Skip to main content

Anomaly Detection based on Prediction - A Step Closer to General Artificial Intelligence

Anomaly detection refers to the problem of finding patterns that do not conform to expected behavior [1]. In the last article "Understanding Neocortex to Create Intelligence", we explored how applications based on the workings of neocortex create intelligence. Pattern recognition along with prediction makes human brains the ultimate intelligent machines. Prediction help humans to detect anomalies in the environment. Before every action is taken, neocortex predicts the outcome. If there is a deviation from the expected outcome, neocortex detects anomalies, and will take necessary steps to handle them. A system which claims to be intelligent, should have anomaly detection in place.

Recent findings using research on neocortex have made it possible to create applications that does anomaly detection. Numenta’s NuPIC using Hierarchical Temporal Memory (HTM) framework is able to do inference and prediction, and hence anomaly detection. HTM accurately predicts anomalies in real world applications such as server anomalies, stock anomalies and geospatial tracking.

All the anomaly detection techniques such as simple threshold, statistical, distance based or supervised methods do not exploit prediction, and hence need a lot of manual intervention. Also those techniques don’t have the capability of neocortex to do continuous learning on streaming data.

Frameworks like HTM are taking anomaly detection closer or even surpassing human like capabilities. This is indeed a step closer to General Artificial Intelligence.

REFERENCES:

[1] Chandola V., Banerjee, A. and Kumar A., 2009.  "Anomaly detection: A survey." ACM computing surveys (CSUR), 41(3), p.15.

[2] J. Hawkins, S. Ahmad, and D. Dubinsky. (2014) The Science of Anomaly Detection [Online technical report]. Redwood City, CA: Numenta, Inc. Available: http://numenta.com/#technology

Comments

  1. Are you trying to move in or out of Jind? or near rohtak Find the most famous, reputed and the very best of all Packers and Movers by simply calling or talking to Airavat Movers and Packers
    good luck
    Ai & Artificial Intelligence Course in Chennai
    PHP Training in Chennai
    Ethical Hacking Course in Chennai Blue Prism Training in Chennai
    UiPath Training in Chennai

    ReplyDelete
  2. Nice! you are sharing such helpful and easy to understandable blog. i have no words for say i just say thanks because it is helpful for me.





    Dot Net Training in Chennai | Dot Net Training in anna nagar | Dot Net Training in omr | Dot Net Training in porur | Dot Net Training in tambaram | Dot Net Training in velachery

    ReplyDelete
  3. Extremely overall quite fascinating post. I was searching for this sort of data and delighted in perusing this one. Continue posting. A debt of gratitude is in order for sharing.data science training Hyderabad

    ReplyDelete
  4. Pretty article! I found some useful information in your blog, it was awesome to read, thanks for sharing this great content to my vision, keep sharing.
    DevOps Training in Hyderabad
    DevOps Course in Hyderabad

    ReplyDelete

  5. Great to become visiting your weblog once more, it has been a very long time for me. Pleasantly this article i've been sat tight fosuch a long time. I will require this post to add up to my task in the school, and it has identical subject along with your review. Much appreciated, great offer. data science course in nagpur

    ReplyDelete

Post a Comment

Popular posts from this blog

Understanding Generative Adverserial Networks - Part 1

This is a two part series on understanding Generative Adversarial Networks (GANs). This part deals with the conceptual understanding of GANs. In the second part we will try to understand the mathematics behind GANs. Generative networks have been in use for quite a while now. And so have discriminative networks. But only in 2014 did someone get the brilliant idea of using them together. These are the generative adversarial networks. This kind of deep learning model was invented by Ian Goodfellow . When we work with data already labelled, it’s called supervised learning. It’s much easier compared to unsupervised learning, which has no predefined labels, making the task more vague.  "Generative Adversarial Networks is the most interesting idea in the last ten years in Machine Learning." - Yann LeCun In this post, we’ll discuss what GANs are and how they work, at a higher , more abstract level. Since 2014, many variations of the traditional GAN have co

Implement XOR in Tensorflow

XOR is considered as the 'Hello World' of Neural Networks. It seems like the best problem to try your first TensorFlow program. Tensorflow makes it easy to build a neural network with few tweaks. All you have to do is make a graph and you have a neural network that learns the XOR function. Why XOR? Well, XOR is the reason why backpropogation was invented in the first place. A single layer perceptron although quite successful in learning the AND and OR functions, can't learn XOR (Table 1) as it is just a linear classifier, and XOR is a linearly inseparable pattern (Figure 1). Thus the single layer perceptron goes into a panic mode while learning XOR – it can't just do that.  Deep Propogation algorithm comes for the rescue. It learns an XOR by adding two lines L1 and L2 (Figure 2). This post assumes you know how the backpropogation algorithm works. Following are the steps to implement the ne