Skip to main content

Understanding Neocortex to Create Intelligence

There are two approaches to create intelligence in machines. One is to understand how human brain creates intelligence and replicate the methods used by it to create AI. The other is to take a fresh engineering approach to create intelligence.

There is an ongoing debate as to which approach is feasible or better. Few companies like Numenta and Vicarious have invested in understanding the neocortex. They have launched successful applications that work on the principles of neocortex. Other big companies including Google, Microsoft and Facebook use deep learning to create applications that can do major classification tasks. But deep learning does not exploit the concepts of neocortex, and hence is not up to the mark to do continuous learning. The deep learning models first go through a training phase and then the models are used to do classification.

The idea that makes neocortex truly intelligent is that it is able to do pattern matching and prediction. This has given humans the ability to survive and dominate the animal kingdom. Neocortex, a thin layer formed on top of the brain, started gaining prominence in mammals. A rat with a small sheet of neocortex is able to travel in a maze. It uses the trained patterns to predict the route in a maze. Researchers like Jeff Hawkins, founder of Numenta, believes that this ability of neocortex if replicated in machines will make them truly intelligent. He has created an architecture that exploits the hierarchy of neocortex. He calls it Hierarchical Temporal Memory (HTM). HTM is able to perform online learning like humans, where it doesn't need to use stored models, but the models get updated continuously.

Other approaches that model the neocortex have come into prominence and have received funding. The Blue Brain Project conducted by IBM has received a funding of $4.9 million. The idea is to artificially link the neurons in the computer by placing thirty million synapses in their proper three dimensional position. The Human Brain Project funded by European Union is attempting to provide a collaborative informatics infrastructure and first draft rodent and human whole brain models. Henry Markram is the director of both Blue Brain Project and Human Brain Project. He believes that the best way to figure out something is to try to build it from scratch.

Researchers at CereLabs believe that both approaches are essential for the success of a true AI company. We are keeping a close eye on deep learning as well as architectures like HTM. Although deep learning is giving us ready made solutions to create intelligence on your data, architectures like HTM will provide general intelligence solutions. We are betting on General Artificial Intelligence based on the concepts of neocortex. Are you?

Comments

Popular posts from this blog

Implement XOR in Tensorflow

XOR is considered as the 'Hello World' of Neural Networks. It seems like the best problem to try your first TensorFlow program.

Tensorflow makes it easy to build a neural network with few tweaks. All you have to do is make a graph and you have a neural network that learns the XOR function.

Why XOR? Well, XOR is the reason why backpropogation was invented in the first place. A single layer perceptron although quite successful in learning the AND and OR functions, can't learn XOR (Table 1) as it is just a linear classifier, and XOR is a linearly inseparable pattern (Figure 1). Thus the single layer perceptron goes into a panic mode while learning XOR – it can't just do that. 

Deep Propogation algorithm comes for the rescue. It learns an XOR by adding two lines L1 and L2 (Figure 2). This post assumes you know how the backpropogation algorithm works.



Following are the steps to implement the neural network in Figure 3 for XOR in Tensorflow:
1. Import necessary libraries
impo…

From Cats to Convolutional Neural Networks

Widely used in image recognition, Convolutional Neural Networks (CNNs) consist of multiple layers of neuron collection which look at small window of the input image, called receptive fields.
The history of Convolutional Neural Networks begins with a famous experiment “Receptive Fields of Single Neurons in the Cat’s Striate Cortex” conducted by Hubel and Wiesel. The experiment confirmed the long belief of neurobiologists and psychologists that the neurons in the brain act as feature detectors.
The first neural network model that drew inspiration from the hierarchy model of the visual nervous system proposed by Hubel and Wiesel was Neocognitron invented by Kunihiko Fukushima, and had the ability of performing unsupervised learning. Kunihiko Fukushima’s approach was commendable as it was the first neural network model having the capability of pattern recognition similar to human brain. The model gave a lot of insight and helped future understanding of the brain.
A successful advancement i…

Understanding Projection Pursuit Regression

The following article gives an overview of the paper "Projection Pursuit Regression” published by Friedman J. H and Stuetzle W. You will need basic background of Machine Learning and Regression before understanding this article. The algorithms and images are taken from the paper. (http://www.stat.washington.edu/courses/stat527/s13/readings/FriedmanStuetzle_JASA_1981.pdf
What is Regression? Regression is a machine learning technology used to predict a response variable given multiple predictor variables or features. The main distinction is that the response to be predicted is any real value and not just any class or cluster name. Hence though similar to Classification in terms of making a prediction, it is largely different given what it’s predicting. 
A simple to understand real world problem of regression would be predicting the sale price of a particular house based on it’s square footage, given that we have data of similar houses sold in that area in the past. The regression so…