Skip to main content

The Value of Data

Although, there is not a simple answer for what came first - Chicken or Egg?, in Machine Learning, there is an easy answer. Data came first before any function. Machine learning is all about learning from data. The learning algorithm tries to learn a function that can either classify the data into different categories, or learn the function itself that plots the data.

There are two popular ways in which a Machine Learning algorithm can be taught to learn a function, but in both cases it needs data.

  • Supervised Learning: We give the algorithm a lot of data with both input and output, and it learns the function. In case of regression problems, the function approximately plots the function that understands the data. In case of classification problems, the function tries to classify the data.

  • Unsupervised Learning: We give the algorithm a lot of input data with no output, and it tries to find patterns in the data. The algorithm classifies the data based on the similarities of the data points. This method is called clustering.


In both the cases, enough data is needed for the algorithm to learn the function. Most of the machine learning algorithms are provided with three kinds of data -

  • Training Data: This data is used by the algorithm to learn the function, based on which it tries to generalize.

  • Validation Data: There is a high chance that the algorithm might overfit the training data, and will fail on any other data. To protect it from doing so, validation data is used. Validation data helps the algorithm to correlate how accurate it works on both known and unknown data.

  • Test Data: Once the function is learnt, it is tested on the test data. Here the algorithm checks whether it is also able to generalize on the test data, and thus able to stand a better chance in generalizing future data it has to do inference on.


Two interesting questions emerge, both having elegant answers.

Question. What are the parameters to decide on the amount of data needed for the algorithm to learn a function?
Answer. This might come with experience, but the more the merrier. Today’s Machine Learning algorithms such as Neural Networks are so powerful that if not enough data is given, they overfit easily. Also once your model is fine tuned and no more optimization is possible, it will only do better with more data.

Question. What if there are outliers in the data?
Answer. To get better performance, such outliers should be filtered from the data, otherwise the algorithm might get confused, and thus create a function that tries to learn the outlier too. It also depends on how sensitive some algorithms are to such outliers.

In today’s generation where companies having more data are richer than companies with less data, it seems data will decide the future.

P.S: You don’t get any spam emails in your inbox, thanks to billions of email that your spam filter has been trained on.

Comments

Post a comment

Popular posts from this blog

Anomaly Detection based on Prediction - A Step Closer to General Artificial Intelligence

Anomaly detection refers to the problem of finding patterns that do not conform to expected behavior [1]. In the last article "Understanding Neocortex to Create Intelligence", we explored how applications based on the workings of neocortex create intelligence. Pattern recognition along with prediction makes human brains the ultimate intelligent machines. Prediction help humans to detect anomalies in the environment. Before every action is taken, neocortex predicts the outcome. If there is a deviation from the expected outcome, neocortex detects anomalies, and will take necessary steps to handle them. A system which claims to be intelligent, should have anomaly detection in place.
Recent findings using research on neocortex have made it possible to create applications that does anomaly detection. Numenta’s NuPIC using Hierarchical Temporal Memory (HTM) framework is able to do inference and prediction, and hence anomaly detection. HTM accurately predicts anomalies in real worl…

Implement XOR in Tensorflow

XOR is considered as the 'Hello World' of Neural Networks. It seems like the best problem to try your first TensorFlow program.

Tensorflow makes it easy to build a neural network with few tweaks. All you have to do is make a graph and you have a neural network that learns the XOR function.

Why XOR? Well, XOR is the reason why backpropogation was invented in the first place. A single layer perceptron although quite successful in learning the AND and OR functions, can't learn XOR (Table 1) as it is just a linear classifier, and XOR is a linearly inseparable pattern (Figure 1). Thus the single layer perceptron goes into a panic mode while learning XOR – it can't just do that. 

Deep Propogation algorithm comes for the rescue. It learns an XOR by adding two lines L1 and L2 (Figure 2). This post assumes you know how the backpropogation algorithm works.



Following are the steps to implement the neural network in Figure 3 for XOR in Tensorflow:
1. Import necessary libraries
impo…

Understanding Projection Pursuit Regression

The following article gives an overview of the paper "Projection Pursuit Regression” published by Friedman J. H and Stuetzle W. You will need basic background of Machine Learning and Regression before understanding this article. The algorithms and images are taken from the paper. (http://www.stat.washington.edu/courses/stat527/s13/readings/FriedmanStuetzle_JASA_1981.pdf
What is Regression? Regression is a machine learning technology used to predict a response variable given multiple predictor variables or features. The main distinction is that the response to be predicted is any real value and not just any class or cluster name. Hence though similar to Classification in terms of making a prediction, it is largely different given what it’s predicting. 
A simple to understand real world problem of regression would be predicting the sale price of a particular house based on it’s square footage, given that we have data of similar houses sold in that area in the past. The regression so…