Artificial Neural Networks

We want our machines to learn everything on their own as much as possible. Over the past few decades, researchers have come up with many theories and formulations about how we can achieve this in the best possible way. This realm is called machine learning. We come up with algorithms to teach the machines how to learn. I have discussed more about machine learning here. Human brain seems to achieve this rather effortlessly. Our ultimate goal is to make the machines as good as our brains, or even better. The formulation of Artificial Neural Network (ANN) is an attempt towards this.  

What is a neural network?

Neural networks refers to the central nervous system in the human brain. It consists of neurons, axons, dendrites and synapses. These are the basic elements of the system and each of them have different functionalities. The transfer of information in the brain occurs with the help of electrical signals and neurons are responsible for this. Neuron is an electrically excitable cell which helps in the transfer of information to and from the brain. The interconnections between neurons form the biological neural network. There are about 100 billion neurons in a human brain.

What is an artificial neural network?

Now we take inspiration from the human brain and try to mimic the functionality using a mathematical formulation. Even after all the advances in technology, human brain is still very obscure and we don’t completely understand how it functions. ANN refers to a mathematical model which is inspired by the biological neural network. It consists of artificial neurons, which act as simple processing elements. ANNs are parallel cascaded systems, which means that information is processed simultaneously by multiple processing elements and the output is fed to the next stage of neurons. Most ANNs are adaptive and they change their structure based on the input, output and the feedback, just like the neurons in our brain.

How do we use this in machine learning?

ANNs are mainly used as pattern classifiers in the realm of machine learning. Since it is adaptive, each parameter can change during its operation and it is deployed for solving the problem at hand. This is called the training phase. ANNs need a lot of training data to come up with a robust model.

ANN is developed with a step-by-step procedure which optimizes a criterion commonly known as the learning rule. The input/output training data is fundamental for these networks as it conveys the information which is necessary to discover the optimal operating point. This concept is a bit similar to Reinforcement Learning. If you have read my previous blog about machine learning, I use the analogy of a baby named John. Just like how John updates his knowledge based on his experiences, an ANN also updates itself using the feedback. In addition, the non linear nature makes neural network a very flexible system.

Why feedback?

An ANN is basically a system which receives an input, processes the data, and provides an output. Once an input is presented to the neural network, and a corresponding desired or target response is set at the output, an error is composed from the difference of the desired response and the real system output.

The error information is fed back to the system which then makes all the required adjustments to its parameters in a systematic fashion. This process is repeated until the desired output is acceptable. It is important to notice that the performance hinges heavily on the data. What I mean to say is that if you don’t provide a wide variety of data, then the ANN might fail in a few scenarios.

If we consider the example of baby John discovering eating items, we can say that the error will be more if he chews something really hard by mistake. It means John will have to be very careful about these kind of objects. But if he chews something soft which is not a fruit, the error is not that large. This object didn’t cause any harm to his teeth, it’s just that he cannot bite it off and eat it. If he already knew that such objects exist, he will probably not attempt to eat them. This is why ANNs need wide variety of data to train.

There are many different types of ANNs and all of them have their own algorithms and formulations. ANNs are one of the more recent machine learning algorithms to be developed. The field of work is very interdisciplinary. They are being increasingly used to find patterns in data and in the field of artificial intelligence in general.

————————————————————————————————-