A Complete Guide To Artificial Neural Network In Machine Learning

This Tutorial Explains What Is Artificial Neural Network, How Does An ANN Work, Structure and Types of ANN & Neural Network Architecture:

In this Machine Learning Training For All, we explored all about Types of Machine Learning in our previous tutorial.

Here, in this tutorial, discuss the various algorithms in Neural Networks, along with the comparison between machine learning and ANN. Before we learn how ANN contributes to machine learning, we need to know what an Artificial Neural Network is and brief knowledge about machine learning.

Let’s explore more about Machine Learning And Artificial Neural Network!!

=> Read Through The Complete Machine Learning Training Series

What Is Machine Learning?

Machine Learning is a field of science that allow computers the capability to learn and act without being programmed explicitly. It is a subfield of Artificial Intelligence.

What Is An Artificial Neural Network?

ANN is a non-linear model that is widely used in Machine Learning and has a promising future in the field of Artificial Intelligence.

Artificial Neural Network is analogous to a biological neural network. A biological neural network is a structure of billions of interconnected neurons in a human brain. The human brain comprises of neurons that send information to various parts of the body in response to an action performed.

Similar to this, an Artificial Neural Network (ANN) is a computational network in science that resembles the characteristics of a human brain. ANN can model as the original neurons of the human brain, hence ANN processing parts are called Artificial Neurons.

ANN consist of a large number of interconnected neurons that are inspired by the working of a brain. These neurons have the capabilities to learn, generalize the training data and derive results from complicated data.

These networks are used in the areas of classification & prediction, pattern & trend identifications, optimization problems, etc. ANN learns from the training data (input and target output known) without any programming.

The learned neural network is called an expert system with the capability to analyze information and answer the questions of a specific field.

The formal definition of ANN given by Dr.Robert Hecht-Nielson, inventor of one first neuro computers is:

“…a computing system made up of a number of simple, highly interconnected processing elements, which process information by their dynamic state response to external inputs”.

Structure Of A Biological Neural Network

A biological neural network consists of:

• Soma: This is also called the cell body. It is where the cell nucleus is located.
• Dendrites: These are tree-like networks that are connected to the cell body. It is made of the nerve fiber.
• Axon: Axon carries the signal from the cell body. It splits into strands and each strand ends in a bulb-like structure called synapse. The electrical signals are passed between the synapse and dendrites.

[image source] 

Comparison Of Biological Neuron And Artificial Neuron

Biological NeuronArtificial Neuron

It is made of cells.The cells correspond to neurons.

It has dendrites which are interconnections between cell body.The connection weights correspond to dendrites.

Soma receives the input.Soma is similar to net input weight.

The axon receives the signal.The output of ANN corresponds to axon.

Characteristics Of ANN

• Non Linearity: The mechanism followed in ANN for the generation of the input signal is nonlinear.
• Supervised Learning: The input and output are mapped and the ANN is trained with the training dataset.
• Unsupervised Learning: The target output is not given, so the ANN will learn on its own by discovering the features in the input patterns.
• Adaptive Nature: The connection weights in the nodes of ANN are capable to adjust themselves to give the desired output.
• Biological Neuron Analogy: The ANN has a human brain-inspired structure and functionality.
• Fault Tolerance: These networks are highly tolerant as the information is distributed in layers and computation occurs in real-time.

Structure Of ANN

Artificial Neural Networks are processing elements either in the form of algorithms or hardware devices modeled after the neuronal structure of a human brain cerebral cortex.

These networks are also simply called Neural Networks. The NN is formed of many layers. The multiple layers that are interconnected are often called “Multilayer Perceptron”. The neurons in one layer are called “Nodes”. These nodes have an “Activation function”.

The ANN has 3 main layers:

• Input Layer: The input patterns are fed to the input layers. There is one input layer.
• Hidden Layers: There can be one or more hidden layers. The processing that takes place in the inner layers is called “hidden layers”. The hidden layers calculate the output based on the “weights” which is the “sum of weighted synapse connections”. The hidden layers refine the input by removing redundant information and send the information to the next hidden layer for further processing.
• Output Layer: This hidden layer connects to the “output layer” where the output is shown.

Activation Function

The activation function is an internal state of a neuron. It is a function of input that the neuron receives. The activation function is used to convert the input signal on the node of ANN to an output signal.

What Is An Artificial Neuron?

An Artificial Neural Network consists of highly interconnected processing elements called nodes or neurons.

These neurons work in parallel and are organized in an architecture. The nodes are connected to each other by connection links. Each neuron carries a weight that contains information about the input signal.

How Does Artificial Neuron Work?

An artificial neuron receives an input. These inputs have a weight called “synapse”. These neurons (also called nodes) have “activation function”. This activation function works on the input and processes it to give an output.

The weighted sum of inputs becomes an input signal to the activation function to give one output. These input weights are adjustable so that the neural network can adjust its parameters to give the desired output.

Few Common Activation Functions That Are Used In Artificial Neural Network Are:

#1) Identity Function

It can be defined as f(x) = x for all values of x. This is a linear function where the output is the same as the input.

#2) Binary Step Function

This function is used in single layer networks to convert the net input to output. The output is binary i.e. 0 or 1. The t represents the threshold value.

[image source]

#3) Bipolar Step Function

The bipolar step function has bipolar outputs (+1 or -1) for the net input. T represents the threshold value.

#4) Sigmoidal Function

It is used in backpropagation networks.

It is of two types:

• Binary Sigmoid function: It is also called as the unipolar sigmoid function or logistic sigmoid function. The range of sigmoidal functional is 0 to 1.

• Bipolar Sigmoid: The bipolar sigmoidal function ranges from -1 to +1. It is similar to the hyperbolic tangent function.

[image source]

#5) RampFunction

The weighted sum of Inputs means the “product of the weight of input and value of input” summed together for all inputs.

Let I= {I1, I2, I3… In} be the input pattern to neuron.

Let W= {W1, W2, W3… Wn} be the weight associated with each input to the node.

Weighted Sum of Inputs = Y = (?Wi *Ii) for i =1 to n

Basic Models Of ANN

The artificial neural network models consist of 3 entities:

• Weights or synaptic connections
• The learning rule used for adjusting the weights
• Activation functions of the neuron

Neural Network Architecture

In ANN the neurons are interconnected and the output of each neuron is connected to the next neuron through weights. The architecture of these interconnections is important in an ANN. This arrangement is in the form of layers and the connection between the layers and within the layer is the neural network architecture.

The most commonly known network architectures are:

1. Single-Layer Feed-Forward Network
2. Multi-Layer Feed-Forward Network
3. Single Node With Its Own Feedback
4. Single-Layer Recurrent Network
5. Multi-Layer Recurrent Network

Let us have a look at each of these in detail.

#1) Single-Layer Feed-Forward Network

A layer is a network formed of neurons. These neurons are connected to the other neurons of the next layer. For a single layer, there are only the input and output layers. The input layer is connected to the output layer nodes with weights.

All the input nodes are connected to each of the output nodes. The term feed-forward depicts that there is no feedback sent from the output layer to the input layer. This forms a single layer feed-forward network.

[image source]

#2) Multi-Layer Feed-Forward Network

The Multi-layer network consists of one or more layers between the input and output. The input layer just receives a signal and buffers it while the output layer shows the output. The layers between the input and output are called the hidden layers.

The hidden layers are not in contact with the external environment. With more number of hidden layers, the output response is more efficient. The nodes in the previous layer are connected to each node in the next layer.

As there is no output layer connected to the input or hidden layers, it forms a multi-layer feed-forward network.

#3) Single Node With Its Own Feedback

The networks where the output layer output is sent back as an input to the input layer or the other hidden layers is called Feedback Networks. In single-node feedback systems, there is a single input layer where the output is redirected back as feedback.

#4) Single Layer Recurrent Network

In a single layer recurrent network, the feedback network forms a closed loop. In this model, a single neuron receives feedback to itself or the other neurons in the network or both.

#5) Multi-Layer Recurrent Network

In Multi-Layer recurrent network, multiple hidden layers exist and the output is redirected back to the neurons of the previous layers and other neurons in the same layers or same neuron itself.

Example Of Artificial Neuron Network

Let’s take the below network with the given input and calculate the net input neuron and obtain the output of the neuron Y with activation function as binary sigmoidal.

The input has 3 neurons X1, X2 and X3, and single output Y.

The weights associated with the inputs are: {0.2, 0.1, -0.3}

Inputs= {0.3, 0.5, 0.6}

Net input ={x1*w1+x2*w2+ x3*w3}

Net input = (0.3*0.2) + (0.5*0.1) + (0.6*-0.3)

Net input= -0.07

Output for Binary Sigmoidal:

X is -0.07

Output comes out to be 0.517

Comparison Between Machine Learning And ANN

Machine LearningArtificial Neural Network

Machine Learning learns from input data and discovers output data patterns of interest.ANN are used in machine learning algorithms to train the system using synapses, nodes and connection links.

ML is a subset of the field of artificial intelligence.ANN is also a part of the Artificial Intelligence field of science and a subset of machine learning.

ML algorithms learn from data fed to the algorithm for decision making purpose. Some of these algorithms are classification. Clustering, association data mining.ANN is a deep learning science that analyses the data with logical structures as humans do.

Some of theANN learning schemes are Hebbian, Perceptron, Back propagation, etc.

ML algorithms have self-learning capabilities but would require human intervention if the outcome is inaccurate.ANN algorithms have capabilities to adjust themselves using connection weights if the outcome comes out to be wrong.

ML algorithms requires programming skills, data structure and big data database knowledge.ANN also requires strong skills in mathematics, probability , data structures, etc.

ML programs can predict the outcome for learned set of data and adjust itself for new data.ANN can learn and make intelligent decision on their own for new data but it is deeper than machine learning.

ML is applied in eCommerce, healthcare, product recommendations, etc.ANN is applied in finance domain, machine learning and artificial intelligence.

Supervised and Unsupervised learning fall under machine learning.Learning such as Kohenen, radial bias, feed-forward neural network fall under ANN.

Some examples of ML are Google search results etc.Some examples of ANN are face recognition, image recognition, etc.

Neural Networks And Deep Learning

Deep Learning networks contain several hidden layers between the input and the output. These networks are distinguished by the depth of the hidden layers in them. The input data passes through multiple steps before the output is shown.

These networks differ from the earlier NN such as perceptron which had a single hidden layer and was called Shallow Networks. Each hidden layer in the deep learning network trains the data with certain features based on the output of the previous layer.

The data passes through many layers of nonlinear function at the node. The more the number of layers, the more complex features can be recognized as the next layer will perform aggregation of features from the previous layers.

Multiple hidden layers in the network increase complexity and abstraction. This depth is also termed as a feature hierarchy. Due to this, deep learning networks are capable of handling high dimensional data.

Some examples of deep learning networks include clustering of millions of images based on its characteristics and similarities, filtering of email messages, applying filters to messages in CRM, identifying speech, etc.

Deep Learning Networks can be trained on both labeled and unlabeled set of data. For the unlabeled set of data, the networks such as Boltzmann selection machines perform automatic feature extraction.

The network learns automatically by analyzing the input through sampling and minimizing the difference in output and distribution of input. The neural network here finds correlations between the features and outcomes.

The deep learning networks trained on labeled data can be applied to unstructured data. The more the training data fed to the network, the more accurate it will become.

The ability of the network to learn from unlabeled data is an advantage over the other learning algorithms.

Artificial Neural Network Applications

Neural Networks have been successfully used in a variety of solutions as shown below.

#1) Pattern Recognition: ANN is used in pattern recognition, image recognition, visualization of images, handwriting, speech, and other such tasks.

#2) Optimization Problems: Problems such as finding the shortest route, scheduling and manufacturing where problem constraints are to be satisfied and optimal solutions need to be achieved are using NNs.

#3) Forecasting: NN can predict the outcome for situations by analyzing past trends. Applications such as banking, stock market, weather forecasting use Neural Networks.

#4) Control Systems: Control systems such as computer products, chemical products, and robotics use neural networks.

Limitations Of Neural Networks

Enlisted below are some of the drawbacks of Neural Networks.

• These networks are black boxes for the user as the user does not have any roles except feeding the input and observing the output. The user is unaware of the training happening in the algorithm.
• These algorithms are rather slow and require many iterations (also called epochs) to give accurate results. This is because the CPU computes the weights, activation function of each node separately thereby making it consume time as well as resources. It also causes a problem with a large amount of data.

Conclusion

In this tutorial, we learned about Artificial Neural Network, its analogy to Biological Neuron and Types of Neural Network.

ANN falls under machine learning. It is a computational model composed of multiple neuron nodes. These nodes receive input, process the input using activation function and pass the output to the next layers.

The inputs are associated with connection link weights called synapse. A basic ANN consists of an input layer, weights, activation function, hidden layer, and an output layer.

The activation functions are used to convert the input to the output. Some of them are binary, bipolar, sigmoidal and a ramp function. There are different types of ANN such as Single-Layer Feed Forward, Multilayer Feed Forward, Recurrent networks, etc. based on the number of hidden layers and feedback mechanisms.

ANN with many hidden layers between the input and output form deep learning network. The deep learning networks have high complexity and abstraction level that makes them capable of computing high dimensional data with thousands of parameters.

ANN is used in the field of forecasting, image processing, control systems, etc. These have been successfully applied as a solution to the array of problems in science.

We hope this tutorial explained everything you need to know about Artificial Neural Networks!!

=> Visit Here For The Exclusive Machine Learning Series