Types of Neural Networks

types-of-neural-networks-

Mục Lục

Introduction to Neural Networks

Types of Neural Networks are the concepts that define how the neural network structure works in computation resembling the human brain functionality for decision making. There are several types of neural networks available such as feed-forward neural network, Radial Basis Function (RBF) Neural Network, Multilayer Perceptron, Convolutional Neural Network, Recurrent Neural Network(RNN), Modular Neural Network and Sequence to sequence models. Each of the neural network types is specific to certain business scenarios and data patterns. Neural network algorithms could be highly optimized through the learning and relearning process with multiple iterations of data processing. Neural networks augment Artificial Intelligence.

Now that we have an intuition that what neural networks are. Before looking at types of neural networks, let us see neural networks work. Since neural networks are close to replicating how our brain works, it will add an intuition of our best shot at Artificial Intelligence.

Start Your Free Data Science Course

Hadoop, Data Science, Statistics & others

Let us compare it to the nervous system of the human body to have a clear intuition of the work of the neural networks. The first layer gets the raw input similar to the audio nerve in the ears. Next, it processes the signal to the next layer of neurons. The output from the first layer is fed to different neurons in the next layer each performing distinct processing and finally, the processed signals reach the brain to provide a decision to respond. Now in neural networks, the first layers receive the raw input and send it to subsequent layers each processing it in parallel. Each of these nodes in the layer has its own knowledge sphere and own rules of programming learned by itself. Now, having a brief introduction of how neural networks works let us look at different types of Neural Networks.

1. Feed-Forward Neural Network

This is a basic neural network that can exist in the entire domain of neural networks. As the name suggests, the motion of this network is only forward, and it moves till the point it reaches the output node. There is no back feedback to improve the nodes in different layers and not much self-learning mechanism. Below is a simple representation one-layer neural network.

Types of Neural Networks 1

In the above diagram, the data moves in the forward direction with 3 nodes in Layer 1 having a distinct function to process within itself. These have found useful usage in face recognition modeling and computer vision.

2. Radial Basis Function (RBF) Neural Network

The main intuition in these types of neural networks is the distance of data points with respect to the center. These neural networks have typically 2 layers (One is the hidden and other is the output layer). The hidden layer has a typical radial basis function. This function helps in reasonable interpolation while fitting the data to it. This comes with the intuition that the points closer are similar in nature and have a similarity with k-NN. The intuition goes like this: “The predicted target output of an item will behave similar as other items that have close resemblance of the predictor variables.”

3. Multilayer Perceptron

Now, slowly we would move to neural networks having more than 2 layers, i.e. more than one hidden layer. In a Multilayer Perceptron, the main intuition of using this method is when the data is not linearly separable. Each node in a layer consists of a non-linear activation function for processing. These functions are typically Sigmoid/Logistic Function, tanh/Hyperbolic Tangent function, ReLU (Rectified Linear Unit), Softmax. This neural network is fully connected and also has the capability to learn by itself by changing the weights of connection after each data point is processed and the amount of error it generates.

Types of Neural Networks 2

4. Convolutional Neural Network

Now coming on to Convolutional Neural Network, this type of neural network is an advanced version of Multilayer Perceptron. In this type, there is one or more than one convolutional layer. Now the basic question is what exactly is a convolutional layer? Convolution is nothing but a simple filtering mechanism that enables an activation. When this filtering mechanism is repeated, it yields the location and strength of a detected feature. As a result of this ability, these networks are widely used in image processing, natural language processing, recommender systems so as to yield effective results of the important feature detected.

5. Recurrent Neural Network

As the name suggests, in this network something recurs. Now to mention this network the output of a particular layer is saved and is put back into the input again. Here the first layer will be a simple feed-forward neural network and subsequently, each node will retain information in the next layers. On doing this, if the prediction is wrong the network will try to re-learn and learn it effectively to the right prediction. This is widely used in text-to-speech conversion. The main building block of this network is storing in memory will influence the better prediction of what is coming next.

6. Modular Neural Network

Coming to the last but not the least neural network type, i.e. Modular Neural Network. As the name suggests modularity is the basic foundation block of this neural network. Modularity means that independently functioning different networks carry out sub-tasks and since they do not interact with each other the computation speed increases and lead to large complex process work significantly faster by processing individual components. Similar to how independently the left and right side of the brain handles things independently, yet be one, a Modular neural network is an analogous situation to this biological situation.

Conclusion

As evident from the above, we have a lot of types, but here in this section, we have gone through the most used neural networks in the industry. This might not be the exhaustive list of different types of Neural Network, but here we have tried to capture the maximum and widely used ones. And as evident from the algorithm on how neural network works, it has huge potential to learn, re-learn and grow organically unlike machine learning which gets stagnated after few iterations.