Neural Network Models Explained – Seldon

Artificial neural network models are behind many of the most complex applications of machine learning. Classification, regression problems, and sentiment analysis are some of the ways artificial neural networks are being leveraged today. As an emerging field, there are many different types of artificial neural networks. They vary for a variety of reasons, such as complexity, network architecture, density, and the flow of data. But the different types share a common goal of modelling and attempting to replicate the behaviour of neurons to improve machine learning.

Artificial neural networks have a wide range of uses in machine learning. Each type of artificial neural network model has different strengths and use cases. Overall, they are mainly used to solve more complex problems than would be possible with more traditional machine learning techniques. Examples may include complex natural language processing and machine learning-power language translation, which all rely on artificial neural networks. Recurrent neural networks are often utilised for analysis sentiment or translating text too. The depth and scale of the neural architecture means a non-linear decision making process can be achieved.

Artificial neural networks are used in the deep learning form of machine learning. It’s called deep learning as models use the ‘deep’, multi-layered architecture of an artificial neural network. As each layer of an artificial neural network can process data, models can build an abstract understanding of the data. This architecture means models can perform increasingly complex tasks, for example understanding natural language or categorising complex file types.

Artificial neural networks are already used in machine learning to power:

• Recommendation systems for customers, users and consumers in products like streaming services or e-commerce.
• To power virtual assistance and speech recognition software.
• Complex image, audio and document classification models, for example in facial recognition software.
• In automatic feature extraction from raw, unlabelled data.

There are different types of artificial neural networks which vary in complexity. This guide explores the different types of artificial neural networks, including what they are and how they’re used.

What are artificial neural networks?

Artificial neural networks are designed to replicate the behaviour of neural networks found in human or animal brains. By mirroring and modelling the behaviour of neurons, machine learning gains the model architecture to process increasingly complex data. There are a variety of different types of artificial neural networks, with many early iterations seeming simple in comparison to emerging techniques. For example, artificial neural networks are used as the architecture for complex deep learning models.

Artificial neurons or nodes are modelled as a simplified version of neurons found in the brain. Each artificial neuron is connected to other nodes, though the density and amount of connections differ with each type of artificial neural network. The network is usually grouped into layers of nodes, which exist between the input and output layer. This multi-layered network architecture is also known as a deep neural network because of the depth of these layers. These different layers in the artificial neural network models can learn different features of data. Hidden hierarchical layers allow the understanding of complex concepts or patterns from processed data.

The structure of artificial neural networks represent a simplified reflection of the complexity of the human or animal brain. A web of interconnected artificial nodes mimic the behaviour of neurons within a nervous system. These artificial neural networks are much less complex than a human brain, but are still incredibly powerful at performing tasks such as classification. Data starts in the input layer and leaves from the output layer. But with the more complex artificial neural networks, data will move between many different layers in a non-linear way.

Complex artificial neural networks are developed so that models can mirror the nonlinear decision-making process of the human brain. This means models can be trained to make complex decisions or understand abstract concepts and objects. The model will build from low-level features to complex features, understanding complex concepts. Each node within the network is weighted depending on its influence on other artificial neural network nodes.

Like other machine learning models, optimisation of artificial neural networks is based on a loss function. This is the difference between a predicted and actual output. The weighting of each node and layer is adjusted by the model to achieve a minimum loss. Artificial neural network models can understand multiple levels of data features, and any hierarchical relationship between features. So when used for a classification problem, an artificial neural network model can understand complex concepts by processing multiple layers of features.

5 types of neural network models explained

There are many different types of artificial neural networks, varying in complexity. They share the intended goal of mirroring the function of the human brain to solve complex problems or tasks. The structure of each type of artificial neural network in some way mirrors neurons and synapses. However, they differ in terms of complexity, use cases, and structure. Differences also include how artificial neurons are modelled within each type of artificial neural network, and the connections between each node. Other differences include how the data may flow through the artificial neural network, and the density of the nodes.

5 examples of the different types of artificial neural network include:

• Feedforward artificial neural networks
• Perceptron and Multilayer Perceptron neural networks
• Radial basis function artificial neural networks
• Recurrent neural networks
• Modular neural networks

Feedforward artificial neural networks

As the name suggests, a Feedforward artificial neural network is when data moves in one direction between the input and output nodes. Data moves forward through layers of nodes, and won’t cycle backwards through the same layers. Although there may be many different layers with many different nodes, the one-way movement of data makes Feedforward neural networks relatively simple. Feedforward artificial neural network models are mainly used for simplistic classification problems. Models will perform beyond the scope of a traditional machine learning model, but don’t meet the level of abstraction found in a deep learning model.

Perceptron and Multilayer Perceptron neural networks

A perceptron is one of the earliest and simplest models of a neuron. A Perceptron model is a binary classifier, separating data into two different classifications. As a linear model it is one of the simplest examples of a type of artificial neural network.

Multilayer Perceptron artificial neural networks adds complexity and density, with the capacity for many hidden layers between the input and output layer. Each individual node on a specific layer is connected to every node on the next layer. This means Multilayer Perceptron models are fully connected networks, and can be leveraged for deep learning.

They’re used for more complex problems and tasks such as complex classification or voice recognition. Because of the model’s depth and complexity, processing and model maintenance can be resource and time-consuming.

Radial basis function artificial neural networks

Radial basis function neural networks usually have an input layer, a layer with radial basis function nodes with different parameters, and an output layer. Models can be used to perform classification, regression for time series, and to control systems. Radial basis functions calculate the absolute value between a centre point and a given point. In the case of classification, a radial basis function calculates the distance between an input and a learned classification. If the input is closest to a specific tag, it is classified as such.

A common use for radial basis function neural networks is in system control, such as systems that control power restoration after a power cut. The artificial neural network can understand the priority order to restoring power, prioritising repairs to the greatest number of people or core services.

Recurrent neural networks

Recurrent neural networks are powerful tools when a model is designed to process sequential data. The model will move data forward and loop it backwards to previous steps in the artificial neural network to best achieve a task and improve predictions. The layers between the input and output layers are recurrent, in that relevant information is looped back and retained. Memory of outputs from a layer is looped back to the input where it is held to improve the process for the next input.

The flow of data is similar to Feedforward artificial neural networks, but each node will retain information needed to improve each step. Because of this, models can better understand the context of an input and refine the prediction of an output. For example, a predictive text system may use memory of a previous word in a string of words to better predict the outcome of the next word. A recurrent artificial neural network would be better suited to understand the sentiment behind a whole sentence compared to more traditional machine learning models.

Recurrent neural networks are also used within sequence to sequence models, which are used for natural language processing. Two recurrent neural networks are used within these models, which consists of a simultaneous encoder and decoder. These models are used for reactive chatbots, translating language, or to summarise documents.

Modular neural networks

A Modular artificial neural network consists of a series of networks or components that work together (though independently) to achieve a task. A complex task can therefore be broken down into smaller components. If applied to data processing or the computing process, the speed of the processing will be increased as smaller components can work in tandem.

Each component network is performing a different subtask which when combined completes the overall tasks and output. This type of artificial neural network is beneficial as it can make complex processes more efficient, and can be applied to a range of environments.

Challenges of artificial neural network models

Although there is huge potential for leveraging artificial neural networks in machine learning, the approach comes with some challenges. Models are complex, and it can be difficult to explain the reasoning behind a decision in what in many cases is a black box operation. This makes the issue of explainability a significant challenge and consideration.

With all types of machine learning models, the accuracy of the final model depends heavily on the quantity and quality of training data available. A model built with an artificial neural network needs even more data and resources to train than a traditional machine learning model. This means millions of data points in contrast to the hundreds of thousands needed by a traditional machine learning model.

The most complex artificial neural networks are often referred to as deep neural networks, referencing the multi-layered network architecture. Deep learning models are usually trained using labelled training data, which is data with a defined input and output. This is known as supervised machine learning, unlike unsupervised machine learning which uses unlabelled, raw training data. The model will learn the features and patterns within the labelled training data, and learn to perform an intended task through the examples in the training data. Artificial neural networks need a huge amount of training data, more so then more traditional machine learning algorithms. This is in the realm of big data, so many millions of data points may be required.

The need for such a large array of labelled, quality data is a limiting factor to being able to develop artificial neural network models. Organisations are therefore limited to those that have access to the required big data. The most powerful artificial neural network models have complex, multi-layered architecture. These models require a huge amount of resources and power to process datasets. This requires powerful, resource-intensive GPU units and system architecture. Again, the level of resources required is a limiting factor and challenge for organisations.

The method of transfer learning is often used to lower the resource intensity. In this process, existing knowledge from other models and existing artificial neural networks can be transferred or adapted when developing a new model. This streamlines development as models aren’t built from scratch each time, but  can be built from elements of existing models.

Machine learning deployment for every organisation

Seldon moves machine learning from POC to production to scale, reducing time-to-value so models can get to work up to 85% quicker. In this rapidly changing environment, Seldon can give you the edge you need to supercharge your performance.

With Seldon Deploy, your business can efficiently manage and monitor machine learning, minimise risk, and understand how machine learning models impact decisions and business processes. Meaning you know your team has done its due diligence in creating a more equitable system while boosting performance.

Deploy machine learning in your organisations effectively and efficiently. Talk to our team about machine learning solutions today.