Shallow Neural Network Time-Series Prediction and Modeling – MATLAB & Simulink

This example shows how to train a shallow neural network to fit time series data using the Neural Net Time Series app.

Open the Neural Net Time Series app using ntstool.

Select Network

You can use the Neural Net Time Series app to solve three different kinds of time series problems.

• In the first type of time series problem, you would like to predict future values of a time series y(t) from past values of that time series and past values of a second time series x(t). This form of prediction is called nonlinear autoregressive network with exogenous (external) input, or NARX.

• In the second type of time series problem, there is only one series involved. The future values of a time series y(t) are predicted only from past values of that series. This form of prediction is called nonlinear autoregressive, or NAR.

• The third time series problem is similar to the first type, in that two series are involved, an input series (predictors) x(t) and an output series (responses) y(t). Here you want to predict values of y(t) from previous values of x(t), but without knowledge of previous values of y(t).

For this example, use a NARX network. Click Select Network > NARX Network.

Select Data

The Neural Net Time Series app has example data to help you get started training a neural network.

To import example pH neutralization process data, select Import > More Example Data Sets > Import pH Neutralization Data Set. You can use this data set to train a neural network to predict the pH of a solution using acid and base solution flow. If you import your own data from file or the workspace, you must specify the predictors and responses.

Information about the imported data appears in the Model Summary. This data set contains 2001 time steps. The predictors have two features (acid and base solution flow) and the responses have a single feature (solution pH).

Split the data into training, validation, and test sets. Keep the default settings. The data is split into:

• 70% for training.

• 15% to validate that the network is generalizing and to stop training before overfitting.

• 15% to independently test network generalization.

For more information on data division, see Divide Data for Optimal Neural Network Training.

Create Network

The standard NARX network is a two-layer feedforward network, with a sigmoid transfer function in the hidden layer and a linear transfer function in the output layer. This network also uses tapped delay lines to store previous values of the x(t) and y(t) sequences. Note that the output of the NARX network, y(t), is fed back to the input of the network (through delays), since y(t) is a function of y(t–1),y(t–2),…,y(t–d). However, for efficient training this feedback loop can be opened.

Because the true output is available during the training of the network, you can use the open-loop architecture shown below, in which the true output is used instead of feeding back the estimated output. This has two advantages. The first is that the input to the feedforward network is more accurate. The second is that the resulting network has a purely feedforward architecture, and therefore a more efficient algorithm can be used for training. This network is discussed in more detail in Design Time Series NARX Feedback Neural Networks.

The Layer size value defines the number of hidden neurons. Keep the default layer size, 10. Change the Time delay value to 4. You might want to adjust these numbers if the network training performance is poor.

You can see the network architecture in the Network pane.

Train Network

To train the network, select Train > Train with Levenberg-Marquardt. This is the default training algorithm and the same as clicking Train.

Training with Levenberg-Marquardt (trainlm) is recommended for most problems. For noisy or small problems, Bayesian Regularization (trainbr) can obtain a better solution, at the cost of taking longer. For large problems, Scaled Conjugate Gradient (trainscg) is recommended as it uses gradient calculations which are more memory efficient than the Jacobian calculations the other two algorithms use.

In the Training pane, you can see the training progress. Training continues until one of the stopping criteria is met. In this example, training continues until the validation error increases consecutively for six iterations (“Met validation criterion”).

Analyze Results

The Model Summary contains information about the training algorithm and the training results for each data set.

You can further analyze the results by generating plots. To plot the error autocorrelation, in the Plots section, click Error Autocorrelation. The autocorrelation plot describes how the prediction errors are related in time. For a perfect prediction model, there should only be one nonzero value of the autocorrelation function, and it should occur at zero lag (this is the mean square error). This would mean that the prediction errors were completely uncorrelated with each other (white noise). If there was significant correlation in the prediction errors, then it should be possible to improve the prediction – perhaps by increasing the number of delays in the tapped delay lines. In this case, the correlations, except for the one at zero lag, fall approximately within the 95% confidence limits around zero, so the model seems to be adequate. If even more accurate results were required, you could retrain the network. This will change the initial weights and biases of the network, and may produce an improved network after retraining.

View the input-error cross-correlation plot to obtain additional verification of network performance. In the Plots section, click Input-Error Correlation. The input-error cross-correlation plot illustrates how the errors are correlated with the input sequence x(t). For a perfect prediction model, all of the correlations should be zero. If the input is correlated with the error, then it should be possible to improve the prediction, perhaps by increasing the number of delays in the tapped delay lines. In this case, most of the correlations fall within the confidence bounds around zero.

In the Plots section, click Response. This displays the outputs, responses (targets), and errors versus time. It also indicates which time points were selected for training, testing, and validation.

If you are unhappy with the network performance, you can do one of the following:

• Train the network again.

• Increase the number of hidden neurons.

• Use a larger training data set.

If performance on the training set is good but the test set performance is poor, this could indicate the model is overfitting. Decreasing the layer size, and therefore decreasing the number of neurons, can reduce the overfitting.

You can also evaluate the network performance on an additional test set. To load additional test data to evaluate the network with, in the Test section, click Test. The Model Summary displays the additional test data results. You can also generate plots to analyze the additional test data results.

Generate Code

Select Generate Code > Generate Simple Training Script to create MATLAB code to reproduce the previous steps from the command line. Creating MATLAB code can be helpful if you want to learn how to use the command-line functionality of the toolbox to customize the training process. In Fit Time Series Data Using Command-Line Functions, you will investigate the generated scripts in more detail.

Export Network

You can export your trained network to the workspace or Simulink®. You can also deploy the network with MATLAB Compiler™ tools and other MATLAB code generation tools. To export your trained network and results, select Export Model > Export to Workspace.