In this second part, you’ll use your network to make predictions, and also compare its performance to two standard libraries (scikit-learn and Keras).

In the last post , you created a 2-layer neural network from scratch and now have a better understanding of how neural networks work.

Write every line of code and understand why it works

Passionate about machine learning? Same! We’re curating each week’s biggest stories, best tutorials, and latest research so you don’t have to. Sign up for weekly updates delivered to your inbox .

👉🏽 Full code in Google Colab here:

Training and Testing the Neural Network

In case you’ve forgotten what your network looks like, here’s the code:

To train and test the network, you’ll read in the heart disease dataset, preprocess it like you would in any other machine learning task, specifically, you’ll:

• Split the data into train and test sets,
• Standardize it using StandardScaler,
• Initialize a model from your NeuralNetwork class, and finally,
• Train and predict with the Model

In the code block above, first, you get the training data, excluding the label—this is done with the drop function. The drop function removes the specified column from the dataset and returns the remaining features. Next, you saved the label to a variable calledy_label.

Next, you split the data into train and test set, with the test set taking 10 percent of the overall data. Then, you standardize the dataset. Standardization is important when working with neural networks, as it has a serious effect on the training time and network performance.

In the next code section, you’ll initialize the neural network and train it using the fit function.

Here, you initialize the neural network with the default parameters:

• layers ==> [13,8,1]
• learning_rate ==> 0.001
• iterations ==> 100

Then, you passed in the training data (Xtrain,ytrain) to the fit method. This is the training phase of the model.

Now that you’re done training, let’s plot the loss using the handy plot_loss function in the NeuralNetwork class:

nn.plot_loss()

You can see that the loss starts pretty high, then quickly goes down, and starts approaching zero. This gives you an insight into how the network learns. Now, let’s try increasing the learning rate and the size of the hidden layer, and see how it affects the loss.

You can clearly see that the loss is lower than the previous one. So these sets of parameters might actually be doing better than the first. To confirm this, let’s show the accuracy on both the train and test set.

For the first Architecture, we have the following accuracies:

For the second network, I had the same set of accuracies. This suggests that the second model is overfitting the data and the first model is actually better.

And just so you know, those are pretty great accuracies for a Neural Network you built from scratch!

Working with Python Libraries

Just to spice it up, let’s compare your implementation with standard libraries like Keras and scikit-learn. If you don’t have these libraries installed on your machine, you either have to install them or use Google Colab for this section.

A 2-Layer Neural Network with scikit-learn

The MLPClassifier in the scikit-learn package contains an implementation of a neural network. To use it, first, you have to import it from the sklearn.neural_network class, and initialize the architecture:

Notice that you specify just the number of hidden nodes when using the MLPClassifier—this is because the size of the input feature is inferred from the dimension of the input data. Also, you can specify the learning rate and the number of iterations. To keep things fair, you’ll use the same parameters you used for your own Neural Net. That is 8 hidden nodes, a learning rate of 0.001 and 100 iterations.

Let’s train this network and calculate its accuracy:

Well, can you believe it! Your accuracy is actually higher than that of scikit-learn. This does not mean that you can use your Neural Network in production though. The catch here is that many standard implementations of neural networks use different optimization strategies regarding weight initialization, model training, gradient updates, adaptive learning rates, regularization, and so on, and these are important to consider when using models in industry.

While we can implement most of these optimizations in our implementation, we’ll refrain from doing that in this post. If interested, check out these amazing articles:

Next, let’s see how to create a neural network with Keras.

A 2-Layer Neural Network with Keras

Keras is an open-source deep-learning library written in Python. It was designed to make experimentation with deep learning libraries faster and easier. In this article, you’ll be using the Tensorflow Implementation of Keras (tf.keras). This implementation is similar to normal Keras. The only difference is the way it is imported.

To create a neural network model in tf.keras, you have to import the Sequential, Layers and Dense modules. The Sequential module can accept a series of layers stacked on top of each other.

Let’s demonstrate this below:

In the code section above, you created a Sequential model. This tells Keras that you want to create stacks of layers. Next, you add two Dense layers. The first Dense layer has an input shape of 13 and 8 hidden nodes, while the second Dense layer, which is your output, has a single node and uses the sigmoid activation function.

Next, compile the model by passing in the loss function, an optimizer that tells the network how to learn, and a metric to be calculated:

There are many types of optimizers you can choose from, and each may affect the network’s performance. Start with the Adam optimizer here, but feel free to try other ones here. Also, use a binary cross-entropy loss function, given that you’re working on a binary classification task.

After compiling, you’ll train the network and evaluate it:

And that’s it! The accuracies of Keras on both the train and test sets are similar to what you have in your own Neural Network.

Summary

That’s it… You’ve built, trained, and tested a neural network from scratch, and also compared the performance with 2 standard deep learning libraries.

In summary, to create a neural network from scratch, you have to perform the following:

1. Get training data and target variable
2. Initialize the weights and biases
3. Compute forward propagation
4. Compute backpropagation
5. Update weights and bias
6. Repeat steps 2-4 for n times

Conclusion

I hope this has been an effective introduction to Neural Networks, AI and deep learning in general. More importantly, I hope you’ve learned the steps and challenges in creating a Neural Network from scratch, using just Python and Numpy. While your network is not state-of-art, I’m sure this post has helped you understand how neural network works.

There are lots of other things that go into effectively optimizing a neural network for production. This is the reason why you won’t need to use a “built-from-scratch” Neural Network in the industry, instead, you’ll use existing libraries that have been efficiently optimized.

Congratulations, you’re well on your way to becoming a great AI engineer. If you have any questions, suggestions, or feedback, don’t hesitate to use the comment section below.

Connect with me on Twitter.

Connect with me on LinkedIn.