How To Create a Neural Network In Python – With And Without Keras

There are two ways to create a neural network in Python:

From Scratch
– this can be a good learning exercise, as it will teach you how neural networks work from the ground up
Using a Neural Network Library
– packages like Keras and TensorFlow simplify the building of neural networks by abstracting away the low-level code. If you’re already familiar with how neural networks work, this is the fastest and easiest way to create one.

No matter which method you choose, working with a neural network to make a prediction is essentially the same:

Import the libraries
. For example:

import numpy as np
Define/create input data
. For example, use

numpy

to create a dataset and an array of data values.
Add weights and bias
(if applicable) to input features. These are learnable parameters, meaning that they can be adjusted during training.
- Weights = input parameters that influences output
- Bias = an
  
  extra threshold value added to the output
Train the network
against known, good data in order to find the correct values for the weights and biases.
Test the Network
against a set of test data to see how it performs.
Fit the model
with hyperparameters (parameters whose values are used to control the learning process), calculate accuracy, and make a prediction.

Create a Neural Network from Scratch

In this example, I’ll use Python code and the numpy and scipy libraries to create a simple neural network with two nodes.

# Import python libraries required in this example:
import numpy as np
from scipy.special import expit as activation_function
from scipy.stats import truncnorm

# DEFINE THE NETWORK

# Generate random numbers within a truncated (bounded) 
# normal distribution:
def truncated_normal(mean=0, sd=1, low=0, upp=10):
    return truncnorm(
        (low - mean) / sd, (upp - mean) / sd, loc=mean, scale=sd)

# Create the ‘Nnetwork’ class and define its arguments:
# Set the number of neurons/nodes for each layer
# and initialize the weight matrices:  
class Nnetwork:

    def __init__(self, 
                 no_of_in_nodes, 
                 no_of_out_nodes, 
                 no_of_hidden_nodes,
                 learning_rate):
        self.no_of_in_nodes = no_of_in_nodes
        self.no_of_out_nodes = no_of_out_nodes
        self.no_of_hidden_nodes = no_of_hidden_nodes
        self.learning_rate = learning_rate 
        self.create_weight_matrices()
        
    def create_weight_matrices(self):
        """ A method to initialize the weight matrices of the neural network"""
        rad = 1 / np.sqrt(self.no_of_in_nodes)
        X = truncated_normal(mean=0, sd=1, low=-rad, upp=rad)
        self.weights_in_hidden = X.rvs((self.no_of_hidden_nodes, 
                                       self.no_of_in_nodes))
        rad = 1 / np.sqrt(self.no_of_hidden_nodes)
        X = truncated_normal(mean=0, sd=1, low=-rad, upp=rad)
        self.weights_hidden_out = X.rvs((self.no_of_out_nodes, 
                                        self.no_of_hidden_nodes))

    def train(self, input_vector, target_vector):
        pass # More work is needed to train the network
            
    def run(self, input_vector):
        """
        running the network with an input vector 'input_vector'. 
        'input_vector' can be tuple, list or ndarray
        """
        # Turn the input vector into a column vector:
        input_vector = np.array(input_vector, ndmin=2).T
        # activation_function() implements the expit function,
        # which is an implementation of the sigmoid function:
        input_hidden = activation_function(self.weights_in_hidden @   input_vector)
        output_vector = activation_function(self.weights_hidden_out @ input_hidden)
        return output_vector 

# RUN THE NETWORK AND GET A RESULT

# Initialize an instance of the class:  
simple_network = Nnetwork(no_of_in_nodes=2, 
                               no_of_out_nodes=2, 
                               no_of_hidden_nodes=4,
                               learning_rate=0.6)

# Run simple_network for arrays, lists and tuples with shape (2):
# and get a result:
simple_network.run([(3, 4)])

Figure 1. Array defined by the random values of the weights:

array defined by the random values of the weights

Create a Neural Network Using Keras

It’s difficult to replicate exactly the Python code in the previous example using Keras, so we’ll create a similar 2-node network model instead.

# Import python libraries required in this example:
from keras.models import Sequential
from keras.layers import Dense, Activation
import numpy as np

# Use numpy arrays to store inputs (x) and outputs (y):
x = np.array([[0,0], [0,1], [1,0], [1,1]])
y = np.array([[0], [1], [1], [0]]) 

# Define the network model and its arguments. 
# Set the number of neurons/nodes for each layer:
model = Sequential()
model.add(Dense(2, input_shape=(2,)))
model.add(Activation('sigmoid'))
model.add(Dense(1))
model.add(Activation('sigmoid')) 

# Compile the model and calculate its accuracy:
model.compile(loss='mean_squared_error', optimizer='sgd', metrics=['accuracy']) 

# Print a summary of the Keras model:
model.summary()

Figure 2. Summary of the Keras model:

Summary of the keras model