Automated Signature Verification Using Siamese Network

Signature verification is a critical task in identity verification. Unlike other verification (e.g face) it requires a detailed analysis of genuine and forged signatures. This task has huge applications in banks where they have to verify signatures on a day-to-day basis. As we humans are prone to error let’s see if AI can help us in cracking this.

SIAMESE NETWORK:

Siamese network has been very popular for tasks like face verification where we take two facial images as input and compare their encodings by means of a similarity score. This score tells us how similar they are instead of classifying into two different classes. The following block diagram shows the structure of a basic siamese network.

(Siamese Network)

Here to the network, we provide two images as input which goes through two different convolutional neural networks which are exact replica of each other. Both the images pass through the network and produce a certain embedding. We try to find out the difference between the encoding by means of a loss function (e.g euclidean distance). Thereby we add a dense layer with a sigmoid unit to output a similarity score. The key idea is, more the similar are two images more similar would be their encodings. Hence the similarity score that is produced by the sigmoid unit.

Signature Verification:

For any machine learning problem, we need a dataset to check the validity of our model. In this case, we can use the ICDAR Signature Verification Database. It is already labeled providing zero label to dissimilar pair of signatures and 1 to similar signatures.

Preprocessing:

In the preprocessing stage, we will resize all the images into a certain size as all the images available can be of different sizes. We can take the help of methods like bilinear interpolation to achieve the same. We shall also make the background pixels zero by inverting it as most of the background pixels would be white or would be having the value 255. So by subtracting 255 from it will make it zero or we can say the background pixels have been zero.

Modelling:

After the preprocessing step we shall use our siamese network to get the encodings of the image. We would use keras library to do the required task. Siamese network works such a way that we have to increase the similarity of signatures that belong to the same person and decrease the similarity of dissimilar signatures. This way the parameters of our network are updated. This procedure ensures that the highest level representation (i.e output from last layer) will have a similar feature space. This is possible because the function computed by both the convolutional neural networks is apparently the same. We can create the model in keras as below

def signature(input_shape):
     left_img=Input(input_shape)
     right_img=Input(input_shape)
 
     model=Sequential()
    model.add(Conv2D(96,  (11,11),activation=’relu’,input_shape=input_shape))
    model.add(MaxPooling2D())
    model.add(Conv2D(256(5,5),
             activation=’relu’,input_shape=input_shape))
    model.add(Conv2D(384,      (3,3),activation=’relu’,input_shape=input_shape))
    model.add(MaxPool2D())
   model.add(Conv2D(256(3,3),activation=’relu’,input_shape=input_shape
)
    model.add(Flatten())
    model.add(Dense(128,activation=’sigmoid’))
    enc_1=model(left_img)
    enc_2=model(right_img)
    L1_layer=Lambda(lambda tensors:K.abs(tensors[0]-tensors[1]))
    L1_distance=L1_layer([enc1,enc2])
  prediction=Dense(1,activation=’sigmoid’,bias_initializer=initialize_bias)(L1_distance)
 signature_mod=Model(inputs=[left_img,right_img],ouputs=prediction)
 return signature_mod

The above code snippet shows the structure of a siamese network that we are going to use to find similarity or dissimilarity amongst signatures. As we can see initially we have used 96 (11*11) kernel which detects low-level features from images. Similarly in the next layers, we have used kernels of sizes (5*5),(3*3), etc. In the end, we have flattened the layers and converted them into a 128-dimensional vector by a dense layer. Now to compute the similarity between the encodings we have taken the help of the Keras lambda layers which computes the euclidean distance between the vectors. As an advanced approach, we can use the contrastive loss as well. The final dense layer with a sigmoid activation function shall compute the similarity score of the signatures.

(Image Source:Research Gate)

We will train the network with the already available labels and dataset. We can use the adam optimizer with binary-crossentropy as the loss function. Also, we can play with more number of hidden layers and various numbers of kernels to find a better model. Also, advanced distance metrics can be used to compute the distance between encodings.