Explanation of the Neural Net

While abstraction is essential to solve problems efficiently, software engineers are required to keep the bigger picture in mind. Abstraction is useful but abstraction without the knowledge of the entire problem can lead to failure in the developmental process. The previous chapters were very theoretical and abstract. Here is an example, that practically implements the concepts from above to show a real-world implementation.

The dataset can be found HERE.
The code can be found HERE but might be subject to change.

Problem Statement

The reading of a multimeter screen should be saved in a database. Since the multimeter has no serial port and therefore cannot be connected to a computer, the only option is to use computervision. The algorithm needs to read the image and save the numerical values to a csv file. To simplify the task, the camera gets mounted in a fixed position above the screen. This keeps the pixels occupied by the 7-segment display roughly at the same place. The advantage is, that the locations can be hardcoded, and manual bounding boxes can be used to extract the individual images from the original. The algorithm controlling the camera pre-processes the images to appear monochrome. Unfortunately, the jpg format only allows the storage of RGB images, disallowing the potential for data compression and requiring slight adaptations of the neural net.

Data

The raw data consists of images in jpg format. With simple algorithms these images get converted to a monochrome style and 3 individual digits get extracted and are passed on as individual jpg files. The localisation of the digits has been completed by traditional algorithms (algorithms that were manually programmed). The desired output of the network would be the corresponding float value to the original image, containing all 3 digits. Due to the digits being passed on individually only individual digits are being returned by the network. These individual digits must be reassembled by an algorithm. The floating point on the LCD keeps its position, simplifying the process. The training data needs to be composed manually. This very time-consuming and monotonous process must be done by humans. The individual digits are labelled with the digit they display. Each digit has a corresponding integer (0 to 9). The PyTorch framework provides the class ImageFolder for handling image datasets. (View the documentation for the folder structure.)

ANN

The ANN of this example is no deep neural network since it is only comprised of 2 hidden layers. The architecture of the layers must adapt to the input and output data. The neuron layers are defined in the class constructor. The net consists of a convolutional layer, a pooling layer, another convolutional layer, and a fully connected layer for the output. The forward-pass defines the dataflow and implements some matrix transformations alongside the neuron layers. These transformations are necessary to link layers with different properties like in this case the second convolutional layer with the linear output layer. In the first convolutional layer, the kernel is moved along the image and compresses 3x3 pixels to a single one. This pixel gets handed over to the next layer. There are 3 input channels due to the image being in rgb format. The 3 input channels signal that these convolutions are taking place 3 times, for each colour once. There are 8 output channels passing the data to the next layer. To link individual layers, the following layer must take input in the same shape as the output from the previous layer. Therefore, the output layer takes in 44400 values which is exactly the output of the third layer (convolutional). The output layer outputs a tensor of 10 float values. The index of the largest float value is the prediction of model. The larger a value the more confident the model is, that the input image belongs to that label.

Training

This example uses the CrossEntropyLoss as a loss function and the Adam optimizer.
Before the training process the dataset is handed over to a dataloader. Dataloaders are used to simplify the handling of datasets. Here for example the train_loader sets the batch size to 64 samples per batch. More important: the train_loader balances the dataset, so no unwanted bias of the network occurs. The training process involves cycling over the entire dataset (now accessed via the dataloader) multiple times. Each iteration is considered an epoch. Within each epoch all the data of the dataset is passed through the model. This data is split into batches, the reason for this decision being that the backpropagation only occurs after an entire batch and not after each datum. This saves time and helps to prevent overfitting of the model. After the batch has been processed, the loss function is used to calculate the loss between the outputs and the desired targets. The loss then gets backpropagated over the entire network, so the error contribution of each individual neuron gets calculated. Finally, the Optimizer optimizes the weights of the model according to the calculated error contribution from each neuron. The learning rate specified at the initialisation of the optimizer defines the size of the weight manipulation, the higher the learning rate, the larger the optimisation steps. Once the training process is completed, the model should be capable of classifying similar images to the ones it was trained on.

Evaluation

After the training is completed, the accuracy of the model must be evaluated by passing another dataset, that holds images that the model was not trained on, through the model. Percentual accuracy (correct predicted samples / total number of samples * 100) of the model gets calculated at the end of the evaluation process. The evaluation process is very important to ensure the functionality of the model. From experience, it can be said that the initial model architectures and hyperparameters result in poor results. Only after a lot of debugging and parameter variation the predictions start getting satisfactory. At this point it should be mentioned that there are superior ways for evaluation as pointed out in this Stanford lecture.

Deployment

The final phase of implementing machine learning is the deployment of the neural network. After training and evaluation (both must show satisfactory results), the model is ready to be deployed in its intended application. The model is used like a function and gets called with the input image as a parameter. The return value of this function is the prediction. The status of the model gets saved after training and evaluation. All the weights are now available after loading the saved model from a file. Finally, it must be ensured that the input and output of the neural net has the desired format. The Core of the network can only handle PyTorch tensors so the images need to be transformed to tensors and the output must be transformed into an integer.