https://onedrive.live.com/?authkey=%21AH9ufP8gPMnMMLA&cid=2D07F5043DA09852&id=2D07F5043DA09852%2186606&parId=2D07F5043DA09852%2186340&o=OneUp

Machine learning is used in image processing in a wide range of scenarios – e.g. passport control, facial recognition in security scenarios, handwriting recognition, robot vision and driverless vehicles.

An analysis of the principles of machine learning that are applied to image processing can reveal more general machine learning principles that can be applied in other scenarios – voice recognition, for example.

Turning Pictures Into Numbers.

Let’s start by exploring how images can be represented digitally, because once we have numbers in an array we can perform machine learning methods on them.

Open and run 5.5 Classifying Images/Visual Experiment/Vizexp.py

Figure 63. Image, code, data, and colour map

The resulting plot is a colour map of the data contained in a PNG file.

Note the following line in the code –

M = misc.imread('Singlecolour.png', flatten=True)

By changing ('Singlecolour.png'...) we can instruct the algorithm to work with other data files - in this case, PNG files.

Note that when we run the program with it reading the file ‘Singlecolour.png’, it shows an array comprising of just one number (112.4...), and the distribution map of that colour is flat and even.

Now Change 'Singlecolour.png' to 'B&W.png' and Run the code again.

When we change the file to B&W.png we can see that the array now has two colours – 0, Black, and 255, white. The amount of memory that the file takes is bigger, partly because it has more information than the Singlecolour.png file.

Figure 64. More features in the colour map due to more colours present in the image file

Finally, when we look at RGBpic.png we see a full range of numbers, a much wider distribution, and a much bigger file size.

Figure 65. Again, more features in the colour map due to more data in the image file

Looking at them all together –

Figure 66. Each image file and their corresponding colour map

Turning Number Into Pictures.

So we can convert a picture to numbers in an array, but we can also create a picture from numbers in an array.

Open 5.5 Classifying Images/Pic to and From Data/Generate image from array.py

Figure 67. Turning numbers into images

Here, we’ve reduced the colour palate from 255 colours to just 4. Here, black is 0 and white is 3, and we have two shades of grey between.

Run 1. Generate_image_from_array.py

Figure 68. Output from number to image algorithm

Your task

Now experiment with the numbers in the array.

'Round-Trip" Image Data.

Now let’s bring picture into an array, then turn it back into a picture. The purpose of this is just to illustrate how we can process a full cycle of image processing.

Open and run 5.5 Classifying Images/Pic to and From Data/2. Image_to_array_to_csv_and_back.py

The process works like this:

First the program reads the file, in this case ‘Face.png’

Next, the array is exported as a .txt file

Then this external text file is read by the program, and a new array is created, from which a picture can be plotted.

Figure 69. Image to data and back to image

Let’s look at how the code works -

Figure 70. Image 'round-trip' code analysis

Handwriting Recognition.

So now we know how images can be processed, lets now explore a practical application – handwriting recognition. We will do this by classifying images.

In the following exercise, we are going explore how we can compare arrays to find similarities in order to recognise an image.

Will use 10 image files, each with a handwritten digit between 0 and 9.

We will then bring in another digit and see if the algorithm can find the correct match.

Note that this is an extremely crude version of image recognition, but it shows some basic techniques.

In the first instance, we are looking for a handwritten digit ‘8’.

https://onedrive.live.com/?authkey=%21AH9ufP8gPMnMMLA&cid=2D07F5043DA09852&id=2D07F5043DA09852%2186606&parId=2D07F5043DA09852%2186340&o=OneUp

Figure 71. Classifying Images video

Classifying Images

First look in the folder - 5.5 Classifying Images/Digit Recognition/

Note that 0.png - 9.png is the training data.

10.png, 11.png, and 12.png are test data.

Click on 10.png to Open it using Image Viewer.

Figure 72. Handwritten digits - open with Image Viewer

Test data 10.png looks like digit '8'.

Now open and run Digit_recogniser.py and note that the image that appears is the same as 8.png in the Training Data.

Figure 73. Image recognition code recognises a hand written digit

It worked!

Let’s now see how this happened.

The goal of the code is to turn the training images and test data into arrays, simplify, compare and find the closet fit.

Figure 74. Test, training and recognised digits

Looking deeper, we see that the algorithm is searching through arrays to find the training data array with the best fit for the test data.

The test data (10.png) array has a value of 14479...

The Array for training data 0.png has a value of 54479...

The Array for training data 8.png has a value of 14481... which is the closet fit

Figure 75. The recognition mechanism

Let’s now look at the code in Digit_recogniser.py

The 4th line, starting with traindigits = [] sets up the first set of arrays. It reads each training file and the test image file, turns it into a numbered array.

traindigits = []

for i in range(11):

It then calculates a coefficient for each array, in other words it finds patterns which it enumerates and assigns to the array. This makes it easier to compare arrays.

  B, c, D = np.linalg.svd(A, full_matrices=False) 

We next feed in the number of the test image file.

testdig = traindigits[10] 

Then, we look for the training data array that has the lowest difference between itself and the test digit array

recogniseddigit = min(traindigits[:10], key=lambda e: sum((e['singular']-testdig['singular'])**2))

Finally, we show the test data image file that corresponds to the test data array with the lowest difference.

plt.imshow(recogniseddigit['original'], interpolation='nearest', cmap=plt.cm.Greys_r)

plt.show()

Looking at it all together

Figure 76. Digit recogniser code analysis

It's critical to note that a commercial application of image recognition would use much more training data than we have done here. For example, each of the digits below may have been inputted as hundreds, thousands or even millions of different images. This way, accuracy can be greatly increased.

Your Task.

Try different images by renaming them to 10.png

Figure 77. Sequence for trying the digit recogniser with other digits