1.Introduction

In this assignment, my aim is to implement an automatic bottles faults detection system based on embedded image processing techniques using Matlab software. There were seven different kinds of faults including cap missing, labels missing, blanked labels, non-straight labels, deformed bottles, over and under fillings. For each different faults, I designed some respective functions to detect and report them. Besides, all images contains three bottles given by a shot screen of camera as shown in figure 1.1 below. Besides, the most significant thing is that I only focus on the middle bottles of all testing images (Edward 2019).

Figure 1.1: Sample normal image

2. Basic Workflow & Image Preprocessing

2.1 Basic workflow

The below flowchart illustrates the overall workflow of all faults detection. This flowchart mainly shows the logic relationship between each fault detection.

Figure 2.1: The workflow of faults detection

As shown in figure 2.1, this system starts at detecting missing of bottles, and then detects whether bottles are deformed, cap missing, over or under filling or not. After that, it will check if the label missed. Furtherly, it will check if the labels were blank or not straight provided that the label not missed.

Image Preprocessing

In this section, it will start the detection process. The first thing is image preprocessing when the system receives an image. 

function [gray_img,bin_img]=pre_process(filename)
m_img=imread(filename);  
gray_img=rgb2gray(m_img);
best_thre_value=graythresh(gray_img);
bin_img=im2bw(gray_img,best_thre_value);
end

First of all, I use imread() this function to load an image and save in m_img object. After that, I applied rgb2gray() this function to convert the true color image into gray scale. Finally, I used both graythresh() and im2bw() two functions together to find the optimal threshold which convert the grayscale image into binary image furtherly. All detections focus on the comparison of black and white pixels within binary images. The example image was shown below.

Figure 3.1: Binary Image

3.Faults Detection Implementation

In this section, I mainly focus on the functionality of each fault detection technique.

3.1 Missing Bottles

The first thing the system need to do is to ignore the missing bottles of the image. The code segment below illustrates the process. 

function [judge]=missing_bottle(bin_img)
m_zone=bin_img(:,135:220);
[counts,bins]=imhist(m_zone);
judge=0;
if counts(1,1)<10 judge=1;
else
judge=0; end
end

As I mentioned previously, I have got the binary images, and I selected an interesting area (The blue triangle) from the image, and observed the white and black pixels using imhist() function which gives the distribution of black and white pixels. I observed a common pattern from all missing bottles images and selected a rectangular area (the columns from 135 to 220) as interesting area as shown in figure 4.1. And I select a threshold as condition that counts (1, 1) (The number of black pixels) is less than 10.

Figure 4.1 Original missing bottle image with binary image.

3.2 Missing Caps

For the detection of missing caps, I also used the technique which is similar to the detection of missing bottles. While this time I picked different interesting area of image. The code segment below illustrates this process. 

function [judge]=missing_cap(bin_img)
zone_area=bin_img(1:50,120:250);
[counts,bins]=imhist(zone_area); judge=0;
if counts(1,1) <200 judge=1;
else
judge=0;
end end

For this time, the area I selected is around the cap (rows from 1 to 50, columns from 120 to 250) (The blue triangle area).

</tr>
Figure 3.2.1: Normal Image Figure 3.2.2: Tap missing image
Figure 3.2.3 Binary Image with cap area Figure 3.2.4 Binary Image without cap area

These four pictures shows the colorful image with cap, colorful image without cap, binary image with cap and binary image without cap. Obviously, the image which missed tap have more white pixels, and I set the threshold of black pixels (200) to detect them. In fact, there exist another one default, over-filling, I would discuss in later section.

3.3 Labels

3.3.1 Label missing & Label is blank

Another one fault is about label missing. Furtherly, this fault could be classified into sub- problems, not-straight label and blanked label providing that label is not missing. For the label faults, I detected the label missing and blanked label at the same time. The reason is that there are some logic relationship between these two faults.

Figure 3.3.1 Label Missing Figure 3.3.2 Label not printing
Figure 3.3.3 Missing label binary image area Figure 3.3.4 Blanked label binary image area

Same as two previous faults detection, I also selected an interesting area (row from 185:250, and column from 130 to 225). The figure 3.3.1 and 3.3.2 are two colorful images about missing label and blanked label images respectively. The figure 3.3.3 and 3.3.4 are two binary images of them. Obviously, compared with these two binary images, the missing label image is filled with black and blanked bottle is filled with white respectively. Therefore, I also set two lower thresholds to detect if label missed or blanked label. While, these two events are conflicted, if there was no label detected, and there would not be blank label definitely. The below code segment shows the detection process of missing label and blanked label. I declared two bool variables, judge1 (for missing label) and judge2 (for blanked label). 

function [judge1,judge2]=missing_or_blank_label(bin_img)
zone_area=bin_img(185:250,130:225);
[counts,bins]=imhist(zone_area);
judge1=0; judge2=0;
if counts(1,1) <70 judge1=1;
end
if counts(2,1) <10 judge2=1; judge1=0;
end
end

3.3.2 Label not straight

Another one possible fault existing in labels is not-straight label. For this fault, I selected an area around the top boundary of the label. (rows from 180 to 190, columns from 150 to 200)

Figure 3.3.2.1 Normal Image Figure 3.3.2.2 Label not straight Image
Figure 3.3.2.3 Binary Image for normal area Figure 3.3.2.4 Binary Image for fault area
Figure 3.3.2.5 Label not straight image Figure 3.3.2.6 binary image for faults

As shown in figure 3.3.2.3 and 3.3.2.4 clearly, the proportion of white and black pixels of normal bottle was in a specific range, as I observed form all faults images in folder. The code segment below illustrates the process. (The black pixels should be in the range from 230 to 300, and white pixels should be in range from 240 to 330). 

function [judge]=label_notstright(bin_img)
zone_area=bin_img(180:190,150:200);
[counts,bins]=imhist(zone_area);
judge=1;
if counts(1,1)>=230 & counts(1,1)<=300 judge=0;
end
if counts(2,1)>=240 & counts(2,1)<=330 judge=0;
end
end

3.5 Over-filling

Besides, over-filling is also a significant fault. For this case, the area I selected is the small part above than normal liquid level, as shown in figures 3.5.1 and 3.5.2. (rows from 120 to 130, and columns from 150 to 200).

Figure 3.5.1 Normal Image Figure 3.5.2 Over-filling Image
Figure 3.5.3 Binary Image selected area Figure 3.5.4 Over-filling selected area

The code below segment illustrates this process. Typically, compared with the pixels distribution, I easily observed that the fault bottle was filled with black pixels. Hence, the basic idea is to compare the amount of black pixels. 

function[judge]=over_filling1(bin_img)
zone_area=bin_img(120:130,150:200);
figure;
imshow(zone_area);
[counts,bins]=imhist(zone_area);
judge=0;
if counts(1,1) >30 judge=1;
else
judge=0; end
end

3.6 Under-filling

Similarly, the detection of under-filling is similar to over-filling.

Figure 3.6.1 Normal Image Figure 3.6.2 under-filling image
Figure 3.6.3 Binary Image for normal area Figure 3.6.4 Binary Image for faults area

Obviously, observed from figure 3.6.3 and 3.6.4, and I can concluded that the under-filling area was filled with white. While black pixels take accounts of large amount in the normal image. The reason why there was a small white area is that the reflection of lights, but it will not affect my detection. Compared with the over-filling detection, the core idea is to compare the number of white pixels. I set a threshold 150 to filler them. 

function[judge]=under_filling(bin_img)
zone_area=bin_img(150:165,150:200);
[counts,bins]=imhist(zone_area);
judge=0;
if counts(2,1)>150
%White pixels number judge=1;
else
judge=0;
end end

——————— Continuing In The Next Post —————————