Chapter 13: Contour Analysis – Practical Guide to Machine Vision Software: An Introduction with LabVIEW

13
Contour Analysis

Contour analysis is often used to detect defects in objects by analyzing the object's contours. Boundary curves of objects are connected to produce contours. For this purpose, the objects in the image should have clear boundary edges to accurately distinguish its location from the image background. Since boundary curves are a set of edge points, the color image should be converted to 8 bit grayscale or binary image for the edge detection.

Figure 13.1 shows the basic process for contour analysis to find defects in objects.

Figure 13.1 Defect detection process using contour analysis.

To analyze the contour of objects, curvature profile is often used. The curvature profile is effective in detecting defects in places where the contour changes abruptly. However, some objects may have abrupt contour changes even though there are no defects. In such cases, defect detection using the curvature may be impossible.

On the other hand, you may use a reference template image to detect defects in an object by the comparison of contours. Alternatively, if the contour has a standard geometric general shape, such as line, circle ellipse, polynomial, and B-spline, the contour of the object may be numerically fitted to find abnormal parts in an object.

The example for detecting defects of an object can be found from the following folder:

Figure 13.2 shows the front panel for the example VI, Contour Defect Inspection.vi.

Figure 13.2 Example VI for defect detection using contour difference (reference: circle).

As seen in Figure 13.2, certain types of defects can be effectively located by using contour analysis. The method shown in Figure 13.2 uses fitted circle for comparison. The difference (distance) in the contours compared with the fitted circular shape is used to find defects in this object.

In some cases, curvature profiles can be used alone and do not require comparison with a fitted shape, reference template, or reference contour. Applying the method of curvature profiles may be easier to use in some applications. Figure 13.3 (contour analysis.vi) shows the example of using curvature profile to detect abnormalities in a simple pattern.

Figure 13.3 Example VI for defect detection using curvature profile (Contour Analysis.vi).

13.1 Contour Analysis

13.1.1 Image Acquisition Using a USB Camera

  1. Drag the Vision Acquisition Express icon from the function palette to the block diagram.
  2. Complete the creation of a continuous acquisition with inline processing as seen in Figure 13.5. The details can be referenced from previous chapters.
  3. Right click the mouse on the front panel image display in Figure 13.5b and save the acquired image for image processing using Vision Assistant.

Figure 13.4 Defect detection using contour.

Figure 13.5 Continuous image acquisition. (a) Block diagram. (b) Front panel.

13.1.2 Contour Analysis Using Vision Assistant

  1. Drag the Vision Assistant Express icon from the functions palette to a block diagram.
  2. Select Open Image to open the previously saved image file for contour analysis.
  3. Convert the color image to grayscale or binary for detecting edges since contour of an object (boundary edge curve) is extracted from a grayscale or binary image. For this process, select Luminance plane from HSL from Color Plane Extraction function.
  4. Select the Contour Analysis function icon from Processing Functions: Machine Vision in Vision Assistant, as seen in Figure 13.6.
  5. Appropriate parameter values need to be selected for the contour analysis setup. Figure 13.7 shows the example of the Extract Contours tab under the Contour Analysis Setup to extract contours of the objects. The setup values should be adjusted by observing the perimeter contour line overlay on the objects in the displayed image. If the setup values are correct, the overlaid line lies exactly along the contour of the objects within ROI, as seen in Figure 13.8.

Figure 13.6 Contour analysis.

Figure 13.7 Contour analysis setup.

Figure 13.8 Reference image for contour analysis.

13.1.3 Defect Detection Using Curvature

Select the Analyze Curvature tab to verify from the Curvature Profile that the contour extraction is successful. Since there are no defects, the graph in Figure 13.9 shows no significant variation throughout the curvature of the object's boundary.

Figure 13.9 Curvature analysis of contours with no defects.

For the case of the notched ellipse, Figure 13.10b shows how the curvature profile reveals defects, as seen in Figure 13.10a. Note that you can change Kernel size such that the curvature profile can adequately detect possible defects. You can see the sudden large value change in curvature profile as seen in Figure 13.10b . In this way, you can identify the location of the defect as well as determine its severity. Note that the value of curvature profile is not related to amount of defects, but to the rate of change of the contour. For example, the defect in Figure 13.10a is smaller in size than that of the defects along and . However, the curvature graph shows a significant value in defect at . The detection of the defect based on curvature is effective in identifying defects due to irregularity of the contour (smoothness).

Figure 13.10 Curvature profile of an object with defect. (a) Defects. (b) Curvature profile with defects.

13.1.4 Defect Detection by Comparing Contours

In some cases, objects may intentionally have irregularities to their shape, which do not indicate a defect in the object. In the presence of an irregular shape, which is normal for the object, large values of curvature profile may be observed. Thus, defect detection using the curvature profile method may not be appropriate. In these cases, the comparison of contours using a reference template image is perhaps better choice for the detection of defects.

The two different approaches may be used to compare contours: the use of fitted data with a reference contour as seen in Figure 13.11 and the use of reference template image as seen in Figure 13.11 . In this section, the method using the reference template will be discussed. This method uses the contour of an object and a reference template to detect defects by comparing the two contours. To use the reference template method, we need to create a template file. Complete the following steps to detect defects based on template image.

  1. Select New (Figure 13.11 ) to create template file. A pop-up window will appear to create a new contour template as seen in Figure 13.12.
  2. Identify the reference template image area by using a ROI, as seen in Figure 13.12 .
  3. Select OK and save the template contour to a file in the form of a .png image. As a suggestion, save the contour in a file named reference.png. After saving the reference contour, you can drag out a ROI around the target object to detect defects by means of distance (Distance) between the contour and the reference contour. Figure 13.13b shows the distance graph of the defective object. From Figure 13.13a and b, the point shows the largest difference and is easy to detect based on the compare contour method. However, even though the curvature is very large, the defect area at in Figure 13.13 may not be noticed using this detection method since the contour distance is relatively small.

Figure 13.11 Contour analysis setup.

Figure 13.12 Contour template.

Figure 13.13 Defects detection using template contour. (a) Defects. (b) Contour analysis setup.

The defects can be detected by comparing the distance against a threshold tolerance. If the distance is larger than the threshold at a location, the location can be identified as a defective part. An overlay on the image can be used to show the detection results effectively. The details of this will be discussed later.

13.1.5 VI Creation

When the contour analysis setup is complete, the results of the analysis are indicated as an overlay on image, as seen in Figure 13.14.

Figure 13.14 Vision Assistant Express for contour analysis.

As a final step, click on Select Controls to choose the Controls and Indicators, as seen in Figure 13.15.

Figure 13.15 Selection of controls and indicators.

As seen in Figure 13.15, you can access two different results of contour analysis in the forms of Curvature Profile and Distance. In this way, you can choose contour results according to your requirement of defect detection.

13.2 VIs for Contour Analysis

13.2.1 Main VI

After finishing the selection of controls and indicators, a Vision Assistant Express VI can be created, as seen in Figure 13.16 . To build up the main VI, you may want to connect the inputs and outputs of the Vision Assistant Express VI as seen in Figure 13.16.

Figure 13.16 Main VI for contour analysis.

In this example, two different methods (with and without reference contour) are compared to detect defects using contour analysis. The two methods include the use of curvature profile and distance between contours.

The template image, which was saved previously using Vision Assistant, is used for contour comparison based on distance.

The acquired image (Image_In) is used as the input of the SubVI and the ROI (Figure 13.16 ) is used to define the image area of the object for defect detection so that the object in the ROI is compared with the template image. Note that the Contour Analysis method based on distance is related to the direct comparison of the target object with the template object. Therefore, the template and target objects should be matched properly since the target object may be rotated as well as translated in the ROI region. Therefore, the matching process should be included in the contour analysis to acquire the intended results.

As an alternative method, curvature profile of the object can be used to detect defects without a reference contour.

The results of Contour Comparison between target and template contour points are in the form of cluster array, as seen in Figure 13.17. In the cluster, you can find the template contour (Template Point) and matched contour locations (Target Point) in their X and Y locations. Also, the distance between two contours can be obtained.

Figure 13.17 Contour analysis result (Distances).

For better understanding of extracted contours of objects, the Template Points and Target Points can be plotted on a graph, as seen in Figure 13.18.

Figure 13.18 Template and target points.

Based on the contour of the template and target points, curvature and distance can be obtained as seen in Figure 13.19a and b, respectively.

Figure 13.19 Curvature profile and distance. (a) Curvature profile. (b) Distance.

As seen in Figure 13.19, a horizontal line cursor can be added to the graph and used to graphically modify the threshold value from within the graph. In this way, the setting and modifying of threshold value can be made easy and straightforward. To add the line cursor, right click the mouse on waveform. Then, in the pop-up window, you can add cursors from Cursors tab of Graph Properties menu, as seen in Figure 13.20.

Figure 13.20 Cursor creation from waveform graph.

You can now move the cursor on the graph and the value at the cursor can be used in the block diagram. To access this value, right click the mouse on the graph indicator in block diagram (Figure 13.16 ). You will then see the pop-up window to select Create»Property Node»Cursor»Cursor Position» Cursor Y. From the created property node, you can obtain the cursor value from the graph, as seen in Figure 13.16 . The value can then be used as the threshold value for identifying defects.

13.2.2 Overlay for Defects

An overlay of the template contours and target contours for comparison are automatically generated from Vision Assistant Express. However, you may want to add a highlight overlay to indicate the defect area. To detect possible defects, a threshold value is used to identify severe defects by comparing the value with the curvature variation or the distance. If the distance value is higher than the threshold, the location can be classified as defective part. The defective parts can be effectively shown on the image display by using overlay SubVI. Figure 13.21 shows the inputs and outputs of the SubVI.

Figure 13.21 Inputs and outputs of overlay SubVI for defects.

Figure 13.22 shows the block diagram for the SubVI to overlay multiple lines on defected parts. To overlay highlights on defective parts, the locations for the distance or curvature profile greater than the threshold value will be added to an array receiving the defect location information.

Figure 13.22 Block diagram for defect detection and overlay. (a) Distances above threshold value. (b) Distances below the threshold value.

For defect detection, we will discuss the method of using the distance values. As seen in Figure 13.22a, if the distance is larger than the threshold value, the size of array of detects location will be increased accordingly () and the array will be plotted using overlay multiline function. On the other hand, if the distance is lower than threshold value, the locations will not be used for overlay, as seen in Figure 13.22b.

To overlay on defect locations, the overlay function of IMAQ Overlay Multiple Line 2 in Figure 13.23 is used, which can be found from Vision and Motion»Vision Utility»Overlay. As seen in Figure 13.22, line segments (Line End Points) specified with X and Y location data are used as input to the IMAQ Overlay Multiple Lines 2 function.

Figure 13.23 Overlay Multiple Line function.

Figure 13.24 shows the overlay results that use the threshold value from the Distance graph. As seen in Figure 13.24, the defect location can be identified and effectively indicated on image display by using the overlay SubVI.

Figure 13.24 Identified defects based on the threshold value.

Note that defect detection may not always be successful because of image distortions due to perspective errors. When object under inspection is not located on the same position where template image is taken, the results are likely to be affected by perspective errors depending on camera alignment. The perspective issue in image will be discussed in Section 13.2.3 and Chapter 14.

13.2.3 Perspective Errors in Images

As seen in Figure 13.25, the shape of an object and its contour is affected by the alignment of the camera. So, if the camera is not exactly perpendicular to the object, perspective errors will occur. The distortions include size as well as the shape. Due to the distortions, it may be difficult to determine the defects from the contours analysis.

Figure 13.25 Distorted image due to perspective error (size, shape).

Therefore, it is advisable to position your camera perpendicular to objects under inspection. In the case that the camera position may not be controlled, the distortion can be corrected using software. The calibration method for correcting distortions using the software will be discussed in Chapter 14.

Figure 13.26 Example images for defect detection.

Exercise 13.1

Find images in C:\Program Files\National Instruments\Vision\Examples\Images\Cans.

Use Vision Assistant Express to build your own VI to detect the defective part of the cans (Figure 13.26). You may refer to the Contour Defect Inspection.vi in Figure 13.2.