Chapter 11: OCR (Optical Character Recognition) – Practical Guide to Machine Vision Software: An Introduction with LabVIEW

11
OCR (Optical Character Recognition)

OCR (Optical Character Recognition) is machine vision software that translates images of characters into text. To identify characters in an image, the LabVIEW software needs to be trained to identify each character. A data set of trained characters is created and saved as a character set file in advance. The set of trained characters are compared with character objects in an image for recognition.

The example of OCR provided in LabVIEW can be found from the following folders:

Figure 11.1 shows the screen-captured image of the example OCR.vi. In this example, characters in the image are recognized and converted into a character string.

Figure 11.1 Example VI for OCR.

Some industrial applications of OCR for machine vision include inspecting labels, sorting, tracking packages, and verifying parts during manufacturing process.

11.1 OCR Using Vision Assistant

11.1.1 Character Training Using Vision Assistant

The OCR function using Vision Assistant offers an excellent procedure to train the software to recognize the imaged characters. For this purpose, complete the following steps:

  1. Launch Vision Assistant and select Open Image to train characters in the image. The image from Figure 11.2 will be used to train the numeric characters as seen in Figure 11.4.
  2. Select the OCR/OCV function icon (Figure 11.5) from Processing Functions: Identification» OCR/OCV. You will then see the menu for the OCR/OCV Setup, as shown in Figure 11.6.
  3. Select New Character Set File from the menu in Figure 11.6. A pop-up window will appear, as seen in Figure 11.7, to train characters.
  4. Drag out a rectangle to create a ROI area around the characters to train in the image as seen in Figure 11.7 . The OCR function automatically separates each character from image background. As a result, there will be small rectangles that will indicate the location of the found image for each character. The character sets can be selected in groups or individually. Then, Train All Characters ( ) is selected to begin the training process.
  5. Key in the corresponding character string in the proper sequence so that the software can equate the imaged characters with the correct text, as seen in Figure 11.7 .
  6. Select Train, as seen in Figure 11.7 , to train the characters in the ROI. It can be the case that the text characters to be recognized are of different fonts. Training the software with all the character fonts to be recognized will produce a more accurate and reliable result. In this example, two different fonts of numeric characters and the math symbols are trained.
  7. Select the Edit Character Set File tab in Figure 11.7 . You will then see the trained characters as shown in Figure 11.8.
  8. You may rename or delete the trained character sets from the Edit Character Set File Tab.
  9. Save the character set file by selecting the File Save icon in Figure 11.8 as the final step for training of a character set. As a suggestion, you might save it as a file named char.abc.

Figure 11.2 Characters to be trained.

Figure 11.3 Characters to be identified.

Figure 11.4 Open image file of characters to be trained.

Figure 11.5 OCR/OCV function.

Figure 11.6 OCR Setup.

Figure 11.7 OCR Training interface.

Figure 11.8 Edit character set file.

11.1.2 Character Identification Using Vision Assistant

In this section, a USB camera is used for image acquisition. To identify characters from acquired images, OCR will be discussed and used to build a VI. This process will use the previously saved character set file. To gain understanding of real-time text identification using OCR, a set of characters is printed out on paper and then a camera is used to take images of the printed paper. Note that the size of the character and image quality of the acquired image can be significantly different from the image that was used for training.

  1. Launch NI Vision Assistant and select New Script from File menu.
  2. Select Acquire Images in Figure 11.9 .
  3. Select Acquire Image (1394, GigE, or USB) if you are using USB camera.
  4. Select the Acquire Continuous Images icon to continuously acquire images (Figure 11.9 ).
  5. Select the Store Acquired Image in Browser icon (Figure 11.9 ) to save the acquired image for image processing using Vision Assistant. Then, select the Image Process icon button in Figure 11.9 to start image processing.
  6. Select OCR/OCV from Processing Functions: Identification, as seen previously in Figure 11.4.
  7. Select Train Tab from OCR/OCV Setup.
  8. Select Character Set Path to load the Characters Set File (Figure 11.10 ), which was saved during the training process (Figure 11.8).
  9. Use the mouse to drag out a ROI rectangle around the characters as seen in Figure 11.10 .
  10. The recognized characters are then indicated by rectangular outlines. The images of the trained characters are also overlaid on the recognized characters. The characters at the bottom of the rectangles correspond to the recognized text. The identified results from the OCR process are shown in Figure 11.10 . Note that the condition of the acquired image using digital camera is significantly different from that of the image used for training character. Nonetheless, the characters from the camera are identified as seen in Figure 11.10.

    You can check whether the characters with different fonts are identified correctly by placing ROI around other examples of imaged characters, as seen in Figure 11.11. As we can see in this figure, some characters may not be identified due to differences in font. Here, the unrecognized character is marked with a question mark, “?.”

    By lowering the Acceptance Level score value in the Read Options tab as seen in Figure 11.12, previously unrecognized characters may be identified as similarly trained characters.

    For example, if the Acceptance Level is reduced from 700 to 300, as seen in Figure 11.12, every character except 74 can be identified. The two numeric characters of 7 and 4 were overlapped in the horizontal directions and cannot be separated by using rectangle ROI. The two characters might be considered as a single character, as seen in Figure 11.13a. Note that there is a trade-off in selecting an Acceptance Level. If a high value is used for the acceptance level, an identified character is likely to match a trained character perfectly. However, there may be other missed characters. On the other hand, more characters can be identified by lowering Acceptance Level, although some characters may not be identified correctly due to the low tolerance. For example, 1 (one) is identified as 7 (seven) in Figure 11.13b. To improve identification results, more character sets with different fonts should be trained during the OCR setup training process.

  11. If the identified results are acceptable, select OK (Figure 11.10 ).

Figure 11.9 Acquisition of image characters.

Figure 11.10 OCR/OCV setup.

Figure 11.11 OCR identification results using acceptance level of 700.

Figure 11.12 Acceptance Level.

Figure 11.13 OCR identification failures. (a) Improper spacing. (b) Incorrect identification due to low acceptance level.

11.2 VI for OCR

11.2.1 VI Creation for OCR Using Vision Assistant

In this section, an OCR VI, which performs real-time character identification, will be discussed. We will use the OCR function to identify characters from an imaged printed paper. A USB camera is used to acquire images of the printed characters. To identify the characters from the acquired image, a VI created using Vision Assistant will be modified as a SubVI.

You need to first select Tools»Create LabVIEW VI (located in the NI Vision Assistant tool bar menu) upon finishing the OCR setup in Vision Assistant (Section 11.1). From the LabVIEW Create Wizard – Step 3 of 4, select Image Control (Figure 11.14) to use this VI as a SubVI, which will receive acquired image from main VI.

Figure 11.14 Selection of the image source.

By selecting Next in the LabVIEW VI Creation Wizard – Step 4 of 4, you can select Controls and Indicators, as seen in Figure 11.15.

Figure 11.15 Selection of controls and indicators.

Here, Image Src (source) and Image Dst Out (destination) are automatically selected for controls and indicators in Figure 11.15 so that an acquired image can be received from main VI and the processed image can then be returned.

11.2.2 SubVI for OCR

To read characters within an image, the LabVIEW VI created from Vision Assistant is modified to a SubVI having inputs and outputs as seen in Figure 11.16.

Figure 11.16 OCR SubVI.

Figure 11.17 shows the block diagram of a SubVI for performing the OCR, which has been slightly modified from the VI created with Vision Assistant. As seen in Figure 11.17, controls of ROI Descriptor and the acquired Image are used as inputs of the SubVI to receive information from the main VI. As a result of OCR processing, the character string (Read String) is obtained and used as an output of the SubVI.

Figure 11.17 OCR SubVI block diagram.

To use the OCR function, a character set file having trained character information is required. For this purpose, the character sets file (file name: char.abc) saved previously in this chapter is used. Figure 11.17 shows the control for the path of the character set file. The previously selected Parameters values for OCR character identification can be found in Figure 11.17 . After reading in the character set file (Figure 11.17 ), the characters in the image are identified by using the OCR Read Text function (Figure 11.17 ). The resulting recognized text corresponding to each identified character is in the Read String indicator shown in Figure 11.17 . To draw an overlay on each identified character, the overlay function in Figure 11.17 is used. Note that IMAQ Overlay ROI shown in Figure 11.18 is used since ROI descriptor bounding the characters from the OCR Read Text function is compatible with this overlay function.

Figure 11.18 Overlay function.

Figure 11.19 shows the first result in the Character Reports indicator in Figure 11.17 . The Character Reports provides useful information for debugging purpose. The Character Reports has the form of a cluster array with the size of the array equal to the number of characters identified. Each array element of Character Reports contains information such as character value, score, character location, and character size.

Figure 11.19 Character Reports.

11.2.3 Main VI

Figure 11.20 shows the block diagram of a main VI for real-time OCR analysis. The SubVI for OCR described in Figure 11.16 has been used to perform the OCR analysis.

Figure 11.20 Main VI for real-time OCR analysis.

Figures 11.21 and 11.22 show the character identification results when the main VI is executed. To identify characters in the image display, a ROI is placed around the character set to be identified. As seen in Figure 11.21, character identification is possible even though character fonts, as well as the font size, are different from the character set file. Also, you may rotate the paper to check if OCR function can read characters properly. In this example, character identification was possible in cases where the paper is slightly rotated.

Figure 11.21 Character identification results (success).

Figure 11.22 Character identification results (failure).

However, if the image is not clear and/or the characters do not have proper spacing, character identification may fail as seen in Figure 11.22. Also, the characters in the image need to have a minimum size, which is a measure of the total number of pixels in a character. It is recommended in NI Vision Concept manual that the character size be more than 25 pixels to successfully identify characters using the OCR function.

Note that identified character sets include numbers. If a numeric calculation is needed from the numerals identified, they will need to be converted from text to numeric values before mathematics functions are used.

Figure 11.23 OCR identification and the numeric calculation.

Figure 11.24 LCD image.

Exercise 11.1

Create an application that can read number and symbol characters of an equation, complete the calculation, and return the results. For example, the results of Figure 11.23 would be 98.

Exercise 11.2

Find image files from C:\Program files\National Instruments\Vision\Examples\Images\Lcd. Build an application that reads the characters from the images, as seen in Figure 11.24.