Role of Machine Vision in Sustainable Manufacturing

MAY 27, 2010 (ASIA PACIFIC) - As resource shortages continue to be the norm in manufacturing industry, it is essential for a company to operate in an environmentally sustainable way. One of the Factory Automation forefront technology delivering value of minimizing resource consumption is “Machine Vision”.

It would be difficult for a company to sustain its competitive advantage if it solely focuses on its financial performance and negate its social strategy from corporate strategy. Some of the most successful global organizations are able to incorporate its corporate strategy with its social to develop true sustainable competitive advantages in the market, leading it to sustainable long term financial performance and social performance.

Global warming, pollution in cities, water contamination, soil depletion, food shortages and many other problems arise due to our ignorance in the past to take care of the environment. Resource shortages would continue to be the norm for several more decades. In this situation, the manufacturing industry will be constrained by scarcity of resources and increased costs of raw materials such as crude oil, metals, commodities etc to pursue their manufacturing outputs.

Only those manufacturing companies that have the capability to operate in an environmentally sustainable way would survive. It means embarking on systems and processes which minimize the use of resources to produce more outputs. In layman terms, this is principled around zero waste and zero defects management. In the past, the Automation industry has concentrated on production line productivity and manufacturing costs reduction as key performance indicators. But, now it is increasingly applied to reduce scrap, eliminate defects and to minimize resource consumption. One forefront technologies delivering this value is “Machine Vision”. Machine Vision is a technology that combines human eye and judgment by camera and artificial intelligence technologies.

For example, one of the world’s top biscuit manufacturing companies installed vision sensor in its baking line. The idea is to detect the colour of the biscuit after baking. In the past, the production inspector and operators were controlling line speed and oven temperature based on colour of the biscuit coming out of the oven. The colour of biscuit is one of the indicators to tell if a biscuit is baked properly or not. There were several problems arising from this arrangement as different operators would view and interpret the colours differently and making judgement as per their own personal experience.

In the company’s pursuit to maintain stringent quality, slight deviation of biscuit colour was considered unacceptable. Since, there was no common benchmark, the scrap rate was high. Being one of the largest food companies in the world, the company has an environmental policy in place that also includes efficient use of resources. It means minimizing usage of raw material, water and energy.

The company executives scouted for various options and solutions and finally settled for “Vision Sensor with Real Colour capability” technology. This is how it works, the Real Colour Vision sensor converts colour data into numeric digits of R, G, B. (Example, a colour is registered as R=225, G=150, B=130) and each of it can be set to different threshold values. With the introduction of the Vision Sensor, there were two immediate benefits.

First, it was possible to set a clear reference for the colour of the biscuit after baking and thus eliminating ambiguity arising from person to person judgement. Second, it is now possible to arrive at the required baking condition speedily. Biscuit recipe now includes R, G and B data from vision sensor, in addition to temperature and conveyor speed, enabling to make close loop feedback control to achieve required baking condition without any mistake (fig.1, fig.2).

As a result, this company has managed to reduce its consumption of flour, sugar, dairy products, water and energy by a significant margin. This is not just bottom line improvement but also a contribution to the social responsibility philosophy of the business.

Fig.1: Inspecting biscuit colour by Real Colour Vision Sensing

Fig.2: Real Colour Sensing interactive menu

You can easily specify any colour by just clicking it on the screen. The colour chart on the screen, that shows the colour you have chosen, enables intuitive operation even for fine adjustments.

Despite its obvious benefits, the machine vision is so far adopted by limited industry and top players only. Some of the identified barriers for its widespread adoption are technology gap, cost and lack of user friendliness.

As far as the cost of machine vision is concerned, the average price range of a complete vision system has come down dramatically as technology advances. Similarly, user friendliness of machine vision has improved drastically after stand alone machine vision entered the market. Standalone vision sensor does not require complicated integration of camera, frame grabbers and software programming. Even though the price has reduced and user friendliness has increased, several applications are turned down by machine vision specialists each year. Some of these applications if solved can realize significant reduction in waste and contribution to environment. This is enough motivation for machine vision producing companies that are equally passionate about environment. While the consumer demands for various products to be inspected by machine vision have increased, the levels of challenges have increased too. This is because of the gap between current technology and the user requirements.

Machine vision has progressed from monochrome to colour vision, camera resolution has jumped from 300K pixels to several mega pixels, and processor speeds have increased several folds. High resolution cameras and high speed processors cannot make any difference if the processing software is not as powerful. For example in the case study of detecting biscuit colour, the Real Colour Technology is the key.

Real Colour

Since the inception of machine vision, monochrome processing is dominating the industry. In monochrome technology information of colour is reduced at the time of image pick up (fig.3). Colour information is reduced to detecting brightness only.

After several years of development, came the colour image processing. The two most common colour technologies are colour pick up and colour grey. In colour pick up technique, only the selected colour is extracted from the colour image and processed, while remaining colour from the image blackened. Colour pick allows up to eight colours to be picked and processed simultaneously. While in the colour grey technique, one of the selected colour is converted to grey scale of 256 levels of black-and-white brightness and the contrasts of specific colour is enhanced. The problem with colour pick up technique is that it is highly sensitive to lighting condition variation. While the problem with colour grey is that it is limited to single colour and processing efficiency is dependent on how colour filter is set (fig.4). As a result, both the colour image and colour grey are losing lots of information of the original image after the image is picked up by a colour camera Moreover, this technique cannot detect subtle changes in images with low contrast.

Fig.3: Differentiating Real Colour from traditional technologies

Fig.4: Traditional Colour Vision converts colour image into black and white

Real Colour technology (patent pending) introduced by Omron in 2006 managed to break through the above constraints. Real Colour technique is based on the concept of human eye where an image is processed with lots of information. There is no information loss either during image capturing or during processing.

Different colours are represented as different positions in the 3D RGB space (fig.5). Subtle variations in colour can be recognised by representing them as distances between different colour pixels comprising this space.

Fig.5: Real Colour Sensing

One of the advantages of Real Colour sensing processing is stable measurements in different inspection environments. One the other hand the drawback is that it requires a high performance processing chips. At the moment, Omron is using custom make Dual Mega ARCS Engine that can do multi-processing (fig.6).

Fig.6: ARCS (Advanced Real Color Sensing)

Examples where Real Colour can make a difference

One example where Real Colour technology can make a difference is inspecting foreign particle in connector molding (fig.7). Real colour sensing can inspect any kinds of defects, including oil stain. Traditional colour vision cannot detect defects that are not defined. Another example is inspecting harness of colour sequence (fig.8). Traditional colour vision processing using colour pick up can only see limited colour and each colour sequence need to be taught. Colour grey would see only one colour. But, Real colour can see inspect any colour sequence by comparing it with the register model.

Fig.7: Inspecting foreign particles in connector molding

Real Colour Sensing can inspect foreign particles where colour of defect cannot be defined, such as oil stain.

Traditional Vision Sensors can detect only pre-defined colour defects. It cannot detect colour that is not defined. Real Colour Sensing can inspect any colour sequence by comparing it with the OK item. In case of Traditional Vision Sensor, every kind of sequence has to be taught.

Fig.8: Inspecting Harness Colour Sequence (top), Inspecting Quality of Gold Plating on PCB (bottom)

Real Colour technology creates wonders for colour objects but it can do little for reflective shiny metallic objects. Traditionally Vision specialists have been using lighting techniques and filters to get clear images, but success is not always guaranteed. A new technology called HDR is emerging which utilizes software power to eliminate effects of reflection to get clear image.
 

High Dynamic Range or HDR

It is said that 90% of the task in machine vision is about capturing the clear image. One of the most difficult tasks in machine vision is the generation of clear image for individual inspection. Image processing becomes easy if clear images are captured regardless of lighting variation, reflection and poor contrast. One of the problems is also the limited dynamic range of cameras. Luckily, one of the leading players in the market, Omron has developed HDR technology (patent pending) to minimize the effects of lighting condition. HDR stands for High Dynamic Range.
 

HDR Image Generation Technology

Dynamic range means the imaging hardware’s ability to tell differences in luminosity. The higher dynamic range the hardware scores, the clearer images it can generate when imaging objects with a strong contrast in luminosity. A machine vision processor featuring the HDR image Generation technology takes two or more imaging of a work piece at different levels of luminosity by changing the shutter speed and synthesizes them into a single image rapidly (fig.9).
 

Fig.9: HDR Image Generation

 

HDR High-Contrast Technology

An image processor loaded with HDR technology can enhance the contrast in the area to be inspected by overlapping and synthesizing two or more images taken at the same shutter speed. After the synthesis, noise contents are suppressed while the area to be inspected is amplified by integration (fig.10).

Fig.10: HDR High Contrast


HDR can make significant difference while inspecting metallic object. One of them is punched or laser marked code on automotive components. Fig.11 shows two individual images, left hand side top image is facing problem of overexposure while bottom image underexposed. As a result, both these images cannot be utilized for processing. On the other hand, fig.11 right hand image showing the same object captured using HDR technology. The surface of the object as well 2D code is clearly visible with enough contrast required for reliable processing.

 

Comparing conventional images against HDR images

Fig.11: Conventional image vs. HDR image

Other examples are inspecting electrical components with shiny metallic pins and black molding body under the same lighting condition, inspecting shiny and cylindrically curved objects (fig.12).



Fig.12: Conventional image vs. HDR image


Conclusion:

Yesterday’s businesses were oblivious to their negative impact on the environment. Today’s businesses are striving for zero impact on the environment while ensuring profitability. After all, sustainable manufacturing means meeting the needs of present without compromising the ability of future generation to meet their needs. Factory Automation industry play a critical role by providing products, services and technologies that would help manufacturing industries to realize sustainable environmentally friendly operations. It is not just new business opportunities for the industry; it is also the social responsibility.
 

For more information about Omron's Machine Vision, please refer to:

Vision Sensors / Machine Vision Systems