MVTEC HALCON
Pick-and-place robots have been used for a number of years in such diverse applications as PCB manufacturing and food packaging. In most of these systems, the size, shape, and surface characteristics of the objects to be handled are well-known. Because of this, system developers can choose from a range of robotic grippers specifically designed to handle individual products.
In many cases, however, the irregular nature of objects such as light bulbs, bottles, glassware, linen, and books necessitates the development of specialized handling equipment. To alleviate the need for such development costs, the Helmut-Schmidt-University (Hamburg, Germany) has developed a specialized gripper. By combining this gripper with off-the-shelf machine vision cameras and software, the team has built an automated packaging system for pick and place.
Today, many automation setups use 3-D laser sensors to perform object measurement. In these systems, the geometric dimensions of objects are compared with a 3-D CAD model to allow pick-and-place systems to accurately predict their location and position.However, where flexible packages such as bags or sacks need to be handled, more specialized picking systems that incorporate sophisticated handling mechanisms coupled with robotic vision systems need to be deployed.
In the design of the flexible gripper, a double-walled hose is filled with a fluid material such as water or air. This hose is glued into a cylindrical ferrule which forms the outer part of the gripper and allows the gripper to pick objects (see Fig. 1).
Before the flexible gripper can be used to pick an object, its location and position must be measured. To detect objects, the system developed at the Helmut-Schmidt-University uses non-contact stereo measurement that is first used to create an elevation profile of the object and then segment single objects.
By mounting the cameras from MaxxVision (Stuttgart, Germany) 200 mm apart, images from the cameras overlap, allowing an object's elevation distance to be calculated by a binocular stereo operator included in the HALCON image processing library from MVTec (see Fig. 2). To calibrate the cameras and the four stereo pairs, twenty-one different positions of a calibration plate were imaged and evaluated by HALCON's camera calibration algorithms.
To calculate the elevation distance, a gray value correlation analysis is performed for each of the four stereo pairs. Because the pixel distance, the camera distance, and the distance between cameras are known, a matrix elevation profile of gray values can be calculated geometrically.
Objects with low-contrast surfaces constitute a particular challenge caused by the absence of distinctive points. To overcome this, a Flexpoint laser pattern generator module from Laser Components (Olching, Germany) was used to project a laser dot matrix onto the surface with a dot distance of approximately 5 mm.
To avoid reflections and to detect transparent packing materials, an ultrasonic nebulizer from Hirtz & Co. (Cologne, Germany) sprays the package with a nondestructive thin water fumes.
To perform image segmentation, objects with known surfaces are compared using correspondence analysis with template models. To segment unknown objects, edge-based and region-based segmentations are used. Edge-based methods detect edges by using gray scale correlation inside the elevation profile.
While such methods are fast, they often result in generating incomplete edges. More computationally intensive region-based methods subdivide the elevation profile into homogeneous areas of the same height or same surface gradient.
Using these segmentation techniques, known and unknown objects can be identified. If, for example, such segmentation results in a known pattern, then the object can be properly identified. To identify unknown objects, their geometric parameters and color information is used. These characteristic data are fed to the input layer of a neural network, also implemented using HALCON, which relates the object to an object class. With every object, the neural network is trained again resulting in an increasing classification quality.
After identification, the known height and position of each object are transferred to the robotic palletizer for picking. By incorporating the flexible grabber on thigh x-y-z stage, the vision-based robotic system can be used to pick and place a number of disparate objects.
Authors: Rainer Bruns, Björn Cleves, and Dr. Lutz Kreutzer, MVTec Software GmbH
Article kindly provided by Vision Systems Design. All product names, trademarks and images of the products/trademarks are copyright by their holders. All rights reserved.
Published on: August 26, 2013
