MVTEC HALCON
Fully automated part picking by robots is a major technical challenge. A particular difficulty is that the components are not always delivered to the robot in an orderly fashion, but rather unsorted and even overlapping each other. This means that humans still have to intervene frequently.
The Spanish company Tekniker has developed an application based on machine vision that enables fully automated bin-picking with two robots. A particular highlight is the 3D vision technology "Surfaced-based Matching", which reliably recognizes the delivered components and enables precise gripping.
Tekniker, based in Eibar in northern Spain, specializes in the development of technology solutions for automation, industrial robotics, and sensor technology, among others.
One of Tekniker's customers is the CIKAUTXO GROUP, a Spanish automotive supplier. On behalf of the company, Tekniker has developed a machine-vision-controlled robot system for automated palletizing components. Robots remove components that are lying randomly in a box and then place them in an orderly manner into a container that is specially adapted to the geometric features of the parts. They are then ready for further processing in the production workflow.
"Up to now, the palletizing of parts was carried out purely manually. Our goal was to automate this process end-to-end in order to speed it up, increase productivity, and save costs. In addition, our workers were to be relieved," explains Kepa Laka, Automation and Robotics Area Manager at CIKAUTXO GROUP. To implement process automation, one option was to implement a mechatronic system. However, this approach offers a low level of flexibility and reliability and, on top of that, generates a lot of noise. Therefore, Tekniker decided to use a solution based on machine vision.
The system setup consists of two industrial robots, a 3D scanner, a depth camera, a PC running the machine vision software, and a PLC to control the cell. In the process, one of the robots first removes the stamped parts from the input bin using a magnetic multifunction gripper and deposits them on two conveyor belts. The removal is stopped as soon as a certain volume of parts is reached on both belts. This is calculated using data provided by the depth camera as well as 2D and 3D vision tools in HALCON.
HALCON also continuously processes the data from the 3D scanner and provides the robots with the position and orientation of the parts best suited for gripping on the conveyor belt.
With this data, the second robot already begins to pick up the parts from the conveyor belts and, also with the help of 3D vision, aligns them so that it can sort them into two target bins in an orderly manner. And, after the first robot stops picking from the input bin, it "helps" the second with its task. Incidentally, this works at a speed of 8 seconds per part. During this process, the parts must be inserted into guide bars within the target container with the front facing upwards and with a tolerance of 0.5 millimeters.
The particular challenge in this setup is that the robots need to precisely recognize the position and orientation of components in 3D space in order to pick them up and put them down again safely.
"To implement this requirement, we have developed and integrated the SMARTPICKING software. The software is able to accurately identify objects by analyzing 3D data. At the heart of the SMARTPICKING solution is the machine vision software MVTec HALCON. It provides sophisticated algorithms and tools to precisely locate the parts lying chaotically on the conveyor belt and transmit the coordinates of the pick point to the robots," explains Jorge Molina, researcher at smart and autonomous systems department at Tekniker.
In concrete terms, classic 3D-based image processing methods are used. First and foremost, this includes the HALCON-feature "Surface-based Matching". The technology uses data from 3D point clouds captured by the 3D scanner. The data is further processed using different filters and preprocessing operators to improve the pose estimation of the parts. This allows for a particularly robust positioning of objects – regardless of the nature of the surface. Now, the robots can grip the parts unerringly. This works for a wide range of different parts as they occur in industrial production processes. In addition, for a safe gripping process, the robots must also be able to determine the relative position of the 3D camera in relation to the respective object exactly in their coordinate system. For this purpose, a so-called hand-eye calibration is performed with the help of MVTec HALCON. This calibration is essential to establish the relationship between camera, robot, and the parts to be gripped. Only then does the robot "know" exactly where its gripper is in relation to the object to be gripped – i.e., where it must move it in order not to grasp at nothing.
Possible collisions between the robot arms and individual objects (components, containers, etc.) must be avoided during the entire pick-and-place workflow. This is where the "3D Object Processing" technology integrated in HALCON comes in. This makes it possible to determine those parts that lie favorably on the conveyor belt and can thus initially be gripped without collision. Based on this initial selection, the software then determines the optimal candidate using various decision criteria such as the height and orientation of the parts or the position of the gripping point and returns the coordinates of its position to the robot.
In this way, the robot can safely carry out the next gripping process. The controlled sequence allows the removal of components in such a way that collisions and thus disruptions to the workflow are reliably prevented. The software also detects the remaining capacity of the target container and determines whether further parts can be added to the container.
"Thanks to HALCON, we can automate and optimize the robotic palletizing process smoothly and end-to-end," Jorge Molina sums up. "We are able to increase process speed by up to 100 percent and, on top of that, relieve our employees of routine tasks," confirms Kepa Laka.
Published on: November 24, 2023
