Machine Vision automates vaccine production
Robotics | HALCON | 3D VisionA global company needed to quickly ramp up vaccine production. Goldfuß engineering GmbH and SIMON IBV GmbH developed a robot-based solution for loading and unloading trolleys with the vaccine vials. The machine vision software HALCON ensures the seamless automation of the robot cell.
To quickly produce a large amount of vaccine against a globally circulating virus, very large production capacities had to be built up within a very short time. A high degree of automation was essential for this. In one step of the automated production process, a robot places vaccine-filled vials from a conveyor belt into drawers and later removes them again. Special trolleys serve as a buffer and transfer system between filling, quality control, and packaging.
Goldfuß engineering GmbH was commissioned by a global producer of active pharmaceutical ingredients to develop a new robot cell for the automated loading and unloading of the trolleys. SIMON IBV GmbH was responsible for developing the optical 3D system that enables the robot to grip the vials independently without damaging them. Such an automation solution was previously not available on the market. In comparable applications, the vials are loaded manually into trays or boxes after filling, manually stacked on pallets or in Kanban trolleys, and subsequently depalletized again.
Robot loads and unloads trolleys with vaccine
The trolleys have nine drawers, each capable of holding 24 rows of 46 vials each. In total, one trolley can therefore hold almost 10,000 vials. By buffering such a large quantity, it is possible to flexibly control the capacity utilization of the packaging system. Since the trolleys are mobile, they can be parked in cooling chambers so that the cold chain is not interrupted. Employees manually position the trolleys on the system, which means that the exact position always varies.
The particular challenge in implementing this was that the robots always have to work precisely despite these deviations. In addition, the weight of the trolleys and therefore the position of the drawers is constantly changing during loading and unloading. To maintain the necessary positional accuracy when gripping the vials, the precise position for each row must be recalculated each time. To successfully address this challenge and ensure fully automated vaccine production, a high-performance 3D vision system is required. Another advantage: due to the automated process, samples can be easily taken from the trolleys and, after quality assurance approval, returned to the packaging process.
3D vision technologies automate handling in the robot cell
In addition to the robot, the hardware components of the robot cell include high-resolution 3D cameras using the stereometric method with pattern projection and industrial computer technology with fast processors for PC-based evaluation. MVTec HALCON machine vision software has been integrated for high-performance image processing. SIMAVIS® from SIMON IBV is used as the user interface for visualization and simple operation of the robot cell.
As part of the process, an employee first pushes a trolley into one of two possible positions. A 3D camera within the robot cell pinpoints the trolley and checks whether the drawers are open or closed. The programmable logic controller (PLC) contains information on whether the trolley currently being processed is to be loaded or unloaded, which drawer is to be opened, and how many vials are already there and where they are located. The 3D camera takes a picture of the drawer to be opened. HALCON uses this to create a coordinates system and transmits it to the robot so that it can open the drawer. In the next step, the 3D camera takes a picture of the drawer's contents. This allows the system to determine how many vials are stored and their exact location. It also checks whether any individual vials are standing upright or have tipped over and therefore cannot be picked up by the robot. The gripper can pick up 46 vials at a time. Once a drawer has been completely emptied, the 3D camera once again determines the position of the drawer handle so that the robot can close it again. This process is then repeated with the remaining eight drawers until the trolley has been completely unloaded.
High demands on 3D vision technologies
“The development of the robot cell took place under tremendous time pressure. The demand for the vaccine had to be met as quickly as possible,” explains Stephan Trunk from Goldfuß engineering. The position of the vials must be detected with an accuracy of 0.1 millimeters – in a working area of 800 x 600 mm and a depth of 600 mm, this is no easy task. To enable the vaccine to be provided quickly, loading and unloading must be carried out swiftly. At the same time, the extremely valuable contents of the vials had to be handled with care.
“The vision system also faced two special challenges. On the one hand, it was necessary to work with different materials. The glass of the vials and the metal of the trolleys have transparent or reflective surfaces and are therefore difficult to detect. On the other hand, the system will only work if the robots can operate autonomously in three-dimensional space. To do this, however, the machine vision software must also include powerful 3D vision technologies,” says Daniel Simon, authorized signatory and responsible for technical sales at SIMON IBV.
Machine vision software HALCON ensures maximum performance and robustness
To achieve the necessary performance and robustness, the project managers chose the MVTec HALCON machine vision software. “Based on our many years of experience with HALCON, we knew that the software has an extensive library with many extremely powerful methods,” adds Daniel Simon.
Various machine vision technologies are used in the robot cell – such as hand-eye calibration. This involves synchronizing the robot's and the camera's coordinates systems so that the robot's movements can be precisely matched to the camera's images. This makes it possible to determine the exact positions and orientations of the vials in relation to the robot. At the same time, HALCON's 3D vision technology “Stereo Vision” is also utilized. It is designed for 3D reconstruction and is particularly useful for large or medium-sized structured objects. In addition, the technology enables quality control and position detection of three-dimensional objects and calculates 3D coordinates on object surfaces.
Robot cell for vaccine production developed in just six months
The system was put into operation in July 2021. The strict requirements — in terms of speed and precision, for example — were also met. “We have managed to develop a completely new robot cell and get it up and running with process stability in just six months. We are proud to have contributed to accelerating the production of a vaccine against a dangerous virus,” summarizes Stephan Trunk from Goldfuß engineering. And Daniel Simon adds: “This project illustrates all of the potential and possibilities offered by machine vision. Thanks to the successful implementation, we are very motivated to automate other challenging tasks.”