3D vision technology – 3D matching
Please note: Once you watch the video, data will be transmitted to Youtube/Google. For more information, see Google Privacy.
With 3D matching, it is possible to recognize arbitrary 3D objects and to determine their 3D pose even with only one camera or depth sensor.
The 3D matching of MVTec HALCON is particularly used for 3D alignment, i.e., determining the 3D position and orientation of 3D objects from common or distance images, e.g., within automotive and robotics applications, pick-and-place applications (see an example in the video on the right), and bin picking.
Deep 3D Matching
Deep learning and rule-based methods achieve unprecedented detection rates
Deep 3D Matching is a new deep-learning-based technology for fast and robust 3D object detection and pose estimation using 2D images. It requires minimal parametrization and delivers high performance, making it ideal for applications such as bin picking and robotic handling – even in challenging conditions.
- Bin picking in metal processing and intralogistics
- Robot-assisted quality assurance in the automotive industry
- Object recognition with CAD models in multi-variant production
- Flexible robot guidance with camera support
As one of the first technologies of its kind, Deep 3D Matching combines advanced deep learning algorithms with rule-based methods to achieve unprecedented and extremely robust recognition rates. The technology is highly resilient to changes in background, object arrangement, and the colors or materials of the parts. Even foreign objects in the image do not negatively affect the results.
Flexible camera setup
Applications are best implemented with a flexible camera setup: a variable number of 2D cameras generate images from different perspectives. This reduces ambiguous and false-positive results to a minimum. The more cameras are installed, the more robust the identification rates. The ideal number of cameras depends on the specific application.
- Low calibration effort
- Reduced hardware costs thanks to the use of inexpensive 2D cameras
- Flexible positioning of the cameras, e.g. on a movable robot arm, to increase accuracy and extend the field of view.
Efficient training with synthetically generated image data
Deep 3D Matching also streamlines training:
only synthetically generated image data is required. These can be generated fully automatically, using the CAD data of the objects to be recognized.
- Huge cost and time savings
- No effort required for labelling the images
- Low level of parameterization required for robust recognition rates
If desired, model training can also be commissioned as a paid service through MVTec.
Generic box finder
For pick-and-place applications, HALCON provides the generic box finder. It allows users to locate boxes of different sizes within a predefined range of height, width, and depth, removing the need to train a model. This makes many applications much more efficient – especially within the logistics and pharmaceutical industries, where usually boxes in a large variety of different sizes are used.
A further possibility is the measuring of geometric features and locating defects on complex 3D objects after 3D alignment.
Shape- and surface-based 3D matching
To determine the 3D pose from common images, shape-based 3D matching is used. It extends the technology of shape-based matching to 3D by using multiple 2D views of the 3D object, represented by its CAD model.
To determine the 3D pose from distance images, surface-based 3D matching is used. It combines 3D point cloud data and edge information from distance images. This allows the robust pose determination even for smooth objects without prominent edges, which would not show significant gray value edges in common images.
To gain the highest accuracy, both methods provide a refinement of the pose in the full 3D space.
