interfaces mvtec

Interface Documentation

Image Acquisition Interface for OpenNI compliant 3D Sensors

Interface: OpenNI
Revision: 6.0
Date: 2014-10-31
HALCON Version: 12.0
OpenNI: Legacy Interface

General

This page provides the documentation of the HALCON image acquisition interface for OpenNI compliant 3D sensors like PrimeSense Carmine, ASUS Xtion, Fotonic E-series, Microsoft Kinect for Windows, or the Microsoft Kinect. Registered customers can download the latest revision of this interface from the MVTec WWW server.

General system requirements for using OpenNI

  • Intel compatible PC with Windows Vista/7, Windows Vista/7 x64, also WoW64 (using 32-bit HALCON on 64-bit Windows).
  • HALCON image acquisition interface hAcqOpenNI.dll or hAcqOpenNIxl.dll, respectively. If you have properly installed the interface, both DLLs should reside in bin\%HALCONARCH% within the HALCON base directory %HALCONROOT% you have chosen during the installation of HALCON.
  • Successful installation of OpenNI Build for Windows x86 (32-bit) v1.5.7.10 Development Edition (or higher) or OpenNI Build for Windows x64 (64-bit) v1.5.7.10 Development Edition (or higher). Please install the OpenNI runtime, before you start with the installation of the specific device driver.
  • To get further information about the installation process of your device, please see the device specific sections below.
  • After the installation process you can use the NiSimpleRead Viewer provided by the installation of OpenNI to verify that the setup was successful.

Using PrimeSense Carmine

The HALCON OpenNI Interface supports all Carmine models from Prime Sense.
  • Successful installation of the PrimeSense driver version 5.1.6.6 for (32-bit) (or higher) or version 5.1.6.6 for (64-bit) (or higher).
  • Check inside the Windows Device Manager, that the driver version is PrimeSense PS1080 3.1.3.1 (22.05.2012)

Using ASUS Xtion

The HALCON OpenNI Interface supports the ASUS Xtion series, the ASUS Xtion Pro and the ASUS Xtion Pro Live camera.
  • Successful installation of the PrimeSense driver version 5.1.6.6 for (32-bit) (or higher) or version 5.1.6.6 for (64-bit) (or higher).
  • Check inside the Windows Device Manager, that the driver version is PrimeSense PS1080 3.1.3.1 (22.05.2012)
  • In case of an Kinect10.dll error, please verify that the Microsoft Kinect service is disabled and all Kinect for Windows installations are removed from the system.

Using Fotonic E-series

The Fotonic E-series is - in contrast to the original Microsoft Kinect sensor - based on the Fotonic SDK and needs an extra bridge library to run with the HALCON OpenNI interface.
  • Gigabit Ethernet network adapter with native driver installed.
  • Successful installation of the SensorKinect driver SensorKinect-Win-OpenSource32-5.1.2.1 (32-bit)(or higher) or SensorKinect-Win-OpenSource64-5.1.2.1 (64-bit)(or higher).
  • Successful installation of FotonicNiPlugin.dll 32-bit or 64-bit. If you do not have this file, please contact Fotonic or the vendor from which you bought the sensor. To register this dll, please use the niReg.exe program provided by OpenNI. Please note, that you need sufficient permissions to register the dll.
  • To open the device in HALCON via open_framegrabber, the 'camera_type' parameter must be set to a valid configuration file (xml). It is not sufficient to use the value 'default'.
  • Using different color spaces than 'default' is not supported.

Using Kinect for Windows

Using Microsoft Kinect

  • Successful installation of the SensorKinect driver SensorKinect-Win-OpenSource32-5.1.2.1 (32-bit)(or higher) or SensorKinect-Win-OpenSource64-5.1.2.1 (64-bit)(or higher).
    Note that the audio device of the Microsoft Kinect sensor is not supported by the provided OpenNI driver and is also not necessary for using the HALCON OpenNI interface.
  • Check inside the Windows Device Manager, that the driver version for the entry Camera is 3.1.3.1 (22.05.2012).
  • Attention: For using the Microsoft Kinect 3D sensor it is not sufficient to install the OpenNI driver module. You have to replace the driver by the installation packages SensorKinect-Win-OpenSource[32|64]-5.1.2.1 as specified above. The link redirects to the github web site where you can find the needed installation packages.
  • Visual Studio C++ 2008 Redistributable Runtime Package, particularly msvcp90.dll. If this package is not installed please download and install it from the Microsoft Download Center for Windows x86 or Windows x64. Note: It is not sufficient to copy the missing files!

Features

  • Support of grab_image and grab_image_async to get the image (rgb) or the infrared image (gray).
  • Support of grab_data and grab_data_async to get the image (rgb or gray), the depth image (uint2), and the calibrated X, Y, and Z images (each float).
  • Support of multiple sensors.
  • Support of the Microsoft Kinect for Windows via kinect-mssdk-openni-bridge.

Limitations

  • No support of external or software trigger.
  • No controlling of the motor of the Kinect sensor. This feature is currently not supported by the driver.
  • Due to sensor restrictions of the Kinect sensor, the depth and color image are not synchronized.
  • If the image type inside the xml configuration file is set to 'IR' and full resolution, or the 'ColorSpace' is set to 'grayd' and full resolution, a destroyed depth image will be returned. This is a limitation of the currently used driver. Please use the image type 'Image' inside the xml file, or the 'ColorSpace' with 'rgbd' and full resolution instead. If a VGA resolution is sufficient, you can also use the 'IR' image type inside the xml file with VGA resolution, or a 'ColorSpace' with 'grayd' and VGA resolution to get a valid depth image.

Camera Calibration

To get the calibrated X, Y, and Z images the camera calibration data is per default provided by the SDK. If you have created your own calibration data, which could improve the results of the X,Y and Z images, you can set parameters with set_framegrabber_param(...,'cam_param_file','campar.dat'). It is also possible to set the camera calibration data via the corresponding tuple with set_framegrabber_param(...,'cam_param',...). For further information see also the documentation of the operator camera_calibration. If you want to calibrate the 3D sensor on your own, please note that you have to calibrate the rgb and infrared images separately.

For further processing of the 3D data, the operator xyz_to_object_model_3d can be used that transforms an image triple with the X, Y and Z coordinates of 3D points to a 3D object model.

Parameters for info_framegrabber

Parameter Value List Type Kind Description
'bits_per_channel' [] Unused.
'camera_type' ['default', '<path>'] string pre-defined Default value and path for camera configuration file.
'color_space' ['default', 'gray', 'grayd', 'rgb', 'rgbd'] string pre-defined Values for color space.
'defaults' [1, 1, 0, 0, 0, 0, 'progressive', 8, 'default', -1.0, 'false', 'default', '0', 0, 0] mixed pre-defined Default values for open_framegrabber.
'device' ['0', '1', '2',...] string dynamic Returns a pre-defined list of possible device numbers.
'external_trigger' [] Unused.
'field' [] Unused.
'general' [] string pre-defined Information about the HALCON OpenNI interface.
'generic' [] Unused.
'horizontal_resolution' [] Unused.
'image_height' [] Unsupported query.
'image_width' [] Unsupported query.
'info_boards' ['device:<device_id>'] string dynamic A list of the available devices.
'parameters' ['<parameters>'] string pre-defined Pre-defined parameters of the HALCON interface.
'parameters_readonly' ['<parameters>'] string pre-defined Pre-defined read-only parameters of the HALCON interface.
'parameters_writeonly' ['<parameters>'] string pre-defined Pre-defined write-only parameters of the HALCON interface.
'port' [] Unused.
'revision' '<revision>' string pre-defined Revision number of the OpenNI interface.
'start_column' [] Unsupported query.
'start_row' [] Unsupported query.
'vertical_resolution' [] Unused.

Parameters for open_framegrabber

Parameter Values Default Type Description
Name 'OpenNI' string Name of the HALCON interface.
HorizontalResolution --- 1 Ignored.
VerticalResolution --- 1 Ignored.
ImageWidth 0, 640, 1280 0 integer Width of the desired image part ('0' stands for the complete image). The depth image and the corresponding XYZ images, which could be acquired via grab_data or grab_data_async, are always in VGA resolution.
ImageHeight 0, <height> 0 integer Height of the desired image part ('0' stands for the complete image). The depth image and the corresponding XYZ images, which could be acquired via grab_data or grab_data_async, are always in VGA resolution.
StartRow --- Ignored.
StartColumn --- Ignored.
Field --- Ignored.
BitsPerChannel --- Ignored.
ColorSpace 'default', 'gray', 'grayd', 'rgb', 'rgbd' 'default' string If the 'ColorSpace' parameter is set to 'gray' the infrared image is acquired by the device. If 'rgb' is selected, the device will provide a three channel rgb image. If 'grayd' or 'rgbd' is set, the call of grab_data or grab_data_async will provide an image and also the depth information. With 'default' the color space 'rgbd' will be set. If this color space is not supported, 'grayd' will be used instead.
Generic --- Ignored.
ExternalTrigger --- Ignored.
CameraType 'default', '<xml_file>' 'default' string With 'default' the device initialization is done without the use of the xml file. If a xml file is given, the 'ColorSpace' parameter will be ignored.
Device 'default', '<id>' '0' string Opens a specific device. All connected devices could be queried with info_framegrabber(..,'info_boards',...). Each device must be connected to its own USB bus. To connect a camera only to another USB port is not sufficient. If multiple sensors are connected, the device that corresponds to '0' is connected to the first USB bus.
If a xml file is set at the 'CameraType' parameter, the device parameter will be ignored and the connection to multiple devices is not supported.
Port --- Ignored.
LineIn --- Ignored.

Parameters for set_framegrabber_param

Parameter Values Type Description
'alternative_viewpoint' 'disable', 'enable' string Point of view from the rgb image. If the parameter is set to 'enable' the point of view from the depth image and rgb image is equal.
'cam_param' [<Focus(float), Kappa(float), Sx(float), Sy(float), Cx(float), Cy(float), ImageWidth(int), ImageHeight(int)>] mixed Sets the camera calibration parameters directly. If successful, grab_data and grab_data_async will acquire the calibrated XYZ images based on this information. If no calibration data is set or an empty tuple is used as parameter, the calibration for the XYZ images is done by the SDK.
The query of the parameter will only return default camera parameters, if an infrared image (gray image) is acquired and no camera calibration parameters were set before.
'cam_param_file' '<cam_par_file>' string Loads camera calibration parameters via a camera calibration file. If successful, grab_data and grab_data_async will deliver the calibrated XYZ images based on the calibration data, which is included in this file. If no calibration file is set or an empty tuple is used as parameter, the XYZ calibration is done by the SDK.
'mirror_image' 'false', 'true' string Flip the image on the 3D sensor from right to left.
'xyz_images' 'disable', 'enable' string Controls the grab_data/grab_data_async image tuple: if set to 'enable', grab_data and grab_data_async will deliver also the calibrated XYZ images. Note that this parameter will be set automatically to 'enable' if the camera calibration data was set by the parameters 'cam_param_file' or 'cam_param'.

Parameters for get_framegrabber_param

There may exist additional read-only parameters with the following postfixes:
  • '_description': These parameters provide the tool-tip of the corresponding parameter as a string.
  • '_range': These parameters provide the minimum, maximum, step width, and default values for the corresponding integer or float parameter as a tuple with 4 elements, e.g., get_framegrabber_param(..,'Shutter_range',..) will return the output tuple [min,max,step,default]. Optionally, this tuple can also contain additional valid string values like 'auto' or 'manual'.
  • '_values': These parameters provide the valid value list for the corresponding parameter as a tuple, e.g., get_framegrabber_param(..,'volatile_values',..) will return the output tuple ['enable','disable'].

All these postfixed parameter names are not returned when calling info_framegrabber(..,'parameters',..) and are used to enable the easy parameterization via a generic graphical user interface, particularly the HDevelop Image Acquisition Assistant.

Parameter Values Default Type Kind Description
'alternative_viewpoint' 'disable', 'enable' 'disable' string pre-defined Point of view from the rgb image. If the parameter is set to 'enable' the point of view from the depth image and rgb image is equal.
'bits_per_channel' '<default>' 8 integer pre-defined The value is not used, so a default value is returned.
'cam_param' [<Focus(float), Kappa(float), Sx(float), Sy(float), Cx(float), Cy(float), ImageWidth(int), ImageHeight(int)>] mixed dynamic Parameters for camera calibration.
'cam_param_file' '<cam_par_file>' string dynamic Name of the used camera calibration file.
'camera_type' '<xml_file>' 'default' string pre-defined Current camera type.
'color_space' 'gray', 'grayd', 'rgb', 'rgbd' 'default' string pre-defined Desired color space and thus the number of image channels of the resulting HALCON image.
'device' '<device_id>' '0' string pre-defined Current device id.
'external_trigger' <default> 'false' string pre-defined The value is not used, so a default value is returned.
'field' '<default>' 'progressive' string pre-defined The value is not used, so a default value is returned.
'generic' <default> -1 integer pre-defined The value is not used, so a default value is returned.
'horizontal_resolution' <resolution> 1 integer pre-defined Current value of horizontal resolution.
'image_height' <height> 0 integer pre-defined Height of the desired image part ('0' stands for the complete image).
'image_width' <width> 0 integer pre-defined Width of the desired image part ('0' stands for the complete image).
'line_in' <default> 0 integer pre-defined The value is not used, so a default value is returned.
'mirror_image' 'false', 'true' 'false' string pre-defined Flip the image on the 3D sensor from right to left.
'name' 'OpenNI' string pre-defined Name of the HALCON interface.
'port' <default> 0 integer pre-defined The value is not used, so a default value is returned.
'revision' '<revision>' string pre-defined Revision number of the OpenNI interface.
'start_column' <column> 0 integer pre-defined Returns the current start column of the HALCON image.
'start_row' <row> 0 integer pre-defined Returns the current start row of the HALCON image.
'vertical_resolution' <resolution> 1 integer pre-defined Current value of vertical resolution.
'xyz_images' 'disable', 'enable' 'enable' string pre-defined Status of xyz images in grab_data/grab_data_async.

Operator set_framegrabber_lut

Not supported by this interface.

Operator get_framegrabber_lut

Not supported by this interface.

Operator set_framegrabber_callback

Not supported by this interface.

Operator get_framegrabber_callback

Not supported by this interface.

Operator grab_image_start

Starts a new asynchronous grab. See also grab_image_start.

Operator grab_image

grab_image starts a new synchronous grab. See also grab_image. Note that the interface converts the image from the device to the desired image format specified by the parameters 'image_width', 'image_height', 'start_row', 'start_column', 'bits_per_channel', and 'color_space'.

Operator grab_image_async

grab_image_async returns an image and starts the next asynchronous grab. See also grab_image_async. Note that the interface converts the image from the device to the desired image format specified by the parameters 'image_width', 'image_height', 'start_row', 'start_column', 'bits_per_channel', and 'color_space'.

Operator grab_data

Parameter Type Count Description
'image' 1...5 Depending on the parameters 'color_space' and 'xyz_images' the following data is returned:
  • image as byte
  • depth image as uint2
  • calibrated x image as float
  • calibrated y image as float
  • calibrated z image as float
'contours' Not used.
'region' Not used.
'data' Not used.

Operator grab_data_async

Same return values as grab_data, but here an asynchronous grab is started.

Operator close_framegrabber

This operator closes the device. See also close_framegrabber.

HDevelop Examples

For this interface there are the following examples available:
  • openni_2cameras.hdev - Shows the usage of two cameras.
  • openni_create_camera_calibration.hdev - Example for the calibration of the sensor.
  • openni_parameters.hdev - Lists all parameters of a device.
  • openni_simple.hdev - A simple example to show the usage of the interface.
  • openni_surface_based_3d_matching.hdev - Shows the possibilities of the sensor with surfaced based 3D matching.

Release Notes

  • Revision 6.0 (Oct 31, 2014):
    • HALCON 12 version of the interface.
  • Revision 5.2 Addendum (Apr 30, 2014):
    • Adapted the documentation. Now the required installation package OpenNI Build v1.5.7.10 and PrimeSense driver v5.1.6.6 could be downloaded from the MVTec download server. The packages OpenNI Build 1.5.4.0 (or higher) and the PrimeSense driver 5.1.2.1 are still binary compatible with this interface.
    • With the new installation packages the Kinect for Windows SDK and the kinect-mssdk-openni-bridge is no longer required by the Kinect for Windows device.
  • Revision 5.2 (Sep 6, 2013):
    • Support of the Fotonic E-series.
    • Support of the PrimeSense Carmine series.
    • Improved open_framegrabber. If an unsupported frame rate is specified via the xml file, a corrected setting will be used instead.
    • Fixed bug in open_framegrabber if an unsupported device name was used.
    • Created and adapted System Requirements section for each supported device series.
    • Added support of Kinect for Windows SDK 1.6 and support of kinect-mssdk-openni-bridge-v1.6.0.0-for-1.5.2.23.
  • Revision 5.1 (Aug 28, 2012):
    • Support of the new Microsoft Kinect for Windows via kinect-mssdk-openni-bridge.
  • Revision 5.0 (May 15, 2012):
    • HALCON 11 version of the interface (included in HALCON 11 DVD).
  • Revision 4.2 (Dec 8, 2011):
    • Added support for Windows x64.
  • Revision 4.1 (Sep 19, 2011):
    • Adapted to OpenNI Build v1.1.0.41 or higher.
    • Added support of multiple connected sensors.
    • Added functionality in info_framegrabber(..,'info_boards',...) and info_framegrabber(..,'device',...) to enumerate the currently connected devices.
    • Added support of internal provided calibration data.
    • Adapted query of the 'cam_param' parameter. Together with an infrared image, now the parameter returns default camera parameters.
    • Implemented 'ImageHeight' parameter in open_framegrabber to support software cropping.
    • Added new parameter 'alternative_viewpoint'.
    • Improved internal buffer handling to achieve a better performance.
    • Renamed the calibration file extension inside the HDevelop examples from .cal to .dat.
    • Adapted the device initialization in open_framegrabber. Now the use of an xml file is optional.
    • Improved implementation of info_framegrabber(..,'camera_type',...) to return the OpenNI installation environment path.
    • Added new HALCON HDevelop example program to show the usage of multiple sensors.
    • Added new HALCON HDevelop example program to create a camera-specific calibration file.
    • Adapted existing HDevelop example programs to open the device without the use of the xml file.
  • Revision 4.0 (May 31, 2011):
    • First official release.
  • Revision 4.0 beta (Mar 1, 2011):
    • First official beta release (as HALCON Kinect interface).