Image Acquisition Interface for OpenNI compliant 3D Sensors
| Interface: | OpenNI |
| Revision: | 4.3 |
| Date: | 2012-08-28 |
| HALCON Version: | 10.0 |
- General
- System Requirements
- Using Kinect for Windows
- Features
- Limitations
- Camera Calibration
- Parameters for info_framegrabber
- Parameters for open_framegrabber
- Parameters for set_framegrabber_param
- Parameters for get_framegrabber_param
- Operator set_framegrabber_lut
- Operator get_framegrabber_lut
- Operator set_framegrabber_callback
- Operator get_framegrabber_callback
- Operator grab_image_start
- Operator grab_image
- Operator grab_image_async
- Operator grab_data
- Operator grab_data_async
- Operator close_framegrabber
- HDevelop Examples
- Release Notes
General
This page provides the documentation of the HALCON image acquisition interface for OpenNI compliant 3D sensors like
Microsoft Kinect, Microsoft
Kinect for Windows, or ASUS Xtion PRO which are
based on the PrimeSensor technology.
Registered customers can download the
latest revision of this interface from the
MVTec WWW server.
System Requirements
- Intel compatible PC with Windows XP/Vista/7, Windows XP/Vista/7 x64, also WoW64 (using 32-bit HALCON on 64-bit Windows).
- To use the Microsoft Kinect for Windows, please see the chapter "Using Kinect for Windows" to get detailed information about installing the necessary runtime and driver setup.
- Successful installation of OpenNI Build for Windows x86 (32-bit) v1.1.0.41 Development Edition (or higher) or OpenNI Build for Windows x64 (64-bit) v1.4.0.2 Development Edition (or higher). Please install OpenNI, before you start with the installation of the SensorKinect driver.
- Successful installation of the
SensorKinect driver
SensorKinect-Win-OpenSource32-5.0.1 (or
higher) or
SensorKinect-Win-OpenSource64-5.0.5.1 (or
higher) including driver v3.1.2.0. Note that the audio device of the Kinect
sensor is not supported by this driver and is also not necessary for
using the HALCON OpenNI interface. If you have trouble
connecting to the device, please check if the installed driver version is
actually the correct version.
Attention: For using the Microsoft Kinect 3D sensor it is not sufficient to install the driver module from the OpenNI web site! Please download the driver module from the github web site. Please check the NiViewer to verify the software driver is correctly installed. - Visual Studio C++ 2008 Redistributable Runtime Package, particularly msvcp90.dll. If this package is not installed please download and install it from the Microsoft Download Center for Windows x86 or Windows x64. Note: It is not sufficient to copy the missing files!
- HALCON image acquisition interface hAcqOpenNI.dll or hAcqOpenNIxl.dll, respectively. If you have properly installed the interface, both DLLs should reside in bin\%HALCONARCH% within the HALCON base directory %HALCONROOT% you have chosen during the installation of HALCON.
Using Kinect for Windows
The Microsoft Kinect for Windows is - in contrast to the original Microsoft Kinect sensor - based on the Microsoft Kinect for Windows SDK 1.5
and needs a different configuration to run with the HALCON OpenNI
interface. Currently, the use of this camera is only supported for 32-bit
HALCON (including WoW64, i.e,, using 32-bit HALCON on 64-bit Windows).
Please see the following system requirements to configure the device:
- Successful installation of OpenNI Build for Windows x86 (32-bit) v1.5.2.23 Development Edition (or higher).
- Successful installation of Kinect for Windows SDK 1.5
- Successful installation of kinect-mssdk-openni-bridge v1.5.0.2-for-1.5.2.23. The kinect-mssdk-bridge library must be registered once.
- To verify the installation, you can use the NiSimpleRead Viewer provided during the installation of OpenNI.
- To open the device in HALCON via of open_framegrabber, the 'camera_type' parameter must be set to a valid configuration file (xml). It is not sufficient to use the value 'default'.
- To use the near mode, this configuration must be set inside the configuration file.
- Please see the installation instruction README.txt in the kinect-mssdk-bridge folder to get additional information.
Features
- Support of grab_image and grab_image_async to get the image (rgb) or the infrared image (gray).
- Support of grab_data and grab_data_async to get the image (rgb or gray), the depth image (uint2), and the calibrated X, Y, and Z images (each float).
- Support of multiple sensors.
- Support of the Microsoft Kinect for Windows via kinect-mssdk-openni-bridge.
Limitations
- No support of external or software trigger.
- No controlling of the motor of the Kinect sensor. This feature is currently not supported by the driver.
- Due to sensor restrictions of the Kinect sensor, the depth and color image are not synchronized.
- If the image type inside the xml configuration file is set to 'IR' and full resolution or the 'ColorSpace' is set to 'grayd' and full resolution, a destroyed depth image will be returned. This is a limitation of the currently used driver. Please use the image type 'Image' inside the xml file, or the 'ColorSpace' with 'rgbd' and full resolution instead. If a VGA resolution is sufficient, you can also use the 'IR' image type inside the xml file with VGA resolution, or a 'ColorSpace' with 'grayd' and VGA resolution to get a valid depth image.
Camera Calibration
To get the calibrated X, Y, and Z images the camera calibration data is per default provided by the SDK. If you have created your own calibration data
which could improve the results of the X,Y and Z images, you can set
parameters with
set_framegrabber_param(...,'cam_param_file','campar.dat').
It is also possible to set the camera calibration data via the corresponding
tuple with set_framegrabber_param(...,'cam_param',...), see also
operator camera_calibration.
If you want to calibrate the 3D sensor on your own, please note that you
have to calibrate the rgb and infrared images separately.
For further processing of the 3D data, the operator xyz_to_object_model_3d can be used that transforms an image triple with the X, Y and Z coordinates of 3D points to a 3D object model.
For further processing of the 3D data, the operator xyz_to_object_model_3d can be used that transforms an image triple with the X, Y and Z coordinates of 3D points to a 3D object model.
Parameters for info_framegrabber
| Parameter | Value List | Type | Kind | Description |
|---|---|---|---|---|
| 'bits_per_channel' | [] | Unused. | ||
| 'camera_type' | ['default', '<path>'] | string | pre-defined | Default value and path for camera configuration file. |
| 'color_space' | ['default', 'gray', 'grayd', 'rgb', 'rgbd'] | string | pre-defined | Values for color space. |
| 'defaults' | [1, 1, 0, 0, 0, 0, 'progressive', 8, 'default', -1.0, 'false', 'default', '0', 0, 0] | mixed | pre-defined | Default values for open_framegrabber. |
| 'device' | ['0', '1', '2',...] | string | dynamic | Returns a pre-defined list of possible device numbers. |
| 'external_trigger' | [] | Unused. | ||
| 'field' | [] | Unused. | ||
| 'general' | [] | string | pre-defined | Information about the HALCON OpenNI interface. |
| 'generic' | [] | Unused. | ||
| 'horizontal_resolution' | [] | Unused. | ||
| 'image_height' | [] | Unsupported query. | ||
| 'image_width' | [] | Unsupported query. | ||
| 'info_boards' | ['device:<device_id>'] | string | dynamic | A list of the available devices. |
| 'parameters' | ['<parameters>'] | string | pre-defined | Pre-defined parameters of the HALCON interface. |
| 'parameters_readonly' | ['<parameters>'] | string | pre-defined | Pre-defined read-only parameters of the HALCON interface. |
| 'parameters_writeonly' | ['<parameters>'] | string | pre-defined | Pre-defined write-only parameters of the HALCON interface. |
| 'port' | [] | Unused. | ||
| 'revision' | '<revision>' | string | pre-defined | Revision number of the OpenNI interface. |
| 'start_column' | [] | Unsupported query. | ||
| 'start_row' | [] | Unsupported query. | ||
| 'vertical_resolution' | [] | Unused. |
Parameters for open_framegrabber
| Parameter | Values | Default | Type | Description |
|---|---|---|---|---|
| Name | 'OpenNI' | string | Name of the HALCON interface. | |
| HorizontalResolution | --- | 1 | Ignored. | |
| VerticalResolution | --- | 1 | Ignored. | |
| ImageWidth | 0, 640, 1280 | 0 | integer | Width of the desired image part ('0' stands for the complete image). The depth image and the corresponding XYZ images, which could be acquired via grab_data or grab_data_async, are always in VGA resolution. |
| ImageHeight | 0, <height> | 0 | integer | Height of the desired image part ('0' stands for the complete image). The depth image and the corresponding XYZ images, which could be acquired via grab_data or grab_data_async, are always in VGA resolution. |
| StartRow | --- | Ignored. | ||
| StartColumn | --- | Ignored. | ||
| Field | --- | Ignored. | ||
| BitsPerChannel | --- | Ignored. | ||
| ColorSpace | 'default', 'gray', 'grayd', 'rgb', 'rgbd' | 'default' | string | If the 'ColorSpace' parameter is set to 'gray' the infrared image is acquired by the device. If 'rgb' is selected, the device will provide a three channel rgb image. If 'grayd' or 'rgbd' is set, the call of grab_data or grab_data_async will provide an image and also the depth information. With 'default' the color space 'rgbd' will be set. If this color space is not supported, 'grayd' will be used instead. |
| Generic | --- | Ignored. | ||
| ExternalTrigger | --- | Ignored. | ||
| CameraType | 'default', '<xml_file>' | 'default' | string | With 'default' the device initialization is done without the use of the xml file. If a xml file is given, the 'ColorSpace' parameter will be ignored. |
| Device | 'default', '<id>' | '0' | string | Opens a specific device. All connected devices could be queried with info_framegrabber(..,'info_boards',...). Each device must be
connected to its own USB bus. To connect a camera only to another USB
port is not sufficient. If multiple sensors are connected, the device
that corresponds to '0' is connected to the first USB bus. If a xml file is set at the 'CameraType' parameter, the device parameter will be ignored and the connection to multiple devices is not supported. |
| Port | --- | Ignored. | ||
| LineIn | --- | Ignored. |
Parameters for set_framegrabber_param
| Parameter | Values | Type | Description |
|---|---|---|---|
| 'alternative_viewpoint' | 'disable', 'enable' | string | Point of view from the rgb image. If the parameter is set to 'enable' the point of view from the depth image and rgb image is equal. |
| 'cam_param' | [<Focus(float), Kappa(float), Sx(float), Sy(float), Cx(float), Cy(float), ImageWidth(int), ImageHeight(int)>] | mixed | Sets the camera calibration parameters directly. If successful, grab_data and grab_data_async will acquire the calibrated XYZ images based on this
information. If no calibration data is set or an empty tuple is used as
parameter, the calibration for the XYZ images is done by the SDK. The query of the parameter will only return default camera parameters, if an infrared image (gray image) is acquired and no camera calibration parameters were set before. |
| 'cam_param_file' | '<cam_par_file>' | string | Loads camera calibration parameters via a camera calibration file. If successful, grab_data and grab_data_async will deliver the calibrated XYZ images based on the calibration data, which is included in this file. If no calibration file is set or an empty tuple is used as parameter, the XYZ calibration is done by the SDK. |
| 'mirror_image' | 'false', 'true' | string | Flip the image on the 3D sensor from right to left. |
| 'xyz_images' | 'disable', 'enable' | string | Controls the grab_data/grab_data_async image tuple: if set to 'enable', grab_data and grab_data_async will deliver also the calibrated XYZ images. Note that this parameter will be set automatically to 'enable' if the camera calibration data was set by the parameters 'cam_param_file' or 'cam_param'. |
Parameters for get_framegrabber_param
There may exist additional read-only parameters with the following postfixes:
- '_description': These parameters provide the tool-tip of the corresponding parameter as a string.
- '_range': These parameters provide the minimum, maximum, step width, and default values for the corresponding integer or float parameter as a tuple with 4 elements, e.g., get_framegrabber_param(..,'Shutter_range',..) will return the output tuple [min,max,step,default]. Optionally, this tuple can also contain additional valid string values like 'auto' or 'manual'.
- '_values': These parameters provide the valid value list for the corresponding parameter as a tuple, e.g., get_framegrabber_param(..,'volatile_values',..) will return the output tuple ['enable','disable'].
All these postfixed parameter names are not returned when calling info_framegrabber(..,'parameters',..) and are used to enable the easy parameterization via a generic graphical user interface, particularly the HDevelop Image Acquisition Assistant.
| Parameter | Values | Default | Type | Kind | Description |
|---|---|---|---|---|---|
| 'alternative_viewpoint' | 'disable', 'enable' | 'disable' | string | pre-defined | Point of view from the rgb image. If the parameter is set to 'enable' the point of view from the depth image and rgb image is equal. |
| 'bits_per_channel' | '<default>' | 8 | integer | pre-defined | The value is not used, so a default value is returned. |
| 'cam_param' | [<Focus(float), Kappa(float), Sx(float), Sy(float), Cx(float), Cy(float), ImageWidth(int), ImageHeight(int)>] | mixed | dynamic | Parameters for camera calibration. | |
| 'cam_param_file' | '<cam_par_file>' | string | dynamic | Name of the used camera calibration file. | |
| 'camera_type' | '<xml_file>' | 'default' | string | pre-defined | Current camera type. |
| 'color_space' | 'gray', 'grayd', 'rgb', 'rgbd' | 'default' | string | pre-defined | Desired color space and thus the number of image channels of the resulting HALCON image. |
| 'device' | '<device_id>' | '0' | string | pre-defined | Current device id. |
| 'external_trigger' | <default> | 'false' | string | pre-defined | The value is not used, so a default value is returned. |
| 'field' | '<default>' | 'progressive' | string | pre-defined | The value is not used, so a default value is returned. |
| 'generic' | <default> | -1 | integer | pre-defined | The value is not used, so a default value is returned. |
| 'horizontal_resolution' | <resolution> | 1 | integer | pre-defined | Current value of horizontal resolution. |
| 'image_height' | <height> | 0 | integer | pre-defined | Height of the desired image part ('0' stands for the complete image). |
| 'image_width' | <width> | 0 | integer | pre-defined | Width of the desired image part ('0' stands for the complete image). |
| 'line_in' | <default> | 0 | integer | pre-defined | The value is not used, so a default value is returned. |
| 'mirror_image' | 'false', 'true' | 'false' | string | pre-defined | Flip the image on the 3D sensor from right to left. |
| 'name' | 'OpenNI' | string | pre-defined | Name of the HALCON interface. | |
| 'port' | <default> | 0 | integer | pre-defined | The value is not used, so a default value is returned. |
| 'revision' | '<revision>' | string | pre-defined | Revision number of the OpenNI interface. | |
| 'start_column' | <column> | 0 | integer | pre-defined | Returns the current start column of the HALCON image. |
| 'start_row' | <row> | 0 | integer | pre-defined | Returns the current start row of the HALCON image. |
| 'vertical_resolution' | <resolution> | 1 | integer | pre-defined | Current value of vertical resolution. |
| 'xyz_images' | 'disable', 'enable' | 'enable' | string | pre-defined | Status of xyz images in grab_data/grab_data_async. |
Operator set_framegrabber_lut
Not supported by this interface.
Operator get_framegrabber_lut
Not supported by this interface.
Operator set_framegrabber_callback
Not supported by this interface.
Operator get_framegrabber_callback
Not supported by this interface.
Operator grab_image_start
Starts a new asynchronous grab. See also grab_image_start.
Operator grab_image
grab_image starts a new synchronous grab. See also grab_image. Note that
the interface converts the image from the device to the desired image format
specified by the parameters 'image_width', 'image_height', 'start_row',
'start_column', 'bits_per_channel', and 'color_space'.
Operator grab_image_async
grab_image_async returns an image and starts the next asynchronous grab. See also grab_image_async.
Note that
the interface converts the image from the device to the desired image format
specified by the parameters 'image_width', 'image_height', 'start_row',
'start_column', 'bits_per_channel', and 'color_space'.
Operator grab_data
| Parameter | Type | Count | Description |
|---|---|---|---|
| 'image' | 1...5 | Depending on the parameters 'color_space' and 'xyz_images' the following data is returned:
|
|
| 'contours' | Not used. | ||
| 'region' | Not used. | ||
| 'data' | Not used. |
Operator grab_data_async
Same return values as grab_data, but here an asynchronous grab is started.
Operator close_framegrabber
This operator closes the device. See also close_framegrabber.
HDevelop Examples
For this interface there are the following examples available:
- openni_2cameras.hdev - Shows the usage of two cameras.
- openni_create_camera_calibration.hdev - Example for the calibration of the sensor.
- openni_parameters.hdev - Lists all parameters of a device.
- openni_simple.hdev - A simple example to show the usage of the interface.
- openni_surface_based_3d_matching.hdev - Shows the possibilities of the sensor with surfaced based 3D matching.
Release Notes
- Revision 4.3 (Aug 28, 2012):
- Support of the new Microsoft Kinect for Windows via kinect-mssdk-openni-bridge.
- Revision 4.2 (Dec 8, 2011):
- Added support for Windows x64.
- Revision 4.1 (Sep 19, 2011):
- Adapted to OpenNI Build v1.1.0.41 or higher.
- Added support of multiple connected sensors.
- Added functionality in info_framegrabber(..,'info_boards',...) and info_framegrabber(..,'device',...) to enumerate the currently connected devices.
- Added support of internal provided calibration data.
- Adapted query of the 'cam_param' parameter. Together with an infrared image, now the parameter returns default camera parameters.
- Implemented 'ImageHeight' parameter in open_framegrabber to support software cropping.
- Added new parameter 'alternative_viewpoint'.
- Improved internal buffer handling to achieve a better performance.
- Renamed the calibration file extension inside the HDevelop examples from .cal to .dat.
- Adapted the device initialization in open_framegrabber. Now the use of an xml file is optional.
- Improved implementation of info_framegrabber(..,'camera_type',...) to return the OpenNI installation environment path.
- Added new HALCON HDevelop example program to show the usage of multiple sensors.
- Added new HALCON HDevelop example program to create a camera-specific calibration file.
- Adapted existing HDevelop example programs to open the device without the use of the xml file.
- Revision 4.0 (May 31, 2011):
- First official release.
- Revision 4.0 beta (Mar 1, 2011):
- First official beta release (as HALCON Kinect interface).
