Interface Documentation
Image Acquisition Interface for OpenNI compliant 3D Sensors
Interface: | OpenNI |
Revision: | 6.0 |
Date: | 2014-10-31 |
HALCON Version: | 12.0 |
OpenNI: Legacy Interface
- General
- General system requirements for using OpenNI
- Using PrimeSense Carmine
- Using ASUS Xtion
- Using Fotonic E-series
- Using Kinect for Windows
- Using Microsoft Kinect
- Features
- Limitations
- Camera Calibration
- Parameters for info_framegrabber
- Parameters for open_framegrabber
- Parameters for set_framegrabber_param
- Parameters for get_framegrabber_param
- Operator set_framegrabber_lut
- Operator get_framegrabber_lut
- Operator set_framegrabber_callback
- Operator get_framegrabber_callback
- Operator grab_image_start
- Operator grab_image
- Operator grab_image_async
- Operator grab_data
- Operator grab_data_async
- Operator close_framegrabber
- HDevelop Examples
- Release Notes
General
This page provides the documentation of the HALCON image acquisition interface for OpenNI compliant 3D sensors like
PrimeSense Carmine,
ASUS Xtion,
Fotonic E-series,
Microsoft Kinect for Windows,
or the Microsoft Kinect.
Registered customers can download the
latest revision of this interface from the
MVTec WWW server.
General system requirements for using OpenNI
- Intel compatible PC with Windows Vista/7, Windows Vista/7 x64, also WoW64 (using 32-bit HALCON on 64-bit Windows).
- HALCON image acquisition interface hAcqOpenNI.dll or hAcqOpenNIxl.dll, respectively. If you have properly installed the interface, both DLLs should reside in bin\%HALCONARCH% within the HALCON base directory %HALCONROOT% you have chosen during the installation of HALCON.
- Successful installation of OpenNI Build for Windows x86 (32-bit) v1.5.7.10 Development Edition (or higher) or OpenNI Build for Windows x64 (64-bit) v1.5.7.10 Development Edition (or higher). Please install the OpenNI runtime, before you start with the installation of the specific device driver.
- To get further information about the installation process of your device, please see the device specific sections below.
- After the installation process you can use the NiSimpleRead Viewer provided by the installation of OpenNI to verify that the setup was successful.
Using PrimeSense Carmine
The HALCON OpenNI Interface supports all Carmine models from Prime Sense.
- Successful installation of the PrimeSense driver version 5.1.6.6 for (32-bit) (or higher) or version 5.1.6.6 for (64-bit) (or higher).
- Check inside the Windows Device Manager, that the driver version is PrimeSense PS1080 3.1.3.1 (22.05.2012)
Using ASUS Xtion
The HALCON OpenNI Interface supports the ASUS Xtion series, the ASUS Xtion Pro and the ASUS Xtion Pro Live camera.
- Successful installation of the PrimeSense driver version 5.1.6.6 for (32-bit) (or higher) or version 5.1.6.6 for (64-bit) (or higher).
- Check inside the Windows Device Manager, that the driver version is PrimeSense PS1080 3.1.3.1 (22.05.2012)
- In case of an Kinect10.dll error, please verify that the Microsoft Kinect service is disabled and all Kinect for Windows installations are removed from the system.
Using Fotonic E-series
The Fotonic E-series is - in contrast to the original Microsoft Kinect sensor - based on the Fotonic SDK and needs an extra bridge
library to run with the HALCON OpenNI interface.
- Gigabit Ethernet network adapter with native driver installed.
- Successful installation of the SensorKinect driver SensorKinect-Win-OpenSource32-5.1.2.1 (32-bit)(or higher) or SensorKinect-Win-OpenSource64-5.1.2.1 (64-bit)(or higher).
- Successful installation of FotonicNiPlugin.dll 32-bit or 64-bit. If you do not have this file, please contact Fotonic or the vendor from which you bought the sensor. To register this dll, please use the niReg.exe program provided by OpenNI. Please note, that you need sufficient permissions to register the dll.
- To open the device in HALCON via open_framegrabber, the 'camera_type' parameter must be set to a valid configuration file (xml). It is not sufficient to use the value 'default'.
- Using different color spaces than 'default' is not supported.
Using Kinect for Windows
- Successful installation of the SensorKinect driver SensorKinect-Win-OpenSource32-5.1.2.1 (32-bit)(or higher) or SensorKinect-Win-OpenSource64-5.1.2.1 (64-bit)(or higher).
- Check inside the Camera Manager, that the an PrimeSense entry is shown and the driver version for Kinect for Windows camera has the driver version 3.1.3.1 with the date 22.05.2012.
- To use the near mode, this configuration must be set inside the configuration file (xml).
Using Microsoft Kinect
- Successful installation of the SensorKinect driver
SensorKinect-Win-OpenSource32-5.1.2.1
(32-bit)(or higher) or
SensorKinect-Win-OpenSource64-5.1.2.1
(64-bit)(or higher).
Note that the audio device of the Microsoft Kinect sensor is not supported by the provided OpenNI driver and is also not necessary for using the HALCON OpenNI interface. - Check inside the Windows Device Manager, that the driver version for the entry Camera is 3.1.3.1 (22.05.2012).
- Attention: For using the Microsoft Kinect 3D sensor it is not sufficient to install the OpenNI driver module. You have to replace the driver by the installation packages SensorKinect-Win-OpenSource[32|64]-5.1.2.1 as specified above. The link redirects to the github web site where you can find the needed installation packages.
- Visual Studio C++ 2008 Redistributable Runtime Package, particularly msvcp90.dll. If this package is not installed please download and install it from the Microsoft Download Center for Windows x86 or Windows x64. Note: It is not sufficient to copy the missing files!
Features
- Support of grab_image and grab_image_async to get the image (rgb) or the infrared image (gray).
- Support of grab_data and grab_data_async to get the image (rgb or gray), the depth image (uint2), and the calibrated X, Y, and Z images (each float).
- Support of multiple sensors.
- Support of the Microsoft Kinect for Windows via kinect-mssdk-openni-bridge.
Limitations
- No support of external or software trigger.
- No controlling of the motor of the Kinect sensor. This feature is currently not supported by the driver.
- Due to sensor restrictions of the Kinect sensor, the depth and color image are not synchronized.
- If the image type inside the xml configuration file is set to 'IR' and full resolution, or the 'ColorSpace' is set to 'grayd' and full resolution, a destroyed depth image will be returned. This is a limitation of the currently used driver. Please use the image type 'Image' inside the xml file, or the 'ColorSpace' with 'rgbd' and full resolution instead. If a VGA resolution is sufficient, you can also use the 'IR' image type inside the xml file with VGA resolution, or a 'ColorSpace' with 'grayd' and VGA resolution to get a valid depth image.
Camera Calibration
To get the calibrated X, Y, and Z images the camera calibration data is per default provided by the SDK. If you have created your own calibration data,
which could improve the results of the X,Y and Z images, you can set
parameters with
set_framegrabber_param(...,'cam_param_file','campar.dat').
It is also possible to set the camera calibration data via the corresponding
tuple with set_framegrabber_param(...,'cam_param',...). For further
information see also the documentation of the operator camera_calibration.
If you want to calibrate the 3D sensor on your own, please note that you
have to calibrate the rgb and infrared images separately.
For further processing of the 3D data, the operator xyz_to_object_model_3d can be used that transforms an image triple with the X, Y and Z coordinates of 3D points to a 3D object model.
For further processing of the 3D data, the operator xyz_to_object_model_3d can be used that transforms an image triple with the X, Y and Z coordinates of 3D points to a 3D object model.
Parameters for info_framegrabber
Parameter | Value List | Type | Kind | Description |
---|---|---|---|---|
'bits_per_channel' | [] | Unused. | ||
'camera_type' | ['default', '<path>'] | string | pre-defined | Default value and path for camera configuration file. |
'color_space' | ['default', 'gray', 'grayd', 'rgb', 'rgbd'] | string | pre-defined | Values for color space. |
'defaults' | [1, 1, 0, 0, 0, 0, 'progressive', 8, 'default', -1.0, 'false', 'default', '0', 0, 0] | mixed | pre-defined | Default values for open_framegrabber. |
'device' | ['0', '1', '2',...] | string | dynamic | Returns a pre-defined list of possible device numbers. |
'external_trigger' | [] | Unused. | ||
'field' | [] | Unused. | ||
'general' | [] | string | pre-defined | Information about the HALCON OpenNI interface. |
'generic' | [] | Unused. | ||
'horizontal_resolution' | [] | Unused. | ||
'image_height' | [] | Unsupported query. | ||
'image_width' | [] | Unsupported query. | ||
'info_boards' | ['device:<device_id>'] | string | dynamic | A list of the available devices. |
'parameters' | ['<parameters>'] | string | pre-defined | Pre-defined parameters of the HALCON interface. |
'parameters_readonly' | ['<parameters>'] | string | pre-defined | Pre-defined read-only parameters of the HALCON interface. |
'parameters_writeonly' | ['<parameters>'] | string | pre-defined | Pre-defined write-only parameters of the HALCON interface. |
'port' | [] | Unused. | ||
'revision' | '<revision>' | string | pre-defined | Revision number of the OpenNI interface. |
'start_column' | [] | Unsupported query. | ||
'start_row' | [] | Unsupported query. | ||
'vertical_resolution' | [] | Unused. |
Parameters for open_framegrabber
Parameter | Values | Default | Type | Description |
---|---|---|---|---|
Name | 'OpenNI' | string | Name of the HALCON interface. | |
HorizontalResolution | --- | 1 | Ignored. | |
VerticalResolution | --- | 1 | Ignored. | |
ImageWidth | 0, 640, 1280 | 0 | integer | Width of the desired image part ('0' stands for the complete image). The depth image and the corresponding XYZ images, which could be acquired via grab_data or grab_data_async, are always in VGA resolution. |
ImageHeight | 0, <height> | 0 | integer | Height of the desired image part ('0' stands for the complete image). The depth image and the corresponding XYZ images, which could be acquired via grab_data or grab_data_async, are always in VGA resolution. |
StartRow | --- | Ignored. | ||
StartColumn | --- | Ignored. | ||
Field | --- | Ignored. | ||
BitsPerChannel | --- | Ignored. | ||
ColorSpace | 'default', 'gray', 'grayd', 'rgb', 'rgbd' | 'default' | string | If the 'ColorSpace' parameter is set to 'gray' the infrared image is acquired by the device. If 'rgb' is selected, the device will provide a three channel rgb image. If 'grayd' or 'rgbd' is set, the call of grab_data or grab_data_async will provide an image and also the depth information. With 'default' the color space 'rgbd' will be set. If this color space is not supported, 'grayd' will be used instead. |
Generic | --- | Ignored. | ||
ExternalTrigger | --- | Ignored. | ||
CameraType | 'default', '<xml_file>' | 'default' | string | With 'default' the device initialization is done without the use of the xml file. If a xml file is given, the 'ColorSpace' parameter will be ignored. |
Device | 'default', '<id>' | '0' | string | Opens a specific device. All connected devices could be queried with info_framegrabber(..,'info_boards',...). Each device must be
connected to its own USB bus. To connect a camera only to another USB
port is not sufficient. If multiple sensors are connected, the device
that corresponds to '0' is connected to the first USB bus. If a xml file is set at the 'CameraType' parameter, the device parameter will be ignored and the connection to multiple devices is not supported. |
Port | --- | Ignored. | ||
LineIn | --- | Ignored. |
Parameters for set_framegrabber_param
Parameter | Values | Type | Description |
---|---|---|---|
'alternative_viewpoint' | 'disable', 'enable' | string | Point of view from the rgb image. If the parameter is set to 'enable' the point of view from the depth image and rgb image is equal. |
'cam_param' | [<Focus(float), Kappa(float), Sx(float), Sy(float), Cx(float), Cy(float), ImageWidth(int), ImageHeight(int)>] | mixed | Sets the camera calibration parameters directly. If successful, grab_data and grab_data_async will acquire the calibrated XYZ images based on this
information. If no calibration data is set or an empty tuple is used as
parameter, the calibration for the XYZ images is done by the SDK. The query of the parameter will only return default camera parameters, if an infrared image (gray image) is acquired and no camera calibration parameters were set before. |
'cam_param_file' | '<cam_par_file>' | string | Loads camera calibration parameters via a camera calibration file. If successful, grab_data and grab_data_async will deliver the calibrated XYZ images based on the calibration data, which is included in this file. If no calibration file is set or an empty tuple is used as parameter, the XYZ calibration is done by the SDK. |
'mirror_image' | 'false', 'true' | string | Flip the image on the 3D sensor from right to left. |
'xyz_images' | 'disable', 'enable' | string | Controls the grab_data/grab_data_async image tuple: if set to 'enable', grab_data and grab_data_async will deliver also the calibrated XYZ images. Note that this parameter will be set automatically to 'enable' if the camera calibration data was set by the parameters 'cam_param_file' or 'cam_param'. |
Parameters for get_framegrabber_param
There may exist additional read-only parameters with the following postfixes:
- '_description': These parameters provide the tool-tip of the corresponding parameter as a string.
- '_range': These parameters provide the minimum, maximum, step width, and default values for the corresponding integer or float parameter as a tuple with 4 elements, e.g., get_framegrabber_param(..,'Shutter_range',..) will return the output tuple [min,max,step,default]. Optionally, this tuple can also contain additional valid string values like 'auto' or 'manual'.
- '_values': These parameters provide the valid value list for the corresponding parameter as a tuple, e.g., get_framegrabber_param(..,'volatile_values',..) will return the output tuple ['enable','disable'].
All these postfixed parameter names are not returned when calling info_framegrabber(..,'parameters',..) and are used to enable the easy parameterization via a generic graphical user interface, particularly the HDevelop Image Acquisition Assistant.
Parameter | Values | Default | Type | Kind | Description |
---|---|---|---|---|---|
'alternative_viewpoint' | 'disable', 'enable' | 'disable' | string | pre-defined | Point of view from the rgb image. If the parameter is set to 'enable' the point of view from the depth image and rgb image is equal. |
'bits_per_channel' | '<default>' | 8 | integer | pre-defined | The value is not used, so a default value is returned. |
'cam_param' | [<Focus(float), Kappa(float), Sx(float), Sy(float), Cx(float), Cy(float), ImageWidth(int), ImageHeight(int)>] | mixed | dynamic | Parameters for camera calibration. | |
'cam_param_file' | '<cam_par_file>' | string | dynamic | Name of the used camera calibration file. | |
'camera_type' | '<xml_file>' | 'default' | string | pre-defined | Current camera type. |
'color_space' | 'gray', 'grayd', 'rgb', 'rgbd' | 'default' | string | pre-defined | Desired color space and thus the number of image channels of the resulting HALCON image. |
'device' | '<device_id>' | '0' | string | pre-defined | Current device id. |
'external_trigger' | <default> | 'false' | string | pre-defined | The value is not used, so a default value is returned. |
'field' | '<default>' | 'progressive' | string | pre-defined | The value is not used, so a default value is returned. |
'generic' | <default> | -1 | integer | pre-defined | The value is not used, so a default value is returned. |
'horizontal_resolution' | <resolution> | 1 | integer | pre-defined | Current value of horizontal resolution. |
'image_height' | <height> | 0 | integer | pre-defined | Height of the desired image part ('0' stands for the complete image). |
'image_width' | <width> | 0 | integer | pre-defined | Width of the desired image part ('0' stands for the complete image). |
'line_in' | <default> | 0 | integer | pre-defined | The value is not used, so a default value is returned. |
'mirror_image' | 'false', 'true' | 'false' | string | pre-defined | Flip the image on the 3D sensor from right to left. |
'name' | 'OpenNI' | string | pre-defined | Name of the HALCON interface. | |
'port' | <default> | 0 | integer | pre-defined | The value is not used, so a default value is returned. |
'revision' | '<revision>' | string | pre-defined | Revision number of the OpenNI interface. | |
'start_column' | <column> | 0 | integer | pre-defined | Returns the current start column of the HALCON image. |
'start_row' | <row> | 0 | integer | pre-defined | Returns the current start row of the HALCON image. |
'vertical_resolution' | <resolution> | 1 | integer | pre-defined | Current value of vertical resolution. |
'xyz_images' | 'disable', 'enable' | 'enable' | string | pre-defined | Status of xyz images in grab_data/grab_data_async. |
Operator set_framegrabber_lut
Not supported by this interface.
Operator get_framegrabber_lut
Not supported by this interface.
Operator set_framegrabber_callback
Not supported by this interface.
Operator get_framegrabber_callback
Not supported by this interface.
Operator grab_image_start
Starts a new asynchronous grab. See also grab_image_start.
Operator grab_image
grab_image starts a new synchronous grab. See also grab_image. Note that
the interface converts the image from the device to the desired image format
specified by the parameters 'image_width', 'image_height', 'start_row',
'start_column', 'bits_per_channel', and 'color_space'.
Operator grab_image_async
grab_image_async returns an image and starts the next asynchronous grab. See also grab_image_async.
Note that
the interface converts the image from the device to the desired image format
specified by the parameters 'image_width', 'image_height', 'start_row',
'start_column', 'bits_per_channel', and 'color_space'.
Operator grab_data
Parameter | Type | Count | Description |
---|---|---|---|
'image' | 1...5 | Depending on the parameters 'color_space' and 'xyz_images' the following data is returned:
|
|
'contours' | Not used. | ||
'region' | Not used. | ||
'data' | Not used. |
Operator grab_data_async
Same return values as grab_data, but here an asynchronous grab is started.
Operator close_framegrabber
This operator closes the device. See also close_framegrabber.
HDevelop Examples
For this interface there are the following examples available:
- openni_2cameras.hdev - Shows the usage of two cameras.
- openni_create_camera_calibration.hdev - Example for the calibration of the sensor.
- openni_parameters.hdev - Lists all parameters of a device.
- openni_simple.hdev - A simple example to show the usage of the interface.
- openni_surface_based_3d_matching.hdev - Shows the possibilities of the sensor with surfaced based 3D matching.
Release Notes
- Revision 6.0 (Oct 31, 2014):
- HALCON 12 version of the interface.
- Revision 5.2 Addendum (Apr 30, 2014):
- Adapted the documentation. Now the required installation package OpenNI Build v1.5.7.10 and PrimeSense driver v5.1.6.6 could be downloaded from the MVTec download server. The packages OpenNI Build 1.5.4.0 (or higher) and the PrimeSense driver 5.1.2.1 are still binary compatible with this interface.
- With the new installation packages the Kinect for Windows SDK and the kinect-mssdk-openni-bridge is no longer required by the Kinect for Windows device.
- Revision 5.2 (Sep 6, 2013):
- Support of the Fotonic E-series.
- Support of the PrimeSense Carmine series.
- Improved open_framegrabber. If an unsupported frame rate is specified via the xml file, a corrected setting will be used instead.
- Fixed bug in open_framegrabber if an unsupported device name was used.
- Created and adapted System Requirements section for each supported device series.
- Added support of Kinect for Windows SDK 1.6 and support of kinect-mssdk-openni-bridge-v1.6.0.0-for-1.5.2.23.
- Revision 5.1 (Aug 28, 2012):
- Support of the new Microsoft Kinect for Windows via kinect-mssdk-openni-bridge.
- Revision 5.0 (May 15, 2012):
- HALCON 11 version of the interface (included in HALCON 11 DVD).
- Revision 4.2 (Dec 8, 2011):
- Added support for Windows x64.
- Revision 4.1 (Sep 19, 2011):
- Adapted to OpenNI Build v1.1.0.41 or higher.
- Added support of multiple connected sensors.
- Added functionality in info_framegrabber(..,'info_boards',...) and info_framegrabber(..,'device',...) to enumerate the currently connected devices.
- Added support of internal provided calibration data.
- Adapted query of the 'cam_param' parameter. Together with an infrared image, now the parameter returns default camera parameters.
- Implemented 'ImageHeight' parameter in open_framegrabber to support software cropping.
- Added new parameter 'alternative_viewpoint'.
- Improved internal buffer handling to achieve a better performance.
- Renamed the calibration file extension inside the HDevelop examples from .cal to .dat.
- Adapted the device initialization in open_framegrabber. Now the use of an xml file is optional.
- Improved implementation of info_framegrabber(..,'camera_type',...) to return the OpenNI installation environment path.
- Added new HALCON HDevelop example program to show the usage of multiple sensors.
- Added new HALCON HDevelop example program to create a camera-specific calibration file.
- Adapted existing HDevelop example programs to open the device without the use of the xml file.
- Revision 4.0 (May 31, 2011):
- First official release.
- Revision 4.0 beta (Mar 1, 2011):
- First official beta release (as HALCON Kinect interface).