Image Acquisition Interface for Intel RealSense devices

This page provides the documentation of the HALCON RealSense image acquisition interface, which is based on GenICam GenTL and the RealSense SDK 2.0 to acquire images from RealSense devices. Registered customers can download the latest revision of this interface from the MVTec WWW server.
The interface internally contains a GenTL producer for RealSense devices which is referred to as 'RealSense Producer' in this document.

• General:
  • GenICam GenApi. The corresponding files are part of the HALCON RealSense package and are located in the directory genicam within the HALCON base directory %HALCONROOT%. See section GenICam GenApi for more details.
  • The computer has to be equipped with a USB 3.0 interface to connect the RealSense Device.
• Windows:
  • Windows 10 operating system.
  • HALCON image acquisition interface hAcqRealSense.dll or hAcqRealSensexl.dll, respectively. If you have properly installed the interface, all these DLLs should reside in bin\%HALCONARCH% within the HALCON base directory %HALCONROOT% you have chosen during the installation of HALCON.
    
  • The realsense2 shared library (realsense2.dll) must be present and found in the system in version 2.54.2. Please download and install the RealSense SDK package for Windows 10. Make sure that the realsense2.dll is found in the windows PATH variable, or copy it to bin\%HALCONARCH% within the HALCON base directory %HALCONROOT% you have chosen during the installation of HALCON.
    
• Linux:
  • Linux system with a working installation of the RealSense SDK. For additional system requirements please check the 'Supported Platforms' of the SDK.
  • HALCON image acquisition interface hAcqRealSense.so or hAcqRealSensexl.so, respectively. If you have properly installed the interface, the shared objects should reside in lib\$HALCONARCH within the HALCON base directory $HALCONROOT you have chosen during the installation of HALCON.
    
  • The realsense2 shared library (librealsense2.so) must be present and found in the system in version 2.54.2.
  • Please follow the RealSense instructions for installation on linux. Specific versions of the packages can be installed via sudo apt-get install <package>=<version>. If for your platform no dpkg packages are available, follow the build from source instructions.

MVTec interfaces for digital I/O and image acquisition are always compatible to a range of HALCON versions. Therefore, the versioning scheme both describes the compatibility of the interface and also the revision of the interface itself. An interface version always consists of three numbers, separated by dots, i.e. 24.11.5. The first two numbers describe the minimum HALCON version the interface is compatible with. For the example version 24.11.5 this means that the interface is compatible with all HALCON versions since HALCON 24.11. The last number describes the revision of the interface, in this example this is revision 5.

Only when installing or updating the interface manually follow these steps:

• Windows: Extract the archive containing the interface files to the HALCON base directory %HALCONROOT% (Note: Administrator privileges may be required for this step). Additionally, you have to move the interface examples to the directory %HALCONEXAMPLES% manually.

The interface is based on the RealSense SDK 2.0. The version used is 2.54.2. For a RealSense device to be usable with this interface it must provide at least a depth stream in the RealSense 'Z16' format. Compatibility has been tested with a device of the D400 and of the L500 family of devices.

• Automatic generation of an ObjectModel3D.
• Automatic handling of the color overlay for the ObjectModel3D.
• Invalid points are automatically excluded from the generated ObjectModel3D.
• Synchronous and asynchronous grabbing.
• Support of feature persistence.

• No external triggering.
• No support of software trigger.
• Only AcquisitionMode 'Continuous' is available.
• No support for RealSense infrared streams.
• No access to RealSense 'advanced mode' parameters.

The connection to the RealSense device is established through the open_framegrabber operator. The operator provides a set of parameters allowing to configure the basic properties of the images received from the device (see the corresponding section). In most use cases the parameter can be set to their default value. Almost all of the parameters can be also modified at runtime through set_framegrabber_param. This section concentrates solely on the 'Device' parameter of open_framegrabber, identifying the device that should be opened.

A connection to the device accessible through the RealSense interface is established using the device ID which is guaranteed to be unique per device and stays persistent across multiple sessions. The list of all devices currently available in the system can be obtained using info_framegrabber parameter 'device' or 'info_boards' – see corresponding section. Both parameters return a list of devices, each specified using entities in the format ' | device:<device id> | unique_name:<unique name> | ... '. The device id and the unique name are guaranteed to be always the same for each device. To increase compatibility with other HALCON image acquisition Interfaces they are both listed. The full string (obtained from the info_framegrabber call) can be directly used for the 'Device' parameter of open_framegrabber, which uses the same format. Only the ' | device:<device id>' string entry is mandatory for open_framegrabber, the others might be omitted.

When a pure string without any 'token:' and without the ' | ' separator is used, it is interpreted directly as the device ID.
As a special case, the string 'default' can be used – in this case the HALCON acquisition interface will use the first available device (devices currently reported as non-accessible will be skipped). However this can result in non deterministic opening of devices when multiple devices are connected to the system.

RealSense devices typically consist of multiple modules, where each module provides one type of data stream. For example, the depth module provides the depth data, the color module provides the color data which can be overlaid to generate colored point clouds. Each module might consist of multiple sub-components as well (i.e. the depth module of a D400 device consists of two infrared sensors and an infrared laser projector).
Configuration options are managed on a per-module basis. This means some options might be only available for one of the modules, some options might be available for both. As an example, the 'ExposureTime' feature can be configured for the depth and the color module, the 'LaserPower' feature can only be configured for the depth module. To cover this use case while keeping features like 'ExposureTime' singular, the SFNC (GenICam Standard Feature Naming Convention specifies the 'SourceSelector'. It allows to switch the target of the configuration option.
Even if a specific feature exists only for one of the modules, it will not be configurable while the 'SourceSelector' is pointing to the wrong module.

The usage concept is illustrated by the following example, which configures the 'ExposureTime' for each module differently:
set_framegrabber_param (AcqHandle,'SourceSelector','Depth')
set_framegrabber_param (AcqHandle,'ExposureTime',10000)
set_framegrabber_param (AcqHandle,'SourceSelector','Color')
set_framegrabber_param (AcqHandle,'ExposureTime',15000)

The names of the features reflect the names of the RealSense SDK sensor options, except where the SFNC dictates a specific different name. For each feature that maps to a RealSense SDK option, this document states the option explicitly (for example RS_OPTION_BACKLIGHT_COMPENSATION and so on). Those identifiers are contained in the RealSense SDK header file 'librealsense2/h/rs_option.h'.

The image acquisition can be either synchronous (grab_image/grab_data) or asynchronous (grab_image_start/grab_image_async/grab_data_async), see the reference documentation of the operators.
The interface fully configures and controls the acquisition process on the camera. Note that some of the camera features might be locked by GenICam when an acquisition is active.
With synchronous grab (grab_image/grab_data), a new acquisition is started internally for each image, so that the application always gets a new image. Before delivering the image, the acquisition is stopped again, so between individual grab_image/grab_data calls, all acquisition related features remain unlocked.
With asynchronous grabbing, started explicitly by grab_image_start or implicitly by grab_image_async/grab_data_async, the interface keeps the acquisition running internally, collecting further images to be delivered through future grab_image_async/grab_data_async calls. The acquisition related features are locked, until the acquisition is stopped using set_framegrabber_param(..., 'do_abort_grab', ...).
The HALCON acquisition interface properly recognizes the 'Continuous', 'SingleFrame' and 'MultiFrame' acquisition modes configured on the device and adjusts the acquisition control logic accordingly.
Furthermore, the interface takes over exclusive access to several remote device features essential for the acquisition control (AcquisitionStart, AcquisitionStop, AcquisitionAbort, TLParamsLocked). The user application has no direct way to control these features.
The differences between the "image" and "data" version of the grab operators is documented in Acquisition – Grab Operators.

The acquisition interface provides two mechanisms for acquisition of the image (or other) data from the device, grab_image/grab_image_async and grab_data/grab_data_async. Internally, both mechanisms work exactly the same (in particular how they acquire and process the data from the device), they differ in the way how the outputs are provided to the application.
The grab_image/grab_image_async operators can only output a single image. For RealSense devices this single image will contain the Z coordinates of the 3D points. It will be returned as an image of the image type 'real', where each pixel is a float value. This approach is also currently used by the HDevelop's Image Acquisition Assistant.
For RealSense devices the grab_image/grab_image_async operators are too limited to deal with all the data available. Instead the grab_data/grab_data_async operators should be used. The image tuple returned in the first output variable of the operators always contains the X, Y and Z planes of the 3D points. All of those are images of the 'real' image type. Additionally, if activated and if the device provides the necessary videostreams, the image tuple will contain the color overlay and the confidence map. To switch those additional components on and off, please refer to the SFNC features 'ComponentSelector' and 'ComponentEnable'. The operators can also return an automatically generated ObjectModel3D in the 'Data' output variable.
The structure of the provided outputs can also be queried with the help of the 'image_contents', 'data_contents' and related parameters.

Per default the grab_data operators will only output the various available images or image planes. To work with 3D data the 3D coordinates have to be combined into a point-cloud. In HALCON terminology this is called an ObjectModel3D. The RealSense acquisition interface provides mechanisms to automatically generate such an ObjectModel3D. To activate this feature use the parameter 'create_objectmodel3d'. Invalid points will automatically be excluded from the resulting ObjectModel3D. Additionally the process can be configured using the parameters 'coordinate_transform_mode', 'confidence_mode', 'confidence_threshold' and 'add_objectmodel3d_overlay_attrib'. Please refer to their documentation in the corresponding section.

The ObjectModel3D can also be generated manually by extracting the X, Y and Z planes from the image tuple and using the HALCON operator 'xyz_to_object_model_3d'.

During open_framegrabber the largest available resolution of the depth datastream is chosen automatically. If available, the color datastream and the confidence overlay are activated automatically.

The interface allocates 4 buffers for the acquisition engine by default (the number of buffers can be changed through the 'Generic' parameter of open_framegrabber).
Whenever a new image is acquired successfully and passed to the application as a HALCON image, the interface keeps the buffer locked (not returning it to the acquisition engine) until a new grab-related operator is called by the application or the acquisition is aborted using set_framegrabber_param(..., 'do_abort_grab', ...). During this period, it is fully safe to query information about this "last acquired" buffer – for example query buffer properties through get_framegrabber_param parameters such as 'buffer_timestamp', 'buffer_is_incomplete', 'image_width' and 'image_height'.
When a new grab-related operator is called by the application, the interface returns the buffer to the acquisition engine and buffer-related queries are not valid anymore.

It can happen, that the camera is temporarily or constantly acquiring data in higher speed than the application is processing them. In such case the streaming engine of the RealSense Producer decides how to treat the acquired buffers based on the '[Stream]StreamBufferHandlingMode' parameter.

• This interface uses GenApi version 3.5.0, for more details refer to the GenICam homepage. The corresponding files are part of the HALCON runtime installation and are located in the directory genicam within the HALCON base directory %HALCONROOT% or $HALCONROOT, respectively. This version is the same as the officially released version at the time of writing.
• The HALCON RealSense interface sets all necessary environment variables on its own and ignores other installed GenICam packages by default.
  If you want to use another GenICam package, you need to set the environment variable HALCON_USE_EXTERNAL_GENAPI. This skips the step of setting all necessary variables and paths internally, so you have to make sure they are set correctly. Please note that it might not be possible to use different GenApi versions with different interfaces at the same time and that you must use the required GenApi version for this interface.
• The caching of device XML files is activated to speed up processing, writing to a temporary directory subject to availability of %TEMP%, %TMP%, $TMPDIR, /tmp or %HALCONROOT%.
• The remote device control features as well as the RealSense Producer control features of this interface are controlled via GenApi.

The features of a device or RealSense Producer are described by GenICam feature description files (XML files) that are automatically parsed and offered to the user. The HALCON RealSense image acquisition interface provides access to the features exposed through the following GenICam feature description files:

• Features of the connected RealSense device are wrapped by the RealSense Producer and presented via a GenICam compatible XML file. Names for those features follow the SFNC recommendation as far as applicable.
• Features of the RealSense Producer are exposed through a set of GenICam description files, one for each internal entity to control the device tree:
  • The "system" – representing the overall behavior of the acquisition interface
  • The interface used to connect the device to the system.
  • A proxy (called "local device") to the device, controlling the RealSense Producer view of the device.
  • The data stream used for the acquisition (if the device provides any data streams).

In most cases the GenICam description files are provided by the device and the RealSense Producer. It is possible to store a copy of these files corresponding to an open connection by using the parameter 'do_write_configuration' (see set_framegrabber_param). The command will write all the GenICam description files and an "ini" file (details in set_framegrabber_param) with various information, including the location of the written GenICam description files, in the following format:

 
RemoteFile=%PATH_TO_GENICAM_FILE_OF_THE_DEVICE% 
SystemFile=%PATH_TO_GENICAM_FILE_OF_THE_SYSTEM_MODULE% 
InterfaceFile=%PATH_TO_GENICAM_FILE_OF_THE_INTERFACE_MODULE% 
DeviceFile=%PATH_TO_GENICAM_FILE_OF_THE_LOCAL_DEVICE_MODULE% 
StreamFile=%PATH_TO_GENICAM_FILE_OF_THE_STREAM_MODULE% 

The same ini-file format can be reused to force the HALCON acquisition interface to load an alternative XML file for one or more of these entities. This can be useful, e.g., for updates or troubleshooting. The files listed in the ini file will be used for the given entity instead of the original ones. For the entities excluded from the ini file, the GenICam description file will be searched and loaded the usual way. To apply the ini file, pass its full path to open_framegrabber in the 'CameraType' parameter.

Note that the ini-file can be reused also for other purposes such as storing/restoring configuration as described in Parameters – Persisting Device Status. Be aware that when persistence files are specified, they have priority over other explicit settings passed to open_framegrabber.

The following groups of parameters exist:

• Internal parameters of the MVTec GenTL Consumer itself (GenICamTL acquisition interface). These are named following the "underscore" naming style, e.g., color_space, and are all lowercase. However, MVTec EasyParams are prefixed with "[Consumer]", e.g., [Consumer]gain.
• GenICam-based parameters of the device, usually a camera, use by convention the "CamelCase" style, e.g., ExposureTime.
• GenICam-based parameters of the individual RealSense Producer modules (system, interface, device and data stream) use by convention the same style but are prefixed with the module name in square brackets. Beware that the system and interface modules might potentially be shared by multiple opened devices, so changing their configuration might have side effects on other connections as well. The following prefixes are used:
  • "[System]", e.g., [System]TLVendorName.
  • "[Interface]", e.g., [Interface]InterfaceType.
  • "[Device]", e.g., [Device]DeviceID.
  • "[Stream]", e.g., [Stream]StreamBufferHandlingMode.

The MVTec EasyParams is a group of MVTec GenTL Consumer parameters based on the regular GenICam features of the device. These parameters allow easy access to the most commonly used camera parameters.
The list of EasyParams can be obtained calling 'available_easyparam_names'. Note that 'available_param_names' does not include the EasyParams. The MVTec EasyParams start with the prefix [Consumer], indicating they are internal parameters of the HALCON GenTL consumer itself.
Since HALCON 22.11 they are available via an own tab in the HDevelop Image Acquisition Assistant.
If your device does not support one of the EasyParams, querying its access will return 'ni' (not implemented). If instead 'na' (not available) is returned, the EasyParam is supported but currently not available, e.g., it depends on the current state of another EasyParam or the access of one or more GenICam features, or because the image acquisition is running. In the latter case, use set_framegrabber_param(..., 'do_abort_grab', ...) to stop the acquisition.
List of MVTec EasyParams:

• General device information:
  • [Consumer]info_general
• Exposure feature:
  • [Consumer]exposure_auto
  • [Consumer]exposure
• Gain feature:
  • [Consumer]gain_auto
  • [Consumer]gain

Detailed information of each MVTec EasyParam can be found in sections about getting framegrabber parameters and setting framegrabber parameters.
See also the HDevelop example realsense_easyparams.hdev.

HALCON native parameter types are

• integer – signed integer, 64-bit on 64-bit platforms, 32-bit on 32-bit ones
• real – floating point type
• string – classical string

When accessing GenICam-based features, the GenICam data types must be mapped to the parameter types recognized by HALCON. GenICam offers the following types:

• IInteger – integer value, mapped directly to HALCON's integer parameter type. It will also accept real parameters. When the GenICam feature in question has a representation associated that identifies it as IPV4Address, MACAddress or HexNumber, the corresponding string representation will also be accepted. It should be highlighted that a problem can occur when running HALCON on a 32-bit platform (where integer parameters are 32-bit wide) and accessing a GenICam feature that is naturally 64-bit. In such cases the full range of the feature will not be available to the application and its value might be truncated. Also the sign of the parameter might be misinterpreted. To solve this problem, the parameter 'split_param_values_into_dwords' can be used. This maps the 64-bit IInteger value into two 32-bit integer values and vice versa.
• IFloat – floating point value, mapped directly to HALCON's real parameter type. It will also accept integer parameters.
• IString – string value, mapped directly to HALCON's string parameter type.
• IBoolean – boolean type is handled through HALCON's integer parameters, a value of 0 (zero) means logical false, other values mean logical true.
• IEnumeration – enumeration is a type that allows selecting from a set of values that are primarily identified by their name (the values are called 'enum entries'). Enumerations are interfaced through HALCON's string parameter type, while the enum entry names are used as the parameter values.
• ICommand – The command type allows to "execute" actions. To execute a command, use set_framegrabber_param with an integer parameter of any value (the value is ignored). Some commands' execution might take longer time and the command provides feedback when it is done. To query the status, use get_framegrabber_param for that command. It will return an integer value of 1 if it "is done" (command execution has finished), 0 if it has not finished yet. Note that if some other features are designed to be dependent on the command, querying the "is done" status might be essential to get the depending features invalidated (and updated with new values) as soon as the command execution finishes.
• IRegister – The register data type is used for plain memory blobs that cannot be mapped to any other data type. They are interfaced through HALCON's string parameter type, the register value is the hexadecimal string representation of the register's memory.

The current status of the acquisition device settings (values of all the parameters defining its working state) might be persisted while no acquisition is active, i.e. before grab_image_start or grab_image_async or after set_framegrabber_param(..., 'do_abort_grab', 1). This applies not only to actual device parameters, but also to parameters configuring the RealSense Producer and internal parameters of the RealSense image acquisition interface. Device parameters are usually kept until the device is powered down. The RealSense Producer modules and the Consumer parameters are kept until close_framegrabber is called. To indicate which parameters to persist, use the parameter 'settings_selector'. The persistence functionality consists of two steps, storing the current configuration to a file and later re-loading it back to the device. The selected module settings, indicated by 'settings_selector', can be stored using the 'do_write_settings' parameter and re-loaded later using the 'do_load_settings' parameter, specifying the desired persistence file path in both cases. To query if a feature can be persisted use the postfix '_streamable'.

Note that while the format of the files is intentionally human readable and the files can be hand-modified if desired, such modifications should be done with care by someone familiar with the GenICam persistence functionality internals and given device. Improper modifications of the files can lead to errors when using it.

The persistence related operations might be blocked by the device and/or RealSense Producer based on the current state, for example while an acquisition is in progress.

The same persistence files can be applied to the entire set of devices of the same type and firmware version. Applying the persistence files to a device of another type or using even different firmware version will probably lead to inconsistencies or will even fail completely .

Apart from the 'do_write_settings', the feature persistence file will also be written together with the ini file documented in the section Selection of GenICam Feature Description File(s) - using parameter 'do_write_configuration'. This command will generate extended version of the persistence file, storing not only the current device configuration, but also contents of its user sets and sequencer sets (if the device supports them). Additionally, it will also generate persistence files for all the RealSense Producer modules (system, interface, device and data stream). The persistence file entries in the ini file will have the format

 
RemotePersistence=%PATH_TO_PERSISTENCE_FILE_OF_THE_DEVICE% 
SystemPersistence=%PATH_TO_PERSISTENCE_FILE_OF_THE_SYSTEM_MODULE% 
InterfacePersistence=%PATH_TO_PERSISTENCE_FILE_OF_THE_INTERFACE_MODULE% 
DevicePersistence=%PATH_TO_PERSISTENCE_FILE_OF_THE_LOCAL_DEVICE_MODULE% 
StreamPersistence=%PATH_TO_PERSISTENCE_FILE_OF_THE_STREAM_MODULE% 

It is possible to receive notifications about changes of any features exposed through the GenICam interface by the camera and RealSense Producer as well as for internal parameters of the acquisition interface.
Note that the notifications might be raised in various circumstances, including:

• The application (you) explicitly changed that feature.
• Another feature has changed and the notified feature "depends" on the changed feature (the dependencies are defined in the GenICam description file).
• Access mode or current range for the feature has changed.
• As a result of regular "polling" in case of uncached features.
• As a result of device event delivery if the feature is connected to that event.
• As a result of new buffer delivery for features corresponding to chunk data.

Note that the notifications are raised whenever the feature is "marked dirty" (its cache invalidated) by one of the actions described above. It does not necessarily mean that its value has really changed, it is up to the application to check this.
Notification callbacks can be registered for individual features using set_framegrabber_callback - see corresponding operator documentation. Additionally, it is possible to use message queues to receive the event notification. In those cases it is necessary to create a message queue and then register the individual feature - see event message queues.

GenICam compatible devices can deliver asynchronous events which optionally carry additional data. It is usually necessary to enable delivery of individual event types using the features 'EventSelector'/'EventNotification' first. For SFNC-compliant events, this is done automatically if the parameter 'event_notification_helper' is enabled.
The decoding of the event data and matching them to the corresponding features, including potential notifications, is performed transparently by the interface.
The actual values might be read through the regular parameter reading mechanism like get_framegrabber_param or by get_message_tuple if you are using message queues to receive events. The choice of the event types to be generated is device-specific. The names of the event related features usually start by convention with a prefix 'Event' (examples might be 'EventFrameTrigger' and 'EventFrameTriggerTimestamp'), however, the device documentation should contain all the information about supported events and their corresponding feature names.
Although the data corresponding to the last delivered event can be in general read at any time, when using callback to receive events it is highly recommended that reading the event data is synchronized to notifications for corresponding event feature(s). Only in such a case it is guaranteed that the read data correspond exactly to the very event instance being notified – and that the feature values are not just being modified through a new instance of the same event. Note that the notifications are raised from context of the event handling/dispatching thread, so when processing the user callback, the event handling mechanism is paused. If multiple data items are associated with the same event, it is enough to register notification just for the actual event feature and read all the data during the callback.
If using message queues to receive events, you can decide to add additional data to be delivered with the corresponding event feature(s), see Event Message Queues. For this case the interface will read all the specified event features as soon as the event is generated and add it to the corresponding message. This guarantees that the delivered information corresponds with the actual value at the time the event was generated.
Besides the asynchronous events generated by the actual device, asynchronous events (optionally including additional data) can be generated by any module of the RealSense Producer (system, interface, device and data stream). The information provided above about handling of the device events applies similarly also to the RealSense Producer events, including enabling/disabling them (typically using 'EventSelector'/'EventNotification' features provided by given module, i.e. with corresponding module prefix in the feature name). For SFNC-compliant events, this is done automatically if the parameter 'event_notification_helper' is enabled.

This interface supports feature change notifications via message queues. Select the desired target feature with set_framegrabber_param(..., 'event_selector', ...). It is the same plain feature name (GenICam feature or internal parameter) as used with set_framegrabber_param, including a possible prefix, such as '[Device]' (refer to the Parameters – Naming Conventions).
Create a message queue at which you want to receive the notifications with create_message_queue and assign it to the selected feature with set_framegrabber_param(..., 'event_message_queue', QueueHandle).
The message queue can be registered for any GenICam based features, i.e., features published by the device and RealSense Producer or for internal features of the acquisition interface. On top of that, the notification can be registered also for internal events occurring within the acquisition interface, in particular 'event_new_buffer' which gets fired whenever the information about last acquired buffer is updated (new buffer grabbed or last buffer info invalidated when restarting the acquisition).
The list of supported targets can be queried by calling get_framegrabber_param(..., 'available_event_names', ...).
One of the important use cases for feature change callbacks is the device event delivery mechanism, see details in Event Data and Feature Change Notifications sections.
A new message would be added to the specified queue whenever a given feature is potentially changed (including its other properties such as range or access mode). Note that it does not necessarily always mean that the feature actually has a new value. A call to set_framegrabber_param(..., 'event_message_queue', 0) unregisters the previously registered message queue from the specified event. Note that the interface keeps just a single registration for every feature, if you attempt to register a new message queue for a feature that already had a message queue registered, the previous registration will be replaced with the new one.
The messages incoming on an event can be retrieved with dequeue_message and will contain at least three tuples. The first tuple (key 'id') is a unique identifier of the acquisition instance the event is coming from. It is a string composed as '<interface>:<unique_name>'. The second tuple (key 'event_name') is the name of the corresponding feature previously specified by 'event_selector'. The third tuple (key 'event_value') contains the value if the corresponding feature if available and readable.
If you decide to add additional data to be delivered with the corresponding event feature(s), add the features of interest with set_framegrabber_param(..., 'event_data', ...). Each event data feature will be appended to the event message with the key being its name and the tuple its value if readable. Note that duplicities possibly specified through 'event_data' are ignored The value of the notified feature itself (already reported in 'event_value') would also be considered as a duplicity and not reported again.

In case of using the HDevelop Image Acquisition Assistant the following hints will help to avoid problems:

• Some parameters depend on special conditions, e.g., a valid buffer or another parameter activated. After opening the camera these conditions may not yet be fulfilled, so the depending parameters are not shown. By using the 'Refresh' button, all parameters are read again and the depending parameters should appear if the conditions are fulfilled then.
• There are also some parameters regarding the image size and the payload size, which can only be changed if no acquisition takes place. The safest way to ensure this is to apply the action parameter 'do_abort_grab'. Please note that 'Update Image' has to be disabled first.
• The behavior of allowing changes to parameters while streaming is active depends on the capabilities of the device. It is possible that some cameras give you control over, e.g., the exposure time, while streaming and others do not.

The parameters of the RealSense device and RealSense Producer are accessed through GenICam and defined in GenICam description file(s) of the respective camera or RealSense Producer, so the parameter set is different for every product (although the parameter naming should adhere to SFNC and GenTL SFNC GenICam standard). A call of get_framegrabber_param(..., 'available_param_names', ...) returns a tuple containing all available parameters of the connected camera and RealSense Producer. See also section about parameter naming convention.
To set e.g. the current gain of the camera AcqHandle refers to (after calling open_framegrabber), the user can call set_framegrabber_param(AcqHandle, 'Gain', 6.0).
Please note that the interface sets the value of a parameter only if the value is valid. Integer and float values not matching the allowed range for given feature are aligned to the closest valid value. Invalid values of other feature types are refused.
Additionally to the GenICam parameters of the camera and of the RealSense Producer, the following HALCON interface parameters are supported by set_framegrabber_param:

There may exist additional read-only parameters with the following postfixes:

• '_access': These parameters provide the access permissions of the corresponding parameter as a string. Possible values are 'ro' (read-only), 'wo' (write-only), 'rw' (read/write), 'na' (not available), and 'ni' (not implemented).
• '_category': These parameters provide the category of the corresponding parameter as a string.
• '_description': These parameters provide the tool-tip of the corresponding parameter as a string.
• '_displayname': These parameters provide the displayname of the corresponding parameter as a string.
• '_longdescription': These parameters provide the description of the corresponding parameter as a string.
• '_range': These parameters provide the minimum, maximum, step width, and current values for the corresponding integer or float parameter as a tuple with 4 elements, e.g., get_framegrabber_param(.., 'Shutter_range', ..) will return the output tuple [min, max, step, current].
• '_representation': These parameters provide the GenICam representation of the corresponding parameter as a string. They only exist for integer-parameters. Possible values are 'Linear', 'Logarithmic', 'Boolean', 'PureNumber', 'HexNumber', 'IPV4Address' and 'MACAddress'. The purpose of the representation is to aid in GUI design.
• '_streamable': These parameters provide the persistence of the corresponding parameter as an integer. Possible values are 0 if false or 1 if true.
• '_string': These parameters provide the integer values formatted as a string according to the '_representation' postfix. Especially useful for the '_representation' values 'IPV4Address', 'MACAddress' and 'HexNumber'.
• '_type': These parameters provide the type of the corresponding parameter as string.
• '_values': These parameters provide the valid value list for the corresponding parameter as a tuple, e.g., get_framegrabber_param(.., 'volatile_values', ..) will return the output tuple ['enable', 'disable'].
• '_visibility': These parameters provide the visibility of the corresponding parameter as a string. Possible values are 'beginner', 'expert', and 'guru'.

All these postfixed parameter names are not returned when calling info_framegrabber(.., 'parameters', ..) and are used to enable the easy parameterization via a generic graphical user interface, particularly the HDevelop Image Acquisition Assistant.

Not supported by this interface.

Not supported by this interface.

This interface supports feature change callbacks via the operators set_framegrabber_callback and get_framegrabber_callback.
The callback can be registered for any GenICam based features, i.e., features published by the device and RealSense Producer through the GenICam description files as well as for internal parameters of the acquisition interface.
The list of supported callback targets can be queried by calling get_framegrabber_param(..., 'available_callback_types', ...).
One of the important use cases for feature change callbacks is the device event delivery mechanism, see details in event data and feature notifications sections. The 'CallbackType' parameter of set_framegrabber_callback defines the feature for which the callback is registered. It is the same plain feature name as used with set_framegrabber_param, including a possible prefix, such as '[Device]' (refer to the parameter naming convention).
The registered callback function would be called whenever a given feature is potentially changed (including its other properties such as range or access mode). Note that it does not necessarily always mean that the feature actually has a new value. If the callback function is set to NULL, the corresponding callback will be unregistered. Note that the interface keeps just a single registration for every feature, if you attempt to register a new callback for a feature that already had a callback registered, the previous registration will be replaced with the new one.

The signature of the callback function is Herror (__stdcall *HAcqCallback)(void *AcqHandle, void *Context, void *UserContext) and uses the following parameters:

• AcqHandle: Acquisition handle of the corresponding image acquisition instance.
• Context: Optional context data of the specific callback. Up to now, this parameter is not used, i.e., Context is set to NULL.
• UserContext: Optional context data of the specific callback. Up to now, this parameter is not used, i.e., UserContext is set to NULL.

Note that the execution time of a user-specific callback function must always be as short as possible since during the execution of a callback function the handling of further internal callbacks might be blocked. This can be achieved by removing the current processing from the user-specific callback function to a separate thread that is controlled via signals or events. The callback function is executed in the context of the underlying interface or driver.
It is also important to understand that the callbacks might be fired as a side-effect of parameter-setting or grab operations, ie. it might be called from within their respective locks. The user is responsible to take this into account in the callback handler to avoid risk of a deadlock.

This interface supports feature callbacks via the operators set_framegrabber_callback and get_framegrabber_callback. For more information see set_framegrabber_callback.

Starts a new asynchronous grab. See also grab_image_start and section about acquisition control. Note that this operator starts acquisition on the RealSense Producer and device and locks features protected during acquisition. If grab_image_start is called repeatedly, the acquisition is restarted on both RealSense Producer and device. That means, that the buffer queue and all stream related statistics parameters like 'buffer_frameid' are reset. The acquisition can be stopped (and the features unlocked) using set_framegrabber_param(..., 'do_abort_grab', ...).

grab_image starts a new synchronous grab of a single image. See also grab_image, section about acquisition control and about grab operators. Note that the interface converts the acquired image to the desired image format specified by the parameters 'bits_per_channel' and 'color_space'. The single image returned will contain the Z values of the 3D points.

grab_image_async returns a single image and starts the next asynchronous grab. See also grab_image_async, section about acquisition control and about grab operators. Note that the interface converts the acquired image to the desired image format specified by the parameters 'bits_per_channel' and 'color_space'.
The 'MaxDelay' parameter of the grab_image_async operator is ignored by the HALCON RealSense acquisition interface, because there is no way to support it reliably . If needed, the application needs to implement alternative functionality on its own. The single image returned will contain the Z values of the 3D points.

grab_data starts a new synchronous grab, resulting possibly in tuple of output images and tuple of data outputs of various kind, depending on the input and configuration. See also grab_data, section about acquisition control and about grab operators. Note that the interface converts the acquired images to the desired image format specified by the parameters 'bits_per_channel' and 'color_space'. The output tuples are described using the 'data_contents', 'image_contents' and related parameters. The image tuple returned will contain the X, Y and Z values of the 3D points. Based on the current configuration and device, it will also contain the color overlay and the confidence map. If activated via the parameter 'create_objectmodel3d' the data variable will contain the ObjectModel3D.

grab_data_async returns acquired images/data and starts the next asynchronous grab. See also grab_data_async, section about acquisition control and about grab operators. Note that the interface converts the acquired image to the desired image format specified by the parameters 'bits_per_channel' and 'color_space'. The output tuples are described using the 'data_contents', 'image_contents' and related parameters.
The 'MaxDelay' parameter of the grab_image_async operator is ignored by the HALCON RealSense acquisition interface, because there is no way to support it reliably . If needed, the application needs to implement alternative functionality on its own. The image tuple returned will contain the X, Y and Z values of the 3D points. Based on the current configuration and device it will also contain the color overlay and the confidence map. If activated via the parameter 'create_objectmodel3d' the data variable will contain the ObjectModel3D.

This operator closes the device. See also close_framegrabber.

For this interface there are the following examples available:

• realsense_component_mapping.hdev - Configure the mapping of depth and color images and demonstrate the implications of each mode.
• realsense_easyparams.hdev - Handling MVTec EasyParams.
• realsense_frame_rate.hdev - Measure the effective frame rate of an acquisition device.
• realsense_parameter_persistence.hdev - Writing parameter values to files and restore parameter values from files.
• realsense_simple.hdev - Use the interface to acquire 2D images.
• realsense_objectmodel3d.hdev - Automatically generation an ObjectModel3D during grab_data.

This interface depends on third-party libraries. See the file third_party_realsense.txt in the HALCON base directory for copyright and license information. Libraries are used without modifications unless explicitly stated in the file.

You can request the source for those third-party libraries licensed under GPL or LGPL via email to info@mvtec.com with the subject "Request source code of third-party libraries".

• Revision 24.11.10 (Apr 20, 2026):
  • The PixelFormat used for the color plane has been adapted to improve compatibility with new versions of HALCON.
  • zlib has been updated to version 1.3.2.
• Revision 24.11.9 (Oct 8, 2025):
  • There was a memory leak when registering for feature change notifications with set_framegrabber_callback or with 'event_message_queue'. This problem has been fixed.
  • A Software Bill of Materials (SBOM) in the System Package Data Exchange (SPDX) format has been added next to each binary file with the same name extended by ".spdx.json".
• Revision 24.11.8 (Apr 16, 2025):
  • The underlying GenApi version has been updated to the latest official release 3.5.0. The previous version failed to load settings that contained empty string parameters. This problem has been fixed.
  • The first access to the MVTec EasyParam [Consumer]exposure_auto and [Consumer]gain_auto could lead to a change in ExposureTime or Gain GenICam device parameter, respectively. This problem has been fixed.
  • Multithread usage of the MVTec EasyParams could lead to abnormal program termination. This problem has been fixed.
• Revision 20.11.7 (Mar 5, 2024):
  • The RealSense SDK 2.0 and thus the dependency to the librealsense has been updated to version 2.54.2.
  • The parameter 'ColorResolution' has been added to control the input resolution of the color sensor.
  • The parameter 'ComponentMappingMode' has been added to control how depth and color planes are mapped.
  • The HDevelop example realsense_component_mapping.hdev has been added to demonstrate the effects of the parameter 'ComponentMappingMode'.
  • The underlying GenApi version has been updated to the latest official release 3.4.2. This fixes a compatibility break with certain persistence files introduced in version 3.4.1.
  • Loading of configuration files during open_framegrabber (with the parameter CameraType) did not raise an exception, even when the file could not be read. This problem has been fixed.
  • When passing GenICam persistence files to open_framegrabber, the remote device was silently ignored. This problem has been fixed.
• Revision 20.11.6 (Jul 31, 2023):
  • The underlying GenApi version has been updated to the latest official release 3.4.1. This improves the validation of the string representation of parameters written through the '_string' postfix.
  • The first access to an MVTec EasyParam could change the value of its equivalent GenICam device parameter. This problem has been fixed.
  • Accessing an MVTec EasyParam could lead to a crash if any of the underlying GenICam parameters had an unexpected type. This problem has been fixed.
  • Querying the '_representation' of an MVTec EasyParam could in some cases lead to a crash. This problem has been fixed.
  • In rare cases, the access of not implementable MVTec EasyParams was erroneously set to 'na', instead of 'ni'. This problem has been fixed.
• Revision 20.11.5 (Apr 5, 2023):
  • The libCLAllSerial shared object has been removed from the redistribution of GenApi 3.4.0.
  • third_party_realsense.txt now contains links to the corresponding software location. See also Usage of third-party libraries.
• Revision 18.11.4 (Oct 21, 2022):
  • The MVTec EasyParams have been implemented to deal with the most common GenICam features in a simple and user-friendly way.
  • The new parameter 'available_easyparam_names' has been added to return the list of the MVTec EasyParams.
  • The HDevelop example realsense_easyparams.hdev has been added to demonstrate how to handle the MVTec EasyParams.
  • On a Unix system, loading GenICam persistence files that contained Windows line endings failed. This problem has been fixed.
• Revision 18.11.3 (May 11, 2022):
  • The underlying GenApi version has been updated to the latest official release 3.4. This fixes a crash during 'do_load_settings' when a device failed to execute the underlying command 'DeviceRegistersStreamingEnd'.
  • On a Unix system, loading Consumer persistence or configuration ini that was written on a Windows system failed. This problem has been fixed. Moreover, writing Consumer persistence or configuration ini files on a Windows system has been changed to use Unix line ending.
  • It is now possible to receive Feature Change Notifications on any parameter listed by 'available_param_names'.
  • A new parameter 'event_new_buffer' has been added which is only available as an event, see Feature Change Notifications and following.
  • There was a memory leak with each message delivered through the Event Message Queues mechanism. This problem has been fixed.
  • The identifier (key 'id') of the messages incoming on an event was not always unique. This problem has been fixed. The device name in the string ':' now corresponds with the 'unique_name' from info_framegrabber.
• Revision 18.11.2 (Oct 20, 2021):
  • This interface now is also available for Linux and Linux for Arm (64-bit) systems.
  • The description of the system requirements for Windows have been changed to clarify that full compatibility can only be expected with the correct version of the RealSense SDK installed.
  • The distribution package of this interface now contains files providing meta information for the HDevelop example browser.
• Revision 18.11.1 (May 14, 2021):
  • First official release.