ClassesClasses | | Operators

gen_binocular_rectification_mapT_gen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMap (Operator)

Name

gen_binocular_rectification_mapT_gen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMap — Generate transformation maps that describe the mapping of the images of a binocular camera pair to a common rectified image plane.

Signature

gen_binocular_rectification_map( : Map1, Map2 : CamParam1, CamParam2, RelPose, SubSampling, Method, MapType : CamParamRect1, CamParamRect2, CamPoseRect1, CamPoseRect2, RelPoseRect)

Herror T_gen_binocular_rectification_map(Hobject* Map1, Hobject* Map2, const Htuple CamParam1, const Htuple CamParam2, const Htuple RelPose, const Htuple SubSampling, const Htuple Method, const Htuple MapType, Htuple* CamParamRect1, Htuple* CamParamRect2, Htuple* CamPoseRect1, Htuple* CamPoseRect2, Htuple* RelPoseRect)

void GenBinocularRectificationMap(HObject* Map1, HObject* Map2, const HTuple& CamParam1, const HTuple& CamParam2, const HTuple& RelPose, const HTuple& SubSampling, const HTuple& Method, const HTuple& MapType, HTuple* CamParamRect1, HTuple* CamParamRect2, HTuple* CamPoseRect1, HTuple* CamPoseRect2, HTuple* RelPoseRect)

HImage HImage::GenBinocularRectificationMap(const HCamPar& CamParam1, const HCamPar& CamParam2, const HPose& RelPose, double SubSampling, const HString& Method, const HString& MapType, HCamPar* CamParamRect1, HCamPar* CamParamRect2, HPose* CamPoseRect1, HPose* CamPoseRect2, HPose* RelPoseRect)

HImage HImage::GenBinocularRectificationMap(const HCamPar& CamParam1, const HCamPar& CamParam2, const HPose& RelPose, double SubSampling, const char* Method, const char* MapType, HCamPar* CamParamRect1, HCamPar* CamParamRect2, HPose* CamPoseRect1, HPose* CamPoseRect2, HPose* RelPoseRect)

HImage HCamPar::GenBinocularRectificationMap(HImage* Map2, const HCamPar& CamParam2, const HPose& RelPose, double SubSampling, const HString& Method, const HString& MapType, HCamPar* CamParamRect1, HCamPar* CamParamRect2, HPose* CamPoseRect1, HPose* CamPoseRect2, HPose* RelPoseRect) const

HImage HCamPar::GenBinocularRectificationMap(HImage* Map2, const HCamPar& CamParam2, const HPose& RelPose, double SubSampling, const char* Method, const char* MapType, HCamPar* CamParamRect1, HCamPar* CamParamRect2, HPose* CamPoseRect1, HPose* CamPoseRect2, HPose* RelPoseRect) const

HImage HPose::GenBinocularRectificationMap(HImage* Map2, const HCamPar& CamParam1, const HCamPar& CamParam2, double SubSampling, const HString& Method, const HString& MapType, HCamPar* CamParamRect1, HCamPar* CamParamRect2, HPose* CamPoseRect1, HPose* CamPoseRect2, HPose* RelPoseRect) const

HImage HPose::GenBinocularRectificationMap(HImage* Map2, const HCamPar& CamParam1, const HCamPar& CamParam2, double SubSampling, const char* Method, const char* MapType, HCamPar* CamParamRect1, HCamPar* CamParamRect2, HPose* CamPoseRect1, HPose* CamPoseRect2, HPose* RelPoseRect) const

static void HOperatorSet.GenBinocularRectificationMap(out HObject map1, out HObject map2, HTuple camParam1, HTuple camParam2, HTuple relPose, HTuple subSampling, HTuple method, HTuple mapType, out HTuple camParamRect1, out HTuple camParamRect2, out HTuple camPoseRect1, out HTuple camPoseRect2, out HTuple relPoseRect)

HImage HImage.GenBinocularRectificationMap(HCamPar camParam1, HCamPar camParam2, HPose relPose, double subSampling, string method, string mapType, out HCamPar camParamRect1, out HCamPar camParamRect2, out HPose camPoseRect1, out HPose camPoseRect2, out HPose relPoseRect)

HImage HCamPar.GenBinocularRectificationMap(out HImage map2, HCamPar camParam2, HPose relPose, double subSampling, string method, string mapType, out HCamPar camParamRect1, out HCamPar camParamRect2, out HPose camPoseRect1, out HPose camPoseRect2, out HPose relPoseRect)

HImage HPose.GenBinocularRectificationMap(out HImage map2, HCamPar camParam1, HCamPar camParam2, double subSampling, string method, string mapType, out HCamPar camParamRect1, out HCamPar camParamRect2, out HPose camPoseRect1, out HPose camPoseRect2, out HPose relPoseRect)

Description

Given a pair of stereo images, rectification determines a transformation of each image plane in a way that pairs of conjugate epipolar lines become collinear and parallel to the horizontal image axes. This is required for an efficient calculation of disparities or distances with operators such as binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparityBinocularDisparity or binocular_distancebinocular_distanceBinocularDistanceBinocularDistanceBinocularDistance. The rectified images can be thought of as acquired by a new stereo rig, obtained by rotating and, in case of telecentric cameras, translating the original cameras. The projection centers (i.e., in the telecentric case, the direction of the optical axes) are maintained. For perspective cameras, the image planes are additionally transformed into a common plane, which means that the focal lengths are set equal, and the optical axes are parallel. For a stereo setup of mixed type (i.e., one perspective and one telecentric camera), the image planes are also transformed into a common plane, as described below.

To achieve the transformation map for rectified images gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapGenBinocularRectificationMap requires the internal camera parameters CamParam1CamParam1CamParam1CamParam1camParam1 of camera 1 and CamParam2CamParam2CamParam2CamParam2camParam2 of camera 2, as well as the relative pose RelPoseRelPoseRelPoseRelPoserelPose defining a point transformation from camera 2 to camera 1. These parameters can be obtained, e.g., from the operator calibrate_camerascalibrate_camerasCalibrateCamerasCalibrateCamerasCalibrateCameras.

The internal camera parameters, modified by the rectification, are returned in CamParamRect1CamParamRect1CamParamRect1CamParamRect1camParamRect1 for camera 1 and CamParamRect2CamParamRect2CamParamRect2CamParamRect2camParamRect2 for camera 2, respectively. The rotation and, in case of telecentric cameras, translation of the rectified camera in relation to the original camera is specified by CamPoseRect1CamPoseRect1CamPoseRect1CamPoseRect1camPoseRect1 and CamPoseRect2CamPoseRect2CamPoseRect2CamPoseRect2camPoseRect2, respectively. Finally, RelPoseRectRelPoseRectRelPoseRectRelPoseRectrelPoseRect returns the modified relative pose of the rectified camera system 2 in relation to the rectified camera system 1.

Rectification Method

For perspective cameras, RelPoseRectRelPoseRectRelPoseRectRelPoseRectrelPoseRect has only a translation in x. Generally, the transformations are defined in a way that the rectified camera 1 is left of the rectified camera 2. This means that the optical center of camera 2 has a positive x coordinate of the rectified coordinate system of camera 1.

The projection onto a common plane has many degrees of freedom which are implicitly restricted by selecting a certain method in MethodMethodMethodMethodmethod (currently only one method available):

For telecentric cameras, the parameter MethodMethodMethodMethodmethod is ignored. The relative pose of both cameras is not uniquely defined in such a system, since the cameras return identical images no matter how they are translated along their optical axis. Yet, in order to define an absolute distance measurement to the cameras, a standard position of both cameras is considered. This position is defined as follows: Both cameras are translated along their optical axes until their distance is one meter and until the line between the cameras (baseline) forms the same angle with both optical axes (i.e. the baseline and the optical axes form an isosceles triangle). The optical axes remain unchanged. The relative pose of the rectified cameras RelPoseRectRelPoseRectRelPoseRectRelPoseRectrelPoseRect may be different from the relative pose of the original cameras RelPoseRelPoseRelPoseRelPoserelPose.

For a stereo setup of mixed type (i.e., one perspective and one telecentric camera), the parameter MethodMethodMethodMethodmethod is ignored. The rectified image plane is determined uniquely from the geometry of the perspective camera and the relative pose of the two cameras. The normal of the rectified image plane is the vector that points from the projection center of the perspective camera to the point on the optical axis of the telecentric camera that has the shortest distance from the projection center of the perspective camera. This is also the z-axis of the rectified perspective camera. The geometric base of the mixed camera system is a line that passes through the projection center of the perspective camera and has the same direction as the z-axis of the telecentric camera, i.e., the base is parallel to the viewing direction of the telecentric camera. The x-axis of the rectified perspective camera is given by the base and the y-axis is constructed to form a right-handed coordinate system. To rectify the telecentric camera, its optical axis must be shifted to the base and the image plane must be tilted by or . To achieve this, a special type of object-side telecentric camera that is able to handle this special rectification goemetry (indicated by a negative image plane distance ImagePlaneDistImagePlaneDistImagePlaneDistImagePlaneDistimagePlaneDist) must be used for the rectified telecentric camera. The representation of this special camera type should be regarded as a black box because it is used only for rectification purposes in HALCON (for this reason, it is not documented in camera_calibrationcamera_calibrationCameraCalibrationCameraCalibrationCameraCalibration). The rectified telecentric camera has the same orientation as the original telecentric camera, while its origin is translated to a point on the base.

Rectification Maps

The mapping functions for the images of camera 1 and camera 2 are returned in the images Map1Map1Map1Map1map1 and Map2Map2Map2Map2map2. MapTypeMapTypeMapTypeMapTypemapType is used to specify the type of the output maps. If 'nearest_neighbor'"nearest_neighbor""nearest_neighbor""nearest_neighbor""nearest_neighbor" is chosen, both maps consist of one image containing one channel, in which for each pixel of the resulting image the linearized coordinate of the pixel of the input image is stored that is the nearest neighbor to the transformed coordinates. If 'bilinear'"bilinear""bilinear""bilinear""bilinear" interpolation is chosen, both maps consists of one image containing five channels. In the first channel for each pixel in the resulting image the linearized coordinates of the pixel in the input image is stored that is in the upper left position relative to the transformed coordinates. The four other channels contain the weights of the four neighboring pixels of the transformed coordinates which are used for the bilinear interpolation, in the following order:

2 3
4 5

The second channel, for example, contains the weights of the pixels that lie to the upper left relative to the transformed coordinates. If 'coord_map_sub_pix'"coord_map_sub_pix""coord_map_sub_pix""coord_map_sub_pix""coord_map_sub_pix" is chosen, both maps consist of one vector field image, in which for each pixel of the resulting image the subpixel precise coordinates in the input image are stored.

The size and resolution of the maps and of the transformed images can be adjusted by the SubSamplingSubSamplingSubSamplingSubSamplingsubSampling parameter which applies a sub-sampling factor to the original images.

If you want to re-use the created map in another program, you can save it as a multi-channel image with the operator write_imagewrite_imageWriteImageWriteImageWriteImage, using the format 'tiff'"tiff""tiff""tiff""tiff".

Execution Information

Parameters

Map1Map1Map1Map1map1 (output_object)  (multichannel-)image objectHImageHImageHobject * (int4 / uint2 / vector_field)

Image containing the mapping data of camera 1.

Map2Map2Map2Map2map2 (output_object)  (multichannel-)image objectHImageHImageHobject * (int4 / uint2 / vector_field)

Image containing the mapping data of camera 2.

CamParam1CamParam1CamParam1CamParam1camParam1 (input_control)  campar HCamPar, HTupleHTupleHtuple (real / integer / string) (double / int / long / string) (double / Hlong / HString) (double / Hlong / char*)

Internal parameters of camera 1.

CamParam2CamParam2CamParam2CamParam2camParam2 (input_control)  campar HCamPar, HTupleHTupleHtuple (real / integer / string) (double / int / long / string) (double / Hlong / HString) (double / Hlong / char*)

Internal parameters of camera 2.

RelPoseRelPoseRelPoseRelPoserelPose (input_control)  pose HPose, HTupleHTupleHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong)

Point transformation from camera 2 to camera 1.

Number of elements: 7

SubSamplingSubSamplingSubSamplingSubSamplingsubSampling (input_control)  real HTupleHTupleHtuple (real) (double) (double) (double)

Subsampling factor.

Default value: 1.0

Suggested values: 0.5, 0.66, 1.0, 1.5, 2.0, 3.0, 4.0

MethodMethodMethodMethodmethod (input_control)  string HTupleHTupleHtuple (string) (string) (HString) (char*)

Type of rectification.

Default value: 'geometric' "geometric" "geometric" "geometric" "geometric"

List of values: 'geometric'"geometric""geometric""geometric""geometric"

MapTypeMapTypeMapTypeMapTypemapType (input_control)  string HTupleHTupleHtuple (string) (string) (HString) (char*)

Type of mapping.

Default value: 'bilinear' "bilinear" "bilinear" "bilinear" "bilinear"

List of values: 'bilinear'"bilinear""bilinear""bilinear""bilinear", 'coord_map_sub_pix'"coord_map_sub_pix""coord_map_sub_pix""coord_map_sub_pix""coord_map_sub_pix", 'nearest_neighbor'"nearest_neighbor""nearest_neighbor""nearest_neighbor""nearest_neighbor"

CamParamRect1CamParamRect1CamParamRect1CamParamRect1camParamRect1 (output_control)  campar HCamPar, HTupleHTupleHtuple (real / integer / string) (double / int / long / string) (double / Hlong / HString) (double / Hlong / char*)

Rectified internal parameters of camera 1.

CamParamRect2CamParamRect2CamParamRect2CamParamRect2camParamRect2 (output_control)  campar HCamPar, HTupleHTupleHtuple (real / integer / string) (double / int / long / string) (double / Hlong / HString) (double / Hlong / char*)

Rectified internal parameters of camera 2.

CamPoseRect1CamPoseRect1CamPoseRect1CamPoseRect1camPoseRect1 (output_control)  pose HPose, HTupleHTupleHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong)

Point transformation from the rectified camera 1 to the original camera 1.

Number of elements: 7

CamPoseRect2CamPoseRect2CamPoseRect2CamPoseRect2camPoseRect2 (output_control)  pose HPose, HTupleHTupleHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong)

Point transformation from the rectified camera 1 to the original camera 1.

Number of elements: 7

RelPoseRectRelPoseRectRelPoseRectRelPoseRectrelPoseRect (output_control)  pose HPose, HTupleHTupleHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong)

Point transformation from the rectified camera 2 to the rectified camera 1.

Number of elements: 7

Example (HDevelop)

* Set internal and external stereo parameters.
* Note that, typically, these values are the result of a prior
* calibration.
gen_cam_par_area_scan_division (0.01, -665, 5.2e-006, 5.2e-006, \
                                622, 517, 1280, 1024, CamParam1)
gen_cam_par_area_scan_division (0.01, -731, 5.2e-006, 5.2e-006, \
                                654, 519, 1280, 1024, CamParam2)
create_pose (0.1535,-0.0037,0.0447,0.17,319.84,359.89, \
             'Rp+T', 'gba', 'point', RelPose)

* Compute the mapping for rectified images.
gen_binocular_rectification_map (Map1, Map2, CamParam1, CamParam2, \
                                 RelPose, 1,'geometric', 'bilinear', \
                                 CamParamRect1, CamParamRect2, \
                                 CamPoseRect1, CamPoseRect2, \
                                 RelPoseRect)

* Compute the disparities in online images.
while (1)
  grab_image_async (Image1, AcqHandle1, -1)
  map_image (Image1, Map1, ImageMapped1)

  grab_image_async (Image2, AcqHandle2, -1)
  map_image (Image2, Map2, ImageMapped2)

  binocular_disparity(ImageMapped1, ImageMapped2, Disparity, Score, \
                      'sad', 11, 11, 20, -40, 20, 2, 25, \
                      'left_right_check', 'interpolation')
endwhile

Example (HDevelop)

* Set internal and external stereo parameters.
* Note that, typically, these values are the result of a prior
* calibration.
gen_cam_par_area_scan_division (0.01, -665, 5.2e-006, 5.2e-006, \
                                622, 517, 1280, 1024, CamParam1)
gen_cam_par_area_scan_division (0.01, -731, 5.2e-006, 5.2e-006, \
                                654, 519, 1280, 1024, CamParam2)
create_pose (0.1535,-0.0037,0.0447,0.17,319.84,359.89, \
             'Rp+T', 'gba', 'point', RelPose)

* Compute the mapping for rectified images.
gen_binocular_rectification_map (Map1, Map2, CamParam1, CamParam2, \
                                 RelPose, 1,'geometric', 'bilinear', \
                                 CamParamRect1, CamParamRect2, \
                                 CamPoseRect1, CamPoseRect2, \
                                 RelPoseRect)

* Compute the disparities in online images.
while (1)
  grab_image_async (Image1, AcqHandle1, -1)
  map_image (Image1, Map1, ImageMapped1)

  grab_image_async (Image2, AcqHandle2, -1)
  map_image (Image2, Map2, ImageMapped2)

  binocular_disparity(ImageMapped1, ImageMapped2, Disparity, Score, \
                      'sad', 11, 11, 20, -40, 20, 2, 25, \
                      'left_right_check', 'interpolation')
endwhile

Example (HDevelop)

* Set internal and external stereo parameters.
* Note that, typically, these values are the result of a prior
* calibration.
gen_cam_par_area_scan_division (0.01, -665, 5.2e-006, 5.2e-006, \
                                622, 517, 1280, 1024, CamParam1)
gen_cam_par_area_scan_division (0.01, -731, 5.2e-006, 5.2e-006, \
                                654, 519, 1280, 1024, CamParam2)
create_pose (0.1535,-0.0037,0.0447,0.17,319.84,359.89, \
             'Rp+T', 'gba', 'point', RelPose)

* Compute the mapping for rectified images.
gen_binocular_rectification_map (Map1, Map2, CamParam1, CamParam2, \
                                 RelPose, 1,'geometric', 'bilinear', \
                                 CamParamRect1, CamParamRect2, \
                                 CamPoseRect1, CamPoseRect2, \
                                 RelPoseRect)

* Compute the disparities in online images.
while (1)
  grab_image_async (Image1, AcqHandle1, -1)
  map_image (Image1, Map1, ImageMapped1)

  grab_image_async (Image2, AcqHandle2, -1)
  map_image (Image2, Map2, ImageMapped2)

  binocular_disparity(ImageMapped1, ImageMapped2, Disparity, Score, \
                      'sad', 11, 11, 20, -40, 20, 2, 25, \
                      'left_right_check', 'interpolation')
endwhile

Example (HDevelop)

* Set internal and external stereo parameters.
* Note that, typically, these values are the result of a prior
* calibration.
gen_cam_par_area_scan_division (0.01, -665, 5.2e-006, 5.2e-006, \
                                622, 517, 1280, 1024, CamParam1)
gen_cam_par_area_scan_division (0.01, -731, 5.2e-006, 5.2e-006, \
                                654, 519, 1280, 1024, CamParam2)
create_pose (0.1535,-0.0037,0.0447,0.17,319.84,359.89, \
             'Rp+T', 'gba', 'point', RelPose)

* Compute the mapping for rectified images.
gen_binocular_rectification_map (Map1, Map2, CamParam1, CamParam2, \
                                 RelPose, 1,'geometric', 'bilinear', \
                                 CamParamRect1, CamParamRect2, \
                                 CamPoseRect1, CamPoseRect2, \
                                 RelPoseRect)

* Compute the disparities in online images.
while (1)
  grab_image_async (Image1, AcqHandle1, -1)
  map_image (Image1, Map1, ImageMapped1)

  grab_image_async (Image2, AcqHandle2, -1)
  map_image (Image2, Map2, ImageMapped2)

  binocular_disparity(ImageMapped1, ImageMapped2, Disparity, Score, \
                      'sad', 11, 11, 20, -40, 20, 2, 25, \
                      'left_right_check', 'interpolation')
endwhile

Example (HDevelop)

* Set internal and external stereo parameters.
* Note that, typically, these values are the result of a prior
* calibration.
gen_cam_par_area_scan_division (0.01, -665, 5.2e-006, 5.2e-006, \
                                622, 517, 1280, 1024, CamParam1)
gen_cam_par_area_scan_division (0.01, -731, 5.2e-006, 5.2e-006, \
                                654, 519, 1280, 1024, CamParam2)
create_pose (0.1535,-0.0037,0.0447,0.17,319.84,359.89, \
             'Rp+T', 'gba', 'point', RelPose)

* Compute the mapping for rectified images.
gen_binocular_rectification_map (Map1, Map2, CamParam1, CamParam2, \
                                 RelPose, 1,'geometric', 'bilinear', \
                                 CamParamRect1, CamParamRect2, \
                                 CamPoseRect1, CamPoseRect2, \
                                 RelPoseRect)

* Compute the disparities in online images.
while (1)
  grab_image_async (Image1, AcqHandle1, -1)
  map_image (Image1, Map1, ImageMapped1)

  grab_image_async (Image2, AcqHandle2, -1)
  map_image (Image2, Map2, ImageMapped2)

  binocular_disparity(ImageMapped1, ImageMapped2, Disparity, Score, \
                      'sad', 11, 11, 20, -40, 20, 2, 25, \
                      'left_right_check', 'interpolation')
endwhile

Result

gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapGenBinocularRectificationMap returns 2 (H_MSG_TRUE) if all parameter values are correct. If necessary, an exception is raised.

Possible Predecessors

binocular_calibrationbinocular_calibrationBinocularCalibrationBinocularCalibrationBinocularCalibration

Possible Successors

map_imagemap_imageMapImageMapImageMapImage

Alternatives

gen_image_to_world_plane_mapgen_image_to_world_plane_mapGenImageToWorldPlaneMapGenImageToWorldPlaneMapGenImageToWorldPlaneMap

See also

map_imagemap_imageMapImageMapImageMapImage, binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparityBinocularDisparity, binocular_distancebinocular_distanceBinocularDistanceBinocularDistanceBinocularDistance, binocular_disparity_mgbinocular_disparity_mgBinocularDisparityMgBinocularDisparityMgBinocularDisparityMg, binocular_distance_mgbinocular_distance_mgBinocularDistanceMgBinocularDistanceMgBinocularDistanceMg, binocular_disparity_msbinocular_disparity_msBinocularDisparityMsBinocularDisparityMsBinocularDisparityMs, binocular_distance_msbinocular_distance_msBinocularDistanceMsBinocularDistanceMsBinocularDistanceMs, gen_image_to_world_plane_mapgen_image_to_world_plane_mapGenImageToWorldPlaneMapGenImageToWorldPlaneMapGenImageToWorldPlaneMap, contour_to_world_plane_xldcontour_to_world_plane_xldContourToWorldPlaneXldContourToWorldPlaneXldContourToWorldPlaneXld, image_points_to_world_planeimage_points_to_world_planeImagePointsToWorldPlaneImagePointsToWorldPlaneImagePointsToWorldPlane

Module

3D Metrology


ClassesClasses | | Operators