binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparitybinocular_disparity (Operator)

Name

binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparitybinocular_disparity — Compute the disparities of a rectified image pair using correlation techniques.

Signature

binocular_disparity(ImageRect1, ImageRect2 : Disparity, Score : Method, MaskWidth, MaskHeight, TextureThresh, MinDisparity, MaxDisparity, NumLevels, ScoreThresh, Filter, SubDisparity : )

Herror binocular_disparity(const Hobject ImageRect1, const Hobject ImageRect2, Hobject* Disparity, Hobject* Score, const char* Method, const Hlong MaskWidth, const Hlong MaskHeight, double TextureThresh, const Hlong MinDisparity, const Hlong MaxDisparity, const Hlong NumLevels, double ScoreThresh, const char* Filter, const char* SubDisparity)

Herror T_binocular_disparity(const Hobject ImageRect1, const Hobject ImageRect2, Hobject* Disparity, Hobject* Score, const Htuple Method, const Htuple MaskWidth, const Htuple MaskHeight, const Htuple TextureThresh, const Htuple MinDisparity, const Htuple MaxDisparity, const Htuple NumLevels, const Htuple ScoreThresh, const Htuple Filter, const Htuple SubDisparity)

void BinocularDisparity(const HObject& ImageRect1, const HObject& ImageRect2, HObject* Disparity, HObject* Score, const HTuple& Method, const HTuple& MaskWidth, const HTuple& MaskHeight, const HTuple& TextureThresh, const HTuple& MinDisparity, const HTuple& MaxDisparity, const HTuple& NumLevels, const HTuple& ScoreThresh, const HTuple& Filter, const HTuple& SubDisparity)

HImage HImage::BinocularDisparity(const HImage& ImageRect2, HImage* Score, const HString& Method, Hlong MaskWidth, Hlong MaskHeight, const HTuple& TextureThresh, Hlong MinDisparity, Hlong MaxDisparity, Hlong NumLevels, const HTuple& ScoreThresh, const HTuple& Filter, const HString& SubDisparity) const

HImage HImage::BinocularDisparity(const HImage& ImageRect2, HImage* Score, const HString& Method, Hlong MaskWidth, Hlong MaskHeight, double TextureThresh, Hlong MinDisparity, Hlong MaxDisparity, Hlong NumLevels, double ScoreThresh, const HString& Filter, const HString& SubDisparity) const

HImage HImage::BinocularDisparity(const HImage& ImageRect2, HImage* Score, const char* Method, Hlong MaskWidth, Hlong MaskHeight, double TextureThresh, Hlong MinDisparity, Hlong MaxDisparity, Hlong NumLevels, double ScoreThresh, const char* Filter, const char* SubDisparity) const

HImage HImage::BinocularDisparity(const HImage& ImageRect2, HImage* Score, const wchar_t* Method, Hlong MaskWidth, Hlong MaskHeight, double TextureThresh, Hlong MinDisparity, Hlong MaxDisparity, Hlong NumLevels, double ScoreThresh, const wchar_t* Filter, const wchar_t* SubDisparity) const   (Windows only)

static void HOperatorSet.BinocularDisparity(HObject imageRect1, HObject imageRect2, out HObject disparity, out HObject score, HTuple method, HTuple maskWidth, HTuple maskHeight, HTuple textureThresh, HTuple minDisparity, HTuple maxDisparity, HTuple numLevels, HTuple scoreThresh, HTuple filter, HTuple subDisparity)

HImage HImage.BinocularDisparity(HImage imageRect2, out HImage score, string method, int maskWidth, int maskHeight, HTuple textureThresh, int minDisparity, int maxDisparity, int numLevels, HTuple scoreThresh, HTuple filter, string subDisparity)

HImage HImage.BinocularDisparity(HImage imageRect2, out HImage score, string method, int maskWidth, int maskHeight, double textureThresh, int minDisparity, int maxDisparity, int numLevels, double scoreThresh, string filter, string subDisparity)

def binocular_disparity(image_rect_1: HObject, image_rect_2: HObject, method: str, mask_width: int, mask_height: int, texture_thresh: Union[float, int], min_disparity: int, max_disparity: int, num_levels: int, score_thresh: Union[float, int], filter: MaybeSequence[str], sub_disparity: str) -> Tuple[HObject, HObject]

Description

binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparityBinocularDisparitybinocular_disparity computes pixel-wise correspondences between two rectified images using correlation techniques. Different from binocular_distancebinocular_distanceBinocularDistanceBinocularDistanceBinocularDistancebinocular_distance the results are not transformed into distance values.

The algorithm requires a reference image ImageRect1ImageRect1ImageRect1ImageRect1imageRect1image_rect_1 and a search image ImageRect2ImageRect2ImageRect2ImageRect2imageRect2image_rect_2 which must be rectified, i.e., corresponding epipolar lines are parallel and lie on identical image rows ( ). In case this assumption is violated the images can be rectified by using the operators calibrate_camerascalibrate_camerasCalibrateCamerasCalibrateCamerasCalibrateCamerascalibrate_cameras, gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapGenBinocularRectificationMapgen_binocular_rectification_map, and map_imagemap_imageMapImageMapImageMapImagemap_image. Hence, given a pixel in the reference image ImageRect1ImageRect1ImageRect1ImageRect1imageRect1image_rect_1 the homologous pixel in ImageRect2ImageRect2ImageRect2ImageRect2imageRect2image_rect_2 is selected by searching along the corresponding row in ImageRect2ImageRect2ImageRect2ImageRect2imageRect2image_rect_2 and matching a local neighborhood within a rectangular window of size MaskWidthMaskWidthMaskWidthMaskWidthmaskWidthmask_width and MaskHeightMaskHeightMaskHeightMaskHeightmaskHeightmask_height. The pixel correspondences are returned in the single-channel DisparityDisparityDisparityDisparitydisparitydisparity image which specifies for each pixel of the reference image ImageRect1ImageRect1ImageRect1ImageRect1imageRect1image_rect_1 a suitable matching pixel of ImageRect2ImageRect2ImageRect2ImageRect2imageRect2image_rect_2 according to the equation . A quality measure for each disparity value is returned in ScoreScoreScoreScorescorescore, containing the best result of the matching function S of a reference pixel. For the matching, the gray values of the original unprocessed images are used.

The used matching function is defined by the parameter MethodMethodMethodMethodmethodmethod allocating three different kinds of correlation:

with

, , , : row and column coordinates of the corresponding pixels of the two input images,

, : gray values of the unprocessed input images,

: size of correlation window

: mean value within the correlation window of width and height .

Note that the methods 'sad'"sad""sad""sad""sad""sad" and 'ssd'"ssd""ssd""ssd""ssd""ssd" compare the gray values of the pixels within a mask window directly, whereas 'ncc'"ncc""ncc""ncc""ncc""ncc" compensates for the mean gray value and its variance within the mask window. Therefore, if the two images differ in brightness and contrast, this method should be preferred. For images with similar brightness and contrast 'sad'"sad""sad""sad""sad""sad" and 'ssd'"ssd""ssd""ssd""ssd""ssd" are to be preferred as they are faster because of less complex internal computations.

It should be noted, that the quality of correlation for rising is falling in methods 'sad'"sad""sad""sad""sad""sad" and 'ssd'"ssd""ssd""ssd""ssd""ssd" (the best quality value is 0) but rising in method 'ncc'"ncc""ncc""ncc""ncc""ncc" (the best quality value is 1.0).

The size of the correlation window, referenced by and , has to be odd numbered and is passed in MaskWidthMaskWidthMaskWidthMaskWidthmaskWidthmask_width and MaskHeightMaskHeightMaskHeightMaskHeightmaskHeightmask_height. The search space is confined by the minimum and maximum disparity value MinDisparityMinDisparityMinDisparityMinDisparityminDisparitymin_disparity and MaxDisparityMaxDisparityMaxDisparityMaxDisparitymaxDisparitymax_disparity. Due to pixel values not defined beyond the image border the resulting domain of DisparityDisparityDisparityDisparitydisparitydisparity and ScoreScoreScoreScorescorescore is not set along the image border within a margin of height (MaskHeightMaskHeightMaskHeightMaskHeightmaskHeightmask_height-1)/2 at the top and bottom border and of width (MaskWidthMaskWidthMaskWidthMaskWidthmaskWidthmask_width-1)/2 at the left and right border. For the same reason, the maximum disparity range is reduced at the left and right image border.

Since matching turns out to be highly unreliable when dealing with poorly textured areas, the minimum statistical spread of gray values within the correlation window can be defined in TextureThreshTextureThreshTextureThreshTextureThreshtextureThreshtexture_thresh. This threshold is applied on both input images ImageRect1ImageRect1ImageRect1ImageRect1imageRect1image_rect_1 and ImageRect2ImageRect2ImageRect2ImageRect2imageRect2image_rect_2. In addition, ScoreThreshScoreThreshScoreThreshScoreThreshscoreThreshscore_thresh guarantees the matching quality and defines the maximum ('sad'"sad""sad""sad""sad""sad",'ssd'"ssd""ssd""ssd""ssd""ssd") or, respectively, minimum ('ncc'"ncc""ncc""ncc""ncc""ncc") score value of the correlation function. Setting FilterFilterFilterFilterfilterfilter to 'left_right_check'"left_right_check""left_right_check""left_right_check""left_right_check""left_right_check", moreover, increases the robustness of the returned matches, as the result relies on a concurrent direct and reverse match, whereas 'none'"none""none""none""none""none" switches it off.

The number of pyramid levels used to improve the time response of binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparityBinocularDisparitybinocular_disparity is determined by NumLevelsNumLevelsNumLevelsNumLevelsnumLevelsnum_levels. Following a coarse-to-fine scheme disparity images of higher levels are computed and segmented into rectangular subimages of similar disparity to reduce the disparity range on the next lower pyramid level. TextureThreshTextureThreshTextureThreshTextureThreshtextureThreshtexture_thresh and ScoreThreshScoreThreshScoreThreshScoreThreshscoreThreshscore_thresh are applied on every level and the returned domain of the DisparityDisparityDisparityDisparitydisparitydisparity and ScoreScoreScoreScorescorescore images arises from the intersection of the resulting domains of every single level. Generally, pyramid structures are the more advantageous the more the disparity image can be segmented into regions of homogeneous disparities and the bigger the disparity range is specified. As a drawback, coarse pyramid levels might loose important texture information which can result in deficient disparity values.

Finally, the value 'interpolation'"interpolation""interpolation""interpolation""interpolation""interpolation" for parameter SubDisparitySubDisparitySubDisparitySubDisparitysubDisparitysub_disparity performs subpixel refinement of disparities. It is switched off by setting the parameter to 'none'"none""none""none""none""none".

Execution Information

Parameters

ImageRect1ImageRect1ImageRect1ImageRect1imageRect1image_rect_1 (input_object)  singlechannelimage objectHImageHObjectHImageHobject (byte)

Rectified image of camera 1.

ImageRect2ImageRect2ImageRect2ImageRect2imageRect2image_rect_2 (input_object)  singlechannelimage objectHImageHObjectHImageHobject (byte)

Rectified image of camera 2.

DisparityDisparityDisparityDisparitydisparitydisparity (output_object)  singlechannelimage objectHImageHObjectHImageHobject * (real)

Disparity map.

ScoreScoreScoreScorescorescore (output_object)  singlechannelimage objectHImageHObjectHImageHobject * (real)

Evaluation of the disparity values.

MethodMethodMethodMethodmethodmethod (input_control)  string HTuplestrHTupleHtuple (string) (string) (HString) (char*)

Matching function.

Default value: 'ncc' "ncc" "ncc" "ncc" "ncc" "ncc"

List of values: 'ncc'"ncc""ncc""ncc""ncc""ncc", 'sad'"sad""sad""sad""sad""sad", 'ssd'"ssd""ssd""ssd""ssd""ssd"

MaskWidthMaskWidthMaskWidthMaskWidthmaskWidthmask_width (input_control)  integer HTupleintHTupleHtuple (integer) (int / long) (Hlong) (Hlong)

Width of the correlation window.

Default value: 11

Suggested values: 5, 7, 9, 11, 21

Restriction: 3 <= MaskWidth && odd(MaskWidth)

MaskHeightMaskHeightMaskHeightMaskHeightmaskHeightmask_height (input_control)  integer HTupleintHTupleHtuple (integer) (int / long) (Hlong) (Hlong)

Height of the correlation window.

Default value: 11

Suggested values: 5, 7, 9, 11, 21

Restriction: 3 <= MaskHeight && odd(MaskHeight)

TextureThreshTextureThreshTextureThreshTextureThreshtextureThreshtexture_thresh (input_control)  real HTupleUnion[float, int]HTupleHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong)

Variance threshold of textured image regions.

Default value: 0.0

Suggested values: 0.0, 10.0, 30.0

Restriction: 0.0 <= TextureThresh

MinDisparityMinDisparityMinDisparityMinDisparityminDisparitymin_disparity (input_control)  integer HTupleintHTupleHtuple (integer) (int / long) (Hlong) (Hlong)

Minimum of the expected disparities.

Default value: -30

Typical range of values: -32768 ≤ MinDisparity MinDisparity MinDisparity MinDisparity minDisparity min_disparity ≤ 32767

MaxDisparityMaxDisparityMaxDisparityMaxDisparitymaxDisparitymax_disparity (input_control)  integer HTupleintHTupleHtuple (integer) (int / long) (Hlong) (Hlong)

Maximum of the expected disparities.

Default value: 30

Typical range of values: -32768 ≤ MaxDisparity MaxDisparity MaxDisparity MaxDisparity maxDisparity max_disparity ≤ 32767

NumLevelsNumLevelsNumLevelsNumLevelsnumLevelsnum_levels (input_control)  integer HTupleintHTupleHtuple (integer) (int / long) (Hlong) (Hlong)

Number of pyramid levels.

Default value: 1

Suggested values: 1, 2, 3, 4

Restriction: 1 <= NumLevels

ScoreThreshScoreThreshScoreThreshScoreThreshscoreThreshscore_thresh (input_control)  real HTupleUnion[float, int]HTupleHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong)

Threshold of the correlation function.

Default value: 0.5

Suggested values: -1.0, 0.0, 0.3, 0.5, 0.7

FilterFilterFilterFilterfilterfilter (input_control)  string(-array) HTupleMaybeSequence[str]HTupleHtuple (string) (string) (HString) (char*)

Downstream filters.

Default value: 'none' "none" "none" "none" "none" "none"

List of values: 'left_right_check'"left_right_check""left_right_check""left_right_check""left_right_check""left_right_check", 'none'"none""none""none""none""none"

SubDisparitySubDisparitySubDisparitySubDisparitysubDisparitysub_disparity (input_control)  string HTuplestrHTupleHtuple (string) (string) (HString) (char*)

Subpixel interpolation of disparities.

Default value: 'none' "none" "none" "none" "none" "none"

List of values: 'interpolation'"interpolation""interpolation""interpolation""interpolation""interpolation", 'none'"none""none""none""none""none"

Example (HDevelop)

* Set internal and external stereo parameters.
* Note that, typically, these values are the result of a prior
* calibration.
gen_cam_par_area_scan_division (0.01, -665, 5.2e-006, 5.2e-006, \
                                622, 517, 1280, 1024, CamParam1)
gen_cam_par_area_scan_division (0.01, -731, 5.2e-006, 5.2e-006, \
                                654, 519, 1280, 1024, CamParam2)
create_pose (0.1535,-0.0037,0.0447,0.17,319.84,359.89, \
             'Rp+T', 'gba', 'point', RelPose)

* Compute the mapping for rectified images.
gen_binocular_rectification_map (Map1, Map2, CamParam1, CamParam2, RelPose, \
                                 1, 'viewing_direction', 'bilinear', \
                                 CamParamRect1,CamParamRect2, \
                                 Cam1PoseRect1, Cam2PoseRect2,RelPoseRect)

* Compute the disparities in online images.
while (1)
  grab_image_async (Image1, AcqHandle1, -1)
  map_image (Image1, Map1, ImageRect1)

  grab_image_async (Image2, AcqHandle2, -1)
  map_image (Image2, Map2, ImageRect2)

  binocular_disparity(ImageRect1, ImageRect2, Disparity, Score, 'sad', \
                      11, 11, 20, -40, 20, 2, 25, 'left_right_check', \
                      'interpolation')
endwhile

Example (HDevelop)

* Set internal and external stereo parameters.
* Note that, typically, these values are the result of a prior
* calibration.
gen_cam_par_area_scan_division (0.01, -665, 5.2e-006, 5.2e-006, \
                                622, 517, 1280, 1024, CamParam1)
gen_cam_par_area_scan_division (0.01, -731, 5.2e-006, 5.2e-006, \
                                654, 519, 1280, 1024, CamParam2)
create_pose (0.1535,-0.0037,0.0447,0.17,319.84,359.89, \
             'Rp+T', 'gba', 'point', RelPose)

* Compute the mapping for rectified images.
gen_binocular_rectification_map (Map1, Map2, CamParam1, CamParam2, RelPose, \
                                 1, 'viewing_direction', 'bilinear', \
                                 CamParamRect1,CamParamRect2, \
                                 Cam1PoseRect1, Cam2PoseRect2,RelPoseRect)

* Compute the disparities in online images.
while (1)
  grab_image_async (Image1, AcqHandle1, -1)
  map_image (Image1, Map1, ImageRect1)

  grab_image_async (Image2, AcqHandle2, -1)
  map_image (Image2, Map2, ImageRect2)

  binocular_disparity(ImageRect1, ImageRect2, Disparity, Score, 'sad', \
                      11, 11, 20, -40, 20, 2, 25, 'left_right_check', \
                      'interpolation')
endwhile

Example (C++)

// Set internal and external stereo parameters.
// Note that, typically, these values are the result of a prior
// calibration.
gen_cam_par_area_scan_division (0.01, -665, 5.2e-006, 5.2e-006,
                                622, 517, 1280, 1024, &CamParam1);
gen_cam_par_area_scan_division (0.01, -731, 5.2e-006, 5.2e-006,
                                654, 519, 1280, 1024, &CamParam2);
create_pose (0.1535,-0.0037,0.0447,0.17,319.84,359.89,
             "Rp+T", "gba", "point", &RelPose);
// Compute the mapping for rectified images.
gen_binocular_rectification_map(&Map1,&Map2,CamParam1,CamParam2,RelPose,1,
                                "viewing_direction","bilinear",
                                &CamParamRect1,&CamParamRect2,
                                &CamPoseRect1,&CamPoseRect2,&RelPosRect);

// Compute the disparities in online images.
while (1)
{
  grab_image_async(&Image1,AcqHandle1,-1);
  map_image(Image1,Map1,&ImageRect1);

  grab_image_async(&Image2,AcqHandle2,-1);
  map_image(Image2,Map2,&ImageRect2);

  binocular_disparity(ImageRect1,ImageRect2,&Disparity,&Score,"sad",
                      11,11,20,-40,20,2,25,"left_right_check",
                      "interpolation");
}

Example (HDevelop)

* Set internal and external stereo parameters.
* Note that, typically, these values are the result of a prior
* calibration.
gen_cam_par_area_scan_division (0.01, -665, 5.2e-006, 5.2e-006, \
                                622, 517, 1280, 1024, CamParam1)
gen_cam_par_area_scan_division (0.01, -731, 5.2e-006, 5.2e-006, \
                                654, 519, 1280, 1024, CamParam2)
create_pose (0.1535,-0.0037,0.0447,0.17,319.84,359.89, \
             'Rp+T', 'gba', 'point', RelPose)

* Compute the mapping for rectified images.
gen_binocular_rectification_map (Map1, Map2, CamParam1, CamParam2, RelPose, \
                                 1, 'viewing_direction', 'bilinear', \
                                 CamParamRect1,CamParamRect2, \
                                 Cam1PoseRect1, Cam2PoseRect2,RelPoseRect)

* Compute the disparities in online images.
while (1)
  grab_image_async (Image1, AcqHandle1, -1)
  map_image (Image1, Map1, ImageRect1)

  grab_image_async (Image2, AcqHandle2, -1)
  map_image (Image2, Map2, ImageRect2)

  binocular_disparity(ImageRect1, ImageRect2, Disparity, Score, 'sad', \
                      11, 11, 20, -40, 20, 2, 25, 'left_right_check', \
                      'interpolation')
endwhile

Example (HDevelop)

* Set internal and external stereo parameters.
* Note that, typically, these values are the result of a prior
* calibration.
gen_cam_par_area_scan_division (0.01, -665, 5.2e-006, 5.2e-006, \
                                622, 517, 1280, 1024, CamParam1)
gen_cam_par_area_scan_division (0.01, -731, 5.2e-006, 5.2e-006, \
                                654, 519, 1280, 1024, CamParam2)
create_pose (0.1535,-0.0037,0.0447,0.17,319.84,359.89, \
             'Rp+T', 'gba', 'point', RelPose)

* Compute the mapping for rectified images.
gen_binocular_rectification_map (Map1, Map2, CamParam1, CamParam2, RelPose, \
                                 1, 'viewing_direction', 'bilinear', \
                                 CamParamRect1,CamParamRect2, \
                                 Cam1PoseRect1, Cam2PoseRect2,RelPoseRect)

* Compute the disparities in online images.
while (1)
  grab_image_async (Image1, AcqHandle1, -1)
  map_image (Image1, Map1, ImageRect1)

  grab_image_async (Image2, AcqHandle2, -1)
  map_image (Image2, Map2, ImageRect2)

  binocular_disparity(ImageRect1, ImageRect2, Disparity, Score, 'sad', \
                      11, 11, 20, -40, 20, 2, 25, 'left_right_check', \
                      'interpolation')
endwhile

Result

binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparityBinocularDisparitybinocular_disparity returns TRUE if all parameter values are correct. If necessary, an exception is raised.

Possible Predecessors

map_imagemap_imageMapImageMapImageMapImagemap_image

Possible Successors

thresholdthresholdThresholdThresholdThresholdthreshold, disparity_to_distancedisparity_to_distanceDisparityToDistanceDisparityToDistanceDisparityToDistancedisparity_to_distance, disparity_image_to_xyzdisparity_image_to_xyzDisparityImageToXyzDisparityImageToXyzDisparityImageToXyzdisparity_image_to_xyz

Alternatives

binocular_disparity_mgbinocular_disparity_mgBinocularDisparityMgBinocularDisparityMgBinocularDisparityMgbinocular_disparity_mg, binocular_disparity_msbinocular_disparity_msBinocularDisparityMsBinocularDisparityMsBinocularDisparityMsbinocular_disparity_ms, binocular_distancebinocular_distanceBinocularDistanceBinocularDistanceBinocularDistancebinocular_distance, binocular_distance_mgbinocular_distance_mgBinocularDistanceMgBinocularDistanceMgBinocularDistanceMgbinocular_distance_mg, binocular_distance_msbinocular_distance_msBinocularDistanceMsBinocularDistanceMsBinocularDistanceMsbinocular_distance_ms

See also

map_imagemap_imageMapImageMapImageMapImagemap_image, gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapGenBinocularRectificationMapgen_binocular_rectification_map, binocular_calibrationbinocular_calibrationBinocularCalibrationBinocularCalibrationBinocularCalibrationbinocular_calibration

Module

3D Metrology