binocular_disparity (Operator)

Name

binocular_disparity — Compute the disparities of a rectified image pair using correlation techniques.

Synopsis

binocular_disparity(Image1, Image2 : Disparity, Score : Method, MaskWidth, MaskHeight, TextureThresh, MinDisparity, MaxDisparity, NumLevels, ScoreThresh, Filter, SubDisparity : )

Description

binocular_disparity computes pixel-wise correspondences between two rectified images using correlation techniques. Different from binocular_distance the results are not transformed into distance values.

The algorithm requires a reference image Image1 and a search image Image2 which must be rectified, i.e., corresponding epipolar lines are parallel and lie on identical image rows ( r1=r2 ). In case this assumption is violated the images can be rectified by using the operators binocular_calibration, gen_binocular_rectification_map, and map_image. Hence, given a pixel in the reference image Image1 the homologous pixel in Image2 is selected by searching along the corresponding row in Image2 and matching a local neighborhood within a rectangular window of size MaskWidth and MaskHeight. The pixel correspondences are returned in the single-channel Disparity image d(r1,c1) which specifies for each pixel (r1,c1) of the reference image Image1 a suitable matching pixel (r2,c2) of Image2 according to the equation c2 = c1 + d(r1,c1) . A quality measure for each disparity value is returned in Score, containing the best result of the matching function S of a reference pixel. For the matching, the gray values of the original unprocessed images are used.

The used matching function is defined by the parameter Method allocating three different kinds of correlation:

'sad': Summed Absolute Differences  
       S(r1,c1,d) := SUM(|g1(r1',c1')-g2(r1',c1'+d)|), 0=<=S<=255,
'ssd': Summed Squared Differences   
       S(r1,c1,d) := SUM((g1(r1',c1')-g2(r1',c1'+d))²), 0<=S<=65025, 
'ncc': Normalized Cross Correlation 
       S(r1,c1,d) :=
          SUM(g1(r1',c1')-gm1(r1,c1))*(g2(r1',c1'+d)-gm2(r1,c1+d)))/
          SQRT(SUM((g1(r1',c1')-gm1(r1,c1))²)*
               SUM((g2(r1',c1+d)-gm2(r1,c1+d))²)),
       -1.0<=S<=1.0,

whereas SUM(x) is the averaged summation of x within a window of
size (2m+1) x (2n+1). Especially gm(r,c) is the mean value of
g(r',c') within this window at row r and column c. g1 and g2 are the gray
values of the unprocessed input image.

Note that the methods 'sad' and 'ssd' compare the gray values of the pixels within a mask window directly, whereas 'ncc' compensates for the mean gray value and its variance within the mask window. Therefore, if the two images differ in brightness and contrast, this method should be preferred. For images with similar brightness and contrast 'sad' and 'ssd' are to be preferred as they are faster because of less complex internal computations.

It should be noted, that the quality of correlation for rising S is falling in methods 'sad' and 'ssd' (the best quality value is 0) but rising in method 'ncc' (the best quality value is 1.0).

The size of the correlation window, referenced by 2m+1 and 2n+1, has to be odd numbered and is passed in MaskWidth and MaskHeight. The search space is confined by the minimum and maximum disparity value MinDisparity and MaxDisparity. Due to pixel values not defined beyond the image border the resulting domain of Disparity and Score is not set along the image border within a margin of height (MaskHeight-1)/2 at the top and bottom border and of width (MaskWidth-1)/2 at the left and right border. For the same reason, the maximum disparity range is reduced at the left and right image border.

Since matching turns out to be highly unreliable when dealing with poorly textured areas, the minimum statistical spread of gray values within the correlation window can be defined in TextureThresh. This threshold is applied on both input images Image1 and Image2. In addition, ScoreThresh guarantees the matching quality and defines the maximum ('sad','ssd') or, respectively, minimum ('ncc') score value of the correlation function. Setting Filter to 'left_right_check', moreover, increases the robustness of the returned matches, as the result relies on a concurrent direct and reverse match, whereas 'none' switches it off.

The number of pyramid levels used to improve the time response of binocular_disparity is determined by NumLevels. Following a coarse-to-fine scheme disparity images of higher levels are computed and segmented into rectangular subimages of similar disparity to reduce the disparity range on the next lower pyramid level. TextureThresh and ScoreThresh are applied on every level and the returned domain of the Disparity and Score images arises from the intersection of the resulting domains of every single level. Generally, pyramid structures are the more advantageous the more the disparity image can be segmented into regions of homogeneous disparities and the bigger the disparity range is specified. As a drawback, coarse pyramid levels might loose important texture information which can result in deficient disparity values.

Finally, the value 'interpolation' for parameter SubDisparity performs subpixel refinement of disparities. It is switched off by setting the parameter to 'none'.

Parameters

Image1 (input_object)	(multichannel-)image → object (byte)
Rectified image of camera 1.

Image2 (input_object)	(multichannel-)image → object (byte)
Rectified image of camera 2.

Disparity (output_object)	singlechannelimage → object (real)
Disparity map.

Score (output_object)	singlechannelimage → object (real)
Evaluation of the disparity values.

Method (input_control)	string → (string)
Matching function.
Default value: 'ncc'
List of values: 'sad', 'ssd', 'ncc'

MaskWidth (input_control)	integer → (integer)
Width of the correlation window.
Default value: 11
Suggested values: 5, 7, 9, 11, 21
Restriction: (3 <= MaskWidth) && odd(MaskWidth)

MaskHeight (input_control)	integer → (integer)
Height of the correlation window.
Default value: 11
Suggested values: 5, 7, 9, 11, 21
Restriction: (3 <= MaskHeight) && odd(MaskHeight)

TextureThresh (input_control)	real → (real / integer)
Variance threshold of textured image regions.
Default value: 0.0
Suggested values: 0.0, 10.0, 30.0
Restriction: 0.0 <= TextureThresh

MinDisparity (input_control)	integer → (integer)
Minimum of the expected disparities.
Default value: -30
Typical range of values: -32768 ≤ MinDisparity ≤ 32767

MaxDisparity (input_control)	integer → (integer)
Maximum of the expected disparities.
Default value: 30
Typical range of values: -32768 ≤ MaxDisparity ≤ 32767

NumLevels (input_control)	integer → (integer)
Number of pyramid levels.
Default value: 1
Suggested values: 1, 2, 3, 4
Restriction: 1 <= NumLevels

ScoreThresh (input_control)	real → (real / integer)
Threshold of the correlation function.
Default value: 0.5
Suggested values: -1.0, 0.0, 0.3, 0.5, 0.7

Filter (input_control)	string(-array) → (string)
Downstream filters.
Default value: 'none'
List of values: 'none', 'left_right_check'

SubDisparity (input_control)	string → (string)
Subpixel interpolation of disparities.
Default value: 'none'
List of values: 'none', 'interpolation'

Example

* read the internal and external stereo parameters
read_cam_par ('cam_left.dat', CamParam1)
read_cam_par ('cam_right.dat', CamParam2)
read_pose ('relpos.dat', RelPose)

* compute the mapping for rectified images
gen_binocular_rectification_map (Map1, Map2, CamParam1, CamParam2, RelPose, \
  'geometric', 'bilinear', CamParamRect1,CamParamRect2, Cam1PoseRect1, \
  Cam2PoseRect2,RelPoseRect)

* compute the disparities in online images
while (1)
  grab_image_async (Image1, FGHandle1, -1)
  map_image (Image1, Map1, ImageMapped1)

  grab_image_async (Image2, FGHandle2, -1)
  map_image (Image2, Map2, ImageMapped2)

  binocular_disparity(ImageMapped1, ImageMapped2, Disparity, Score, 'sad', \
    11, 11, 20, -40, 20, 2, 25, 'left_right_check','interpolation')
endwhile

Result

binocular_disparity returns 2 (H_MSG_TRUE) if all parameter values are correct. If necessary, an exception is raised.

Parallelization Information

binocular_disparity is reentrant and automatically parallelized (on domain level).

Possible Predecessors

map_image

Possible Successors

threshold, disparity_to_distance, disparity_image_to_xyz

Alternatives

binocular_disparity_mg, binocular_distance, binocular_distance_mg

See also

map_image, gen_binocular_rectification_map, binocular_calibration

Module

3D Metrology