gen_binocular_proj_rectificationT_gen_binocular_proj_rectificationGenBinocularProjRectificationGenBinocularProjRectificationgen_binocular_proj_rectification (Operator)

Name

gen_binocular_proj_rectificationT_gen_binocular_proj_rectificationGenBinocularProjRectificationGenBinocularProjRectificationgen_binocular_proj_rectification — Compute the projective rectification of weakly calibrated binocular stereo images.

Signature

Description

A binocular stereo setup is called weakly calibrated if the fundamental matrix, which describes the projective relation between the two images, is known. Rectification is the process of finding a suitable set of transformations, that transform both images such that all corresponding epipolar lines become collinear and parallel to the horizontal axes. The rectified images can be thought of as acquired by a stereo configuration where the left and right image plane are identical and the difference between both image centers is a horizontal translation. Note that rectification can only be performed if both of the epipoles are located outside the images.

Typically, the fundamental matrix is calculated beforehand with match_fundamental_matrix_ransacmatch_fundamental_matrix_ransacMatchFundamentalMatrixRansacMatchFundamentalMatrixRansacMatchFundamentalMatrixRansacmatch_fundamental_matrix_ransac and FMatrixFMatrixFMatrixFMatrixFMatrixfmatrix is the basis for the computation of the two homographies H1H1H1H1h1h1 and H2H2H2H2h2h2, which describe the rectifications for the left image and the right image respectively. Since a projective rectification is an underdetermined problem, additional constraints are defined: the algorithm chooses the set of homographies that minimizes the projective distortion induced by the homographies in both images. For the computation of this cost function the dimensions of the images must be provided in Width1Width1Width1Width1width1width_1, Height1Height1Height1Height1height1height_1, Width2Width2Width2Width2width2width_2, Height2Height2Height2Height2height2height_2. After rectification the fundamental matrix is always of the canonical form

In the case of a known covariance matrix CovFMatCovFMatCovFMatCovFMatcovFMatcov_fmat of the fundamental matrix FMatrixFMatrixFMatrixFMatrixFMatrixfmatrix, the covariance matrix CovFMatRectCovFMatRectCovFMatRectCovFMatRectcovFMatRectcov_fmat_rect of the above rectified fundamental matrix is calculated. This can help for an improved stereo matching process because the covariance matrix defines in terms of probabilities the image domain where to find a corresponding match.

Similar to the operator gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapGenBinocularRectificationMapgen_binocular_rectification_map the output images Map1Map1Map1Map1map1map_1 and Map2Map2Map2Map2map2map_2 describe the transformation, also called mapping, of the original images to the rectified ones. The parameter MappingMappingMappingMappingmappingmapping specifies whether bilinear interpolation ('bilinear_map'"bilinear_map""bilinear_map""bilinear_map""bilinear_map""bilinear_map") should be applied between the pixels in the input image or whether the gray value of the nearest neighboring pixel should be taken ('nn_map'"nn_map""nn_map""nn_map""nn_map""nn_map"). The size and resolution of the maps and of the transformed images can be adjusted by the parameter SubSamplingSubSamplingSubSamplingSubSamplingsubSamplingsub_sampling, which applies a sub-sampling factor to the original images. For example, a factor of two will halve the image sizes. If just the two homographies are required MappingMappingMappingMappingmappingmapping can be set to 'no_map'"no_map""no_map""no_map""no_map""no_map" and no maps will be returned. For speed reasons, this option should be used if for a specific stereo configuration the images must be rectified only once. If the stereo setup is fixed, the maps should be generated only once and both images should be rectified with map_imagemap_imageMapImageMapImageMapImagemap_image; this will result in the smallest computational cost for on-line rectification.

The channels 2 to 5, which are of type uint2, contain the weights of the relevant pixels for the bilinear interpolation.

Based on the rectified images, the disparity be computed using binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparityBinocularDisparitybinocular_disparity. In contrast to stereo with fully calibrated cameras, using the operator gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapGenBinocularRectificationMapgen_binocular_rectification_map and its successors, metric depth information can not be derived for weakly calibrated cameras. The disparity map gives just a qualitative depth ordering of the scene.

Execution Information

• Multithreading type: reentrant (runs in parallel with non-exclusive operators).
• Multithreading scope: global (may be called from any thread).
• Processed without parallelization.

Parameters

Example (HDevelop)

* Rectify an image pair using a map.
get_image_size (Image1, Width1, Height1)
get_image_size (Image2, Width2, Height2)
points_harris (Image1, 3, 1, 0.2, 10000, Row1, Col1)
points_harris (Image2, 3, 1, 0.2, 10000, Row2, Col2)
match_fundamental_matrix_ransac (Image1, Image2, Row1, Col1, Row2, Col2, \
                                 'ncc', 21, 0, 200, 20, 50, 0, 0.9, \
                                 'gold_standard', 0.3, 1, FMatrix, \
                                 CovFMat, Error, Points1, Points2)
gen_binocular_proj_rectification (Map1, Map2, FMatrix, [], Width1, \
                                  Height1, Width2, Height2, 1, \
                                  'bilinear_map', CovFMatRect, H1, H2)
map_image (Image1, Map1, Image1Rect)
map_image (Image2, Map2, Image2Rect)

* Rectify an image pair without using a map.
get_image_size (Image1, Width1, Height1)
get_image_size (Image2, Width2, Height2)
points_harris (Image1, 3, 1, 0.2, 10000, Row1, Col1)
points_harris (Image2, 3, 1, 0.2, 10000, Row2, Col2)
match_fundamental_matrix_ransac (Image1, Image2, Row1, Col1, Row2, Col2, \
                                 'ncc', 21, 0, 200, 20, 50, 0, 0.9, \
                                 'gold_standard', 0.3, 1, FMatrix, \
                                 CovFMat, Error, Points1, Points2)
gen_binocular_proj_rectification (Map1, Map2, FMatrix, [], Width1, \
                                  Height1, Width2, Height2, 1, \
                                  'no_map', CovFMatRect, H1, H2)
* Determine the maximum extent of the two rectified images.
projective_trans_point_2d (H1, [0,0,Height1,Height1], \
                           [0,Width1,0,Width1], [1,1,1,1], R1, C1, W1)
R1 := int(floor(R1/W1))
C1 := int(floor(C1/W1))
projective_trans_point_2d (H2, [0,0,Height2,Height2], \
                           [0,Width2,0,Width2], [1,1,1,1], R2, C2, W2)
R2 := int(floor(R2/W2))
C2 := int(floor(C2/W2))
WidthRect := max([C1,C2])
HeightRect := max([R1,R2])
projective_trans_image_size (Image1, Image1Rect, H1, 'bilinear', \
                             WidthRect, HeightRect, 'false')
projective_trans_image_size (Image2, Image2Rect, H2, 'bilinear', \
                             WidthRect, HeightRect, 'false')

Possible Predecessors

match_fundamental_matrix_ransacmatch_fundamental_matrix_ransacMatchFundamentalMatrixRansacMatchFundamentalMatrixRansacMatchFundamentalMatrixRansacmatch_fundamental_matrix_ransac, vector_to_fundamental_matrixvector_to_fundamental_matrixVectorToFundamentalMatrixVectorToFundamentalMatrixVectorToFundamentalMatrixvector_to_fundamental_matrix

Possible Successors

map_imagemap_imageMapImageMapImageMapImagemap_image, projective_trans_imageprojective_trans_imageProjectiveTransImageProjectiveTransImageProjectiveTransImageprojective_trans_image, binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparityBinocularDisparitybinocular_disparity

Alternatives

gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapGenBinocularRectificationMapgen_binocular_rectification_map

References

J. Gluckmann and S.K. Nayar: “Rectifying transformations that minimize resampling effects”; IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2001, vol I, pages 111-117.

Module

3D Metrology