gen_binocular_proj_rectificationT_gen_binocular_proj_rectificationGenBinocularProjRectificationGenBinocularProjRectificationgen_binocular_proj_rectification (Operator)

Name

gen_binocular_proj_rectificationT_gen_binocular_proj_rectificationGenBinocularProjRectificationGenBinocularProjRectificationgen_binocular_proj_rectification — Compute the projective rectification of weakly calibrated binocular stereo images.

Signature

Description

A binocular stereo setup is called weakly calibrated if the fundamental matrix, which describes the projective relation between the two images, is known. Rectification is the process of finding a suitable set of transformations, that transform both images such that all corresponding epipolar lines become collinear and parallel to the horizontal axes. The rectified images can be thought of as acquired by a stereo configuration where the left and right image plane are identical and the difference between both image centers is a horizontal translation. Note that rectification can only be performed if both of the epipoles are located outside the images.

Typically, the fundamental matrix is calculated beforehand with match_fundamental_matrix_ransacmatch_fundamental_matrix_ransacMatchFundamentalMatrixRansacMatchFundamentalMatrixRansacmatch_fundamental_matrix_ransac and FMatrixFMatrixFMatrixFMatrixfmatrix is the basis for the computation of the two homographies H1H1H1h1h1 and H2H2H2h2h2, which describe the rectifications for the left image and the right image respectively. Since a projective rectification is an underdetermined problem, additional constraints are defined: the algorithm chooses the set of homographies that minimizes the projective distortion induced by the homographies in both images. For the computation of this cost function the dimensions of the images must be provided in Width1Width1Width1width1width_1, Height1Height1Height1height1height_1, Width2Width2Width2width2width_2, Height2Height2Height2height2height_2. After rectification the fundamental matrix is always of the canonical form

In the case of a known covariance matrix CovFMatCovFMatCovFMatcovFMatcov_fmat of the fundamental matrix FMatrixFMatrixFMatrixFMatrixfmatrix, the covariance matrix CovFMatRectCovFMatRectCovFMatRectcovFMatRectcov_fmat_rect of the above rectified fundamental matrix is calculated. This can help for an improved stereo matching process because the covariance matrix defines in terms of probabilities the image domain where to find a corresponding match.

Similar to the operator gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapgen_binocular_rectification_map the output images Map1Map1Map1map1map_1 and Map2Map2Map2map2map_2 describe the transformation, also called mapping, of the original images to the rectified ones. The parameter MappingMappingMappingmappingmapping specifies whether bilinear interpolation ('bilinear_map'"bilinear_map""bilinear_map""bilinear_map""bilinear_map") should be applied between the pixels in the input image or whether the gray value of the nearest neighboring pixel should be taken ('nn_map'"nn_map""nn_map""nn_map""nn_map"). The size and resolution of the maps and of the transformed images can be adjusted by the parameter SubSamplingSubSamplingSubSamplingsubSamplingsub_sampling, which applies a sub-sampling factor to the original images. For example, a factor of two will halve the image sizes. If just the two homographies are required MappingMappingMappingmappingmapping can be set to 'no_map'"no_map""no_map""no_map""no_map" and no maps will be returned. For speed reasons, this option should be used if for a specific stereo configuration the images must be rectified only once. If the stereo setup is fixed, the maps should be generated only once and both images should be rectified with map_imagemap_imageMapImageMapImagemap_image; this will result in the smallest computational cost for on-line rectification.

When using the maps, the transformed images are of the same size as their maps. Each pixel in the map contains the description of how the new pixel at this position is generated. The images Map1Map1Map1map1map_1 and Map2Map2Map2map2map_2 are single channel images if MappingMappingMappingmappingmapping is set to 'nn_map'"nn_map""nn_map""nn_map""nn_map" and five channel images if it is set to 'bilinear_map'"bilinear_map""bilinear_map""bilinear_map""bilinear_map". In the first channel, which is of type int4, the pixels contain the linear coordinates of their reference pixels in the original image. With MappingMappingMappingmappingmapping equal to 'no_map'"no_map""no_map""no_map""no_map" this reference pixel is the nearest neighbor to the back-transformed pixel coordinates of the map. In the case of bilinear interpolation the reference pixel is the next upper left pixel relative to the back-transformed coordinates. The following scheme shows the ordering of the pixels in the original image next to the back-transformed pixel coordinates, where the reference pixel takes the number 2.

The channels 2 to 5, which are of type uint2, contain the weights of the relevant pixels for the bilinear interpolation.

Based on the rectified images, the disparity be computed using binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparitybinocular_disparity. In contrast to stereo with fully calibrated cameras, using the operator gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapgen_binocular_rectification_map and its successors, metric depth information can not be derived for weakly calibrated cameras. The disparity map gives just a qualitative depth ordering of the scene.

Execution Information

• Multithreading type: reentrant (runs in parallel with non-exclusive operators).
• Multithreading scope: global (may be called from any thread).
• Processed without parallelization.

Parameters

Example (HDevelop)

* Rectify an image pair using a map.
get_image_size (Image1, Width1, Height1)
get_image_size (Image2, Width2, Height2)
points_harris (Image1, 3, 1, 0.2, 10000, Row1, Col1)
points_harris (Image2, 3, 1, 0.2, 10000, Row2, Col2)
match_fundamental_matrix_ransac (Image1, Image2, Row1, Col1, Row2, Col2, \
                                 'ncc', 21, 0, 200, 20, 50, 0, 0.9, \
                                 'gold_standard', 0.3, 1, FMatrix, \
                                 CovFMat, Error, Points1, Points2)
gen_binocular_proj_rectification (Map1, Map2, FMatrix, [], Width1, \
                                  Height1, Width2, Height2, 1, \
                                  'bilinear_map', CovFMatRect, H1, H2)
map_image (Image1, Map1, Image1Rect)
map_image (Image2, Map2, Image2Rect)

* Rectify an image pair without using a map.
get_image_size (Image1, Width1, Height1)
get_image_size (Image2, Width2, Height2)
points_harris (Image1, 3, 1, 0.2, 10000, Row1, Col1)
points_harris (Image2, 3, 1, 0.2, 10000, Row2, Col2)
match_fundamental_matrix_ransac (Image1, Image2, Row1, Col1, Row2, Col2, \
                                 'ncc', 21, 0, 200, 20, 50, 0, 0.9, \
                                 'gold_standard', 0.3, 1, FMatrix, \
                                 CovFMat, Error, Points1, Points2)
gen_binocular_proj_rectification (Map1, Map2, FMatrix, [], Width1, \
                                  Height1, Width2, Height2, 1, \
                                  'no_map', CovFMatRect, H1, H2)
* Determine the maximum extent of the two rectified images.
projective_trans_point_2d (H1, [0,0,Height1,Height1], \
                           [0,Width1,0,Width1], [1,1,1,1], R1, C1, W1)
R1 := int(floor(R1/W1))
C1 := int(floor(C1/W1))
projective_trans_point_2d (H2, [0,0,Height2,Height2], \
                           [0,Width2,0,Width2], [1,1,1,1], R2, C2, W2)
R2 := int(floor(R2/W2))
C2 := int(floor(C2/W2))
WidthRect := max([C1,C2])
HeightRect := max([R1,R2])
projective_trans_image_size (Image1, Image1Rect, H1, 'bilinear', \
                             WidthRect, HeightRect, 'false')
projective_trans_image_size (Image2, Image2Rect, H2, 'bilinear', \
                             WidthRect, HeightRect, 'false')

Possible Predecessors

match_fundamental_matrix_ransacmatch_fundamental_matrix_ransacMatchFundamentalMatrixRansacMatchFundamentalMatrixRansacmatch_fundamental_matrix_ransac, vector_to_fundamental_matrixvector_to_fundamental_matrixVectorToFundamentalMatrixVectorToFundamentalMatrixvector_to_fundamental_matrix

Possible Successors

map_imagemap_imageMapImageMapImagemap_image, projective_trans_imageprojective_trans_imageProjectiveTransImageProjectiveTransImageprojective_trans_image, binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparitybinocular_disparity

Alternatives

gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapgen_binocular_rectification_map

References

J. Gluckmann and S.K. Nayar: “Rectifying transformations that minimize resampling effects”; IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2001, vol I, pages 111-117.

Module

3D Metrology

---