find_uncalib_descriptor_modelT_find_uncalib_descriptor_modelFindUncalibDescriptorModelfind_uncalib_descriptor_modelFindUncalibDescriptorModelFindUncalibDescriptorModel (Operator)

Name

find_uncalib_descriptor_modelT_find_uncalib_descriptor_modelFindUncalibDescriptorModelfind_uncalib_descriptor_modelFindUncalibDescriptorModelFindUncalibDescriptorModel — Find the best matches of a descriptor model in an image.

Signature

Description

The operator find_uncalib_descriptor_modelfind_uncalib_descriptor_modelFindUncalibDescriptorModelfind_uncalib_descriptor_modelFindUncalibDescriptorModelFindUncalibDescriptorModel finds the best matches of a descriptor model ModelIDModelIDModelIDModelIDModelIDmodelID in ImageImageImageImageImageimage. The descriptor model must have been created previously by calling create_uncalib_descriptor_modelcreate_uncalib_descriptor_modelCreateUncalibDescriptorModelcreate_uncalib_descriptor_modelCreateUncalibDescriptorModelCreateUncalibDescriptorModel, create_calib_descriptor_modelcreate_calib_descriptor_modelCreateCalibDescriptorModelcreate_calib_descriptor_modelCreateCalibDescriptorModelCreateCalibDescriptorModel or read_descriptor_modelread_descriptor_modelReadDescriptorModelread_descriptor_modelReadDescriptorModelReadDescriptorModel.

A match is only accepted if its score exceeds the value of MinScoreMinScoreMinScoreMinScoreMinScoreminScore. This criterion is based on the 'inlier_ratio' score that is described in details below. The main result of the operator find_uncalib_descriptor_modelfind_uncalib_descriptor_modelFindUncalibDescriptorModelfind_uncalib_descriptor_modelFindUncalibDescriptorModelFindUncalibDescriptorModel for each match is a 3x3 matrix HomMat2DHomMat2DHomMat2DHomMat2DHomMat2DhomMat2D, which describes a 2D projection of model points to search image points and is represented by a tuple of 9 elements in row-major order. If more matches of the searched object (template) appear and pass the MinScoreMinScoreMinScoreMinScoreMinScoreminScore criterion, the resulting multiple homographies are concatenated. The number of objects actually found is then equal to NumObjects = |HomMat2DHomMat2DHomMat2DHomMat2DHomMat2DhomMat2D|/9.

The detection process is divided into three parts. First, interest points are extracted from the search image (only inside the domain of the search image). This is done using the point detector and its parameters selected once during the generation of the model. However, DetectorParamNameDetectorParamNameDetectorParamNameDetectorParamNameDetectorParamNamedetectorParamName and DetectorParamValueDetectorParamValueDetectorParamValueDetectorParamValueDetectorParamValuedetectorParamValue can be used to specify different detector parameter values during the find_uncalib_descriptor_modelfind_uncalib_descriptor_modelFindUncalibDescriptorModelfind_uncalib_descriptor_modelFindUncalibDescriptorModelFindUncalibDescriptorModel call. By changing these parameters, it is possible to adjust to illumination changes between the model generation and the online detection. However, it is recommended to use the same values used to create the model (pass an empty tuple).

The second step of the detection process is to calculate correspondences between the model points and the points that were detected. The run time parameters of the descriptor can be adjusted with DescriptorParamNameDescriptorParamNameDescriptorParamNameDescriptorParamNameDescriptorParamNamedescriptorParamName and DescriptorParamValueDescriptorParamValueDescriptorParamValueDescriptorParamValueDescriptorParamValuedescriptorParamValue:

'min_score_descr'"min_score_descr""min_score_descr""min_score_descr""min_score_descr""min_score_descr":

is the minimal classifier score for an interest point to be regarded as a potential match. The score function is between 0.0 and 1.0, but typically only values between 0.0 and 0.1 make sense. Increasing 'min_score_descr'"min_score_descr""min_score_descr""min_score_descr""min_score_descr""min_score_descr" can increase significantly the detection speed. Note, however, that using 'min_score_descr'"min_score_descr""min_score_descr""min_score_descr""min_score_descr""min_score_descr" might have negative effect on the robustness of the detection process, especially when only few points can be found. Typical values are [0.0 .. 0.1], default value is 0.0.

'guided_matching'"guided_matching""guided_matching""guided_matching""guided_matching""guided_matching":

enhances the accuracy of the object recognition when switched on. Note that it increases the computational costs up to 10% in some cases. Possible values are ['on'"on""on""on""on""on", 'off'"off""off""off""off""off"], default value is 'on'"on""on""on""on""on".

The last step is the estimation of a homography that describes the point correspondences. The homography is a 2D projection, which describes a transformation from model points to points in ImageImageImageImageImageimage. Here, Natural 3D Markers (N3Ms) are utilized to identify robustly the point corresondences (see references).

Additionally to the estimated homography in HomMat2DHomMat2DHomMat2DHomMat2DHomMat2DhomMat2D, the operator returns one or more ScoreScoreScoreScoreScorescore estimations per object instance as specified by the user in a tuple ScoreTypeScoreTypeScoreTypeScoreTypeScoreTypescoreType. Currently the following values for ScoreTypeScoreTypeScoreTypeScoreTypeScoreTypescoreType are supported:

'num_points'"num_points""num_points""num_points""num_points""num_points":

number of point correspondences per instance. An object instance should be considered good, if it has 10 or more point correspondences with the model. Fewer points are insufficient, because any random 4 point correspondences define a mathematically correct homography between two images.

'inlier_ratio'"inlier_ratio""inlier_ratio""inlier_ratio""inlier_ratio""inlier_ratio":

the ratio of the number of point correspondences to the number of model points. Although this value can have values of [0.0 .. 1.0], it is rather unlikely that this ratio can reach 1.0. Yet, objects having inlier ratio less than 0.1, should be disregarded.

Note that the resulting scores for more than one object instance will be concatenated in ScoreScoreScoreScoreScorescore, such that |ScoreScoreScoreScoreScorescore| = NumObjects*|ScoreTypeScoreTypeScoreTypeScoreTypeScoreTypescoreType|.

The point correspondences for each object can be queried with get_descriptor_model_pointsget_descriptor_model_pointsGetDescriptorModelPointsget_descriptor_model_pointsGetDescriptorModelPointsGetDescriptorModelPoints.

Attention

A descriptor model 'ModelID'"ModelID""ModelID""ModelID""ModelID""ModelID" cannot be shared between two or more user's threads. Different descriptor models can be used independently and safely in different threads.

Note that the domain of the search image should contain the whole object to be searched for because interest points are only extracted inside the domain of the search image. This means that if the domain does not contain the full object to be searched for, the resulting ScoreScoreScoreScoreScorescore will decrease. Note also that matches may be found even if the reference point (origin) of the model lies outside of the domain of the search image. Both is in contrast to shape-based matching, where the domain of the search image defines the search space for the reference point of the model.

Parallelization

• Multithreading type: reentrant (runs in parallel with non-exclusive operators).
• Multithreading scope: global (may be called from any thread).
• Processed without parallelization.

Parameters

Example (HDevelop)

create_uncalib_descriptor_model (ImageReduced,'harris',[],[], \
                                 [],[],42,ModelID)

get_descriptor_model_params (ModelID,DetectorType, \
                             DetectorParamName,DetectorParamValue, \
                             DescriptorParamName,DescriptorParamValue)
write_descriptor_model (ModelID,'simple_example.dsm')
clear_descriptor_model (ModelID)

read_descriptor_model ('simple_example.dsm',ModelID)
find_uncalib_descriptor_model (SearchImage,ModelID,[],[],[],[],0.2,1, \
                               ['num_points','inlier_ratio'],HomMat2D,Score)
clear_descriptor_model (ModelID)

Possible Predecessors

create_uncalib_descriptor_modelcreate_uncalib_descriptor_modelCreateUncalibDescriptorModelcreate_uncalib_descriptor_modelCreateUncalibDescriptorModelCreateUncalibDescriptorModel, create_calib_descriptor_modelcreate_calib_descriptor_modelCreateCalibDescriptorModelcreate_calib_descriptor_modelCreateCalibDescriptorModelCreateCalibDescriptorModel, read_descriptor_modelread_descriptor_modelReadDescriptorModelread_descriptor_modelReadDescriptorModelReadDescriptorModel

See also

create_uncalib_descriptor_modelcreate_uncalib_descriptor_modelCreateUncalibDescriptorModelcreate_uncalib_descriptor_modelCreateUncalibDescriptorModelCreateUncalibDescriptorModel, create_calib_descriptor_modelcreate_calib_descriptor_modelCreateCalibDescriptorModelcreate_calib_descriptor_modelCreateCalibDescriptorModelCreateCalibDescriptorModel, find_calib_descriptor_modelfind_calib_descriptor_modelFindCalibDescriptorModelfind_calib_descriptor_modelFindCalibDescriptorModelFindCalibDescriptorModel, get_descriptor_model_pointsget_descriptor_model_pointsGetDescriptorModelPointsget_descriptor_model_pointsGetDescriptorModelPointsGetDescriptorModelPoints

References

S. Hinterstoisser, S. Benhimane, and N. Navab: “N3M: Natural 3D Markers for Real-Time Object Detection and Pose Estimation.“ IEEE 11th International Conference on Computer Vision, 2007. pp. 1-7, ICCV 2007.

Module

Matching