| Table of Contents / 3D Matching / Surface-Based | Operators |
find_surface_model — Find the best matches of a surface model in a 3D scene.
find_surface_model( : : SurfaceModelID, ObjectModel3DID, RelSamplingDistance, KeyPointFraction, MinScore, ReturnResultHandle, GenParamName, GenParamValue : Pose, Score, SurfaceMatchingResultID)
The operator find_surface_model finds the best matches of the surface model SurfaceModelID in the 3D scene ObjectModel3DID. The surface model SurfaceModelID must have been created previously with create_surface_model or read_surface_model.
The matching of the surface model requires that the 3D object model contains points and normals. The following combinations are possible:
points and point normals;
points and a 2D-Mapping, e.g. an XYZ image triple converted with xyz_to_object_model_3d.
The 3D pose of the found object instance is returned in Pose. It describes the pose of the surface model in the scene coordinate system. The pose refers to the original coordinate system of the 3D object model that was passed to create_surface_model. Additionally, for each match a score is returned in Score. The value and interpretation of the score depends on whether pose refinement is enabled or not. If pose refinement is disabled, the score is the approximate number of points from the subsampled scene that lie on the found object. If pose refinement is enabled, the approximate fraction of the object's surface that is visible in the scene is returned, as value between 0 and 1. Both score types are described in more detail below.
The parameter MinScore can be used to filter the results. Only matches with a score exceeding the value of MinScore are returned. If MinScore is set to zero, all matches are returned.
If ReturnResultHandle is set to 'true', a surface matching result handle is returned in SurfaceMatchingResultID. Additional details of the matching process can be queried with the operator get_surface_matching_result using that handle. The returned handle must be freed by using clear_surface_matching_result to avoid memory leaks.
The parameters GenParamName and GenParamValue are used to set generic parameters. Both get a tuple of equal length, where the tuple passed to GenParamName contains the names of the parameters to set, and the tuple passed to GenParamValue contains the corresponding values. The possible parameter names and values are described below.
The matching is divided into three steps:
Approximate matching: The approximate poses of the instances of the surface model in the scene are searched.
First, points are sampled uniformly from the scene passed in ObjectModel3DID. The sampling distance is controlled with the parameter RelSamplingDistance, and is given relative to the diameter of the surface model. Decreasing RelSamplingDistance leads to more sampled points, and in turn to a more stable but slower matching. Increasing RelSamplingDistance reduces the number of sampled scene points, which leads to a less stable but faster matching. The operator get_surface_matching_result can be used to retrieve the sampled scene points for visual inspection. For a robust matching it is recommended that at least 50-100 scene points are sampled for each object instance.
A set of key points is selected from the sampled scene points. The number of selected key points is controlled with the parameter KeyPointFraction. For example, if the value is set to 0.1, 10% of the sampled scene points are used as key points. For stable results it is important that each instance of the object is covered by several key points. Increasing KeyPointFraction means that more key points are selected from the scene, resulting in a slower but more stable matching. Decreasing KeyPointFraction has the inverse effect and results in a faster but less stable matching. The operator get_surface_matching_result can be used to retrieve the selected key points for visual inspection.
For each selected key point, the optimum pose of the surface model is computed under the assumption that the key point lies on the surface of the object. This is done by pairing the key point with all other sampled scene points and finding the point pairs on the surface model that have a similar distance and relative orientation. The similarity is defined by the parameters 'feat_step_size_rel' and 'feat_angle_resolution' in create_surface_model. The pose for which the largest number of points from the sampled scene lie on the object is considered to be the best pose for this key point. The number of sampled scene points on the object is considered to be the score of the pose.
From all key points the poses with the best scores are then selected and used as approximate poses. The maximum number of returned poses is set with the generic parameter 'num_matches'. If the pose refinement is disabled, the score described above is returned for each pose in Score. The value of the score depends on the amount of surface of the instance that is visible in the scene and on the sampling rate of the scene. Only poses whose score exceeds MinScore are returned. To determine a good threshold for MinScore, it is recommended to test the matching on several scenes.
Note that the resulting poses from this step are only approximate. The error in the pose is proportional to the sampling rates of the surface model given in create_surface_model, and is typically less than 5% of the object's diameter.
The following generic parameters control the approximate matching and can be set with GenParamName and GenParamValue:
Sets the maximum number of matches that are returned.
Suggested values: 1, 2, 5
Default value: 1
Assertion: 'num_matches' > 0
Sets the threshold on the difference in translation of two poses, such that the poses are considered to be 'similar'. The threshold is set relative to the diameter of the object. Once a pose with a high score is found, all other poses that are similar under this definition are ignored. The parameter such defines by how much two matches may overlap. Only one of the parameters 'max_overlap_dist_rel' and 'max_overlap_dist_abs' should be set. If both are set, only the value of the last parameter is used.
Suggested values: 0.1, 0.5, 1
Default value: 0.5
Assertion: 'max_overlap_dist_rel' >= 0
Sets the threshold on the difference in translation of two poses, such that the poses are considered to be 'similar'. The threshold is set as absolute value. For a description of the threshold, see 'max_overlap_dist_rel' above. Only one of the parameters 'max_overlap_dist_rel' and 'max_overlap_dist_abs' should be set. If both are set, only the value of the last parameter is used.
Suggested values: 1, 2, 3
Assertion: 'max_overlap_dist_abs' >= 0
Sparse pose refinement: In this second step, the approximate poses found in the previous step are further refined. This increases the accuracy of the poses and the significance of the score value.
The sparse pose refinement uses the sampled scene points from the approximate matching. The pose is optimized such that the distances from the sampled scene points to the plane of the closest model point are minimal. The plane of each model point is defined as the plane perpendicular to its normal.
The sparse pose refinement is enabled by default. It can be disabled by setting the generic parameter 'sparse_pose_refinement' to 'false'. Since each key point produces one pose candidate, the total number of pose candidates to be optimized is proportional to the number of key points. For large scenes with much clutter, i.e., scene parts that do not belong to the object of interest, it can be faster to disable the sparse pose refinement.
The score of each pose is recomputed after the sparse pose refinement by counting the number of model points that have a correspondent scene point, and dividing it by the total number of points on the model. This results in a value between 0 and 1 that is approximately the fraction of the model surface that is visible in the scene (see below). This score is more accurate and stable than the score coming from the approximate matching. It still depends on the sampling distance of the scene set in RelSamplingDistance.
The following generic parameters control the sparse pose refinement and can be set with GenParamName and GenParamValue:
Enables or disables the sparse pose refinement.
Value list: 'true', 'false'
Default value: 'true'
Set the type of the score that is returned. Several different scores can be computed and returned after the pose refinement. This parameter has no effect if both the sparse and the dense pose refinement are disabled.
Note that for the computation of the score after the sparse pose refinement, the sampled scene points are used. For the computation of the score after the dense pose refinement, all scene points are used (see below). The score value after the sparse pose refinement therefore depends on the sampling distance of the scene.
The following score types are supported:
Count the number of sampled scene points that lie on the surface of the found object. This score is more accurate and stable than the score coming from the approximate matching. It depends on the sampling distance of the scene set in RelSamplingDistance. The returned score will be between zero and the number of points in the sampled scene.
Count the number of sampled model points that were detected in the scene. A model point is defined to be 'detected' if there is a scene point close to it. The returned score will be between zero and the number of points in the sampled model.
Compute the fraction of visible model points by dividing the score value from 'num_model_points' by the number of model points. The returned score will be between zero and one, and is approximately the fraction of the model surface that is visible in the scene. This is the default score type.
Value list: 'model_point_fraction', 'num_scene_points', 'num_model_points'
Default value: 'model_point_fraction'
Dense pose refinement: Accurately refines the poses found in the previous steps. This step works similar to the sparse pose refinement and minimizes the distances between the scene points and the planes of the closest model points. The difference is that
only the 'num_matches' poses with the best scores from the previous step are refined;
all points from the scene passed in ObjectModel3DID are used for the refinement.
Taking all points from the scene increases the accuracy of the refinement but is slower than refining on the subsampled scene points. The dense pose refinement is enabled by default, but can be disabled with the generic parameter 'dense_pose_refinement'.
After the dense pose refinement, the score of each match is recomputed. The threshold for considering a point to be 'on' the object is set with the generic parameter 'pose_ref_scoring_dist_rel' or 'pose_ref_scoring_dist_abs' (see below).
The following generic parameters influence the accuracy and speed of the dense pose refinement and can be set with GenParamName and GenParamValue:
Enables or disables the dense pose refinement.
Value list: 'true', 'false'
Default value: 'true'
Number of iterations for the dense pose refinement. Increasing the number of iteration leads to a more accurate pose at the expense of runtime. However, once convergence is reached, the accuracy can no longer be increased, even if the number of steps is increased. Note that this parameter is ignored if the dense pose refinement is disabled.
Suggested values: 1, 3, 5, 20
Default value: 5
Assertion: 'pose_ref_num_steps' > 0
Set the rate of scene points to be used for the dense pose refinement. For example, if this value is set to 5, every 5th point from the scene is used for pose refinement. This parameter allows an easy tradeoff between speed and accuracy of the pose refinement: Increasing the value leads to less points being used and in turn to a faster but less accurate pose refinement. Decreasing the value has the inverse effect. Note that this parameter is ignored if the dense pose refinement is disabled.
Suggested values: 1, 2, 5, 10
Default value: 2
Assertion: 'pose_ref_sub_sampling' > 0
Set the distance threshold for dense pose refinement relative to the diameter of the surface model. Only scene points that are closer to the object than this distance are used for the optimization. Scene points further away are ignored. Note that this parameter is ignored if the dense pose refinement is disabled.
Suggested values: 0.03, 0.05, 0.1, 0.2
Default value: 0.1
Assertion: 0 < 'pose_ref_dist_threshold_rel'
Set the distance threshold for scoring relative to the diameter of the surface model. See the following 'pose_ref_scoring_dist_abs' for a detailed description. Only one of the parameters 'pose_ref_scoring_dist_rel' and 'pose_ref_scoring_dist_abs' can be set. If both are set, only the value of the last parameter is used. Note that this parameter is ignored if the dense pose refinement is disabled.
Suggested values: 0.2, 0.01, 0.005, 0.0001
Default value: 0.005
Assertion: 0 < 'pose_ref_scoring_dist_rel'
Set the distance threshold for scoring. Only scene points that are closer to the object than this distance are considered to be 'on the model' when computing the score after the pose refinement. All other scene points are considered not to be on the model. The value should correspond to the amount of noise on the coordinates of the scene points. Note that this parameter is ignored if the dense pose refinement is disabled. Only one of the parameters 'pose_ref_scoring_dist_rel' and 'pose_ref_scoring_dist_abs' can be set. If both are set, only the value of the last parameter is used.
Set the type of the score that is returned. Several different scores can be computed and returned after the pose refinement. This parameter has no effect if both the sparse and the dense pose refinement are disabled.
For all score types, 'pose_ref_scoring_dist_rel' or 'pose_ref_scoring_dist_abs' controls how close a scene point needs to be to a model point in order to classify it as being on the model.
Note that for the computation of the score after the sparse pose refinement, the sampled scene points are used (see above). For the computation of the score after the dense pose refinement, all scene points are used (see below). The score value after the dense pose refinement does therefore not depend on the sampling distance of the scene.
The following score types are supported:
Count the number of sampled scene points that lie on the surface of the found object. This pose is more accurate and stable than the score coming from the approximate matching. It depends on the sampling distance of the scene set in RelSamplingDistance. The returned score will be between zero and the number of points in the sampled scene.
Count the number of sampled model points that were detected in the scene. A model point is defined to be 'detected' if there is a scene point close to it. The returned score will be between zero and the number of points in the sampled model.
Compute the fraction of visible model points by dividing the score value from 'num_model_points' by the number of model points. The returned score will be between zero and one, and is approximately the fraction of the model surface that is visible in the scene. This is the default score type.
Value list: 'model_point_fraction', 'num_scene_points', 'num_model_points'
Default value: 'model_point_fraction'
Handle of the surface model.
Handle of the 3D object model containing the scene.
Scene sampling distance relative to the diameter of the surface model.
Default value: 0.07
Suggested values: 0.1, 0.07, 0.05, 0.04, 0.03
Restriction: (0 < RelSamplingDistance) < 1
Fraction of sampled scene points used as key points.
Default value: 0.2
Suggested values: 0.3, 0.2, 0.1, 0.05
Restriction: (0 < KeyPointFraction) <= 1
Minimum score of the returned poses.
Default value: 0
Restriction: MinScore >= 0
Enable returning a result handle in SurfaceMatchingResultID.
Default value: 'false'
Suggested values: 'true', 'false'
Names of the generic parameters.
Default value: []
List of values: 'num_matches', 'max_overlap_dist_rel', 'max_overlap_dist_abs', 'sparse_pose_refinement', 'dense_pose_refinement', 'pose_ref_num_steps', 'pose_ref_sub_sampling', 'pose_ref_dist_threshold_rel', 'pose_ref_scoring_dist_rel', 'pose_ref_scoring_dist_abs', 'score_type'
Values of the generic parameters.
Default value: []
Suggested values: 0, 1, 'true', 'false', 0.005, 0.01, 0.03, 0.05, 0.1, 'num_scene_points', 'model_point_fraction', 'num_model_points'
3D pose of the surface model in the scene.
Score of the found instances of the surface model.
Handle of the matching result, if enabled in ReturnResultHandle.
find_surface_model returns 2 (H_MSG_TRUE) if all parameters are correct. If necessary, an exception is raised.
read_object_model_3d, xyz_to_object_model_3d, get_object_model_3d_params, read_surface_model, create_surface_model, get_surface_model_param
refine_surface_model_pose, get_surface_matching_result, clear_surface_matching_result, clear_object_model_3d
3D Metrology
| Table of Contents / 3D Matching / Surface-Based | Operators |
| HALCON Reference Manual 10.0.2 | Copyright © 1996-2011 MVTec Software GmbH |