ClassesClassesClassesClasses | | | | Operators

get_rectangle_poseT_get_rectangle_poseGetRectanglePoseget_rectangle_poseGetRectanglePoseGetRectanglePose (Operator)


get_rectangle_poseT_get_rectangle_poseGetRectanglePoseget_rectangle_poseGetRectanglePoseGetRectanglePose — Determine the 3D pose of a rectangle from its perspective 2D projection


get_rectangle_pose(Contour : : CameraParam, Width, Height, WeightingMode, ClippingFactor : Pose, CovPose, Error)

Herror T_get_rectangle_pose(const Hobject Contour, const Htuple CameraParam, const Htuple Width, const Htuple Height, const Htuple WeightingMode, const Htuple ClippingFactor, Htuple* Pose, Htuple* CovPose, Htuple* Error)

Herror get_rectangle_pose(Hobject Contour, const HTuple& CameraParam, const HTuple& Width, const HTuple& Height, const HTuple& WeightingMode, const HTuple& ClippingFactor, HTuple* Pose, HTuple* CovPose, HTuple* Error)

HTuple HXLD::GetRectanglePose(const HTuple& CameraParam, const HTuple& Width, const HTuple& Height, const HTuple& WeightingMode, const HTuple& ClippingFactor, HTuple* CovPose, HTuple* Error) const

HTuple HXLDArray::GetRectanglePose(const HTuple& CameraParam, const HTuple& Width, const HTuple& Height, const HTuple& WeightingMode, const HTuple& ClippingFactor, HTuple* CovPose, HTuple* Error) const

void GetRectanglePose(const HObject& Contour, const HTuple& CameraParam, const HTuple& Width, const HTuple& Height, const HTuple& WeightingMode, const HTuple& ClippingFactor, HTuple* Pose, HTuple* CovPose, HTuple* Error)

HPose HXLD::GetRectanglePose(const HTuple& CameraParam, const HTuple& Width, const HTuple& Height, const HString& WeightingMode, double ClippingFactor, HTuple* CovPose, HTuple* Error) const

HPose HXLD::GetRectanglePose(const HTuple& CameraParam, double Width, double Height, const HString& WeightingMode, double ClippingFactor, HTuple* CovPose, HTuple* Error) const

HPose HXLD::GetRectanglePose(const HTuple& CameraParam, double Width, double Height, const char* WeightingMode, double ClippingFactor, HTuple* CovPose, HTuple* Error) const

void HOperatorSetX.GetRectanglePose(
[in] IHUntypedObjectX* Contour, [in] VARIANT CameraParam, [in] VARIANT Width, [in] VARIANT Height, [in] VARIANT WeightingMode, [in] VARIANT ClippingFactor, [out] VARIANT* Pose, [out] VARIANT* CovPose, [out] VARIANT* Error)

VARIANT HXLDX.GetRectanglePose(
[in] VARIANT CameraParam, [in] VARIANT Width, [in] VARIANT Height, [in] BSTR WeightingMode, [in] double ClippingFactor, [out] VARIANT* CovPose, [out] VARIANT* Error)

VARIANT HXLDContX.GetRectanglePose(
[in] VARIANT CameraParam, [in] VARIANT Width, [in] VARIANT Height, [in] BSTR WeightingMode, [in] double ClippingFactor, [out] VARIANT* CovPose, [out] VARIANT* Error)

VARIANT HXLDPolyX.GetRectanglePose(
[in] VARIANT CameraParam, [in] VARIANT Width, [in] VARIANT Height, [in] BSTR WeightingMode, [in] double ClippingFactor, [out] VARIANT* CovPose, [out] VARIANT* Error)

VARIANT HXLDParaX.GetRectanglePose(
[in] VARIANT CameraParam, [in] VARIANT Width, [in] VARIANT Height, [in] BSTR WeightingMode, [in] double ClippingFactor, [out] VARIANT* CovPose, [out] VARIANT* Error)

VARIANT HXLDModParaX.GetRectanglePose(
[in] VARIANT CameraParam, [in] VARIANT Width, [in] VARIANT Height, [in] BSTR WeightingMode, [in] double ClippingFactor, [out] VARIANT* CovPose, [out] VARIANT* Error)

VARIANT HXLDExtParaX.GetRectanglePose(
[in] VARIANT CameraParam, [in] VARIANT Width, [in] VARIANT Height, [in] BSTR WeightingMode, [in] double ClippingFactor, [out] VARIANT* CovPose, [out] VARIANT* Error)

static void HOperatorSet.GetRectanglePose(HObject contour, HTuple cameraParam, HTuple width, HTuple height, HTuple weightingMode, HTuple clippingFactor, out HTuple pose, out HTuple covPose, out HTuple error)

HPose HXLD.GetRectanglePose(HTuple cameraParam, HTuple width, HTuple height, string weightingMode, double clippingFactor, out HTuple covPose, out HTuple error)

HPose HXLD.GetRectanglePose(HTuple cameraParam, double width, double height, string weightingMode, double clippingFactor, out HTuple covPose, out HTuple error)


A rectangle in space is projected as a general quadrangle into the image. get_rectangle_poseget_rectangle_poseGetRectanglePoseget_rectangle_poseGetRectanglePoseGetRectanglePose determines the PosePosePosePosePosepose of the rectangle from this projection (ContourContourContourContourContourcontour).

The algorithm works as follows: First, ContourContourContourContourContourcontour is segmented into four line segments and their intersections are considered as corners of the contour. The corners together with the internal camera parameters (CameraParamCameraParamCameraParamCameraParamCameraParamcameraParam) and the rectangle size in meters (WidthWidthWidthWidthWidthwidth, HeightHeightHeightHeightHeightheight) are used for an initial estimation of the rectangle pose. Then, the final PosePosePosePosePosepose is refined with a non-linear optimization by minimizing the geometrical distance of the contour points from the back projection of the rectangle in the image.

The operator supports only area-scan pinhole (projective) cameras. An error is returned if CameraParamCameraParamCameraParamCameraParamCameraParamcameraParam specifies a line-scan or a telecentric camera (see also calibrate_camerascalibrate_camerasCalibrateCamerascalibrate_camerasCalibrateCamerasCalibrateCameras).

WidthWidthWidthWidthWidthwidth and HeightHeightHeightHeightHeightheight specify the size of the rectangle in x and y dimensions, respectively, in its coordinate system. The origin of this coordinate system is in the center of the rectangle. The z axis points away from the camera.

The arguments WeightingModeWeightingModeWeightingModeWeightingModeWeightingModeweightingMode and ClippingFactorClippingFactorClippingFactorClippingFactorClippingFactorclippingFactor can be used to damp the impact of outliers on the algorithm. If WeightingModeWeightingModeWeightingModeWeightingModeWeightingModeweightingMode is set to 'tukey' or 'huber', the contour points are weighted based on the approach of Tukey or Huber respectively. In such a case a robust error statistics is used to estimate the standard deviation of the distances of the contour points from the backprojected rectangle excluding outliers. The parameter ClippingFactorClippingFactorClippingFactorClippingFactorClippingFactorclippingFactor (a scaling factor for the standard deviation) controls the amount of damping outliers: The smaller the value chosen for ClippingFactorClippingFactorClippingFactorClippingFactorClippingFactorclippingFactor the more outliers are detected. See a discussion about the properties of the different weighting modes in fit_line_contour_xldfit_line_contour_xldFitLineContourXldfit_line_contour_xldFitLineContourXldFitLineContourXld. Note that, unlike by fit_line_contour_xldfit_line_contour_xldFitLineContourXldfit_line_contour_xldFitLineContourXldFitLineContourXld, for the rectangle pose estimation the approach of Huber is recommended.


The resulting PosePosePosePosePosepose is of code-0 (see create_posecreate_poseCreatePosecreate_poseCreatePoseCreatePose) and represents the pose of the center of the rectangle. You can compute the pose of the corners of the rectangle as follows:

  set_origin_pose (PosePosePosePosePosepose,  WidthWidthWidthWidthWidthwidth/2, -HeightHeightHeightHeightHeightheight/2, 0, PoseCorner1)
  set_origin_pose (PosePosePosePosePosepose,  WidthWidthWidthWidthWidthwidth/2,  HeightHeightHeightHeightHeightheight/2, 0, PoseCorner2)
  set_origin_pose (PosePosePosePosePosepose, -WidthWidthWidthWidthWidthwidth/2,  HeightHeightHeightHeightHeightheight/2, 0, PoseCorner3)
  set_origin_pose (PosePosePosePosePosepose, -WidthWidthWidthWidthWidthwidth/2, -HeightHeightHeightHeightHeightheight/2, 0, PoseCorner4)

A rectangle is symmetric with respect to its x, y, and z axis and one and the same contour can represent a rectangle in 4 different poses. The angles in PosePosePosePosePosepose are normalized to be in the range [-90; 90] degrees and the rest of the 4 possible poses can be computed by combining flips around the corresponding axis:

  * NOTE: the following code works ONLY for pose of code-0
  *       as it is returned by get_rectangle_poseget_rectangle_poseGetRectanglePoseget_rectangle_poseGetRectanglePoseGetRectanglePose
  * flip around z-axis
  PoseFlippedZ := PosePosePosePosePosepose
  PoseFlippedZ[5] :=  PoseFlippedZ[5]+180
  * flip around y-axis
  PoseFlippedY := PosePosePosePosePosepose
  PoseFlippedY[4] :=  PoseFlippedY[4]+180
  PoseFlippedY[5] := -PoseFlippedY[5]
  * flip around x-axis
  PoseFlippedX := PosePosePosePosePosepose
  PoseFlippedX[3] :=  PoseFlippedX[3]+180
  PoseFlippedX[4] := -PoseFlippedX[4]
  PoseFlippedX[5] := -PoseFlippedX[5]

Note that if the rectangle is a square (WidthWidthWidthWidthWidthwidth == HeightHeightHeightHeightHeightheight) the number of alternative poses is 8.

If more than one contour are given in ContourContourContourContourContourcontour, a corresponding tuple of values for both WidthWidthWidthWidthWidthwidth and HeightHeightHeightHeightHeightheight has to be provided as well. Yet, if only one value is provided for each of these arguments, then this value is applied for each processed contour. A pose is estimated for each processed contour and all poses are concatenated in PosePosePosePosePosepose (see the example below).

Accuracy of the pose

The accuracy of the estimated pose depends on the following three factors:

In order to achieve an accurate pose estimation, there are three corresponding criteria that should be considered:

The ratio WidthWidthWidthWidthWidthwidth/HeightHeightHeightHeightHeightheight should fulfill

        1/3 < WidthWidthWidthWidthWidthwidth/HeightHeightHeightHeightHeightheight < 3

For a rectangular object deviating from this criterion, its longer side dominates the determination of its pose. This causes instability in the estimation of the angle around the longer rectangle's axis. In the extreme case when one of the dimensions is 0, the rectangle is in fact a line segment, whose pose cannot be estimated.

Secondly, the lengths of each side of the contour should be at least 20 pixels. An error is returned if a side of the contour is less than 5 pixels long.

Thirdly, the more the contour appears projectively distorted, the more stable the algorithm works. Therefore, the pose of a rectangle tilted with respect to the image plane can be estimated accurately, whereas the pose of an rectangle parallel to the image plane of the camera could be unstable. This is further discussed in the next paragraph. Additionally, there is a rule of thumb that ensures projective distortion: the rectangle should be placed in space such that its size in x and y dimension in the camera coordinate system should not be less than 1/10th of its distance from the camera in z direction.

get_rectangle_poseget_rectangle_poseGetRectanglePoseget_rectangle_poseGetRectanglePoseGetRectanglePose provides two measures for the accuracy of the estimated PosePosePosePosePosepose. ErrorErrorErrorErrorErrorerror is the average pixel error between the contour points and the modeled rectangle backprojected on the image. If ErrorErrorErrorErrorErrorerror is exceeding 0.5, this is an indication that the algorithm did not converge properly, and the resulting PosePosePosePosePosepose should not be used. CovPoseCovPoseCovPoseCovPoseCovPosecovPose contains 36 entries representing the 6×6 covariance matrix of the first 6 entries of PosePosePosePosePosepose. The above mentioned case of instability of the angle about the longer rectangle's axis be detected by checking that the absolute values of the variances and covariances of the rotations around the x and y axis (CovPoseCovPoseCovPoseCovPoseCovPosecovPose[21],CovPoseCovPoseCovPoseCovPoseCovPosecovPose[28], and CovPoseCovPoseCovPoseCovPoseCovPosecovPose[22] == CovPoseCovPoseCovPoseCovPoseCovPosecovPose[27]) do not exceed 0.05. Further, unusually increased values of any of the covariances and especially of the variances (the 6 values on the diagonal of CovPoseCovPoseCovPoseCovPoseCovPosecovPose with indices 0, 7, 14, 21, 28 and 35, respectively) indicate a poor quality of PosePosePosePosePosepose.



ContourContourContourContourContourcontour (input_object)  xld(-array) objectHXLDHXLDHXLDHXLDXHobject

Contour(s) to be examined.

CameraParamCameraParamCameraParamCameraParamCameraParamcameraParam (input_control)  number-array HTupleHTupleHTupleVARIANTHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong) (double / Hlong) (double / Hlong)

Internal camera parameters.

Number of elements: CameraParam == 8 || CameraParam == 12

WidthWidthWidthWidthWidthwidth (input_control)  number(-array) HTupleHTupleHTupleVARIANTHtuple (real) (double) (double) (double) (double) (double)

Width of the rectangle in meters.

Restriction: Width > 0

HeightHeightHeightHeightHeightheight (input_control)  number(-array) HTupleHTupleHTupleVARIANTHtuple (real) (double) (double) (double) (double) (double)

Height of the rectangle in meters.

Restriction: Height > 0

WeightingModeWeightingModeWeightingModeWeightingModeWeightingModeweightingMode (input_control)  string HTupleHTupleHTupleVARIANTHtuple (string) (string) (HString) (char*) (BSTR) (char*)

Weighting mode for the optimization phase.

Default value: 'nonweighted' "nonweighted" "nonweighted" "nonweighted" "nonweighted" "nonweighted"

List of values: 'huber'"huber""huber""huber""huber""huber", 'nonweighted'"nonweighted""nonweighted""nonweighted""nonweighted""nonweighted", 'tukey'"tukey""tukey""tukey""tukey""tukey"

ClippingFactorClippingFactorClippingFactorClippingFactorClippingFactorclippingFactor (input_control)  number HTupleHTupleHTupleVARIANTHtuple (real) (double) (double) (double) (double) (double)

Clipping factor for the elimination of outliers (typical: 1.0 for 'huber' and 3.0 for 'tukey').

Default value: 2.0

Suggested values: 1.0, 1.5, 2.0, 2.5, 3.0

Restriction: ClippingFactor > 0

PosePosePosePosePosepose (output_control)  pose HPose, HTupleHTupleHTupleHPoseX, VARIANTHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong) (double / Hlong) (double / Hlong)

3D pose of the rectangle.

Number of elements: Pose == 7 * Contour

CovPoseCovPoseCovPoseCovPoseCovPosecovPose (output_control)  number-array HTupleHTupleHTupleVARIANTHtuple (real) (double) (double) (double) (double) (double)

Covariances of the pose values.

Number of elements: CovPose == 36 * Contour

ErrorErrorErrorErrorErrorerror (output_control)  number-array HTupleHTupleHTupleVARIANTHtuple (real) (double) (double) (double) (double) (double)

Root-mean-square value of the final residual error.

Number of elements: Error == Contour

Example (HDevelop)

* Process an image with several rectangles of the same size appearing
* as light objects
RectWidth := 0.04
RectHeight := 0.025
read_cam_par ('campar.dat', CameraParam)
read_image (Image, 'tea_boxes')
* find light objects in the image
mean_image (Image, ImageMean, 201, 201)
dyn_threshold (Image, ImageMean, Region, 5, 'light')
* fill gaps in the objects
fill_up (Region, RegionFillUp)
* extract rectangular contours
* NOTE: for a real application, this step might require some additional
*       pre- or postprocessing
reduce_domain (Image, RegionSelected, ImageReduced)
edges_sub_pix (ImageReduced, Edges, 'canny', 0.7, 20, 30)
* get the pose of all contours found
get_rectangle_pose (Edges, CameraParam, RectWidth, RectHeight, 'huber', 2, \
                    Poses, CovPose, Error)
NumPoses := |Poses|/7
for I := 0 to NumPoses-1 by 1
  Pose := Poses[I*7:I*7+6]
  * use the Pose here
  * ...


get_rectangle_poseget_rectangle_poseGetRectanglePoseget_rectangle_poseGetRectanglePoseGetRectanglePose returns 2 (H_MSG_TRUE) if all parameter values are correct and the position of the rectangle has been determined successfully. If the provided contour(s) cannot be segmented as a quadrangle get_rectangle_poseget_rectangle_poseGetRectanglePoseget_rectangle_poseGetRectanglePoseGetRectanglePose returns H_ERR_FIT_QUADRANGLE. If further necessary, an exception is raised.

Possible Predecessors


See also

get_circle_poseget_circle_poseGetCirclePoseget_circle_poseGetCirclePoseGetCirclePose, set_origin_poseset_origin_poseSetOriginPoseset_origin_poseSetOriginPoseSetOriginPose, camera_calibrationcamera_calibrationCameraCalibrationcamera_calibrationCameraCalibrationCameraCalibration


G.Schweighofer and A.Pinz: “Robust Pose Estimation from a Planar Target”; Transactions on Pattern Analysis and Machine Intelligence (PAMI), 28(12):2024-2030, 2006


3D Metrology

ClassesClassesClassesClasses | | | | Operators