HALCON Website / HALCON Operator Reference / Filters / Scene Flow
ClassesClasses | | Operators

scene_flow_calibT_scene_flow_calibSceneFlowCalibSceneFlowCalib (Operator)

Name

scene_flow_calibT_scene_flow_calibSceneFlowCalibSceneFlowCalib — Compute the calibrated scene flow between two stereo image pairs.

Signature

scene_flow_calib(ImageRect1T1, ImageRect2T1, ImageRect1T2, ImageRect2T2, Disparity : : SmoothingFlow, SmoothingDisparity, GenParamName, GenParamValue, CamParamRect1, CamParamRect2, RelPoseRect : ObjectModel3D)

Herror T_scene_flow_calib(const Hobject ImageRect1T1, const Hobject ImageRect2T1, const Hobject ImageRect1T2, const Hobject ImageRect2T2, const Hobject Disparity, const Htuple SmoothingFlow, const Htuple SmoothingDisparity, const Htuple GenParamName, const Htuple GenParamValue, const Htuple CamParamRect1, const Htuple CamParamRect2, const Htuple RelPoseRect, Htuple* ObjectModel3D)

void SceneFlowCalib(const HObject& ImageRect1T1, const HObject& ImageRect2T1, const HObject& ImageRect1T2, const HObject& ImageRect2T2, const HObject& Disparity, const HTuple& SmoothingFlow, const HTuple& SmoothingDisparity, const HTuple& GenParamName, const HTuple& GenParamValue, const HTuple& CamParamRect1, const HTuple& CamParamRect2, const HTuple& RelPoseRect, HTuple* ObjectModel3D)

static HObjectModel3DArray HObjectModel3D::SceneFlowCalib(const HImage& ImageRect1T1, const HImage& ImageRect2T1, const HImage& ImageRect1T2, const HImage& ImageRect2T2, const HImage& Disparity, const HTuple& SmoothingFlow, const HTuple& SmoothingDisparity, const HTuple& GenParamName, const HTuple& GenParamValue, const HCamPar& CamParamRect1, const HCamPar& CamParamRect2, const HPose& RelPoseRect)

void HObjectModel3D::SceneFlowCalib(const HImage& ImageRect1T1, const HImage& ImageRect2T1, const HImage& ImageRect1T2, const HImage& ImageRect2T2, const HImage& Disparity, double SmoothingFlow, double SmoothingDisparity, const HString& GenParamName, const HString& GenParamValue, const HCamPar& CamParamRect1, const HCamPar& CamParamRect2, const HPose& RelPoseRect)

void HObjectModel3D::SceneFlowCalib(const HImage& ImageRect1T1, const HImage& ImageRect2T1, const HImage& ImageRect1T2, const HImage& ImageRect2T2, const HImage& Disparity, double SmoothingFlow, double SmoothingDisparity, const char* GenParamName, const char* GenParamValue, const HCamPar& CamParamRect1, const HCamPar& CamParamRect2, const HPose& RelPoseRect)

void HObjectModel3D::SceneFlowCalib(const HImage& ImageRect1T1, const HImage& ImageRect2T1, const HImage& ImageRect1T2, const HImage& ImageRect2T2, const HImage& Disparity, double SmoothingFlow, double SmoothingDisparity, const wchar_t* GenParamName, const wchar_t* GenParamValue, const HCamPar& CamParamRect1, const HCamPar& CamParamRect2, const HPose& RelPoseRect)   (Windows only)

HObjectModel3DArray HCamPar::SceneFlowCalib(const HImage& ImageRect1T1, const HImage& ImageRect2T1, const HImage& ImageRect1T2, const HImage& ImageRect2T2, const HImage& Disparity, const HTuple& SmoothingFlow, const HTuple& SmoothingDisparity, const HTuple& GenParamName, const HTuple& GenParamValue, const HCamPar& CamParamRect2, const HPose& RelPoseRect) const

HObjectModel3D HCamPar::SceneFlowCalib(const HImage& ImageRect1T1, const HImage& ImageRect2T1, const HImage& ImageRect1T2, const HImage& ImageRect2T2, const HImage& Disparity, double SmoothingFlow, double SmoothingDisparity, const HString& GenParamName, const HString& GenParamValue, const HCamPar& CamParamRect2, const HPose& RelPoseRect) const

HObjectModel3D HCamPar::SceneFlowCalib(const HImage& ImageRect1T1, const HImage& ImageRect2T1, const HImage& ImageRect1T2, const HImage& ImageRect2T2, const HImage& Disparity, double SmoothingFlow, double SmoothingDisparity, const char* GenParamName, const char* GenParamValue, const HCamPar& CamParamRect2, const HPose& RelPoseRect) const

HObjectModel3D HCamPar::SceneFlowCalib(const HImage& ImageRect1T1, const HImage& ImageRect2T1, const HImage& ImageRect1T2, const HImage& ImageRect2T2, const HImage& Disparity, double SmoothingFlow, double SmoothingDisparity, const wchar_t* GenParamName, const wchar_t* GenParamValue, const HCamPar& CamParamRect2, const HPose& RelPoseRect) const   (Windows only)

HObjectModel3DArray HPose::SceneFlowCalib(const HImage& ImageRect1T1, const HImage& ImageRect2T1, const HImage& ImageRect1T2, const HImage& ImageRect2T2, const HImage& Disparity, const HTuple& SmoothingFlow, const HTuple& SmoothingDisparity, const HTuple& GenParamName, const HTuple& GenParamValue, const HCamPar& CamParamRect1, const HCamPar& CamParamRect2) const

HObjectModel3D HPose::SceneFlowCalib(const HImage& ImageRect1T1, const HImage& ImageRect2T1, const HImage& ImageRect1T2, const HImage& ImageRect2T2, const HImage& Disparity, double SmoothingFlow, double SmoothingDisparity, const HString& GenParamName, const HString& GenParamValue, const HCamPar& CamParamRect1, const HCamPar& CamParamRect2) const

HObjectModel3D HPose::SceneFlowCalib(const HImage& ImageRect1T1, const HImage& ImageRect2T1, const HImage& ImageRect1T2, const HImage& ImageRect2T2, const HImage& Disparity, double SmoothingFlow, double SmoothingDisparity, const char* GenParamName, const char* GenParamValue, const HCamPar& CamParamRect1, const HCamPar& CamParamRect2) const

HObjectModel3D HPose::SceneFlowCalib(const HImage& ImageRect1T1, const HImage& ImageRect2T1, const HImage& ImageRect1T2, const HImage& ImageRect2T2, const HImage& Disparity, double SmoothingFlow, double SmoothingDisparity, const wchar_t* GenParamName, const wchar_t* GenParamValue, const HCamPar& CamParamRect1, const HCamPar& CamParamRect2) const   (Windows only)

static void HOperatorSet.SceneFlowCalib(HObject imageRect1T1, HObject imageRect2T1, HObject imageRect1T2, HObject imageRect2T2, HObject disparity, HTuple smoothingFlow, HTuple smoothingDisparity, HTuple genParamName, HTuple genParamValue, HTuple camParamRect1, HTuple camParamRect2, HTuple relPoseRect, out HTuple objectModel3D)

static HObjectModel3D[] HObjectModel3D.SceneFlowCalib(HImage imageRect1T1, HImage imageRect2T1, HImage imageRect1T2, HImage imageRect2T2, HImage disparity, HTuple smoothingFlow, HTuple smoothingDisparity, HTuple genParamName, HTuple genParamValue, HCamPar camParamRect1, HCamPar camParamRect2, HPose relPoseRect)

void HObjectModel3D.SceneFlowCalib(HImage imageRect1T1, HImage imageRect2T1, HImage imageRect1T2, HImage imageRect2T2, HImage disparity, double smoothingFlow, double smoothingDisparity, string genParamName, string genParamValue, HCamPar camParamRect1, HCamPar camParamRect2, HPose relPoseRect)

HObjectModel3D[] HCamPar.SceneFlowCalib(HImage imageRect1T1, HImage imageRect2T1, HImage imageRect1T2, HImage imageRect2T2, HImage disparity, HTuple smoothingFlow, HTuple smoothingDisparity, HTuple genParamName, HTuple genParamValue, HCamPar camParamRect2, HPose relPoseRect)

HObjectModel3D HCamPar.SceneFlowCalib(HImage imageRect1T1, HImage imageRect2T1, HImage imageRect1T2, HImage imageRect2T2, HImage disparity, double smoothingFlow, double smoothingDisparity, string genParamName, string genParamValue, HCamPar camParamRect2, HPose relPoseRect)

HObjectModel3D[] HPose.SceneFlowCalib(HImage imageRect1T1, HImage imageRect2T1, HImage imageRect1T2, HImage imageRect2T2, HImage disparity, HTuple smoothingFlow, HTuple smoothingDisparity, HTuple genParamName, HTuple genParamValue, HCamPar camParamRect1, HCamPar camParamRect2)

HObjectModel3D HPose.SceneFlowCalib(HImage imageRect1T1, HImage imageRect2T1, HImage imageRect1T2, HImage imageRect2T2, HImage disparity, double smoothingFlow, double smoothingDisparity, string genParamName, string genParamValue, HCamPar camParamRect1, HCamPar camParamRect2)

Description

scene_flow_calibscene_flow_calibSceneFlowCalibSceneFlowCalibSceneFlowCalib computes the calibrated scene flow between two consecutive rectified stereo image pairs. The scene flow is the three-dimensional position and motion of surface points in a dynamic scene. The movement in the images can be caused by objects that move in the world or by a movement of the camera (or both) between the acquisition of the two image pairs.

The scene flow is returned as the 3D object model ObjectModel3DObjectModel3DObjectModel3DObjectModel3DobjectModel3D. The 3D object model contains the coordinates of the reconstructed 3D points. Furthermore, the 3D flow is encoded in the 3D object model by the attributes '&flow_x'"&flow_x""&flow_x""&flow_x""&flow_x", '&flow_y'"&flow_y""&flow_y""&flow_y""&flow_y", and '&flow_z'"&flow_z""&flow_z""&flow_z""&flow_z".

The two consecutive stereo image pairs of the image sequence are passed in ImageRect1T1ImageRect1T1ImageRect1T1ImageRect1T1imageRect1T1, ImageRect2T1ImageRect2T1ImageRect2T1ImageRect2T1imageRect2T1, ImageRect1T2ImageRect1T2ImageRect1T2ImageRect1T2imageRect1T2, and ImageRect2T2ImageRect2T2ImageRect2T2ImageRect2T2imageRect2T2. Each stereo image pair must be rectified. Note that the images can be rectified by using the operators calibrate_camerascalibrate_camerasCalibrateCamerasCalibrateCamerasCalibrateCameras, gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapGenBinocularRectificationMap, and map_imagemap_imageMapImageMapImageMapImage.

The camera geometry of the rectified binocular camera system is specified by its internal camera parameters CamParamRect1CamParamRect1CamParamRect1CamParamRect1camParamRect1 of the rectified camera 1 and CamParamRect2CamParamRect2CamParamRect2CamParamRect2camParamRect2 of the rectified camera 2, and the pose RelPoseRectRelPoseRectRelPoseRectRelPoseRectrelPoseRect that defines the pose of the rectified camera 2 in relation to the rectified camera 1. These camera parameters can be obtained from the operators calibrate_camerascalibrate_camerasCalibrateCamerasCalibrateCamerasCalibrateCameras and gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapGenBinocularRectificationMap. The focal length and scale factor of the rectified camera system 1 and 2 must be equal.

Furthermore, a single-channel DisparityDisparityDisparityDisparitydisparity image is required, which specifies for each pixel (r,c1) of the image ImageRect1T1ImageRect1T1ImageRect1T1ImageRect1T1imageRect1T1 a matching pixel (r,c2) of ImageRect2T1ImageRect2T1ImageRect2T1ImageRect2T1imageRect2T1 according to the equation c2=c1+d(r,c1), where d(r,c) is the DisparityDisparityDisparityDisparitydisparity at pixel (r,c). The disparity image can be computed using binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparityBinocularDisparity or binocular_disparity_mgbinocular_disparity_mgBinocularDisparityMgBinocularDisparityMgBinocularDisparityMg.

To calculate the calibrated scene flow, internally scene_flow_uncalibscene_flow_uncalibSceneFlowUncalibSceneFlowUncalibSceneFlowUncalib is first executed. The results are then converted to 3D points and 3D flow vectors using the stereo camera geometry parameters described above.

For a description of the remaining parameters of scene_flow_calibscene_flow_calibSceneFlowCalibSceneFlowCalibSceneFlowCalib, please refer to scene_flow_uncalibscene_flow_uncalibSceneFlowUncalibSceneFlowUncalibSceneFlowUncalib.

Execution Information

This operator returns a handle. Note that the state of an instance of this handle type may be changed by specific operators even though the handle is used as an input parameter by those operators.

Parameters

ImageRect1T1ImageRect1T1ImageRect1T1ImageRect1T1imageRect1T1 (input_object)  singlechannelimage(-array) → objectHImageHImageHobject (byte / uint2 / real)

Input image 1 at time .

ImageRect2T1ImageRect2T1ImageRect2T1ImageRect2T1imageRect2T1 (input_object)  singlechannelimage(-array) → objectHImageHImageHobject (byte / uint2 / real)

Input image 2 at time .

ImageRect1T2ImageRect1T2ImageRect1T2ImageRect1T2imageRect1T2 (input_object)  singlechannelimage(-array) → objectHImageHImageHobject (byte / uint2 / real)

Input image 1 at time .

ImageRect2T2ImageRect2T2ImageRect2T2ImageRect2T2imageRect2T2 (input_object)  singlechannelimage(-array) → objectHImageHImageHobject (byte / uint2 / real)

Input image 2 at time .

DisparityDisparityDisparityDisparitydisparity (input_object)  singlechannelimage(-array) → objectHImageHImageHobject (real)

Disparity between input images 1 and 2 at time .

SmoothingFlowSmoothingFlowSmoothingFlowSmoothingFlowsmoothingFlow (input_control)  number → HTupleHTupleHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong)

Weight of the regularization term relative to the data term (derivatives of the optical flow).

Default value: 40.0

Suggested values: 10.0, 20.0, 30.0, 40.0, 50.0, 60.0, 70.0, 80.0, 90.0, 100.0

Restriction: SmoothingFlow > 0.0

SmoothingDisparitySmoothingDisparitySmoothingDisparitySmoothingDisparitysmoothingDisparity (input_control)  number → HTupleHTupleHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong)

Weight of the regularization term relative to the data term (derivatives of the disparity change).

Default value: 40.0

Suggested values: 10.0, 20.0, 30.0, 40.0, 50.0, 60.0, 70.0, 80.0, 90.0, 100.0

Restriction: SmoothingDisparity > 0.0

GenParamNameGenParamNameGenParamNameGenParamNamegenParamName (input_control)  attribute.name(-array) → HTupleHTupleHtuple (string) (string) (HString) (char*)

Parameter name(s) for the algorithm.

Default value: 'default_parameters' "default_parameters" "default_parameters" "default_parameters" "default_parameters"

Suggested values: 'default_parameters'"default_parameters""default_parameters""default_parameters""default_parameters", 'warp_levels'"warp_levels""warp_levels""warp_levels""warp_levels", 'warp_zoom_factor'"warp_zoom_factor""warp_zoom_factor""warp_zoom_factor""warp_zoom_factor", 'warp_last_level'"warp_last_level""warp_last_level""warp_last_level""warp_last_level", 'outer_iter'"outer_iter""outer_iter""outer_iter""outer_iter", 'inner_iter'"inner_iter""inner_iter""inner_iter""inner_iter", 'sor_iter'"sor_iter""sor_iter""sor_iter""sor_iter", 'omega'"omega""omega""omega""omega"

GenParamValueGenParamValueGenParamValueGenParamValuegenParamValue (input_control)  attribute.value(-array) → HTupleHTupleHtuple (string / integer / real) (string / int / long / double) (HString / Hlong / double) (char* / Hlong / double)

Parameter value(s) for the algorithm.

Default value: 'accurate' "accurate" "accurate" "accurate" "accurate"

Suggested values: 'very_accurate'"very_accurate""very_accurate""very_accurate""very_accurate", 'accurate'"accurate""accurate""accurate""accurate", 'fast'"fast""fast""fast""fast", 'very_fast'"very_fast""very_fast""very_fast""very_fast", 0, 1, 2, 3, 4, 5, 6, 0.5, 0.6, 0.7, 0.75, 3, 5, 7, 2, 3, 1.9

CamParamRect1CamParamRect1CamParamRect1CamParamRect1camParamRect1 (input_control)  campar → HCamPar, HTupleHTupleHtuple (real / integer / string) (double / int / long / string) (double / Hlong / HString) (double / Hlong / char*)

Internal camera parameters of the rectified camera 1.

CamParamRect2CamParamRect2CamParamRect2CamParamRect2camParamRect2 (input_control)  campar → HCamPar, HTupleHTupleHtuple (real / integer / string) (double / int / long / string) (double / Hlong / HString) (double / Hlong / char*)

Internal camera parameters of the rectified camera 2.

RelPoseRectRelPoseRectRelPoseRectRelPoseRectrelPoseRect (input_control)  pose → HPose, HTupleHTupleHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong)

Pose of the rectified camera 2 in relation to the rectified camera 1.

Number of elements: 7

ObjectModel3DObjectModel3DObjectModel3DObjectModel3DobjectModel3D (output_control)  object_model_3d(-array) → HObjectModel3D, HTupleHTupleHtuple (handle) (IntPtr) (HHandle) (handle)

Handle of the 3D object model.

Result

If the parameter values are correct, the operator scene_flow_calibscene_flow_calibSceneFlowCalibSceneFlowCalibSceneFlowCalib returns the value 2 (H_MSG_TRUE). If the input is empty (no input images are available) the behavior can be set via set_system('no_object_result',<Result>)set_system("no_object_result",<Result>)SetSystem("no_object_result",<Result>)SetSystem("no_object_result",<Result>)SetSystem("no_object_result",<Result>). If necessary, an exception is raised.

Possible Predecessors

binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparityBinocularDisparity, binocular_disparity_mgbinocular_disparity_mgBinocularDisparityMgBinocularDisparityMgBinocularDisparityMg

Alternatives

scene_flow_uncalibscene_flow_uncalibSceneFlowUncalibSceneFlowUncalibSceneFlowUncalib, optical_flow_mgoptical_flow_mgOpticalFlowMgOpticalFlowMgOpticalFlowMg

References

A. Wedel, C. Rabe, T. Vaudrey, T. Brox, U. Franke and D. Cremers: “Efficient dense scene flow from sparse or dense stereo data”; In: Proceedings of the 10th European Conference on Computer Vision: Part I, pages 739-751. Springer-Verlag, 2008.

Module

Foundation

ClassesClasses | | Operators
HALCON Operator Reference 18.11.3.0 Copyright © 1996-2020 MVTec Software GmbH