photometric_stereoT_photometric_stereoPhotometricStereoPhotometricStereophotometric_stereo (Operator)

Name

photometric_stereoT_photometric_stereoPhotometricStereoPhotometricStereophotometric_stereo — Reconstruct a surface according to the photometric stereo technique.

Signature

photometric_stereo(Images : HeightField, Gradient, Albedo : Slants, Tilts, ResultType, ReconstructionMethod, GenParamName, GenParamValue : )

Herror T_photometric_stereo(const Hobject Images, Hobject* HeightField, Hobject* Gradient, Hobject* Albedo, const Htuple Slants, const Htuple Tilts, const Htuple ResultType, const Htuple ReconstructionMethod, const Htuple GenParamName, const Htuple GenParamValue)

void PhotometricStereo(const HObject& Images, HObject* HeightField, HObject* Gradient, HObject* Albedo, const HTuple& Slants, const HTuple& Tilts, const HTuple& ResultType, const HTuple& ReconstructionMethod, const HTuple& GenParamName, const HTuple& GenParamValue)

HImage HImage::PhotometricStereo(HImage* Gradient, HImage* Albedo, const HTuple& Slants, const HTuple& Tilts, const HTuple& ResultType, const HString& ReconstructionMethod, const HTuple& GenParamName, const HTuple& GenParamValue) const

HImage HImage::PhotometricStereo(HImage* Gradient, HImage* Albedo, const HTuple& Slants, const HTuple& Tilts, const HTuple& ResultType, const char* ReconstructionMethod, const HTuple& GenParamName, const HTuple& GenParamValue) const

HImage HImage::PhotometricStereo(HImage* Gradient, HImage* Albedo, const HTuple& Slants, const HTuple& Tilts, const HTuple& ResultType, const wchar_t* ReconstructionMethod, const HTuple& GenParamName, const HTuple& GenParamValue) const   (Windows only)

static void HOperatorSet.PhotometricStereo(HObject images, out HObject heightField, out HObject gradient, out HObject albedo, HTuple slants, HTuple tilts, HTuple resultType, HTuple reconstructionMethod, HTuple genParamName, HTuple genParamValue)

HImage HImage.PhotometricStereo(out HImage gradient, out HImage albedo, HTuple slants, HTuple tilts, HTuple resultType, string reconstructionMethod, HTuple genParamName, HTuple genParamValue)

def photometric_stereo(images: HObject, slants: Sequence[Union[float, int]], tilts: Sequence[Union[float, int]], result_type: Sequence[str], reconstruction_method: str, gen_param_name: Sequence[str], gen_param_value: Sequence[Union[int, float, str]]) -> Tuple[HObject, HObject, HObject]

Description

photometric_stereophotometric_stereoPhotometricStereoPhotometricStereoPhotometricStereophotometric_stereo can be used to separate the three-dimensional shape of an object from its two-dimensional texture, e.g., its print image. The operator requires at least three images of the same object taken with different and known directions of illumination. Note, that the point of view of the camera must be the same for all images.

The three-dimensional shape of the object is primarily computed as the local gradients of the three-dimensional surface. Those gradients can be further integrated to obtain a height field, i.e., an image in which the pixel values correspond to a relative height. The two-dimensional texture is called albedo and corresponds to the local light absorption and reflection characteristics of the surface exclusive of any shading effect.

Typical applications of photometric stereo

Typical applications of photometric stereo are to detect small inconsistencies in a surface that represent, e.g., defects, or to exclude the influence of the direction of light from images that are used, e.g., for the print inspection of non flat characters. Note that photometric stereo is not suitable for the reconstruction of absolute heights, i.e., it is no alternative to typical 3D reconstruction algorithms like depth from focus or sheet of light.

Limitations of photometric stereo

photometric_stereophotometric_stereoPhotometricStereoPhotometricStereoPhotometricStereophotometric_stereo is based on the algorithm of Woodham and therefore assumes on the one hand that the camera performs an orthoscopic projection. That is, you must use a telecentric lens or a lens with a long focal distance. On the other hand, it assumes that each of the light sources delivers a parallel and uniform beam of light. That is, you must use telecentric illumination sources with uniform intensity or, as an alternative, distant point light sources. Additionally, the object must have Lambertian reflectance characteristics, i.e., it must reflect incoming light in a diffuse way. Objects or regions of an object that have specular reflectance characteristics (i.e., mirroring or glossy surfaces) cannot be processed correctly and thus lead to erroneous results.

The acquisition setup

The camera with a telecentric lens must be placed orthogonally, i.e., perpendicular, to the scene that should be reconstructed. The orientation of the camera with respect to the scene must not change during the acquisition of the images. In contrast, the orientation of the illumination with respect to the camera must change for at least three gray value images.

Specifying the directions of illumination

For each image, the directions of illumination must be specified as angles within the parameters SlantsSlantsSlantsSlantsslantsslants and TiltsTiltsTiltsTiltstiltstilts, which describe the direction of the illumination in relation to the scene. To understand the meaning of the parameters SlantsSlantsSlantsSlantsslantsslants and TiltsTiltsTiltsTiltstiltstilts, remember that the illumination source is assumed to produce parallel light rays, the camera has a telecentric lens, and the camera is placed orthogonal to the scene to reconstruct:

SlantsSlantsSlantsSlantsslantsslants
The SlantsSlantsSlantsSlantsslantsslants angle is the angle between the optical axis of the camera and the direction of the illumination.
image/svg+xml Slant
Side view
TiltsTiltsTiltsTiltstiltstilts
The TiltsTiltsTiltsTiltstiltstilts angle is measured within the object plane or any plane that is parallel to it, e.g., the image plane. In particular, it describes the angle between the direction that points from the center of the image to the right and the direction of light that is projected into the plane. That is, when looking at the image (or the corresponding scene), a tilt angle of 0 means that the light comes from the right, a tilt angle of 90 means that the light is coming from the top, a tilt angle of 180 means that the light is coming from the left, etc.
image/svg+xml 90° 180° 270° Tilt
Top view

As stated before, photometric stereo requires at least three images with different directions of illumination. However, the three-dimensional geometry of objects typically leads to shadow casting. In the shadow regions, the number of effectively available directions of illumination is reduced, which leads to ambiguities. To nevertheless get a robust result, redundancy is needed. Therefore, typically more than three light sources with different directions should be used. But note that an increasing number of illumination directions also leads to a higher number of images to be processed and therefore to a higher processing time. In most applications, a number of four to six light sources is reasonable. As a rule of thumb, the slant angles should be chosen between 30° and 60°. The tilt angles typically should be equally distributed around the object to be measured. Please note that the directions of illumination must be selected such that they do not lie in the same plane (i.e., the illumination directions must be independent), otherwise the computing fails and an exception is thrown.

Input images and domains of definition

The input images must be provided in an image array (ImagesImagesImagesImagesimagesimages). Each image must have been taken with a different direction of illumination as stated above. If the images are primarily stored in a multi-channel image, they can be easily converted to an image array using image_to_channelsimage_to_channelsImageToChannelsImageToChannelsImageToChannelsimage_to_channels. As an alternative, the image array can be created using concat_objconcat_objConcatObjConcatObjConcatObjconcat_obj.

photometric_stereophotometric_stereoPhotometricStereoPhotometricStereoPhotometricStereophotometric_stereo relies on the evaluation of the "photometric information", i.e., the gray values stored in the images. Therefore, this information should be unbiased and accurate. We recommend to ensure that the camera that is used to acquire the images has a linear characteristic. You can use the operator radiometric_self_calibrationradiometric_self_calibrationRadiometricSelfCalibrationRadiometricSelfCalibrationRadiometricSelfCalibrationradiometric_self_calibration to determine the characteristic of your camera and the operator lut_translut_transLutTransLutTransLutTranslut_trans to correct the gray value information in case of a non linear characteristic. Additionally, if accurate measurements are required, we recommend to utilize the full dynamic range of the camera since this leads to more accurate gray value information. For the same reason, using images with a bit depth higher than 8 (e.g., uint2 images instead of byte images) leads to a better accuracy.

The domain of definition of the input images determines which algorithm is used internally to process the ImagesImagesImagesImagesimagesimages. Three algorithms are available:

Output images

The operator can return the images for the reconstructed GradientGradientGradientGradientgradientgradient, AlbedoAlbedoAlbedoAlbedoalbedoalbedo, and the HeightFieldHeightFieldHeightFieldHeightFieldheightFieldheight_field of the surface:

By default, all of these iconic objects are returned, i.e., the parameter ResultTypeResultTypeResultTypeResultTyperesultTyperesult_type is set to 'all'"all""all""all""all""all". In case that only some of these results are needed, the parameter ResultTypeResultTypeResultTypeResultTyperesultTyperesult_type can be set to a tuple specifying only the required results among the values 'gradient'"gradient""gradient""gradient""gradient""gradient", 'albedo'"albedo""albedo""albedo""albedo""albedo", and 'height_field'"height_field""height_field""height_field""height_field""height_field". Note that in certain applications like surface inspection tasks only the GradientGradientGradientGradientgradientgradient or AlbedoAlbedoAlbedoAlbedoalbedoalbedo images are required. Here, one can significantly increase the processing speed by not reconstructing the surface, i.e., by passing only 'gradient'"gradient""gradient""gradient""gradient""gradient" and 'albedo'"albedo""albedo""albedo""albedo""albedo" but not 'height_field'"height_field""height_field""height_field""height_field""height_field" to ResultTypeResultTypeResultTypeResultTyperesultTyperesult_type.

Note that internally photometric_stereophotometric_stereoPhotometricStereoPhotometricStereoPhotometricStereophotometric_stereo first determines the gradient values and, if required, integrates these values in order to obtain the height field. This integration is performed by the same algorithms that are provided by the operator reconstruct_height_field_from_gradientreconstruct_height_field_from_gradientReconstructHeightFieldFromGradientReconstructHeightFieldFromGradientReconstructHeightFieldFromGradientreconstruct_height_field_from_gradient and that can be controlled by the parameters ReconstructionMethodReconstructionMethodReconstructionMethodReconstructionMethodreconstructionMethodreconstruction_method, GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name, and GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value. Please, refer to the operator reconstruct_height_field_from_gradientreconstruct_height_field_from_gradientReconstructHeightFieldFromGradientReconstructHeightFieldFromGradientReconstructHeightFieldFromGradientreconstruct_height_field_from_gradient for more information on these parameters. If ResultTypeResultTypeResultTypeResultTyperesultTyperesult_type is set such that 'height_field'"height_field""height_field""height_field""height_field""height_field" is not one of the results, the parameters ReconstructionMethodReconstructionMethodReconstructionMethodReconstructionMethodreconstructionMethodreconstruction_method, GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name, and GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value are ignored.

Attention

Note that photometric_stereophotometric_stereoPhotometricStereoPhotometricStereoPhotometricStereophotometric_stereo assumes square pixels. Additionally, it assumes that the heights are computed on a lattice with step width 1 in object space. If this is not the case, i.e., if the pixel size of the camera projected into the object space differs from 1, the returned height values must be multiplied by the actual step width (value of the pixel size projected into the object space). The size of the pixel in object space is computed by dividing the size of the pixel in the camera by the magnification of the (telecentric) lens.

Execution Information

Parameters

ImagesImagesImagesImagesimagesimages (input_object)  singlechannelimage(-array) objectHImageHObjectHImageHobject (byte / uint2)

Array with at least three input images with different directions of illumination.

HeightFieldHeightFieldHeightFieldHeightFieldheightFieldheight_field (output_object)  image objectHImageHObjectHImageHobject * (real)

Reconstructed height field.

GradientGradientGradientGradientgradientgradient (output_object)  image objectHImageHObjectHImageHobject * (vector_field)

The gradient field of the surface.

AlbedoAlbedoAlbedoAlbedoalbedoalbedo (output_object)  image objectHImageHObjectHImageHobject * (real)

The albedo of the surface.

SlantsSlantsSlantsSlantsslantsslants (input_control)  angle.deg-array HTupleSequence[Union[float, int]]HTupleHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong)

Angle between the camera and the direction of illumination (in degrees).

Default value: 45.0

Suggested values: 1.0, 5.0, 10.0, 20.0, 40.0, 60.0, 90.0

Typical range of values: 0.0 ≤ Slants Slants Slants Slants slants slants ≤ 180.0 (lin)

Minimum increment: 0.01

Recommended increment: 10.0

TiltsTiltsTiltsTiltstiltstilts (input_control)  angle.deg-array HTupleSequence[Union[float, int]]HTupleHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong)

Angle of the direction of illumination within the object plane (in degrees).

Default value: 45.0

Suggested values: 1.0, 5.0, 10.0, 20.0, 40.0, 60.0, 90.0

Typical range of values: 0.0 ≤ Tilts Tilts Tilts Tilts tilts tilts ≤ 360.0 (lin)

Minimum increment: 0.01

Recommended increment: 10.0

ResultTypeResultTypeResultTypeResultTyperesultTyperesult_type (input_control)  string-array HTupleSequence[str]HTupleHtuple (string) (string) (HString) (char*)

Types of the requested results.

Default value: 'all' "all" "all" "all" "all" "all"

List of values: [], 'albedo'"albedo""albedo""albedo""albedo""albedo", 'all'"all""all""all""all""all", 'gradient'"gradient""gradient""gradient""gradient""gradient", 'height_field'"height_field""height_field""height_field""height_field""height_field", 'normalized_surface_normal'"normalized_surface_normal""normalized_surface_normal""normalized_surface_normal""normalized_surface_normal""normalized_surface_normal"

ReconstructionMethodReconstructionMethodReconstructionMethodReconstructionMethodreconstructionMethodreconstruction_method (input_control)  string HTuplestrHTupleHtuple (string) (string) (HString) (char*)

Type of the reconstruction method.

Default value: 'poisson' "poisson" "poisson" "poisson" "poisson" "poisson"

List of values: 'fft_cyclic'"fft_cyclic""fft_cyclic""fft_cyclic""fft_cyclic""fft_cyclic", 'poisson'"poisson""poisson""poisson""poisson""poisson", 'rft_cyclic'"rft_cyclic""rft_cyclic""rft_cyclic""rft_cyclic""rft_cyclic"

GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name (input_control)  string-array HTupleSequence[str]HTupleHtuple (string) (string) (HString) (char*)

Names of the generic parameters.

Default value: []

List of values: 'caching'"caching""caching""caching""caching""caching", 'optimize_speed'"optimize_speed""optimize_speed""optimize_speed""optimize_speed""optimize_speed"

GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value (input_control)  integer-array HTupleSequence[Union[int, float, str]]HTupleHtuple (integer / real / string) (int / long / double / string) (Hlong / double / HString) (Hlong / double / char*)

Values of the generic parameters.

Default value: []

List of values: 'exhaustive'"exhaustive""exhaustive""exhaustive""exhaustive""exhaustive", 'free_cache'"free_cache""free_cache""free_cache""free_cache""free_cache", 'no_cache'"no_cache""no_cache""no_cache""no_cache""no_cache", 'patient'"patient""patient""patient""patient""patient", 'standard'"standard""standard""standard""standard""standard", 'use_cache'"use_cache""use_cache""use_cache""use_cache""use_cache"

Result

If the parameters are valid, photometric_stereophotometric_stereoPhotometricStereoPhotometricStereoPhotometricStereophotometric_stereo returns the value 2 (H_MSG_TRUE). If necessary, an exception is raised.

Possible Predecessors

optimize_fft_speedoptimize_fft_speedOptimizeFftSpeedOptimizeFftSpeedOptimizeFftSpeedoptimize_fft_speed

Module

3D Metrology