adjust_mosaic_imagesT_adjust_mosaic_imagesAdjustMosaicImagesAdjustMosaicImages (Operator)

Name

adjust_mosaic_imagesT_adjust_mosaic_imagesAdjustMosaicImagesAdjustMosaicImages — Apply an automatic color correction to panorama images.

Signature

adjust_mosaic_images(Images : CorrectedImages : From, To, ReferenceImage, HomMatrices2D, EstimationMethod, EstimateParameters, OECFModel : )

Herror T_adjust_mosaic_images(const Hobject Images, Hobject* CorrectedImages, const Htuple From, const Htuple To, const Htuple ReferenceImage, const Htuple HomMatrices2D, const Htuple EstimationMethod, const Htuple EstimateParameters, const Htuple OECFModel)

void AdjustMosaicImages(const HObject& Images, HObject* CorrectedImages, const HTuple& From, const HTuple& To, const HTuple& ReferenceImage, const HTuple& HomMatrices2D, const HTuple& EstimationMethod, const HTuple& EstimateParameters, const HTuple& OECFModel)

HImage HImage::AdjustMosaicImages(const HTuple& From, const HTuple& To, Hlong ReferenceImage, const HTuple& HomMatrices2D, const HString& EstimationMethod, const HTuple& EstimateParameters, const HString& OECFModel) const

HImage HImage::AdjustMosaicImages(const HTuple& From, const HTuple& To, Hlong ReferenceImage, const HTuple& HomMatrices2D, const HString& EstimationMethod, const HString& EstimateParameters, const HString& OECFModel) const

HImage HImage::AdjustMosaicImages(const HTuple& From, const HTuple& To, Hlong ReferenceImage, const HTuple& HomMatrices2D, const char* EstimationMethod, const char* EstimateParameters, const char* OECFModel) const

HImage HImage::AdjustMosaicImages(const HTuple& From, const HTuple& To, Hlong ReferenceImage, const HTuple& HomMatrices2D, const wchar_t* EstimationMethod, const wchar_t* EstimateParameters, const wchar_t* OECFModel) const   (Windows only)

static void HOperatorSet.AdjustMosaicImages(HObject images, out HObject correctedImages, HTuple from, HTuple to, HTuple referenceImage, HTuple homMatrices2D, HTuple estimationMethod, HTuple estimateParameters, HTuple OECFModel)

HImage HImage.AdjustMosaicImages(HTuple from, HTuple to, int referenceImage, HTuple homMatrices2D, string estimationMethod, HTuple estimateParameters, string OECFModel)

HImage HImage.AdjustMosaicImages(HTuple from, HTuple to, int referenceImage, HTuple homMatrices2D, string estimationMethod, string estimateParameters, string OECFModel)

Description

adjust_mosaic_imagesadjust_mosaic_imagesAdjustMosaicImagesAdjustMosaicImagesAdjustMosaicImages performs the radiometric adjustment of images in panoramas. The images to be corrected have to be passed in ImagesImagesImagesImagesimages, the corrected images will be returned in CorrectedImagesCorrectedImagesCorrectedImagesCorrectedImagescorrectedImages.

The parameters FromFromFromFromfrom and ToToToToto must contain the source and destination indices of all image pairs in the panorama. The projective 3x3-matrix of each image pair has to be passed in HomMatrices2DHomMatrices2DHomMatrices2DHomMatrices2DhomMatrices2D. The image, which will be used as the reference for brightness and white balance, is selected with the parameter ReferenceImageReferenceImageReferenceImageReferenceImagereferenceImage.

This means, that one image specifies the "ideal" brightness and white balance settings. All other images will be corrected such that their brightness and white balance match that of the reference image. In other words, the reference image will not be changed, but all other images will be.

EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod is used for choosing whether a fast but less accurate, or a slower but more accurate determination method should be used. This is done by setting EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod either to 'standard'"standard""standard""standard""standard" or 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard".

The methods based on 'standard'"standard""standard""standard""standard" use only the average gray value difference of all images in the overlap area between each image pair. With 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard" the gray value difference for each pixel in the overlap area is taken into account explicitly.

The error function that is minimized in all cases is computed as the summed square of the difference between the corresponding gray values.

The availability of the individual method is depending on the selected EstimateParametersEstimateParametersEstimateParametersEstimateParametersestimateParameters, which determines the model to be used for estimating the radiometric adjustment terms. It is always possible to determine the amount of vignetting in the images by selecting 'vignetting'"vignetting""vignetting""vignetting""vignetting". However, if selected, EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod must be set to 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard". For the remainder of the radiometric adjustment three different options are available:

Estimation of vignetting in images is based on the commonly known approach. This approach assumes that vignetting does not exist in the center of the image and increases with the opening angle by the above equation.

1. Image adjustment with the additive model. This should only be used to adjust images with very small differences in exposure or white balance. To choose this method, EstimateParametersEstimateParametersEstimateParametersEstimateParametersestimateParameters must be set to 'add_gray'"add_gray""add_gray""add_gray""add_gray". This model can be selected either exclusively and only with EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod = 'standard'"standard""standard""standard""standard" or in combination with EstimateParametersEstimateParametersEstimateParametersEstimateParametersestimateParameters = 'vignetting'"vignetting""vignetting""vignetting""vignetting" and only with EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod = 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard".

This model is based on the assumption, that the gray value differences between the images can be corrected by adding an individual value to each image accept the reference image. Basically, the modification to every image can be expressed as a call of scale_imagescale_imageScaleImageScaleImageScaleImage:

scale_image(Image_n,CorrectedImage_n,1.0,Add_n)

where Add_n is the correction term for this image.

2. Image adjustment with the linear model. In this model, images are expected to be taken with a camera using a linear transfer function. The adjustment terms are consequently represented as multiplication factors. To select this model, EstimateParametersEstimateParametersEstimateParametersEstimateParametersestimateParameters must be set to 'mult_gray'"mult_gray""mult_gray""mult_gray""mult_gray". It can be called with EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod = 'standard'"standard""standard""standard""standard" or EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod = 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard". A combined call with EstimateParametersEstimateParametersEstimateParametersEstimateParametersestimateParameters = 'vignetting'"vignetting""vignetting""vignetting""vignetting" is also possible, EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod must be set to 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard" in that case.

This model is based on the assumption that the gray value differences between the images can be corrected by multiplying the gray values in each image by a factor. Basically, the modification to every image can again be expressed as a call of scale_imagescale_imageScaleImageScaleImageScaleImage:

scale_image(Image_n,CorrectedImage_n,Mult_n,0)

where Mult_n is the correction term for this image.

3. Image adjustment with the calibrated model. In this model, images are assumed to be taken with a camera using a nonlinear transfer function. A function of the OECF class selected with OECFModelOECFModelOECFModelOECFModelOECFModel is used to approximate the actually used OECF in the process of image acquisition. As with the linear model, the correction terms are represented as multiplication factors. This model can be selected by choosing EstimateParametersEstimateParametersEstimateParametersEstimateParametersestimateParameters = ['mult_gray','response']["mult_gray","response"]["mult_gray","response"]["mult_gray","response"]["mult_gray","response"] and must be called with EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod = 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard". It is possible to determine the amount of vignetting as well in this case by choosing EstimateParametersEstimateParametersEstimateParametersEstimateParametersestimateParameters = 'vignetting'"vignetting""vignetting""vignetting""vignetting".

This model is similar to the linear model. However, in this case the camera may have a nonlinear response. This means that before the gray values of the images can be multiplied by their respective correction factor, the gray values must be backprojected to a linear response. To do so, the camera's response must be determined. Since the response usually does not change over an image sequence, this parameter is assumed to be constant throughout the whole image sequence.

Any kind of function could be considered to be used as an OECF. As in the operator radiometric_self_calibrationradiometric_self_calibrationRadiometricSelfCalibrationRadiometricSelfCalibrationRadiometricSelfCalibration, a polynomial fitting might be used, but for typical images in a mosaicking application this would not work very well. The reason for this is that polynomial fitting has too many parameters that need to be determined. Instead, only simpler types of response functions can be estimated. Currently, only so-called Laguerre-functions are available.

The response of a Laguerre-type OECF is determined by only one parameter called Phi. In a first step, the whole gray value spectrum (in case of 8bit images the values 0 to 255) is converted to floating point numbers in the interval [0:1]. Then, the OECF backprojection is calculated based on this and the resulting gray values are once again converted to the original interval.

The inverse transform of the gray values back to linear values based on a Laguerre-type OECF is described by the following equation:

with I_l the linear gray value and I_nl the (nonlinear) gray value.

The parameter OECFModelOECFModelOECFModelOECFModelOECFModel is only used if the calibrated model has been chosen. Otherwise, any input for OECFModelOECFModelOECFModelOECFModelOECFModel will be ignored.

The parameter EstimateParametersEstimateParametersEstimateParametersEstimateParametersestimateParameters can also be used to influence the performance and memory consumption of the operator. With 'no_cache'"no_cache""no_cache""no_cache""no_cache" the internal caching mechanism can be disabled. This switch only has and influence if EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod is set to 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard". Otherwise this switch will be ignored. When disabling the internal caching, the operator uses far less memory, but therefore calculates the corresponding grayvalue pairs in each iteration of the minimization algorithm again. Therefore, disabling caching is only advisable if all physical memory is used up at some point of the calculation and the operating system starts using swap space.

A second option to influence the performance is using subsampling. When setting EstimateParametersEstimateParametersEstimateParametersEstimateParametersestimateParameters to 'subsampling_2'"subsampling_2""subsampling_2""subsampling_2""subsampling_2", images are internally zoomed down by a factor of 2. Despite the suggested value list, not only factors of 2 and 4 are available, but any integer number might be specified by appending it to subsampling_ in EstimateParametersEstimateParametersEstimateParametersEstimateParametersestimateParameters. With this, the amount of image data is tremendously reduced, which leads to a much faster computation of the internal minimization. In fact, using moderate subsampling might even lead to better results since it also decreases the influence of slightly misaligned pixels. Although subsampling also influences the minimization if EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod is set to 'standard'"standard""standard""standard""standard", it is mostly useful for 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard".

Some more general remarks on using adjust_mosaic_images in applications:

Execution Information

Parameters

ImagesImagesImagesImagesimages (input_object)  (multichannel-)image-array objectHImageHImageHobject (byte)

Input images.

CorrectedImagesCorrectedImagesCorrectedImagesCorrectedImagescorrectedImages (output_object)  (multichannel-)image-array objectHImageHImageHobject * (byte)

Output images.

FromFromFromFromfrom (input_control)  integer-array HTupleHTupleHtuple (integer) (int / long) (Hlong) (Hlong)

List of source images.

ToToToToto (input_control)  integer-array HTupleHTupleHtuple (integer) (int / long) (Hlong) (Hlong)

List of destination images.

ReferenceImageReferenceImageReferenceImageReferenceImagereferenceImage (input_control)  integer HTupleHTupleHtuple (integer) (int / long) (Hlong) (Hlong)

Reference image.

HomMatrices2DHomMatrices2DHomMatrices2DHomMatrices2DhomMatrices2D (input_control)  real-array HTupleHTupleHtuple (real) (double) (double) (double)

Projective matrices.

EstimationMethodEstimationMethodEstimationMethodEstimationMethodestimationMethod (input_control)  string HTupleHTupleHtuple (string) (string) (HString) (char*)

Estimation algorithm for the correction.

Default value: 'standard' "standard" "standard" "standard" "standard"

List of values: 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard", 'standard'"standard""standard""standard""standard"

EstimateParametersEstimateParametersEstimateParametersEstimateParametersestimateParameters (input_control)  string(-array) HTupleHTupleHtuple (string) (string) (HString) (char*)

Parameters to be estimated.

Default value: ['mult_gray'] ["mult_gray"] ["mult_gray"] ["mult_gray"] ["mult_gray"]

Suggested values: 'add_gray'"add_gray""add_gray""add_gray""add_gray", 'mult_gray'"mult_gray""mult_gray""mult_gray""mult_gray", 'response'"response""response""response""response", 'vignetting'"vignetting""vignetting""vignetting""vignetting", 'subsampling_2'"subsampling_2""subsampling_2""subsampling_2""subsampling_2", 'subsampling_4'"subsampling_4""subsampling_4""subsampling_4""subsampling_4", 'no_cache'"no_cache""no_cache""no_cache""no_cache"

OECFModelOECFModelOECFModelOECFModelOECFModel (input_control)  string HTupleHTupleHtuple (string) (string) (HString) (char*)

Model of OECF to be used.

Default value: ['laguerre'] ["laguerre"] ["laguerre"] ["laguerre"] ["laguerre"]

List of values: 'laguerre'"laguerre""laguerre""laguerre""laguerre"

Example (HDevelop)

* For the input data to stationary_camera_self_calibration, please
* refer to the example for stationary_camera_self_calibration.
stationary_camera_self_calibration (4, 640, 480, 1, From, To, \
                                    HomMatrices2D, Rows1, Cols1, \
                                    Rows2, Cols2, NumMatches, \
                                    'gold_standard', \
                                    ['focus','principal_point'], \
                                    'true', CameraMatrix, Kappa, \
                                    RotationMatrices, X, Y, Z, Error)
adjust_mosaic_images(Images,CorrectedImages,From,To,1,HomMatrices2D, \
                     'gold_standard',['mult_gray','response'],'laguerre')

Result

If the parameters are valid, the operator adjust_mosaic_imagesadjust_mosaic_imagesAdjustMosaicImagesAdjustMosaicImagesAdjustMosaicImages returns the value 2 (H_MSG_TRUE). If necessary an exception is raised.

Possible Predecessors

stationary_camera_self_calibrationstationary_camera_self_calibrationStationaryCameraSelfCalibrationStationaryCameraSelfCalibrationStationaryCameraSelfCalibration

Possible Successors

gen_spherical_mosaicgen_spherical_mosaicGenSphericalMosaicGenSphericalMosaicGenSphericalMosaic

References

David Hasler, Sabine Süsstrunk: Mapping colour in image stitching applications. Journal of Visual Communication and Image Representation, 15(1):65-90, 2004.

Module

Foundation