ClassesClassesClassesClasses | | | | Operators

inpainting_cedinpainting_cedInpaintingCedinpainting_cedInpaintingCedInpaintingCed (Operator)


inpainting_cedinpainting_cedInpaintingCedinpainting_cedInpaintingCedInpaintingCed — Perform an inpainting by coherence enhancing diffusion.


inpainting_ced(Image, Region : InpaintedImage : Sigma, Rho, Theta, Iterations : )

Herror inpainting_ced(const Hobject Image, const Hobject Region, Hobject* InpaintedImage, double Sigma, double Rho, double Theta, const Hlong Iterations)

Herror T_inpainting_ced(const Hobject Image, const Hobject Region, Hobject* InpaintedImage, const Htuple Sigma, const Htuple Rho, const Htuple Theta, const Htuple Iterations)

Herror inpainting_ced(Hobject Image, Hobject Region, Hobject* InpaintedImage, const HTuple& Sigma, const HTuple& Rho, const HTuple& Theta, const HTuple& Iterations)

HImage HImage::InpaintingCed(const HRegion& Region, const HTuple& Sigma, const HTuple& Rho, const HTuple& Theta, const HTuple& Iterations) const

HImageArray HImageArray::InpaintingCed(const HRegion& Region, const HTuple& Sigma, const HTuple& Rho, const HTuple& Theta, const HTuple& Iterations) const

void InpaintingCed(const HObject& Image, const HObject& Region, HObject* InpaintedImage, const HTuple& Sigma, const HTuple& Rho, const HTuple& Theta, const HTuple& Iterations)

HImage HImage::InpaintingCed(const HRegion& Region, double Sigma, double Rho, double Theta, Hlong Iterations) const

void HOperatorSetX.InpaintingCed(
[in] IHUntypedObjectX* Image, [in] IHUntypedObjectX* Region, [out] IHUntypedObjectX*InpaintedImage, [in] VARIANT Sigma, [in] VARIANT Rho, [in] VARIANT Theta, [in] VARIANT Iterations)

IHImageX* HImageX.InpaintingCed(
[in] IHRegionX* Region, [in] double Sigma, [in] double Rho, [in] double Theta, [in] Hlong Iterations)

static void HOperatorSet.InpaintingCed(HObject image, HObject region, out HObject inpaintedImage, HTuple sigma, HTuple rho, HTuple theta, HTuple iterations)

HImage HImage.InpaintingCed(HRegion region, double sigma, double rho, double theta, int iterations)


The operator inpainting_cedinpainting_cedInpaintingCedinpainting_cedInpaintingCedInpaintingCed performs an anisotropic diffusion process on the region RegionRegionRegionRegionRegionregion of the input image ImageImageImageImageImageimage with the objective of completing discontinuous image edges diffusively by increasing the coherence of the image structures contained in ImageImageImageImageImageimage and without smoothing these edges perpendicular to their dominating direction. The mechanism is the same as in the operator coherence_enhancing_diffcoherence_enhancing_diffCoherenceEnhancingDiffcoherence_enhancing_diffCoherenceEnhancingDiffCoherenceEnhancingDiff, which is based on a discretization of the anisotropic diffusion equation

  u_t = div(G(u) grad u)

formulated by Weickert. With a 2x2 coefficient matrix G that depends on the gray values in ImageImageImageImageImageimage, this is an enhancement of the mean curvature flow or intrinsic heat equation

  u_t = div(grad u/|grad u|) |grad u| = curv(u) |grad u|

on the gray value function u defined by the input image ImageImageImageImageImageimage at a time t_0 = 0. The smoothing operator mean_curvature_flowmean_curvature_flowMeanCurvatureFlowmean_curvature_flowMeanCurvatureFlowMeanCurvatureFlow is a direct application of the mean curvature flow equation. With the operator inpainting_mcfinpainting_mcfInpaintingMcfinpainting_mcfInpaintingMcfInpaintingMcf, it can also be used for image inpainting. The discrete diffusion equation is solved in IterationsIterationsIterationsIterationsIterationsiterations time steps of length ThetaThetaThetaThetaThetatheta, so that the output image InpaintedImageInpaintedImageInpaintedImageInpaintedImageInpaintedImageinpaintedImage contains the gray value function at the time IterationsIterationsIterationsIterationsIterationsiterations * ThetaThetaThetaThetaThetatheta .

To detect the image direction more robustly, in particular on noisy input data, an additional isotropic smoothing step can precede the computation of the gray value gradients. The parameter SigmaSigmaSigmaSigmaSigmasigma determines the magnitude of the smoothing by means of the standard deviation of a corresponding Gaussian convolution kernel, as used in the operator isotropic_diffusionisotropic_diffusionIsotropicDiffusionisotropic_diffusionIsotropicDiffusionIsotropicDiffusion for isotropic image smoothing.

Similar to the operator inpainting_mcfinpainting_mcfInpaintingMcfinpainting_mcfInpaintingMcfInpaintingMcf, the structure of the image data in RegionRegionRegionRegionRegionregion is simplified by smoothing the level lines of ImageImageImageImageImageimage. By this, image errors and unwanted objects can be removed from the image, while the edges in the neighborhood are extended continuously. This procedure is called image inpainting. The objective is to introduce a minimum amount of artefacts or smoothing effects, so that the image manipulation is least visible to a human beholder.

While the matrix G is given by

  G_MCF(u) = I - (grad u (grad u)^T)/|grad u|^2

in the case of the operator inpainting_mcfinpainting_mcfInpaintingMcfinpainting_mcfInpaintingMcfInpaintingMcf, where I denotes the unit matrix, G_MCF is again smoothed componentwise by a Gaussian filter of standard deviation RhoRhoRhoRhoRhorho for coherence_enhancing_diffcoherence_enhancing_diffCoherenceEnhancingDiffcoherence_enhancing_diffCoherenceEnhancingDiffCoherenceEnhancingDiff. Then, the final coefficient matrix

  G_CED = g_1((l_1-l_2)^2) w_1 (w_1)^T + g_2((l_1-l_2)^2) w_2 (w_2)^T

is constructed from the eigenvalues l_1,l_2 and eigenvectors w_1,w_2 of the resulting intermediate matrix, where the functions

  g_1(p) = 0.001
  g_2(p) = 0.001 + 0.999 exp(-1/p)

were determined empirically and taken from the publication of Weickert.

Hence, the diffusion direction in mean_curvature_flowmean_curvature_flowMeanCurvatureFlowmean_curvature_flowMeanCurvatureFlowMeanCurvatureFlow is only determined by the local direction of the gray value gradient, while G_CED considers the macroscopic structure of the image objects on the scale RhoRhoRhoRhoRhorho and the magnitude of the diffusion in coherence_enhancing_diffcoherence_enhancing_diffCoherenceEnhancingDiffcoherence_enhancing_diffCoherenceEnhancingDiffCoherenceEnhancingDiff depends on how well this structure is defined.

To achieve the highest possible consistency of the newly created edges with the image data from the neighbourhood, the gray values are not mirrored at the border of RegionRegionRegionRegionRegionregion to compute the convolution with the smoothing filter mask of scale RhoRhoRhoRhoRhorho on the pixels close to the border, although this would be the common approach for filter operators. Instead, the existence of gray values on a band of width ceil(3.1*Rho)+2 pixels around RegionRegionRegionRegionRegionregion is presumed and these values are used in the convolution. This means that RegionRegionRegionRegionRegionregion must keep this much distance to the border of the image matrix ImageImageImageImageImageimage. By involving the gray values and directional information from this extended area, it can be achieved that the continuation of the edges is not only continuous, but also smooth, which means without kinks. Please note that the inpainting progress is restricted to those pixels that are included in the ROI of the input image ImageImageImageImageImageimage. If the ROI does not include the entire region RegionRegionRegionRegionRegionregion, a band around the intersection of RegionRegionRegionRegionRegionregion and the ROI is used to define the boundary values.

To decrease the number of iterations required for attaining a satisfactory result, it may be useful to initialize the gray value matrix in RegionRegionRegionRegionRegionregion with the harmonic interpolant, a continuous function of minimal curvature, by applying the operator harmonic_interpolationharmonic_interpolationHarmonicInterpolationharmonic_interpolationHarmonicInterpolationHarmonicInterpolation to ImageImageImageImageImageimage before calling inpainting_cedinpainting_cedInpaintingCedinpainting_cedInpaintingCedInpaintingCed.



ImageImageImageImageImageimage (input_object)  (multichannel-)image(-array) objectHImageHImageHImageHImageXHobject (byte / uint2 / real)

Input image.

RegionRegionRegionRegionRegionregion (input_object)  region objectHRegionHRegionHRegionHRegionXHobject

Inpainting region.

InpaintedImageInpaintedImageInpaintedImageInpaintedImageInpaintedImageinpaintedImage (output_object)  image(-array) objectHImageHImageHImageHImageXHobject * (byte / uint2 / real)

Output image.

SigmaSigmaSigmaSigmaSigmasigma (input_control)  real HTupleHTupleHTupleVARIANTHtuple (real) (double) (double) (double) (double) (double)

Smoothing for derivative operator.

Default value: 0.5

Suggested values: 0.0, 0.1, 0.5, 1.0

Restriction: Sigma >= 0

RhoRhoRhoRhoRhorho (input_control)  real HTupleHTupleHTupleVARIANTHtuple (real) (double) (double) (double) (double) (double)

Smoothing for diffusion coefficients.

Default value: 3.0

Suggested values: 0.0, 1.0, 3.0, 5.0, 10.0, 30.0

Restriction: Rho >= 0

ThetaThetaThetaThetaThetatheta (input_control)  real HTupleHTupleHTupleVARIANTHtuple (real) (double) (double) (double) (double) (double)

Time step.

Default value: 0.5

Suggested values: 0.1, 0.2, 0.3, 0.4, 0.5

Restriction: 0 < Theta <= 0.5

IterationsIterationsIterationsIterationsIterationsiterations (input_control)  integer HTupleHTupleHTupleVARIANTHtuple (integer) (int / long) (Hlong) (Hlong) (Hlong) (Hlong)

Number of iterations.

Default value: 10

Suggested values: 1, 5, 10, 20, 50, 100, 500

Restriction: Iterations >= 1

Example (HDevelop)

read_image (Image, 'fabrik')
gen_rectangle1 (Rectangle, 270, 180, 320, 230)
harmonic_interpolation (Image, Rectangle, InpaintedImage, 0.01)
inpainting_ced (InpaintedImage, Rectangle, InpaintedImage2, \
                0.5, 3.0, 0.5, 1000)


harmonic_interpolationharmonic_interpolationHarmonicInterpolationharmonic_interpolationHarmonicInterpolationHarmonicInterpolation, inpainting_ctinpainting_ctInpaintingCtinpainting_ctInpaintingCtInpaintingCt, inpainting_anisoinpainting_anisoInpaintingAnisoinpainting_anisoInpaintingAnisoInpaintingAniso, inpainting_mcfinpainting_mcfInpaintingMcfinpainting_mcfInpaintingMcfInpaintingMcf, inpainting_textureinpainting_textureInpaintingTextureinpainting_textureInpaintingTextureInpaintingTexture


J. Weickert, V. Hlavac, R. Sara; “Multiscale texture enhancement”; Computer analysis of images and patterns, Lecture Notes in Computer Science, Vol. 970, pp. 230-237; Springer, Berlin; 1995.
J. Weickert, B. ter Haar Romeny, L. Florack, J. Koenderink, M. Viergever; “A review of nonlinear diffusion filtering”; Scale-Space Theory in Computer Vision, Lecture Notes in Comp. Science, Vol. 1252, pp. 3-28; Springer, Berlin; 1997.



ClassesClassesClassesClasses | | | | Operators