Modeling the Blur/Noise Trade-Off in Image Restoration

random motion point-spread functions


INTRODUCTION SOFTWARE PEOPLE LINKS REFERENCES


Problem

Motion blur and noise are strictly related by the exposure time: photographers, before acquiring pictures of moving objects or dim scenes, always consider whether motion blur may occur (for example, due to scene or camera motion), and carefully set the exposure time.

The trade-off is between long exposures that reduce the noise at the cost of increasing the blur, and short exposures that reduce the blur at the cost of increasing the noise. Often there is no satisfactory compromise, and the captured image is inevitably too blurry or too noisy (see Figure 1).


Blur/noise trade-off (uniform motion)
Blur/noise trade-off (camera shake)
Figure 1. An illustration of the blur/noise trade-off in image acquisition under uniform motion (top) and camera shake (bottom): for each of the two scenarios, the raw-data observations were acquired using different exposure times along the same motion trajectory (an estimate of the corresponding point-spread function is shown at the lower left of each image). Short exposures are corrupted by overwhelming noise, while long exposures are dominated by blur. The image intensities have been normalized for the sake of visualization. Further details concerning the image acquisition settings can be found here.


Possible Remedy

Under many circumstances, blur and noise can be compensated exclusively by means of digital image restoration. Typically, the effectiveness of any restoration algorithm depends on the blur and noise amounts, as shown in Figure 2.

A desirable feature for a digital camera would be to determine -- before shooting a picture -- which is the exposure time that balances at best the blur and noise, in order to ease the restoration task. Our results go in this direction, as we investigate the blur/noise trade-off to determine the optimal exposure time when images are affected by motion blur.


Effect of the blur/noise trade-off on image deconvolution (uniform motion)
Effect of the blur/noise trade-off on image deconvolution (camera shake)
Figure 2. The results of the Anisotropic LPA-ICI Deconvolution [Foi et al., 2005] applied to the blurred/noisy observations in Figure 1. In case of uniform motion (top) the restoration errors show a clear minimum, which corresponds to the optimal exposure time: beyond such optimal value, the error increases. In case of camera shake (bottom) the restoration error practically stabilizes after reaching its point of minimum, and thus even long exposure times can be allowed. Therefore different scenarios require different acquisition strategies to maximize the quality of the restored image.


Optimal Exposure Time and Restoration-Error Model

For the special case of uniform rectilinear blur, we propose a simple mathematical formulation based on the signal-to-noise ratio in Fourier domain. This allows us to study how the performance of any image deconvolution algorithm varies with respect to the exposure time, and demonstrates the existence and finiteness of an optimal exposure time that maximizes the restoration performance, balancing the amount of blur and noise in the observation.

To investigate more general classes of motion blur, such as blur due to camera shake, we developed a methodology for deriving a statistical model of the restoration error for a given deblurring algorithm in case of arbitrary motions, including random motion.

More specifically, each restoration-error model describes how the expected restoration error of a particular image-deblurring algorithm varies as the blur due to camera motion develops over time along with the PSF trajectory. Each model simultaneously takes into account the exposure time, its interplay with the sensor noise, and the motion randomness.

Restoration-error models allow to design customized image acquisition strategies to maximize the performance of the deblurring algorithm that will be used on the acquired pictures. In particular, in controlled imaging scenarios where the evolution of the PSF trajectory along with the exposure time can be statistically studied or analytically formulated, the restoration-error model can tell whether there exists an optimal exposure, i.e. an exposure time that minimizes the restoration error achievable by the corresponding deblurring algorithm. Then, whenever it exists, the optimal exposure time can be determined from the restoration-error model.

Figure 3 illustrates the restoration-error model for the Anisotropic LPA-ICI Deconvolution [Foi et al., 2005].


Restoration-Error Model for the Anisotropic LPA-ICI Deconvolution
Figure 3. A view of the restoration-error model computed for the Anisotropic LPA-ICI Deconvolution [Foi et al., 2005]. The red and green curves represent the temporal evolution of the RMSE of images restored from rectilinear uniform motion and from camera shake, respectively. The trends of these curves well interpret the restoration performance achieved in the images of Figure 2.

Software (hosted at Politecnico di Milano)

The Blur/Noise Trade-Off DataSet new
Matlab Package for Generating Synthetic Observations According to the Blur/Noise Trade-Off new



Any unauthorized use of the provided software and files for industrial or profit-oriented activities is expressively prohibited. By downloading any of the files contained in this site, you implicitly agree to all the terms of the TUT limited license. Please read the TUT limited license PDF before you proceed with downloading any of the files.



People People

  Giacomo Boracchi
  Alessandro Foi



Links Links

   LASIP Anisotropic Nonparametric Image Restoration DemoBox (Anisotropic LPA-ICI Deconvolution)
   Deconvolution Using a Sparse Natural-Image Prior



References References

PDFG. Boracchi and A. Foi, “Modeling the Performance of Image Restoration from Motion Blur”, IEEE Trans. Image Process. vol. 21, no. 8, pp. 3502-3517, August 2012.  DOI doi:10.1109/TIP.2012.2192126

PDFG. Boracchi and A. Foi, “Uniform motion blur in Poissonian noise: blur/noise trade-off”, IEEE Trans. Image Process., vol. 20, no. 2, pp. 592-598, February 2011.  DOI doi:10.1109/TIP.2010.2062196 PDFraw images (86-Mb zip file)



back to main page