Local 3D Filtering for Still- and Motion-Image Restoration

The local 3D filtering (LF3D) project is devoted to development of highly-effective techniques for still- and motion-image restoration. The research is focused on algorithms based on filtering in local 3D transform domain.

Introduction

Applications

People

Publications

Introduction

Digital still- and motion-images (video) are essential part of contemporary life. Being produced by a variety of physical devices, they are a crucial part of entertainment, medical, scientific, industrial, military, and other applications. However, imperfect acquisition instruments, natural phenomena, transmission errors, or inept processing distort the visual signal so that its true intensities are impaired. The distortions include but are not limited to: blurring, additive and signal-dependent noise, nonuniform illumination, and geometric distortions. Often, they heavily obstruct, or even make impossible, a subsequent processing, analysis, or interpretation of the visual information. Therefore, substantial research has been conducted in the past two decades on the problem of reconstructing estimates of image and video signals from distorted data. This work belongs to the field of signal restoration.

This project is devoted to a sophisticated and effective approach to the signal restoration problem based on local transform-domain filtering (LTDF) applied to image and video signals. The local filtering assumes stationarity and homogeneity of the visual signal within local patches in order to produce sparse local representation by applying a unitary transform. These assumptions are found to be satisfied readily due to the characteristics of natural image and video signals, which are otherwise very unlikely from global perspective.

We propose an effective solution for the problem of reconstructing image and video signals corrupted with additive white Gaussian noise (AWGN). Our denoising method is based on effective filtering in 3D transform domain by combining sliding-window transform processing with block-matching. We process noisy video and image signals in sliding local windows (blocks). We apply block matching for each processed block in order to find neighboring blocks that exhibit similarity to it. The search-neighborhood in case of video denoising is a local 3D spatial-temporal neighborhood. The selected (matched) blocks are stacked together to form a 3D array. In this manner we induce high correlation along the dimension of the array in which the blocks are stacked. We exploit this correlation by applying a 3D decorrelating unitary transform which produces a sparse representation of the true signal in 3D transform domain. We achieve efficient noise attenuation by applying a shrinkage operator (e.g. hard-thresholding or Wiener filtering) on the transform coefficients. This results in improved denoising performance and effective detail preservation in the local estimates of the matched blocks, which are reconstructed by an inverse 3D transform of the filtered coefficients. After processing all blocks, we compute the final estimate of the signal as a weighted average of all overlapping local block-estimates. Because of this overlap, the representation of the signal by the local estimates is overcomplete which helps to avoid blocking artifacts and further improves the estimation ability.

Currently, we actively work on improving the adaptivity of the LTDF technique to the structures of the true signal. We do this by restricting the support of the transforms to regions where stationarity of the signal is readily fulfilled. In this manner we ensure high correlation along the spatial dimension, which improves the energy compaction of the local unitary transforms. We propose to suppress AWGN in variable-sized sliding 3D processing windows, rather than using fixed-sized ones. For every spatial position in each frame we use a block-matching algorithm to collect highly correlated blocks from neighboring frames and form 3D arrays for all predefined window sizes by stacking the matched blocks. An optimal window size is then selected according to the ICI rule and the AWGN is then attenuated by coefficient shrinkage in the 3D transform domain in the manner described above.

Our experiments show that the proposed algorithms outperform all, known to us, image and video denoising methods, both in terms of objective criteria and visual quality. Our approach provides a generalized restoration framework that allows for effective computational scalability by controlling the level overcompleteness.