This project is devoted to a sophisticated and effective approach to the signal restoration problem based on local transform-domain filtering (LTDF) applied to image and video signals. The local filtering assumes stationarity and homogeneity of the visual signal within local patches in order to produce sparse local representation by applying a unitary transform. These assumptions are found to be satisfied readily due to the characteristics of natural image and video signals, which are otherwise very unlikely from global perspective.
We propose an effective solution for the problem of reconstructing image and video signals corrupted with additive white Gaussian noise (AWGN). Our denoising method is based on effective filtering in 3D transform domain by combining sliding-window transform processing with block-matching. We process noisy video and image signals in sliding local windows (blocks). We apply block matching for each processed block in order to find neighboring blocks that exhibit similarity to it. The search-neighborhood in case of video denoising is a local 3D spatial-temporal neighborhood. The selected (matched) blocks are stacked together to form a 3D array. In this manner we induce high correlation along the dimension of the array in which the blocks are stacked. We exploit this correlation by applying a 3D decorrelating unitary transform which produces a sparse representation of the true signal in 3D transform domain. We achieve efficient noise attenuation by applying a shrinkage operator (e.g. hard-thresholding or Wiener filtering) on the transform coefficients. This results in improved denoising performance and effective detail preservation in the local estimates of the matched blocks, which are reconstructed by an inverse 3D transform of the filtered coefficients. After processing all blocks, we compute the final estimate of the signal as a weighted average of all overlapping local block-estimates. Because of this overlap, the representation of the signal by the local estimates is overcomplete which helps to avoid blocking artifacts and further improves the estimation ability.
Currently, we actively work on improving the adaptivity of the LTDF technique to the structures of the true signal. We do this by restricting the support of the transforms to regions where stationarity of the signal is readily fulfilled. In this manner we ensure high correlation along the spatial dimension, which improves the energy compaction of the local unitary transforms. We propose to suppress AWGN in variable-sized sliding 3D processing windows, rather than using fixed-sized ones. For every spatial position in each frame we use a block-matching algorithm to collect highly correlated blocks from neighboring frames and form 3D arrays for all predefined window sizes by stacking the matched blocks. An optimal window size is then selected according to the ICI rule and the AWGN is then attenuated by coefficient shrinkage in the 3D transform domain in the manner described above.
Our experiments show that the proposed algorithms outperform all, known to us, image and video denoising methods, both in terms of objective criteria and visual quality. Our approach provides a generalized restoration framework that allows for effective computational scalability by controlling the level overcompleteness.
The project is lead by Professor Karen Egiazarian. The main contributors are:
Rusanovskyy, Dmytro, MSc
Dabov, Kostadin, BSc
Dabov, K., A. Foi, V. Katkovnik, and K. Egiazarian, "Image denoising with block-matching and 3D filtering," in Electronic Imaging'06, Proc. SPIE 6064, no. 6064A-30, San Jose, California USA, 2006.
Rusanovskyy, D., K.Dabov, and K. Egiazarian, "Moving-Window Varying Size 3D Transform-Based Video Denoising," in Int'l Workshop on Video Process. and Quality Metrics, VPQM'06, Scottdale, USA, 2006.
D. Rusanovskyy and K. Egiazarian, "Video Denoising Algorithm in Sliding 3D DCT Domain," in Advanced Concepts for Intelligent Video Systems, ACIVS'05, Antwerp, Belgium, September 20-23, 2005.