Finnish Academy of Science, Project No. 287150, 2015-2019

Name of responsible leader: Professor Karen Egiazarian.
Title of research project: "Sparse complex domain wave field imaging ("SPARSE").

Keywords: Digital holography, Compressive sensing, Interferometric imaging, Inverse problems, Fourier optics and signal processing, Phase retrieval, Sparse and compressive sensing computational imaging, Superresolution.

Project description

A laser generated monochromatic coherent light field is an unique instrument for precise measuring and sensing in different scenarios varying from the precise control of surfaces to accurate visualization of structures invisible in the ordinary light. It becomes possible because any inhomogeneity in the media of the laser beam propagation results in the path-length (phase) shift of the beam. Two types of the problems are studied: precise reconstruction and precise superresolution of the wave field and the phase from noisy intensity measurements.
Revolutionary novel approaches and novel computational techniques will be developed for these problems in the project. These novel approaches and methods will constitute a clear breakthrough, in accuracy and quality of visualization, as compared with the current state-of-the-art. These results are beneficial for nanoscience and nanotechnology used across chemistry, biology, physics, materials science, optical metrology and engineering.

Funding period 1.9.2015-30.8.2019, Academy of Finland, Decision No. 287150.

  1. Katkovnik V., Egiazarian K, Multi-frequency phase retrieval from noisy data, Proceedings of EUSIPCO 2018, Rome, 2018, and, 5 p. (code) (paper) (web page)

    Abstract. The phase retrieval from multi-frequency intensity (power) observations is considered. The object to be reconstructed is complex-valued. A novel algorithm is presented that accomplishes both the object phase (absolute phase) retrieval and denoising for Poissonian and Gaussian measurements. The algorithm is derived from the maximum likelihood formulation with Block Matching 3D (BM3D) sparsity priors. These priors result in two filtering: one is in the complex domain for complex-valued multi-frequency object images and another one in the real domain for the object phase. The algorithm is iterative with alternating projections between the object and measurement variables. The simulation experiments are produced for Fourier transform image formation and random phase modulations of the object, then the observations are random object diffraction patterns. The results demonstrate the success of the algorithm for reconstruction of the complex phase objects with the high-accuracy performance even for very noisy data.

  2. V. Katkovnik, M. Ponomarenko and K. Egiazarian, "Complex-valued image denosing based on group-wise complex-domain sparsity", arXiv:1711.00362, 2017, 36 p. (code) (paper) (web page)

    Abstract. Phase imaging and wavefront reconstruction from noisy observations of complex exponent is a topic of this paper. It is a highly non-linear problem because the exponent is a 2 Pi-periodic function of phase. The reconstruction of phase and amplitude is difficult. Even with an additive Gaussian noise in observations distributions of noisy components in phase and amplitude are signal dependent and non-Gaussian. Additional difficulties follow from a prior unknown correlation of phase and amplitude in real life scenarios. In this paper, we propose a new class of non-iterative and iterative complex domain filters based on group-wise sparsity in complex domain. This sparsity is based on the techniques implemented in Block-Matching 3D filtering (BM3D) and 3D/4D High-Order Singular Decomposition (HOSVD) exploited for spectrum design, analysis and filtering. The introduced algorithms are a generalization of the ideas used in the CD-BM3D algorithms presented in our previous publications. The algorithms are implemented as a MATLAB Toolbox. The efficiency of the algorithms is demonstrated by simulation tests.

  3. V. Katkovnik, "Phase retrieval from noisy data based on sparse approximation of object phase and amplitude", 2017, (demo code) (paper) (web page)

    Abstract. A variational approach to reconstruction of phase and amplitude of a complex-valued object from Poissonian intensity observations is developed. The observation model corresponds to the typical optical setups with a phase modulation of wavefronts. The transform domain sparsity is applied for the amplitude and phase modeling. It is demonstrated that this modeling results in the essential advantage of the developed algorithm for heavily noisy observations corresponding to a short exposure time in optical experiments. We consider also two simplified versions of this algorithm where the sparsity modeling of phase and amplitude is omitted. In the simulation study we compare the developed algorithms versus the Gerchberg-Saxton and truncation Wirtinger flow algorithms. The latter algorithm being the maximum likelihood based is the state-of-the-art for the phase retrieval from Poissonian observations. For noisy and very noisy observations the proposed algorithm demonstrates a valuable advantage.

  4. Katkovnik V., Hogasten N. , Egiazarian K., A novel two- and multi-level binary phase mask design for enhanced depth-of-focus arXiv preprint arXiv:1707.07752, 2017, 15 p. (paper)

    Abstract. This paper introduces a two-and multi-level binary phase mask design for improved depth of focus. A novel technique is proposed incorporating cubic and generalized cubic wavefront coding (WFC). The obtained system is optical-electronic requiring computational deblurring post-processing, in order to obtain a sharp image from the observed blurred data. A midwave infrared (MWIR) system is simulated showing that this design will produce high quality images even for large amounts of defocus. It is furthermore shown that this technique can be used to design a flat, single optical element, systems where the phase mask performs both the function of focusing and phase modulation. It is demonstrated that in this lensless design the WFC coding components can be omitted and WFC effects are achieved as a result of the proposed algorithm for phase mask design which uses the quadratic phase of the thin refractive lens as the input signal.

  5. Katkovnik, V., Shevkunov, I., Petrov, N.V. & Egiazarian, K. Computational super-resolution phase retrieval from multiple phase-coded diffraction patterns: simulation study and experiments. Optica, 4(7):786-794, Optical Society of America, 2017. (code) (paper)

    Abstract. Computational super-resolution inverse diffraction phase retrieval is considered. The optical setup is lensless with a spatial light modulator (SLM) for aperture phase coding. The paper is focused on experimental tests of the Super-Resolution Sparse Phase Amplitude Retrieval (SR- SPAR) algorithm. We start from simulation tests and go to physical experiments. Both sim- ulation tests and experiments demonstrate a good quality imaging for super-resolution with the factor 4, which means that the computational pixels of the reconstructed object are 4 times smaller than the sensor pixels.

  6. Katkovnik, V. & Egiazarian, K. Sparse super-resolution phase retrieval from phase-coded noisy intensity patterns. Optical Engineering, 56(9), 094103, doi:10.1117/1.OE.56.9.094103, 2017. (draft paper) (demo code)

    Abstract. We consider computational super-resolution inverse diffraction problem for phase retrieval from phase-coded intensity observations. The optical setup includes a thin lens and a spatial light modulator (SLM) for phase coding. The designed algorithm is targeted on optimal solution for Poissonian noisy observations. One of the essential instruments of this design is a complex-domain sparsity applied for complex valued object (phase and amplitude) to be reconstructed. Simulation experiments demonstrate that good quality imaging can be achieved for high-level of the super-resolution with factor 32, what means that the pixels of the reconstructed object pixels are 32 times smaller than sensor and SLM pixels. In wavelength this super-resolution corresponds to the object pixels as small as a quarter of the wavelength.

  7. Katkovnik, V., Ponomarenko, M. & Egiazarian, K. Sparse approximations in complex domain based on BM3D modeling. Signal Processing, Elsevier, 141:96-108, December 2017. (demo code) (draft paper) (webpage)

    Abstract. In this paper the concept of sparsity for complex-valued variables is introduced in the following three types: directly in the complex domain and for two real-valued pairs phase/amplitude and real/imaginary parts of complex variables. The nonlocal block-matching technique is used for sparsity implementation and filter design for each type of sparsity. These filters are complex domain generalizations of Block Matching 3D collaborative (BM3D) filters based on the high-order singular value decomposition (HOSVD) used in order to generate group-wise adaptive analysis/synthesis transforms. Complex domain denoising is developed as a test-problem for comparison of the designed filters and the different types of sparsity modeling.

  8. Katkovnik, V. & Egiazarian, K. Sparse phase imaging based on complex domain nonlocal BM3D techniques. Digital Signal Processing, Academic Press, 63:72-85, 2017. (code) (paper) (webpage)

    Abstract. The paper is addressed to 2D phase and amplitude estimation of complex-valued signals - that is, in particular, to estimation of modulo-2π interferometric phase images from periodic and noisy observations. These degradation mechanisms make phase image estimation a challenging problem. A sparse nonlocal data-adaptive imaging formalized in complex domain is used for phase and amplitude image reconstruction. Following the procedure of patch-based technique, the image is partitioned into small overlapping square patches. Block Matching Three Dimensional (BM3D) technique is developed for forming complex domain sparse spectral representations of complex-valued data. High Order Singular Value Decomposition (HOSVD) applied to BM3D groups enables the design of the orthonormal complex domain 3D transforms which are data adaptive and different for each BM3Ds group. An iterative version of the complex domain BM3D is designed from variational formulation of the problem. The convergence of this algorithm is shown. The effectiveness of the new sparse coding based algorithms is illustrated in simulation experiments where they demonstrate the state-of-the-art performance.

  9. Katkovnik V., Ponomarenko M. & Egiazarian K., Complex domain nonlocal block-matching denoising based on high-order singular value decomposition (HOSVD), 25th European Signal Processing Conference (EUSIPCO), 2017, pp.748-752. (draft paper)

    Abstract. Block matching 3D collaborative filtering (BM3D) is one of the most popular denoising technique based on data sparsity concept applied to specially structured data. In this paper we develop this technique for complex domain, i.e. for application to complex-valued data. Sparsity as an approximation technique can be addressed directly to complex-valued variables or to real-valued pairs phase/amplitude and real/imaginary parts of complex-valued variables. As a result we arrive to various ways of development and obtain a set of quite different algorithms. The algorithms proposed in this paper are composed from two components: nonlocal patch-wise grouping and highorder singular value decomposition (HOSVD) for grouped data processing. The latter gives data adaptive complex-valued bases for complex-valued data or real-valued bases for joint processing of the pairs phase/amplitude, real/imaginary parts of complexvalued variables. Comparative study of the developed algorithms is produced in order to select the most efficient ones.

  10. Ponomarenko M., Pismenskova M. & Egiazarian K., Weighted MSE Based Spatially Adaptive BM3D, 5th European Signal Processing Conference (EUSIPCO), 2017, pp.763-767. (draft paper)

    Abstract. Weighted MSE (wMSE), recently introduced modification of MSE, is an image quality metric used to estimate visual quality of filtered images. It provides better than MSE correspondence to a human perception in consideration of distortions introduced by image filters. In this paper, wMSE is used both as a criterion to evaluate filtering efficiency of the modification of BM3D filter with spatially varying parameters, as well as to train a specially designed neural network to predict filters’ parameters. Extensive analysis on three image datasets demonstrates that the proposed modification of BM3D provides lower values of wMSE than those of BM3D, both effectively suppressing noise in homogeneous regions as well as preserving fine details and texture.

  11. Katkovnik, V., Shevkunov, I., Petrov, N.V. & Egiazarian, K. Computational wavelength resolution for in-line lensless holography: phase-coded diffraction patterns and wavefront group-sparsity. In Digital Optical Technologies 2017, Proceedings of SPIE Vol. 10335:1033509, Munich, 26–28 June 2017.

    Abstract. In-line lensless holography is considered with a random phase modulation at the object plane. The forward wavefront propagation is modelled using the Fourier transform with the angular spectrum transfer function. The multiple intensities (holograms) recorded by the sensor are random due to the random phase modulation and noisy with Poissonian noise distribution. The algorithm is designed for optimal phase/amplitude reconstruction from Poissonian data. It is shown by computational experiments that high-accuracy reconstructions can be achieved with resolution going up to the two thirds of the wavelength. With respect to the sensor pixel size it is a 32 super-resolution. The algorithm is designed for optimal super-resolution phase/amplitude reconstruction from Poissonian data. The algorithm design is based on the general methodology developed for phase retrieval with a pixel-wise resolution in V. Katkovnik, ”Phase retrieval from noisy data based on sparse approximation of object phase and amplitude”,

  12. Egiazarian K. & Katkovnik V., Alternating Group Sparsity for Image Restoration, Workshop SPARS 2017, Lisbon, on June 5-8, 2017. (paper)

  13. Egiazarian K. & Katkovnik V., Complex Domain Nonlocal Group-Wise Sparsity: Toward Wavelength Super-Resolution Phase Imaging in Coherent Optics, Workshop SPARS 2017, Lisbon, on June 5-8, 2017. (paper)

  14. Katkovnik V., Complex domain (phase/amplitude) sparsity and data processing in coherent optics, Summer School on “Digital Holography and related Techniques”, Stuttgart, 21st to 23rd of June 2017.

    Abstract. In optics, a monochromatic wavefront is represented as a complex-valued signal, where amplitude and phase are unknown variables of interest. This wavefront phase cannot be measured directly because all measurement instruments are sensitive with respect to the intensity but not to phase. Accordingly, one of the main targets of data processing is to extract phase information from measured intensities. For instance, in interferometry and holography the phase is retrieved using special reference wavefronts. Under modeling of wavefront, we understand any mathematical tools for prediction, interpolation, denoising, etc., installing links between the values of wavefronts at different coordinates. In computational imaging, sparse and redundant representations (sparsity) have been successfully developed the last years as a general modeling instrument. It is based on the assumption that there exists a small number of basic functions such that image can be represented exactly or approximately with a good accuracy. In term of statistics, a sparse representation can be thought as a low-order parametric approximation. In this classical form, the sparsity concept is used in parametric approximations just zeroing small amplitude components. A specific point of the sparsity is that the sparse basis is unknown in advance and should be designed. In this lecture, we consider methods and algorithms for sparse modeling of wavefronts and their applications for phase imaging.

  15. Katkovnik, V., Shevkunov, I., Petrov, N.V. & Egiazarian, K. High-accuracy off-axis wavefront reconstruction from noisy data: local least square with multiple adaptive windows. Optics express, 24(22):25068-25083, Optical Society of America, 2016. (paper) (webpage)

    Abstract. A variational algorithm to object wavefront reconstruction from noisy intensity observations is developed for the off-axis holography scenario with imaging in the acquisition plane. The algorithm is based on the local least square technique proposed in paper [J. Opt.Soc. Am. A, 21, 367 (2004)]. First, multiple reconstructions of the wavefront are produced for various size and various directional windows applied for localization of estimation. At the second stage, a special statistical rule is applied in order to select the best window size estimate for each pixel of the image and for each of the directional windows. At the third final stage the estimates of the different directions obtained for each pixel are aggregated in the final one. Simulation experiments and real data processing prove that the developed algorithm demonstrate the performance of the extraordinary quality and accuracy for both the phase and amplitude of the object wavefront.

  16. Katkovnik, V. Sparse phase retrieval from noisy data: variational formulation and algorithms. In 3D Image Acquisition and Display: Technology, Perception and Applications, pages JT3A-42, 2016.

    Abstract. A variational approach to wavefront reconstruction from multiple noisy Poissonian intensity observations is developed. Sparse modeling of amplitude and absolute phase of the object is one of the key elements of the derived algorithm.

  17. Trujillo-Sevilla, J., Katkovnik, V., Javidi, B. & Rodriguez-Ramos, J. Restoring Integral Images from Focal Stacks Using Compressed Sensing Techniques. Journal of Display Technology, 12(7):349-354, IEEE, 2016.

    Abstract. This paper contains an original development of the compressed sensing technique for restoring integral images from a number of observed 2D images. The proposed data acquisition uses a conventional camera equipped with a horizontal 1D mask placed in the pupil plane of the lens. The compressed sensing style algorithm developed is based on a sparsity hypothesis imposed on 2D cross sections of the light field.

  18. Sahin, E., Katkovnik, V. & Gotchev, A. Super-resolution in a defocused plenoptic camera: a wave-optics-based approach. Optics letters, 41(5):998-1001, Optical Society of America, 2016. (paper)

    Abstract. Plenoptic cameras enable the capture of a light field with a single device. However, with traditional light field rendering procedures, they can provide only low-resolution two-dimensional images. Super-resolution is considered to overcome this drawback. In this study, we present a super-resolution method for the defocussed plenoptic camera (Plenoptic 1.0), where the imaging system is modeled using wave optics principles and utilizing low-resolution depth information of the scene. We are particularly interested in super-resolution of in-focus and near in-focus scene regions, which constitute the most challenging cases. The simulation results show that the employed wave-optics model makes super-resolution possible for such regions as long as sufficiently accurate depth information is available.

  19. Egiazarian, K. & Katkovnik, V. Single image super-resolution via BM3D sparse coding. In Signal Processing Conference (EUSIPCO), 2015 23rd European, pages 2849-2853, 2015. (paper)

    Abstract. In this paper, a novel single image super-resolution (SISR) algorithm is proposed. It is based on the BM3D (Block- Matching and 3D filtering) paradigm, where both sparsity and nonlocal patch self-similarity priors are utilized. The algorithm is derived from a variational formulation of the problem and has a structure typical for iterative back-projection super-resolution methods. They are characterized by updating high-resolution image which is calculated using the previous estimate and upsampled low-resolution error. The developed method is thoroughly compared with the state-of-the-art SISR both for noiseless and noisy data, demonstrating superior performance objectively and subjectively.

  20. Katkovnik, V., Shevkunov, I., Petrov, N. & Egiazarian, K. Wavefront reconstruction in digital off-axis holography via sparse coding of amplitude and absolute phase. Optics letters, 40(10):2417-2420, Optical Society of America, 2015. (paper)

    Abstract. This work presents the new method for wavefront reconstruction from a digital hologram recorded in off-axis configuration. The main feature of the proposed algorithm is a good ability for noise filtration due to the original formulation of the problem taking into account the presence of noise in the recorded intensity distribution and the sparse phase and amplitude reconstruction approach with the data-adaptive block-matching 3D technique. Basically, the sparsity assumes that low dimensional models can be used for phase and amplitude approximations. This low dimensionality enables strong suppression of noisy components and accurate revealing of the main features of the signals of interest. The principal point is that dictionaries of these sparse models are not known in advance and reconstructed from given noisy observations in a multiobjective optimization procedure. We show experimental results demonstrating the effectiveness of our approach.

  21. Katkovnik, V., Shevkunov, I.A., Petrov, N.V. & Egiazarian, K. Sparse approximations of phase and amplitude for wave field reconstruction from noisy data. In SPIE Optics+ Optoelectronics, pages 950802-950802, 2015. (paper)

    Abstract. The topic of sparse representations (SR) of images has attracted tremendous interest from the research community in the last ten years. This interest stems from the fundamental role that the low dimensional models play in many signal and image processing areas, i.e., real world images can be well approximated by a linear combination of a small number of atoms (i.e., patches of images) taken from a large frame, often termed dictionary. The principal point is that these large dictionaries as well as the elements of these dictionaries taken for approximation are not known in advance and should be taken from given noisy observations. The sparse phase and amplitude reconstruction (SPAR) algorithm has been developed for monochromatic coherent wave field reconstruction, for phase-shifting interferometry and holography. In this paper the SPAR technique is extended to off-axis holography. Pragmatically, SPAR representations are result in design of efficient data-adaptive filters. We develop and study the algorithm where these filters are applied for denoising of phase and amplitude in object and sensor planes. This algorithm is iterative and developed as a maximum likelihood optimal solution provided that the noise in intensity measurements is Gaussian. The multiple simulation and real data experiments demonstrate the advance performance of the new technique.

  22. Hongxing, H., Bioucas-Dias, J.M. & Katkovnik, V. Interferometric phase image estimation via sparse coding in the complex domain. IEEE Transactions on Geoscience and Remote Sensing, 53(5):2587-2602, IEEE, 2015. (Matlab code) (paper) (webpage)

    Abstract. This paper addresses interferometric phase image estimation, i.e., the estimation of phase modulo-2π images from sinusoidal 2π -periodic and noisy observations. These degradation mechanisms make interferometric phase image estimation a quite challenging problem. We tackle this challenge by reformulating the true estimation problem as a sparse regression, often termed sparse coding, in the complex domain. Following the standard procedure in patch-based image restoration, the image is partitioned into small overlapping square patches, and the vector corresponding to each patch is modeled as a sparse linear combination of vectors, termed the atoms, taken from a set called dictionary. Aiming at optimal sparse representations, and thus at optimal noise removing capabilities, the dictionary is learned from the data that it represents via matrix factorization with sparsity constraints on the code (i.e., the regression coefficients) enforced by the l1 norm. The effectiveness of the new sparse-coding-based approach to interferometric phase estimation, termed the SpInPHASE, is illustrated in a series of experiments with simulated and real data where it outperforms the state-of-the-art.

Go to Computational Imaging Group webpage...