Finnish Academy of Science, Project No. 287150, 2015-2019

Name of responsible leader: Professor Karen Egiazarian.
Title of research project: "Sparse complex domain wave field imaging ("SPARSE").

Keywords: Digital holography, Compressive sensing, Interferometric imaging, Inverse problems, Fourier optics and signal processing, Phase retrieval, Sparse and compressive sensing computational imaging, Superresolution.

Project description

A laser generated monochromatic coherent light field is an unique instrument for precise measuring and sensing in different scenarios varying from the precise control of surfaces to accurate visualization of structures invisible in the ordinary light. It becomes possible because any inhomogeneity in the media of the laser beam propagation results in the path-length (phase) shift of the beam. Two types of the problems are studied: precise reconstruction and precise superresolution of the wave field and the phase from noisy intensity measurements.
Revolutionary novel approaches and novel computational techniques will be developed for these problems in the project. These novel approaches and methods will constitute a clear breakthrough, in accuracy and quality of visualization, as compared with the current state-of-the-art. These results are beneficial for nanoscience and nanotechnology used across chemistry, biology, physics, materials science, optical metrology and engineering.

Funding period 1.9.2015-30.8.2019, Academy of Finland, Decision No. 287150.

Publications
  1. Kulya, M., Petrov, N.V., Tsypkin, A., Egiazarian, K. & Katkovnik, V. Hyperspectral data denoising for terahertz pulse time-domain holography. Optics Express, 27(13):18456-18476, Optical Society of America, 2019. (paper)

    Abstract. We investigated data denoising in hyperspectral terahertz pulse time-domain holography. Using the block-matching algorithms adapted for spatio-temporal and spatio-spectral volumetric data we studied and optimized parameters of these algorithms to improve phase image reconstruction quality. We propose a sequential application of the two algorithms oriented on work in temporal and spectral domains. Experimental data demonstrate the improvement in the quality of the resultant time-domain images as well as phase images and object's relief. The simulation results are proved by comparison with the experimental ones.

  2. Kocsis, P., Shevkunov, I., Katkovnik, V. & Egiazarian, K. Single exposure lensless subpixel phase imaging. In Digital Optical Technologies 2019, 11062:1106212, 2019. (web page)

    Abstract. Lensless phase-retrieval system with phase modulation of free propagation wavefront is proposed. Contrary to the traditional super-resolution phase-retrieval, the method in this paper requires a single observation only and uses advanced SR-SPAR iterative technique. Successful object imaging relies on modulation of the object wavefront with a random phase-mask, which generates enlarged intensity patterns, allowing us to extract more information than it is possible without such a mask. The achieved high-quality super-resolution phase-imaging is demonstrated by simulation-tests produced with the parameters corresponding to the physical prototype of the considered optical system.

  3. Katkovnik, V., Shevkunov, I., Claus, D., Pedrini, G. & Egiazarian, K. Hyperspectral phase imaging with denoising in SVD image subspace. In Digital Holography and Three-Dimensional Imaging, pages W1B-2, 2019. (webpage)

    Abstract. We propose a modified denoising algorithm for hyperspectral data. The algorithm is based on a complex domain block-matching 3D filter, on estimation of the noise correlation matrix and on dimension reduction of the Singular Value Decomposition (SVD) eigenspace.

  4. Kulya, M., Petrov, N., Egiazarian, K. & Katkovnik, V. Hyperspectral terahertz pulse time-domain holography: noise filtering. In Digital Holography and Three-Dimensional Imaging, pages Th4B-4, 2019. (webpage)

    Abstract. Using the block-matching denoising algorithms adapted to spatio-temporal and spatio-spectral volumetric data, we studied and optimized the parameters of these algorithms to improve the phase/amplitude image reconstruction in hyperspectral terahertz pulse time-domain holography. We propose a sequential application of two different algorithms oriented on work in temporal and spectral domains. Experimental data demonstrates essential improvement in the quality of the resulting phase imaging.

  5. Katkovnik, V., Ponomarenko, M. & Egiazarian, K. Lensless broadband diffractive imaging with improved depth of focus: wavefront modulation by multilevel phase masks. Journal of Modern Optics, 66(3):335-352, Taylor \& Francis, 2018. (paper) (webpage)

    Abstract. his paper introduces a novel lensless full color diffractive computational imaging system with a planar Multilevel Phase Mask (MPM) as a diffractive optical element (DOE). The novelty concerns: a methodology of MPM design for improved depth of focus (DoF); design of PSFs for RGB imaging and an inverse imaging algorithm with sparse color image modelling simultaneous for all RGB channels. MPMs are step-wise invariant. The cubic wavefront coding (WFC) is incorporated in MPMs with optimization of number of levels and width of invariant steps. This design of MPM makes the system robust with respect to defocus (improves DoF) and diminish chromatic aberrations typical for DOEs. Broadband multichannel test-images are exploited for design and testing of the lensless system. We consider two alternative optical setups: Wavelength Multiplexing (WM) and Wavelength Division (WD). In WM, the light beam is broadband multichannel with light sources radiating all wavelengths simultaneously and a CMOS sensor is equipped with a Bayer color filter array (CFA) for registration of spectral measurements. In this setup, a single MPM is designed for the broadband multichannel light beams. In WD, separate exposures of RGB channels are registered by a broadband grey-scale CCD sensor. Different MPMs are designed for each of the RGB channels. Simulation experiments demonstrate the essentially extended DoF of the designed lensless systems and the advanced accuracy and quality of imaging with respect to the corresponding WM and WD systems with refractive lenses. Due to robustness of the designed lensless system to chromatic aberrations, this advantage has a place even with respect to the lens-system.

  6. Cazac, V., Achimova, E., Abashkin, V., Meshalkin, A., Shevkunov, I. & Katkovnik, V. Noise minimizing by harmonic phase diffraction gratings in off-axis lensless digital holography. In Advanced Topics in Optoelectronics, Microelectronics, and Nanotechnologies IX, 10977:109771Z, 2018. (webpage)

    Abstract. Off-axis lensless holography is considered with a sinusoidal phase modulation at the object plane. The variational algorithm for phase and amplitude reconstruction is based on the algorithm proposed in the paper 1[V. Katkovnik, I. A. Shevkunov, N. V. Petrov, and K. Egiazarian, "Wavefront reconstruction in digital off-axis holography via sparse coding of amplitude and absolute phase", Opt. Lett. 40, 2417-2420 (2015)]. The forward wavefront propagation is modelled using the Fourier transform with the angular spectrum transfer function. The multiple intensities (holograms) recorded by the sensor vary in dependence to the angle of the phase diffraction grating. The i.i.d. Gaussian noise is added to observations to make them closer to real experimental conditions. The root mean square error (RMSE) values of the phase reconstructions were compared in two scenarios: with and without the diffraction grating. Computational experiments showed that with sinusoidal phase modulation RMSE values are decreased about 20%. These results support the conclusion on advantage of the proposed phase modulation gratings in off-axis lensless digital holography.

  7. Katkovnik, V., Ponomarenko, M. & Egiazarian, K. Optimization of Hybrid Optics with Multilevel Phase Mask for Improved Depth of Focus Broadband Imaging. In 2018 7th European Workshop on Visual Information Processing (EUVIP), pages 1-6, 2018. (webpage)

    Abstract. This paper is devoted to designing of hybrid refractive/diffractive optics for high-quality imaging with improved depth of focus (DoF). The novelty of the concept is in sharing of the optical power of the refractive lens between the lens and a multilevel phase mask (MLM) as a diffractive optical element (DOE). The efficiency of the design is confirmed by numerical results. Broadband multiwavelength test-images are exploited for the design and testing of the system. It is shown that the obtained hybrid optical system in combination with computational inverse imaging provides both a better image quality (due to the robustness of the designed optics to chromatic aberrations) and an extended depth of focus as compared with the refractive lens and the diffractive lensless design with MLM.

  8. Krishnan, J., Bioucas-Dias, J. & Katkovnik, V. Dictionary Learning Phase Retrieval from Noisy Diffraction Patterns. Sensors, 18(11):4006, Multidisciplinary Digital Publishing Institute, 2018. (paper) (webpage)

    Abstract. This paper proposes a novel algorithm for image phase retrieval, ie, for recovering complex-valued images from the amplitudes of noisy linear combinations (often the Fourier transform) of the sought complex images. The algorithm is developed using the alternating projection framework and is aimed to obtain high performance for heavily noisy (Poissonian or Gaussian) observations. The estimation of the target images is reformulated as a sparse regression, often termed sparse coding, in the complex domain. This is accomplished by learning a complex domain dictionary from the data it represents via matrix factorization with sparsity constraints on the code (ie, the regression coefficients). Our algorithm, termed dictionary learning phase retrieval (DLPR), jointly learns the referred to dictionary and reconstructs the unknown target image. The effectiveness of DLPR is illustrated through experiments conducted on complex images, simulated and real, where it shows noticeable advantages over the state-of-the-art competitors.

  9. Shevkunov, I., Katkovnik, V., Petrov, N.V. & Egiazarian, K. Super-resolution microscopy for biological specimens: lensless phase retrieval in noisy conditions. Biomedical optics express, 9(11):5511-5523, Optical Society of America, 2018. (paper) (webpage)

    Abstract. The paper is devoted to a computational super-resolution microscopy. A complex- valued wavefront of a transparent biological cellular specimen is restored from multiple intensity diffraction patterns registered with noise. For this problem, the recently developed lensless super-resolution phase retrieval algorithm [Optica, 4(7), 786 (2017)] is modified and tuned. This algorithm is based on a random phase coding of the wavefront and on a sparse complex-domain approximation of the specimen. It is demonstrated in experiments, that the reliable phase and amplitude imaging of the specimen is achieved for the low signal-to-noise ratio provided a low dynamic range of observations. The filtering in the observation domain and specimen variables are specific features of the applied algorithm. If these filtering are omitted the algorithm becomes a super-resolution version of the standard iterative phase retrieval algorithms. In comparison with this simplified algorithm with no filtering, our algorithm shows a valuable improvement in imaging with much smaller number of observations and shorter exposure time. In this way, presented algorithm demonstrates ability to work in a low radiation photon-limited mode.

  10. Katkovnik, V., Hogasten, N. & Egiazarian, K. A novel binary and multilevel phase masks for enhanced depth-of-focus infrared imaging. In 2018 52nd Asilomar Conference on Signals, Systems, and Computers, pages 386-390, 2018. (webpage)

    Abstract. This paper introduces novel binary and multilevel phase masks (BPMs) for improved depth of focus (DoF) in infrared imaging. The procedure developed for design of BPMs combines two ideas: cubic wavefront coding (WFC) of continuous absolute phase and original discretization of this phase profile. Both these ingredients of the design make the optical system robust with respect to defocus and uncontrolled variations in the optical system. The method allows us to design a flat and very thin BPM using piecewise invariant diffractive optical element (DOE) of simple geometry. The BPMs are used for wavefront modulation in two optical setups: a lensless system and a lens/BPM optical hybrid. Computational inverse imaging (deblurring) is applied in order to reconstruct a sharp image from respective blurred observations. The optical system is optimized end-to-end, including both the BPMs and the deblurring algorithm. Simulation experiments demonstrated for midwave infrared (MWIR) imaging, show high quality imaging even for very large amounts of defocus. Imaging systems incorporating this method show advantages over conventional optical systems with a refractive lens.

  11. Katkovnik, V. & Egiazarian, K. Multi-frequency phase retrieval from noisy data. In 2018 26th European Signal Processing Conference (EUSIPCO), pages 2200-2204, 2018. (paper) (webpage)

    Abstract. The phase retrieval from multi-frequency intensity (power) observations is considered. The object to be reconstructed is complex-valued. A novel algorithm is presented that accomplishes both the object phase (absolute phase) retrieval and denoising for Poissonian and Gaussian measurements. The algorithm is derived from the maximum likelihood formulation with Block Matching 3D (BM3D) sparsity priors. These priors result in two filtering: one in the complex domain for complex-valued multi-frequency object images and another one in the real domain for the object absolute phase. The algorithm is iterative with alternating projections between the object and measurement variables. The simulation experiments are produced for Fourier transform image formation and random phase modulations of the object, then observations are random object diffraction patterns. The simulation results demonstrate the success of the algorithm for reconstruction of the complex phase objects with the high-accuracy performance even for a high dynamic range of the absolute phase and very noisy data.

  12. Katkovnik, V., Shevkunov, I., Petrov, N.V. & Egiazarian, K. Multiwavelength surface contouring from phase-coded noisy diffraction patterns: wavelength-division optical setup. Optical Engineering, 57(8):085105, International Society for Optics and Photonics, 2018. (paper) (webpage)

    Abstract. We propose an algorithm for absolute phase retrieval from multiwavelength noisy phase coded diffraction patterns. A lensless optical system is considered with a set of successive single wavelength experiments (wavelength-division setup). The phase masks are applied for modulation of the multiwavelength object wavefronts. The algorithm uses the forward/backward propagation for coherent light beams and sparsely encoding wavefronts, which leads to the complex-domain block-matching three-dimensional filtering. The key-element of the algorithm is an original aggregation of the multiwavelength object wavefronts for high-dynamic-range absolute phase reconstruction. Simulation tests demonstrate that the developed approach leads to the effective solutions explicitly using the sparsity for noise suppression and high-accuracy object absolute phase reconstruction from noisy data.

  13. Achimova, E., Abaskin, V., Cazac, V., Meshalkin, A., Pedrini, G., Claus, D., Shevkunov, I. & Katkovnik, V. Surface topography studied by off-axis digital holography. In Novel Optical Materials and Applications, pages NoW1J-7, 2018. (webpage)

    Abstract. The topography of surface relief gratings patterned on As 2 S 3--Se nano-multilayers was investigated by digital holographic microscopy. For the high-accuracy phase reconstruction of the topography we used the sparse wavefront modeling. Experimental results are presented.

  14. Cruz, C., Foi, A., Katkovnik, V. & Egiazarian, K. Nonlocality-reinforced convolutional neural networks for image denoising. IEEE Signal Processing Letters, 25(8):1216-1220, IEEE, 2018. (paper) (webpage)

    Abstract. We introduce a paradigm for nonlocal sparsity reinforced deep convolutional neural network denoising. It is a combination of a local multiscale denoising by a convolutional neural network (CNN) based denoiser and a nonlocal denoising based on a nonlocal filter (NLF), exploiting the mutual similarities between groups of patches. CNN models are leveraged with noise levels that progressively decrease at every iteration of our framework, while their output is regularized by a nonlocal prior implicit within the NLF. Unlike complicated neural networks that embed the nonlocality prior within the layers of the network, our framework is modular, and it uses standard pretrained CNNs together with standard nonlocal filters. An instance of the proposed framework, called NN3D, is evaluated over large grayscale image datasets showing state-of-the-art performance.

  15. Katkovnik, V., Shevkunov, I., Petrov, N.V. & Eguiazarian, K. Multiwavelength surface contouring from phase-coded diffraction patterns. In Unconventional Optical Imaging, 10677:106771B, 2018. (webpage)

    Abstract. We propose a new algorithm for absolute phase retrieval from multiwavelength noisy phase coded diffraction patterns in the task of surface contouring. A lensless optical setup is considered with a set of successive single wavelength experiments. The phase masks are applied for modulation of the multiwavelength object wavefronts. The algorithm uses the forward and backward propagation for coherent light beams and sparsely encoding wavefronts which leads to the complex-domain block-matching 3D filtering. The key-element of the algorithm is an original aggregation of the multiwavelength object wavefronts for high-dynamic-range profile measurement. Numerical experiments demonstrate that the developed approach leads to the effective solutions explicitly using the sparsity for noise suppression and high-accuracy object profile reconstruction.

  16. Katkovnik, V., Shevkunov, I., Petrov, N. & Eguiazarian, K. Multiwavelength absolute phase retrieval from noisy diffractive patterns: wavelength multiplexing algorithm. Applied Sciences, 8(5):719, Multidisciplinary Digital Publishing Institute, 2018. (paper) (webpage)

    Abstract. We study the problem of multiwavelength absolute phase retrieval from noisy diffraction patterns. The system is lensless with multiwavelength coherent input light beams and random phase masks applied for wavefront modulation. The light beams are formed by light sources radiating all wavelengths simultaneously. A sensor equipped by a Color Filter Array (CFA) is used for spectral measurement registration. The developed algorithm targeted on optimal phase retrieval from noisy observations is based on maximum likelihood technique. The algorithm is specified for Poissonian and Gaussian noise distributions. One of the key elements of the algorithm is an original sparse modeling of the multiwavelength complex-valued wavefronts based on the complex-domain block-matching 3D filtering. Presented numerical experiments are restricted to noisy Poissonian observations. They demonstrate that the developed algorithm leads to effective solutions explicitly using the sparsity for noise suppression and enabling accurate reconstruction of absolute phase of high-dynamic range.

  17. Ponomarenko, M., Katkovnik, V. & Egiazarian, K. Methods and tools for denoising of complex-valued images based on block-matching and high order singular value decomposition. Electronic Imaging, 2018(13):1-6, Society for Imaging Science and Technology, 2018. (paper) (webpage)

    Abstract. Noise suppression in complex-valued data is an important task for a wide class of applications, in particular concerning the phase retrieval in coherent imaging. The approaches based on BM3D techniques are ones of the most successful in the field. In this paper, we propose and develop a new class of BM3D-style algorithms, which use high order (3D and 4D) singular value decomposition (HOSVD) for transform design in complex domain. This set of the novel algorithms is implemented as a toolbox In Matlab. This development is produced for various types of the complex-domain sparsity: directly in complex domain, real/imaginary and phase/ amplitude parts of complex-valued variables. The group-wise transform design is combined with the different kinds of thresholding including multivariable Wiener filtering. The toolbox includes iterative and non-iterative novel complex-domain algorithms (filters). The efficiency of the developed algorithms is demonstrated on denoising problems with an additive Gaussian complex-valued noise. A special set of the complex-valued test-images was developed with spatially varying correlated phase and amplitudes imitating data typical for optical interferometry and holography. It is shown that for this class of the test-images the developed algorithms demonstrate the state-of-the-art performance.

  18. Cazac, V., Meshalkin, A., Achimova, E., Abashkin, V., Katkovnik, V., Shevkunov, I., Claus, D. & Pedrini, G. Surface relief and refractive index gratings patterned in chalcogenide glasses and studied by off-axis digital holography. Applied optics, 57(3):507-513, Optical Society of America, 2018. (paper) (webpage)

    Abstract. Surface relief gratings and refractive index gratings are formed by direct holographic recording in amorphous chalcogenide nano-multilayer structures As_2S_3-Se and thin films As_2S_3. The evolution of the grating parameters, such as the modulation of refractive index and relief depth in dependence of the holographic exposure, is investigated. Off-axis digital holographic microscopy is applied for the measurement of the photoinduced phase gratings. For the high-accuracy reconstruction of the wavefront (amplitude and phase) transmitted by the fabricated gratings, we used a computational technique based on the sparse modeling of phase and amplitude. Both topography and refractive index maps of recorded gratings are revealed. Their separated contribution in diffraction efficiency is estimated.

  19. Kocsis, P., Shevkunov, I., Katkovnik, V. & Egiazarian, K. Single exposure lensless phase imaging. In EUVIP Workshop, 2018. (paper)

    Abstract. This paper is devoted to development of a phase retrieval imaging technique based on phase modulation of the object wavefront and angular spectrum free propagation from object to the modulation phase mask and from this mask to the camera. The free propagation is modelled by the angular spectrum operator [1]. In contrast with the traditional microscopy, our system is lensless, which can provide high resolution reconstruction with large field of view while the setup became portable and cost-effective. A random phase mask between the object and camera creates a coded diffraction pattern without twin image effects. The complex-domain book-matching 3D (BM3D) filtering is applied in order to reach pixel resolution with a one single wavelength experiment. A spatial light modulator (SLM) is used for the phase-mask generation and the image is captured by a CMOS sensor of the camera. Previously developed optical setup and image reconstruction algorithm [2] based on multiple observations are modified here to use only one observation. A sparse modelling of the complex-valued object to be reconstructed leads to its BM3D filtering [2, 3].

  20. Cruz, C., Mehta, R., Katkovnik, V. & Egiazarian, K.O. Single image super-resolution based on Wiener filter in similarity domain. IEEE Transactions on Image Processing, 27(3):1376-1389, IEEE, 2017. (paper) (webpage)

    Abstract. Abstract: Single image super-resolution (SISR) is an ill-posed problem aiming at estimating a plausible high-resolution (HR) image from a single low-resolution image. Current state-of-the-art SISR methods are patch-based. They use either external data or internal self-similarity to learn a prior for an HR image. External data-based methods utilize a large number of patches from the training data, while self-similarity-based approaches leverage one or more similar patches from the input image. In this paper, we propose a self-similarity-based approach that is able to use large groups of similar patches extracted from the input image to solve the SISR problem. We introduce a novel prior leading to the collaborative filtering of patch groups in a 1D similarity domain and couple it with an iterative back-projection framework. The performance of the proposed algorithm is evaluated on a number of SISR benchmark data sets. Without using any external data, the proposed approach outperforms the current non-convolutional neural network-based methods on the tested data sets for various scaling factors. On certain data sets, the gain is over 1 dB, when compared with the recent method A+. For high sampling rate (x4), the proposed method performs similarly to very recent state-of-the-art deep convolutional network-based approaches.

  21. Katkovnik, V., Hogasten, N. & Egiazarian, K. A novel two-and multi-level binary phase mask design for enhanced depth-of-focus. arXiv preprint arXiv:1707.07752, 2017. (paper) (webpage)

    Abstract. This paper introduces a two-and multi-level phase mask design for improved depth of focus. A novel technique is proposed incorporating cubic and generalized cubic wavefront coding (WFC). The obtained system is optical-electronic requiring computational deblurring post-processing, in order to obtain a sharp image from the observed blurred data. A midwave infrared (MWIR) system is simulated showing that this design will produce high quality images even for large amounts of defocus. It is furthermore shown that this technique can be used to design a flat, single optical element, systems where the phase mask performs both the function of focusing and phase modulation. It is demonstrated that in this lensless design the WFC coding components can be omitted and WFC effects are achieved as a result of the proposed algorithm for phase mask design which uses the quadratic phase of the thin refractive lens as the input signal.

  22. Katkovnik V., Egiazarian K, Multi-frequency phase retrieval from noisy data, Proceedings of EUSIPCO 2018, Rome, 2018, and https://arxiv.org/abs/1802.02011, 5 p. (code) (paper) (web page)

    Abstract. The phase retrieval from multi-frequency intensity (power) observations is considered. The object to be reconstructed is complex-valued. A novel algorithm is presented that accomplishes both the object phase (absolute phase) retrieval and denoising for Poissonian and Gaussian measurements. The algorithm is derived from the maximum likelihood formulation with Block Matching 3D (BM3D) sparsity priors. These priors result in two filtering: one is in the complex domain for complex-valued multi-frequency object images and another one in the real domain for the object phase. The algorithm is iterative with alternating projections between the object and measurement variables. The simulation experiments are produced for Fourier transform image formation and random phase modulations of the object, then the observations are random object diffraction patterns. The results demonstrate the success of the algorithm for reconstruction of the complex phase objects with the high-accuracy performance even for very noisy data.

  23. V. Katkovnik, M. Ponomarenko and K. Egiazarian, "Complex-valued image denosing based on group-wise complex-domain sparsity", arXiv:1711.00362, 2017, 36 p. (code) (paper) (web page)

    Abstract. Phase imaging and wavefront reconstruction from noisy observations of complex exponent is a topic of this paper. It is a highly non-linear problem because the exponent is a 2 Pi-periodic function of phase. The reconstruction of phase and amplitude is difficult. Even with an additive Gaussian noise in observations distributions of noisy components in phase and amplitude are signal dependent and non-Gaussian. Additional difficulties follow from a prior unknown correlation of phase and amplitude in real life scenarios. In this paper, we propose a new class of non-iterative and iterative complex domain filters based on group-wise sparsity in complex domain. This sparsity is based on the techniques implemented in Block-Matching 3D filtering (BM3D) and 3D/4D High-Order Singular Decomposition (HOSVD) exploited for spectrum design, analysis and filtering. The introduced algorithms are a generalization of the ideas used in the CD-BM3D algorithms presented in our previous publications. The algorithms are implemented as a MATLAB Toolbox. The efficiency of the algorithms is demonstrated by simulation tests.

  24. V. Katkovnik, "Phase retrieval from noisy data based on sparse approximation of object phase and amplitude", 2017, https://arxiv.org/abs/1709.01071 (demo code) (paper) (web page)

    Abstract. A variational approach to reconstruction of phase and amplitude of a complex-valued object from Poissonian intensity observations is developed. The observation model corresponds to the typical optical setups with a phase modulation of wavefronts. The transform domain sparsity is applied for the amplitude and phase modeling. It is demonstrated that this modeling results in the essential advantage of the developed algorithm for heavily noisy observations corresponding to a short exposure time in optical experiments. We consider also two simplified versions of this algorithm where the sparsity modeling of phase and amplitude is omitted. In the simulation study we compare the developed algorithms versus the Gerchberg-Saxton and truncation Wirtinger flow algorithms. The latter algorithm being the maximum likelihood based is the state-of-the-art for the phase retrieval from Poissonian observations. For noisy and very noisy observations the proposed algorithm demonstrates a valuable advantage.

  25. Katkovnik V., Hogasten N. , Egiazarian K., A novel two- and multi-level binary phase mask design for enhanced depth-of-focus arXiv preprint arXiv:1707.07752, 2017, 15 p. (paper)

    Abstract. This paper introduces a two-and multi-level binary phase mask design for improved depth of focus. A novel technique is proposed incorporating cubic and generalized cubic wavefront coding (WFC). The obtained system is optical-electronic requiring computational deblurring post-processing, in order to obtain a sharp image from the observed blurred data. A midwave infrared (MWIR) system is simulated showing that this design will produce high quality images even for large amounts of defocus. It is furthermore shown that this technique can be used to design a flat, single optical element, systems where the phase mask performs both the function of focusing and phase modulation. It is demonstrated that in this lensless design the WFC coding components can be omitted and WFC effects are achieved as a result of the proposed algorithm for phase mask design which uses the quadratic phase of the thin refractive lens as the input signal.

  26. Katkovnik, V., Shevkunov, I., Petrov, N.V. & Egiazarian, K. Computational super-resolution phase retrieval from multiple phase-coded diffraction patterns: simulation study and experiments. Optica, 4(7):786-794, Optical Society of America, 2017. (code) (paper)

    Abstract. Computational super-resolution inverse diffraction phase retrieval is considered. The optical setup is lensless with a spatial light modulator (SLM) for aperture phase coding. The paper is focused on experimental tests of the Super-Resolution Sparse Phase Amplitude Retrieval (SR- SPAR) algorithm. We start from simulation tests and go to physical experiments. Both sim- ulation tests and experiments demonstrate a good quality imaging for super-resolution with the factor 4, which means that the computational pixels of the reconstructed object are 4 times smaller than the sensor pixels.

  27. Katkovnik, V. & Egiazarian, K. Sparse super-resolution phase retrieval from phase-coded noisy intensity patterns. Optical Engineering, 56(9), 094103, doi:10.1117/1.OE.56.9.094103, 2017. (draft paper) (demo code)

    Abstract. We consider computational super-resolution inverse diffraction problem for phase retrieval from phase-coded intensity observations. The optical setup includes a thin lens and a spatial light modulator (SLM) for phase coding. The designed algorithm is targeted on optimal solution for Poissonian noisy observations. One of the essential instruments of this design is a complex-domain sparsity applied for complex valued object (phase and amplitude) to be reconstructed. Simulation experiments demonstrate that good quality imaging can be achieved for high-level of the super-resolution with factor 32, what means that the pixels of the reconstructed object pixels are 32 times smaller than sensor and SLM pixels. In wavelength this super-resolution corresponds to the object pixels as small as a quarter of the wavelength.

  28. Katkovnik, V., Ponomarenko, M. & Egiazarian, K. Sparse approximations in complex domain based on BM3D modeling. Signal Processing, Elsevier, 141:96-108, December 2017. (demo code) (draft paper) (webpage)

    Abstract. In this paper the concept of sparsity for complex-valued variables is introduced in the following three types: directly in the complex domain and for two real-valued pairs phase/amplitude and real/imaginary parts of complex variables. The nonlocal block-matching technique is used for sparsity implementation and filter design for each type of sparsity. These filters are complex domain generalizations of Block Matching 3D collaborative (BM3D) filters based on the high-order singular value decomposition (HOSVD) used in order to generate group-wise adaptive analysis/synthesis transforms. Complex domain denoising is developed as a test-problem for comparison of the designed filters and the different types of sparsity modeling.

  29. Katkovnik, V. & Egiazarian, K. Sparse phase imaging based on complex domain nonlocal BM3D techniques. Digital Signal Processing, Academic Press, 63:72-85, 2017. (code) (paper) (webpage)

    Abstract. The paper is addressed to 2D phase and amplitude estimation of complex-valued signals - that is, in particular, to estimation of modulo-2π interferometric phase images from periodic and noisy observations. These degradation mechanisms make phase image estimation a challenging problem. A sparse nonlocal data-adaptive imaging formalized in complex domain is used for phase and amplitude image reconstruction. Following the procedure of patch-based technique, the image is partitioned into small overlapping square patches. Block Matching Three Dimensional (BM3D) technique is developed for forming complex domain sparse spectral representations of complex-valued data. High Order Singular Value Decomposition (HOSVD) applied to BM3D groups enables the design of the orthonormal complex domain 3D transforms which are data adaptive and different for each BM3Ds group. An iterative version of the complex domain BM3D is designed from variational formulation of the problem. The convergence of this algorithm is shown. The effectiveness of the new sparse coding based algorithms is illustrated in simulation experiments where they demonstrate the state-of-the-art performance.

  30. Katkovnik V., Ponomarenko M. & Egiazarian K., Complex domain nonlocal block-matching denoising based on high-order singular value decomposition (HOSVD), 25th European Signal Processing Conference (EUSIPCO), 2017, pp.748-752. (draft paper)

    Abstract. Block matching 3D collaborative filtering (BM3D) is one of the most popular denoising technique based on data sparsity concept applied to specially structured data. In this paper we develop this technique for complex domain, i.e. for application to complex-valued data. Sparsity as an approximation technique can be addressed directly to complex-valued variables or to real-valued pairs phase/amplitude and real/imaginary parts of complex-valued variables. As a result we arrive to various ways of development and obtain a set of quite different algorithms. The algorithms proposed in this paper are composed from two components: nonlocal patch-wise grouping and highorder singular value decomposition (HOSVD) for grouped data processing. The latter gives data adaptive complex-valued bases for complex-valued data or real-valued bases for joint processing of the pairs phase/amplitude, real/imaginary parts of complexvalued variables. Comparative study of the developed algorithms is produced in order to select the most efficient ones.

  31. Ponomarenko M., Pismenskova M. & Egiazarian K., Weighted MSE Based Spatially Adaptive BM3D, 5th European Signal Processing Conference (EUSIPCO), 2017, pp.763-767. (draft paper)

    Abstract. Weighted MSE (wMSE), recently introduced modification of MSE, is an image quality metric used to estimate visual quality of filtered images. It provides better than MSE correspondence to a human perception in consideration of distortions introduced by image filters. In this paper, wMSE is used both as a criterion to evaluate filtering efficiency of the modification of BM3D filter with spatially varying parameters, as well as to train a specially designed neural network to predict filters’ parameters. Extensive analysis on three image datasets demonstrates that the proposed modification of BM3D provides lower values of wMSE than those of BM3D, both effectively suppressing noise in homogeneous regions as well as preserving fine details and texture.

  32. Katkovnik, V., Shevkunov, I., Petrov, N.V. & Egiazarian, K. Computational wavelength resolution for in-line lensless holography: phase-coded diffraction patterns and wavefront group-sparsity. In Digital Optical Technologies 2017, Proceedings of SPIE Vol. 10335:1033509, Munich, 26–28 June 2017.

    Abstract. In-line lensless holography is considered with a random phase modulation at the object plane. The forward wavefront propagation is modelled using the Fourier transform with the angular spectrum transfer function. The multiple intensities (holograms) recorded by the sensor are random due to the random phase modulation and noisy with Poissonian noise distribution. The algorithm is designed for optimal phase/amplitude reconstruction from Poissonian data. It is shown by computational experiments that high-accuracy reconstructions can be achieved with resolution going up to the two thirds of the wavelength. With respect to the sensor pixel size it is a 32 super-resolution. The algorithm is designed for optimal super-resolution phase/amplitude reconstruction from Poissonian data. The algorithm design is based on the general methodology developed for phase retrieval with a pixel-wise resolution in V. Katkovnik, ”Phase retrieval from noisy data based on sparse approximation of object phase and amplitude”, http://www.cs.tut.fi/~lasip/DDT/pdfs/Single_column.pdf.

  33. Egiazarian K. & Katkovnik V., Alternating Group Sparsity for Image Restoration, Workshop SPARS 2017, Lisbon, on June 5-8, 2017. (paper)

  34. Egiazarian K. & Katkovnik V., Complex Domain Nonlocal Group-Wise Sparsity: Toward Wavelength Super-Resolution Phase Imaging in Coherent Optics, Workshop SPARS 2017, Lisbon, on June 5-8, 2017. (paper)

  35. Katkovnik V., Complex domain (phase/amplitude) sparsity and data processing in coherent optics, Summer School on “Digital Holography and related Techniques”, Stuttgart, 21st to 23rd of June 2017.

    Abstract. In optics, a monochromatic wavefront is represented as a complex-valued signal, where amplitude and phase are unknown variables of interest. This wavefront phase cannot be measured directly because all measurement instruments are sensitive with respect to the intensity but not to phase. Accordingly, one of the main targets of data processing is to extract phase information from measured intensities. For instance, in interferometry and holography the phase is retrieved using special reference wavefronts. Under modeling of wavefront, we understand any mathematical tools for prediction, interpolation, denoising, etc., installing links between the values of wavefronts at different coordinates. In computational imaging, sparse and redundant representations (sparsity) have been successfully developed the last years as a general modeling instrument. It is based on the assumption that there exists a small number of basic functions such that image can be represented exactly or approximately with a good accuracy. In term of statistics, a sparse representation can be thought as a low-order parametric approximation. In this classical form, the sparsity concept is used in parametric approximations just zeroing small amplitude components. A specific point of the sparsity is that the sparse basis is unknown in advance and should be designed. In this lecture, we consider methods and algorithms for sparse modeling of wavefronts and their applications for phase imaging.

  36. Katkovnik, V., Shevkunov, I., Petrov, N.V. & Egiazarian, K. High-accuracy off-axis wavefront reconstruction from noisy data: local least square with multiple adaptive windows. Optics express, 24(22):25068-25083, Optical Society of America, 2016. (paper) (webpage)

    Abstract. A variational algorithm to object wavefront reconstruction from noisy intensity observations is developed for the off-axis holography scenario with imaging in the acquisition plane. The algorithm is based on the local least square technique proposed in paper [J. Opt.Soc. Am. A, 21, 367 (2004)]. First, multiple reconstructions of the wavefront are produced for various size and various directional windows applied for localization of estimation. At the second stage, a special statistical rule is applied in order to select the best window size estimate for each pixel of the image and for each of the directional windows. At the third final stage the estimates of the different directions obtained for each pixel are aggregated in the final one. Simulation experiments and real data processing prove that the developed algorithm demonstrate the performance of the extraordinary quality and accuracy for both the phase and amplitude of the object wavefront.

  37. Katkovnik, V. Sparse phase retrieval from noisy data: variational formulation and algorithms. In 3D Image Acquisition and Display: Technology, Perception and Applications, pages JT3A-42, 2016.

    Abstract. A variational approach to wavefront reconstruction from multiple noisy Poissonian intensity observations is developed. Sparse modeling of amplitude and absolute phase of the object is one of the key elements of the derived algorithm.

  38. Trujillo-Sevilla, J., Katkovnik, V., Javidi, B. & Rodriguez-Ramos, J. Restoring Integral Images from Focal Stacks Using Compressed Sensing Techniques. Journal of Display Technology, 12(7):349-354, IEEE, 2016.

    Abstract. This paper contains an original development of the compressed sensing technique for restoring integral images from a number of observed 2D images. The proposed data acquisition uses a conventional camera equipped with a horizontal 1D mask placed in the pupil plane of the lens. The compressed sensing style algorithm developed is based on a sparsity hypothesis imposed on 2D cross sections of the light field.

  39. Sahin, E., Katkovnik, V. & Gotchev, A. Super-resolution in a defocused plenoptic camera: a wave-optics-based approach. Optics letters, 41(5):998-1001, Optical Society of America, 2016. (paper)

    Abstract. Plenoptic cameras enable the capture of a light field with a single device. However, with traditional light field rendering procedures, they can provide only low-resolution two-dimensional images. Super-resolution is considered to overcome this drawback. In this study, we present a super-resolution method for the defocussed plenoptic camera (Plenoptic 1.0), where the imaging system is modeled using wave optics principles and utilizing low-resolution depth information of the scene. We are particularly interested in super-resolution of in-focus and near in-focus scene regions, which constitute the most challenging cases. The simulation results show that the employed wave-optics model makes super-resolution possible for such regions as long as sufficiently accurate depth information is available.

  40. Egiazarian, K. & Katkovnik, V. Single image super-resolution via BM3D sparse coding. In Signal Processing Conference (EUSIPCO), 2015 23rd European, pages 2849-2853, 2015. (paper)

    Abstract. In this paper, a novel single image super-resolution (SISR) algorithm is proposed. It is based on the BM3D (Block- Matching and 3D filtering) paradigm, where both sparsity and nonlocal patch self-similarity priors are utilized. The algorithm is derived from a variational formulation of the problem and has a structure typical for iterative back-projection super-resolution methods. They are characterized by updating high-resolution image which is calculated using the previous estimate and upsampled low-resolution error. The developed method is thoroughly compared with the state-of-the-art SISR both for noiseless and noisy data, demonstrating superior performance objectively and subjectively.

  41. Katkovnik, V., Shevkunov, I., Petrov, N. & Egiazarian, K. Wavefront reconstruction in digital off-axis holography via sparse coding of amplitude and absolute phase. Optics letters, 40(10):2417-2420, Optical Society of America, 2015. (paper)

    Abstract. This work presents the new method for wavefront reconstruction from a digital hologram recorded in off-axis configuration. The main feature of the proposed algorithm is a good ability for noise filtration due to the original formulation of the problem taking into account the presence of noise in the recorded intensity distribution and the sparse phase and amplitude reconstruction approach with the data-adaptive block-matching 3D technique. Basically, the sparsity assumes that low dimensional models can be used for phase and amplitude approximations. This low dimensionality enables strong suppression of noisy components and accurate revealing of the main features of the signals of interest. The principal point is that dictionaries of these sparse models are not known in advance and reconstructed from given noisy observations in a multiobjective optimization procedure. We show experimental results demonstrating the effectiveness of our approach.

  42. Katkovnik, V., Shevkunov, I.A., Petrov, N.V. & Egiazarian, K. Sparse approximations of phase and amplitude for wave field reconstruction from noisy data. In SPIE Optics+ Optoelectronics, pages 950802-950802, 2015. (paper)

    Abstract. The topic of sparse representations (SR) of images has attracted tremendous interest from the research community in the last ten years. This interest stems from the fundamental role that the low dimensional models play in many signal and image processing areas, i.e., real world images can be well approximated by a linear combination of a small number of atoms (i.e., patches of images) taken from a large frame, often termed dictionary. The principal point is that these large dictionaries as well as the elements of these dictionaries taken for approximation are not known in advance and should be taken from given noisy observations. The sparse phase and amplitude reconstruction (SPAR) algorithm has been developed for monochromatic coherent wave field reconstruction, for phase-shifting interferometry and holography. In this paper the SPAR technique is extended to off-axis holography. Pragmatically, SPAR representations are result in design of efficient data-adaptive filters. We develop and study the algorithm where these filters are applied for denoising of phase and amplitude in object and sensor planes. This algorithm is iterative and developed as a maximum likelihood optimal solution provided that the noise in intensity measurements is Gaussian. The multiple simulation and real data experiments demonstrate the advance performance of the new technique.

  43. Hongxing, H., Bioucas-Dias, J.M. & Katkovnik, V. Interferometric phase image estimation via sparse coding in the complex domain. IEEE Transactions on Geoscience and Remote Sensing, 53(5):2587-2602, IEEE, 2015. (Matlab code) (paper) (webpage)

    Abstract. This paper addresses interferometric phase image estimation, i.e., the estimation of phase modulo-2π images from sinusoidal 2π -periodic and noisy observations. These degradation mechanisms make interferometric phase image estimation a quite challenging problem. We tackle this challenge by reformulating the true estimation problem as a sparse regression, often termed sparse coding, in the complex domain. Following the standard procedure in patch-based image restoration, the image is partitioned into small overlapping square patches, and the vector corresponding to each patch is modeled as a sparse linear combination of vectors, termed the atoms, taken from a set called dictionary. Aiming at optimal sparse representations, and thus at optimal noise removing capabilities, the dictionary is learned from the data that it represents via matrix factorization with sparsity constraints on the code (i.e., the regression coefficients) enforced by the l1 norm. The effectiveness of the new sparse-coding-based approach to interferometric phase estimation, termed the SpInPHASE, is illustrated in a series of experiments with simulated and real data where it outperforms the state-of-the-art.



Go to Computational Imaging Group webpage...