General Introduction of image restoration

Instead of giving a rigorous definition for image restoration, we are going to show you what image restoration is through an intuitive way, i.e., a number of practical examples.

If you lose the track while reading, please, do not go away, everything that will be mentioned in this text, will be additionally explained through our blog over time.

Imagine that now you are given a corrupt image, is there any possibility to restore a clean, original image? Nowadays, we have been able to deal with some kinds of image corruptions. First of all, we need to analyze the sources of corruption, which may come in many forms such as noise, motion blur, camera mis-focus, and portions missing. Considering different sources of corruption, image restoration could be divided into three main categories: denoising, deblurring, and inpainting.
  

1. What is image denoising?

Image denoising is to remove unwanted noise in order to restore the original image. Huge number of techniques have been developed to address different types of noise, such as salt and pepper noise, uniform noise, Gaussian noise, and periodic noise.
   

1.1 Salt and pepper noise

Salt and pepper noise is generally caused due to errors in transmission. It presents itself as sparsely (sparsely – thinly scattered or distributed) occurring white and black pixels (Fig. 1).

Fig. 1. Left: original image. Middle: image with salt and pepper noise. Right: denoised result.

1.2 Uniform noise

The uniform noise caused by quantizing the pixels of image (quantization – every pixel receive the new value from small discrete set) to a number of distinct levels is known as quantization noise. Uniform noise can be analytically described as a uniform distributed random variable (Fig. 2).

Fig. 2. Left: original image. Middle: image with uniform noise. Right: denoised version.

1.3 Gaussian noise

Principal sources of Gaussian noise in digital images arise during acquisition e.g. sensor noise caused by poor illumination and/or high temperature, and/or transmission e.g. electronic circuit noise. Each pixel in the noisy image is the sum of the true pixel value and a random Gaussian distributed noise value (Fig. 3).

Fig. 3. Left: original image. Middle: image corrupted by Gaussian noise. Right: denoised version.

1.4 Periodic noise

Periodic noise mainly comes from electrical or electromechanical interference during image acquisition (Fig. 4).

Fig. 4. Left: image with periodic noise. Right: denoised result via inverse fourier transform

2. What is image deblurring?

Deblurring is the process of removing blurring artifacts from images.
Where does blur come from?

• Optical blur: camera is out-of-focus (Fig. 5)

Fig. 5. An image that is partially in focus, but mostly out of focus in varying degrees.

• Motion blur: camera or object is moving (Fig. 6)

Fig. 6. Motion blurring happens when you're photographing a moving subject.

Practical application 1: Law enforcement (Fig. 7)

Fig. 7. License plates from a surveillance system and theirsreconstructions [1].

Practical application 2: video surveillance (Fig. 8)

Fig. 8. Deblurring results of a typical surveillance scene in an urban environment [2].

Practical application 3: astronomical imaging(Fig. 9)

Fig. 9. Left: original image from Hubble Space Telescope. Right: Deblurred result.

3. What is image Inpainting?

Given image with significant portions missing or damaged, inpainting is to reconstitute missing regions with data consistent with the rest of the image.

Example 1: repairing photographs (Fig. 10)

Fig. 10. Left: an 1865 Photograph of Abraham Lincoln taken by Alexander Gardner (courtesy of Wing Yung and Ajeet Shankar from Harvard University). Middle: one showing inpainting mask. Right: restored Image [3].

Example 2: Remove unwanted objects(Fig. 11)

Fig. 11. Left: original input image. Middle: the manually selected target region which we want to remove. Right: the inpainting result.The missing portion of rainbow is reconstructed convincingly [4].

References:
[1] Svoboda, P., Hradis, M., Marsik, L., & Zemcik, P. (2016). CNN for license plate motion deblurring. arXiv preprint arXiv:1602.07873.
[2] Thorpe, C., Li, F., Li, Z., Yu, Z., Saunders, D., & Yu, J. (2013). A coprime blur scheme for data security in video surveillance. IEEE transactions on pattern analysis and machine intelligence, 35(12), 3066-3072.
[3] Richard, M. M. O. B. B., & Chang, M. Y. S. (2001, September). Fast digital image inpainting. In Appeared in the Proceedings of the International Conference on Visualization, Imaging and Image Processing (VIIP 2001), Marbella, Spain (pp. 106-107).
[4] Criminisi, A., Pérez, P., & Toyama, K. (2004). Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on image processing, 13(9), 1200-1212.