Auto Encoder Based Image Inpainting Model Using Multi Layer Latent Representations

Abstract

Image inpainting is used in computer vision to reconstruct images after removal of unwanted objects and in the construction of damaged images in a visually acceptable manner. Use of this technique is mainly found in the areas of image editing. Although many algorithms have been developed by researchers over the years, these mainly fall into the reconstruction of small regions or objects with less structural complexities. With the advancement of machine learning techniques, innovative ideas have emerged which led to the development of mechanisms to reconstruct more complex structural variations in large regions of the images. In this research, a considerably large region of a damaged image has been inpainted using a convolutional auto encoder with an encoder-decoder combination. It is equipped with a novel approach to modify the latent space of the input image. These latent representations are created from multiple layers of the encoder to form a Multi Layer Latent Representation (MLLR). This MLLR is fed to the decoder which generates the image by applying the transpose convolution operation. The quality of the inpainted images generated from our model is compared with the images generated from the model having a single latent representation without the MLLR. Peak Signal to Noise Ratio (PSNR) and Structured Similarity Index Metrics (SSIM) are used in the evaluation. Empirical analysis indicate that the model is able to provide SSIM values over 0.9 for the reconstructed images with damaged areas that consist of 12% of the image surface.