Reflection symmetry aware image retargeting

Highlights

• Symmetry aware image retargeting operator that preserves the reflection symmetry.

• Reflection symmetry region using symmetry axis and symmetric point pairs.

• An optimization framework to make the reflected seam satisfy the seam constraint.

• Propagate the optimized reflected seam in the non-symmetry region.

• Novel evaluation framework for the reflection symmetry aware image retargeting.

Abstract

Reflection symmetry is one of the most commonly occurring and prominent visual attributes present in the real world. With an increase in the display devices of different sizes and aspect ratios, the images captured from the real world need to be resized to fit to the display device. In this paper, we propose a novel image retargeting approach which preserves the reflection symmetry present in the image during the image retargeting process. We detect the symmetry region present in the image using symmetry axis detection and object proposals. We propose a novel framework for finding an optimized reflected seam for the least energy seam defined by the seam carving approach. The symmetry axis and the symmetric object are preserved by adding or removing a seam and its reflected counterpart together. We show better preservation of symmetry axis, preservation of shape of the symmetric object, and quality of image retargeting when compared to the existing methods using three quantitative measures along with the qualitative results.

Introduction

Reflection symmetry is the most frequently occurring type of symmetry in natural objects and man-made objects. While retargeting an image, there have been attempts to preserve the translation symmetry while summarizing an image [1]. However, retargeting of images containing reflective symmetric objects is still an unsolved problem. The standard seam carving method and its variations, in order to preserve the geometry of an object, first generate an energy map. This energy map has high values at the pixels corresponding to the object to be preserved. However, the energy map itself is not symmetric, i.e., it may not have the same value at the reflective symmetric pixels. Therefore, if a seam ${\mathcal{S}}_i$ passes through one side of a reflective symmetric object, then it is not necessary that another seam ${\mathcal{S}}_j$ will pass through the pixels which are mirror reflections of the pixels of the seam ${\mathcal{S}}_i$. Furthermore, finding a reflective symmetric energy map can be a computationally prohibitive task [2]. Therefore, instead of finding a reflective symmetric energy map, we only find the reflection symmetry axis of a symmetric object which is a computationally tractable problem.

In this work, we propose a novel and efficient image retargeting approach which preserves the objects exhibiting reflection symmetry. We find the reflection symmetry axis from the image. However, just the symmetry axis is not sufficient to give complete information about the location and the shape of the object in the image. We use object proposal which is better in describing a symmetry region given the symmetry axis and the symmetric point pairs. In order to preserve reflection symmetry, we propose a symmetry aware seam carving approach. We first find a seam using the traditional seam carving approach. Now, in order to preserve the reflection symmetry, we find the reflective seam corresponding to this seam using the symmetry axis and the object proposal we have estimated. Then, we remove or add (for shrinking or expanding the input image) both the seams. However, we observe that a seam after reflection about the symmetry axis may not follow the criteria to be a seam. The reason is that the symmetry axis may not always be a vertical line. Therefore, integer pixel locations may not be integers after reflection. Also, a reflective vertical (horizontal) seam does not follow an 8-connected path of pixels in the image from top to bottom (left to right), containing only one pixel in each row (column) of the image. We propose an optimization based approach in order to make the reflection of a seam also to be a seam. After reflection and optimization of a seam, we propagate an optimized reflected seam outside the symmetry region. The primary contributions of the proposed work are as follows.

• We introduce a symmetry aware image retargeting operator that preserves the reflection symmetry present in an image. We find the reflection symmetry region using the symmetry axis, the symmetric point pairs and object proposals avoiding the need for object segmentation or symmetry map.

• We reflect the seam in the symmetry region about the symmetry axis. For preserving the symmetry region, we propose an optimization framework to make the reflected seam satisfy the seam constraint. We further use propagation to complete the optimized seam in the non-symmetry region.

• We propose a novel evaluation framework to validate the performance of the reflection symmetry aware image retargeting.

The rest of the paper is structured as follows. In Section 2, we present a brief overview of the symmetry detection and the image retargeting algorithms. Section 3 explains the proposed approach of the reflection symmetry aware image retargeting. The evaluation framework and the results are discussed in Section 4. We conclude the paper in Section 5 with the future scope.