Ariel Shamir
Abstract
Traditional image resizing techniques are oblivious to the content of the image when changing its width or height. In contrast, media (i.e., image and video) retargeting take s content into account. For example, one would like to change the aspect ratio of a video without making human figures look too fat or too skinny, or change the size of an image by automatically removing "unnecessary" portions while keeping the "important" features intact. We propose a simple operator; we term seam carving to support image and video retargeting. A seam is an optimal 1D path of pixels in an image, or a 2D manifold in a video cube, going from top to bottom, or left to right. Optimality is defined by minimizing an energy function that assigns costs to pixels. We show that computing a seam reduces to a dynamic programming problem for images and a graph min-cut search for video. We demonstrate that several image and video operations, such as aspect ratio correction, size change, and object removal, can be recast as a successive operation of the seam carving operator.
- Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. Interactive digital photomontage. ACM Trans. Graph, 23, 3 (2004), 294--302. Google Scholar
Digital Library
- Avidan, S. and Shamir, A. Seam carving for content-aware image resizing. ACM Trans. Graph. 26, 3 (2007), 10. Google ScholarDigital Library
- Boykov, Y. and Jolly, M.-P. Interactive graph cuts for optimal boundary & region segmentation of objects in n-d images. In International Conference on Computer Vision (ICCV), volume I. 2001,105--112.Google ScholarCross Ref
- Chen, L., Xie, X., Fan, X., Ma, W., Zhang, H., and Zhou, H. A visual attention model for adapting images on small displays. Multimedia Systems 9, 4 (2003), 353--364.Google ScholarDigital Library
- Davis, J. Mosaics of scenes with moving objects. In Proceedings of CVPR, 1998. Google ScholarDigital Library
- Efros, A.A. and Freeman, W.T. Image quilting for texture synthesis and transfer. In SIGGRAPH 2001, Computer Graphics Proceedings. E. Fiume, ed. ACM/ACM SIGGRAPH, 2001, 341--346. Google ScholarDigital Library
- Fan, X., Xie, X., Zhou, H.-Q., and Ma, W.-Y. Looking into video frames n small displays. In MULTIMEDIA '03: Proceedings of the 11th ACM International Conference on Multimedia. ACM, 2003, 247--250. Google ScholarDigital Library
- Gal, R., Sorkine, O., and Cohen-Or, D. Feature-aware texturing. In Eurographics Symposium on Rendering, 2006. Google ScholarDigital Library
- Itti, L., Koch, C., and Neibur, E. A model of saliency-based visual attention for rapid scene analysis. PAMI20,11 (1999), 1254--1259. Google ScholarDigital Library
- Jia, J., Sun, J., Tang, C.-K., and Shum, H.-Y. Drag-and-drop pasting. In Proceedings of SIGGRAPH, 2006. Google ScholarDigital Library
- Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Graph. 22, 3 (2003), 277--286. Google ScholarDigital Library
- Liu, F and Gleicher, M. Automatic image retargeting with fisheye-view warping. In ACM UIST, 2005,153--162. Google ScholarDigital Library
- Liu, F. and Gleicher, M.Video retargeting: automating pan and scan. In MULTIMEDIA '06: Proceedings of the 14th annual ACM International Conference on Multimedia. ACM. 2006, 241--250. Google ScholarDigital Library
- Liu, H., Xie, X., Ma, W., and Zhang, H. Automatic browsing of large pictures on mobile devices. Proceedings of the 11th ACM International Conference on Multimedia, 2003,148--155. Google ScholarDigital Library
- Roditty, L. and Zwick, U. On dynamic shortest paths problems. In Proceedings of the 12th Annual European Symposium on Algorithms (ESA), 2004, 580--591.Google ScholarCross Ref
- Rother, C, Bordeaux, L., Hamadi, Y., and Blake, A. Autocollage. In Proceedings of SIGGRAPH 2006, 2006. Google ScholarDigital Library
- Rubinsteing, M., Shamir, A., and Avidan, S. Improved seam carving for video retargeting. ACM Trans. Graph. 27, 3 (2008), 10. Google ScholarDigital Library
- Santella, A., Agrawala, M., DeCarlo, D., Salesin, D., and Cohen, M. Gaze-based interaction for semi-automatic photo cropping. In ACM Human Factors in Computing Systems (CHI), 2006. 771--780. Google ScholarDigital Library
- Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. Video textures. In SIGGRAPH'OO: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques. ACM/Addison-Wesley, 2000, 489--498. Google ScholarDigital Library
- Setlur, V., Takagi, S., Raskar, R., Gleicher, M., and Gooch, B. Automatic image retargeting. In The Mobile and Ubiquitous Multimedia (MUM). ACM. 2005. Google ScholarDigital Library
- Simakov, D., Caspi, Y., Shechtman, E., and Irani, M. Summarizing visual data using bidirectional similarity. In Proceedings of CVPR, 2008.Google ScholarCross Ref
- Suh, B., Ling, H., Bederson, B.B., and Jacobs, D.W. Automatic thumbnail cropping and its effectiveness. In UIST03: Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology. ACM, New York, NY, 95--104. Google ScholarDigital Library
- Viola, P. and Jones, M. Rapid object detection using a boosted cascade of simple features. In Conference on Computer Vision and Pattern Recognition (CVPR), 2001.Google ScholarCross Ref
- Wang, J., Bhat, P., Colburn, R. A., Agrawala, M., and Cohen, M. F. Interactive video cutout. ACM Trans. Graph. 24, 3 (2005), 585--594. Google ScholarDigital Library
- Wang, J., Reinders, M., Lagendijk, R., Lindenberg, J., and Kankanhalli, M. Video content presentation on tiny devices. In IEEE International Conference on Multimedia and Expo (ICME), volume 3, 2004,1711--1714.Google Scholar
- Wang, J., Xu, Y., Shum, H.-Y., and Cohen, M. F. Video tooning. ACM Trans. Graph. 23, 3 (2004), 574--583. Google ScholarDigital Library
- Wang, Y.-S., Tai, C.-L., Sorkine, O., and Lee, T.-Y. Optimized scale-and-stretch for image resizing. ACM Trans. Graph. Proceedings of ACM SIGGRAPH ASIA 27, 5 (2008). Google ScholarDigital Library
- Wei, L.-Y., Han, J., Zhou, K., Bao, H., Guo, B., Shum, H.-Y. Inverse texture synthesis. ACM Trans. Graph. 27,3 (2008), 1--9. Google ScholarDigital Library
- Wolf, L., Guttmann, M., and Cohen-Or, D. Non-homogeneous content-driven video-retargeting. In Proceedings of the 11th IEEE International Conference on Computer Vision (ICCV07), 2007,1--6.Google ScholarCross Ref
Index Terms
- Seam carving for media retargeting
Recommendations
Seam carving for content-aware image resizing
Effective resizing of images should not only use geometric constraints, but consider the image content as well. We present a simple image operator called seam carving that supports content-aware image resizing for both reduction and expansion. A seam is ...
Seam carving for content-aware image resizing
SIGGRAPH '07: ACM SIGGRAPH 2007 papersEffective resizing of images should not only use geometric constraints, but consider the image content as well. We present a simple image operator called seam carving that supports content-aware image resizing for both reduction and expansion. A seam is ...
Improved seam carving for video retargeting
Video, like images, should support content aware resizing. We present video retargeting using an improved seam carving operator. Instead of removing 1D seams from 2D images we remove 2D seam manifolds from 3D space-time volumes. To achieve this we ...
Comments