Skip to main content
SpringerLink
Log in

On Symmetry and Multiple-View Geometry: Structure, Pose, and Calibration from a Single Image

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this paper, we provide a principled explanation of how knowledge in global 3-D structural invariants, typically captured by a group action on a symmetric structure, can dramatically facilitate the task of reconstructing a 3-D scene from one or more images. More importantly, since every symmetric structure admits a “canonical” coordinate frame with respect to which the group action can be naturally represented, the canonical pose between the viewer and this canonical frame can be recovered too, which explains why symmetric objects (e.g., buildings) provide us overwhelming clues to their orientation and position. We give the necessary and sufficient conditions in terms of the symmetry (group) admitted by a structure under which this pose can be uniquely determined. We also characterize, when such conditions are not satisfied, to what extent this pose can be recovered. We show how algorithms from conventional multiple-view geometry, after properly modified and extended, can be directly applied to perform such recovery, from all “hidden images” of one image of the symmetric structure. We also apply our results to a wide range of applications in computer vision and image processing such as camera self-calibration, image segmentation and global orientation, large baseline feature matching, image rendering and photo editing, as well as visual illusions (caused by symmetry if incorrectly assumed).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • A. Criminisi, I. and Zisserman, A. 1999. Single view metrology. In IEEE International Conference on Computer Vision, pp. 434–441.

  • A. Criminisi, I. and Zisserman, A. 2000. Single view metrology. Int. Journal on Computer Vision.

  • Bieberbach, L. 1910. Uber die bewegungsgruppen des ndimensionalen euklidischen raumes mit einem endlichen fundamentalbereich. Nachr, 75–84.

  • Bulatov (www). Polyhedra collection at http://www.physics.orst.edu/~bulatov/polyhedra/index.html.

  • Carlsson, S. 1998. Symmetry in perspective. In European Conference on Computer Vision.

  • Cham, T.J. and Cipolla, R. 1996. Geometric saliency of curve correspondances and grouping of symmetric comntours. In European Conference on Computer Vision, pp. 385-398.

  • Comanicu, D. and Meer, P. 2002. Mean shift: A robust approach toward feature space analysis. In IEEE Transactions on Pattern Analysis & Machine Intelligence, pp. 603–619.

  • Fedorov, E.S. 1885. The elements of the study of figures. Zapiski Imperatorskogo S. Peterburgskogo Mineralogichesgo Obshchestva [Proc. S. Peterb. Mineral. Soc.], 2(21):1–289.

    Google Scholar 

  • Fedorov, E.S. 1891a. Symmetry in the plane. Zapiski Imperatorskogo S. Peterburgskogo Mineralogichesgo Obshchestva [Proc. S. Peterb. Mineral. Soc.], 2(28):345–390.

    Google Scholar 

  • Fedorov, E.S. 1891b. Symmetry of finite figures. Zapiski Imperatorskogo S. Peterburgskogo Mineralogichesgo Obshchestva [Proc. S. Peterb. Mineral. Soc.], 2(28):1–146.

    Google Scholar 

  • Fedorov, E.S. 1971. Symmetry of Crystals. Translated from the 1949 Russian Edition. American Crystallographic Association.

  • Fishler, M. and Bolles, R. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381–395.

    Article  Google Scholar 

  • Garding, J. 1992. Shape from texture for smooth curved surfaces in perspective projection. Journal of Mathematical Imaging and Vision, 2(4):327–350.

    Google Scholar 

  • Garding, J. 1993. Shape from texture and contour by weak isotropy. J. of Artificial Intelligence, 64(2):243–297.

    Article  Google Scholar 

  • Gibson, J. 1950. The Perception of the Visual World. Houghton Mifflin.

  • Gool, L.V., Moons, T., and Proesmans, M. 1996. Mirror and point symmetry under perspective skewing. In Int. Conference on Computer Vision & Pattern Recognition, pp. 285–292.

  • Grünbaum, B. and Shephard, G.C. 1987. Tilings and Patterns. W. H. Freeman and Company.

  • Hong,W., Ma, Y., and Yu, Y. 2004. Reconstruction of 3-D deformed symmetric curves from perspective images without discrete features. In European Conference on Computer Vision.

  • Huang, K., Hong, W., and Ma, Y. 2003. Symmetry-based photoediting. In 9th International Conference on Computer Vision, Workshop on Higher Level Knowledge, Nice, France.

  • Huang, K., Hong, W., Yang, A.Y., and Ma, Y. 2004. Large baseline matching and reconstruction from symmetry cells. In IEEE Int. Conference on Robotics & Automation.

  • Kanatani, K. 1997. Comments on symmetry as a continuous feature. IEEE Transactions on Pattern Analysis & Machine Intelligence, 19(2):246–247.

    Google Scholar 

  • Kiryati, N. and Gofman, Y. 1998. Detecting symmetry in grey level images: The global optimization approach. Int. Journal on Computer Vision, 29(1):29–45.

    Article  Google Scholar 

  • Leung, T. and Malik, J. 1997. On perpendicular texture or: Why do we see more flowers in the distance? In Int. Conference on Computer Vision & Pattern Recognition, pp. 807–813.

  • Liebowitz, D. and Zisserman, A. 1998. Detecting rotational symmetries using normalized convolution. In Int. Conference on Computer Vision & Pattern Recognition, pp. 482–488.

  • Liu, J., Walker, E., and Mundy, J. 1995. Characterizing the stability of 3D invariants derived from 3D translational symmetry. In S.Z. Li, D.P. Mital, E.K. Teoh, and H. Wang (eds.), Recent Developments in Computer Vision, Springer, Berlin, pp. 415–424.

    Google Scholar 

  • Liu, Y. and Colline, R. 1998. Frieze and wallpaper symmetry groups classification under affine and perspective distortion. Technical Report, CMU-RI-TR-98-37, The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA.

    Google Scholar 

  • Ma, Y., KoŠecká, J., and Huang, K. 2002. Rank deficiency condition of the multiple-view matrix for mixed point and line features. In Asian Conference on Computer Vision.

  • Ma, Y., KoŠecká, J., Soatto, S., and Sastry, S. 2003. An Invitation to 3-D Vision: From Images to Geometric Models. Springer-Verlag.

  • Ma, Y., Vidal, R., KoŠecká, J., and Sastry, S. 2000. Kruppa's equations revisited: Its degeneracy, renormalization, and relations to chirality. In European Conference on Computer Vision.

  • Malik, J. and Rosenholtz, R. 1997. Computing local surface orientation and shape from texture for curved surfaces. Int. Journal on Computer Vision, 23:149–168.

    Article  Google Scholar 

  • Marola, G. 1989. On the detection of the axes of symmetry of symmetric and almost symmetric planar images. IEEE Transactions on Pattern Analysis & Machine Intelligence, 11(1):104–108.

    Google Scholar 

  • Marr, D. 1982. Vision. W. H. Freeman and Company.

  • Martin, G.E. 1975. The Foundations of Geometry and the Non-Euclidean Plane. Springer-Verlag.

  • Mitsumoto, H., Tamura, S., Okazaki, K., and Fukui, Y. 1992. 3-D reconstruction using mirror images based on a plane symmetry recovering method. IEEE Transactions on Pattern Analysis & Machine Intelligence, 14(9):941–946.

    Google Scholar 

  • Mukherjee, D.P., Zisserman, A., and Brady, J.M. 1995. Shape from symmetry—Detecting and exploiting symmetry in affine images. Phil. Trans. Royal Soc. London A, 351:77–106.

    Google Scholar 

  • Plamer, S.E. 1999. Vision Science: Photons to Phenomenology. The MIT Press.

  • Rosenholtz, R. and Malik, J. 1997. Surface orientation from texture: Isotropy or homogeneity(or both)? Vision Research, 37(16):2283–2293.

    Article  Google Scholar 

  • Rothwell, C.A., Forsyth, D.A., Zisserman, A., and Mundy, J.L. 1993. Extracting projective structure from single perspective views of3D point sets. In IEEE International Conference on Computer Vision, pp. 573–582.

  • Schaffalitzky, F. and Zisserman, A. 2000. Planar grouping for automatic detection of vanishing lines and points. Image and Vision Computing, 18:647–658.

    Article  Google Scholar 

  • Sun, C. and Sherrah, J. 1997. 3D symmetry detection using the extended gaussian image. IEEE Transactions on Pattern Analysis & Machine Intelligence, 19(2):164–169.

    Google Scholar 

  • Vetter, T. and Poggio, T. 1994. Symmetric 3D objects are an easy case for 2D object recognition. Spatial Vision, 8:443–453.

    Google Scholar 

  • Weng, J., Huang, T.S., and Ahuja, N. 1993. Motion and Structure from Image Sequences. Springer Verlag.

  • Weyl, H. 1952. Symmetry. Princeton Univ. Press.

  • Witkin, A.P. 1988. Recovering surface shape and orientation from texture. J. of Artificial Intelligence, 17:17–45.

    Article  Google Scholar 

  • Yang, A.Y., Rao, S., Huang, K., Hong, W., and Ma, Y. 2003. Geometric segmentation of perspective images based on symmetry groups. In IEEE International Conference on Computer Vision.

  • Zabrodsky, H., Peleg, S., and Avnir, D. 1995. Symmetry as a continuous feature. IEEE Transactions on Pattern Analysis & Machine Intelligence, 17(12):1154–1166.

    Google Scholar 

  • Zabrodsky, H. and Weinshall, D. 1997. Using bilateral symmetry to improve 3D reconstruction from image sequences. Comp. Vision and Image Understanding, 67:48–57.

    Article  Google Scholar 

  • Zhang, Z., Deriche, R., Faugeras, O., and Luong, Q.-T. 1995. A robust technique for matching two uncalibrated images through the recovery of the unknownepipolar geometry. Artificial Intelligence, 78(1/2):87–119.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical & Computer Engineering, University of Illinois at Urbana-Champaign, 1308 West Main St., Urbana, IL, 61801, USA

    Wei Hong, Allen Yang Yang, Kun Huang & Yi Ma

Authors
  1. Wei Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Allen Yang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kun Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yi Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hong, W., Yang, A.Y., Huang, K. et al. On Symmetry and Multiple-View Geometry: Structure, Pose, and Calibration from a Single Image. International Journal of Computer Vision 60, 241–265 (2004). https://doi.org/10.1023/B:VISI.0000036837.76476.10

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:VISI.0000036837.76476.10

Search

Navigation