Skip to main content
SpringerLink
Log in

3D object recognition: Representation and matching

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

Three-dimensional object recognition entails a number of fundamental problems in computer vision: representation of a 3D object, identification of the object from its image, estimation of its position and orientation, and registration of multiple views of the object for automatic model construction. This paper surveys three of those topics, namely representation, matching, and pose estimation. It also presents an overview of the free-form surface matching problem, and describes COSMOS, our framework for representing and recognizing free-form objects. The COSMOS system recognizes arbitrarily curved 3D rigid objects from a single view using dense surface data. We present both the theoretical aspects and the experimental results of a prototype recognition system based on COSMOS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Arman F. and Aggarwal J.K. 1993. Model-based object recognition in dense range images—A review. ACM Computing Surveys 25(1): 5–43.

    Google Scholar 

  • Bajcsy R. and Solina F. 1987. Three-dimensional object representation revisited. In: Proc. First IEEE International Conference on Computer Vision, London, pp. 231–240.

  • Barrow H.G. and Burstall R.M. 1976. Subgraph isomorphism, matching relational structures and maximal cliques. Information Processing Letters 4: 83–84.

    Google Scholar 

  • Basri R. and Ullman S. 1988. The alignment of objects with smooth surfaces. In: Proc. Second IEEE International Conference on Computer Vision, Tarpon Springs, FL, pp. 482–488.

  • Besl P.J. 1988. Surfaces in Range Image Understanding, Springer Series in Perception Engineering. Springer-Verlag.

  • Besl P.J. 1990. The free-form surface matching problem. In: Freeman H. (Ed.), MachineVision for Three-Dimensional Scenes. Academic Press, pp. 25–71.

  • Besl P.J. and Jain R. 1985. Three-dimensional object recognition. ACM Computing Surveys 17: 75–145.

    Google Scholar 

  • Besl P.J. and McKay N.D. 1992. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence 14(2): 239–256.

    Google Scholar 

  • Biederman I. 1987. Recognition-by-components: A theory of human image understanding. Psychological Review 94(2): 115–147.

    Google Scholar 

  • Bolles R. and Horaud P. 1986. 3DPO: A three-dimensional part orientation system. International Journal of Robotics Research 5(3): 3–26.

    Google Scholar 

  • Borges D.L. and Fisher R.B. 1997. Class-based recognition of 3D objects represented by volumetric primitives. Image and Vision Computing 15: 655–664.

    Google Scholar 

  • Breen D., Whitaker R., Rose E., and Tuceryan M. 1996. Interactive occlusion and automatic object placement for augmented reality. In: Proc. Eurographics '96, Poitiers, France. Elsevier Science Publishers, B.V., pp. 11–22.

  • Brooks R.A. 1983. Model-based three-dimensional interpretations of two-dimensional images. IEEE Transactions on Pattern Analysis and Machine Intelligence 5: 140–150.

    Google Scholar 

  • Chen C. and Kak A. 1989. A robot vision system for recognizing 3-D objects in low-order polynomial time. IEEE Transactions on Systems, Man, and Cybernetics 19(6): 1535–1563.

    Google Scholar 

  • Chen T.-W. and Lin W.-C. 1994. A neural network approach to CSGbased 3-D object recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 16(7): 719–726.

    Google Scholar 

  • Chen J.-L. and Stockman G.C. 1996. Determining pose of 3D objects with curved surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence 18: 52–56.

    Google Scholar 

  • Chin R.T. and Dyer C.R. 1986. Model-based recognition in robot vision. ACM Computing Surveys 18(1): 67–108.

    Google Scholar 

  • Connell J.H. and Brady M. 1987. Generating and generalizing models of visual objects. Artificial Intelligence 31: 159–183.

    Google Scholar 

  • Delingette H., Hebert M., and Ikeuchi K. 1993. A spherical representation for the recognition of curved objects. In: Proc. Fourth IEEE International Conference on Computer Vision, Berlin, pp. 103–112.

  • Dickinson S.J., Pentland A.P., and Rosenfeld A. 1992. 3-D shape recovery using distributed aspect matching. IEEE Transactions on Pattern Analysis and Machine Intelligence 14: 174–198.

    Google Scholar 

  • Dorai C. 1996. COSMOS: A framework for representation and recognition of 3D free-form objects. PhD Thesis, Department of Computer Science, Michigan State University, East Lansing.

    Google Scholar 

  • Dorai C. and Jain A.K. 1997a. COSMOS—A representation scheme for 3D free-form objects. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(10): 1115–1130.

    Google Scholar 

  • Dorai C. and Jain A.K. 1997b. Shape spectrumbased viewgrouping and matching of 3D free-form objects. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(10): 1139–1146.

    Google Scholar 

  • Eggert D. and Bowyer K. 1993. Computing the perspective projection aspect graph of solids of revolution. IEEE Transactions on Pattern Analysis and Machine Intelligence 15: 109–128.

    Google Scholar 

  • Executive Office of the President, Office of Science and Technology Policy. 1989. The Federal High Performance Computing Program. Washington, D.C.

  • Fan T.-J., Medioni G., and Nevatia R. 1989. Recognizing 3-D objects using surface descriptions. IEEE Transactions on Pattern Analysis and Machine Intelligence 11(11): 1140–1157.

    Google Scholar 

  • Faugeras O. and Hebert M. 1986. The representation, recognition, and locating of 3-D objects. International Journal of Robotics Research 5(3): 27–52.

    Google Scholar 

  • Ferrie F.P., Mathur S., and Soucy G. 1993. Feature extraction for 3-D model building and object recognition. In: Jain A.K. and Flynn P.J. (Eds.), Three-Dimensional Object Recognition Systems. Elsevier Science Publishers, B.V., Amsterdam, The Netherlands, pp. 57–88.

    Google Scholar 

  • Fischler M. and Bolles R. 1981. Random consensus: A paradigm for model-fitting with applications in image analysis and automated cartography. Communications of the ACM 24: 381–395.

    Google Scholar 

  • Flynn P.J. and Jain A.K. 1991. BONSAI: 3D object recognition using constrained search. IEEE Transactions on Pattern Analysis and Machine Intelligence 13(10): 1066–1075.

    Google Scholar 

  • Flynn P.J. and Jain A.K. 1992. 3D object recognition using invariant feature indexing of interpretation tables. CVGIP: Image Understanding 55(2): 119–129.

    Google Scholar 

  • Flynn P.J. and Jain A.K. 1994. Three-dimensional object recognition. In: Young T.Y. (Ed.), Handbook of Pattern Recognition and Image Processing, Vol. 2. Academic Press, ch. 14, pp. 497–541.

  • Grimson W.E.L. and Lozano-Pérez T. 1984. Model-based recognition and localization from sparse range or tactile data. International Journal of Robotics Research 3: 3–35.

    Google Scholar 

  • Grimson W.E.L. and Lozano-Pérez T. 1987. Localizing overlapping parts by searching the interpretation tree. IEEE Transactions on Pattern Analysis and Machine Intelligence 9: 469–482.

    Google Scholar 

  • Grimson W., Lozano-Perez T., Wells W.M. III, Ettinger G., White S., and Kikinis R. 1994. An automatic registration method for frameless stereotaxy, image guided surgery, and enhanced reality visualization. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, Seattle, Washington, pp. 430–436.

  • Gupta A. and Bajcsy R. 1992. Surface and volumetric segmentation of range images using biquadrics and superquadrics. In: Proc. 11th International Conference on Pattern Recognition, The Hague, The Netherlands, pp. 158–162.

  • Gupta A., Bogoni L., and Bajcsy R. 1989. Quantitative and qualitative measures for the evaluation of the superquadric models. In: Proc. IEEE Workshop on Interpretation of 3D Scenes, Austin, pp. 162–169.

  • Heap A.J. and Hogg D.C. 1995. Automated pivot location for the Cartesian-polar hybrid point distribution model. In: Proc. 6th British Machine Vision Conference, Birmingham, U.K., pp. 97–106.

  • Horn B.K.P. 1984. Extended Gaussian image. In: Proceedings of the IEEE 72: 1671–1686.

    Google Scholar 

  • Horowitz B. and Pentland A.P. 1991. Recovery of non-rigid motion and structure. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, Maui, Hawaii, pp. 288–293.

  • Huttenlocher D.P. and Ullman S. 1990. Recognizing solid objects by alignment with an image. International Journal of Computer Vision 5(2): 195–212.

    Google Scholar 

  • Ikeuchi K. and Hong K.S. 1991. Determining linear shape change: Toward automatic generation of object recognition programs. CVGIP: Image Understanding 53(2): 154–170.

    Google Scholar 

  • Jain A.K. and Dubes R.C. 1988. Algorithms for Clustering Data. NJ, Prentice Hall, Englewood Cliffs.

    Google Scholar 

  • Jain A.K. and Flynn P.J. (Eds.). 1993. 3D Object Recognition Systems. Elsevier Science Publishers, B.V., Amsterdam, The Netherlands.

    Google Scholar 

  • Jain A.K. and Hoffman R.L. 1988. Evidence-based recognition of 3-D objects. IEEE Transactions on Pattern Analysis and Machine Intelligence 10: 783–802.

    Google Scholar 

  • Jarvis R. 1993. Range sensing for computer vision. In: Jain A.K. and Flynn P.J. (Eds.), Three-dimensional Object Recognition Systems. Elsevier Science Publishers, B.V., Amsterdam, The Netherlands, pp. 17–56.

    Google Scholar 

  • Kalawsky R.S. 1993. The Science of Virtual Reality and Virtual Environments. Addison Wesley.

  • Kang S.B. and Ikeuchi K. 1993. The complex EGI: A new representation for 3-D pose determination. IEEE Transactions on Pattern Analysis and Machine Intelligence 15(7): 707–721.

    Google Scholar 

  • Keren D., Cooper D., and Subrahmonia J. 1994. Describing complicated objects by implicit polynomials. IEEE Transactions on Pattern Analysis and Machine Intelligence 16: 38–53.

    Google Scholar 

  • Koenderink J.J. and van Doorn A.J. 1979. Internal representation of solid shape with respect to vision. Biological Cybernetics 32(4): 211–216.

    Google Scholar 

  • Koenderink J.J. and van Doorn A.J. 1992. Surface shape and curvature scales. Image and Vision Computing 10: 557–565.

    Google Scholar 

  • Krishnapuram R. and Casasent D. 1989. Determination of threedimensional object location and orientation from range images. IEEE Transactions on Pattern Analysis and Machine Intelligence 11(11): 1158–1167.

    Google Scholar 

  • Lamdan Y., Schwartz J.T., and Wolfson H.J. 1990. Affine invariant model-based object recognition. IEEE Transactions on Robotics and Automation 6(5): 578–589.

    Google Scholar 

  • Lamdan Y. and Wolfson H. 1988. Geometric hashing: A general and efficient model-based recognition scheme. In: Proc. Second IEEE International Conference on Computer Vision, Tarpon Springs, Florida, pp. 238–249.

  • Lanitis A., Taylor C.J., and Cootes T.F. 1997. Automatic interpretation and coding of face images using flexible models. IEEE Transactions on Pattern Analysis and Machine Intelligence 19: 743–756.

    Google Scholar 

  • Liang P. and Taubes C.H. 1994. Orientation-based differential geometric representations for computer vision applications. IEEE Transactions on Pattern Analysis and Machine Intelligence 16(3): 249–258.

    Google Scholar 

  • Lowe D.G. 1987. Three-dimensional object recognition from single two-dimensional images. Artificial Intelligence 31: 355–395.

    Google Scholar 

  • Marr D. and Nishihara H.K. 1978. Representation and recognition of the spatial organization of three-dimensional shapes. Proc. Royal Society, London, ser. B 200: 269–294.

    Google Scholar 

  • Matsuo H. and Iwata A. 1994. 3-D object recognition using MEGI model from range data. In: Proc. 12th International Conference on Pattern Recognition, Jerusalem, Israel, pp. 843–846.

  • Mercer C.R. and Beheim G. 1990. Fiber-optic projected-fringe digital interferometry. In: Hologram Interferometry and Speckle Metrology Proceedings, Bethel, CT. Society for Experimental Mechanics, pp. 210–216.

  • Murase H. and Nayar S.K. 1995. Visual learning and recognition of 3-D objects from appearance. International Journal of Computer Vision 14(1): 5–24.

    Google Scholar 

  • Nalwa V.S. 1989. Representing oriented piecewise C2 surfaces. International Journal of Computer Vision 3: 131–153.

    Google Scholar 

  • Pentland A.P. 1986. Perceptual organization and the representation of natural form. Artificial Intelligence. 28: 293–331.

    Google Scholar 

  • Pentland A.P. 1990. Automatic extraction of deformable part models. International Journal of Computer Vision 4: 107–126.

    Google Scholar 

  • Plantinga W.H. and Dyer C.R. 1990. Visibility, occlusion, and the aspect graph. International Journal of Computer Vision 5: 137–160.

    Google Scholar 

  • Poggio T. and Edelman S. 1990. A network that learns to recognize three-dimensional objects. Nature 343: 263–266.

    Google Scholar 

  • Ponce J., Hebert M., and Zisserman A. 1996. Report on the 1996 international workshop on object representation in computer vision. In: Ponce J., Zisserman A., and Hebert M. (Eds.), Proc. Intl. Workshop on Object Representation in Computer Vision II, pp. 1–8.

  • Ponce J., Hoogs A., and Kreigman D.J. 1992. On using CAD models to compute the pose of curved 3D objects. CVGIP: Image Understanding 55(2): 184–197.

    Google Scholar 

  • Ponce J., Kriegman D.J., Petitjean S., Sullivan S., Taubin G., and Vijayakumar B. 1993. Representations and algorithms for 3D curved object recognition. In: Jain A.K. and Flynn P.J. (Eds.), Three-Dimensional Object Recognition Systems. Elsevier Science Publishers, B.V., Amsterdam, The Netherlands, pp. 17–56.

    Google Scholar 

  • Raja N.S. and Jain A.K. 1994. Obtaining generic parts from range images using a multi-view representation. CVGIP: Image Understanding 60: 44–64.

    Google Scholar 

  • Roberts, L. 1965. Machine perception of three-dimensional solids. In: Tippett J.T., Berkowitz D.A., Clapp L.C., Koester C.J., and Alexander Vanderburgh J. (Eds.), Optical and Electro-Optical Information Processing. MIT Press, Cambridge, Massachusetts, pp. 159–197.

    Google Scholar 

  • Sallam M. and Bowyer K. 1994. Registering time sequences of mammograms using a two-dimensional image unwarping technique. In: Second International Workshop on Digital Mammography, pp. 121–130.

  • Samet H. 1990. The Design and Analysis of Spatial Data Structures. Addison-Wesley.

  • Seales W.B. and Dyer C.R. 1992. Viewpoint from occluding contour. CVGIP: Image Understanding 55(2): 198–211.

    Google Scholar 

  • Silberberg T.M., Davis L., and Harwood H. 1984. An iterative Hough procedure for three-dimensional object recognition. Pattern Recognition 17(6): 621–629.

    Google Scholar 

  • Sinha S.S. and Jain R. 1994. Range image analysis. In: Young T.Y. (Ed.), Handbook of Pattern Recognition and Image Processing: Computer Vision, Vol. 2. Academic Press, ch. 14, pp. 185–237.

  • Solina F. and Bajcsy R. 1990. Recovery of parametric models from range images: The case for superquadrics with global deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence 12: 131–147.

    Google Scholar 

  • Stark L. and Bowyer K.W. 1991. Achieving generalized object recognition through reasoning about association of function to structure. IEEE Transactions on Pattern Analysis and Machine Intelligence 13: 1097–1104.

    Google Scholar 

  • Stein F. and Medioni G. 1992. Structural indexing: Efficient 3-D object recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 14(2): 125–145.

    Google Scholar 

  • Stockman G.C. 1987. Object recognition and localization via pose clustering. Computer Vision, Graphics, and Image Processing 40: 361–387.

    Google Scholar 

  • Suetens P., Fua P., and Hanson A.J. 1992. Computational strategies for object recognition. ACM Computing Surveys 24: 5–61.

    Google Scholar 

  • Swets D.L. 1996. The self-organizing hierarchical optimal subspace learning and inference framework for object recognition. PhD Thesis, Michigan State University, Department of Computer Science, East Lansing, Michigan.

    Google Scholar 

  • Taubin G., Cukierman F., Sullivan S., Ponce J., and Kreigman D.J. 1992. Parametrized and fitting bounded algebraic curves and surfaces. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, Champaign, Illinois, pp. 103–108.

  • Terzopoulos D. and Metaxas D. 1991. Dynamic 3D models with local and global deformations: Deformable superquadrics. IEEE Transactions on Pattern Analysis and Machine Intelligence 13: 703–714.

    Google Scholar 

  • Turney J.L., Mudge T.N., and Volz R.A. 1985. Recognizing partially occluded parts. IEEE Transactions on Pattern Analysis and Machine Intelligence 7(4): 410–421.

    Google Scholar 

  • Ullman S. and Basri R. 1991. Recognition by linear combination of models. IEEE Transactions on Pattern Analysis and Machine Intelligence 13: 992–1006.

    Google Scholar 

  • Umeyama S. 1993. Parameterized point pattern matching and its application to recognition of object families. IEEE Transactions on Pattern Analysis and Machine Intelligence 15(2): 136–144.

    Google Scholar 

  • Vayda A. and Kak A. 1991. A robot vision systems for recognition of generic shaped objects. CVGIP: Image Understanding, 54: 1–46.

    Google Scholar 

  • Vemuri B. and Aggarwal J. 1987. Representation and recognition of objects from dense range maps. IEEE Transactions on Circuits and Systems CAS-34: 1351–1363.

    Google Scholar 

  • Weld D.S. (Ed.). 1995. The role of intelligent systems in the National Information Infrastructure. AI Magazine 16(3): 45–64.

    Google Scholar 

  • Wong A.K.C., Lu S.W., and Rioux M. 1989. Recognition and shape synthesis of 3D objects based on attributed hypergraphs. IEEE Transactions on Pattern Analysis and Machine Intelligence 11(3): 279–290.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Michigan State University, East Lansing, Michigan, 48824

    Anil K. Jain

  2. IBM T.J. Watson Research Center, P.O. Box 704, Yorktown Heights, New York, 10598

    Chitra Dorai

Authors
  1. Anil K. Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chitra Dorai
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jain, A.K., Dorai, C. 3D object recognition: Representation and matching. Statistics and Computing 10, 167–182 (2000). https://doi.org/10.1023/A:1008998410728

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008998410728

Search

Navigation