Skip to main content
Log in

Visual learning and recognition of 3-d objects from appearance

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

The problem of automatically learning object models for recognition and pose estimation is addressed. In contrast to the traditional approach, the recognition problem is formulated as one of matching appearance rather than shape. The appearance of an object in a two-dimensional image depends on its shape, reflectance properties, pose in the scene, and the illumination conditions. While shape and reflectance are intrinsic properties and constant for a rigid object, pose and illumination vary from scene to scene. A compact representation of object appearance is proposed that is parametrized by pose and illumination. For each object of interest, a large set of images is obtained by automatically varying pose and illumination. This image set is compressed to obtain a low-dimensional subspace, called the eigenspace, in which the object is represented as a manifold. Given an unknown input image, the recognition system projects the image to eigenspace. The object is recognized based on the manifold it lies on. The exact position of the projection on the manifold determines the object's pose in the image.

A variety of experiments are conducted using objects with complex appearance characteristics. The performance of the recognition and pose estimation algorithms is studied using over a thousand input images of sample objects. Sensitivity of recognition to the number of eigenspace dimensions and the number of learning samples is analyzed. For the objects used, appearance representation in eigenspaces with less than 20 dimensions produces accurate recognition results with an average pose estimation error of about 1.0 degree. A near real-time recognition system with 20 complex objects in the database has been developed. The paper is concluded with a discussion on various issues related to the proposed learning and recognition methodology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Besl, P.J., and Jain, R.C. 1985. “Three-Dimensional Object Recognition,”ACM Computing Surveys, Vol. 17, No. 1, pp. 75–145.

    Google Scholar 

  • Chen, W.H., Smith, H., and Fralick, S.C. 1977. “A Fast Computational Algorithm for the Discrete Cosine Transform,”IEEE Transactions on Communications, Vol.25, pp.1004–1009 Vol. 18, No. 1, pp. 67–108.

    Google Scholar 

  • Edelman, S. and Weinshall, D. 1991. “A self-organizing multiple-view representation of 3D objects,”Biological Cybernetics Vol. 64, pp. 209–219.

    Google Scholar 

  • Edelman, S., Bulthoff, H., and Weinshall, D. 1989. “Stimulus familiarity determines recognition strategy for novel 3D objects,” A.I. Memo No. 1138, AI Lab, MIT.

  • Fan, T.J., Medioni, G., and Nevatia, R. 1988. “Recognizing 3-D Objects Using Surface Descriptions,”Proc. of Intl. Conference on Computer Vision, pp. 474–481, Florida.

  • Fukunaga, K. 1990.Introduction to Statistical Pattern Recognition, Academic Press, London.

    Google Scholar 

  • Householder, A.S. 1964.The theory of matrices in numerical analysis, Dover Publications, New York.

    Google Scholar 

  • Huttenlocher, D.P., and Ullman, S. 1990. “Recognizing solid objects by alignment with an image,”International Journal of Computer Vision, Vol. 5, No. 2, pp. 195–212.

    Google Scholar 

  • Ikeuchi, K., and Suehiro, T. 1992. “Recognizing Assembly Tasks using Face-Contact Relations,”Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 154–160.

  • Koenderink, J.J., and van Doorn, A.J. 1979. “The internal representation of solid shape with respect to vision,”Biological Cybernetics Vol. 32, pp. 211–216.

    Google Scholar 

  • Mukherjee, S., and Nayar, S.K. 1994. “Appearance Based Recognition of 3D Objects using RBF Networks,” Technical Report, Department of Computer Science, Columbia University (in preparation).

  • Murakami, H., and Kumar, V. 1982. “Efficient Calculation of Primary Images from a Set of Images,”IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 4, No. 5, pp. 511–515.

    Google Scholar 

  • Murase, H., Kimura, E., Yoshimura, M., and Miyake, Y. 1981. “An Improvement of the Auto-Correlation Matrix in Pattern Matching Method and Its Application to Handprinted ‘HI-RAGANA’,”Trans. IECE, Vol. J64-D, No. 3, pp. 276–283.

    Google Scholar 

  • Murase, H., and Lindenbaum, M. 1992. “Spatial Temporal Adaptive Method for Partial Eigenstructure Decomposition of Large Images,”NTT Technical Report No. 6527. AlsoIEEE Transactions on Image Processing (in press).

  • Murase, H., and Nayar, S.K., 1994. “Illumination Planning for Object Recognition in Structured Environments,”Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 31–38.

  • Nayar, S.K., Murase, H., and Nene, S.A. 1994. “Learning, Positioning, and Tracking Visual Appearance,”Proc. IEEE Conf. on Robotics and Automation, pp. 3237–3244.

  • Nene, S.A., and Nayar, S.K. 1994. “Binary Search Through Multiple Dimensions,” Technical Report CUCS-018-94, Department of Computer Science, Columbia University.

  • Oja, E. 1983.Subspace methods of Pattern Recognition, Research Studies Press, Hertfordshire.

    Google Scholar 

  • Poggio, T., and Girosi, F. 1990. “Networks for Approximation and Learning,”Proceedings of the IEEE, Vol. 78, No. 9, pp. 1481–1497.

    Google Scholar 

  • Poggio, T., and Edelman, S. 1990. “A networks that learns to recognize three-dimensional objects,”Nature, Vol. 343, pp. 263–266.

    Google Scholar 

  • Press, W., Flannery, B.P., Teukolsky, S.A., and Vetterling, W.T. 1988.Numerical Recipes in C, Cambridge University Press, Cambridge.

    Google Scholar 

  • Sirovich, L., and Kirby, M. 1987. “Low dimensional procedure for the characterization of human faces,” Journal of Optical Society of America, Vol. 4, No. 3, pp. 519–524.

    Google Scholar 

  • Tarr, M., and Pinker, S. 1989. “Mental rotation and orientation- dependence in shape recognition,”Cognitive Psychology, Vol. 21, pp. 233–282.

    Google Scholar 

  • Turk, M.A., and Pentland, A.P. 1991. “Face Recognition Using Eigenfaces,”Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–591.

  • Ullman, S., and Basri, R. 1991. “Recognition by Linear Combination of Models,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, No. 10, pp. 992–1006.

    Google Scholar 

  • Yang, X., Sarkar, T.K., and Arvas, E. 1989. “A Survey of Conjugate Gradient Algorithms for Solution of Extreme Eigen-Problems of a Symmetric Matrix,”IEEE Transactions on Acoustics, Speech and Signal Processing, Vol. 37, No. 10, pp. 1550–1555.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Murase, H., Nayar, S.K. Visual learning and recognition of 3-d objects from appearance. Int J Comput Vision 14, 5–24 (1995). https://doi.org/10.1007/BF01421486

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01421486

Keywords

Navigation