Abstract
Specular reflections and interreflections produce strong highlights in brightness images. These highlights can cause vision algorithms for segmentation, shape from shading, binocular stereo, and motion estimation to produce erroneous results. A technique is developed for separating the specular and diffuse components of reflection from images. The approach is to use color and polarization information, simultaneously, to obtain constraints on the reflection components at each image point. Polarization yields local and independent estimates of the color of specular reflection. The result is a linear subspace in color space in which the local diffuse component must lie. This subspace constraint is applied to neighboring image points to determine the diffuse component. In contrast to previous separation algorithms, the proposed method can handle highlights on surfaces with substantial texture, smoothly varying diffuse reflectance, and varying material properties. The separation algorithm is applied to several complex scenes with textured objects and strong interreflections. The separation results are then used to solve three problems pertinent to visual perception; determining illumination color, estimating illumination direction, and shape recovery.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Batchelor, B.G., Hill, D.A., and Hodgson, D.C. 1985. Automated Visual Inspection. IFS Publications Ltd.: Bedford, England.
Bajcsy, R., Lee, S.W., and Leonardis, A. 1990. Color image segmentation with detection of highlights and local illumination induced by interreflections. Proc. of Intl. Conf. on Pattern Recognition (ICPR), Atlantic City, NJ.
Beckmann, P. and Spizzichino, A. 1963. The Scattering of Electromagnetic Waves from Rough Surfaces. Pergamon: New York.
Born, M. and Wolf, E. 1965. Principles of Optics. Pergamon: London.
Boult, T.E. and Wolff, L.B. 1991. Physically based edge labeling. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Maui, Hawaii, pp. 656–663.
Brelstaff, G. and Blake, A. 1992. Detecting specular reflections using Lambertian constraints. Proc. of Intl. Conf. on Computer Vision (ICCV), Tampa, Florida, pp. 297–302.
Gershon, R. 1987. The use of color in computational vision. Ph.D. thesis, University of Toronto.
Handbook of Laser Science and Technology. M.J. Marvin (Ed.), CRC Press Inc., Vol. 4, 1988.
Handbook of Optics. W.G. Driscoll (Ed.), McGraw Hill Inc., 1978.
Klinker, G.J. 1993. A Physical Approach to Color Image Understanding. A.K. Peters, Wellesley: Massachusetts.
Klinker, G.J., Shafer, S.A., and Kanade, T. 1990. The measurement of highlights in color images. International Journal of Computer Vision, 2(1):7–32.
Koshikawa, K. 1979. A polarimetric approach to shape understanding. Proc. of Intl. Joint Conf. on Artificial Intelligence (IJCAI), pp. 493–495.
Lee, S.W. 1991. Understanding of surface reflection in computer vision by color and multiple views. Ph.D. thesis, University of Pennsylvania.
Mersch, S. 1984. Polarized lighting for machine vision applications. Proc. of RI/SME Third Annual Applied Machine Vision Conf., Schaumburg, pp. 40–54.
Nayar, S.K., Ikeuchi, K., and Kanade, T. 1991. Surface reflection: Physical and geometrical perspectives. IEEE Trans. on Pattern Analysis and Machine Intelligence, 13(7):611–634.
Novak, C. and Shafer, S. 1992. Anatomy of a color histogram. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 599–605.
Oren, M. and Nayar, S.K. 1994. Seeing beyond Lambert's law. Proc. Third European Conference on Computer Vision, 2:269–280.
Polarization and Remote Sensing. W.G. Egan (Ed.), SPIE, Vol. 1747, July, 1992.
Sato, Y. and Ikeuchi, K. 1993. Temporal-color space analysis of reflection. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 570–576.
Shafer, S. 1985. Using color to separate reflection components. Color Research and Applications, 10:210–218.
Shafer, S., Kanade, T., Klinker, G.J., and Novak, C.L. 1993. Physics-based models for early vision by machine. Proc. SPIE Conf. on Perceiving, Measuring and Using Color, vol. 1250.
Torrance, K. and Sparrow, E. 1967. Theory for off-specular reflection from roughened surfaces. Journal of the Optical Society of America, 57:1105–1114.
Wolff, L.B. and Boult, T.E. 1991. Constraining object features using a polarization reflectance model. IEEE Trans. on Pattern Analysis and Machine Intelligence, 13(7):635–657.
Wolff, L.B. 1992. A diffuse reflectance model for dielectric surfaces. Proc. SPIE Conf. on Optics, Illumination, and Image Sensing for Machine Vision, vol. 1822, pp. 60–73.
Woodham, R.J. 1980. Photometric method for determining surface orientation from multiple images. Optical Engineering, 19(1):139–144.
Rights and permissions
About this article
Cite this article
Nayar, S.K., Fang, XS. & Boult, T. Separation of Reflection Components Using Color and Polarization. International Journal of Computer Vision 21, 163–186 (1997). https://doi.org/10.1023/A:1007937815113
Issue Date:
DOI: https://doi.org/10.1023/A:1007937815113