Skip to main content
SpringerLink
Log in

Entropy Minimization for Shadow Removal

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Recently, a method for removing shadows from colour images was developed (Finlayson et al. in IEEE Trans. Pattern Anal. Mach. Intell. 28:59–68, 2006) that relies upon finding a special direction in a 2D chromaticity feature space. This “invariant direction” is that for which particular colour features, when projected into 1D, produce a greyscale image which is approximately invariant to intensity and colour of scene illumination. Thus shadows, which are in essence a particular type of lighting, are greatly attenuated. The main approach to finding this special angle is a camera calibration: a colour target is imaged under many different lights, and the direction that best makes colour patch images equal across illuminants is the invariant direction. Here, we take a different approach. In this work, instead of a camera calibration we aim at finding the invariant direction from evidence in the colour image itself. Specifically, we recognize that producing a 1D projection in the correct invariant direction will result in a 1D distribution of pixel values that have smaller entropy than projecting in the wrong direction. The reason is that the correct projection results in a probability distribution spike, for pixels all the same except differing by the lighting that produced their observed RGB values and therefore lying along a line with orientation equal to the invariant direction. Hence we seek that projection which produces a type of intrinsic, independent of lighting reflectance-information only image by minimizing entropy, and from there go on to remove shadows as previously. To be able to develop an effective description of the entropy-minimization task, we go over to the quadratic entropy, rather than Shannon’s definition. Replacing the observed pixels with a kernel density probability distribution, the quadratic entropy can be written as a very simple formulation, and can be evaluated using the efficient Fast Gauss Transform. The entropy, written in this embodiment, has the advantage that it is more insensitive to quantization than is the usual definition. The resulting algorithm is quite reliable, and the shadow removal step produces good shadow-free colour image results whenever strong shadow edges are present in the image. In most cases studied, entropy has a strong minimum for the invariant direction, revealing a new property of image formation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Barrow, H., & Tenenbaum, J. (1978). Recovering intrinsic scene characteristics from images. In A. Hanson & E. Riseman (Eds.), Computer vision systems (pp. 3–26). New York: Academic Press.

    Google Scholar 

  • Baxter, B., & Roussos, G. (2002). A new error estimate of the fast Gauss transform. SIAM Journal on Scientific Computing, 24(1), 257–259.

    Article  MATH  MathSciNet  Google Scholar 

  • Beatson, R., & Greengard, L. (1997). A short course on fast multipole methods. In M. Ainsworth, J. Levesley, W. Light, & M. Marletta (Eds.), Wavelets, multilevel methods and elliptic PDEs. Oxford: Oxford University Press.

    Google Scholar 

  • Bergner, S., Drew, M. S., & Möller (2009). A tool to create illuminant and reflectance spectra for light-driven graphics and visualization. ACM Transactions on Graphics, 28(5), 1–11.

    Article  Google Scholar 

  • Cho, J.-H., Kwon, T.-G., Jang, D.-G., & Hwang, C.-S. (2005). Moving cast shadow detection and removal for visual traffic surveillance. In: Australian conference on artificial intelligence (pp. 746–755).

  • CIE, (1995). Method of measuring and specifying colour rendering properties of light sources. Publication 13.3, ISBN 978-3900734572. http://www.cie.co.at/publ/abst/13-3-95.html.

  • Comaniciu, D., & Meer, P. (2002). Mean shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 603–619.

    Article  Google Scholar 

  • Daly, S. (1992). The visible difference predictor: an algorithm for the assessment of image fidelity. In: A. Rogowitz and Klein (Eds.): Proceedings of SPIE: Vol. 1666. Human vision, visual processing, and digital display III (pp. 2–15).

  • Drew, M., Chen, C., Hordley, S., & Finlayson, G. (2002). Sensor transforms for invariant image enhancement. In: Tenth color imaging conference: color, science, systems and applications (pp. 325–329).

  • Drew, M., Finlayson, G., & Hordley, S. (2003). Recovery of chromaticity image free from shadows via illumination invariance. In: IEEE workshop on color and photometric methods in computer vision, ICCV’03 (pp. 32–39).

  • Drew, M., Salahuddin, M., & Fathi, A. (2007). A standardized workflow for illumination-invariant image extraction. In: 15th color imaging conference: color, science, systems and applications.

  • Elgammal, A., Duraiswami, R., & Davis, L. S. (2003). Efficient kernel density estimation using the fast Gauss transform with applications to color modeling and tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(11), 1499–1504.

    Article  Google Scholar 

  • Finlayson, G., & Drew, M. (2001). 4-sensor camera calibration for image representation invariant to shading, shadows, lighting, and specularities. In: ICCV’01: international conference on computer vision (pp. II: 473–480).

  • Finlayson, G., & Hordley, S. (2001). Colour constancy at a pixel. Journal of the Optical Society of America A, 18(2), 253–264.

    Article  Google Scholar 

  • Finlayson, G., Drew, M., & Funt, B. (1994). Spectral sharpening: sensor transformations for improved color constancy. Journal of the Optical Society of America A, 11(5), 1553–1563.

    Article  Google Scholar 

  • Finlayson, G., Hordley, S., & Drew, M. (2002). Removing shadows from images. In Lecture Notes in Computer Science : Vol. 2353. ECCV 2002: European conference on computer vision (pp. 823–836). Berlin: Springer.

    Chapter  Google Scholar 

  • Finlayson, G., Drew, M., & Lu, C. (2004). Intrinsic images by entropy minimization. In Lecture Notes in Computer Science : Vol. 3023. ECCV 2004: European conference on computer vision (pp. 582–595). Berlin: Springer.

    Google Scholar 

  • Finlayson, G., Hordley, S., Lu, C., & Drew, M. (2006). On the removal of shadows from images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 59–68.

    Article  Google Scholar 

  • Finlayson, G., Fredembach, C., & Drew, M. S. (2007). Detecting illumination in images. In: ICCV’07: international conference on computer vision.

  • Greengard, L., & Strain, J. (1991). The fast Gauss transform. SIAM Journal on Scientific and Statistical Computing, 12(1), 79–94.

    Article  MATH  MathSciNet  Google Scholar 

  • Hsu, E., Mertens, T., Paris, S., Avidan, S., & Durand, F. (2008). Light mixture estimation for spatially varying white balance. ACM Transactions on Graphics, 27(3), 1–7.

    Article  Google Scholar 

  • Jiang, H., & Drew, M. (2003) Shadow-resistant tracking in video. In: ICME’03: international conference on multimedia and expo (Vol. III, pp. 77–80).

  • Jiang, H., & Drew, M. (2007). Shadow resistant tracking using inertia constraints. Pattern Recognition, 40, 1929–1945.

    Article  MATH  Google Scholar 

  • Land, E., & McCann, J. (1971). Lightness and retinex theory. Journal of the Optical Society of America, 61, 1–11.

    Article  Google Scholar 

  • Li, Z.-N., & Drew, M. (2004). Fundamentals of multimedia. New York: Prentice-Hall.

    Google Scholar 

  • Liu, Z., Huang, K., Tan, T., & Wang, L. (2006). Cast shadow removal with GMM for surface reflectance component. In: ICPR06 (pp. 727–730).

  • Martel-Brisson, N., & Zaccarin, A. (2007). Learning and removing cast shadows through a multidistribution approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 1133–1146.

    Article  Google Scholar 

  • Nadimi, S., & Bhanu, B. (2004). Physical models for moving shadow and object detection in video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 1079–1087.

    Article  Google Scholar 

  • Parzen, E. (1962). On estimation of a probability density function and mode. Annals of Mathematical Statistics, 33, 1065–1076.

    Article  MATH  MathSciNet  Google Scholar 

  • Prati, A., Mikic, I., Trivedi, M., & Cucchiara, R. (2003). Detecting moving shadows: algorithms and evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25, 918–923.

    Article  Google Scholar 

  • Ramanath, R., Snyder, W., Yoo, Y., & Drew, M. S. (2005). Color image processing pipeline in digital still cameras. IEEE Signal Processing, 22(1), 34–43.

    Article  Google Scholar 

  • Renyi, A. (1987). A diary on information theory. New York: Wiley.

    MATH  Google Scholar 

  • Scott, D. (1992). Multivariate density estimation: theory, practice and visualization. New York: Wiley and Kegan Paul.

    Book  MATH  Google Scholar 

  • Stauder, J., Mech, R., & Ostermann, J. (1999). Detection of moving cast shadows for object segmentation. IEEE Transactions on Multimedia, 1, 65–76.

    Article  Google Scholar 

  • Tappen, M., Freeman, W., & Adelson, E. (2003). Recovering intrinsic images from a single image. In: Advances in neural information processing systems 15.

  • Tappen, M., Freeman, W., & Adelson, E. (2005). Recovering intrinsic images from a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 1459–1472.

    Article  Google Scholar 

  • Vrhel, M., Gershon, R., & Iwan, L. (1994). Measurement and analysis of object reflectance spectra. Color Research and Application, 19, 4–9.

    Google Scholar 

  • Weiss, Y. (2001). Deriving intrinsic images from image sequences. In: ICCV01 (Vol. II, pp. 68–75.

  • Weyrich, T., Matusik, W., Pfister, H., Bickel, B., Donner, C., Tu, C., McAndless, J., Lee, J., Ngan, A., Wann Jensen, H., & Gross, M. (2006). Analysis of human faces using a measurement-based skin reflectance model. ACM Transactions on Graphics, 25, 1013–1024.

    Article  Google Scholar 

  • Wyszecki, G., & Stiles, W. (1982). Color science: concepts and methods, quantitative data and formulas (2nd ed.). New York: Wiley,

    Google Scholar 

  • Xu, D., & Principe, J. (1998). Learning from examples with quadratic mutual information. In: Neural networks for signal processing (pp. 155–164).

  • Yang, C., Duraiswami, R., Gumerov, N., & Davis, L. (2003). Improved fast Gauss transform and efficient kernel density estimation. In: International conference on computer vision (pp. 464–471).

Download references

Author information

Authors and Affiliations

  1. University of East Anglia, Norwich, UK

    Graham D. Finlayson

  2. Simon Fraser University, Vancouver, Canada

    Mark S. Drew

  3. Micron Technology, Inc., San Jose, CA, USA

    Cheng Lu

Authors
  1. Graham D. Finlayson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mark S. Drew
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cheng Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mark S. Drew.

Additional information

G.D. Finlayson’s work was supported by the Leverhulme Trust.

M.S. Drew’s work was supported by the Natural Sciences and Engineering Research Council of Canada.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Finlayson, G.D., Drew, M.S. & Lu, C. Entropy Minimization for Shadow Removal. Int J Comput Vis 85, 35–57 (2009). https://doi.org/10.1007/s11263-009-0243-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-009-0243-z

Keywords

Search

Navigation