Skip to main content
SpringerLink
Log in

DaLI: Deformation and Light Invariant Descriptor

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Recent advances in 3D shape analysis and recognition have shown that heat diffusion theory can be effectively used to describe local features of deforming and scaling surfaces. In this paper, we show how this description can be used to characterize 2D image patches, and introduce DaLI, a novel feature point descriptor with high resilience to non-rigid image transformations and illumination changes. In order to build the descriptor, 2D image patches are initially treated as 3D surfaces. Patches are then described in terms of a heat kernel signature, which captures both local and global information, and shows a high degree of invariance to non-linear image warps. In addition, by further applying a logarithmic sampling and a Fourier transform, invariance to photometric changes is achieved. Finally, the descriptor is compacted by mapping it onto a low dimensional subspace computed using Principal Component Analysis, allowing for an efficient matching. A thorough experimental validation demonstrates that DaLI is significantly more discriminative and robust to illuminations changes and image transformations than state of the art descriptors, even those specifically designed to describe non-rigid deformations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. http://www.iri.upc.edu/people/esimo/research/dali/

  2. Again, we only compare against DAISY and SIFT, as these are the descriptors which have been more competitive in the experiments with the full dataset.

    Fig. 18
    figure 18

    Mean detection accuracy on two real world videos from Moreno-Noguer and Fua (2013). In the top row we show three example frames from each video. In the bottom row we plot the accuracy for each frame for three descriptors: DaLI, DAISY and SIFT. Additionally the mean for each descriptor is displayed as a dashed line

    Full size image

  3. http://www.iri.upc.edu/people/esimo/research/dali/

References

  • Aflalo, Y., Bronstein, E. M., Bronstein, M. M., & Kimmel, R. (2011). Deformable shape retrieval by learning diffusion kernels. In In Proc. SSVM.

  • Aubry, M., Schlickewei, U., & Cremers, D. (2011). The wave kernel signature: A quantum mechanical approach to shape analysis. In Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on (pp. 1626–1633).

  • Bay, H., Tuytelaars, T., & Gool, L. V. (2006). SURF: Speeded up robust features. In European Conference on Computer Vision (pp. 404–417).

  • Belongie, S., Malik, J., & Puzicha, J. (2002). Shape matching and object recognition using shape contexts. IEEE Transactions Pattern Analysis and Machine Intelligence, 24(4), 509–522.

    Article  Google Scholar 

  • Berg, A., Berg, T., & Malik, J. (2005). Shape matching and object recognition using low distortion correspondences. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 26–33).

  • Bronstein, A., Bronstein, M., Bruckstein, A., & Kimmel, R. (2007). Analysis of two-dimensional non-rigid shapes. International Journal of Computer Vision, 78(1), 67–88.

    Article  Google Scholar 

  • Bronstein, M., & Kokkinos, I. (2010). Scale-invariant heat kernel signatures for non-rigid shape recognition. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 1704–1711).

  • Cai, H., Mikolajczyk, K., & Matas, J. (2011). Learning linear discriminant projections for dimensionality reduction of image descriptors. IEEE Transactions Pattern Analysis and Machine Intelligence, 33(2), 338–352.

    Article  Google Scholar 

  • Calonder, M., Lepetit, V., Ozuysa, M., Trzcinski, T., Strecha, C., & Fua, P. (2012). BRIEF: Computing a local binary descriptor very fast. IEEE Transactions Pattern Analysis and Machine Intelligence, 34(7), 1281–1298.

    Article  Google Scholar 

  • Chavel, I. (1984). Eigenvalues in Riemannian geometry. London: London Academic Press.

    MATH  Google Scholar 

  • Cheng, H., Liu, Z., Zheng, N., & Yang, J. (2008). A deformable local image descriptor. In IEEE Conference on Computer Vision and Pattern Recognition.

  • Cho, M., Lee, J., & Lee, K. (2009). Feature correspondence and deformable object matching via agglomerative correspondence clustering. In International Conference on Computer Vision (pp. 1280–1287).

  • Fan, B., Wu, F., & Hu, Z. (2012). Rotationally invariant descriptors using intensity order pooling. IEEE Transactions Pattern Analysis and Machine Intelligence, 34(10), 2031–2045.

    Article  Google Scholar 

  • Gębal, K., Bærentzen, J. A., Aanæs, H., & Larsen, R. (2009). Shape analysis using the auto diffusion function. In Proceedings of the Symposium on Geometry Processing, SGP ’09 (pp. 1405–1413).

  • de Goes, F., Goldenstein, S., & Velho, L. (2008). A hierarchical segmentation of articulated bodies. In Proceedings of the Symposium on Geometry Processing, SGP ’08 (pp. 1349–1356).

  • Gupta, R., & Mittal, A. (2007). Illumination and Affine-Invariant Point Matching using an Ordinal Approach. In International Conference on Computer Vision.

  • Gupta, R., & Mittal, A. (2008). Smd: A locally stable monotonic change invariant feature descriptor. In European Conference on Computer Vision (pp. 265–277).

  • Gupta, R., Patil, H., & Mittal, A. (2010). Robust order-based methods for feature description. In IEEE Conference on Computer Vision and Pattern Recognition.

  • Heikkilä, M., Pietikäinen, M., & Schmid, C. (2009). Description of interest regions with local binary patterns. Pattern Recognition, 42(3), 425–436.

    Article  MATH  Google Scholar 

  • Ke, Y., & Sukthankar, R. (2004). PCA-SIFT: a more distinctive representation for local image descriptors. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 506–513).

  • Kokkinos, I., Bronstein, M., & Yuille, A. (2012). Dense Scale Invariant Descriptors for Images and Surfaces. Research Report RR-7914, INRIA.

  • Kovnatsky, A., Bronstein, M., Bronstein, A., & Kimmel, R. (2011). Photometric heat kernel signatures. In International Conference on Scale Space and Variational Methods in Computer Vision (pp. 616–627).

  • Leordeanu, M., & Hebert, M. (2005). A spectral technique for correspondence problems using pairwise constraints. In International Conference on Computer Vision (pp. 1482–1489).

  • Lévy, B. (2006). Laplace-Beltrami Eigenfunctions: Towards an Algorithm that Understands Geometry. In IEEE International Conference on Shape Modeling and Applications - SMI 2006 (p. 13).

  • Ling, H., & Jacobs, D. (2005). Deformation invariant image matching. In International Conference on Computer Vision (pp. 1466–1473).

  • Ling, H., & Jacobs, D. (2007). Shape classification using the inner-distance. IEEE Transactions Pattern Analysis and Machine Intelligence, 29(2), 286–299.

    Article  Google Scholar 

  • Ling, H., Yang, X., & Latecki, L. (2010). Balancing deformability and discriminability for shape matching. In European Conference on Computer Vision.

  • Litman, R., & Bronstein, A. (2014). Learning spectral descriptors for deformable shape correspondence. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 36(1), 171–180.

    Article  Google Scholar 

  • Lowe, D. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    Article  Google Scholar 

  • Mikolajczyk, K., & Schmid, C. (2005). A performance evaluation of local descriptors. IEEE Transactions Pattern Analysis and Machine Intelligence, 10(27), 1615–1630.

    Article  Google Scholar 

  • Morel, J., & Yu, G. (2009). ASIFT: A new framework for fully affine invariant image comparison. SIAM Journal on Imaging Sciences, 2(2), 438–469.

    Article  MATH  MathSciNet  Google Scholar 

  • Moreno-Noguer, F. (2011). Deformation and illumination invariant feature point descriptor. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 1593–1600).

  • Moreno-Noguer, F., & Fua, P. (2013). Stochastic exploration of ambiguities for nonrigid shape recovery. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 35(2), 463–475.

    Article  Google Scholar 

  • Philbin, J., Isard, M., Sivic, J., & Zisserman, A. (2010). Descriptor learning for efficient retrieval. In European Conference on Computer Vision (pp. 677–691).

  • Pinkall, U., & Polthier, K. (1993). Computing discrete minimal surfaces and their conjugates. Experimental Mathematics, 2(1), 15–36.

    Article  MATH  MathSciNet  Google Scholar 

  • Raviv, D., Bronstein, M. M., Sochen, N., Bronstein, A. M., & Kimmel, R. (2011). Affine-invariant diffusion geometry for the analysis of deformable 3d shapes. In IEEE Conference on Computer Vision and Pattern Recognition.

  • Reuter, M., Wolter, F., & Peinecke, N. (2006). Laplace-beltrami spectra as ’shape-dna’ of surfaces and solids. Computer Aided Design, 38(4), 342–366.

    Article  Google Scholar 

  • Rustamov, R. (2007). Laplace-beltrami eigenfunctions for deformation invariant shape representation. In Eurographics Symposium on Geometry Processing (pp. 225–233).

  • Sanchez, J., Ostlund, J., Fua, P., & Moreno-Noguer, F. (2010). Simultaneous pose, correspondence and non-rigid shape. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 1189–1196).

  • Serradell, E., Glowacki, P., Kybic, J., Moreno-Noguer, F., & Fua, P. (2012). Robust non-rigid registration of 2d and 3d graphs. In IEEE Conference on Computer Vision and Pattern Recognition.

  • Shi, L., Yu, Y., & Feng, N. B. W. W. (2006). A fast multigrid algorithm for mesh deformation. ACM SIGGRAPH, 25(3), 1108–1117.

    Article  Google Scholar 

  • Sochen, N., Kimmel, R., & Malladi, R. (1998). A general framework for low level vision. IEEE Transactions on Image Processing, 7(3), 310–318.

    Article  MATH  MathSciNet  Google Scholar 

  • Strecha, C., Bronstein, A. M., Bronstein, M. M., & Fua, P. (2012). LDAHash: Improved matching with smaller descriptors. IEEE Transactions Pattern Analysis and Machine Intelligence, 34(1), 66–78.

    Article  Google Scholar 

  • Sun, J., Ovsjanikov, M., & Guibas, L. (2009). A concise and provably informative multi-scale signature based on heat diffusion. In Eurographics Symposium on Geometry Processing (pp. 1383–1392).

  • Tang, F., Lim, S.H., Chang, N., & Tao, H. (2009). A novel feature descriptor invariant to complex brightness changes. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 2631–2638).

  • Tola, E., Lepetit, V., & Fua, P. (2010). Daisy: An efficient dense descriptor applied to wide-baseline stereo. IEEE Transactions Pattern Analysis and Machine Intelligence, 32(5), 815–830.

    Article  Google Scholar 

  • Torresani, L., Kolmogorov, V., & Rother, C. (2008). Feature correspondence via graph matching: Models and global optimization. In European Conference on Computer Vision (pp. 596–609).

  • Trulls, E., Kokkinos, I., Sanfeliu, A., & Moreno-Noguer, F. (2013). Dense segmentation-aware descriptors. In IEEE Conference on Computer Vision and Pattern Recognition.

  • Trulls, E., Tsogkas, S., Kokkinos, I., Sanfeliu, A., & Moreno-Noguer, F. (2014). Segmentation-aware deformable part models. In IEEE Conference on Computer Vision and Pattern Recognition.

  • Vaxman, A., Ben-Chen, M., & Gotsman, C. (2010). A multi-resolution approach to heat kernels on discrete surfaces. ACM SIGGRAPH, 29(4), 121.

    Article  Google Scholar 

  • Vedaldi, A., & Fulkerson, B. (2008). VLFeat: An open and portable library of computer vision algorithms. http://www.vlfeat.org/

  • Vedaldi, A., & Soatto, S. (2005). Features for recognition: Viewpoint invariance for non-planar scenes. In International Conference on Computer Vision (pp. 1474–1481).

  • Wang, Z., Fan, B., & Wu, F. (2011). Local intensity order pattern for feature description. In International Conference on Computer Vision (pp. 603–610).

  • Wesseling, P. (2004). An Introduction to multigrid methods. Chichester: Wiley.

    Google Scholar 

  • Yanowitz, S., & Bruckstein, A. (1989). A new method for image segmentation. Computer Vision, Graphics, and Image Processing, 46(1), 82–95.

    Article  Google Scholar 

  • Yezzi, A. (1998). Modified curvature motion for image smoothing and enhancement. IEEE Transactions on Image Processing, 7(3), 345–352.

    Article  Google Scholar 

Download references

Acknowledgments

This work has been partially funded by the Spanish Ministry of Economy and Competitiveness under Projects ERA-Net Chistera project ViSen PCIN-2013-047 and PAU+ DPI2011-27510, and by the EU Project IntellAct FP7-ICT2009-6-269959.

Author information

Authors and Affiliations

  1. Institut de Robòtica i Informàtica Industrial, CSIC-UPC, 08028, Barcelona, Spain

    Edgar Simo-Serra, Carme Torras & Francesc Moreno-Noguer

Authors
  1. Edgar Simo-Serra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carme Torras
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Francesc Moreno-Noguer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francesc Moreno-Noguer.

Additional information

Communicated by Ron Kimmel.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Simo-Serra, E., Torras, C. & Moreno-Noguer, F. DaLI: Deformation and Light Invariant Descriptor. Int J Comput Vis 115, 136–154 (2015). https://doi.org/10.1007/s11263-015-0805-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-015-0805-1

Keywords

Search

Navigation