Skip to main content
SpringerLink
Log in

Optimal Non-Linear Models for Sparsity and Sampling

  • Published:
Journal of Fourier Analysis and Applications Aims and scope Submit manuscript

Abstract

Given a set of vectors (the data) in a Hilbert space ℋ, we prove the existence of an optimal collection of subspaces minimizing the sum of the square of the distances between each vector and its closest subspace in the collection. This collection of subspaces gives the best sparse representation for the given data, in a sense defined in the paper, and provides an optimal model for sampling in union of subspaces. The results are proved in a general setting and then applied to the case of low dimensional subspaces of ℝN and to infinite dimensional shift-invariant spaces in L 2(ℝd). We also present an iterative search algorithm for finding the solution subspaces. These results are tightly connected to the new emergent theories of compressed sensing and dictionary design, signal models for signals with finite rate of innovation, and the subspace segmentation problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aharon, M., Elad, M., Bruckstein, A.M.: The k-svd: An algorithm for designing of overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)

    Article  Google Scholar 

  2. Aharon, M., Elad, M., Bruckstein, A.M.: On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them. Linear Algebra Appl. 416(1), 48–67 (2006). MR2232919 (2007a:94026)

    Article  MATH  MathSciNet  Google Scholar 

  3. Aldroubi, A., Gröchenig, K.-H.: Non-uniform sampling in shift-invariant space. SIAM Rev. 43(4), 585–620 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  4. Aldroubi, A., Cabrelli, C.A., Hardin, D., Molter, U.M.: Optimal shift invariant spaces and their parseval frame generators. Appl. Comput. Harmon. Anal. 23, 273–283 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  5. Aldroubi, A., Cabrelli, C., Hardin, D.P., Molter, U., Rodado, E.: Determining sets of shift invariant spaces. In: Proceedings of ICWA (Chenai, India), 2003

  6. Baraniuk, R., Davenport, M., DeVore, R.A., Wakin, M.: A simple proof of the restricted isometry property for random matrices. Constr. Approx. (2007). doi:10.1007/s00365-007-9003-x

  7. Bownik, M.: The structure of shift-invariant subspaces of L 2(ℝn). J. Funct. Anal. 177, 282–309 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  8. Candès, E., Romberg, J.: Quantitative robust uncertainty principles and optimally sparse decompositions. Found. Comput. Math. 6, 227–254 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  9. Candès, E., Tao, T.: Near optimal signal recovery from random projections: Universal encoding strategies. IEEE Trans. Inf. Theory 52, 5406–5425 (2006)

    Article  Google Scholar 

  10. Candès, E., Romberg, J., Tao, T.: Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52, 489–509 (2006)

    Article  Google Scholar 

  11. Casazza, P.G.: The art of frame theory. Taiwan. J. Math. 4(2), 129–201 (2000)

    MATH  MathSciNet  Google Scholar 

  12. Christensen, O.: An Introduction to Frames and Riesz Basis. Applied and Numerical Harmonic Analysis. Birkhäuser, Basel (2003)

    Google Scholar 

  13. DeVore, R.A.: Deterministic constructions of compressed sensing matrices. J. Complex. 23, 918–925 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  14. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52, 1289–1306 (2006)

    Article  MathSciNet  Google Scholar 

  15. Dragotti, P.L., Vetterli, M., Blu, T.: Sampling moments and reconstructing signals of finite rate of innovation: Shannon meets strang-fix. IEEE Trans. Signal Process. 55, 1741–1757 (2007)

    Article  MathSciNet  Google Scholar 

  16. Eckart, C., Young, G.: The approximation of one matrix by another of lower rank. Psychometrica 1, 211–218 (1936)

    Article  Google Scholar 

  17. Gribonval, R., Nielsen, M.: Sparse decompositions in unions of bases. IEEE Trans. Inf. Theory 49, 3320–3325 (2003)

    Article  MathSciNet  Google Scholar 

  18. Gröchenig, K.: Foundations of Time-Frequency Analysis. Appl. Numer. Harmon. Anal. Birkhäuser, Basel (2001)

    MATH  Google Scholar 

  19. Hernández, E., Weiss, G.: A First Course on Wavelets. CRC Press, Boca Raton (1996)

    MATH  Google Scholar 

  20. Horn, R., Johnson, C.: Matrix Analysis. Cambridge University Press, Cambridge (1985)

    MATH  Google Scholar 

  21. Lu, Y., Do, M.N.: A theory for sampling signals from a union of subspaces. IEEE Trans. Signal Process. 56, 2334–2345 (2008)

    Article  Google Scholar 

  22. Ma, Y., Derksen, H., Hong, W., Wright, J.: Segmentation of multivariate mixed data via lossy coding and compression. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 29(9), 1546–1562 (2007)

    Article  Google Scholar 

  23. Ma, Y., Yang, A., Derksen, H., Fossum, R.: Estimation of subspace arrangements with applications in modeling and segmenting mixed data. SIAM Rev. 50, 413–458 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  24. Maravic, I., Vetterli, M.: Sampling and reconstruction of signals with finite rate of innovation in the presence of noise. IEEE Trans. Signal Process. 53, 2788–2805 (2005)

    Article  MathSciNet  Google Scholar 

  25. Rauhut, H., Schass, K., Vandergheynst, P.: Compressed sensing and redundant dictionaries. IEEE Trans. Inf. Theory 4, 2210–2219 (2008)

    Article  Google Scholar 

  26. Schmidt, E.: Zur theorie der linearen und nichtlinearen integralgleichungen. i teil. entwicklung willkürlichen funktionen nach system vorgeschriebener. Math. Ann. 63, 433–476 (1907)

    Article  MathSciNet  Google Scholar 

  27. Tropp, J.A.: Greed is good: Algorithmic results for sparse approximation. IEEE Trans. Inf. Theory 50, 2231–2242 (2004)

    Article  MathSciNet  Google Scholar 

  28. Vidal, R., Ma, Y., Sastry, S.: Generalized principal component analysis (gpca). IEEE Trans. Pattern Anal. Mach. Intell. 27, 1–15 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Vanderbilt University, 1326 Stevenson Center, Nashville, TN, 37240, USA

    Akram Aldroubi

  2. Departamento de Matemática, Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires, Ciudad Universitaria, Pabellón I, 1428 Capital Federal, Buenos Aires, Argentina

    Carlos Cabrelli & Ursula Molter

  3. Conicet, Buenos Aires, Argentina

    Carlos Cabrelli & Ursula Molter

Authors
  1. Akram Aldroubi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Cabrelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ursula Molter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Akram Aldroubi.

Additional information

Communicated by Michael Elad.

The research of Akram Aldroubi is supported in part by NSF Grant DMS-0504788. The research of Carlos Cabrelli and Ursula Molter is partially supported by Grants: PICT 15033, CONICET, PIP 5650, UBACyT X058 and X108.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Aldroubi, A., Cabrelli, C. & Molter, U. Optimal Non-Linear Models for Sparsity and Sampling. J Fourier Anal Appl 14, 793–812 (2008). https://doi.org/10.1007/s00041-008-9040-2

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00041-008-9040-2

Keywords

Mathematics Subject Classification (2000)

Search

Navigation