Skip to main content
SpringerLink
Log in

Model Selection Criteria for a Linear Model to Solve Discrete Ill-Posed Problems on the Basis of Singular Decomposition and Random Projection

  • NEW TOOLS OF CYBERNETICS, INFORMATICS, COMPUTER ENGINEERING, AND SYSTEMS ANALYSIS
  • Published:
Cybernetics and Systems Analysis Aims and scope

Abstract

Criteria are developed to determine the optimal number of components of a linear model in solving a discrete ill-posed problem by the methods of truncated singular value decomposition and random projection. To this end, the behavior of dependencies of the error vector of the solution and the restoration error of the vector of the right side on the model dimensionality and their minima is investigated. An experimental investigation of the developed criteria was also pursued and its results are provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. V. K. Khmelevskiy and V. M. Bondarenko, Electric Exploration [in Russian], Nedra, Moscow (1999).

    Google Scholar 

  2. Yu. L. Zabulonov, Yu. M. Korostil, and E. G. Revunova, “Optimization of inverse problem solution to obtain the distribution density function for surface contaminations,” Modeling and Information Technologies, Iss. 39, 77–83 (2006).

  3. D. A. Rachkovskij and E. G. Revunova, “Intelligent gamma-ray data processing for environmental monitoring,” in: Intelligent Data Analysis in Global Monitoring for Environmental and Security (2009), pp. 124–145.

  4. P. C. Hansen, Rank-Deficient and Discrete Ill-Posed Problems. Numerical Aspects of Linear Inversion, SIAM, Philadelphia (1998).

    Book  Google Scholar 

  5. A. N. Tikhonov and V. Y. Arsenin, Solution of Ill-Posed Problems, V. H. Winston, Washington (1977).

    MATH  Google Scholar 

  6. H. Akaike, “A new look at the statistical model identification,” IEEE Transactions on Automatic Control, 19, No. 6, 716–723 (1974).

    Article  MathSciNet  MATH  Google Scholar 

  7. C. L. Mallows, “Some comments on Cp,” Technometrics, 15, No. 4, 661–675 (1973).

    MATH  Google Scholar 

  8. M. Hansen and B. Yu, “Model selection and minimum description length principle,” J. Amer. Statist. Assoc., 96, 746–774 (2001).

    Article  MathSciNet  MATH  Google Scholar 

  9. E. G. Revunova and D. A. Rachkovskij, “Using randomized algorithms for solving discrete ill-posed problems,” International Journal Information Theories and Applications, 16, No. 2, 176–192 (2009).

    MATH  Google Scholar 

  10. E. G. Revunova, “Study of error components for solution of the inverse problem using random projections,” Mathematical Machines and Systems, No. 4, 33–42 (2010).

    Google Scholar 

  11. E. G. Revunova, “Using model selection criteria for solving discrete ill-posed problems by randomized algorithms,” in: Proc. 4th International Workshop on Inductive Modelling (IWIM’2011), Kyiv (2011), pp. 89–97.

  12. D. A. Rachkovskij and E. G. Revunova, “Randomized method for solving discrete ill-posed problems,” Cybernetics and Systems Analysis, 48, No. 4, 621–635 (2012).

    Article  MathSciNet  MATH  Google Scholar 

  13. E. G. Revunova, “A randomization approach in problems of signal recovery from the results of indirect measurements,” Cybernetics and Computer Engineering, Iss. 173, 35–46 (2013).

  14. E. G. Revunova, “Randomization approach to the reconstruction of signals resulted from indirect measurements,” in: Proc. 4th International Conference on Inductive Modelling (ICIM’2013), Kyiv (2013), pp. 203–208.

  15. E. G. Revunova, “Investigation of a method for solving discrete ill-posed problems on the basis of random projection,” USiM, No. 4 (252), 41–47 (2014).

  16. E. G. Revunova, “Analytical study of error components for the solution of discrete ill-posed problems using random projections,” Cybernetics and Systems Analysis, 51, No. 6, 978–991 (2015).

    Article  MathSciNet  MATH  Google Scholar 

  17. E. G. Revunova and A. V. Tyshchuk, “A model selection criterion for solution of discrete ill-posed problems based on the singular value decomposition” in: Proc. 7th International Workshop on Inductive Modelling (IWIM’ 2015), Kyiv–Zhukyn (2015), pp. 43–47.

  18. D. A. Rachkovskij, I. S. Misuno, and S. V. Slipchenko, “Randomized projective methods for construction of binary sparse vector representations,” Cybernetics and Systems Analysis, 48, No. 1, 146–156 (2012).

    Article  MATH  Google Scholar 

  19. D. A. Rachkovskij, “Vector data transformation using random binary matrices,” Cybernetics and Systems Analysis, 50, No. 6, 960–968 (2014).

    Article  MathSciNet  MATH  Google Scholar 

  20. E. M. Kussul, T. N. Baidyk, V. V. Lukovich, and D. A. Rachkovskij, “Adaptive neural network classifier with multifloat input coding,” in: Proc. 6th Intern. Conf. “Neural Networks and Their Industrial and Cognitive Applications (Neuro-Nimes’93)” (1993), pp. 209–216.

  21. V. V. Lukovich, A. D. Goltsev, and D. A. Rachkovskij, “Neural network classifiers for micromechanical equipment diagnostics and micromechanical product quality inspection,” in: Proc. 5th European Congress on Intelligent Techniques and Soft Computing (EUFIT’97), 1 (1997), pp. 534–536.

  22. E. M. Kussul, L. M. Kasatkina, D. A. Rachkovskij, and D. C. Wunsch, “Application of random threshold neural networks for diagnostics of micro machine tool condition,” Neural Networks Proceedings: IEEE World Congress on Computational Intelligence, 1 (1998), pp. 241–244.

    Google Scholar 

  23. A. B. Markman, D. A. Rachkovskij, I. S. Misuno, and E. G. Revunova, “Analogical reasoning techniques in intelligent counterterrorism systems,” International Journal Information Theories and Applications, 10, No. 2, 139–146 (2003).

    Google Scholar 

  24. J.-P. Dedieu and D. Novitsky, “Symplectic methods for the approximation of the exponential map and the Newton iteration on Riemannian submanifolds,” Journal of Complexity, 21, 487–501 (2005).

    Article  MathSciNet  MATH  Google Scholar 

  25. A. M. Reznik, A. A. Galinskaya, O. K. Dekhtyarenko, and D. W. Nowicki, “Preprocessing of matrix QCM sensors data for the classification by means of neural network,” Sensors and Actuators, B, 106, 158–163 (2005).

  26. D. W. Nowicki and O. K. Dekhtyarenko, “Averaging on Riemannian manifolds and unsupervised learning using neural associative memory,” in: Proc. ESANN 2005, 1, Bruges, Belgium (2005), pp. 27–29.

  27. D. A. Rachkovskij and S. V. Slipchenko, “Similarity-based retrieval with structure-sensitive sparse binary distributed representations,” Computational Intelligence, 28, No. 1, 106–129 (2012).

    Article  MathSciNet  Google Scholar 

  28. V. I. Gritsenko, D. A. Rachkovskij, A. D. Goltsev, V. V. Lukovych, I. S. Misuno, E. G. Revunova, S. V. Slipchenko, A. M. Sokolov, and S. A. Talayev, “Neural network distributed representations for intelligent information technology and modelling of thinking,” Cybernetics and Computer Engineering, Iss. 173, 7–24 (2013).

  29. P. C. Hansen, “Regularization tools: A Matlab package for analysis and solution of discrete ill-posed problems,” Numer. Algorithms, 6, 1–35 (1994).

    Article  MathSciNet  MATH  Google Scholar 

  30. J. W. Demmel, Applied Numerical Linear Algebra, SIAM, Philadelphia (1997).

    Book  MATH  Google Scholar 

  31. R. Horn and C. Johnson, Matrix Analysis [Russian translation], Mir, Moscow (1989).

  32. D. Nowicki, P. Verga, and H. Siegelmann, “Modeling reconsolidation in kernel associative memory,” PLoS ONE 8 (8): e68189 (2013). doi:10.1371/journal.pone.0068189.

  33. D. Nowicki and H. Siegelmann, “Flexible kernel memory,” PLoS ONE (2010). 5 (6): e10955. doi:10.1371/journal.pone.0010955.

  34. N. F. Kirichenko, A. M. Reznik, and S. P. Shchetenyuk, “Matrix pseudoinversion in the problem of design of associative memory,” Cybernetics and Systems Analysis, 37, No. 3, 18–27 (2001).

    Article  MathSciNet  MATH  Google Scholar 

  35. R. D. Fierro, G. H. Golub, P. C. Hansen, and D. P. O’Leary, “Regularization by truncated total least squares,” SIAM Journal on Scientific Computing, 18, No. 1, 1223–1241 (1997).

    Article  MathSciNet  MATH  Google Scholar 

  36. L. Reichel and G. Rodriguez, “Old and new parameter choice rules for discrete ill-posed problems,” Numerical Algorithms, 63, No. 1, 65–87 (2013).

    Article  MathSciNet  MATH  Google Scholar 

  37. P. C. Hansen, “The truncated SVD as a method for regularization,” BIT, 27, 534–553 (1987).

    Article  MathSciNet  MATH  Google Scholar 

  38. R. D. Fierro and P. C. Hansen, “Low-rank revealing two-sided orthogonal decompositions,” Numer. Algorithms, 15, 37–55 (1997).

    Article  MathSciNet  MATH  Google Scholar 

  39. T. F. Chan and P. C. Hansen, “Some applications of the rank revealing QR factorization,” SIAM J. Sci. Stat. Comput., 13, 727–741 (1992).

    Article  MathSciNet  MATH  Google Scholar 

  40. T. F. Chan and P. C. Hansen, “Low-rank revealing QR factorizations,” Numer. Linear Algebra Appl., 1, 33–44 (1994).

    Article  MathSciNet  MATH  Google Scholar 

  41. A. Belloni and V. Chernozhukov, “Least squares after model selection in high-dimensional sparse models,” Bernoulli, 19, No 2, 521–547 (2013).

    Article  MathSciNet  MATH  Google Scholar 

  42. M. Bayati, M. A. Erdogdu, and A. Montanari, “Estimating LASSO risk and noise level,” in: Proceedings of Advances in Neural Information Processing Systems (NIPS 2013) (2013), pp. 944–952.

  43. J. Fan, S. Guo, and N. Hao, “Variance estimation using refitted cross-validation in ultrahigh dimensional regression,” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 74, 1467–9868 (2012).

    MathSciNet  Google Scholar 

  44. V. V. Fedorov, Theory of Optimal Experiments, Academic Press, New York (1972).

    Google Scholar 

  45. G. W. Stewart, “On the perturbation of pseudo-inverses, projections and linear least squares problems,” SIAM Review, 19, No. 4, 634–662 1977.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. International Research and Training Center for Information Technologies and Systems, National Academy of Sciences of Ukraine and Ministry of Education and Science of Ukraine, Kyiv, Ukraine

    E. G. Revunova

Authors
  1. E. G. Revunova
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to E. G. Revunova.

Additional information

Translated from Kibernetika i Sistemnyi Analiz, No. 4, July–August, 2016, pp. 174–192.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Revunova, E.G. Model Selection Criteria for a Linear Model to Solve Discrete Ill-Posed Problems on the Basis of Singular Decomposition and Random Projection. Cybern Syst Anal 52, 647–664 (2016). https://doi.org/10.1007/s10559-016-9868-4

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10559-016-9868-4

Keywords

Search

Navigation