Abstract
The author previously introduced a fast fixed-point algorithm for independent component analysis. The algorithm was derived from objective functions motivated by projection pursuit. In this paper, it is shown that the algorithm is closely connected to maximum likelihood estimation as well. The basic fixed-point algorithm maximizes the likelihood under the constraint of decorrelation, if the score function is used as the nonlinearity. Modifications of the algorithm maximize the likelihood without constraints.
Similar content being viewed by others
References
Amari, S.-I.: Neural learning in structured parameter spaces - natural riemannian gradient, In: Advances in Neural Information Processing 9 (Proc. NIPS*96), MIT Press, Cambridge, MA, 1997, pp. 127–133.
Bell, A. J. and Sejnowski, T. J.: An information-maximization approach to blind separation and blind deconvolution, Neural Computation 7 (1995), 1129–1159.
Cardoso, J. F.: Entropic contrasts for source separation, In: S. Haykin (ed.), Adaptive Unsupervised Learning, 1998.
Cichocki, A. and Unbehauen, R.: Robust neural networks with on-line learning for blind identification and blind separation of sources, IEEE Trans. on Circuits and Systems 43(11) (1996), 894–906.
Comon, P.: Independent component analysis - a new concept? Signal Processing 36 (1994), 287–314.
Huber, P. J.: Projection pursuit, Ann. Statistics 13(2) (1985), 435–475.
Hyvärinen, A.: A family of fixed-point algorithms for independent component analysis, In: Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP'97), Munich, Germany, 1997, pp. 3917–3920.
Hyvärinen, A.: Fast and robust fixed-point algorithms for independent component analysis, IEEE Trans. on Neural Networks, in press.
Hyvärinen, A. and Oja, E.: A fast fixed-point algorithm for independent component analysis, Neural Computation 9(7) (1997), 1483–1492.
Jutten, C. and Herault, J.: Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture, Signal Processing 24 (1991), 1–10.
Lee, T.-W., Girolami, M. and Sejnowski, T. J.: Independent component analysis using an extended infomax algorithm for mixed sub-gaussian and super-gaussian sources, Neural Computation 11 (1999), 609–633.
Luenberger, D. G.: Optimization by Vector Space Methods, John Wiley & Sons, 1969.
Nadal, J.-P. and Parga, N.: Non-linear neurons in the low noise limit: a factorial code maximizes information transfer, Network 5 (1994), 565–581.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hyvärinen, A. The Fixed-Point Algorithm and Maximum Likelihood Estimation for Independent Component Analysis. Neural Processing Letters 10, 1–5 (1999). https://doi.org/10.1023/A:1018647011077
Issue Date:
DOI: https://doi.org/10.1023/A:1018647011077