Abstract
This paper focuses on the problem of local minima of the STRESS function. It turns out that unidimensional scaling is particularly prone to local minima, whereas full dimensional scaling with Euclidean distances has a local minimum that is global. For intermediate dimensionality with Euclidean distances it depends on the dissimilarities how severe the local minimum problem is. For city-block distances in any dimensionality many different local minima are found. A simulation experiment is presented that indicates under what conditions local minima can be expected. We introduce the tunneling method for global minimization, and adjust it for multidimensional scaling with general Minkowski distances. The tunneling method alternates a local search step, in which a local minimum is sought, with a tunneling step in which a different configuration is sought with the same STRESS as the previous local minimum. In this manner successively better local minima are obtained, and experimentation so far shows that the last one is often a global minimum.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arabie, P. (1991). Was Euclid an unnecessarily sophisticated psychologist?Psychometrika, 56, 567–587.
Bailey, R. A., & Gower, J. C. (1990). Approximating a symmetric matrix.Psychometrika, 55, 665–675.
Commandeur, J. J. F. (1992).Missing data in the distance approach to Principal Component Analysis (Research Rep. No. RR-92-07). Leiden: Department of Data Theory.
Critchley, F. (1986). Dimensionality theorems in MDS and HCA. In E. Diday et al. (Eds),Data analysis and informatics, Vol. 4 (pp. 45–70). Amsterdam: North-Holland.
Defays, D. (1978). A short note on a method of seriation.British Journal of Mathematical and Statistical Psychology, 3, 49–53.
de Leeuw, J. (1977). Applications of convex analysis to multidimensional scaling. In J. R. Barra, F. Brodeau, G. Romier, & B. van Cutsem (Eds.),Recent development in statistics (pp. 133–145). Amsterdam: North-Holland.
de Leeuw, J. (1988). Convergence of the majorization method for multidimensional scaling.Journal of Classification, 5, 163–180.
de Leeuw, J. (1993).Fitting distances by least squares. Unpublished manuscript.
de Leeuw, J., & Heiser, W. J. (1977). Convergence of correction matrix algorithms for multidimensional scaling. In J. C. Lingoes, E. Roskam, & I. Borg (Eds.),Geometric representations of relational data (pp. 735–752). Ann Arbor: Mathesis Press.
de Leeuw, J. & Heiser, W. J. (1980). Multidimensional scaling with restrictions on the configuration. In P. R. Krishnaiah (Ed.),Multivariate analysis, Vol. V (pp. 501–522). Amsterdam: North-Holland.
De Soete, G., Hubert, L., & Arabie, P. (1988). On the use of simulated annealing for combinatorial data analysis. In W. Gaul & M. Schader (Eds.),Data, expert, knowledge and decisions (pp. 329–340). Berlin: Springer-Verlag.
Dinkelbach, W. (1967). On nonlinear fractional programming.Management Science, 13, 492–498.
Funk, S. G., Horowitz, A. D., Lipshitz, R., & Young, F. W. (1974). The perceived structure of American ethnic groups: The use of multidimensional scaling in stereotype research.Personality and Social Psychology Bulletin, 1, 66–68.
Gomez, S., & Levy, A. V. (1982). The tunneling method for solving the constrained global optimization problem with non-connected feasible regions. In J. P. Hennart (Ed.),Lecture notes in mathematics, 909 (pp. 34–47). Berlin: Springer-Verlag.
Green, P. E., Carmone, F. J. Jr., & Smith, S. M. (1989).Multidimensional scaling, concepts and applications. Boston: Allyn and Bacon.
Groenen, P. J. F. (1993).The majorization approach to multidimensional scaling: Some problems and extensions. Leiden: DSWO Press.
Groenen, P. J. F., & Heiser, W. J. (1991).An improved tunneling function for finding a decreasing series of local minima (Research Rep. No. RR-91-06). Leiden: Department of Data Theory.
Groenen, P. J. F., de Leeuw, J., & Mathar, R. (1996). Least squares multidimensional scaling with transformed distances. In W. Gaul & D. Pfeifer (Eds.),Studies in classification, data analysis, and knowledge organization (pp. 177–185). Berlin: Springer.
Groenen, P. J. F., Mathar, R., & Heiser, W. J. (1995). The majorization approach to multidimensional scaling for Minkowski distances.Journal of Classification, 12, 3–19.
Hardy, G. H., Littlewood, J. E. & Pólya, G. (1952).Inequalities (2nd ed.). Cambridge: University Press.
Heiser, W. J. (1989). The city-block model for three-way multidimensional scaling. In R. Coppi & S. Bolasco (Eds.),Multiway data analysis (pp. 395–404). Amsterdam: North-Holland.
Heiser, W. J. (1991). A generalized majorization method for least squares multidimensional scaling of pseudodistances that may be negative.Psychometrika, 56, 7–27.
Heiser, W. J. (1995).Convergent computation by iterative majorization: Theory and applications in multidimensional data analysis. In W. J. Krzanowski (Eds.),Recent advances in descriptive multivariate analysis (pp. 157–189). Oxford: Oxford University Press.
Heiser, W. J., & de Leeuw, J. (1977).How to use SMACOF-1 (Research Rep. No. UG-86-02). Leiden: Department of Data Theory.
Heiser, W. J., & Groenen, P. J. F. (1994).Cluster differences scaling with a within clusters loss component and a fuzzy successive approximation strategy to avoid local minima (Research Rep. No. RR-94-03). Leiden: Department of Data Theory.
Hubert, L. J., & Arabie, P. (1986). Unidimensional scaling and combinatorial optimization. In J. de Leeuw, W. J. Heiser, J. Meulman & F. Critchley (Eds.),Multidimensional data analysis (pp. 181–196). Leiden: DSWO Press.
Hubert, L. J., Arabie, P., & Hesson-McInnis, M. (1992). Multidimensional scaling in the city-block metric: A combinatorial approach.Journal of Classification, 9, 211–236.
Kruskal, J. B. (1964a). Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis.Psychometrika, 29, 1–28.
Kruskal, J. B. (1964b). Nonmetric multidimensional scaling: A numerical method.Psychometrika, 29, 115–129.
Kruskal, J. B., Young, F. W., & Seery, J. B. (1977).How to use KYST-2, a very flexible program to do multidimensional scaling and unfolding. Murray Hill, NJ: AT&T Bell Laboratories.
Levy, A. V., & Gomez, S. (1985). The tunneling method applied to global optimization. In P. T. Boggs, R. H. Byrd, & R. B. Schnabel (Eds.),Numerical optimization 1984 (pp. 213–244). Philadelphia: SIAM.
Mathar, R., & Groenen, P. J. F. (1991). Algorithms in convex analysis applied to multidimensional scaling. In E. Diday & Y. Lechevallier (Eds.),Symbolic-numeric data analysis and learning (pp. 45–56). Commack, NY: Nova Science Publishers.
Meulman, J. J. (1986).A distance approach to nonlinear multivariate analysis. Leiden: DSWO Press.
Meulman, J. J. (1992). The integration of multidimensional scaling and multivariate analysis with optimal transformations.Psychometrika, 57, 539–565.
Montalvo, A. (1979).Development of a new algorithm for the global minimization of functions Unpublished doctoral dissertation, Universidad Nacional Autonoma de Mexico.
Robinson, W. S. (1951). A method for chronologically ordering archaeological deposits.American Antiquity, 16, 293–301.
Shepard, R. N. (1962). Analysis of proximities: Multidimensional scaling with an unknown distance function.Psychometrika, 27, 125–140.
Tijssen, R. J. W. (1992).Cartography of science: Scientometric mapping with multidimensional scaling methods. Leiden: DSWO Press.
Torgerson, W. S. (1958).Theory and methods of scaling. New York: Wiley.
Tucker, W. S. (1951).A method for synthesis of factor analysis studies (Personel Research Section Rep. No. 984). Washington DC: Department of the Army.
Vilkov, A. V., Zhidkov, N. P., & Shchedrin, B. M. (1975). A method of finding the global minimum of a function of one variable.USSR Computational Mathematics and Mathematical Physics, 15, 1040–1042.
Wagenaar, W. A., & Padmos, P. (1971). Quantitative interpretation of stress in Kruskal's multidimensional scaling technique.British Journal of Mathematical and Statistical Psychology, 24, 101–110.
Weeks, D. G., & Bentler, P. M. (1982). Restricted multidimensional scaling models for asymmetric proximities.Psychometrika, 47, 201–208.
Zielman, B. (1991).Three-way scaling of asymmetric proximities (Research Rep. No. RR-91-01). Leiden: Department of Data Theory.
Author information
Authors and Affiliations
Additional information
This paper is based on the 1994 Psychometric Society's outstanding thesis award of the first author. The authros would like to thank Robert Tijssen of the CWTS Leiden for kindly making available the co-citation data of the Psychometric literature. This paper is an extended version of the paper presented at the Annual Meeting of the Psychometric Society at Champaign-Urbana, Illin., June 1994.
Rights and permissions
About this article
Cite this article
Groenen, P.J.F., Heiser, W.J. The tunneling method for global optimization in multidimensional scaling. Psychometrika 61, 529–550 (1996). https://doi.org/10.1007/BF02294553
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02294553