Miqing Li
Skip Abstract Section
Abstract
Complexity and variety of modern multiobjective optimisation problems result in the emergence of numerous search techniques, from traditional mathematical programming to various randomised heuristics. A key issue raised consequently is how to evaluate and compare solution sets generated by these multiobjective search techniques. In this article, we provide a comprehensive review of solution set quality evaluation. Starting with an introduction of basic principles and concepts of set quality evaluation, this article summarises and categorises 100 state-of-the-art quality indicators, with the focus on what quality aspects these indicators reflect. This is accompanied in each category by detailed descriptions of several representative indicators and in-depth analyses of their strengths and weaknesses. Furthermore, issues regarding attributes that indicators possess and properties that indicators are desirable to have are discussed, in the hope of motivating researchers to look into these important issues when designing quality indicators and of encouraging practitioners to bear these issues in mind when selecting/using quality indicators. Finally, future trends and potential research directions in the area are suggested, together with some guidelines on these directions.
- S. F. Adra and P. J. Fleming. 2011. Diversity management in evolutionary many-objective optimization. IEEE Trans. Evolution. Comput. 15, 2 (2011), 183--195. Google Scholar
Digital Library
- I. Alaya, C. Solnon, and K. Ghedira. 2007. Ant colony optimization for multi-objective optimization problems. In Proceedings of the 19th IEEE International Conference on Tools with Artificial Intelligence, Vol. 1. 450--457. Google ScholarDigital Library
- M. Asafuddoula, T. Ray, and H. K. Singh. 2015. Characterizing pareto front approximations in many-objective optimization. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’15). ACM, 607--614. Google ScholarDigital Library
- A. Auger, J. Bader, and D. Brockhoff. 2010. Theoretically investigating optimal -distributions for the hypervolume indicator: First results for three objectives. In Proceedings of the International Conference on Parallel Problem Solving from Nature (PPSN’10). 586--596. Google ScholarDigital Library
- A. Auger, J. Bader, D. Brockhoff, and E. Zitzler. 2009. Investigating and exploiting the bias of the weighted hypervolume to articulate user preferences. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’09). 563--570. Google ScholarDigital Library
- A. Auger, J. Bader, D. Brockhoff, and E. Zitzler. 2009. Theory of the hypervolume indicator: Optimal -Distributions and the choice of the reference point. In Proceedings of the 10th ACM SIGEVO Workshop on Foundations of Genetic Algorithms (FOGA’09). 87--102. Google ScholarDigital Library
- J. Bader and E. Zitzler. 2011. HypE: An algorithm for fast hypervolume-based many-objective optimization. Evolution. Comput. 19, 1 (2011), 45--76. Google ScholarDigital Library
- S. Bandyopadhyay, S. K. Pal, and B. Aruna. 2004. Multiobjective GAs, quantitative indices, and pattern classification. IEEE Trans. Syst. Man Cybernet. 34, 5 (2004), 2088--2099. Google ScholarDigital Library
- M. Basseur and E. K. Burke. 2007. Indicator-based multi-objective local search. In Proceedings of the IEEE Congress on Evolutionary Computation. 3100--3107.Google Scholar
- A. Berry and P. Vamplew. 2005. The combative accretion model--multiobjective optimisation without explicit Pareto ranking. In Evolutionary Multi-Criterion Optimization. Springer, 77--91. Google ScholarDigital Library
- N. Beume. 2009. S-metric calculation by considering dominated hypervolume as Klee’s measure problem. Evolution. Comput. 17, 4 (Nov. 2009), 477--492. Google ScholarDigital Library
- N. Beume, C. M. Fonseca, M. Lopez-Ibanez, L. Paquete, and J. Vahrenhold. 2009. On the complexity of computing the hypervolume indicator. IEEE Trans. Evolution. Comput. 13, 5 (2009), 1075--1082. Google ScholarDigital Library
- N. Beume, B. Naujoks, and Emmerich. 2007. SMS-EMOA: Multiobjective selection based on dominated hypervolume. Eur. J. Operat. Res. 181, 3 (2007), 1653--1669.Google ScholarCross Ref
- P. AN Bosman and D. Thierens. 2003. The balance between proximity and diversity in multiobjective evolutionary algorithms. IEEE Trans. Evolution. Comput. 7, 2 (2003), 174--188. Google ScholarDigital Library
- P. AN Bosman and D. Thierens. 2005. The naive MIDEA: A baseline multi-objective EA. In Proceedings of the International Conference on Evolutionary Multi-Criterion Optimization. Springer, 428--442. Google ScholarDigital Library
- B. Bozkurt, J. W. Fowler, E. S. Gel, B. Kim, M. Köksalan, and J. Wallenius. 2010. Quantitative comparison of approximate solution sets for multicriteria optimization problems with weighted Tchebycheff preference function. Operat. Res. 58, 3 (2010), 650--659. Google ScholarDigital Library
- K. Bringmann. 2013. Bringing order to special cases of Klee’s measure problem. In Proceedings of the International Symposium on Mathematical Foundations of Computer Science. Springer, 207--218.Google ScholarCross Ref
- K. Bringmann, S. Cabello, and M. Emmerich. 2017. Maximum volume subset selection for anchored boxes. In Proceedings of the 33rd International Symposium on Computational Geometry (SoCG’17), Vol. 77. 22:1--22:15.Google Scholar
- K. Bringmann and T. Friedrich. 2010. Approximating the volume of unions and intersections of high-dimensional geometric objects. Comput. Geom.: Theory Appl. 43, 6-7 (2010), 601--610. Google ScholarDigital Library
- K. Bringmann and T. Friedrich. 2010. An efficient algorithm for computing hypervolume contributions. Evolution. Comput. 18, 3 (2010), 383--402. Google ScholarDigital Library
- K. Bringmann and T. Friedrich. 2010. The maximum hypervolume set yields near-optimal approximation. In Proceedings of Genetic and Evolutionary Computation Conference (GECCO’10). ACM press, 511--518. Google ScholarDigital Library
- K. Bringmann and T. Friedrich. 2012. Approximating the least hypervolume contributor: NP-hard in general, but fast in practice. Theoret. Comput. Sci. 425 (2012), 104--116. Google ScholarDigital Library
- K. Bringmann and T. Friedrich. 2013. Approximation quality of the hypervolume indicator. Artific. Intell. 195 (2013), 265--290. Google ScholarDigital Library
- K. Bringmann, T. Friedrich, C. Igel, and Voß. 2013. Speeding up many-objective optimization by Monte Carlo approximations. Artific. Intell. 204 (2013), 22--29. Google ScholarDigital Library
- K. Bringmann, T. Friedrich, and P. Klitzke. 2014. Two-dimensional subset selection for hypervolume and epsilon-indicator. In Proceedings of Genetic and Evolutionary Computation Conference (GECCO’14). ACM Press, 589--596. Google ScholarDigital Library
- D. Brockhoff, T. Wagner, and H. Trautmann. 2012. On the properties of the R2 indicator. In Proceedings of Genetic and Evolutionary Computation Conference (GECCO’12). ACM, 465--472. Google ScholarDigital Library
- D. Brockhoff, T. Wagner, and H. Trautmann. 2015. R2 indicator-based multiobjective search. Evolution. Comput. 23, 3 (2015), 369--395. Google ScholarDigital Library
- T. Brunsch and H. Röglin. 2015. Improved smoothed analysis of multiobjective optimization. J. ACM 62, 1 (2015). Google ScholarDigital Library
- L. T. Bui, S. Wesolkowski, A. Bender, H. A. Abbass, and M. Barlow. 2009. A dominance-based stability measure for multi-objective evolutionary algorithms. In Proceedings of the IEEE Congress on Evolutionary Computation. IEEE, 749--756. Google ScholarDigital Library
- X. Cai, H. Sun, and Z. Fan. 2018. A diversity indicator based on reference vectors for many-objective optimization. Info. Sci. 430 (2018), 467--486.Google Scholar
- Y. Cao, B. J. Smucker, and T. J. Robinson. 2015. On using the hypervolume indicator to compare Pareto fronts: Applications to multi-criteria optimal experimental design. J. Stat. Plan. Infer. 160 (2015), 60--74.Google ScholarCross Ref
- W. M. Carlyle, J. W. Fowler, E. S. Gel, and B. Kim. 2003. Quantitative comparison of approximate solution sets for bi-criteria optimization problems. Decis. Sci. 34, 1 (2003), 63--82.Google ScholarCross Ref
- T. M. Chan. 2013. Klee’s measure problem made easy. In Proceedings of the IEEE 54th Annual Symposium on Foundations of Computer Science (FOCS’13). IEEE, 410--419. Google ScholarDigital Library
- C. A. C. Coello. 2000. An updated survey of GA-based multiobjective optimization techniques. ACM Comput. Surv. 32, 2 (2000), 109--143. Google ScholarDigital Library
- C. A. C. Coello, G. B. Lamont, and D. A. Van Veldhuizen. 2007. Evolutionary Algorithms for Solving Multi-objective Problems. Vol. 5. Springer, New York. Google ScholarDigital Library
- C. A. C. Coello, G. T. Pulido, and M. S. Lechuga. 2004. Handling multiple objectives with particle swarm optimization. IEEE Trans. Evolution. Comput. 8, 3 (2004), 256--279. Google ScholarDigital Library
- C. A. C. Coello and M. R. Sierra. 2004. A study of the parallelization of a coevolutionary multi-objective evolutionary algorithm. In Proceedings of the Mexican International Conference on Artificial Intelligence (MICAI’04). 688--697.Google Scholar
- Y. Collette and P. Siarry. 2005. Three new metrics to measure the convergence of metaheuristics toward the Pareto frontier and the aesthetic of a set of solutions in biobjective optimization. Comput. Operat. Res. 32, 4 (2005), 773--792. Google ScholarDigital Library
- P. Czyzak and A. Jaszkiewicz. 1998. Pareto simulated annealing—A metaheuristic technique for multiple-objective combinatorial optimization. J. Multi-Crit. Decis. Anal. 7, 1 (1998), 34--47.Google ScholarCross Ref
- D. Datta and J. R. Figueira. 2012. Some convergence-based M-ary cardinal metrics for comparing performances of multi-objective optimizers. Comput. Operat. Res. 39, 7 (2012), 1754--1762. Google ScholarDigital Library
- P. De, J. B. Ghosh, and C. E. Wells. 1992. Heuristic estimation of the efficient frontier for a bi-criteria scheduling problem. Decis. Sci. 23, 3 (1992), 596--609.Google ScholarCross Ref
- K. Deb, S. Agrawal, A. Pratap, and T. Meyarivan. 2000. A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. In Proceedings of the International Conference on Parallel Problem Solving from Nature (PPSN’00). Springer Berlin / Heidelberg, 849--858. Google ScholarDigital Library
- K. Deb and S. Jain. 2002. Running Performance Metrics for Evolutionary Multi-objective Optimization. Technical Report 2002004. KanGAL, Indian Institute of Technology.Google Scholar
- K. Deb, M. Mohan, and S. Mishra. 2005. Evaluating the ε-domination based multi-objective evolutionary algorithm for a quick computation of Pareto-optimal solutions. Evolution. Comput. 13, 4 (Dec. 2005), 501--525. Google ScholarDigital Library
- K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evolution. Comput. 6, 2 (2002), 182--197. Google ScholarDigital Library
- A. Diaz-Manriquez, G. Toscano-Pulido, C. A. C. Coello, and R. Landa-Becerra. 2013. A ranking method based on the R2 indicator for many-objective optimization. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC’13). IEEE, 1523--1530.Google Scholar
- E. Dilettoso, S. A. Rizzo, and N. Salerno. 2017. A weakly Pareto compliant quality indicator. Math. Comput. Appl. 22, 1 (2017), 25.Google Scholar
- M. Ehrgott. 2006. Multicriteria Optimization. Springer Science 8 Business Media. Google ScholarCross Ref
- A. E. Eiben and J. Smith. 2015. From evolutionary computation to the evolution of things. Nature 521, 7553 (2015), 476--482.Google Scholar
- M. Emmerich, N. Beume, and B. Naujoks. 2005. An EMO algorithm using the hypervolume measure as selection criterion. In Evolutionary Multi-Criterion Optimization. Springer, 62--76. Google ScholarDigital Library
- M. Emmerich and A. Deutz. 2014. Time complexity and zeros of the hypervolume indicator gradient field. In EVOLVE—A Bridge Between Probability, Set Oriented Numerics, and Evolutionary Computation III. Springer International Publishing, 169--193.Google Scholar
- M. Emmerich, A. Deutz, and N. Beume. 2007. Gradient-based/evolutionary relay hybrid for computing Pareto front approximations maximizing the S-metric. In Hybrid Metaheuristics, T. Bartz-Beielstein (Ed.). Springer, 140--156. Google ScholarDigital Library
- M. Emmerich, A. Deutz, and I. Yevseyeva. 2015. A Bayesian approach to portfolio selection in multicriteria group decision making. Procedia Comput. Sci. 64 (2015), 993--1000.Google ScholarCross Ref
- M. Emmerich and C. M. Fonseca. 2011. Computing hypervolume contributions in low dimensions: Asymptotically optimal algorithm and complexity results. In Proceedings of the Conference on Evolutionary Multi-Criterion Optimization (EMO’11). 121--135. Google ScholarDigital Library
- H. Eskandari, C. D. Geiger, and G. B. Lamont. 2007. FastPGA: A dynamic population sizing approach for solving expensive multiobjective optimization problems. In Proceedings of the International Conference on Evolutionary Multi-Criterion Optimization. Springer, 141--155. Google ScholarDigital Library
- A. Farhang-Mehr and S. Azarm. 2003. An information-theoretic metric for assessing multi-objective optimization solution set quality. Trans. ASME J. Mech. Design 125, 4 (2003), 655--663.Google ScholarCross Ref
- A. Farhang-Mehr and S. Azarm. 2003. Minimal sets of quality metrics. In Proceedings of the 2nd International Conference on Evolutionary Multi-Criterion Optimization (EMO’03). 405--417. Google ScholarDigital Library
- S. L. Faulkenberg and M. M. Wiecek. 2010. On the quality of discrete representations in multiple objective programming. Optim. Eng. 11, 3 (2010), 423--440.Google ScholarCross Ref
- J. E. Fieldsend, R. M. Everson, and S. Singh. 2003. Using unconstrained elite archives for multiobjective optimization. IEEE Trans. Evolution. Comput. 7, 3 (2003), 305--323. Google ScholarDigital Library
- M. Fleischer. 2003. The measure of Pareto optima applications to multi-objective metaheuristics. In Proceedings of the International Conference on Evolutionary Multi-Criterion Optimization. Springer, 519--533. Google ScholarDigital Library
- C. M. Fonseca and P. J. Fleming. 1995. An overview of evolutionary algorithms in multiobjective optimization. Evolution. Comput. 3, 1 (1995), 1--16. Google ScholarDigital Library
- C. M. Fonseca and P. J. Fleming. 1996. On the performance assessment and comparison of stochastic multiobjective optimizers. In Proceedings of the International Conference on Parallel Problem Solving from Nature (PPSN’96). Vol. 1141. 584--593. Google ScholarDigital Library
- C. M. Fonseca, L. Paquete, and M. Lopez-Ibanez. 2006. An improved dimension-sweep algorithm for the hypervolume indicator. In Proceedings of the IEEE Congress Evolutionary Computation (CEC’06). 1157--1163.Google Scholar
- J. W. Fowler, B. Kim, W. M. Carlyle, E. S. Gel, and S. Horng. 2005. Evaluating solution sets of a posteriori solution techniques for bi-criteria combinatorial optimization problems. J. Schedul. 8, 1 (2005), 75--96. Google ScholarDigital Library
- T. Friedrich, C. Horoba, and F. Neumann. 2009. Multiplicative approximations and the hypervolume indicator. In Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation Conference (GECCO’09). 571--578. Google ScholarDigital Library
- C. K. Goh and K. C. Tan. 2007. An investigation on noisy environments in evolutionary multiobjective optimization. IEEE Trans. Evolution. Comput. 11, 3 (2007), 354--381. Google ScholarDigital Library
- C. K. Goh and K. C. Tan. 2009. Evolutionary multi-objective optimization in uncertain environments. Issues Algor., Studies Comput. Intell. 186 (2009), 5--18. Google ScholarDigital Library
- D. E. Goldberg. 1989. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley. Google ScholarDigital Library
- A. P. Guerreiro and C. M. Fonseca. 2018. Computing and updating hypervolume contributions in up to four dimensions. IEEE Trans. Evolution. Comput. 22, 3 (2018), 449--463.Google ScholarCross Ref
- A. P. Guerreiro, C. M. Fonseca, and M. Emmerich. 2012. A fast dimension-sweep algorithm for the hypervolume indicator in four dimensions. In Proceedings of the Canadian Conference on Computational Geometry (CCCG’12). 77--82.Google Scholar
- D. Hadka and P. Reed. 2012. Diagnostic assessment of search controls and failure modes in many-objective evolutionary optimization. Evolution. Comput. 20, 3 (2012), 423--452. Google ScholarDigital Library
- M. P. Hansen. 1997. Tabu search for multiobjective optimization: MOTS. In Proceedings of the 13th International Conference on Multiple Criteria Decision Making. 574--586.Google Scholar
- M. P. Hansen and A. Jaszkiewicz. 1998. Evaluating the Quality of Approximations to the Nondominated Set. IMM-REP-1998-7. Institute of Mathematical Modeling, Technical University of Denmark.Google Scholar
- D. P. Hardin and E. B. Saff. 2004. Discretizing manifolds via minimum energy points. Notices AMS 51, 10 (2004), 1186--1194.Google Scholar
- Z. He and G. G. Yen. 2016. Visualization and performance metric in many-objective optimization. IEEE Trans. Evolution. Comput. 20, 3 (2016), 386--402.Google ScholarDigital Library
- M. Helbig and A. P. Engelbrecht. 2014. Benchmarks for dynamic multi-objective optimisation algorithms. ACM Comput. Surv. 46, 3, Article 37 (Jan. 2014), 39 pages. Google ScholarDigital Library
- S. Helbig and D. Pateva. 1994. On several concepts for ε-efficiency. Operat. Res. Spektrum 16, 3 (1994), 179--186. Google ScholarDigital Library
- G. R. Hernández and C. C. A. Coello. 2013. MOMBI: A new metaheuristic for many-objective optimization based on the R2 indicator. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC’13). IEEE, 2488--2495.Google Scholar
- G. R. Hernández and C. C. A. Coello. 2015. Improved metaheuristic based on the R2 indicator for many-objective optimization. In Proceedings of the Genetic and Evolutionary Computation Conference. 679--686. Google ScholarDigital Library
- R. M. Hierons, M. Li, X. Liu, S. Segura, and W. Zheng. 2016. SIP: Optimal product selection from feature models using many-objective evolutionary optimization. ACM Trans. Softw. Eng. Methodol. 25, 2 (2016), 17. Google ScholarDigital Library
- T. Hiroyasu, M. Miki, and S. Watanabe. 2000. The new model of parallel genetic algorithm in multi-objective optimization problems-divided range multi-objective genetic algorithm. In Proceedings of the Congress on Evolutionary Computation, Vol. 1. IEEE, 333--340.Google Scholar
- I. Hupkens and M. Emmerich. 2013. Logarithmic-time updates in SMS-EMOA and hypervolume-based archiving. In EVOLVE—A Bridge Between Probability, Set Oriented Numerics, and Evolutionary Computation IV. Springer International Publishing, 155--169.Google Scholar
- A. Ibrahim, S. Rahnamayan, M. V. Martin, and K. Deb. 2018. 3D-RadVis antenna: Visualization and performance measure for many-objective optimization. Swarm Evolution. Comput. 39 (2018), 157--176.Google ScholarCross Ref
- C. Igel, N. Hansen, and S. Roth. 2007. Covariance matrix adaptation for multi-objective optimization. Evolution. Comput. 15, 1 (Mar. 2007), 1--28. Google ScholarDigital Library
- K. Ikeda, H. Kita, and S. Kobayashi. 2001. Failure of Pareto-based MOEAs: Does non-dominated really mean near to optimal? In Proceedings of the IEEE Congress on Evolutionary Computation (CEC’01), Vol. 2. 957--962.Google Scholar
- A. Inselberg and B. Dimsdale. 1991. Parallel coordinates. In Human-Machine Interactive Systems. Springer, 199--233.Google Scholar
- H. Ishibuchi, R. Imada, Y. Setoguchi, and Y. Nojima. 2017. Reference point specification in hypervolume calculation for fair comparison and efficient search. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’17). ACM, 585--592. Google ScholarDigital Library
- H. Ishibuchi, R. Imada, Y. Setoguchi, and Y. Nojima. 2018. How to specify a reference point in hypervolume calculation for fair performance comparison. Evolution. Comput. 26, 3 (2018), 411--440. Google ScholarDigital Library
- H. Ishibuchi, R. Imada, Y. Setoguchi, and Y. Nojima. 2018. Reference point specification in inverted generational distance for triangular linear Pareto front. IEEE Trans. Evolution. Comput. (2018).Google Scholar
- H. Ishibuchi, H. Masuda, and Y. Nojima. 2015. A study on performance evaluation ability of a modified inverted generational distance indicator. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’15). 695--702. Google ScholarDigital Library
- H. Ishibuchi, H. Masuda, Y. Tanigaki, and Y. Nojima. 2014. Difficulties in specifying reference points to calculate the inverted generational distance for many-objective optimization problems. In Proceedings of the IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making (MCDM’14). 170--177.Google Scholar
- H. Ishibuchi, H. Masuda, Y. Tanigaki, and Y. Nojima. 2015. Modified distance calculation in generational distance and inverted generational distance. In Proceedings of the International Conference on Evolutionary Multi-Criterion Optimization (EMO’15). 110--125.Google Scholar
- H. Ishibuchi and T. Murata. 1998. A multi-objective genetic local search algorithm and its application to flowshop scheduling. IEEE Trans. Syst., Man, Cybernet.—Part C: Appl. Rev. 28, 3 (1998), 392--403. Google ScholarDigital Library
- H. Ishibuchi and Y. Shibata. 2004. Mating scheme for controlling the diversity-convergence balance for multiobjective optimization. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’04). 1259--1271.Google Scholar
- H. Ishibuchi, N. Tsukamoto, and Y. Nojima. 2008. Evolutionary many-objective optimization: A short review. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC’08). 2419--2426.Google Scholar
- H. Ishibuchi, N. Tsukamoto, Y. Sakane, and Y. Nojima. 2010. Indicator-based evolutionary algorithm with hypervolume approximation by achievement scalarizing functions. In Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation Conference (GECCO’10). ACM, 527--534. Google ScholarDigital Library
- H. Ishibuchi, T. Yoshida, and T. Murata. 2003. Balance between genetic search and local search in memetic algorithms for multiobjective permutation flowshop scheduling. IEEE Trans. Evolution. Comput. 7, 2 (2003), 204--223. Google ScholarDigital Library
- A. L. Jaimes and C. C. A. Coello. 2009. Study of preference relations in many-objective optimization. In Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation Conference (GECCO’09). 611--618. Google ScholarDigital Library
- A. Jaszkiewicz. 2018. Improved quick hypervolume algorithm. Comput. Operat. Res. 90 (2018), 72--83. Google ScholarDigital Library
- S. Jiang, Y. S. Ong, J. Zhang, and L. Feng. 2014. Consistencies and contradictions of performance metrics in multiobjective optimization. IEEE Trans. Cybernet. 44, 12 (2014), 2391--2404.Google ScholarCross Ref
- S. Jiang, S. Yang, and M. Li. 2016. On the use of hypervolume for diversity measurement of Pareto front approximations. In Proceedings of the IEEE Symposium Series on Computational Intelligence (SSCI’16). 1--8.Google Scholar
- S. Jiang, J. Zhang, Y. S. Ong, A. N. Zhang, and P. S. Tan. 2015. A simple and fast hypervolume indicator-based multiobjective evolutionary algorithm. IEEE Trans. Cybernet. 45, 10 (2015), 2202--2213.Google ScholarCross Ref
- D. F. Jones, S. K. Mirrazavi, and M. Tamiz. 2002. Multi-objective meta-heuristics: An overview of the current state-of-the-art. Eur. J. Operat. Res. 137, 1 (2002), 1--9.Google ScholarCross Ref
- H. Kaji and H. Kita. 2007. Individual evaluation scheduling for experiment-based evolutionary multi-objective optimization. In Evolutionary Multi-Criterion Optimization. Springer, 645--659. Google ScholarDigital Library
- E. Kasanen, R. Östermark, and M. Zeleny. 1991. Gestalt system of holistic graphics: New management support view of MCDM. Comput. Operat. Res. 18, 2 (1991), 233--239. Google ScholarDigital Library
- B. Kim, E. S. Gel, J. W. Fowler, W. M. Carlyle, and J. Wallenius. 2006. Evaluation of nondominated solution sets for k-objective optimization problems: An exact method and approximations. Eur. J. Operat. Res. 173, 2 (2006), 565--582.Google ScholarCross Ref
- J. D. Knowles. 2002. Local-search and hybrid evolutionary algorithms for Pareto optimization. Ph.D. Dissertation. University of Reading UK.Google Scholar
- J. D. Knowles and D. W. Corne. 1999. The Pareto archived evolution strategy: A new baseline algorithm for Pareto multiobjective optimisation. In Proceedings of the Congress Evolutionary Computation (CEC’99), Vol. 1.Google Scholar
- J. D. Knowles and D. W. Corne. 2002. On metrics for comparing nondominated sets. In Proceedings of the Congress Evolutionary Computation (CEC’02), Vol. 1. 711--716.Google Scholar
- J. D. Knowles and D. W. Corne. 2003. Properties of an adaptive archiving algorithm for storing nondominated vectors. IEEE Trans. Evolution. Comput. 7, 2 (2003), 100--116. Google ScholarDigital Library
- J. D. Knowles, D. W. Corne, and M. Fleischer. 2003. Bounded archiving using the Lebesgue measure. In Proceedings of the Congress on Evolutionary Computation (CEC’03). 2490--2497.Google Scholar
- J. D. Knowles, L. Thiele, and E. Zitzler. 2006. A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers. Technical Report No. 214. Computer Engineering and Networks Laboratory (TIK), ETH Zurich, Switzerland.Google Scholar
- J. Kollat and P. Reed. 2005. The value of online adaptive search: A performance comparison of NSGAII, ε-NSGAII and εMOEA. In Evolutionary Multi-Criterion Optimization. Springer, 386--398. Google ScholarDigital Library
- T. Kuhn, C. M. Fonseca, L. Paquete, S. Ruzika, M. M. Duarte, and J. R. Figueira. 2016. Hypervolume subset selection in two dimensions: Formulations and algorithms. Evolution. Comput. 24, 3 (2016), 411--425. Google ScholarDigital Library
- R. Lacour, K. Klamroth, and C. M. Fonseca. 2017. A box decomposition algorithm to compute the hypervolume indicator. Comput. Operat. Res. 79 (2017), 347--360.Google ScholarCross Ref
- M. Laumanns and R. Zenklusen. 2011. Stochastic convergence of random search methods to fixed size Pareto front approximations. Eur. J. Operat. Res. 213, 2 (2011), 414--421.Google ScholarCross Ref
- Y. W. Leung and Y. Wang. 2003. U-measure: A quality measure for multiobjective programming. IEEE Trans. Syst., Man, Cybernet.—Part A: Syst. Hum. 33, 3 (2003), 337--343. Google ScholarDigital Library
- B. Li, J. Li, K. Tang, and X. Yao. 2015. Many-objective evolutionary algorithms: A survey. Comput. Surveys 48, 1 (2015), 1--35. Google ScholarDigital Library
- B. Li, K. Tang, J. Li, and X. Yao. 2016. Stochastic ranking algorithm for many-objective optimization based on multiple indicators. IEEE Trans. Evolution. Comput. 20, 6 (2016), 924--938.Google ScholarCross Ref
- K. Li and K. Deb. 2016. Performance assessment for preference-based evolutionary multi-objective optimization using reference points. COIN Report 1, 1 (2016), 1--23.Google Scholar
- M. Li, T. Chen, and X. Yao. 2018. A critical review of “A practical guide to select quality indicators for assessing Pareto-based search algorithms in search-based software engineering”: Essay on quality indicator selection for SBSE. In Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging Results Track. Google ScholarDigital Library
- M. Li, C. Grosan, S. Yang, X. Liu, and X. Yao. 2018. Multi-line distance minimization: A visualized many-objective test problem suite. IEEE Trans. Evolution. Comput. 22, 1 (2018), 61--78.Google ScholarCross Ref
- M. Li, L. Hu, and X. Yao. 2019. On comparing two sets of multi-objective solution vectors. arXiv preprint arXiv:1702.00477 (2019).Google Scholar
- M. Li, S. Yang, and X. Liu. 2014. Diversity comparison of Pareto front approximations in many-objective optimization. IEEE Trans. Cybernet. 44, 12 (2014), 2568--2584.Google ScholarCross Ref
- M. Li, S. Yang, and X. Liu. 2014. Shift-based density estimation for Pareto-based algorithms in many-objective optimization. IEEE Trans. Evolution. Comput. 18, 3 (2014), 348--365.Google ScholarCross Ref
- M. Li, S. Yang, and X. Liu. 2015. A performance comparison indicator for Pareto front approximations in many-objective optimization. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’15). 703--710. Google ScholarDigital Library
- M. Li, S. Yang, and X. Liu. 2015. Bi-goal evolution for many-objective optimization problems. Artific. Intell. 228 (2015), 45--65. Google ScholarDigital Library
- M. Li, S. Yang, J. Zheng, and X. Liu. 2014. ETEA: A euclidean minimum spanning tree-based evolutionary algorithm for multiobjective optimization. Evol. Comput. 22, 2 (2014), 189--230. Google ScholarDigital Library
- M. Li and X. Yao. 2019. An empirical investigation of the optimality and monotonicity properties of multiobjective archiving methods. In Proceedings of the Evolutionary Multi-Criterion Optimization (EMO’19), in press.Google Scholar
- M. Li, L. Zhen, and X. Yao. 2017. How to read many-objective solution sets in parallel coordinates. IEEE Comput. Intell. Mag. 12, 4 (2017), 88--97.Google ScholarDigital Library
- M. Li and J. Zheng. 2009. Spread assessment for evolutionary multi-objective optimization. In Proceedings of the Evolutionary Multi-Criterion Optimization (EMO’09). Nantes, France, 216--230. Google ScholarDigital Library
- M. Li, J. Zheng, and G. Xiao. 2008. Uniformity assessment for evolutionary multi-objective optimization. In Proceedings of the IEEE Congress Evolutionary Computation (CEC’08). 625--632.Google Scholar
- Xu-yong Li, Jin-hua Zheng, and Juan Xue. 2005. A diversity metric for multi-objective evolutionary algorithms. In Proceedings of the International Conference on Natural Computation. 68--73. Google ScholarDigital Library
- A. Liefooghe and B. Derbel. 2016. A correlation analysis of set quality indicator values in multiobjective optimization. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’16). ACM, 581--588. Google ScholarDigital Library
- G. Lizárraga-Lizárraga, A. Hernández-Aguirre, and S. Botello-Rionda. 2008. G-Metric: An M-ary quality indicator for the evaluation of non-dominated sets. In Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation. 665--672. Google ScholarDigital Library
- G. Lizárraga-Lizárraga, A. Hernández-Aguirre, and S. Botello-Rionda. 2008. On the possibility to create a compatible--complete unary comparison method for evolutionary multiobjective algorithms. In Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation. 759--760. Google ScholarDigital Library
- G. Lizárraga-Lizárraga, A. Hernández-Aguirre, and S. Botello-Rionda. 2008. Some demonstrations about the cardinality of important sets of non-dominated sets. In Proceedings of the Mexican International Conference on Artificial Intelligence. 440--450. Google ScholarDigital Library
- E. M. Lopez and C. A. C. Coello. 2016. IGD+-EMOA: A multi-objective evolutionary algorithm based on IGD+. In Proceedings of the IEEE Congress on Evolutionary Computation. IEEE, 999--1006.Google ScholarDigital Library
- A. V. Lotov, V. A. Bushenkov, and G. K. Kamenev. 2013. Interactive Decision Maps: Approximation and Visualization of Pareto Frontier. Vol. 89. Springer Science 8 Business Media.Google Scholar
- A. V. Lotov, G. K. Kamenev, and V. E. Berezkin. 2002. Approximation and visualization of the Pareto frontier for nonconvex multicriteria problems. In Doklady Mathematics, Vol. 66. MAIK Nauka/Interperiodica, 260--262.Google Scholar
- L. Mandow and J. L. P. De La Cruz. 2010. Multiobjective A* search with consistent heuristics. J. ACM 57, 5 (2010), 27. Google ScholarDigital Library
- H. Meng, X. Zhang, and S. Liu. 2005. New quality measures for multiobjective programming. Adv. Natur. Comput. (2005), 431--431. Google ScholarDigital Library
- A. Messac and C. A. Mattson. 2004. Normal constraint method with guarantee of even representation of complete Pareto frontier. AIAA J. 42, 10 (2004), 2101--2111.Google ScholarCross Ref
- H. Meunier, E. G. Talbi, and P. Reininger. 2000. A multiobjective genetic algorithm for radio network optimization. In Proceedings of the Congress on Evolutionary Computation (CEC’00), Vol. 1. 317--324.Google Scholar
- K. Miettinen. 1999. Nonlinear Multiobjective Optimization. Kluwer Academic Publishers, Boston.Google Scholar
- K. Miettinen. 2014. Survey of methods to visualize alternatives in multiple criteria decision making problems. OR Spectr. 36 (2014), 3--37. Google ScholarDigital Library
- A. Mohammadi, M. N. Omidvar, and X. Li. 2013. A new performance metric for user-preference based multi-objective evolutionary algorithms. In Proceedings of the IEEE Congress on Evolutionary Computation. IEEE, 2825--2832.Google Scholar
- S. Mostaghim and J. Teich. 2005. A new approach on many objective diversity measurement. In Practical Approaches to Multi-Objective Optimization. Springer.Google Scholar
- A. Mukhopadhyay, U. Maulik, S. Bandyopadhyay, and C. A. C. Coello. 2014. A survey of multiobjective evolutionary algorithms for data mining: Part I. IEEE Trans. Evolution. Comput. 18, 1 (2014), 4--19.Google ScholarCross Ref
- A. Mukhopadhyay, U. Maulik, S. Bandyopadhyay, and C. A. C. Coello. 2014. Survey of multiobjective evolutionary algorithms for data mining: Part II. IEEE Trans. Evolution. Comput. 18, 1 (Feb 2014), 20--35.Google ScholarCross Ref
- M. Nicolini. 2004. Evaluating performance of multi-objective genetic algorithms for water distribution system optimization. In Proceedings of the 6th International Conference on Hydroinformatic. World Scientific, 850--857.Google ScholarCross Ref
- T. Okabe, Y. Jin, and B. Sendhoff. 2003. A critical survey of performance indices for multi-objective optimisation. In Proceedings of the Congress Evolutionary Computation (CEC’03), Vol. 2. 878--885.Google Scholar
- T. Pamulapati, R. Mallipeddi, and P. N. Suganthan. 2018. I<sup>+</sup><sub>SDE</sub>—An indicator for multi and many-objective optimization. IEEE Trans. Evolution. Comput., in press (2018).Google Scholar
- C. H. Papadimitriou and M. Yannakakis. 2000. On the approximability of trade-offs and optimal access of web sources. In Proceedings of the 41st Annual Symposium on Foundations of Computer Science (FOCS’00). 86--92. Google ScholarDigital Library
- D. H. Phan and J. Suzuki. 2013. R2-IBEA: R2 indicator based evolutionary algorithm for multiobjective optimization. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC’13). IEEE, 1836--1845.Google Scholar
- D. H. Phan, J. Suzuki, and I. Hayashi. 2012. Leveraging indicator-based ensemble selection in evolutionary multiobjective optimization algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’12). 497--504. Google ScholarDigital Library
- A. Ponsich, A. L. Jaimes, and C. A. C. Coello. 2013. A survey on multiobjective evolutionary algorithms for the solution of the portfolio optimization problem and other finance and economics applications. IEEE Trans. Evolution. Comput. 17, 3 (2013), 321--344. Google ScholarDigital Library
- R. C. Purshouse and P. J. Fleming. 2007. On the evolutionary optimization of many conflicting objectives. IEEE Trans. Evolution. Comput. 11, 6 (2007), 770--784. Google ScholarDigital Library
- C. Qian, G. Li, C. Feng, and K. Tang. 2018. Distributed Pareto optimization for subset selection. In Proceedings of the International Joint Conference on Artificial Intelligence. 1492--1498. Google ScholarDigital Library
- C. Qian, J. Shi, Y. Yu, K. Tang, and Z. Zhou. 2016. Parallel Pareto optimization for subset selection. In Proceedings of the International Joint Conference on Artificial Intelligence. 1939--1945. Google ScholarDigital Library
- C. Qian, Y. Yu, and Z. Zhou. 2015. Subset selection by Pareto optimization. In Advances in Neural Information Processing Systems. MIT Press, 1774--1782. Google ScholarDigital Library
- M. Ravber, M. Mernik, and M. Črepinšek. 2017. The impact of quality indicators on the rating of multi-objective evolutionary algorithms. Appl. Soft Comput. 55 (2017), 265--275. Google ScholarDigital Library
- N. Riquelme, C. Von Lücken, and B. Baran. 2015. Performance metrics in multi-objective optimization. In Proceedings of the Latin American Computing Conference (CLEI’15). 1--11.Google Scholar
- G. Rote, K. Buchin, K. Bringmann, S. Cabello, and M. Emmerich. 2016. Selecting points that maximize the convex hull volume. In Proceedings of the 19th Japan Conference on Discrete and Computational Geometry, Graphs, and Games. 58--60.Google Scholar
- G. Rudolph, O. Schütze, C. Grimme, C. Domínguez-Medina, and H. Trautmann. 2016. Optimal averaged Hausdorff archives for bi-objective problems: Theoretical and numerical results. Comput. Optim. Appl. 64, 2 (2016), 589--618. Google ScholarDigital Library
- G. Rudolph, O. Schütze, C. Grimme, and H. Trautmann. 2014. An aspiration set EMOA based on averaged Hausdorff distances. In Proceedings of the International Conference on Learning and Intelligent Optimization. 153--156.Google Scholar
- L. M. S. Russo and A. P. Francisco. 2014. Quick hypervolume. IEEE Trans. Evolution. Comput. 18, 4 (2014), 481--502.Google ScholarCross Ref
- S. Sayın. 2000. Measuring the quality of discrete representations of efficient sets in multiple objective mathematical programming. Math. Program. 87, 3 (2000), 543--560.Google ScholarCross Ref
- J. R. Schott. 1995. Fault Tolerant Design Using Single and Multicriteria Genetic Algorithm Optimization. Master’s thesis. Department of Aeronautics and Astronautics, Massachusetts Institute of Technology.Google Scholar
- O. Schutze, X. Esquivel, A. Lara, and C. C. A. Coello. 2012. Using the averaged Hausdorff distance as a performance measure in evolutionary multiobjective optimization. IEEE Trans. Evolution. Comput. 16, 4 (2012), 504--522. Google ScholarDigital Library
- M. R. Sierra and C. A. C. Coello. 2005. Improving PSO-based multi-objective optimization using crowding, mutation and -dominance. In Evolutionary Multi-criterion Optimization, Vol. 3410. Springer, 505--519. Google ScholarDigital Library
- N. Srinivas and K. Deb. 1994. Muiltiobjective optimization using nondominated sorting in genetic algorithms. Evolution. Comput. 2, 3 (1994), 221--248. Google ScholarDigital Library
- Y. Sun, G. G. Yen, and Y. Zhang. 2018. IGD indicator-based evolutionary algorithm for many-objective optimization problems. IEEE Trans. Evolution. Comput., published online.Google ScholarCross Ref
- K. C. Tan, T. H. Lee, and E. F. Khor. 2002. Evolutionary algorithms for multi-objective optimization: Performance assessments and comparisons. Artific. Intell. Rev. 17 (2002), 251--290. Issue 4. Google ScholarDigital Library
- L. Thiele. 2015. Indicator-based selection. In Springer Handbook of Computational Intelligence. Springer, 983--994.Google Scholar
- L. Thiele, K. Miettinen, P. J. Korhonen, and J. Molina. 2009. A preference-based evolutionary algorithm for multi-objective optimization. Evolution. Comput. 17, 3 (2009), 411--436. Google ScholarDigital Library
- Y. Tian, R. Cheng, X. Zhang, F. Cheng, and Y. Jin. 2017. An indicator based multi-objective evolutionary algorithm with reference point adaptation for better versatility. IEEE Trans. Evolution. Comput. published online (2017).Google Scholar
- Y. Tian, R. Cheng, X. Zhang, M. Li, and Y. Jin. 2019. Diversity assessment of multi-objective evolutionary algorithms: Performance metric and benchmark problems. IEEE Comput. Intell. Mag., submitted (2019).Google Scholar
- Y. Tian, X. Zhang, R. Cheng, and Y. Jin. 2016. A multi-objective evolutionary algorithm based on an enhanced inverted generational distance metric. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC’16). 5222--5229.Google Scholar
- H. Trautmann, T. Wagner, and D. Brockhoff. 2013. R2-EMOA: Focused multiobjective search using R2-indicator-based selection. In Proceedings of the International Conference on Learning and Intelligent Optimization. 70--74. Google ScholarDigital Library
- A. Trivedi, D. Srinivasan, K. Sanyal, and A. Ghosh. 2017. A survey of multiobjective evolutionary algorithms based on decomposition. IEEE Trans. Evolution. Comput. 21, 3 (2017), 440--462.Google ScholarDigital Library
- P. A. Tukey and J. W. Tukey. 1981. Preparation; Prechosen Sequences of Views. Wiley, Chapter Interpreting multivariate data, 189--213.Google Scholar
- T. Tusar and B. Filipic. 2015. Visualization of Pareto front approximations in evolutionary multiobjective optimization: A critical review and the prosection method. IEEE Trans. Evolution. Comput. 19, 2 (2015), 225--245.Google ScholarDigital Library
- E. L. Ulungu, J. Teghem, P. H. Fortemps, and D. Tuyttens. 1999. MOSA method: A tool for solving multiobjective combinatorial optimization problems. J. Multicrit. Decis. Anal. 8, 4 (1999), 221.Google ScholarCross Ref
- D. A. Van Veldhuizen. 1999. Multiobjective Evolutionary Algorithms: Classifications, Analyses, and New Innovations. Ph.D. Dissertation. Department of Electrical and Computer Engineering, Graduate School of Engineering, Air Force Institute of Technology, Wright-Patterson AFB, Ohio. Google ScholarDigital Library
- D. A. Van Veldhuizen and G. B. Lamont. 1998. Evolutionary computation and convergence to a Pareto front. In Proceedings of the Genetic Programming Conference—Late-breaking Papers. 221--228.Google Scholar
- D. Vaz, L. Paquete, C. M. Fonseca, K. Klamroth, and M. Stiglmayr. 2015. Representation of the non-dominated set in biobjective discrete optimization. Comput. Operat. Res. 63 (2015), 172--186. Google ScholarDigital Library
- D. Vaz, L. Paquete, and A. Ponte. 2013. A note on the epsilon-indicator subset selection. Theoret. Comput. Sci. 499 (2013), 113--116. Google ScholarDigital Library
- A. Viana and J. P. de Sousa. 2000. Using metaheuristics in multiobjective resource constrained project scheduling. Eur. J. Operat. Res. 120, 2 (2000), 359--374.Google ScholarCross Ref
- C. A. R. Villalobos and C. C. A. Coello. 2012. A new multi-objective evolutionary algorithm based on a performance assessment indicator. In Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation Conference (GECCO’12). ACM, 505--512. Google ScholarDigital Library
- T. Wagner, H. Trautmann, and D. Brockhoff. 2013. Preference Articulation by Means of the R2 Indicator. Springer, 81--95.Google Scholar
- D. J. Walker, R. M. Everson, and J. E. Fieldsend. 2013. Visualising mutually non-dominating solution sets in many-objective optimisation. IEEE Trans. Evolution. Comput. 17, 2 (2013), 165--184. Google ScholarDigital Library
- J. Wallenius, J. S. Dyer, P. C. Fishburn, R. E. Steuer, S. Zionts, and K. Deb. 2008. Multiple criteria decision making, multiattribute utility theory: Recent accomplishments and what lies ahead. Manage. Sci. 54, 7 (2008), 1336--1349. Google ScholarDigital Library
- H. Wang, Y. Jin, and X. Yao. 2017. Diversity assessment in many-objective optimization. IEEE Trans. Cybernet. 47, 6 (2017), 1510--1522.Google ScholarCross Ref
- R. Wang, S. Chen, L. Ma, S. Cheng, and Y. Shi. 2018. Multi-indicator bacterial foraging algorithm with Kriging model for many-objective optimization. In Proceedings of the International Conference on Swarm Intelligence. 530--539.Google Scholar
- S. Wang, S. Ali, T. Yue, Y. Li, and M. Liaaen. 2016. A practical guide to select quality indicators for assessing Pareto-based search algorithms in search-based software engineering. In Proceedings of the 38th International Conference on Software Engineering (ICSE’16). 631--642. Google ScholarDigital Library
- S. Wessing and B. Naujoks. 2010. Sequential parameter optimization for multi-objective problems. In Proceedings of the IEEE Congress on Evolutionary Computation. 1--8.Google Scholar
- L. While, L. Bradstreet, and L. Barone. 2012. A fast way of calculating exact hypervolumes. IEEE Trans. Evolution. Comput. 16, 1 (2012), 86--95. Google ScholarDigital Library
- L. While, P. Hingston, L. Barone, and S. Huband. 2006. A faster algorithm for calculating hypervolume. IEEE Trans. Evolution. Comput. 10, 1 (2006), 29--38. Google ScholarDigital Library
- J. Wu and S. Azarm. 2001. Metrics for quality assessment of a multiobjective design optimization solution set. Trans. ASME J. Mech. Design 123 (2001), 18--25.Google ScholarCross Ref
- Y. Xiang, Y. Zhou, Z. Zheng, and M. Li. 2018. Configuring software product lines by combining many-objective optimization and SAT solvers. ACM Trans. Softw. Eng. Methodol. 26, 4 (2018), 14. Google ScholarDigital Library
- S. Yang, M. Li, X. Liu, and J. Zheng. 2013. A grid-based evolutionary algorithm for many-objective optimization. IEEE Trans. Evolution. Comput. 17, 5 (2013), 721--736. Google ScholarDigital Library
- G. G. Yen and Z. He. 2014. Performance metric ensemble for multiobjective evolutionary algorithms. IEEE Trans. Evolution. Comput. 18, 1 (2014), 131--144.Google ScholarCross Ref
- I. Yevseyeva, A. P. Guerreiro, M. Emmerich, and C. M. Fonseca. 2014. A portfolio optimization approach to selection in multiobjective evolutionary algorithms. In Proceedings of the International Conference on Parallel Problem Solving from Nature (PPSN’14). 672--681.Google Scholar
- H. Yildiz and S. Suri. 2012. On Klee’s measure problem for grounded boxes. In Proceedings of the 28th Annual Symposium on Computational Geometry. ACM, 111--120. Google ScholarDigital Library
- G. Yu, J. Zheng, and X. Li. 2015. An improved performance metric for multiobjective evolutionary algorithms with user preferences. In Proceedings of the IEEE Congress on Evolutionary Computation. 908--915.Google Scholar
- M. Zeleny. 1973. Compromise programming. In Multi Criteria Decision Making, J. L. Cochrane and M. Zeleny (Eds.). University of South Carolina Press, 262--301.Google Scholar
- A. Zhou, Y. Jin, Q. Zhang, B. Sendhoff, and E. Tsang. 2006. Combining model-based and genetics-based offspring generation for multi-objective optimization using a convergence criterion. In Proceedings of the IEEE Congress on Evolutionary Computation. 892--899.Google Scholar
- A. Zhou, B. Y. Qu, H. Li, S. Z. Zhao, P. N. Suganthan, and Q. Zhang. 2011. Multiobjective evolutionary algorithms: A survey of the state of the art. Swarm Evolution. Comput. 1, 1 (2011), 32--49.Google ScholarCross Ref
- E. Zitzler. 1999. Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications. Ph.D. Dissertation. Zurich, Switzerland: Swiss Federal Institute of Technology (ETH).Google Scholar
- E. Zitzler, D. Brockhoff, and L. Thiele. 2007. The hypervolume indicator revisited: On the design of Pareto-compliant indicators via weighted integration. In Proceedings of the International Conference on Evolutionary Multi-Criterion Optimization. Springer, 862--876. Google ScholarDigital Library
- E. Zitzler, K. Deb, and L. Thiele. 2000. Comparison of multiobjective evolutionary algorithms: Empirical results. Evolution. Comput. 8, 2 (2000), 173--195. Google ScholarDigital Library
- E. Zitzler, J. Knowles, and L. Thiele. 2008. Quality assessment of Pareto set approximations. In Multiobjective Optimization, J. Branke, Kalyanmoy Deb, Kaisa Miettinen, and Roman Slowinski (Eds.). Vol. 5252. Springer, Berlin, 373--404. Google ScholarDigital Library
- E. Zitzler and S. Künzli. 2004. Indicator-based selection in multiobjective search. In Proceedings of the International Conference on Parallel Problem Solving from Nature (PPSN’04). 832--842.Google Scholar
- E. Zitzler and L. Thiele. 1998. Multiobjective optimization using evolutionary algorithms—A comparative case study. In Proceedings of the International Conference on Parallel Problem Solving from Nature (PPSN’98). 292--301. Google ScholarDigital Library
- E. Zitzler and L. Thiele. 1999. Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach. IEEE Trans. Evolution. Comput. 3, 4 (1999), 257--271. Google ScholarDigital Library
- E. Zitzler, L. Thiele, M. Laumanns, C. M. Fonseca, and V. G. Da Fonseca. 2003. Performance assessment of multiobjective optimizers: An analysis and review. IEEE Trans. Evolution. Comput. 7, 2 (2003), 117--132. Google ScholarDigital Library
Index Terms
- Quality Evaluation of Solution Sets in Multiobjective Optimisation: A Survey
Recommendations
The impact of Quality Indicators on the rating of Multi-objective Evolutionary Algorithms
The figure displays confidence interval of Quality Indicators GD to R2 which both assess convergence to the Pareto optimal front. We can see that they are in contradiction with each other and that they ranked MOEAs exactly the opposite. CRS4EAs has ...
Hypervolume-based multiobjective optimization: Theoretical foundations and practical implications
In recent years, indicator-based evolutionary algorithms, allowing to implicitly incorporate user preferences into the search, have become widely used in practice to solve multiobjective optimization problems. When using this type of methods, the ...
A Correlation Analysis of Set Quality Indicator Values in Multiobjective Optimization
GECCO '16: Proceedings of the Genetic and Evolutionary Computation Conference 2016A large spectrum of quality indicators has been proposed so far to assess the performance of discrete Pareto set approximations in multiobjective optimization. Such indicators assign, to any solution set, a real-value reflecting a given aspect of ...
Comments