Skip to main content
SpringerLink
Log in

Preference-driven multi-objective GP search for regression models with new dominance principle and performance indicators

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Regression is a multi-objective optimization task, which aims to determine accurate and simple relationship expressions between variables. Multi-objective genetic programming (MOGP) methods are popularly-used for regression, which search for trade-off solutions of all objectives, producing a set of solutions (Pareto front). Yet, users normally are not interested in the whole Pareto front and select certain solutions based on preference for specific tasks. There are existing preference-based multi-objective methods. However, existing techniques may not be compliant to Pareto dominance, leading convergence to be deteriorated, or require extra parameters. To handle these issues, a preference-driven dominance (pd-dominance) principle is designed, which is Pareto-compliant and parameterless. Then it is introduced into two base MOGP methods (NSGP (non-dominated sorting genetic programming) and SPGP (strength Pareto genetic programming)) to form two new preference-driven MOGP methods, i.e. pdNSGP and pdSPGP. In addition, three existing performance indicators for multi-objective optimization are improved to adapt to regression tasks. Results show that pdNSGP and pdSPGP outperform MOGP methods with popular preference techniques. For example on the function Dic3, pdSPGP reaches \(3E-2\) based on a distance measure (the lower the better); while the reference MOGP methods achieve \(1.93E-1\). Moreover, compared with seven reference regression methods, the proposed methods ranks the first three places for most test cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Users provide the value/level for each objective that they aim to achieve [2].

  2. Users provide a weight for each objective, which reflects its importance [2].

  3. Users provide the information that how much gain in one objective correspond to an unit reduce in others [2].

  4. Weka is a set of machine learning algorithms to solve real-world data mining problems [36].

References

  1. Abouhawwash M, Deb K (2021) Reference point based evolutionary multi-objective optimization algorithms with convergence properties using kktpm and asf metrics. J Heuristics 3:1–40

    Google Scholar 

  2. Bechikh S, Kessentini M, Said LB, Ghédira K (2015) Preference incorporation in evolutionary multiobjective optimization: A survey of the state-of-the-art. Advances in Computers 98:141–207

    Article  Google Scholar 

  3. Said LB, Bechikh SK (2010) The r-dominance: A new dominance relation for interactive evolutionary multicriteria decision making. IEEE Transactions on Evolutionary Computation 14(5):801–818

    Article  Google Scholar 

  4. Bi Y, Xue B, Zhang M (2021) Multi-objective genetic programming for feature learning in face recognition. Applied Soft Computing 103(4):107152

    Article  Google Scholar 

  5. Cai X, Xiao Y, Li M (2020) A grid-based inverted generational distance for multi/many-objective optimization. IEEE Transactions on Evolutionary Computation PP(99):1–14

    Google Scholar 

  6. Casadei F, Pappa GL (2021) Multi-region symbolic regression: combining functions under a multi-objective approach. Natural Comput 20(1):753–773

  7. Cheng W, Taylor JMG, Gu T, Tomlins SA, Mukherjee B (2019) Informing a risk prediction model for binary outcomes with external coefficient information. Journal of the Royal Statistical Society Series C 68(PT.1):121–139

    Article  MathSciNet  Google Scholar 

  8. Deb K (2001) Multi-objective optimization using evolutionary algorithms. John Wiley & Sons

  9. Deb K, Jain H (2014) An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part i: Solving problems with box constraints. IEEE Trans. Evolutionary Computation 18(4):577–601

    Article  Google Scholar 

  10. Deb K, Sundar J (2006) Reference point based multi-objective optimization using evolutionary algorithms. In: Genetic and evolutionary computation conference, Seattle, Washington, USA

  11. Deb K, Thiele L, Laumanns M, Zitzler E (2006) Scalable test problems for evolutionary multiobjective optimization. ETH Zurich Research Collection

  12. Deng J, Zhang Q (2020) Combining simple and adaptive monte carlo methods for approximating hypervolume. IEEE Transactions on Evolutionary Computation PP(99):1–12

    Google Scholar 

  13. Dua D, Graff C (2017) UCI machine learning repository. http://archive.ics.uci.edu/ml

  14. Ebta B, Ag C, Af D, Ms E, Gwwf G (2020) Multi-objective optimization for the reliable pollution-routing problem with cross-dock selection using pareto-based algorithms - sciencedirect. J Clean Prod 276:1–60

  15. Falcón-Cardona JG, Coello CAC (2019) Convergence and diversity analysis of indicator-based multi-objective evolutionary algorithms. In: GECCO ’19: The genetic and evolutionary computation conference, pp 1–8

  16. Falcón-Cardona JG, Coello CAC (2020) Indicator-based multi-objective evolutionary algorithms: A comprehensive survey. ACM Computing Surveys 53(2):1–35

    Article  Google Scholar 

  17. Fonseca CM, Fleming PJ (1993) Genetic algorithms for multiobjective optimization: Formulation, discussion and generalization. In: Proceedings of the 5th international conference on genetic algorithms (ICGA’93), Morgan Kaufmann, Urbana-Champaign, IL, USA, pp 416–423

  18. Hja B, Ckk B, Cyc B, Kly B (2019) A multi-objective evolutionary approach for fuzzy regression analysis. Expert Systems with Applications 130:225–235

    Article  Google Scholar 

  19. Horn J, Nafpliotis N, Goldberg DE (1994) A niched pareto genetic algorithm for multiobjective optimization. In: IEEE World congress on computational intelligence. IEEE, pp 82–87

  20. Izadi Rad H, Feng J, Iba H (2018) GP-RVM: Genetic programing-based symbolic regression using relevance vector machine. In: International conference on systems, man, and cybernetics (SMC), pp 1–8

  21. Jk K, Derner E, Babuka R (2021) Multi-objective symbolic regression for physics-aware dynamic modeling. Expert Syst Appl 182:115210

  22. Ke L, Liao M, Deb K, Min G, Yao X (2020) Does preference always help? a holistic study on preference-based evolutionary multi-objective optimisation using reference points. IEEE Transactions on Evolutionary Computation PP(99):1–1

    Google Scholar 

  23. Khor EF, Lee TH, Sathikannan R, Tan KC (2003) An evolutionary algorithm with advanced goal and priority specification for multi-objective optimization. Journal of Artificial Intelligence Research 18:183–215

    Article  MathSciNet  MATH  Google Scholar 

  24. Kiran A, Butt WH, Shaukat A, Farooq MU, Fatima U, Azam F, Anwar Z (2020) Multi-objective regression test suite optimization using three variants of adaptive neuro fuzzy inference system. PLOS ONE 15(12):1–23

  25. Knowles J, Corne D (1999) The pareto archived evolution strategy: A new baseline algorithm for pareto multiobjective optimisation. In: Congress on evolutionary computation, pp 1–8

  26. Koza JR, Keane MA, Streeter MJ, Mydlowec W, Yu J, Lanza G (2006) Genetic programming IV: Routine human-competitive machine intelligence. Springer Science & Business Media, vol 5

  27. Liang J, Liu Y, Xue Y (2020) Preference-driven pareto front exploitation for bloat control in genetic programming. Applied Soft Computing 92:1–18

    Article  Google Scholar 

  28. Liang J, Xue Y, Wang J (2020) Bi-objective memetic gp with dispersion-keeping pareto evaluation for real-world regression. Information Sciences 539:16–35

    Article  MathSciNet  MATH  Google Scholar 

  29. Liu L, Guo Y, Wu Y, Zhu R (2019) An r-dominance-based bare-bones multi-objective particle swarm optimization for attitude maneuver of flexible spacecraft filled with liquid. In: IEEE International conference on networking, sensing and control, pp 263–268

  30. Luo W, Shi L, Lin X, Coello CAC (2019) The \(\widehat{g}\)-dominance relation for preference-based evolutionary multi-objective optimization. In: Congress of evolutionary computation, pp 1–8

  31. Molina J, Santana LV, Hernández-Díaz AG, Coello CAC, Caballero R (2009) g-dominance: Reference point based dominance for multiobjective metaheuristics. European Journal of Operational Research 197(2):685–692

    Article  MATH  Google Scholar 

  32. Neri F, Cotta C, Eiben ÁE, Smith JE, Oca MAMD, Cotta C, Neri F, Neri F, Cotta C, Sudholt D (2012) Handbook of memetic algorithms. Comput Rev 53(10):597

  33. Poli R, Langdon WB, Mcphee NF (2008) A field guide to genetic programming. lulu.com. [S.L.]: Lulu Press (lulu.com) 10(2):229–230

    Google Scholar 

  34. Smits GF, Kotanchek M (2005) Pareto-front exploitation in symbolic regression. Genetic Programming Theory and Practice II 2005:283–299

    Article  Google Scholar 

  35. Tirkolaee EB, Goli A, Weber GW, Szwedzka K (2020) A novel formulation for the sustainable periodic waste collection arc-routing problem: A hybrid multi-objective optimization algorithm. In: 15th International congress on logistics and scm systems

  36. Witten IH, Frank E (2011) Data mining : practical machine learning tools and techniques. Acm Sigmod Record 31(1):76–77

    Article  Google Scholar 

  37. Yu G, Jin Y, Olhofer M (2019) Benchmark problems and performance indicators for search of knee points in multiobjective optimization. IEEE Trans Cybernetics 99:1–14

  38. Zhang C (2014) Genetic programming for symbolic regression. https://pdfs.semanticscholar.org/e5ee/ddd04b8344fd4f39a5836be686886c80df13.pdf

  39. Zitzler E, Laumanns M, Thiele L (2001) Spea2: Improving the strength pareto evolutionary algorithm. In: Proc. evolutionary methods for design optimization and control with applications to industrial problems, pp 95–100

  40. Zitzler E, Thiele L, Laumanns M, Fonseca CM, Fonseca VGD (2003) Performance assessment of multiobjective optimizers: an analysis and review. IEEE Transactions on Evolutionary Computation 7(2):117–132

    Article  Google Scholar 

Download references

Acknowledgements

This work is funded by National Natural Science Foundation of China (grant number: 61902281 and 61876089) and Tianjin Science and Technology Program (grant number: 19PTZWHZ00020).

Author information

Authors and Affiliations

  1. Tianjin Key Laboratory of Autonomous Intelligent Technology and System, Tiangong University, 300387, Tianjin, China

    Jiayu Liang, Ludi Zheng & Han Wu

  2. School of Computer and Software, Nanjing University of Information Science and Technology, 210044, Nanjing, China

    Yu Xue

Authors
  1. Jiayu Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ludi Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Han Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiayu Liang.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, J., Zheng, L., Wu, H. et al. Preference-driven multi-objective GP search for regression models with new dominance principle and performance indicators. Appl Intell 52, 15809–15823 (2022). https://doi.org/10.1007/s10489-022-03228-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03228-6

Keywords

Search

Navigation