A novel harmony search algorithm and its application to data clustering

https://doi.org/10.1016/j.asoc.2020.106273Get rights and content

Highlights

  • A novel harmony search is proposed with three pitch selection and production rules.

  • The rules utilize the gathered knowledge during search in generation of harmonies.

  • A probabilistic self-adaptive selection scheme is used to balance search behaviors.

  • The algorithm is applied on numerical optimization and data clustering problems.

  • Results show the superiority of proposed algorithm to the compared algorithms.

Abstract

This paper presents a variant of harmony search algorithm (HS), called best–worst-mean harmony search (BWM_HS). The main difference between the proposed algorithm and the canonical HS is that it employs a modified memory consideration procedure to utilize more efficiently the accumulated knowledge and experience in harmony memory (HM). To this aim, the random harmony selection scheme of this procedure is replaced with three novel pitch selection and production rules. These rules use the information of the current best and worst harmonies as well as the mean of all harmonies to guide the search process. To further utilize the valuable information of HM, two new harmonies are generated at each iteration where the better one will compete with the current worst harmony. The mean of all harmonies is always employed to produce a new harmony. On the other hand, each pitch of the second one is obtained by the rules that consider the information of the best and worst harmonies. These rules can present either explorative or exploitative search behaviors at different stages of search. Thus, a probabilistic self-adaptive selection scheme decides to choose between them to properly balance the exploration and exploitation abilities. The general performance of BWM_HS for solving optimization problems is evaluated against CEC 2017 benchmark functions and its results are compared with HS and eight state-of-the-art variants of HS. The comparison indicates that the performance of BWM_HS is better than or equal to the compared algorithms with respect to the accuracy, robustness, and convergence speed criteria. Moreover, the performance of BWM_HS in solving clustering problems is investigated by applying it for clustering several well-known benchmark datasets. The experimental results show that, in general, the BWM_HS outperforms other well-known algorithms in the literature and in particular, it significantly improves the statistical results for one dataset.

Introduction

In a modern society, many complex problems arise in science and engineering domains, such as improving the reliability of complex systems designed for industrial engineering [1], estimation of parameters for mathematical models of biological systems [2] or industrial chemical reactions [3], and prediction of protein structure for drug design [4]. It is possible to convert these problems into numerical optimization problems with some objective functions and then develop some processes, called optimization methods, to find the best solution among the possible solutions for maximizing or minimizing the objective functions [5], [6], [7]. Classical mathematical optimization approaches like linear, non-linear, or dynamic programming are exact methods of such processes [8]. However, the capability of these approaches for solving real-world optimization problems is very limited because they are computationally expensive, and in addition, they pose strong presumptions that are hardly met in such problems. As an alternative approach, during the last few decades, meta-heuristic optimization algorithms have come into sight to efficiently tackle the optimization problems in various domains [9], [10], [11], [12], [13], [14], [15], [16], [17], [18].

Harmony search (HS) [19] is an example of meta-heuristic algorithms that, due to its simplicity and effectiveness, has been applied in a wide spectrum of real world applications. For instance, a literature review conducted by Manjarres et al. [20] based on a reference repository which indexes 160 papers until 2012 reveals that HS has been employed in engineering, management, medical, robotic, control, power, and many cross-application areas. This algorithm imitates the improvisation process of musicians when they are looking for a new harmony. More precisely, HS simulates a candidate solution for a problem as a harmony where each decision variable of the problem is considered as a pitch of a different musical instrument. Then, HS moves candidate solutions in the search space to find the optimal solution through application of some search operators that model the pitch selection and adjustment actions in production of right harmonies. Despite significant success of HS applications, several studies have pointed out that search mechanisms of HS exhibit excellent explorative behavior but poor exploitative ability [21], [22]. This inefficiency is originated from the insufficient utilization of history information and experience gathered during search process in harmony memory (HM). To explain more, the pitch selection of canonical HS is implemented by a memory consideration mechanism that uses the value of decision variables of randomly selected harmonies from the current HM to generate a new harmony. In other words, it does not utilize the valuable information hidden in HM to follow specific search direction in the selection process. Thus, each time that a new harmony is generated, the algorithm is guided to a random new search direction that is more helpful for improving the exploration ability.

Several studies have tried to ameliorate the above-mentioned issue through replacing the random selection scheme in memory consideration with selection methods that utilize the fitness of harmonies while the rest of improvisation process remains untouched [23], [24], [25]. On the other hand, many works have designed a modified improvisation process that considers information of different harmonies from HM, either deterministically or probabilistically [22], [23], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37].

Indeed, the performance of any population-based optimization algorithm depends on the extent of balance between its exploitation and exploration abilities during the course of search process. A reasonable balance is enhancing the exploration at the early stages of search to help the algorithm to explore the new regions of a large search space quickly. Then, gradually increasing the exploitation ability of algorithm at the later stages to develop the search space in the vicinity of the current good solutions. However, the harmony selection scheme of the aforementioned works does not take into account the stage of search process to find out which ability needs to be promoted and thus to select the appropriate information from HM. Furthermore, their approaches do not benefit from the population level information and only use the information of a few harmonies. Moreover, except few, they often generate a single new harmony in each iteration, which is not efficient in utilizing the information of HM in comparison with producing more new harmonies.

To overcome the above limitations, this paper develops a new variant of HS that is called best–worst-mean harmony search (BWM_HS). The main contributions reflected in the proposed BWM_HS algorithm are as follows:

  • The random selection scheme in memory consideration of HS is substituted with three novel pitch selection and production rules that utilize the accumulated knowledge and experience in harmony memory (HM) to guide the search process.

  • Two rules employ the individual level information, i.e. the information of current best and worst harmonies in HM. The other rule uses the population level information that is the mean of all harmonies in HM.

  • The proposed algorithm generates two new harmonies (instead of one) to increase the utilization of valuable history information of HM. A new harmony is produced by the rules that use the individual level information and another new one is obtained by the rule that shares the population level of information. To update HM, the better new harmony will compete with the current worst harmony of HM to replace it.

  • Rules that utilize the individual level information are designed with a bias toward a special search direction during different stages of search process such that they boost either the exploitation or the exploration ability. Then, to properly establish the balance between these search behaviors, a probabilistic self-adaptive selection scheme helps the proposed algorithm to choose between them for generating each pitch of a new harmony.

  • Despite all the above-mentioned modifications, the proposed BWM_HS algorithm does not change the computational complexity and the simple framework of canonical HS, and is straightforward to implement, too.

In order to demonstrate the performances of proposed algorithm, it is evaluated against CEC2017 test suite [38] and the obtained results are compared with HS and eight state-of-the-art variants of HS. The comparison indicates that the performance of the proposed algorithm is better than or equal to the considered algorithms in terms of accuracy, convergence speed, and robustness. Moreover, application of the proposed algorithm for solving ten benchmark clustering problems is verified and compared with HS variants and also ten state-of-the-art clustering algorithms. The proposed method has obtained competitive results compared to the other algorithms.

The remainder of this paper is organized as follows. Section 2 begins with describing the standard HS algorithm in Section 2.1. Then, a literature review regarding HS improvements is presented in Section 2.2. Next, the proposed algorithm is elaborated in Section 3. Thereafter, in Section 4, some experiments are performed on CEC 2017 as numerical optimization problems and also on some benchmark clustering problems; and then a comparison is made between the obtained results by the proposed algorithms with several well-known and state-of-the-art algorithms. Finally, the conclusions are summarized in Section 5.

Section snippets

A review of HS algorithm

HS is an optimization algorithm that imitates musicians’ search for finding a perfect state of a harmony [19]. This algorithm, after initialization of parameters and HM, repeatedly executes a search mechanism to create a new harmony. This search mechanism includes randomly selecting a pitch from HM, randomly selecting a pitch from HM and adjusting it, and randomly generating a pitch. The greedy selection between the new generated harmony and the worst harmony in HM is performed to select the

Proposed algorithm

As explained in Section 2.1, in each iteration of the standard HS, a new harmony is produced for updating the HM where pitches of this harmony are generated by applying one of the three following rules: randomly selecting a pitch from HM, randomly selecting a pitch from HM and adjusting it, randomly generating a pitch. Thus, the valuable information and experience already collected during search process and stored in HM is mainly utilized by the first rule. In fact, this rule implements the

Experimental results and discussions

This section extensively evaluates the performance of BWM_HS algorithm. To this end, first in Section 4.1 some experiments are conducted on several well-known numerical optimization problems. Then, in Section 4.2, the application of BWM_HS on ten well-known datasets of clustering problems is investigated.

The BWM_HS algorithm is implemented via MATLAB 2015a environment using windows 10 operating system. All experiments are performed on a computer with a 2.19 GHz Intel(R), Xeon(R), 4 processor,

Conclusion

This paper proposes the BWM_HS algorithm as a new variant of HS that utilizes more efficiently the valuable information stored in HM to guide the search process. For this purpose, the BWM_HS uses a modified memory consideration procedure as the random harmony selection scheme is replaced with three novel pitch selection and production rules. Two new harmonies are generated at each iteration of BWM_HS algorithm by applying these three rules to further utilize the information of HM. To generate a

CRediT authorship contribution statement

Kazem Talaei: Methodology, Investigation, Writing - original draft. Amin Rahati: Supervision, Writing - review & editing, Validation. Lhassane Idoumghar: Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (107)

  • KarimiM. et al.

    Using tournament selection approach to improve harmony search algorithm for modeling of proton exchange membrane fuel cell

    Int. J. Electrochem. Sci.

    (2012)
  • Al-BetarM.A. et al.

    Novel selection schemes for harmony search

    Appl. Math. Comput.

    (2012)
  • Al-BetarM.A. et al.

    Tournament-based harmony search algorithm for non-convex economic load dispatch problem

    Appl. Soft Comput.

    (2016)
  • OmranM.G. et al.

    Global-best harmony search

    Appl. Math. Comput.

    (2008)
  • PanQ.-K. et al.

    A self-adaptive global best harmony search algorithm for continuous optimization problems

    Appl. Math. Comput.

    (2010)
  • EnayatifarR. et al.

    LAHS: a novel harmony search algorithm based on learning automata

    Commun. Nonlinear Sci. Numer. Simul.

    (2013)
  • CastelliM. et al.

    Geometric selective harmony search

    Inform. Sci.

    (2014)
  • ShabaniM. et al.

    Selective refining harmony search: A new optimization algorithm

    Expert Syst. Appl.

    (2017)
  • ZouD. et al.

    Novel global harmony search algorithm for unconstrained problems

    Neurocomputing

    (2010)
  • KumarV. et al.

    Parameter adaptive harmony search algorithm for unimodal and multimodal optimization problems

    J. Comput. Sci.

    (2014)
  • KhaliliM. et al.

    Global dynamic harmony search algorithm: GDHS

    Appl. Math. Comput.

    (2014)
  • MahdaviM. et al.

    An improved harmony search algorithm for solving optimization problems

    Appl. Math. Comput.

    (2007)
  • AlatasB.

    Chaotic harmony search algorithms

    Appl. Math. Comput.

    (2010)
  • LuoK. et al.

    Enhanced self-adaptive global-best harmony search without any extra statistic and external archive

    Inform. Sci.

    (2019)
  • HasanB.H.F. et al.

    Hybridizing harmony search algorithm with different mutation operators for continuous problems

    Appl. Math. Comput.

    (2014)
  • KattanA. et al.

    A dynamic self-adaptive harmony search algorithm for continuous optimization problems

    Appl. Math. Comput.

    (2013)
  • YadavP. et al.

    An intelligent tuned harmony search algorithm for optimisation

    Inform. Sci.

    (2012)
  • OuyangH.-b. et al.

    Improved harmony search algorithm: LHS

    Appl. Soft Comput.

    (2017)
  • RaoR.V. et al.

    Teaching–learning-based optimization: a novel method for constrained mechanical design optimization problems

    Comput. Aided Des.

    (2011)
  • Al-BetarM.A. et al.

    Island-based harmony search for optimization problems

    Expert Syst. Appl.

    (2015)
  • SarkhelR. et al.

    An improved harmony search algorithm embedded with a novel piecewise opposition based learning algorithm

    Eng. Appl. Artif. Intell.

    (2018)
  • LayebA.

    A hybrid quantum inspired harmony search algorithm for 0–1 optimization problems

    J. Comput. Appl. Math.

    (2013)
  • OuyangH.-b. et al.

    Hybrid harmony search particle swarm optimization with global dimension selection

    Inform. Sci.

    (2016)
  • AssadA. et al.

    A hybrid harmony search and simulated annealing algorithm for continuous optimization

    Inform. Sci.

    (2018)
  • XiangW.-l. et al.

    An improved global-best harmony search algorithm for faster optimization

    Expert Syst. Appl.

    (2014)
  • WangC.-M. et al.

    Self-adaptive harmony search algorithm for optimization

    Expert Syst. Appl.

    (2010)
  • AineS. et al.

    Adaptive parameter control of evolutionary algorithms to improve quality-time trade-off

    Appl. Soft Comput.

    (2009)
  • LinA. et al.

    Adaptive comprehensive learning particle swarm optimization with cooperative archive

    Appl. Soft Comput.

    (2019)
  • CarmonaC.J. et al.

    Web usage mining to improve the design of an e-commerce website: OrOliveSur. com

    Expert Syst. Appl.

    (2012)
  • AnayaA.R. et al.

    Application of machine learning techniques to analyse student interactions and improve the collaboration process

    Expert Syst. Appl.

    (2011)
  • BassiouN. et al.

    Long distance bigram models applied to word clustering

    Pattern Recognit.

    (2011)
  • PortelaN.M. et al.

    Semi-supervised clustering for MR brain image segmentation

    Expert Syst. Appl.

    (2014)
  • ZhengB. et al.

    Breast cancer diagnosis based on feature extraction using a hybrid of K-means and support vector machine algorithms

    Expert Syst. Appl.

    (2014)
  • HathawayR.J. et al.

    Local convergence of the fuzzy c-means algorithms

    Pattern Recognit.

    (1986)
  • KarabogaD. et al.

    A novel clustering approach: Artificial bee colony (ABC) algorithm

    Appl. Soft Comput.

    (2011)
  • ShelokarP. et al.

    An ant colony approach for clustering

    Anal. Chim. Acta

    (2004)
  • NandaS.J. et al.

    A survey on nature inspired metaheuristic algorithms for partitional clustering

    Swarm Evol. Comput.

    (2014)
  • MaulikU. et al.

    Genetic algorithm-based clustering technique

    Pattern Recognit.

    (2000)
  • KuoW. et al.

    Optimal Reliability Modeling: Principles and Applications

    (2003)
  • GáborA. et al.

    Robust and efficient parameter estimation in dynamic models of biological systems

    BMC Syst. Biol.

    (2015)
  • Cited by (31)

    • Data clustering using leaders and followers optimization and differential evolution[Formula presented]

      2023, Applied Soft Computing
      Citation Excerpt :

      The exploration and exploitation processes constitute the backbone of the optimization algorithm. Whereas the exploitation process is applied to search for a better solution nearby the current solution, the exploration process is performed to search for new solutions far from the current solution [2,18,34–41]. For this, the exploitation and exploration processes are also called local search and global search respectively.

    • Harmony search algorithm and related variants: A systematic review

      2022, Swarm and Evolutionary Computation
      Citation Excerpt :

      According to Gholami et al., the proposed algorithm can produce comparable results compared to basic HS etc., and other algorithms such as GA and PSO. In the same way, Talaei et al. [78] proposed an improved harmony search algorithm for data clustering to study its ability to solve real-world optimization problems, which proposed algorithm outperforms other algorithms in 66.6% of the dataset clustering. In [81], harmony search algorithm can even be used in Intrusion Detection System (IDS) to prevent cybercrime.

    • A two-stage cooperative scatter search algorithm with multi-population hierarchical learning mechanism

      2022, Expert Systems with Applications
      Citation Excerpt :

      Typical algorithms include water wave optimization (WWO) (Zhao, Zhang, et al., 2020), gravity search algorithm (GSA) (Wang, Gao, Yu, Cai, & Wang, 2021a), charging system search (CSS) (D’Ambrosio, Spiller, & Curti, 2020), black hole search algorithm (BHA) (Pashaei & Aydin, 2017), thermal exchange optimization algorithm (TEO) (Kaveh & Dadras, 2017), and water cycle algorithm (WCA) (Chen, Wang, Dong, & Wang, 2020), etc. There are also some intelligent search algorithms inspired by human behavior, such as the teaching–learning based optimization algorithms (TLBO) (Ma, Zhang, Song, & Chen, 2021), harmony search (HS)(Talaei, Rahati, & Idoumghar, 2020), socio evolution and learning optimization algorithm (SELO) (Kumar, Kulkarni, & Satapathy, 2018), and cultural evolution algorithm (CEA) (Maheri, Jalili, Hosseinzadeh, Khani, & Miryahyavi, 2021), etc. The rest of this paper is organized as follows.

    View all citing articles on Scopus
    View full text