Elsevier

Expert Systems with Applications

Volume 103, 1 August 2018, Pages 184-195
Expert Systems with Applications

Combining NeuroEvolution and Principal Component Analysis to trade in the financial markets

https://doi.org/10.1016/j.eswa.2018.03.012Get rights and content

Highlights

  • A system that uses PCA and NEAT to generate a lucrative trading signal is proposed.

  • PCA is used to reduce the dimensionality of the input financial data.

  • NEAT creates and evolves the neural network that generates the trading signal.

  • The proposed system outperforms the B&H strategy in multiple markets.

  • The PCA method has a big influence in the performance of the system.

Abstract

When investing in the financial market, determining a trading signal that can fulfill the financial performance demands of an investor is a difficult task and a very popular research topic in the financial investment area. This paper presents an approach combining the principal component analysis (PCA) with the NeuroEvolution of Augmenting Topologies (NEAT) to generate a trading signal capable of achieving high returns and daily profits with low associated risk. The proposed approach is tested with real daily data from four financial markets of different sectors and with very different characteristics. Three different fitness functions are considered in the NEAT algorithm and the most robust results are produced by a fitness function that measures the mean daily profit obtained by the generated trading signal. The results achieved show that this approach outperforms the Buy and Hold (B&H) strategy in the markets tested (in the S&P 500 index this system achieves a rate of return of 18.89% while the B&H achieves 15.71% and in the Brent Crude futures contract this system achieves a rate of return of 37.91% while the B&H achieves −9.94%). Furthermore, it’s concluded that the PCA method is vital for the good performance of the proposed approach.

Introduction

When studying the stock market, determining the best time to buy or sell stocks is of key importance. However, obtaining a trading signal that finds good entry and exit points in the stock market is difficult due to the stock market’s noisy, non-stationary and non-linear characteristics. In fact, stock markets are affected by many highly interrelated factors, such as psychological variables associated with the investors as well as economic and political factors.

The efficient market hypothesis (Fama, 1970) states that it’s impossible to gain a competitive advantage over the market because the prices already incorporate and reflect all available information in the market at that moment. On the other hand, some researchers believe that the markets are inefficient, mainly due to the psychological variables associated with the various market participants and the inability of the markets to immediately respond to newly released information and so, there is an opportunity to beat the market and obtain above average returns by using stock market forecasting techniques (Gorgulho, Neves, & Horta, 2011).

In this paper, an approach combining a neuroevolution technique, the NeuroEvolution of Augmenting Topologies (NEAT), with a dimensionality reduction technique, the Principal Component Analysis (PCA), is presented. The PCA technique performs a linear mapping of the high dimensional space input data, consisting of raw financial data with some indicators obtained using technical analysis, to a lower dimensional space such that the variance of the data in the low dimensional representation is maximized. This dimensionality reduction of the data facilitates the identification of patterns by the artificial neural network (ANN) that is generated by NEAT. The neural network generated by NEAT outputs a trading signal that evolves with the help of an adapted genetic algorithm (GA) to a solution that maximizes the returns and daily profits while minimizing the associated risk.

The main contributions of this paper are: the combination of the data dimensionalty reduction technique PCA with the NEAT algorithm to identify the best trading points in the stock market; the use of fitness functions in NEAT that take in consideration not only the returns obtained but also the daily profits, risk and number of days spent with capital invested in the market; the integration of the hyper mutation technique in the NEAT algorithm that adjusts not only the overall mutation rate but also the probability of the mutation to add a new node to the ANN generated by NEAT; the possibility of the ANN originated by NEAT to have different activation functions in the hidden neurons if that helps the evolution process to achieve a better performing solution.

This paper is organized as follows: in Section 2 the related work is discussed. Section 3 presents the architecture and describes the implemented system. In Section 4 the case studies and results are presented and analyzed. Section 5 provides the conclusions obtained by the work developed.

Section snippets

Related work

Finding the best time to buy or sell has remained a difficult challenge as there are several factors that may impact stock prices (Chang & Liu, 2008). With the growth of the trading business, investors tried to find methods and tools to accurately predict the share prices, in order to increase their gains and minimize the risk (Khan, Alin, & Hussain, 2011). Methods like fundamental analysis, technical analysis and machine learning have all been used to attempt to find the best entry (buy) and

System architecture

This paper presents a trading system that uses the NEAT algorithm combined with the PCA technique to detect the best entry and exit points in the market, with the goal of maximizing the returns and daily profits while taking in consideration the risk and the number of days with capital invested in the market. As illustrated in Fig. 2, the system’s architecture is structured on a traditional layered architecture composed by three distinct layers: presentation layer, business logic layer and data

Results

The financial data used to train and test the implemented system are the daily volume and prices (open, high, low and adjusted close) of different markets over the period of 27/03/2006 to 13/04/2017, using 80% of the data to train the system and 20% to test it. To test the robustness of the system to different markets, the experiments are performed in the following markets: S&P 500 index, Brent Crude futures contract, Exxon Mobil Corporation stocks and Home Depot Inc. stocks. The S&P 500 index

Conclusions

This work proposes an approach combining the PCA method for dimensionality reduction with the NEAT algorithm for evolving an ANN with an adapted GA to generate a trading signal capable of achieving high returns and daily profit with low associated risk. This approach, with a fitness function in the NEAT algorithm measuring the mean daily profits, shows robust results over a wide variety of financial markets. One of the major conclusions that can be drawn from the results obtained is that the

Acknowledgement

This work was supported in part by Fundação para a Ciência e a Tecnologia (Project UID/EEA/50008/2013).

References (34)

  • P.-C. Chang et al.

    A TSK type fuzzy rule based system for stock price prediction

    Expert Systems with Applications

    (2008)
  • P.-C. Chang et al.

    A novel model by evolving partially connected neural network for stock price trend forecasting

    Expert Systems with Applications

    (2012)
  • P.-C. Chang et al.

    Evolving neural network for printed circuit board sales forecasting

    Expert Systems with Applications

    (2005)
  • Y.-T. Chang et al.

    Optimization the initial weights of artificial neural networks via genetic algorithm applied to hip bone fracture prediction

    Advances in Fuzzy Systems

    (2012)
  • Chervenski, P., & Ryan, S. (2012). MultiNEAT : Portable software library for performing neuroevolution....
  • W.-C. Chiang et al.

    An adaptive stock index trading decision support system

    Expert Systems with Applications

    (2016)
  • E. Fama

    Efficient capital markets: A review of theory and empirical work

    Journal of Finance

    (1970)
  • Fortier, M. (2007). TA-Lib : Technical analysis library. http://www.ta-lib.org/. [Online; accessed 20-April-2017...
  • A. Gorgulho et al.

    Applying a GA kernel on optimizing technical analysis rules for stock picking and portfolio composition

    Expert Systems with Applications

    (2011)
  • M. Göçken et al.

    Integrating metaheuristics and artificial neural networks for improved stock price prediction

    Expert Systems with Applications

    (2016)
  • Inthachot, M., Boonjing, V., & Intakosum, S. (2016). Artificial neural network and genetic algorithm hybrid...
  • K. Jadav et al.

    Optimizing weights of artificial neural networks using genetic algorithms

    International Journal of Advanced Research in Computer Science and Electronics Engineering (IJARCSEE)

    (2012)
  • I.T. Jolliffe

    Principal component analysis

    (2002)
  • Z.H. Khan et al.

    Price prediction of share market using artificial neural network (ann)

    International Journal of Computer Applications

    (2011)
  • C.D. Kirkpatrick et al.

    Technical analysis: The complete resource for financial market technicians

    (2015)
  • R. Kuo et al.

    A decision support system for order selection in electronic commerce based on fuzzy neural network supported by real-coded genetic algorithm

    Expert Systems with Applications

    (2004)
  • J. Mańdziuk et al.

    ”Dead” chromosomes and their elimination in the neuro-genetic stock index prediction system

    Proceedings of the 16th international conference on neural information processing: Part II

    (2009)
  • Cited by (40)

    • Online composition detection and cluster analysis of Tibetan incense

      2021, Optik
      Citation Excerpt :

      It is often used in exploratory data analysis and pattern recognition [23]. Since PCA method can visualize the best discrimination between substances, it is usually applied with some other techniques to make the problem easier to solve [24–26]. In this part, three common incenses were selected for online detection and analysis of their smoke when burning.

    • MHieR-encoder: Modelling the high-frequency changes across stocks

      2021, Knowledge-Based Systems
      Citation Excerpt :

      Li et al. [24] trained a generalized ANN framework that learns the RSI values of stocks in the Spanish IBEX 35 and gives the buying or selling signals of stocks. Elsewhere, Nadkarni and Neves [25] combined principal component analysis with stochastic time neural networks to predict financial indices such as CSI300 and S&P500. Some reinforce learning-based models are also applied in stock forecasting works.

    • Does short-term technical trading exist in the Vietnamese stock market?

      2021, Borsa Istanbul Review
      Citation Excerpt :

      In particular, Kim and Won (2018) give an example of a hybrid expert system that uses a machine-learning approach. Nadkarni and Neves (2018) demonstrate the importance of isolating the most important factors first in algorithmic trading. Some expert systems are rather focused on predicting the future direction of asset prices instead of the actual returns themselves (Brasileiro et al., 2017; Feuerriegel & Gordon, 2018; Jeong & Kim, 2019; Karhunen, 2019; Malagrino et al., 2018; Nam & Seong, 2019).

    View all citing articles on Scopus
    View full text