Combining NeuroEvolution and Principal Component Analysis to trade in the financial markets
Introduction
When studying the stock market, determining the best time to buy or sell stocks is of key importance. However, obtaining a trading signal that finds good entry and exit points in the stock market is difficult due to the stock market’s noisy, non-stationary and non-linear characteristics. In fact, stock markets are affected by many highly interrelated factors, such as psychological variables associated with the investors as well as economic and political factors.
The efficient market hypothesis (Fama, 1970) states that it’s impossible to gain a competitive advantage over the market because the prices already incorporate and reflect all available information in the market at that moment. On the other hand, some researchers believe that the markets are inefficient, mainly due to the psychological variables associated with the various market participants and the inability of the markets to immediately respond to newly released information and so, there is an opportunity to beat the market and obtain above average returns by using stock market forecasting techniques (Gorgulho, Neves, & Horta, 2011).
In this paper, an approach combining a neuroevolution technique, the NeuroEvolution of Augmenting Topologies (NEAT), with a dimensionality reduction technique, the Principal Component Analysis (PCA), is presented. The PCA technique performs a linear mapping of the high dimensional space input data, consisting of raw financial data with some indicators obtained using technical analysis, to a lower dimensional space such that the variance of the data in the low dimensional representation is maximized. This dimensionality reduction of the data facilitates the identification of patterns by the artificial neural network (ANN) that is generated by NEAT. The neural network generated by NEAT outputs a trading signal that evolves with the help of an adapted genetic algorithm (GA) to a solution that maximizes the returns and daily profits while minimizing the associated risk.
The main contributions of this paper are: the combination of the data dimensionalty reduction technique PCA with the NEAT algorithm to identify the best trading points in the stock market; the use of fitness functions in NEAT that take in consideration not only the returns obtained but also the daily profits, risk and number of days spent with capital invested in the market; the integration of the hyper mutation technique in the NEAT algorithm that adjusts not only the overall mutation rate but also the probability of the mutation to add a new node to the ANN generated by NEAT; the possibility of the ANN originated by NEAT to have different activation functions in the hidden neurons if that helps the evolution process to achieve a better performing solution.
This paper is organized as follows: in Section 2 the related work is discussed. Section 3 presents the architecture and describes the implemented system. In Section 4 the case studies and results are presented and analyzed. Section 5 provides the conclusions obtained by the work developed.
Section snippets
Related work
Finding the best time to buy or sell has remained a difficult challenge as there are several factors that may impact stock prices (Chang & Liu, 2008). With the growth of the trading business, investors tried to find methods and tools to accurately predict the share prices, in order to increase their gains and minimize the risk (Khan, Alin, & Hussain, 2011). Methods like fundamental analysis, technical analysis and machine learning have all been used to attempt to find the best entry (buy) and
System architecture
This paper presents a trading system that uses the NEAT algorithm combined with the PCA technique to detect the best entry and exit points in the market, with the goal of maximizing the returns and daily profits while taking in consideration the risk and the number of days with capital invested in the market. As illustrated in Fig. 2, the system’s architecture is structured on a traditional layered architecture composed by three distinct layers: presentation layer, business logic layer and data
Results
The financial data used to train and test the implemented system are the daily volume and prices (open, high, low and adjusted close) of different markets over the period of 27/03/2006 to 13/04/2017, using 80% of the data to train the system and 20% to test it. To test the robustness of the system to different markets, the experiments are performed in the following markets: S&P 500 index, Brent Crude futures contract, Exxon Mobil Corporation stocks and Home Depot Inc. stocks. The S&P 500 index
Conclusions
This work proposes an approach combining the PCA method for dimensionality reduction with the NEAT algorithm for evolving an ANN with an adapted GA to generate a trading signal capable of achieving high returns and daily profit with low associated risk. This approach, with a fitness function in the NEAT algorithm measuring the mean daily profits, shows robust results over a wide variety of financial markets. One of the major conclusions that can be drawn from the results obtained is that the
Acknowledgement
This work was supported in part by Fundação para a Ciência e a Tecnologia (Project UID/EEA/50008/2013).
References (34)
- et al.
A TSK type fuzzy rule based system for stock price prediction
Expert Systems with Applications
(2008) - et al.
A novel model by evolving partially connected neural network for stock price trend forecasting
Expert Systems with Applications
(2012) - et al.
Evolving neural network for printed circuit board sales forecasting
Expert Systems with Applications
(2005) - et al.
Optimization the initial weights of artificial neural networks via genetic algorithm applied to hip bone fracture prediction
Advances in Fuzzy Systems
(2012) - Chervenski, P., & Ryan, S. (2012). MultiNEAT : Portable software library for performing neuroevolution....
- et al.
An adaptive stock index trading decision support system
Expert Systems with Applications
(2016) Efficient capital markets: A review of theory and empirical work
Journal of Finance
(1970)- Fortier, M. (2007). TA-Lib : Technical analysis library. http://www.ta-lib.org/. [Online; accessed 20-April-2017...
- et al.
Applying a GA kernel on optimizing technical analysis rules for stock picking and portfolio composition
Expert Systems with Applications
(2011) - et al.
Integrating metaheuristics and artificial neural networks for improved stock price prediction
Expert Systems with Applications
(2016)
Optimizing weights of artificial neural networks using genetic algorithms
International Journal of Advanced Research in Computer Science and Electronics Engineering (IJARCSEE)
Principal component analysis
Price prediction of share market using artificial neural network (ann)
International Journal of Computer Applications
Technical analysis: The complete resource for financial market technicians
A decision support system for order selection in electronic commerce based on fuzzy neural network supported by real-coded genetic algorithm
Expert Systems with Applications
”Dead” chromosomes and their elimination in the neuro-genetic stock index prediction system
Proceedings of the 16th international conference on neural information processing: Part II
Cited by (40)
Neuroevolved bi-directional LSTM applied to zero emission zones management in urban transport
2023, Applied Soft ComputingA new oversampling method and improved radial basis function classifier for customer consumption behavior prediction
2022, Expert Systems with ApplicationsOnline composition detection and cluster analysis of Tibetan incense
2021, OptikCitation Excerpt :It is often used in exploratory data analysis and pattern recognition [23]. Since PCA method can visualize the best discrimination between substances, it is usually applied with some other techniques to make the problem easier to solve [24–26]. In this part, three common incenses were selected for online detection and analysis of their smoke when burning.
MHieR-encoder: Modelling the high-frequency changes across stocks
2021, Knowledge-Based SystemsCitation Excerpt :Li et al. [24] trained a generalized ANN framework that learns the RSI values of stocks in the Spanish IBEX 35 and gives the buying or selling signals of stocks. Elsewhere, Nadkarni and Neves [25] combined principal component analysis with stochastic time neural networks to predict financial indices such as CSI300 and S&P500. Some reinforce learning-based models are also applied in stock forecasting works.
Does short-term technical trading exist in the Vietnamese stock market?
2021, Borsa Istanbul ReviewCitation Excerpt :In particular, Kim and Won (2018) give an example of a hybrid expert system that uses a machine-learning approach. Nadkarni and Neves (2018) demonstrate the importance of isolating the most important factors first in algorithmic trading. Some expert systems are rather focused on predicting the future direction of asset prices instead of the actual returns themselves (Brasileiro et al., 2017; Feuerriegel & Gordon, 2018; Jeong & Kim, 2019; Karhunen, 2019; Malagrino et al., 2018; Nam & Seong, 2019).