ABSTRACT
Financial portfolio management is the process of periodically reallocating a fund into different financial investment products, with the goal of achieving the maximum profits. While conventional financial machine learning methods try to predict the price trends, reinforcement learning based portfolio management methods makes trading decisions according to the price changes directly. However, existing reinforcement learning based methods are limited in extracting the price change information at single-scale level, which makes their performance still not satisfactory. In this paper, inspired by the Inception network that has achieved great success in computer vision and can extract multi-scale features simultaneously, we propose a novel Ensemble of Identical Independent Inception (EI$^3$) convolutional neural network, with the objective of addressing the limitation of existing reinforcement learning based portfolio management methods. With EI$^3$, multiple assets can be processed independently while sharing the same network parameters. Moreover, price movement information for each product can be extracted at multiple scales via wide network and then aggregated to make trading decision. Based on EI$^3$, we further propose a recurrent reinforcement learning framework to provide a deep machine learning solution for the portfolio management problem. Comprehensive experiments on the cryptocurrency datasets demonstrate the superiority of our method over existing competitors, in both upswing and downswing environments.
- Amit Agarwal, Elad Hazan, Satyen Kale, and Robert E. Schapire. 2006. Algorithms for portfolio management based on the Newton method. In Proceedings of ICML . 9--16.Google Scholar
- Saud Almahdi and Steve Y. Yang. 2017. An adaptive portfolio trading system: A risk-return portfolio optimization using recurrent reinforcement learning with expected maximum drawdown. Expert Systems with Applications , Vol. 87 (2017), 267--279.Google ScholarCross Ref
- Gerald Appel. 2005. Technical analysis: power tools for active investors .Google Scholar
- Adebiyi Ariyo Ariyo, Aderemi Oluyinka Adewumi, and Charles K. Ayo. 2014. Stock Price Prediction Using the ARIMA Model. In Proceedings of UKSim. 106--112.Google Scholar
- Stelios D Bekiros. 2010. Heterogeneous trading strategies with adaptive fuzzy actor--critic reinforcement learning: A behavioral approach. Journal of Economic Dynamics and Control , Vol. 34, 6 (2010), 1153--1170.Google ScholarCross Ref
- Allan Borodin, Ran El-Yaniv, and Vincent Gogan. 2000. On the Competitive Theory and Practice of Portfolio Selection (Extended Abstract). In Proceedings of LATIN (Lecture Notes in Computer Science), Vol. 1776. 173--196.Google ScholarCross Ref
- Allan Borodin, Ran El-Yaniv, and Vincent Gogan. 2003. Can We Learn to Beat the Best Stock. In Proceedings of NIPS. 345--352.Google Scholar
- Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. 2018. Recurrent neural networks for multivariate time series with missing values. Scientific Reports , Vol. 8, 1 (2018), 1--12.Google ScholarCross Ref
- Kyunghyun Cho, Bart van Merrienboer, cC aglar Gü lcc ehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In Proceedings of EMNLP. 1724--1734.Google Scholar
- Marco Corazza and Francesco Bertoluzzo. 2014. Q-learning-based financial trading systems with applications. University Ca'Foscari of Venice, Dept. of Economics Working Paper Series , Vol. 15 (2014), 1--23.Google Scholar
- Thomas M. Cover. 1991. Universal Portfolios. Mathematical Finance , Vol. 1, 1 (1991), 1--29.Google ScholarCross Ref
- Yue Deng, Feng Bao, Youyong Kong, Zhiquan Ren, and Qionghai Dai. 2017. Deep Direct Reinforcement Learning for Financial Signal Representation and Trading. IEEE Transactions on Neural Networks and Learning Systems , Vol. 28, 3 (2017), 653--664.Google ScholarCross Ref
- Thomas G Fischer. 2018. Reinforcement learning in financial markets - a survey . Technical Report. FAU Discussion Papers in Economics.Google Scholar
- Li Gao and Weiguo Zhang. 2013. Weighted moving average passive aggressive algorithm for online portfolio selection. In Proceedings of IHMSC, Vol. 1. 327--330.Google Scholar
- Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proc. of International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research). 249--256.Google Scholar
- László Györfi, Gábor Lugosi, and Frederic Udina. 2006. Nonparametric kernel-based sequential investment strategies. Mathematical Finance: An International Journal of Mathematics, Statistics and Financial Economics , Vol. 16, 2 (2006), 337--357.Google ScholarCross Ref
- David P. Helmbold, Robert E. Schapire, Yoram Singer, and Manfred K. Warmuth. 1996. On-Line Portfolio Selection Using Multiplicative Updates. In Proceedings of ICML . 243--251.Google Scholar
- Sepp Hochreiter and Jü rgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation , Vol. 9, 8 (1997), 1735--1780.Google ScholarDigital Library
- Dingjiang Huang, Junlong Zhou, Bin Li, Steven C. H. Hoi, and Shuigeng Zhou. 2016. Robust Median Reversion Strategy for Online Portfolio Selection. IEEE Transactions on Knowledge and Data Engineering , Vol. 28, 9 (2016), 2480--2493.Google ScholarDigital Library
- Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. 2017. Densely Connected Convolutional Networks. In Proceedings of CVPR. 2261--2269.Google Scholar
- Zhengyao Jiang and Jinjun Liang. 2017. Cryptocurrency portfolio management with deep reinforcement learning. In Proceedings of IntelliSys. 905--913.Google ScholarCross Ref
- Zhengyao Jiang, Dixing Xu, and Jinjun Liang. 2017. A deep reinforcement learning framework for the financial portfolio management problem. arXiv preprint arXiv:1706.10059 (2017).Google Scholar
- Bin Li and Steven CH Hoi. 2012. On-line portfolio selection with moving average reversion. In Proceedings of ICML . 563--570.Google Scholar
- Bin Li, Steven C.H. Hoi, and Vivekanand Gopalkrishnan. 2011. CORN: Correlation-driven Nonparametric Learning Approach for Portfolio Selection. ACM Transactions on Intelligent Systems and Technology , Vol. 2, 3 (2011), 21:1--21:29.Google ScholarDigital Library
- Bin Li and Steven C. H. Hoi. 2014. Online portfolio selection: A survey. Comput. Surveys , Vol. 46, 3 (2014), 35:1--35:36.Google ScholarDigital Library
- Bin Li, Peilin Zhao, Steven C. H. Hoi, and Vivekanand Gopalkrishnan. 2012. PAMR: Passive aggressive mean reversion strategy for portfolio selection. Machine Learning , Vol. 87, 2 (2012), 221--258.Google ScholarDigital Library
- Malik Magdon-Ismail, Amir F Atiya, Amrit Pratap, and Yaser S Abu-Mostafa. 2004. On the maximum drawdown of a Brownian motion. Journal of Applied Probability , Vol. 41, 1 (2004), 147--161.Google ScholarCross Ref
- Harry Markowitz. 1952. Portfolio selection. The Journal of Finance , Vol. 7, 1 (1952), 77--91.Google Scholar
- John Moody, Lizhong Wu, Yuansong Liao, and Matthew Saffell. 1998. Performance functions and reinforcement learning for trading systems and portfolios. Journal of Forecasting , Vol. 17, 5--6 (1998), 441--470.Google ScholarCross Ref
- Gordon Ritter. 2018. Reinforcement Learning in Finance. Big Data and Machine Learning in Quantitative Investment (2018), 225--250.Google Scholar
- David E Rumelhart, Geoffrey E Hinton, Ronald J Williams, and others. 1988. Learning representations by back-propagating errors. Cognitive Modeling , Vol. 5, 3 (1988), 533--536.Google Scholar
- William F Sharpe. 1994. The sharpe ratio. Journal of portfolio management , Vol. 21, 1 (1994), 49--58.Google ScholarCross Ref
- David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin A. Riedmiller. 2014. Deterministic Policy Gradient Algorithms. In Proceedings of ICML. 387--395.Google Scholar
- Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In Proceedings of ICLR . 1--14.Google Scholar
- Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of CVPR. 1--9.Google ScholarCross Ref
- Dat Thanh Tran, Alexandros Iosifidis, Juho Kanniainen, and Moncef Gabbouj. 2019. Temporal Attention-Augmented Bilinear Network for Financial Time-Series Data Analysis. IEEE Trans. on Neural Networks and Learning Systems , Vol. 30, 5 (2019), 1407--1418.Google ScholarCross Ref
Index Terms
- A Multi-Scale Temporal Feature Aggregation Convolutional Neural Network for Portfolio Management
Recommendations
Reinforcement Learning with Maskable Stock Representation for Portfolio Management in Customizable Stock Pools
WWW '24: Proceedings of the ACM on Web Conference 2024Portfolio management (PM) is a fundamental financial trading task, which explores the optimal periodical reallocation of capitals into different stocks to pursue long-term profits. Reinforcement learning (RL) has recently shown its potential to train ...
Margin Trader: A Reinforcement Learning Framework for Portfolio Management with Margin and Constraints
ICAIF '23: Proceedings of the Fourth ACM International Conference on AI in FinanceIn the field of portfolio management using reinforcement learning, existing approaches have mainly focused on cash-only trading, overlooking the potential benefits and risks of margin trading. Incorporating margin accounts and their constraints, ...
Portfolio management system in equity market neutral using reinforcement learning
AbstractPortfolio management involves position sizing and resource allocation. Traditional and generic portfolio strategies require forecasting of future stock prices as model inputs, which is not a trivial task since those values are difficult to obtain ...
Comments