Skip to main content
SpringerLink
Log in

A hybrid model of dynamic time wrapping and hidden Markov model for forecasting and trading in crude oil market

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

In this study, a hybrid model of hidden Markov model (HMM) and dynamic time wrapping (DTW) is proposed to predict the return of crude oil price movements and trading. First, three indicators are used as inputs of HMM to determine the market state for each month; next, DTW algorithm is applied to match similar price sequences which have the same market state in historical time series, and then to calculate expected returns; Finally, it forecasts the crude oil spot price direction and executes related simulation trading. For design of the trading strategy, we adopt different parameters such as trading thresholds and position-closing thresholds for each market state, and the particle swarm optimization algorithm is applied for parameter optimization of our trading strategy. In experiments, the proposed method is applied for direction forecasting and simulation trading of WTI and Brent crude oil market. Experimental results show that the proposed method yielded the best forecasting and trading performances in average. For instance, in the WTI market, the proposed method produced a hit ratio of about 62.74% and a yield of 34.3% profit per year, and a Sharpe ratio value of 2.274. Furthermore, experimental results of the proposed method were significantly superior to other benchmark methods, demonstrating that the proposed method is not only good at direction prediction and profit making, but also return/risk ratio.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Abbreviations

DTW:

Dynamic time wrapping

HMM:

Hidden Markov model

PSO:

Particle swarm optimization

SVMs:

Support vector machines

ANNs:

Artificial neural networks

ARIMA:

Autoregressive integrated moving average

DP:

Data pre-processing

PTE:

Prediction, trading and evaluation

TL:

Threshold for long

TS:

Threshold for short

MA:

Moving average

PTT:

Profit-taking threshold

LCT:

Loss-cutting threshold

RR:

Return rate

AR:

Accumulated return

SR:

Sharpe ratio

BAH:

Buy and hold

SAH:

Sell and hold

References

  • Ahmed RA, Shabri AB (2014) Daily crude oil price forecasting model using arima, generalized autoregressive conditional heteroscedastic and support vector machines. Am J Appl Sci 11:425–432

    Article  Google Scholar 

  • Alvarez RJ, Cisneros M, Ibarra VC (2012) Multifractal Hurst analysis of crude oil prices. Phys A 313(3):651–670

    MATH  Google Scholar 

  • Ao SI (2011) A hybrid neural network cybernetic system for quantifying cross-market dynamics and business forecasting. Soft Comput 15:1041–1053

    Article  Google Scholar 

  • Bao Y, Zhang X, Yu L, Lai KK, Wang S (2007) Hybridizing wavelet and least squares support vector machines for crude oil price forecasting. In: Proceedings of the 2nd international workshop on intelligent finance, pp 1–15

  • Baum LE, Petrie T (1966) Statistical inference for probabilistic functions of finite state Markov Chains. Ann Math Stat 37:1554–1563

    Article  MathSciNet  Google Scholar 

  • Bon AT, Isah N (2016) Hidden markov model and forward-backward algorithm in crude oil price forecasting. In IOP conference series: materials science and engineering. vol 160. IOP Publishing, p 012067

  • Box GEP, Jenkins GM (1994) Time series analysis: forecasting and control. Prentice Hall, EnglewoodCliffs

    MATH  Google Scholar 

  • Cao DZ, Pang SL, Bai YH (2005) Forecasting exchange rate using support vector machines. In: International conference on machine learning & cybernetics 2005, vol 6. IEEE, pp 3448–3452

  • Chang PC, Fan CY, Lin JL (2008) Integrating a piecewise linear representation method with dynamic time warping system for stock trading decision making. In: 2008 4th international conference on natural computation, vol 2. IEEE, pp 434–438

  • Chen Y, Zhang C, He K, Zheng A (2018) Multi-step-ahead crude oil price forecasting using a hybrid grey wave model. Phys A 501:98–110

    Article  Google Scholar 

  • Chiroma H, Abdul-Kareem S, Abubakar A, Zeki AM, Usman MJ (2014) Orthogonal wavelet support vector machine for predicting crude oil prices. In: Proceedings of the 1st international conference on advanced data and information engineering, pp 193–201

  • Chiroma H, Abdul-Kareem S, Herawan T (2015a) Evolutionary neural network model for west texas intermediate crude oil price prediction. Appl Energy 142:266–273

    Article  Google Scholar 

  • Chiroma H, Abdul-Kareem S, Noor ASM, Abubakar AI, Safa NS, Shuib L et al (2015b) A review on artificial intelligence methodologies for the forecasting of crude oil price. Intell Autom Soft Comput 22(3):449–462

    Article  Google Scholar 

  • Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1:3–18

    Article  Google Scholar 

  • Ding Y (2018) A novel decompose-ensemble methodology with AIC-ANN approach for crude oil forecasting. Energy 154:328–336

    Article  Google Scholar 

  • Donninger C (2017) Trading bull- and bear-markets with a hidden markov model. Working paper. SSRN Electron J

  • Etuk EH (2013) Seasonal Arima modelling of Nigerian monthly crude oil prices. Asian Econ Financ Rev 3:333–340

    Google Scholar 

  • Galib AA, Alam M (2014) Rahman R M. Prediction of stock price based on hidden Markov model and nearest neighbour algorithm. Int J Inf Decis Sci 6:262–292

    Google Scholar 

  • Guo Z, Wang H, Quan L (2013) Financial time series forecasting using LPP and SVM optimized by PSO. Soft Comput 17:805–818

    Article  Google Scholar 

  • Hagiwara K, Fukumizu K (2008) Relation between weight size and degree of over-fitting in neural network regression. Neural Netw 21:48–58

    Article  Google Scholar 

  • Hajizadeh E, Mahootchi M, Esfahanipour A, Kh MM (2019) A new NN-PSO hybrid model for forecasting Euro/Dollar exchange rate volatility. Neural Comput Appl 31(7):2063–2071

    Article  Google Scholar 

  • Hassan M (2009) A combination of hidden markov model and fuzzy model for stock market forecasting. Neurocomputing 72(16–18):3439–3446

    Article  Google Scholar 

  • Hassan M, Nath B (2005) Stock market forecasting using hidden Markov model: a new approach. In: 5th international conference on intelligent systems design and applications (ISDA’05). IEEE, pp 192–196

  • Hassan M, Nath B, Kirley M (2007) A fusion model of hmm, ann and ga for stock market forecasting. Expert Syst Appl 33(1):171–180

    Article  Google Scholar 

  • He K, Yu L, Lai KK (2012) Crude oil price analysis and forecasting using wavelet decomposed ensemble model. Energy 46:564–574

    Article  Google Scholar 

  • Kecskes I, Szekacs L, Fodor JC, Odry P (2013) PSO and GA optimization methods comparison on simulation model of a real hexapod robot. In 2013 IEEE 9th international conference on computational cybernetics (ICCC). IEEE, pp 125–130

  • Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of IEEE international conference on neural networks, pp 1942–1948

  • Latif M, Herawati S (2016) The application of EEMD and neural network based on Polak-Ribiére conjugate gradient algorithm for crude oil prices forecasting. MATEC Web of Conferences, EDP Sciences 58:03013

    Article  Google Scholar 

  • Lee SJ, Jeong SJ (2012) Trading strategies based on pattern recognition in stock futures market using dynamic time warping algorithm. Journal of Convergence Information Technology. 7:185–196

    Google Scholar 

  • Lima CAS Jr, Lapa CMF, do NA Pereira CM, da Cunha JJ, Alvim ACM (2011) Comparison of computational performance of GA and PSO optimization techniques when designing similar systems–Typical PWR core case. Ann Nucl Energy 38(6):1339–1346

    Article  Google Scholar 

  • Liu D, Wei Y, Yang S et al (2013) Electricity price forecast using combined models with adaptive weights selected and errors calibrated by hidden Markov model. Math Probl Eng 3:1–8

    Google Scholar 

  • Lu CJ (2013) Hybridizing nonlinear independent component analysis and support vector; regression with particle swarm optimization for stock index forecasting. Neural Comput Appl 23:2417–2427

    Article  Google Scholar 

  • Niu H, Wang J (2014) Financial time series prediction by a random data-time effective RBF neural network. Soft Comput 18:497–508

    Article  Google Scholar 

  • Ou C, Lin W (2006) Comparison between PSO and GA for parameters optimization of PID controller. In: 2006 International conference on mechatronics and automation. IEEE, pp 2471–2475

  • Park SH, Lee JH, Lee HC (2011) Trend forecasting of financial time series using PIPs detection and continuous HMM. IOS Press, Amsterdam

    Book  Google Scholar 

  • Rabiner LR (1989) A tutorial on hidden markov models and selected applications in speech recognition. Proc IEEE 77:257–286

    Article  Google Scholar 

  • Sharpe WF (1994) The sharpe ratio. J Portfolio Manag 21:49–58

    Article  Google Scholar 

  • Suriya K (2016) Forecasting crude oil price using neural networks. Nonlinear Dyn 44:341–349

    Google Scholar 

  • Wang J, Li X (2018) A combined neural network model for commodity price forecasting with SSA. Soft Comput 22(16):5323–5333

    Article  Google Scholar 

  • Wang B, Wu Z, Zhao Z (2010) Performance comparison of GA, PSO, and DE approaches in estimating low atmospheric refractivity profiles. Wuhan Univ J Nat Sci 15(5):433–439

    Article  Google Scholar 

  • Wang GJ, Xie C, Han F (2012) Similarity measure and topology evolution of foreign exchange markets using dynamic time warping method: evidence from minimal spanning tree. Phys A 391:4136–4146

    Article  Google Scholar 

  • Wang M, Zhao L, Du R, Wang C, Chen L, Tian L, Stanley HE (2018) A novel hybrid method of forecasting crude oil prices using complex network science and artificial intelligence algorithms. Appl Energy 220:480–495

    Article  Google Scholar 

  • Website soruce 1. U.S. Energy information administration website. https://www.eia.gov/. Accessed 28 Dec 2018

  • Website source 2. U.S. Department of the Treasury Website. https://www.treasury.gov/. Accessed 28 Dec 2018

  • Xiao Y, Xiao J, Lu F (2014) Ensemble ANNs-PSO-GA approach for day-ahead stock E-exchange prices forecasting. Int J Comput Int Syst 7:272–290

    Article  Google Scholar 

  • Xie W, Yu L, Xu S, Wang S (2006) A new method for crude oil price forecasting based on support vector machines. In: International conference on computational science. Springer, pp 444–451

  • Yang Y (2010) Crude oil price prediction based on empirical model decomposition and support vector machines. Chin J Manag 7:1884–1889

    Google Scholar 

  • Yin X, Zhao J (2015) A hidden markov model approach to information-based trading: theory and applications. J Appl Econom 30:1210–1234

    Article  MathSciNet  Google Scholar 

  • Yu L, Lai KK, Wang S, He K (2007) Oil price forecasting with an EMD-based multiscale neural network learning paradigm. In: International conference on computational science. Springer, pp 925–932

  • Zhang Y (2004) Prediction of financial time series with hidden Markov models. Doctoral dissertation, Applied Sciences: School of Computing Science

  • Zhang YJ, Wei YM (2011) The dynamic influence of advanced stock market risk on international crude oil returns: an empirical analysis. Quant Financ 11:967–978

    Article  MathSciNet  Google Scholar 

  • Zhang L, Wang F, Xu B, Chi W, Wang Q, Sun T (2018) Prediction of stock prices based on LM-BP neural network and the estimation of overfitting point by RDCI. Neural Comput Appl 30:1425–1444

    Article  Google Scholar 

  • Zheng JW, Li SX, Kun Y (2014) A new hybrid model for forecasting crude oil price and the techniques in the model. Adv Mater Res 974:310–317

    Article  Google Scholar 

  • Zhiqiang G, Huaiqing W, Quan L (2013) Financial time series forecasting using LPP and SVM optimized by PSO. Soft Comput 17(5):805–818

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially funded by Hubei Provincial Department of Education (No. Q20171208), Science Foundation of China Three Gorges University (No. KJ2016A001) and Starting Grant of China Three Gorges University (No. 20170907). In addition, the authors thank the editors and three anonymous reviewers for their helpful comments for improving the quality of the paper.

Author information

Authors and Affiliations

  1. College of Economics and Management, China Three Gorges University, Yichang, 443002, Hubei Province, China

    Shangkun Deng, Youtao Xiang, Boyang Nan & Hongyu Tian

  2. Financial Research Institute, College of Economics and Management, China Three Gorges University, Yichang, 443002, Hubei Province, China

    Shangkun Deng & Hongyu Tian

  3. Laboratory for Advanced Brain Signal Processing, RIKEN, 2-1 Hirosawa, Wako, Saitama Prefecture, 351-0198, Japan

    Zhe Sun

Authors
  1. Shangkun Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Youtao Xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Boyang Nan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hongyu Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhe Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shangkun Deng.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Communicated by V. Loia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deng, S., Xiang, Y., Nan, B. et al. A hybrid model of dynamic time wrapping and hidden Markov model for forecasting and trading in crude oil market. Soft Comput 24, 6655–6672 (2020). https://doi.org/10.1007/s00500-019-04304-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-019-04304-9

Keywords

Search

Navigation