Ensemble approach based on bagging, boosting and stacking for short-term prediction in agribusiness time series

Highlights

• Ensembles and single models are compared for short-term forecasting in agribusiness.

• Soybean and wheat commodities are adopted as case studies.

• Boosting approaches showed lower predictions errors.

• XGB or STACK and RF models are adopted in soybean and wheat cases, respectively.

• Ensemble performance is better than SVR, KNN and MLP performance.

Abstract

The investigation of the accuracy of methods employed to forecast agricultural commodities prices is an important area of study. In this context, the development of effective models is necessary. Regression ensembles can be used for this purpose. An ensemble is a set of combined models which act together to forecast a response variable with lower error. Faced with this, the general contribution of this work is to explore the predictive capability of regression ensembles by comparing ensembles among themselves, as well as with approaches that consider a single model (reference models) in the agribusiness area to forecast prices one month ahead. In this aspect, monthly time series referring to the price paid to producers in the state of Parana, Brazil for a 60 kg bag of soybean (case study 1) and wheat (case study 2) are used. The ensembles bagging (random forests — RF), boosting (gradient boosting machine — GBM and extreme gradient boosting machine — XGB), and stacking (STACK) are adopted. The support vector machine for regression (SVR), multilayer perceptron neural network (MLP) and K-nearest neighbors (KNN) are adopted as reference models. Performance measures such as mean absolute percentage error (MAPE), root mean squared error (RMSE), mean absolute error (MAE), and mean squared error (MSE) are used for models comparison. Friedman and Wilcoxon signed rank tests are applied to evaluate the models’ absolute percentage errors (APE). From the comparison of test set results, MAPE lower than 1% is observed for the best ensemble approaches. In this context, the XGB/STACK (Least Absolute Shrinkage and Selection Operator-KNN-XGB-SVR) and RF models showed better performance for short-term forecasting tasks for case studies 1 and 2, respectively. Better APE (statistically smaller) is observed for XGB/STACK and RF in relation to reference models. Besides that, approaches based on boosting are consistent, providing good results in both case studies. Alongside, a rank according to the performances is: XGB, GBM, RF, STACK, MLP, SVR and KNN. It can be concluded that the ensemble approach presents statistically significant gains, reducing prediction errors for the price series studied. The use of ensembles is recommended to forecast agricultural commodities prices one month ahead, since a more assertive performance is observed, which allows to increase the accuracy of the constructed model and reduce decision-making risk.

Introduction

The operations inherent to agribusiness constitute an important network of activities that have a strong influence on the local, regional, and national economy. Generally, in times of crisis, agribusiness has been one of the areas that have contributed to the growth of different indicators related to the economy, such as the Gross Domestic Product (GDP). In view of the current Brazilian economic scenario, the GDP volume of agribusiness increased by 1.9%, facilitating national GDP growth and inflation control [1]. At the state level, according to recent reports from the Parana Institute for Economic and Social Development [2], the Parana state ended 2017 with a GDP of R$ (BRL) 415.8 billion, equivalent to 6.35% of the Brazilian economy. Regarding production, according to the Ministry of Agriculture, Livestock and Food Supply (MAPA) [3], the area planted with grains is expected to increase by 14.9% in the next ten years. Indeed, this is an important area for the development of the regions, and it also contributes to family income. Faced with this, it is important to know future price quotations, since this may affect the economic planning of small, medium, and large producers.

In this aspect, the forecast for time series of agricultural commodities prices plays an important role in the economic scenario. Also, the precision of the forecasts is as important as the expected results, and it is directly linked to the accuracy of the adopted model. Its importance permeates all areas of knowledge, as forecasts can be used as the basis for revision/implementation of public policies, development of strategic planning in companies, and decisions in the corporate world. In general, forecasting can be understood as a process of idealizing a more probable result, assuming a set of assumptions inherent in several aspects such as technology, environment, pricing, marketing, and production.

To improve the predictive capacity of regression models, among which we can highlight the time series models, several approaches have been proposed in the literature. One of these are regression ensembles [4], [5]. This principle is based on machine learning, whose essence is associated with the idea of divide-to-conquer and which seeks to overcome the limitation of a machine learning model that operates in isolation [6], [7]. Models characterized as ensembles refer to multiple base models (base learners, similar or not) developed to solve the same problem, each of which learns characteristics from data and accordingly makes a prediction. Hence, the final prediction is presented as a combination of the individual predictions. Through this approach we want to obtain a robust system that incorporates the prediction from all base learners. Among the basic ways to build ensembles for regression problems is the grouping by average and weighted average. In general, this approach achieves better results than single prediction models, since it has a better capacity to generalize and adapt in different scenarios [8]. The purpose of using regression ensembles is to generate a set of models to improve the prediction accuracy of the adopted models, whose target variables are numerical [9]. In this aspect, it is a methodology that has been used extensively over years and has been successful in different applications such as energy [10], climate [8], market forecasting [11], finance [12], [13], and chemistry [14].

From this perspective, this paper aims to explore the predictive capacity of regression ensembles by comparing ensembles among themselves, as well as with approaches that consider a single model (reference models), to forecast agricultural commodities prices one month ahead. The main motivation for the choice of this forecast horizon for the area of agribusiness is associated with the fact that short term forecasting allows small, medium and large producers to develop strategic planning for a short period of time to meet their instantaneous necessities. This paper conducts two case studies in which historical data of monthly prices (R$) paid to the producers in the state of Parana, Brazil, are considered for 60 kg bags of soybean (case study 1) and wheat (case study 2). The specific objectives are stated as follows: (i) Identify, by related works, the main variables that act as price drivers in the soybean and wheat cases. In addition, extract the importance of them for each case study; (ii) Present a theoretical revision of the regression ensembles based on bagging (random forests — RF), boosting (gradient boosting machine — GBM and extreme gradient boosting machine — XGB) and stacking (STACK) and (iii) verify if the ensemble approaches outperform single models (K-Nearest Neighbor — KNN, Multilayer Perceptron Neural Network — MLP and Support Vector Regression with linear Kernel— SVR), as well as which structure is better in the adopted cases.

The main contribution of this paper to the soft computing and machine learning literature lies in the application, evaluation and addition of discussions not previously developed regarding the behavior of GBM, RF, STACK and XGB to forecast (in the short-term) agricultural commodities prices, with the use of the price drivers identified in Section 3.2. While the papers identified in Section 2 focus on the study of price volatility by hybrid models, as well as combining decomposition, optimization and machine learning models, this paper proposes the investigation of the performance of bagging, boosting and stacking ensemble approaches in the presence of different features (price drivers) regarding short-term forecasting for the soybean and wheat commodities.

Through these contributions, it is expected that once a few time series models using regression ensembles to predict the agribusiness theme are introduced in the literature, this work will fill a gap in an area of the economy essential to the composition of economic indicators. Seeking to illustrate the superior performance of the ensemble models vis-à-vis reference approaches to forecasting in time series associated with agribusiness, it is expected that this study will stimulate the interest of financial market researchers to use this approach, since it performs well and enables managers to minimize their losses and leverage their earnings.

The remainder of this paper is structured as follows: Section 2 presents a set of related works on the subject of regression ensembles for commodities forecasting. Section 3 describes the data sets used in this paper, as well as the predictors employed in the modeling process. Section 4 introduces the theoretical aspects necessary for the development of this paper. Section 5 presents the adopted methodology. Section 6 details the results and discussions. Finally, Section 7 concludes the paper with general considerations and directions for future research.