Prediction of cryptocurrency’s price using ensemble machine learning algorithms

A cryptocurrency is a cryptographically secured digital currency distributed on a decentralized network using blockchain technology, making them anonymous and untraceable. The advent of cryptocurrencies and blockchain technology has sparked a revolutionary shift in the financial sector (Kayani, 2023; Kayani and Hasan, 2024).

Cryptocurrencies offer substantial returns on investments besides possessing a higher risk factor. A classic example would be the Bitcoin index (in EUR), which stands at a 108.27% compound annual growth rate and a standard deviation of 156.99% between 2011 and 2024. Figure 1 shows the abnormal spike in Bitcoin return during 2021, with significantly higher fluctuations indicating heightened volatility. This calls for accurate forecasting models to combat the large fluctuations in non-stationary cryptocurrencies (Bouteska et al., 2024). Prediction of cryptocurrency prices is also important for several other reasons, such as building trading strategies, risk management, price discovery, market sentiment analysis and business applications.

Fundamentalists and chartists rely on price prediction to make an informed decision. Chartists looking for price patterns and trends can refine their technical analysis based on forecasting. While fundamentalist’s primary focus remains on long-term value assessment, the model’s predictions could potentially serve as an additional data point for considering short-term market sentiment that might influence their investment decisions (Soltani et al., 2023).

The early-stage researchers employed statistical and econometrics models to forecast the price of cryptocurrencies. To predict Bitcoin’s short-term prices, the autoregressive integrated moving average (ARIMA) model was applied by Wirawan et al. (2019). The progressive growth in machine learning (ML) also motivated researchers to compare the forecasting accuracy of various models. Lyu (2022) compared the accuracy of various ML models and found gradient boosting (GB) to be the most efficient in predicting most major cryptocurrencies.

Recent studies on forecasting started applying deep learning (DL) and hybrid models, combining classical models such as the ARIMA model and artificial neural networks (Suhartono et al., 2017). Li et al. (2022) found that the hybrid model improved the forecasting accuracy of foreign currency and demonstrated that the hybrid data decomposition model outperforms econometric, ML and deep-learning models. Chen (2023) predicted the Bitcoin price of the next day and found that random forest (RF) gives better accuracy than long short-term memory (LSTM). An extensive literature review revealed limited studies using ensemble methods to predict the prices of multiple cryptocurrencies and hyper-tune the parameters to improve forecasting accuracy (Derbentseva et al., 2021).

Furthermore, the existing literature predominantly focuses mostly on Bitcoin. Empirical research is necessary to test the forecasting models in a broader range of cryptocurrencies that display diverse price dynamics and volatility, which have evolved over the past few years (Piryonesi and El-Diraby, 2020). ML has the power to identify complex patterns in price movements and overcome the limitations of traditional models to improve forecasting accuracy. In contrast, traditional models like ARIMA may exhibit reliability in some cases (Nakano et al., 2018). Ensemble model combine the output from multiple models, hyper-tune the parameters and improve the forecasting accuracy. A systematic, three-stage approach is followed to create an ensemble model for predicting cryptocurrency prices. First, a dataset of historical prices of the 15 cryptocurrencies selected based on their market capitalization. Then, the base models, random forest regressor (RFR), gradient boosting regressor (GBR), support vector regression (SVR) and MLP regressor, are trained on the training set, optimizing their hyper-parameters. With the base models trained, predictions on the test set are made using each model. Finally, these individual predictions are combined using stacking to create the ensemble model. Root mean squared error (RMSE), mean squared error (MSE), mean absolute error (MAE) and R2 are used to evaluate the ensemble model’s performance on the 15 cryptocurrencies.

The study addresses the critical gaps in the existing literature. First, the existing studies have explored the statistical or ML models in silos. The traditional statistical models fail to capture the non-linear and rapid price movement of cryptocurrencies. Further it carries limitations such as hyperparameter tuning and lacks computational efficiency. The standalone ML models suffer from issues such as overfitting or underfitting adversely affecting the forecasting accuracy. This study bridges the gap by applying a robust ensemble machine leveraging the potential of ML to improve forecasting accuracy. Further the existing studies on forecasting, focus mostly on a single cryptocurrency, Bitcoin in general and few others like Ethereum. The broader cryptocurrency market remains largely unresearched. The study provides a methodological framework to empirically test the forecasting accuracy of ensemble model on a broader set of cryptocurrencies. Ensemble models are flexible in capturing non-linear relationships, complex patterns in the data and provide superior forecasting performance as compared to the standalone models. This paper contributes to the existing theories on predictive analytics, providing empirical evidence by creating an ensemble ML model. In addition, it contributes to the body of knowledge on how to leverage bias-variance trade-offs in forecasting. To fill the research gap, the study aims to address the following questions: (1) How to develop an ensemble model to accurately predict the prices of multiple cryptocurrencies? (2) Does the ensemble model outperform the statistical and econometric model in improving forecasting accuracy? (3) Does the ensemble model provide a better estimate of future price on a broader set of multiple cryptocurrencies that differ in return and volatility dynamics?

The rest of the paper is structured as follows. A brief literature review is detailed in Section 2, outlining the previous papers that used various models to predict the price of cryptocurrencies. Section 3 explores the methodology and the ensemble prediction approach of this study, whereas Section 4 presents the empirical findings, including a comparison of the proposed ensemble method with econometric and ML models. The paper concludes with Section 5, providing avenues for future research in forecasting.

Over the last decade, cryptocurrencies have gone up from an obscure asset to a preferred investment due to millennials' popularity, increased risk-taking propensity, perceived profitability, decentralization and acceptance of cryptocurrency as an investment option.

Cryptocurrency price forecasting is considered one of the financial domain’s most difficult predictions (Livieris et al., 2021). Challenges, complexity and interpretability of models have been classified as a problem of time series forecasting by most successful researchers (Adegboruwa et al., 2019; Hamayel and Owda, 2021; Lahmiri and Bekiros, 2019; Patel et al., 2020; Tandon et al., 2019; Wirawan et al., 2019).

This literature review examines the existing research studies on cryptocurrency price forecasting and the various possible methodologies for the same. Traditional statistical and econometric models have been considered for the analysis, which goes on to assess promising ML techniques. Finally, the potential of combining these methodologies to create hybrid models is analyzed.

The early research on cryptocurrency price forecasting relied on traditional econometric and statistical models. Efficient and immediate solutions for time series problems can be achieved through statistical models (Meenu et al., 2020). Econometric approaches that combine statistical and economic principles, according to Alahmari (2019) are crucial to estimating and predicting economic variables relevant to cryptocurrency prices. Time-series analysis often employs the ARIMA model, which, as discussed above, is a widespread technique (El-Bannany et al., 2020). Yet, the statistical and econometric models in forecasting come along with limitations. The statistical models rely on specific assumptions such as the normal distribution of data, the linear relationship between the variables and the stability of parameters over time. Thus, statistical models perform poorly either with noisy data or with changes in the trends and patterns.

Nikou et al. (2019) found that the ARIMA model, while suitable for price prediction within specific periods, struggles with capturing sharp price fluctuations. Similarly, Greaves and Au (2015) also observed significant prediction errors with ARIMA, particularly in the cases pertaining to volatile markets. Moreover, statistical and econometric models work within a specific framework, thus limiting the model’s ability to handle complexities in real-world data. The presence of non-linear relationships between variables adversely affects the forecasting accuracy.

ML techniques have gained significant traction in cryptocurrency price forecasting as a consequence of advancements in big data technology and artificial intelligence. These techniques analyze complex data patterns and make predictions powerfully (Chevallier et al., 2021; Derbentseva et al., 2021).

There is a solid foundation through traditional statistical and econometric models to understand trends and relationships of financial data. However, cryptocurrency markets present the complexity of non-linear patterns that traditional models might not capture. ML techniques have the ability to identify such patterns with ease, only when provided with large datasets and careful tuning to avoid overfitting.

The benefits of statistical models lie in providing a foundational framework and distinguishing patterns and trends. For ML, the same reflects in uncovering the intricacies of complex non-linear relationships within the data. Through a combined approach, it is possible to tackle the limitations of individual models and optimize prediction accuracy (Madan et al., 2015).

Hybrid models that combine traditional and ML approaches can outperform classical ML models in terms of accuracy (Alahmari, 2019). The strengths of both methodologies come together to optimize accuracy in such a hybrid model.

Wang et al. (2020) studied the relationship between economic policy uncertainty (EPU) and Bitcoin prices. Their findings suggest that EPU has a negative impact on Bitcoin prices in the short term. This effect gradually diminishes over time and has no notable long-term effects. Similarly, positive information shocks lead to higher volatility in the short run, while negative shocks have left no significant impact on cryptocurrency prices (Yin et al., 2021)

Cryptocurrency markets are known for their intense fluctuations and unpredictability. A predictive model which can efficiently navigate price fluctuations is required for effective cryptocurrency price prediction. Comparative studies of statistical models and their strengths and weaknesses are lacking (Lobell and Burke, 2010). ML models are dynamic and can deal with complexities to predict the price with a high degree of accuracy. Researchers used ML models such as DT, GB, SVR and MLP. Contradicting the researchers who supported ML models for price forecasting, Cohn et al. (1996) have concluded that statistically-based learning architectures combined with a mixture of Gaussians and locally weighted regression improve forecasting accuracy. Feature engineering and feature extraction using ML models can improve forecasting accuracy.

Ensemble model synthesize the output from multiple ML models to make an accurate forecast. It works like seeking advice from many sources thus improving the forecasting accuracy. The model improves the forecasting accuracy by merging predictions from multiple models. The learning rate of ensemble model can be effectively improved through optimization and hyperparameter tuning. By combining diverse models, ensemble forecasting accuracy can be improved using features such as maximum vote, stacking, blending, bagging and boosting. The combination of multiple algorithms in an ensemble model makes it capable of outperforming single-base models in predicting cryptocurrency (Yang et al., 2022).

Primarily differing in their approach to value stability and underlying mechanisms, stablecoins and cryptocurrencies are quite different. Stablecoins are engineered to mitigate price volatility by pegging their worth to a stable asset or fiat currency such as the US dollar or Euro. This ensures their consistency in value. This research paper focuses on the prediction of cryptocurrency prices due to their volatility when compared to stable asset prices. By focusing solely on non-stablecoin cryptocurrencies, our research avoids potential complexities introduced by stablecoins, whose value is pegged to external assets and may exhibit less price volatility compared to other cryptocurrencies.

Daily price data of 15 coins, based on market capitalization, is collected to build the forecasting model starting from January 2018 to September 2022. The coins selected for the study are Ethereum (ETH), Binance (BNB), Ripple (XRP), Dogecoin (DOGE), TRON (TRX), Litecoin (LTC), Ethereum Classic (ETC), Stellar (XLM), Monero (XMR), File coin (FIL), Decentraland (MANA), ZCash (ZEC), Dash (DASH), NEM coin (XEM) and Mona coin (MONA), all priced in USD. Data regarding the prices is obtained from Coinmarketcap as it offers comprehensive coverage and is widely used in cryptocurrency research (Vidal-Tomás, 2022).

The research methodology is divided into four stages. The first stage is to examine the forecasting accuracy of statistical models and refine them further to improve the forecasting accuracy. We evaluate the forecasting accuracy of ARIMA in stage two. In Stage Three, ML algorithms, including GB, DT, RF, SVR and MLP, are employed.

The final stage combines the best-performing ML models to optimize the forecasting accuracy further and build the model. The three models are RF With eXtreme Gradient Boosting (XGBoost), Ensemble model with RF, GB, DT and multi-model regression (MLR). R-squared, RMSE, MSE and mean absolute error were employed to evaluate the performance of the models.

A SLR model is applied to forecast future prices when there is a linear relationship between dependent and independent variables. The equation has the form Y = a + bX, where Y is the dependent variable, i.e. the price of cryptocurrencies and X is the independent variable (time), b is the slope of the line and a is the y-intercept.

The ARIMA model, a stochastic process, includes sums of autoregressive and moving average components. The model is applicable when there is autocorrelation among the residuals. The model includes three parameters, i.e. p, d and q.

• p-auto-regressive component

• d-order of differencing

• q-moving average component.

ARIMA model specification

RF, a supervised learning algorithm, is applicable to solving classification and regression problems. The first work on RF goes back to Ho (1995). Breiman (2001) further developed the idea and presented the RF algorithm in 2001. Various DTs are created on randomly selected samples, the forecasted results are produced from each tree and then the best tree is chosen by voting. The key benefits of RF are its generalization capability and minimal sensitivity to hyperparameters (Naing and Htike, 2015). In the prediction of cryptocurrency prices, RF has been used for BTC forecasting (Chevallier et al., 2021) and BTC, ETH and XRP in (Derbentseva et al., 2021).

DT, a type of supervised ML model, made of decision nodes and leaf nodes, resembling a tree-like structure with branches and leaves. The decision node represents features and attributes, and the leaf node represents the outcome. Inducing DT is one of the oldest and most popular techniques for learning discriminatory models, which has been developed independently in the statistical (Breiman, 2017; Kass, 1980) and ML (Hunt et al., 1966; Quinlan, 1983, 1986) communities.

GB, an ML technique, is used to solve regression and classification issues. A collection of weak prediction models, often DT, are produced as outcomes (Madeh and El-Diraby, 2021). GB frequently outperforms RF (Madeh and El-Diraby, 2021; Piryonesi and El-Diraby, 2020), allowing optimization of any differentiable loss function.

The loss function and the base-learner models can be arbitrarily specified on demand. Given a loss function denoted by Ψ(y, f) and/or a custom base-learner h(x, θ), obtaining a solution to the parameter estimates can be difficult.

SVR is used for regression tasks to minimize forecasting errors. It operates by identifying a hyperplane with maximum margin from the training data. Support vectors, kernel functions and hyperparameter tuning are employed to capture complex relationships and handle outliers. This algorithm can powerfully predict continuous numerical values. Built on support vector machines for classification, SVR enables both linear and non-linear regressions. Examples of SVR usage in forecasting cryptocurrency prices can be found in (Chevallier et al., 2021; Khedr et al., 2021).

MLP regressor processes input data with multiple layers of interconnected nodes (neurons) to form non-linear transformations. It employs the backpropagation algorithm to adjust the weights and biases of the network, optimizing the model to minimize the error. Many complicated problems are not linearly separable, so the single hidden layer perceptron, which can only solve linearly separable problems, is not effective (Rosenblatt, 1958). These issues require a solution with multiple hidden layers (Rumelhart et al., 1986). The MPLNN, a feedforward neural network with multiple layers, comprises three layers: an input layer, one or more hidden layers and an output layer.

Three ensemble models have been created and tested to enhance the accuracy of this study.

GB is used in this approach to train an RF ensemble. The boosting technique is applied to the DTs in RF ensembles by updating the weights of the observations based on the ensemble residuals. Then, each tree in the ensemble is trained to correct the errors of the previous trees. This is done by using the updated weights of the observations. Such an iterative process continues as long as the ensemble’s performance continues to improve.

The final ensemble is expected to be more accurate than a simple RF, as the GB algorithm is used to train the DTs.

This method integrated multiple ML model forecasts to increase the forecast’s overall accuracy. This ensemble technique is built to individually develop each of the three models, namely RF, GB and DT, which will then return the average of their predictions. Each model may cause errors, so this method can balance them. This practice increases the overall accuracy of the ensemble model’s predictions. Each model is trained with a distinct set of algorithms and parameters on the same data set.

This approach employs the RFR, the GB, the SVR and the MLP regressor to form an MLR. Each model’s capabilities enhance this ensemble model to increase forecast accuracy. A meta-features data frame is generated by the MLR, which trains a linear regression meta-model using the ensemble of these models. This effectively captures varied patterns and correlations in cryptocurrency data. The MLR technique’s success in lowering prediction errors and offering insights into model performance is validated through the RMSE, MSE, MAE and R2 assessment measures.

Table 1 shows the descriptive statistics of 15 cryptocurrencies. Generally, cryptocurrencies show high volatility and also create volatility clustering. A perfect normal distribution data series has a skewness value of 0, and kurtosis equals 3. The skewness, kurtosis and Jarque–Bera results indicate that cryptocurrencies’ price movements do not follow a normal distribution. This makes it evident that investors face few larger gains and many periods of loss.

Table 2 shows the MAE of cryptocurrency prices. Upon examining the results, it is evident that the performance of the models varies across different coins. Starting with XRP, the RF model exhibits the lowest MAE of 0.019, outperforming other models such as ARIMA, SLR, DT, SVR and MLP. The RF model consistently demonstrates superior accuracy with comparatively lower MAE values than other models for BNB, MONA, XLM, DASH, XMR, FIL and TRX. However, the MLP model shows the lowest MAE values for ETH, LTC, ZEC, MANA and ETC, indicating its effectiveness in predicting the prices. On the other hand, SLR has high MAE values for most coins except for XRP. XRP shows a relatively low MAE compared to other models through SLR. Both RF ensemble with XGBoost and averaging ensemble of RF, GB and DT show low MAE demonstrating that the model performs well for most cryptocurrencies.

According to Table 3 on the MSE of cryptocurrency prices, the results vary in accordance with the performance of models across different coins. First, the RF model outperforms other models such as ARIMA, SLR, DT, GB, RF ensemble with XGBoost and averaging ensemble of RF, GB and DT for XRP, demonstrating the lowest MSE of 0.002. However, it is essential to note that it is unusual for the MLP model to show a negative MSE value for XRP. This might indicate an issue with the model or the data. The RF model consistently exhibits the lowest MSE values for BNB, MONA, XLM, DASH, XMR, FIL and TRX, indicating its superior accuracy when compared to other models. For ETH, LTC, ZEC, MANA and ETC, the MLP model, conversely, showcases the lowest MSE values among the models considered and stands out as the most accurate. It is worth mentioning that the SLR model shows a relatively low MSE value for XRP and performs poorly for most coins, resulting in a significant rise in MSE values compared to other models.

Table 4 results derive variations in the performance of models among different coins. The RF model outperforms other models such as ARIMA, SLR, DT, GB, SVR and MLP and achieves the lowest RMSE for XRP at 0.052. However, it is essential to note that the SLR model presents a relatively low RMSE for XRP. The RF model consistently exhibits the lowest RMSE values for BNB, MONA, XLM, DASH, XMR, FIL, TRX and ZEC, indicating its superior accuracy compared to other models. For ETH, LTC, MANA and ETC, the MLP model conversely stands out as the most accurate. It showcases the lowest RMSE values among the models considered. The ARIMA model shows relatively high RMSE values across the board, while the SVR model performs moderately well for most coins. Additionally, except for XRP and TRX, where it performs relatively better, the SLR model exhibits higher RMSE values for most coins. RF ensemble with XGBoost and Averaging ensemble of RF, GB and DT has relatively low RMSE demonstrating that the model performs well for most cryptocurrencies.

The results of Table 5 show that the RF, DT and GB models have strong predictive power, due to high R-squared values across most coins. Their ability to account for a considerable proportion of the fluctuations in cryptocurrency prices is evident in these models. The ARIMA model has limited explanatory capability for cryptocurrency price movements, as it exhibits comparatively lower R-squared values for all coins. Some coins have relatively higher R-squared values through the SLR model, while others have significantly negative values. Hence, the results are mixed. It is important to note that the SLR model performs worse than a horizontal line (mean) in fitting the data when R-squared values are negative. This implies that the relationships between those specific coins’ dependent and independent variables cannot be captured adequately by the SLR model. There is varied performance across different coins with the MLP model as well. It achieves high R-squared values for some coins, such as BNB and ETH, and presents negative R-squared values for others. This suggests that for those particular coins, the MLP model may not effectively capture the underlying patterns in cryptocurrency prices. For RF ensemble with XGBoost and averaging ensemble model with RF, GB and DT show reasonably high R2 values indicate that the model can explain a considerable percentage of the data variation. However, when certain cryptocurrencies, such as XRP, have lower R2 values for RF ensemble with XGBoost, it is inferred that the model cannot explain a large variation in the data for this coin. Thus, this model’s prediction accuracy is quite poor for this currency. The averaging ensemble model with RF, GB and DT model has R2 values are typically near one, indicating that the model can accurately predict the target variable.

Table 6 and Figure 2 shows the forecasting results of MLR for pre and post-COVID crisis. The results show that the model consistently achieves exceptional performance across all coins. Extremely low values of MAE, MSE and RMSE indicate its high accuracy in predicting cryptocurrency prices covering pre and post-crisis. Additionally, it can be concluded that the model can perfectly explain the variance in cryptocurrency prices, indicating its strong explanatory power, given that the R2 value of all coins is 1.00. The significant performance of the MLR in accurately forecasting cryptocurrency prices across different coins, as the table highlights, is overall indicative of its effectiveness.

The descriptive statistics analysis, helped understand the nature of cryptocurrency price distributions. Cryptocurrencies are known for their high volatility and the presence of volatility clustering, as noted in the literature review. Aligning with findings from the previous research, the skewness and kurtosis values indicate deviations from a normal distribution (Nikou et al., 2019). This supports the need for robust forecasting models capable of handling such volatile and skewed data.

ARIMA, SLR, RF, DT, GB and MLR ensemble models performed well with coins such as XLM, FIL, MANA, DASH, XMR and ZEC. These models adequately reflect the above coins' trends, seasonality and historical price patterns. Furthermore, MLR demonstrates its advantages when applied to coins like BNB and MONA. SLR has performed well for cryptocurrencies that have significant market capitalization, strong liquidity and price movements that have historically been consistent, like XRP, TRX and DOGE. The MLR ensemble approach produces more accurate predictions for coins with higher volatility and irregular price patterns, such as ETH and ETC, by combining ML models to remove errors.

It is worth noting that the intricacies of each coin’s behavior may not be captured as these are broad observations. Macroeconomic, attractiveness and demand and supply variables are all factors that can impact a model’s efficacy. The literature review expresses concern regarding cryptocurrency price forecasting challenges, echoed by the findings, which demonstrate that different models perform variably across different cryptocurrencies. The analysis of the RF ensembles with XGBoost and averaging ensemble model with RF, GB and DT demonstrates the effectiveness of ensemble models combining multiple models for improved forecasting accuracy. The results support that ensemble models can enhance predictive performance. They often outperformed individual models across various cryptocurrencies in the current study.

Cryptocurrency price forecasting carries a significant influence on investment decisions. The present study makes contribution to the theoretical and practical implications on cryptocurrency investments. On the theoretical side, the study advances the conceptual framework on building predictive models by demonstrating the efficiency of ensemble model in forecasting. It shows a new dimension to predictive analytics by empirically validating the accuracy of multiple algorithms pre-COVID and post-COVID period. On the practical side, it offers a valuable forecasting tool to the traders and investors in making informed decision. The novelty of the research is in its attempt to empirically test the forecasting ability of ensemble model across time span and its comparison with econometric model on a broad-based set of cryptocurrencies. Consistent with the expectations, ensemble models created by combining RF, XGBoost, gradient boosting and decision tree (DT) reduce the variance in predicting errors indicating the forecasting accuracy. Interestingly, it is found that when the price movement exhibits autocorrelation, the ARIMA model is best applicable. Compared to other ML models applied in isolation the ensemble model performed better.

The present study contributes to the body of literature on ensemble methods in forecasting crypto price in general, potentially influencing future studies on price forecasting. The study motivates the researchers on empirical testing of our framework on various asset classes. As a result, on the prediction ability of ensemble model, the study will significantly influence the decision-making process of traders and investors. The research benefits the traders and investors to effectively develop a model to forecast cryptocurrency price. The findings highlight the potential of ensemble model in predicting high volatile cryptocurrencies and other financial assets. Investors can design the investment strategies and asset allocation decisions by understanding the relationship between market trends and consumer behavior. Investors can enhance portfolio performance and mitigate risk by incorporating these insights into their decision-making processes. Policymakers can use this information to design more effective regulations and policies promoting economic stability and consumer welfare. The study emphasizes the need for using diversified model to understand the market dynamics and improving trading strategies.

Although the dataset includes crisis period data, anomalies or outliers are yet to be explicitly excluded from the analysis. The models employed in this study still demonstrate high accuracy in predicting cryptocurrency prices despite these outliers, suggesting that the models are robust enough to handle unexpected fluctuations or extreme events in the market. However, the lack of specific analysis on the impact of outliers on model performance is a limitation of the study, as it needs to fully explore the resilience of the forecasting models under adverse market conditions.

Cryptocurrency markets can be influenced by investor sentiment, which can be reflected in price movements. By analyzing these price movements, investors might gain insights into overall market sentiment. However, exploring this link further is certainly an area for future research. While our study focuses on price prediction using the ensemble model, we acknowledge the need for further exploration to definitively link price movements to specific investor behaviors. Complementary techniques, such as sentiment analysis of social media data or transaction volume analysis, could be valuable tools for future research in this area.