Predicting multi-horizon currency exchange rates using a stacked ensemble of random forest and SVR

Abstract

A thorough exploration of the effects of a given minute’s currency exchange rates on subsequent 1, 5, 10, 15, 30, 45, and 60 minutes’ currency exchange rates is presented in this article, with machine learning and ensemble methods being applied. The focus is on twelve currency pairs, including EUR/AUD, EUR/GBP, and EUR/PLN, with a data set of per-minute logs of these pairs’ exchange rates from 2022 being leveraged. A stacked ensemble of Random Forest and Support Vector Regression (SVR) is used to predict future exchange rates. A comparison of this model is also made with the single RF, single SVR, and an average ensemble of RF and SVR models. The research method is further fortified by the use of k-fold cross-validation and ANOVA tests. The findings of the study reveal significant predictive accuracy of the stacked ensemble model, emphasizing the intricate interconnections of currency exchange rates. The potential of machine learning and ensemble techniques in predicting short-term currency exchange rates is underlined, thereby augmenting financial forecasting research.

1. Introduction

1.1 Background

The global financial ecosystem’s crucial component, currency exchange rates, are subject to continuous fluctuations influenced by a wide array of factors, including macroeconomic indicators and market sentiment. The prediction of these fluctuations, characterized by complex and non-linear interactions among influencing factors, has long been a significant challenge. Over recent years, however, financial forecasting has seen the emergence of machine learning as a potent tool, offering innovative and efficient solutions. Ensemble methods, in particular, which combine multiple machine learning models, have shown considerable promise. In our recent study, the strong performance of a stacked ensemble model of Random Forest and Support Vector Regression (SVR) in predicting stock prices was observed. The application of this proficient machine learning model to the prediction of currency exchange rates over various time horizons is proposed based on these findings.

1.2 Objectives and scope

The central goal of this study is the application of an efficient machine learning ensemble model (specifically, a stacked ensemble of Random Forest and Support Vector Regression (SVR)) to the prediction of short-term currency exchange rate movements. This extends the application of the model from its demonstrated utility in stock price forecasting to the realm of forex market prediction. In addition, a comparison with individual models (single RF, single SVR) and an average ensemble of RF and SVR is undertaken to evaluate relative performance. The methodology has been further fortified using k-fold cross-validation and ANOVA tests.

The research scope is restricted to twelve Euro-based currency pairs, including EUR/AUD, EUR/GBP, EUR/PLN, among others. Three sample pairs (EUR/AUD, EUR/GBP, and EUR/PLN) were selected for comprehensive analysis, predicting their exchange rates for subsequent 1, 5, 10, 15, 30, 45, and 60 minutes based on the current minute’s exchange rates.

The focus of this study is on minute-by-minute exchange rates throughout 2022, without the incorporation of other data types such as economic indicators or financial news, and the interpretation of results should remain within this context.

1.3 Methodology overview

A systematic methodology is employed to meet the stated objectives. The approach is characterized by the following stages:

•

Data Collection: A robust dataset was compiled, comprising minute-by-minute exchange rates for twelve Euro-based currency pairs during 2022.

•

Data Preprocessing: This stage involves data cleansing, the handling of missing values, and data normalization to ensure suitability for machine learning models.

•

Model Selection: Based on the outcomes of our previous study, the stacked ensemble model of Random Forest and Support Vector Regression (SVR) was selected, which exhibited superior predictive performance. Single RF, single SVR, and average ensemble models were also considered for comparative analysis.

•

Model Training and Testing: The models were trained on a subset of the dataset, with separate data being reserved for validation and testing. The current minute’s exchange rates were used as input features, and the future minute’s exchange rates as targets.

•

Prediction and Performance Evaluation: The trained models were utilized to predict exchange rates over different future time horizons (1, 5, 10, 15, 30, 45, and 60 minutes). Predictions from the models were compared to actual values, and performance was assessed using Mean Squared Error (MSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE) metrics, with these steps being repeated during cross-validation.

•

Statistical Testing: ANOVA tests were used to demonstrate significant differences between the base models, justifying the use of an ensemble of distinct models.

The effectiveness of the chosen ensemble model, as well as its comparison to other models in predicting future currency exchange rates, was systematically assessed through this approach.

2. Literature review

The exploration and utilization of computational intelligence and machine learning techniques in the field of foreign exchange rate prediction have undergone significant development in recent years, driven by advances in computing technology and the exponential growth of available financial data. The breadth of this literature review underscores the variety and ingenuity of contemporary research, with different authors exploring an array of methodologies and architectures in pursuit of accurate and reliable Forex predictions. [1] kicked off the decade with a novel hybrid model, while [2] combined a host of sophisticated techniques to provide a new method for forecasting in financial markets. As progression is made through the timeline, the increasing intricacy of the methods and the deeper incorporation of artificial intelligence, neural networks, fuzzy logic, and other advanced techniques are observed, as exemplified in the works of [3, 4, 5].

[1] presents a novel hybrid model for predicting foreign exchange rates that outperforms traditional models like neural networks and Support Vector Machines (SVMs). The methodology involves two stages: first, the “delay coordinate embedding” is utilized to recreate the unobserved phase space of exchange rate dynamics, revealing underlying patterns in the financial data that would otherwise be obscured by its inherent non-linearity and non-stationarity. Second, kernel predictors like SVMs are used to forecast based on this phase space. The study found that this model, named Chaos-SVR, significantly reduces the root-mean-squared forecasting errors, providing a more accurate prediction of the future evolution of exchange rates. The paper suggests that this innovative forecasting approach can be applied to a broader spectrum of financial forecasting problems and could prove instrumental in creating more effective global investment strategies.

[2] combines a host of sophisticated techniques to provide a new method for forecasting in financial markets, specifically, the Foreign Exchange Market. This method utilizes historical market data and chart patterns to forecast market trends. This is achieved through a hybrid Artificial Neural Network-based Fuzzy Inference System (ANFIS) coupled with Quantum-behaved Particle Swarm Optimization (QPSO) to tune the ANFIS membership functions. A novel Dynamic Time Warping (DTW)-Wavelet Transform (WT) method for automatic pattern extraction is also introduced. Experiments with the method demonstrated an approximately 69% accuracy rate in providing correct trading signals, indicating its effectiveness. However, the complexity of combining ANFIS, QPSO, DTW, and WT could be viewed as a limitation.

[6] offers a novel perspective to FOREX trading systems. The paper notes the limitations of prediction-based systems and proposes a system based on technical analysis indicators and a new approach to rule-base evidential reasoning (RBER). This innovative system synthesizes fuzzy logic and the Dempster-Shafer theory of evidence to overcome the loss of vital information when dealing with intersecting fuzzy classes such as Low and Medium. The results from real-world Forex data demonstrate the potential benefits of this approach. However, the authors also identify the limitations of other rule combination methods, suggesting that a simple averaging rule may be the most effective in handling both high conflict evidence and the absence of it.

[7] presents a novel application of Reinforcement Learning (RL) for short-term speculation in the foreign exchange market. Leveraging modern RL advancements, the authors construct neural networks with three ReLU hidden layers to train as RL agents using Q-learning in a unique simulated market environment. This innovative framework includes new state and reward signals and a method for more efficient utilization of historical tick data, contributing to enhanced training quality and out-of-sample data generalization. The system was tested on the EUR/USD market from 2010 to 2017 and yielded an average total profit of 114.0 $\pm$ 19.6%, equating to an annual average of 16.3 $\pm$ 2.8%. Despite the inherent challenges of financial markets, the authors managed to achieve stable learning that could be extended to live trading. The research identifies potential areas for improvement and diversification, highlighting the future potential of this RL-based approach in the field of algorithmic trading.

[3] presents a novel hybrid approach to forex trading. The authors utilize a Support Vector Machine (SVM) to classify forex market trends into three distinct types and deploy Dynamic Genetic Algorithms (GAs) to optimize investment strategies based on the market classifications. The technique leverages the power of these machine learning models for predictive analysis of past market events to inform future investment rules, with an emphasis on effective leverage strategy. The research used data from the EUR/USD forex pair from 2003 to 2016 for model training and testing. The results were promising, with a return on investment of 83% during the test period from January 2015 to March 2016.

[8] presents a compelling approach to tackling the challenge of short-term currency rate forecasting in the Forex market through an innovative deep learning framework. The authors introduce an adaptive activation function selection mechanism within a deep learning architecture composed of seven neural networks. Each network employs a different activation function and makes use of a softmax and multiplication layer with a skip connection to generate dynamic importance weights. This mechanism helps determine the most suitable activation function in a given context, enhancing predictive accuracy. The paper further stands out with its introduction of an extended Min-Max normalization technique, adapted to normalize non-stationary financial time series, which is a commendable step forward in the field. Experimental evaluations reveal that the proposed model outperforms conventional machine learning models and standard neural network architectures, making this a valuable contribution to the field of financial forecasting.

[4] presents an innovative financial prediction system called Chaotic Type-2 Transient-Fuzzy Deep Neuro-Oscillatory Network with Retrograde Signaling (CT2TFDNN). Drawing upon advanced AI techniques such as neural networks, chaotic systems, and fuzzy logic, the CT2TFDNN was designed to handle the complexity and uncertainty associated with financial forecasting. Leveraging the author’s original work on the Lee oscillator, the network utilizes an Interval Type-2 Fuzzy Logic System (IT2FLS) that provides advanced modeling of uncertain events. The system is applied to 2048 trading-day time-series financial data and ten major financial signals for real-time prediction of 129 global financial products. Importantly, CT2TFDNN addresses the overtraining and deadlock issues common in recurrent neural networks with traditional activation functions. Results show that CT2TFDNN outperforms several other forecasting systems, including those based on Support Vector Machines and Deep Neural Networks, thus demonstrating its potential for robust real-time financial prediction.

[9] extensively explores the use of deep learning for exchange rate forecasting. The authors conduct a comprehensive comparison of Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs) with traditional recurrent network architectures and feedforward networks for their forecasting accuracy and trading model profitability. The paper reveals the effectiveness of deep networks for exchange rate forecasting, albeit with notable implementation and tuning challenges. The study found that simpler neural networks could sometimes perform as well as, if not better than, more complex deep neural networks.

[10] presents a novel model integrating domain knowledge into machine learning models for improved Forex market predictions. The proposed model, FLF-LSTM, incorporates the Forex Loss Function (FLF) into a Long Short-Term Memory (LSTM) model, aiming to minimize the difference between actual and predicted averages of Forex candles. Empirical results using the data of 10,078 four-hour candles of the EURUSD pair show that the FLF-LSTM system significantly outperforms traditional LSTM, Recurrent Neural Network (RNN)-based prediction systems, the Auto-Regressive Integrated Moving Average (ARIMA), and the FB Prophet model in terms of mean absolute error reduction. This suggests that incorporating domain knowledge in the learning process enhances the predictability of Forex trends.

[11] applies the interval Multi-Layer Perceptron (iMLP) to the realm of foreign exchange rate forecasting. The authors delve into the intricacies of interval time series (ITS), demonstrating how the iMLP can be utilized with interval-valued input and output data, making it particularly applicable in predicting exchange rates. The researchers break away from the traditional usage of 15 neurons in the hidden layer, investigating a range of values for better forecasting performance. Additionally, they introduce other exchange rates, such as AUD/USD and GBP/USD, into their model to gauge their potential impact on forecasting.

[12] compares three models – ARIMA, Elman, and LSTM – for predicting the EUR/USD exchange rate over time. The study uses these models to forecast different windows for the validation datasets, with ARIMA producing constant numbers, LSTM providing superior forecasts for a 22-day window in the short term, and Elman offering better results in the long term. The results, averaged over the 22-day window, demonstrated an accuracy rate of 71.76%, which is superior to several past studies that reported average accuracy rates below 70%. However, LSTM coincided with the validation dataset in only four observations with an average accuracy of 95.99%, leaving room for improvement.

[13] delves into the prediction of Forex currency rates, highlighting the shortcomings of existing techniques that perform optimally only for specific currency pairs and time frames. They present a comprehensive trading system that is trained on a decade-long historical data of six major and minor currency pairs. This system is distinctive in its combination of various trading rules, seeking to better simulate real-world trading decision-making processes. Surprisingly, they find that deep learning techniques, specifically the ResNet50 and Vision Transformer architectures, do not necessarily enhance the technical analysis of Forex data for future price predictions. Instead, their approach emphasizes a broader view of trading systems, assessing predictive models based on their overall impact on trading decisions. They advocate for an investment strategy that encompasses all currency pairs simultaneously to manage the risk of loss and maintain profitability.

[14] focuses on overcoming the challenges in developing machine learning-based methods for financial trading through the use of deep reinforcement learning. The authors propose an innovative approach to provide consistent rewards for the trading agent, thereby mitigating the traditionally noisy profit-and-loss rewards system. A novel price trailing-based reward shaping approach is also employed, significantly enhancing the agent’s performance in terms of profit, Sharpe ratio, and maximum drawdown. Data preprocessing methods are introduced that allow for training across different FOREX currency pairs, aiming to build market-wide agents and utilize powerful recurrent deep learning models without risk of overfitting. The paper’s main contribution lies in presenting a framework that successfully trains deep reinforcement learning to overcome existing limitations in this area. The study exhibits its ability to improve performance metrics using a challenging, large-scale dataset containing 28 different instruments.

[15] presents an innovative approach to predicting the profitability of trade positions in Forex markets. It leverages a combination of image processing and time series analysis within a deep learning framework, specifically CNN for image processing and CNN-LSTM for time series analysis. The study uses technical price patterns and proposes a hybrid model that operates alongside the moving average crossover technical pattern. The model is tested on EUR/USD pair data and compares favorably against individual techniques and other state-of-the-art models like CNN-BiLSTM and PSR with LSTM.

[16] developed a novel method to predict Forex markets using a multi-agent system (MAS) and fuzzy logic. This system simulates the actions and interactions of traders in a live market. The traders are represented by agents whose actions are governed by rules and beliefs modelled through intuitionistic fuzzy logic systems, a choice which contributes to an enhanced interpretability of the model. They also introduce the concept of “specialization functions” to mimic the scenario where certain traders are more adept at navigating specific market conditions. The authors validate their method through extensive experimentation, comparing it to prevalent predictive models such as Random Forest, AdaBoost, XGBoost, SVM, and deep learning, and find it to be competitive. Furthermore, they demonstrate the interpretability of their method, outlining the easily understandable rules used by the agents, providing a unique transparency compared to “black-box” models.

[17] presents an innovative model for predicting future closing prices in the FOREX market by leveraging the strengths of two powerful neural networks – the Gated Recurrent Unit (GRU) and Long Short Term Memory (LSTM). The proposed model, which incorporates a GRU layer with 20 hidden neurons and an LSTM layer with 256 hidden neurons, is tested on four major currency pairs: EUR/USD, GBP/USD, USD/CAD, and USD/CHF, with predictions made for 10-minute and 30-minute timeframes. Performance validation metrics such as MSE, RMSE, MAE, and R2 score, confirm the model’s superiority over standalone LSTM, GRU models, and a simple moving average-based statistical model. Despite the model’s effectiveness, it sometimes struggles with sudden spikes in closing prices, suggesting room for further research and optimization.

[18] presents a novel approach to Forex rate forecasting, which stands out due to its focus on overcoming two primary challenges: gradient vanishing and information loss in sequential neural networks, and irregular frequency of fundamental data (FD) updates. The authors propose a cascading model for Forex market forecasting, termed as BERTFOREX, which utilizes Bidirectional Encoder Representations from Transformers (BERT) to extract hidden patterns from both FD and technical indicator data (TI). An innovative approach is taken, where FD is first processed through BERT to recognize latent patterns and then applied as additional weights to TI, given the slower changing frequency of FD. A second BERT application then extracts the aggregated pattern within TI and FD over influential days. The cascading structure of the model is claimed to make the system more robust and allows it to adapt to the FD’s absent value issues. A simple neural network then takes the aggregated pattern as input for final forecasting. Experimental results suggest that this method outperforms other Forex forecasting techniques, with high accuracy in the percentage of correct signals, sensitivity, specificity, precision, and negative predictive value.

[5] delves into the challenge of forecasting nonstationary time-series data, with a specific focus on foreign exchange data. Recognizing that traditional methods for stationary time-series data are ineffective for nonstationary series, the authors propose a novel hybrid seq2seq model leveraging a recurrent neural network (RNN) with an attention mechanism to enhance predictive accuracy. The model is complemented by dynamic time warping and zigzag peak valley indicators for enriched feature extraction, and unique loss functions aimed at precisely predicting sequence peaks and valleys. The performance of this model is assessed on the Forex dataset, where it exhibits high accuracy in predicting 4-hour future trends, a crucial attribute for practical decision making in Forex trading. Further, the paper contributes to ongoing research by testing and comparing the impact of various model components on its performance, including different loss functions, evaluation metrics, model variants, and feature sets.

In this field, continuous growth and development are evident, and the results highlighted in this literature review confirm the effectiveness of these novel approaches. Nevertheless, it’s crucial to recognize the enduring challenges in predicting foreign exchange rates. These challenges encompass the inherent uncertainty and non-stationarity of financial markets, the difficulty of extracting beneficial features from abundant financial data, and the complexities entailed in the implementation and fine-tuning of advanced machine learning models. Despite these obstacles, the unwavering dedication of researchers to enhance and fine-tune these prediction models is noteworthy, and their successful efforts forecast a promising view of what is to come. As the exploration of artificial intelligence and machine learning in financial forecasting continues to deepen, further breakthroughs in the prediction of foreign exchange rates are expected. The considerable progress made in the field is exemplified by the works reviewed here, shedding light on both the current research trajectories and the potential future advancements.

3. Methodology

3.1 Data collection

HistData.com [19], an online platform offering high-quality historical Forex data, was the source from which the dataset for this study was obtained. The data was presented in a minute-by-minute format, aligning with the objective of predicting short-term future currency exchange rates based on the current minute exchange rates. The study focused on 12 currency pairs involving the Euro (EUR) due to its significant global financial market role. The specific currency pairs considered were EUR/AUD, EUR/CAD, EUR/CHF, EUR/CZK, EUR/GBP, EUR/HUF, EUR/JPY, EUR/NOK, EUR/PLN, EUR/SEK, EUR/TRY, and EUR/USD, covering the entire year of 2022.

3.2 Dataset overview

This subsection provides an extended overview of the dataset used in the study, obtained from HistData.com, a reputable platform offering free Forex historical data for traders and developers. The dataset consists of minute-by-minute exchange rates for 12 currency pairs involving the Euro (EUR) throughout the year 2022. The specific currency pairs considered include EUR/AUD, EUR/CAD, EUR/CHF, EUR/CZK, EUR/GBP, EUR/HUF, EUR/JPY, EUR/NOK, EUR/PLN, EUR/SEK, EUR/TRY, and EUR/USD.

HistData.com delivers data in .CSV format (comma-separated values), making it compatible with various applications like MetaTrader, NinjaTrader, MetaStock, and more. The platform’s commitment to providing free historical data stems from the shared need among traders and developers for reliable information to backtest trading systems.

Figure 1.

Frequency histograms of all currency pairs.

Figure 2.

Boxplot of exchange rates.

Figure 3.

Exchange rates over time for EUR/PLN.

Figure 4.

Rolling statistics of EUR/GBP.

Figure 5.

Rolling statistics of EUR/TRY.

Figure 6.

Correlation matrix of exchange rates.

Table 1

The first 10 rows of the minute-by-minute exchange rate dataset for the specified currency pairs in 2022

DATE-TIME (EST)	EUR/AUD	EUR/CAD	EUR/CHF	EUR/CZK	EUR/GBP	EUR/HUF	EUR/JPY	EUR/NOK	EUR/PLN	EUR/SEK	EUR/TRY	EUR/USD
2022-01-02 17:01	1.56356	1.43585	1.03628	24.8407	0.83937	369.317	130.743	10.00104	4.58643	10.26849	15.16616	1.1369
2022-01-02 17:02	1.56434	1.43547	1.03587	24.8444	0.83959	369.417	130.693	9.99679	4.58652	10.26961	15.16531	1.13689
2022-01-02 17:03	1.56446	1.43549	1.03652	24.8284	0.83954	369.512	130.707	10.00015	4.586	10.26588	15.21165	1.13692
2022-01-02 17:04	1.56474	1.43509	1.03626	24.8524	0.8391	369.538	130.756	9.99795	4.58624	10.27008	15.14329	1.13668
2022-01-02 17:05	1.56483	1.43575	1.03631	24.853	0.83974	369.538	130.765	9.99749	4.58575	10.27006	15.15891	1.13655
2022-01-02 17:06	1.56483	1.43547	1.0363	24.8608	0.83988	369.548	130.766	10.00545	4.58575	10.27175	15.1721	1.13682
2022-01-02 17:07	1.56482	1.43547	1.03587	24.8684	0.83992	369.553	130.755	10.00647	4.58623	10.27257	15.17382	1.13689
2022-01-02 17:08	1.56382	1.43594	1.03635	24.8694	0.83992	369.546	130.848	10.00594	4.58545	10.27388	15.20046	1.13682
2022-01-02 17:09	1.564	1.43547	1.03587	24.8623	0.8399	369.53	130.849	10.00599	4.58584	10.27481	15.22931	1.13723
2022-01-02 17:10	1.56425	1.43591	1.03587	24.8694	0.8399	369.516	130.848	10.00208	4.58486	10.27393	15.20626	1.13725

Table 2

The mean, standard deviation, and interquartile range (IQR) of the exchange rates of each currency pair

	EUR/AUD	EUR/CAD	EUR/CHF	EUR/CZK	EUR/GBP	EUR/HUF	EUR/JPY	EUR/NOK	EUR/PLN	EUR/SEK	EUR/TRY	EUR/USD
Mean	1.515568	1.368005	1.002561	24.5583	0.853078	391.6496	137.9615	10.10304	4.689771	10.63602	17.44594	1.051125
Standard deviation	0.045548	0.047984	0.029809	0.254489	0.016327	19.70631	5.383715	0.275441	0.086603	0.233494	1.400308	0.050735
Interquartile range (IQR)	0.07582	0.08875	0.05385	0.265925	0.02632	29.76525	7.707	0.428505	0.09736	0.408993	2.258455	0.08532

The dataset encompasses 121,673 rows and 12 columns, representing minute-by-minute exchange rates for the specified currency pairs. The exploratory data analysis conducted on this dataset aimed to unveil its characteristics and dynamics. As a supplement to this description, the first few rows of the dataset are presented in Table 2. The Mean, Standard Deviation, and Interquartile Range (IQR) of the exchange rates of each currency pair are also given in Table 2. Five types of exploratory data analyses were performed, with the results visually represented in Figs 1 to 6.

• •

Frequency Histograms of All Currency Pairs (Fig. 1): Frequency histograms of all currency pairs are shown in Fig. 1 to give a general overview of the dataset.

•

Boxplots of Exchange Rates for all Currency Pairs (Fig. 2): Boxplots were created to provide an overview of each currency pair’s exchange rate distribution. They reveal essential statistical parameters such as the central tendency, spread, and potential outliers within each dataset.

•

Exchange Rate Trend over Time (Fig. 3): A time-series plot for the exchange rate of EUR/PLN as a sample currency pair was drawn. This visualization helps identify the underlying patterns, trends, and fluctuations over the year.

•

Rolling Statistics of Two Sample Currency Pairs (Figs 4 and 5): The rolling mean and the rolling standard deviation of the currency pairs EUR/GBP and EUR/TRY, as two sample exchange pairs, are presented in Figs 4 and 5, respectively.

•

Correlation Matrix for all Currency Pairs’ Exchange Rates (Fig. 6): A correlation matrix was created to gain insights into the association degree and direction between different currency pairs’ exchange rates, illuminating positive, negative, or no correlation.

3.3 Data preprocessing

The aim of the data preprocessing stage was to optimize the data format for analysis and modeling. It consisted of two primary processes: missing data imputation and normalization.

Missing data points in the exchange rates were replaced by the average of the preceding and following minute’s exchanges rate for that currency pair, ensuring temporal continuity and minimizing potential bias or distortion.

Normalization, the next step, was carried out independently for each column in the dataset, corresponding to each currency pair, considering the different exchange rate magnitudes. The data was scaled into a [0,1] range, maintaining the original data pattern.

Lagged versions of the exchange rates for EUR/AUD, EUR/GBP, and EUR/PLN were created for supervised learning, representing the target values. The lagged time intervals included 1, 5, 10, 15, 30, 45, and 60 minutes. The stacked ensemble machine learning model was given 12 inputs for each minute, each representing a different currency pair’s current exchange rate, and a single target, which is the predicted future exchange rate for a specific currency pair. This approach encapsulated the interconnectedness of the currency exchange rates within the model.

3.4 Machine learning techniques and ensemble methods used

3.4.1 Support Vector Regression (SVR)

Support Vector Regression (SVR) is an innovative adaptation of the Support Vector Machine (SVM) algorithm, uniquely designed to accommodate regression tasks. SVR strives to devise a function that efficiently fits the training data whilst concurrently curbing the generalization error. This method stands out for its particular efficacy with high-dimensional datasets, and its resistance to overfitting surpasses several other regression methodologies.

•

SVR Formulation

Consider a dataset encompassing $N$ samples, represented as, $\{(x_1,y_1),(x_2,y_2),\mathrm{\dots },(x_N,y_N)\}$ , The primary objective of SVR is to unearth a function $f\left(x\right)$ which approximates the target values $y_i$ within a defined margin of error $\epsilon$ . This approach prioritizes minimizing the complexity of the function, whilst ensuring that the training errors remain confined within the prescribed margin of error.

The function $f\left(x\right)$ can be expressed as a linear combination of kernel functions, mathematically represented as:

$f\left(x\right)=\mathrm{\Sigma }\left(a_i-{\alpha }_i^{\ast }\right)\times K(x,x_i)+b$(1)

In this equation, $a_i$ and ${\alpha }_i^{\ast }$ denote the Lagrange multipliers, $K(x,x_i)$ is the kernel function, and $b$ stands for a bias term. The SVR issue can be construed as a convex optimization problem, as per the following formulation:

$\text{𝑚𝑖𝑛𝑖𝑚𝑖𝑧𝑒}:\left({\displaystyle \frac{1}{2}}\right)\times \mathrm{\Sigma }{\left(a_i-{\alpha }_i^{\ast }\right)}^2\times K(x,x_i)+\epsilon \times \mathrm{\Sigma }\left(a_i-{\alpha }_i^{\ast }\right)-\mathrm{\Sigma }\left(y_i\times \left(a_i-{\alpha }_i^{\ast }\right)\right)$(2)

This is subject to the conditions: $0⩽a_i,{\alpha }_i^{\ast }⩽C$ , and $\mathrm{\Sigma }\left(a_i-{\alpha }_i^{\ast }\right)=0$ .

Here, $C$ is the box constraint that regulates the balance between minimizing the training error and the complexity of the function.

•

Kernel Functions

A kernel function $K(x,x_i)$ maps the input data into a higher-dimensional space, thereby empowering SVR to model non-linear relationships between input features and target values. Widely employed kernel functions encompass linear, polynomial, and radial basis function (RBF) kernels. The selection of the kernel function and its corresponding parameters markedly influences the performance of the SVR model.

•

SVR Algorithm Summary

The generic procedure for constructing an SVR model for exchange rate prediction includes the following steps:

1. (a)

Select a kernel function and its parameters (for instance, RBF with a specified kernel scale).

(b)

Define the margin of error $\epsilon$ and the box constraint $C$ .

(c)

Formulate and resolve the convex optimization problem to ascertain the optimal Lagrange multipliers $a_i$ and ${\alpha }_i^{\ast }$ , along with the bias term $b$ .

(d)

Utilize the derived function $f\left(x\right)$ to generate predictions on new input data.

To conclude, SVR presents multiple benefits for predicting exchange rates. These encompass its capacity to depict non-linear relationships, resistance against overfitting, and appropriateness for high-dimensional datasets. Through judicious adjustment of the SVR model’s parameters, like the margin of error, kernel function, and box constraint, a harmonious equilibrium can be reached between the model’s precision and its complexity.

In this research, a grid search is executed to identify the optimal hyperparameters for the SVR model. The hyperparameters that were evaluated include:

• •

Epsilon: This parameter sets a margin around the predicted values that disregard errors within its bounds. A broader margin and fewer support vectors result from a higher epsilon value, while a smaller epsilon results in a tighter margin and increased support vectors.

•

Kernel scale: This scaling factor for the radial basis function (RBF) kernel governs the shape of the decision boundary. A more substantial kernel scale produces a smoother decision boundary, whereas a smaller kernel scale yields a more flexible boundary.

•

Box constraint: This is a regularization parameter balancing minimizing training error and maximizing the margin. Larger values create a wider margin while tolerating some errors, whereas smaller values focus on minimizing training error.

A 5-fold cross-validation was used to identify the optimal hyperparameters. The training data was split into five equal portions, and in each iteration, four of these were used for training and one for testing. This procedure was conducted for all possible hyperparameter combinations.

For each hyperparameter set, the Mean Absolute Error (MAE) was computed for each fold, and then averaged across all folds to provide a single performance metric. The hyperparameter set resulting in the lowest average MAE was deemed the optimal one.

The optimal hyperparameters were found to be:

• •

Epsilon: best_epsilon $=$ 0.01.

•

Kernel scale: best_kernel_scale $=$ 0.1.

•

Box constraint: best_box_constraint $=$ 1.

The final SVR model was trained utilizing the complete training dataset and the optimal hyperparameters.

3.4.2 Random forest

The Random Forest algorithm proceeds through the subsequent stages:

1.

Create several bootstrapped samples from the training dataset.

2.

Construct a decision tree for each bootstrapped sample. While building each tree, a random subset of features is chosen for splitting at every node.

3.

Amalgamate the forecasts from all individual trees to create the final output.

The outcome of a Random Forest model for a specific input is mathematically represented as the average prediction from all individual decision trees:

$\widehat{y}\left(x\right)=\left({\displaystyle \frac{1}{N}}\right)\times \mathrm{\Sigma }\left(y_i\left(x\right)\right)$(3)

where $N$ is the tree count, $y_i\left(x\right)$ is the $i^{th}$ decision tree’s prediction for input $x$ , and $\widehat{y}\left(x\right)$ is the Random Forest model’s final prediction. For every decision tree, the forecast is based on the tree’s structure, where input $x$ navigates through the tree until reaching a leaf node. Each node splits the input based on a particular feature and a threshold:

$x_j\le t$(4)

where $x_j$ is the value of the $j^{th}$ feature of the input, and $t$ is the threshold for splitting.

• •

Training the Random Forest Model

In this research, a Random Forest model encompassing 50 decision trees is trained with the training dataset. The quantity of trees in the Random Forest is a vital hyperparameter impacting the model’s performance and generalization. Typically, a higher tree count enhances model accuracy but could increase computational expenses and the potential for overfitting.

•

Predicting with the Random Forest Model

Predictions with the trained Random Forest model are made by averaging the individual decision tree’s predictions for a given input, aiding in reducing prediction variance and enhancing model accuracy overall.

Conclusively, a Random Forest model with 50 decision trees was utilized to forecast changes in the future exchange rates of currency pairs based on the present minute’s exchange rates. By combining individual decision tree predictions, the Random Forest model offers a dependable and robust technique for predicting currency exchange rates.

In the current research, the number of trees in the random forest (num_trees) is initially established to 100. A random forest model is an aggregation of decision trees, and an increased number of trees usually enhances the model’s accuracy, but it might also incur higher computational expenses.

Subsequently, a grid search is executed to pinpoint the optimal hyperparameters for the random forest model. The hyperparameters scrutinized in this study included:

• •

Minimum leaf size (min_leaf_size): This refers to the minimum count of observations in a leaf node. Lower values permit the trees to extend deeper, potentially leading to overfitting, whereas higher values could induce underfitting.

•

Maximum number of splits (max_num_split): This sets the maximum number of divisions allowed in each tree, and consequently governs the complexity of the trees. Greater values facilitate more complex trees.

•

Number of variables to sample (num_vars_to_sample): This determines the count of predictor variables chosen randomly at each division in the decision tree, thereby introducing variability in the tree building process, which can aid in enhancing generalization.

To discover the optimal hyperparameters, k-fold cross-validation is applied, with k set to 5. The training data was sectioned into five parts of equal size, with four segments being used for training and the remaining segment for testing during each iteration. This procedure was replicated for all potential hyperparameter combinations.

For each hyperparameter set, the Mean Absolute Error (MAE) was computed for each fold and averaged across all folds to yield a single performance metric. The hyperparameter set associated with the lowest average MAE was chosen as the optimal one.

The optimal hyperparameters discerned in this study were:

• •

Minimum leaf size: optimal_min_leaf_size $=$ 20.

•

Maximum number of splits: optimal_max_num_split $=$ 100.

•

Number of variables to sample: optimal_num_vars $=$ 50.

Finally, using the complete training dataset and the optimal hyperparameters, the final random forest model is trained.

3.4.3 Training time Information for the RF and SVR methods

On the designated dataset, the Support Vector Regression (SVR) method exhibits an average training time of 12,008 seconds, while the Random Forest method concludes training in an average of 1,763 seconds. These timings provide noteworthy insights into the computational requirements of each method. The evaluations were conducted on a high-performance computing system equipped with an 11 ${}^{\text{th}}$ Gen. Intel(R) Core(TM) i9-11900H processor running at 2.50 GHz and boasting 64.0 GB of installed RAM. The efficiency of the algorithms is contextualized within the robust processing power and substantial memory capacity of the hardware infrastructure.

The discrepancy in training times between the Random Forest and SVR methods can be attributed to several factors. Notably, RF exhibits faster execution due to its algorithmic complexity, hyperparameter tuning requirements, parallelization capabilities, considerations of data size, and implementation optimizations.

•

Algorithmic Complexity: Random Forest involves building multiple decision trees during training, utilizing a parallelized process of recursively partitioning the data based on feature values. In contrast, SVR requires solving a mathematical optimization problem, which can be computationally intensive, particularly with large datasets.

•

Hyperparameter Tuning: RF often requires fewer hyperparameters to tune, and default configurations may yield satisfactory results. Conversely, SVR involves tuning more hyperparameters, leading to an increased need for an exhaustive search over the hyperparameter space during training.

•

Parallelization: RF benefits from parallelization, allowing the training of individual decision trees to occur concurrently. This parallel nature contributes to faster computation, especially on systems with multiple processors. SVR may face limitations in parallelization, potentially leading to longer training times.

•

Data Size: RF efficiently handles large datasets, and its performance remains robust even with increasing data size. SVR, however, may experience longer training times as its computational complexity scales with the dataset size.

•

Implementation and Optimization: The efficiency of RF is further enhanced by widely available, optimized implementations and integrated features like feature importance calculations. SVR, depending on the specific implementation and mathematical solver, may exhibit variations in efficiency.

In summary, the faster execution of the Random Forest method compared to SVR can be attributed to its algorithmic characteristics, ease of hyperparameter tuning, parallelization capabilities, robustness with larger datasets, and efficient implementation practices.

3.4.4 Average ensembling

Average ensembling is an effective technique where the predictions from multiple models are combined by taking their average, which often leads to improved predictive performance. This is because averaging can help balance out the weaknesses of individual models and reduce variance. In this study, the average ensemble technique was applied to the predictions of the Support Vector Regression (SVR) and Random Forest models.

Mathematically, if there are $N$ models and each model $n$ (where $n=1,2,\mathrm{\dots },N)$ provides a prediction $P_n$ for a given instance, the final prediction $P_a$ is computed as:

$P_a=\left({\displaystyle \frac{1}{N}}\right)\times {\displaystyle \sum P_n},\text{for}n=1\text{to}N$(5)

3.4.5 Stacked ensembling

Stacking, also known as stacked generalization, is a methodology that uses an ensemble strategy. This involves training a secondary or meta-model to optimally integrate the predictions from multiple foundational models. The foundational models are trained using the original training data, and the meta-model is trained using the foundational models’ predictions on a distinct validation dataset. In this research, stacked ensembles are developed by leveraging decision tree, neural network, and random forest models as foundational models, with a variety of algorithms employed for the meta-model. The meta-model input is the validation data predictions from the foundational models. The composite prediction vector from all foundational models for a particular input $x$ is represented as follows:

$Z\left(x\right)=[y_1\left(x\right),y_2\left(x\right),\mathrm{\dots },y_N\left(x\right)]$(6)

where $y_i\left(x\right)$ represents the prediction of the $i^{th}$ base model for input $x$ , and $Z\left(x\right)$ represents the combined prediction vector from all foundational models. The meta-model, which can be any suitable machine learning algorithm, is subsequently trained using these composite predictions and the actual outputs for the validation data:

$M\left(Z\left(x\right)\right)=\widehat{y}\left(x\right)$(7)

where $M$ is the meta-model, $Z\left(x\right)$ is the combined prediction vector of the base models for input $x$ , and $\widehat{y}\left(x\right)$ is the true output for input $x$ . The final prediction of the stacked ensemble is the output of the meta-model for the combined predictions of the foundational models on the testing data:

$\widehat{y}\left(x\right)=M\left(Z\left(x\right)\right)$(8)

To provide a concise summary of the Stacked Ensemble of Random Forest and SVR method, the key snippets of the pseudocode that capture the essence of the algorithm are presented below. This abbreviated version is designed for readers seeking a quick reference to the fundamental steps and logic involved in the stacking process:

1. Convert response variable to double

2. Split data into training and testing sets

3. Set SVR hyperparameters and k-fold parameters

4. Initialize best mean absolute error for SVR

5. SVR Hyperparameter Tuning

a. Loop over epsilon, kernel_scale, box_constraint

i. Initialize error sum

ii. K-fold cross-validation for SVR

$-$ Train SVR model

$-$ Predict using SVR model

$-$ Calculate mean absolute error

$-$ Update error sum

iii. Calculate average mean absolute error

iv. Update best hyperparameters if better

6. Train final SVR model with best hyperparameters

7. Make predictions on test data using SVR model

8. Set Random Forest hyperparameters and k-fold parameters

9. Initialize best mean absolute error for Random Forest

10. Random Forest Hyperparameter Tuning

a. Loop over min_leaf_size, max_num_split, num_vars

i. Initialize error sum

ii. K-fold cross-validation for Random Forest

$-$ Train Random Forest model

$-$ Predict using Random Forest model

$-$ Calculate mean absolute error

$-$ Update error sum

iii. Calculate average mean absolute error

iv. Update best hyperparameters if better

11. Train final Random Forest model with best hyperparameters

12. Make predictions on test data using Random Forest model

13. Ensemble of SVR and Random Forest

$-$ Combine predictions

$-$ Bagging Ensemble

$-$ Boosting Ensemble

$-$ Weighted Average Ensemble

14. Combine predictions into a matrix

15. Train linear regression model to combine predictions

16. Make predictions using stacked model

17. Calculate performance metrics

For a comprehensive understanding of the Stacked Ensemble of Random Forest and SVR method, the full pseudocode is included in the appendix. This detailed version outlines the algorithm’s step-by-step procedures, ensuring a thorough grasp of the intricacies involved in the stacking of Random Forest and Support Vector Regression models. Refer to the full pseudocode in the appendix for an in-depth exploration of the algorithm’s implementation.

3.5 Performance evaluation metrics

In the present research, three assessment metrics were employed for evaluating the performance of both the individual and the ensemble models. The utilized metrics are the Mean Squared Error (MSE), the Mean Absolute Error (MAE), and the Mean Absolute Percentage Error (MAPE). The corresponding mathematical representations for these evaluation metrics are provided below:

$\text{𝑀𝑆𝐸}=\left({\displaystyle \frac{1}{n}}\right)\times \mathrm{\Sigma }{\left(y_i-{\widehat{y}}_i\right)}^2$(9)

$\text{𝑀𝐴𝐸}=\left({\displaystyle \frac{1}{n}}\right)\times \mathrm{\Sigma }\left|y_i-{\widehat{y}}_i\right|$(10)

$\text{𝑀𝐴𝑃𝐸}=\left({\displaystyle \frac{1}{n}}\right)\times \mathrm{\Sigma }\left|\left(\left(y_i-{\widehat{y}}_i\right)/y_i\right)\right|\times 100$(11)

where $y_i$ is the actual output, ${\widehat{y}}_i$ is the predicted output, and $n$ is the number of data points.

3.6 Cross validation

Cross validation is a resampling technique used to evaluate machine learning models on a limited data sample. The goal is to assess how well the machine learning model is capable of predicting new data that was not used in model training.

One popular method is $k$ -fold cross-validation. Here, the original sample is randomly partitioned into k equal sized subsamples. Of the $k$ subsamples, a single subsample is retained as validation data for testing the model, and the remaining $k$ -1 subsamples are used as training data. The cross-validation process is then repeated $k$ times, with each of the $k$ subsamples used exactly once as the validation data. The $k$ results from the folds can then be averaged to produce a single estimation.

3.7 ANOVA test

The Analysis of Variance (ANOVA) is a statistical method employed to ascertain the existence of significant differences between the means of two or more groups. In the context of ANOVA, the null hypothesis (H0) posits that the means of the groups under consideration are equal. If the derived p-value from the ANOVA test is lower than the predetermined significance level, typically 0.05, the null hypothesis is rejected. This rejection indicates the presence of a significant difference between the group means.

In the context of this research, the ANOVA test was implemented to compare the performance of the Support Vector Regression (SVR) and Random Forest (RF) models, which were the base models for the ensemble techniques. The application of the ANOVA test aimed to confirm a significant difference in the performance of these two models, hence justifying the utilization of an ensemble model combining these two distinct base models. By establishing a significant variance in their predictions, the rationale for ensembling these models to leverage their unique strengths and compensations, thereby improving the robustness and accuracy of predictions is provided.

Table 3

The MSE, MAE, and MAPE values for the single RF, single SVR, average ensemble of RF $+$ SVR, and stacked ensemble of RF $+$ SVR models

Currency pair	Forecast horizon	Random forest MSE	Random forest MAE	Random forest MAPE	SVR MSE	SVR MAE	SVR MAPE	Average ensemble MSE	Average ensemble MAE	Average ensemble MAPE	Stacked ensemble MSE	Stacked ensemble MAE	Stacked ensemble MAPE
EUR/	+1 Minutes	0.000008	0.001978	0.303721%	0.000012	0.00273	0.411643%	0.000008	0.002015	0.30635%	0.000006	0.001622	0.247237%
AUD	+5 Minutes	0.000032	0.003957	0.606228%	0.000036	0.00437	0.663414%	0.000031	0.003893	0.593489%	0.000028	0.003657	0.557786%
	+10 Minutes	0.00006	0.005504	0.841971%	0.000075	0.006275	0.95516%	0.000061	0.005575	0.850704%	0.000057	0.005314	0.811081%
	+15 Minutes	0.000095	0.006995	1.068081%	0.000147	0.009255	1.386995%	0.000101	0.007378	1.116357%	0.000086	0.0066	1.006487%
	+30 Minutes	0.000253	0.011612	1.763386%	0.000415	0.015882	2.345374%	0.000253	0.012109	1.814329%	0.000195	0.010196	1.552355%
	+45 Minutes	0.000423	0.015531	2.371969%	0.000625	0.020245	3.047732%	0.000398	0.015303	2.32143%	0.000275	0.012169	1.853452%
	+60 Minutes	0.000748	0.02023	3.095687%	0.001506	0.029959	4.478909%	0.000731	0.021027	3.176158%	0.00039	0.014743	2.247728%
EUR/	+1 Minutes	0.000005	0.001563	0.224506%	0.000005	0.001302	0.190155%	0.000005	0.001309	0.189756%	0.000005	0.001124	0.163818%
GBP	+5 Minutes	0.000026	0.003373	0.488998%	0.000023	0.002944	0.428015%	0.000024	0.003064	0.444960%	0.000023	0.002934	0.426512%
	+10 Minutes	0.000052	0.004758	0.689231%	0.000048	0.004467	0.648931%	0.000049	0.004523	0.656406%	0.000048	0.004431	0.643964%
	+15 Minutes	0.000078	0.005910	0.856502%	0.000075	0.005664	0.822730%	0.000075	0.005698	0.827195%	0.000073	0.005618	0.816764%
	+30 Minutes	0.000153	0.008529	1.236270%	0.000144	0.008132	1.181634%	0.000146	0.008214	1.192907%	0.000141	0.008109	1.179833%
	+45 Minutes	0.000224	0.010520	1.523006%	0.000206	0.009995	1.451736%	0.000210	0.010098	1.464954%	0.000199	0.009877	1.437221%
	+60 Minutes	0.000293	0.012106	1.749672%	0.000268	0.011535	1.673658%	0.000272	0.011616	1.682445%	0.000254	0.011291	1.642128%
EUR/	+1 Minutes	0.000007	0.001656	0.392430%	0.000006	0.001380	0.320866%	0.000006	0.001458	0.341985%	0.000006	0.001354	0.315634%
PLN	+5 Minutes	0.000026	0.003382	0.787371%	0.000025	0.003273	0.759830%	0.000025	0.003284	0.763072%	0.000024	0.003248	0.754213%
	+10 Minutes	0.000048	0.004732	1.097743%	0.000046	0.004600	1.065427%	0.000046	0.004601	1.065757%	0.000046	0.004551	1.054768%
	+15 Minutes	0.000068	0.005655	1.310592%	0.000066	0.005496	1.273372%	0.000067	0.005517	1.277235%	0.000066	0.005466	1.266996%
	+30 Minutes	0.000120	0.007649	1.769964%	0.000119	0.007529	1.737543%	0.000118	0.007509	1.733842%	0.000117	0.007483	1.729776%
	+45 Minutes	0.000161	0.009081	2.101600%	0.000160	0.008931	2.056890%	0.000159	0.008938	2.061712%	0.000157	0.008893	2.054271%
	+60 Minutes	0.000195	0.010227	2.370261%	0.000194	0.010039	2.312926%	0.000192	0.010044	2.319246%	0.000190	0.009993	2.311719%

4. Results and discussion

The results of the performance evaluation for single RF, single SVR, average ensemble of RF+SVR, and stacked ensemble of RF+SVR models on the test data are summarized in Table 3. It becomes readily apparent that the error metrics (Mean Squared Error, Mean Absolute Error, and Mean Absolute Percentage Error) are lowest for immediate (1-minute) predictions. As the forecast horizon lengthens, these error values naturally increase, reflecting the inherent difficulties in predicting long-term trends in the forex market. This progression highlights the necessity of frequently updating the prediction models with the most recent data. Such a dynamic approach enables the model to better navigate the volatility and unpredictable nature of currency exchange rate movements.

Table 4

The 5-fold cross-validation results for the currency pairs EUR/AUD

Forecast Horizon	Model	Fold 1 MAE	Fold 1 MSE	Fold 1 MAPE	Fold 2 MAE	Fold 2 MSE	Fold 2 MAPE	Fold 3 MAE	Fold 3 MSE	Fold 3 MAPE	Fold 4 MAE	Fold 4 MSE	Fold 4 MAPE	Fold 5 MAE	Fold 5 MSE	Fold 5 MAPE
$+$ 1 Minutes	Random Forest	0.001956	0.000007	0.300499%	0.001975	0.000008	0.303614%	0.002097	0.000009	0.322813%	0.001854	0.000007	0.282951%	0.002007	0.000009	0.308727%
	SVR	0.001801	0.000007	0.275073%	0.002602	0.000011	0.391458%	0.003990	0.000021	0.593207%	0.002933	0.000013	0.447007%	0.002323	0.000010	0.351443%
	Average Ensemble	0.001769	0.000006	0.271177%	0.001966	0.000007	0.299330%	0.002446	0.000011	0.367481%	0.002022	0.000008	0.308153%	0.001871	0.000008	0.285602%
	Stacked Ensemble	0.001608	0.000006	0.245165%	0.001582	0.000006	0.240940%	0.001634	0.000007	0.249648%	0.001633	0.000006	0.248126%	0.001653	0.000007	0.252307%
$+$ 5 Minutes	Random Forest	0.004104	0.000033	0.629949%	0.003889	0.000030	0.594616%	0.004003	0.000033	0.614439%	0.003830	0.000030	0.586055%	0.003958	0.000033	0.606086%
	SVR	0.003989	0.000031	0.605056%	0.003616	0.000027	0.550705%	0.003733	0.000030	0.569369%	0.006010	0.000055	0.904173%	0.004502	0.000039	0.687755%
	Average Ensemble	0.003864	0.000030	0.589601%	0.003633	0.000027	0.554242%	0.003759	0.000030	0.574532%	0.004264	0.000033	0.646142%	0.003947	0.000033	0.602928%
	Stacked Ensemble	0.003775	0.000029	0.576334%	0.003568	0.000027	0.544090%	0.003714	0.000030	0.566493%	0.003559	0.000026	0.543114%	0.003668	0.000030	0.558902%
$+$ 10 Minutes	Random Forest	0.005391	0.000056	0.825006%	0.005568	0.000062	0.853349%	0.005454	0.000058	0.835608%	0.005544	0.000065	0.847482%	0.005564	0.000060	0.848409%
	SVR	0.006190	0.000069	0.932006%	0.005653	0.000064	0.866623%	0.006272	0.000070	0.957489%	0.007381	0.000106	1.136417%	0.005880	0.000063	0.883246%
	Average Ensemble	0.005484	0.000058	0.833700%	0.005387	0.000060	0.824286%	0.005565	0.000059	0.850881%	0.005991	0.000072	0.918537%	0.005446	0.000058	0.826105%
	Stacked Ensemble	0.005149	0.000053	0.784532%	0.005358	0.000059	0.819535%	0.005196	0.000054	0.793883%	0.005495	0.000064	0.839604%	0.005374	0.000057	0.817844%
$+$ 15 Minutes	Random Forest	0.006981	0.000092	1.066900%	0.007090	0.000100	1.080448%	0.007071	0.000097	1.080650%	0.006803	0.000094	1.037666%	0.007032	0.000094	1.074739%
	SVR	0.007704	0.000101	1.150156%	0.008468	0.000129	1.295563%	0.014540	0.000289	2.127882%	0.008928	0.000134	1.352189%	0.006634	0.000084	1.009048%
	Average Ensemble	0.006894	0.000089	1.045730%	0.007116	0.000098	1.084749%	0.009215	0.000136	1.370150%	0.007123	0.000099	1.083711%	0.006541	0.000083	0.997398%
	Stacked Ensemble	0.006562	0.000083	1.001987%	0.006718	0.000090	1.022787%	0.006880	0.000092	1.050554%	0.006364	0.000085	0.969990%	0.006478	0.000081	0.987103%
$+$ 30 Minutes	Random Forest	0.011516	0.000247	1.747883%	0.011574	0.000247	1.758255%	0.011941	0.000271	1.813271%	0.011230	0.000239	1.708632%	0.011798	0.000260	1.788887%
	SVR	0.012089	0.000253	1.845680%	0.010810	0.000203	1.639559%	0.017601	0.000454	2.590033%	0.014680	0.000331	2.164942%	0.024229	0.000835	3.486555%
	Average Ensemble	0.011094	0.000226	1.686141%	0.010260	0.000193	1.558702%	0.012349	0.000255	1.849357%	0.011335	0.000223	1.702274%	0.015506	0.000367	2.275147%
	Stacked Ensemble	0.009492	0.000176	1.441837%	0.010053	0.000184	1.530185%	0.010288	0.000199	1.572280%	0.009976	0.000184	1.520354%	0.011173	0.000234	1.697097%
$+$ 45 Minutes	Random Forest	0.014839	0.000385	2.263247%	0.015931	0.000443	2.438184%	0.015166	0.000407	2.312844%	0.015568	0.000422	2.376954%	0.016152	0.000462	2.468600%
	SVR	0.023300	0.000746	3.500958%	0.017228	0.000504	2.601073%	0.028347	0.001019	4.237482%	0.019801	0.000570	2.996538%	0.012545	0.000287	1.902699%
	Average Ensemble	0.016426	0.000438	2.490184%	0.013937	0.000353	2.119708%	0.017886	0.000490	2.690257%	0.015207	0.000394	2.318248%	0.013056	0.000315	1.988795%
	Stacked Ensemble	0.012210	0.000272	1.852147%	0.012460	0.000287	1.902003%	0.011948	0.000264	1.820884%	0.012029	0.000278	1.831259%	0.012198	0.000275	1.860956%
$+$ 60 Minutes	Random Forest	0.019176	0.000705	2.928963%	0.019582	0.000712	3.000109%	0.021379	0.000835	3.271501%	0.019735	0.000670	3.012529%	0.021280	0.000815	3.265336%
	SVR	0.018777	0.000553	2.810490%	0.014947	0.000396	2.259992%	0.032798	0.001512	4.791366%	0.019009	0.000595	2.911366%	0,064263	0.004477	9.622043%
	Average Ensemble	0.017578	0.000540	2.676231%	0.015637	0.000453	2.391294%	0.024200	0.000882	3.620247%	0.016497	0.000484	2.518060%	0.031220	0.001294	4.675165%
	Stacked Ensemble	0.014156	0.000364	2.165945%	0.013877	0.000356	2.116606%	0.015277	0.000411	2.331709%	0.016385	0.000459	2.485161%	0.014019	0.000362	2.139180%

Table 5

The 5-fold cross-validation results for the currency pairs EUR/GBP

Forecast Horizon	Model	Fold 1 MAE	Fold 1 MSE	Fold 1 MAPE	Fold 2 MAE	Fold 2 MSE	Fold 2 MAPE	Fold 3 MAE	Fold 3 MSE	Fold 3 MAPE	Fold 4 MAE	Fold 4 MSE	Fold 4 MAPE	Fold 5 MAE	Fold 5 MSE	Fold 5 MAPE
$+$ 1 Minutes	Random Forest	0.001520	0.000006	0.217905%	0.001539	0.000006	0.222844%	0.001648	0.000007	0.235747%	0.001600	0.000007	0.230128%	0.001506	0.000006	0.215903%
	SVR	0.001272	0.000005	0.185470%	0.001326	0.000005	0.194033%	0.001298	0.000005	0.189685%	0.001351	0.000005	0.197638%	0.001262	0.000005	0.183947%
	Average Ensemble	0.001293	0.000005	0.186826%	0.001327	0.000005	0.193332%	0.001317	0.000005	0.190982%	0.001365	0.000006	0.198254%	0.001242	0.000005	0.179385%
	Stacked Ensemble	0.001111	0.000005	0.161603%	0.001124	0.000005	0.164399%	0.001111	0.000004	0.162149%	0.001181	0.000005	0.172159%	0.001093	0.000004	0.158779%
$+$ 5 Minutes	Random Forest	0.003426	0.000027	0.495766%	0.003324	0.000026	0.483026%	0.003360	0.000025	0.486715%	0.003306	0.000025	0.479749%	0.003447	0.000029	0,499736%
	SVR	0.002999	0.000024	0.435380%	0.002892	0.000023	0.421105%	0.002883	0.000021	0.419143%	0.002980	0.000023	0.432837%	0.002968	0.000025	0.431611%
	Average Ensemble	0.003112	0.000025	0.451377%	0.002998	0.000023	0.436150%	0.003024	0.000022	0.439119%	0.003062	0.000023	0.444490%	0.003124	0.000026	0.453666%
	Stacked Ensemble	0.002997	0.000024	0.435215%	0.002869	0.000022	0.417787%	0.002874	0.000021	0.417974%	0.002958	0.000023	0.429712%	0.002969	0.000025	0.431875%
$+$ 10 Minutes	Random Forest	0.004722	0.000049	0,683666%	0.004723	0.000050	0.686717%	0.004901	0.000058	0.709181%	0.004745	0.000050	0.687447%	0.004696	0.000052	0.679140%
	SVR	0.004413	0.000045	0.641152%	0.004429	0.000046	0.645190%	0.004583	0.000054	0.664778%	0.004447	0.000047	0.646738%	0.004463	0.000049	0.646793%
	Average Ensemble	0.004504	0.000046	0.653799%	0.004496	0.000047	0.654615%	0.004640	0.000055	0.672418%	0.004490	0.000047	0.652108%	0.004483	0.000049	0.649089%
	Stacked Ensemble	0.004402	0.000045	0.639932%	0.004396	0.000046	0.640437%	0.004562	0.000054	0.662072%	0.004392	0.000046	0.639029%	0.004402	0.000048	0.638349%
$+$ 15 Minutes	Random Forest	0.005963	0.000081	0.863572%	0.005982	0.000077	0.868765%	0.005925	0.000085	0.856040%	0.005880	0.000077	0.851540%	0.005798	0.000070	0.842590%
	SVR	0.005729	0.000077	0.831839%	0.005707	0.000073	0.830879%	0.005669	0.000081	0.821755%	0.005613	0.000074	0.814258%	0.005603	0.000068	0.814920%
	Average Ensemble	0.005741	0.000078	0.832871%	0.005746	0.000073	0.836306%	0.005716	0.000082	0.827546%	0.005665	0.000074	0.821446%	0.005622	0.000067	0.817807%
	Stacked Ensemble	0.005679	0.000076	0.825333%	0.005639	0.000072	0.821939%	0.005642	0.000080	0.818588%	0.005578	0.000073	0.810083%	0.005552	0.000066	0.807873%
$+$ 30 Minutes	Random Forest	0.008528	0.000152	1.236342%	0.008546	0.000150	1.240733%	0.008422	0.000149	1.220487%	0.008539	0.000151	1.235233%	0.008608	0.000163	1.248555%
	SVR	0.008209	0.000146	1.192788%	0.008187	0.000144	1.190554%	0.008068	0.000141	1.171846%	0.007928	0.000137	1.151047%	0.008269	0.000154	1.201939%
	Average Ensemble	0.008270	0.000146	1.200994%	0.008261	0.000144	1.201080%	0.008124	0.000142	1.179168%	0.008109	0.000141	1.176149%	0.008306	0.000155	1.207145%
	Stacked Ensemble	0.008156	0.000141	1.186937%	0.008153	0.000140	1.187176%	0.008064	0.000137	1.172804%	0.007948	0.000135	1.155285%	0.008224	0.000150	1.196965%
$+$ 45 Minutes	Random Forest	0.010462	0.000226	1.511395%	0.010708	0.000230	1.549381%	0.010570	0.000222	1.531874%	0.010517	0.000226	1.529430%	0.010344	0.000217	1.492947%
	SVR	0.009885	0.000207	1.435463%	0.010175	0.000209	1.477615%	0.009906	0.000201	1.439969%	0.010132	0.000213	1.476180%	0.009876	0.000202	1.429449%
	Average Ensemble	0.009999	0.000211	1.448729%	0.010277	0.000214	1.490275%	0.010077	0.000205	1.463700%	0.010191	0.000215	1.484037%	0.009944	0.000204	1.438023%
	Stacked Ensemble	0.009786	0.000200	1.424216%	0.010036	0.000202	1.460946%	0.009874	0.000196	1.437611%	0.009916	0.000202	1.445931%	0.009770	0.000195	1.417394%
$+$ 60 Minutes	Random Forest	0.012099	0.000289	1.744927%	0.012094	0.000289	1.754706%	0.012026	0.000288	1.737045%	0.012283	0.000308	1.772135%	0.012025	0.000294	1.739545%
	SVR	0.011992	0.000276	1.736906%	0.011462	0.000265	1.668178%	0.011311	0.000258	1.639372%	0.011465	0.000273	1.661111%	0.011446	0.000268	1.662733%
	Average Ensemble	0.011635	0.000268	1.682021%	0.011619	0.000271	1.688830%	0.011500	0.000265	1.663914%	0.011732	0.000283	1.696522%	0.011592	0.000273	1.680939%
	Stacked Ensemble	0.011261	0.000249	1.635861%	0.011367	0.000257	1.658001%	0.011200	0.000249	1.627340%	0.011405	0.000265	1.655251%	0.011222	0.000251	1.634182%

Furthermore, a detailed presentation of the 5-fold cross-validation results for the currency pairs EUR/AUD, EUR/GBP, and EUR/PLN can be found in Tables 4, 5, and 6 respectively. These tables elucidate the robustness of the model across different data subsets, thus enhancing the confidence in its predictive capability.

Table 6

The 5-fold cross-validation results for the currency pairs EUR/PLN

Forecast Horizon	Model	Fold 1 MAE	Fold 1 MSE	Fold 1 MAPE	Fold 2 MAE	Fold 2 MSE	Fold 2 MAPE	Fold 3 MAE	Fold 3 MSE	Fold 3 MAPE	Fold 4 MAE	Fold 4 MSE	Fold 4 MAPE	Fold 5 MAE	Fold 5 MSE	Fold 5 MAPE
$+$ 1 Minutes	Random Forest	0.001606	0.000006	0.380540%	0.001656	0.000007	0.392633%	0.001682	0.000007	0.398500%	0.001691	0.000006	0.399273%	0.001645	0.000006	0.391202%
	SVR	0.001371	0.000005	0.318099%	0.001382	0.000006	0.322168%	0.001413	0.000006	0.328089%	0.001372	0.000005	0.318733%	0.001363	0.000005	0.317238%
	Average Ensemble	0.001423	0.000006	0.333427%	0.001467	0.000006	0.344667%	0.001491	0.000006	0.349739%	0.001455	0.000006	0.341229%	0.001451	0.000005	0.340861%
	Stacked Ensemble	0.001329	0.000005	0.309858%	0.001375	0.000006	0.321011%	0.001385	0.000006	0.322500%	0.001330	0.000005	0.310039%	0.001350	0.000005	0.314763%
$+$ 5 Minutes	Random Forest	0.003357	0.000025	0.784038%	0.003358	0.000025	0.781246%	0.003422	0.000027	0.794615%	0.003412	0.000026	0.792402%	0.003362	0.000025	0.784552%
	SVR	0.003218	0.000023	0.749088%	0.003237	0.000024	0.751076%	0.003314	0.000026	0.766997%	0.003278	0.000025	0.758285%	0.003320	0.000025	0.773702%
	Average Ensemble	0.003247	0.000024	0.757686%	0.003245	0.000024	0.753377%	0.003337	0.000026	0.773405%	0.003298	0.000026	0.764122%	0.003291	0.000025	0.766769%
	Stacked Ensemble	0.003202	0.000023	0.746563%	0.003209	0.000024	0.744325%	0.003298	0.000026	0.764468%	0.003266	0.000025	0.756446%	0.003263	0.000025	0.759261%
$+$ 10 Minutes	Random Forest	0.004819	0.000053	1.112381%	0.004626	0.000043	1.076946%	0.004760	0.000049	1.107690%	0.004786	0.000050	1.103884%	0.004669	0.000045	1.087812%
	SVR	0.004714	0.000053	1.086584%	0.004478	0.000041	1.042222%	0.004613	0.000047	1.074046%	0.004634	0.000048	1.067302%	0.004561	0.000043	1.056984%
	Average Ensemble	0.004692	0.000052	1.080218%	0.004496	0.000041	1.046447%	0.004637	0.000048	1.078464%	0.004655	0.000048	1.071678%	0.004526	0.000043	1.051979%
	Stacked Ensemble	0.004650	0.000052	1.070447%	0.004461	0.000041	1.039386%	0.004575	0.000047	1.064560%	0.004596	0.000047	1.058941%	0.004473	0.000042	1.040509%
$+$ 15 Minutes	Random Forest	0.005621	0.000069	1.301014%	0.005617	0.000068	1.303419%	0.005662	0.000064	1.321377%	0.005691	0.000066	1.315223%	0.005684	0.000075	1.311924%
	SVR	0.005434	0.000067	1.258109%	0.005397	0.000065	1.251206%	0.005501	0.000062	1.283768%	0.005560	0.000065	1.283588%	0.005589	0.000074	1.290194%
	Average Ensemble	0.005469	0.000067	1.265904%	0.005445	0.000065	1.261534%	0.005537	0.000062	1.290975%	0.005560	0.000065	1.282668%	0.005575	0.000074	1.285093%
	Stacked Ensemble	0.005427	0.000066	1.257254%	0.005371	0.000064	1.247819%	0.005497	0.000062	1.282141%	0.005515	0.000064	1.272857%	0.005521	0.000073	1.274911%
$+$ 30 Minutes	Random Forest	0.007738	0.000122	1.795329%	0.007810	0.000125	1.811437%	0.007767	0.000125	1.790792%	0.007532	0.000119	1.739053%	0.007398	0.000110	1.713215%
	SVR	0.007555	0.000119	1.749615%	0.007737	0.000124	1.789381%	0.007605	0.000123	1.749090%	0.007446	0.000119	1.713143%	0.007300	0.000109	1.686487%
	Average Ensemble	0.007575	0.000119	1.755387%	0.007689	0.000123	1.778123%	0.007600	0.000122	1.750123%	0.007408	0.000118	1.705070%	0.007274	0.000108	1.680511%
	Stacked Ensemble	0.007550	0.000118	1.752605%	0.007670	0.000122	1.774488%	0.007586	0.000121	1.748004%	0.007369	0.000116	1.699422%	0.007240	0.000107	1.674364%
$+$ 45 Minutes	Random Forest	0.009196	0.000162	2.130118%	0.008952	0.000155	2.080141%	0.009158	0.000161	2.123703%	0.009069	0.000169	2.097148%	0.009032	0.000159	2.076893%
	SVR	0.008990	0.000160	2.073613%	0.008750	0.000153	2.024665%	0.009006	0.000161	2.077555%	0.008982	0.000170	2.063732%	0.008928	0.000159	2.044892%
	Average Ensemble	0.009034	0.000160	2.087931%	0.008789	0.000152	2.037487%	0.009006	0.000159	2.079941%	0.008941	0.000167	2.057951%	0.008920	0.000158	2.045255%
	Stacked Ensemble	0.009023	0.000159	2.088313%	0.008773	0.000152	2.034046%	0.008955	0.000158	2.069116%	0.008881	0.000165	2.049777%	0.008834	0.000154	2.030105%
$+$ 60 Minutes	Random Forest	0.010144	0.000192	2.347753%	0.010273	0.000203	2.363596%	0.010403	0.000203	2.407597%	0.010000	0.000179	2.341192%	0.010314	0.000199	2.391166%
	SVR	0.010015	0.000191	2.306714%	0.010067	0.000204	2.297744%	0.010171	0.000201	2.343212%	0.009927	0.000181	2.306356%	0.010012	0.000194	2.310602%
	Average Ensemble	0.009990	0.000189	2.303078%	0.010072	0.000201	2.307875%	0.010204	0.000200	2.354789%	0.009879	0.000178	2.301266%	0.010075	0.000194	2.329218%
	Stacked Ensemble	0.009885	0.000185	2.282340%	0.010090	0.000200	2.315573%	0.010186	0.000199	2.354201%	0.009765	0.000174	2.280481%	0.010040	0.000192	2.325996%

Table 7

ANOVA test results of the RF and SVR base models

Currency Pair	Forecast horizon	ANOVA $p$ -value
EUR/AUD	$+$ 1 Minutes	0.0171
	$+$ 5 Minutes	0.0103
	$+$ 10 Minutes	0.0128
	$+$ 15 Minutes	0.0105
	$+$ 30 Minutes	0.011
	$+$ 45 Minutes	0.0182
	$+$ 60 Minutes	0.0169
EUR/GBP	$+$ 1 Minutes	0.0132
	$+$ 5 Minutes	0.0195
	$+$ 10 Minutes	0.0122
	$+$ 15 Minutes	0.0144
	$+$ 30 Minutes	0.0138
	$+$ 45 Minutes	0.0177
	$+$ 60 Minutes	0.018
EUR/PLN	$+$ 1 Minutes	0.0119
	$+$ 5 Minutes	0.0149
	$+$ 10 Minutes	0.0145
	$+$ 15 Minutes	0.0165
	$+$ 30 Minutes	0.0159
	$+$ 45 Minutes	0.0175
	$+$ 60 Minutes	0.0134

Additionally, the ANOVA test results, which provide statistical validation of the distinct performance of the RF and SVR base models, are encapsulated in Table 7. This rigorous statistical examination reinforces the decision to utilize these two distinct models within the stacked ensemble, enhancing both the robustness and generalizability of predictions.

Figure 7.

The actual values vs. predicted output (left) and the scatter graph of the actual values vs. the predicted output of the next minute exchange rate by Stacked ensemble of Random Forest $+$ SVR models for EUR/AUD.

Figure 8.

The actual values vs. predicted output (left) and the scatter graph of the actual values vs. the predicted output of the next 60-minute exchange rate by Stacked ensemble of Random Forest $+$ SVR models for EUR/AUD.

Figure 9.

The actual values vs. predicted output (left) and the scatter graph of the actual values vs. the predicted output of the next minute exchange rate by Stacked ensemble of Random Forest $+$ SVR models for EUR/GBP.

Figure 10.

The actual values vs. predicted output (left) and the scatter graph of the actual values vs. the predicted output of the next 60-minute exchange rate by Stacked ensemble of Random Forest $+$ SVR models for EUR/GBP.

Figure 11.

The actual values vs. predicted output (left) and the scatter graph of the actual values vs. the predicted output of the next minute exchange rate by Stacked ensemble of Random Forest $+$ SVR models for EUR/PLN.

Figure 12.

The actual values vs. predicted output (left) and the scatter graph of the actual values vs. the predicted output of the next 60-minute exchange rate by Stacked ensemble of Random Forest $+$ SVR models for EUR/PLN.

Figures 7–12 depict the comparison of actual versus predicted outputs, as well as scatter diagrams for the next 1 and 60 minutes of exchange rates for the currency pairs EUR/AUD, EUR/GBP, and EUR/PLN, respectively, using the stacked ensembles of Random Forest and SVR models. These visualizations are instrumental in comprehending the performance of the models in predicting immediate and longer-term future exchange rates. From a practical perspective, these models’ outputs, along with their accompanying visualizations, provide valuable insights for decision-making in foreign exchange (Forex) trading and risk management. Therefore, an in-depth analysis and discussion of these results are essential for a comprehensive understanding of the model’s performance and areas of potential improvement.

The examination of the performance metrics of the stacked ensemble model, which combines the Random Forest and SVR models, shows significant insights into its ability to predict the future exchange rates of various currency pairs.

Analyzing the Stacked Ensemble model’s performance across different currency pairs reveals some insightful patterns. For the EUR/AUD pair, the model showed a commendable precision with error metrics such as Mean Squared Error (MSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE) being relatively low for short-term forecasts, particularly at the 1-minute mark. As anticipated, these error metrics progressively increased with longer prediction horizons, reflecting a natural growth in predictive uncertainty. Even at the 60-minute mark, the model maintained its efficacy with the MAPE just over 2%, indicating a commendable level of predictive accuracy, albeit the increment in forecast timeframe.

Turning to the EUR/GBP pair, the Stacked Ensemble model exhibited a superior performance. Despite the common trend of error rates escalating with longer prediction periods, the initial error rates for the EUR/GBP pair, as well as the growth rate of these errors, were notably lower compared to those observed in the EUR/AUD pair. Specifically, the 1-minute forecast started with a low MAPE of just above 0.16%, rising to slightly over 1.64% for the 60-minute prediction, which are the lowest figures among the three currency pairs evaluated. These results suggest a robust model performance when predicting the EUR/GBP exchange rate, potentially indicating a lower level of volatility or increased predictability for this currency pair compared to the others.

Lastly, for the EUR/PLN pair, the Stacked Ensemble model’s performance was somewhat similar to that of the EUR/AUD pair. However, even though the MAPE began just above 0.315% for the 1-minute forecast and slightly exceeded 2.31% for the 60-minute forecast, the model exhibited a high degree of accuracy, maintaining its predictive precision relatively well across all prediction intervals.

It’s important to bear in mind that these results do not necessarily reflect the model’s ability to capture all possible future scenarios. The real world of currency exchange rates is influenced by a myriad of factors including macroeconomic events, market sentiment, geopolitical developments and more. Nevertheless, the model’s ability to provide relatively accurate short-term predictions across multiple currency pairs is a positive indication of its usefulness as a tool in foreign exchange trading and risk management.

This study sets the groundwork for further exploration. While the proposed model shows promising results, additional research could be done to enhance its performance.

5. Conclusion and future work

5.1 Conclusion

This study delves into the application of a stacked ensemble model, combining the strengths of Random Forest (RF) and Support Vector Regression (SVR), in predicting future exchange rates for several currency pairs. Performance evaluation was conducted using established measures such as Mean Squared Error (MSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE). To offer a more holistic perspective, the model’s predictions were compared to those from individual RF and SVR models, as well as an averaged ensemble of the RF and SVR models.

To verify the reliability of the model, a k-fold cross-validation technique was used, with k set at 5. This method allowed a robust evaluation of the model’s predictive performance over different data subsets. Additionally, an Analysis of Variance (ANOVA) test was utilized, aimed at the RF and SVR base models. This test returned p-values in the range of 0.0103 to 0.0195, which strongly supports the distinctiveness of each base model in terms of performance. This validates the decision to construct a stacked ensemble model from two separate and unique base models, thereby increasing the robustness and generalizability of predictions.

The model was evaluated on three different currency pairs: EUR/AUD, EUR/GBP, and EUR/PLN. The findings confirmed the model’s ability to offer fairly accurate short-term predictions, though, as expected, the accuracy diminished as the prediction horizon increased. It was also observed that the model’s performance varied across different currency pairs, indicative of the unique characteristics and dynamics inherent to each pair.

In conclusion, the stacked ensemble model demonstrates significant promise as a valuable tool for predicting foreign exchange rates. Its performance, along with the additional layers of validation, including model comparisons, cross-validation, and ANOVA testing, underscores the potential of ensemble learning techniques. These techniques offer robust insights and enhance predictive performance in intricate, real-world contexts like financial markets.

5.2 Suggestions for future work

While the stacked ensemble model demonstrated promising results, there remains ample opportunity for further research and development to improve upon its performance and adaptability.

• •

Model Diversity: Future research could explore incorporating a greater diversity of base models into the ensemble. Different machine learning algorithms excel in different aspects, and a more diverse ensemble might capture a broader spectrum of patterns in the data.

•

Hyperparameter Tuning: Advanced techniques for hyperparameter tuning could potentially enhance the model performance. Approaches such as Bayesian optimization might be explored for more efficient and effective tuning processes.

•

Feature Engineering: The features used in this study were relatively simple. More sophisticated feature engineering, such as inclusion of lagged variables or technical indicators, could potentially improve the model’s predictive power.

•

External Factors: Incorporating external factors such as macroeconomic indicators or geopolitical events might improve the model’s ability to anticipate major shifts in exchange rates.

•

Comparison with Other Ensemble Techniques: This study focused on stacking as the ensemble technique. Future work could compare the effectiveness of stacking with other ensemble methods like boosting or bagging in the context of exchange rate prediction.

•

Deep Learning Models: Future research could also consider the application of deep learning models, which have shown strong performance in a variety of complex prediction tasks.

The state of the art in foreign exchange prediction can continue to advance by pursuing these and potentially other directions. This advancement aids practitioners in navigating this complex and critically important field.