Options markets are the platforms where traders buy/sell contracts, are inherently complicated and volatile, presenting difficulties for algorithmic trading systems. Conventional approaches, which rely on human evaluation of financial documents and charts, are frequently ineffective and prone to mistakes. To overcome this constraint, this study suggests an automated options trading system that uses tailored Proximal Policy Optimization (PPO). The system trains an RL agent incorporating time series data, technical indicators (Momentum Indicators, Level Indicators), and a risk-adjusted reward function defined as cumulative returns penalized for exceeding a 5% daily loss. According to our findings, the PPO-based trading strategy outperforms a number of current trading strategies, such as Moving Averages (MA), Relative Strength Index (RSI), Momentum Trading, and provides a success rate of 85%, indicating that total trades resulted in profit. This provides a robust environment for reviewing stock data, conducting trading simulations, and evaluating performance metrics. The core functionality leverages a Proximal Policy Optimization (PPO) model to forecast trading moves based on historical price movements. This highlights the significance of integrating technical indicators with systematic strategy assessment for reliable automated trading systems. Empirical results demonstrate a predictive accuracy of 85% for profitable trades using the PPO model, superior to state-of-the-art. This facilitates data-driven decision-making, enabling users to identify and implement optimal, risk-managed trading strategies.
LiYNiPChangV. Application of deep reinforcement learning in stock trading strategies and stock forecasting. ComputingJun. 2020; 102: 1305–1322.
2.
DangQ-V. Reinforcement Learning in Stock Trading. In: International Conference on Computer Science, Applied Mathematics and Applications, Hanoi, Vietnam, 2019. [Online]. Available: https://hal.science/hal-02306522. Accessed on: Dec. 11, 2023.
3.
StraderTJRozyckiJJRootTH, et al.Machine learning stock market prediction studies: review and research directions. J. Int. Technol. Inf. ManagJan. 2020; 28: 63–83.
4.
LinWXieLXuH. Deep-reinforcement-learning-based dynamic ensemble model for stock prediction. Electronics (Basel)Jan. 2023; 12: 4483. doi: 10.3390/electronics12(21)4483
5.
ShahDIsahHZulkernineF. Stock market analysis: a review and taxonomy of prediction techniques. Int. J. Financ. StudMay 2019; 7: 26.
6.
MnihV, et al.Human-level control through deep reinforcement learning. NatureFeb. 2015; 518: 529–533.
7.
MnihV, et al.“Asynchronous Methods for Deep Reinforcement Learning.” arXiv, Jun. 16, 2016, doi: 10.48550/arXiv.1602.01783.
8.
SchaulTHorganDGregorK, et al.Universal Value Function Approximators. In: Proceedings of the 32nd International Conference on Machine Learning, PMLR, Jun. 2015, pp. 1312–1320. [Online]. Available: https://proceedings.mlr.press/v37/schaul15.html. Accessed on: Jun. 12, 2024.
GanguliD, et al.Predictability and Surprise in Large Generative Models. In: 2022 ACM Conference on Fairness, Accountability, and Transparency, Jun. 2022, pp. 1747–1764, doi: 10.1145/3531146.3533229.
11.
DengYBaoFKongY, et al.Deep direct reinforcement learning for financial signal representation and trading. IEEE Trans. Neural Netw. Learn. SystMar. 2017; 28: 653–664.
Saeed Marzban, et al.“WaveCorr: Correlation-savvy Deep Reinforcement Learning for Portfolio Management,” ar5iv. [Online]. Available: https://ar5iv.labs.arxiv.org/html/2109.07005. Accessed on: Jul. 09, 2024.
14.
GarzaJ. Reinforcement Learning for Stock Option Trading. In: Proc. of the IEEE Artificial Intelligence Conference Series (AICS), Dec. 2023, pp. 10470596. doi: 10.1109/AICS60730.2023.10470596.
15.
KumarPGorantlaSMandadiAR, et al.Analyzing Deep Reinforcement Learning Strategies for Enhanced Profit Generation and Risk Mitigation in Algorithm Stock Trading. In: Proc. of the IEEE International Conference on Recent Trends and Advances in Computing (ICRTAC), Dec. 2023, pp. 10480823. doi: 10.1109/ICRTAC59277.2023.10480823.
16.
BaiTLangQSongS, et al.Portfolio Optimization in Dynamic Markets: Reinforcement Learning for Investment. In: Proc. of the 2022 Chinese Control Conference (CCC), Jul. 2022, pp. 9901810. doi: 10.23919/CCC55666.2022.9901810.
17.
LiYLiuPWangZ. Stock trading strategies based on deep reinforcement learning. Security and PrivacyMar. 2022; 2022: 4698656.
18.
DeviIVNatarajanBPrabuS, et al.Automated Stock Trading using Reinforcement Learning. In: Proc. of the IEEE Conference on Innovative Computing and Information and Communication Technologies (ICIICS), Nov. 2023, pp. 10421071. doi: 10.1109/ICIICS59993.2023.10421071.
19.
KongMSoJ. Empirical analysis of automated stock trading using deep reinforcement learning. Appl SciJan. 2023; 13: 33.
20.
ZhaoLDengBWuL, et al.Deep reinforcement learning for adaptive stock trading: tackling inconsistent information and dynamic decision environments. J Organ End User ComputJan. 2024; 36: 1–27. doi: 10.4018/joeuc.335083.
21.
GeJQinYLiY, et al.Single stock trading with deep reinforcement learning: A comparative study. In: Proc. of the ACM International Conference on Artificial Intelligence and Machine Learning (ICAIL), Feb. 2022, pp. 3529857. doi: 10.1145/3529836.3529857.
22.
ZouJLouJWangB, et al.“A Novel Deep Reinforcement Learning Based Automated Stock Trading System Using Cascaded LSTM Networks,” arXiv preprint arXiv:2212.02721, Dec. 2022. doi: 10.48550/arXiv.2212.02721.
SattarovO, et al.Recommending cryptocurrency trading points with deep reinforcement learning approach. Appl SciFeb. 2020; 10: 1506. Available: https://doi.org/10.3390/app10041506.
25.
AkenJLiangDLinZ, et al.“The application of deep reinforcement learning in stock trading models,” advances in economics. Business and Management ResearchNov. 2023; 39: 1973. Available: https://doi.org/10.54254/2754-1169/39/20231973.
26.
YousefiN. Deep reinforcement learning for Tehran stock trading. Journal of Next Generation Engineering Sciences and Technologies (JNESTNov. 2022; 1: 71. Available: https://doi.org/10.56741/jnest.v1i02.171.
27.
AnsariY, et al.A deep reinforcement learning-based decision support system for automated stock market trading. IEEE AccessDec. 2022; 10: 966229. Available: https://doi.org/10.1109/ACCESS.2022.3226629.
28.
BaiT.LangQ.SongS., et al., “Feature Fusion Deep Reinforcement Learning Approach for Stock Trading,” in Proc. of the 2022 Chinese Control Conference (CCC), Jul. 2022, pp. 9901810. doi: 10.23919/CCC55666.2022.9901810. Available: https://doi.org/10.23919/CCC55666.2022.9901810.
29.
KabbaniTDumanE. Deep reinforcement learning approach for trading automation in the stock market. IEEE AccessJul. 2022; 10: 20369–20379. Available: https://doi.org/10.1109/ACCESS.2022.3203697.
30.
HeYYangYLiY, et al.A Novel Deep Reinforcement Learning-based Automatic Stock Trading Method and a Case Study. In: Proc. of the IEEE International Conference on Blockchain and Cryptocurrency (ICBC), Nov. 2022, pp. 10087066. doi: 10.1109/iGETblockchain56591.2022.10087066. Available: https://doi.org/10.1109/iGETblockchain56591.2022.10087066.
31.
MalibariNKatibIAMehmoodR. Smart robotic strategies and advice for stock trading using deep transformer reinforcement learning. Appl SciDec. 2022; 12: 12526. Available: https://doi.org/10.3390/app122412526.
CuiDDengAXiaZ, et al.Improved deep reinforcement learning algorithms and applications in quantitative trading in extreme stock markets. EAI Endorsed Transactions on Scalable Information SystemsJun. 2023; 10: e1. Available: https://doi.org/10.4108/eai.2-6-2023.2334605.
LiYNiPChangVI. Application of deep reinforcement learning in stock trading strategies and stock forecasting. ComputingDec. 2019; 102: 2573–2588.
36.
FioriniPMFioriniP-G. A Simple Reinforcement Learning Algorithm for Stock Trading. In: 2021 11th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), Cracow, Poland, 2021, pp. 824-830, doi: 10.1109/IDAACS53288.2021.9660900.
37.
YoonYChoJLeeK. “Multi-agent reinforcement learning for cryptocurrency market prediction,” IEEE transactions on systems. Man, and Cybernetics: SystemsAug. 2022; 52: 4123–4136.
38.
ChenHWangLZhangY. Auto-Trader: deep deterministic policy gradient algorithm for stock trading in the US market. Journal of Financial Data Science2021; 3: 45–67. [Online]. Available: https://jfds.pm-research.com/content/3/2/45 Accessed on: Aug. 20, 2024.
39.
LuoSWuJZhouB. Portfolio management using reinforcement learning: a proximal policy optimization approach. IEEE AccessJan. 2020; 8: 2053–2065.
40.
KimHParkMLeeD. Double deep Q-network for forex trading: a risk-minimizing approach. International Journal of Finance & EconomicsApr. 2021; 26: 2018–2034.
41.
YangSWangZLiJ. Hierarchical reinforcement learning for multi-asset trading in volatile markets. Quantitative FinanceMay 2023; 23: 905–921.
ShenXLuJHuangW. Dueling deep Q-networks for high-frequency trading. IEEE Transactions on Neural Networks and Learning SystemsFeb. 2023; 34: 345–367.
45.
XuHZhangLLiuX. Reinforcement learning with twin-delayed DDPG for futures trading in Chinese markets. IEEE AccessJan. 2021; 9: 3478–3490.
46.
PendharkarPCCusatisP. Trading financial indices with reinforcement learning agents. Expert Syst. Appl2018; 103: 1–13.
47.
Marco CorazzaAS. Q-Learning and SARSA: “A comparison between two intelligent stochastic control approaches for financial trading”. Univ. Ca’ Foscari Venice Dept. Econ. Res. Pap2015; 15: 1–23.
48.
JeongGKimHY. Improving financial trading decisions using deep Q-learning: predicting the number of shares, action strategies, and transfer learning. Expert Syst. Appl2018; 117: 125–138.
