Sage Journals: Discover world-class research

Abstract

This paper develops a novel approach to stock selection by integrating natural language processing techniques with machine learning algorithms to analyze public opinion data in financial markets. Specifically, we employ the Bidirectional Encoder Representations from Transformers (BERT) model to process and classify financial news and social media content, combined with the Light Gradient Boosting Machine (LightGBM) algorithm to select high-potential stocks within identified concept sectors. Using over 18 million Chinese financial text records from 2023 to 2024, we construct a comprehensive framework that captures both market sentiment and stock-specific characteristics. Our strategy consists of three core components: (1) a BERT-based sentiment analyzer that identifies promising concept sectors with strong momentum, (2) a LightGBM-powered stock selection mechanism utilizing a specially designed “Concept-Momentum-Combined” factor alongside conventional financial indicators, and (3) a risk management system combining sentiment anomaly detection with multi-stage Average True Range (ATR) trailing stop mechanisms. Empirical results demonstrate significant outperformance over CSI 800 across various timeframes, with annualized excess returns of 21.55% over a six-month period and maximum drawdowns of only 11.68%. Performance attribution analysis confirms that concept sector selection based on sentiment analysis is the primary driver of excess returns. Our results add to the expanding literature on the use of artificial intelligence in financial markets and provide actionable takeaways for investors who would like to incorporate public opinion data into their investment process.

Keywords

BERT sentiment analysis LightGBM stock selection concept momentum public opinion mining quantitative investment strategy

Get full access to this article

View all access options for this article.

References

Zhou

. The impact of social media sentiment on Chinese stock market: evidence from Guba forum posts. J Financ Econ 2018; 19(1): 36–51.

Chan

Zhang

. Momentum and reversal effects in China's A-share market. Pac Basin Finance J 2016; 40: 384–397.

Baker

Wurgler

. Investor sentiment and the cross‐section of stock returns. J Finance 2006; 61(4): 1645–1680.

Tetlock

. Giving content to investor sentiment: the role of media in the stock market. J Finance 2007; 62(3): 1139–1168.

Devlin

Chang

Lee

, et al. BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, 2019, pp. 4171–4186.

Yang

Zhu

Wang

. BERT-based sentiment analysis for Chinese financial texts. In: Proceedings of the 2020 International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering, 2020, pp. 155–160.

Araci

(2019). FinBERT: financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063.

Fama

. Efficient capital markets: a review of theory and empirical work. J Finance 1970; 25(2): 383–417.

Shiller

. From efficient markets theory to behavioral finance. J Econ Perspect 2003; 17(1): 83–104.

10.

Barberis

Thaler

. A survey of behavioral finance. Handb Econ Finance 2003; 1: 1053–1128.

11.

Bollen

Mao

Zeng

. Twitter mood predicts the stock market. Journal of Computational Science 2011; 2(1): 1–8.

12.

Chen

Jin

. The impact of investor sentiment on stock returns in China's stock market: evidence from social media data. J Financ Res 2019; 42(3): 621–645.

13.

Fisher

Shah

Titman

. Combining value and momentum. J Invest Manag 2016; 14(2): 33–48.

14.

Fama

French

. Common risk factors in the returns on stocks and bonds. J Financ Econ 1993; 33(1): 3–56.

15.

Meng

Finley

, et al. LightGBM: a highly efficient gradient boosting decision tree. Adv Neural Inf Process Syst 2017; 30: 3146–3154.

16.

Chen

Zaki

. Deep learning for financial sentiment analysis: a survey. IEEE Trans Knowl Data Eng 2021; 33(9): 3191–3209.

17.

Sharpe

. Capital asset prices: a theory of market equilibrium under conditions of risk. J Finance 1964; 19(3): 425–442.

18.

Mohammed

Umar

Adams

. Modeling the volatility for some selected beverages stock returns in Nigeria (2012-2021): a garch model approach. Matrix sci math 2022; 6(2): 41–51.

Public sentiment-based stock selection strategy using the BERT model and LightGBM algorithm

Abstract

Keywords

Get full access to this article

References