Clinical trials are essential for discovering new treatments and advancing medical knowledge. However, the high uncertainty of carrying out clinical trials often ends with ineffective results. Therefore, the accurate prediction of clinical trial outcomes has become a significant challenge. Numerous publicly accessible clinical trial reports have been discovered to be beneficial in alleviating this challenge but lack necessary annotations to be formal datasets for deep model training. To address the issue, this paper proposes to construct a new clinical trial dataset by extracting publicly available clinical trial reports from ClinicalTrials.gov and PubMed. In addition, a new two-stage method is proposed for the prediction of clinical trial outcomes across all trial phases. Specifically, our method first employs a prompt template combined with each clinical trial report to prompt a large language model to generate a concise summarization text containing essential information related to the clinical trial outcomes. Subsequently, this summarization text is utilized to train a classifier to predict the outcomes. Extensive experiments were conducted on the dataset, and our method was compared with several state-of-the-art classification models. The results showed that our method achieved the best performance in predicting clinical trial outcomes, especially using small amounts of training data under a data imbalance difficulty.
FuTHuangKXiaoC, et al.HINT: hierarchical interaction network for clinical-trial-outcome predictions. Patterns2022; 3: 10445.
2.
LuoJQiaoZGlassL, et al.Clinicalrisk: a new therapy-related clinical trial dataset for predicting trial status and failure reasons. In: Proceedings of the 32nd ACM international conference on information and knowledge management, 2023, pp.5356–5360.
3.
BlassBE. Basic principles of drug discovery and development. Philadelphia, USA: Elsevier, 2015.
4.
BentleyCCressmanSvan der HoekK, et al.Conducting clinical trials–costs, impacts, and the value of clinical trials networks: a scoping review. Clin Trials2019; 16: 183–193.
5.
FriedmanLMFurbergCDDeMetsDL, et al.Fundamentals of clinical trials. North Bethesda, USA: Springer, 2015.
6.
HeinemannFHuberTMeiselC, et al.Reflection of successful anticancer drug development processes in the literature. Drug Discov Today2016; 21: 1740–1744.
7.
MunosBNiederreiterJRiccaboniM. Improving the prediction of clinical success using machine learning. medRxiv2021.
8.
FollettLGelettaSLaugermanM. Quantifying risk associated with clinical trial termination: a text mining approach. Inf Process Manag2019; 56: 516–525.
9.
GelettaSFollettLLaugermanM. Latent dirichlet allocation in predicting clinical trial terminations. BMC Med Inform Decis Mak2019; 19: 1–12.
10.
JinQTanCChenM, et al.Predicting clinical trial results by implicit evidence integration. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), 2020, pp.1461–1477.
11.
KatsimprasGPaliourasG. Predicting intervention approval in clinical trials through multi-document summarization. In: Proceedings of the 60th annual meeting of the association for computational linguistics (Volume 1: Long Papers), 2022, pp.1947–1957.
12.
LehmanEDeYoungJBarzilayR, et al.Inferring which medical treatments work from reports of clinical trials. In: Proceedings of the 2019 conference of the north American chapter of the association for computational linguistics: Human language technologies, Volume 1 (Long and Short Papers), 2019, pp.3705–3717.
13.
MarshallIJKuiperJBannerE, et al.Automating biomedical evidence synthesis: Robotreviewer. In: Proceedings of the conference. Association for computational linguistics. Meeting, volume 2017, 2017, p.7. NIH Public Access.
14.
WangZSunJ. Trial2vec: zero-shot clinical trial document similarity search using self-supervision. In: 2022 Findings of the association for computational linguistics: EMNLP 2022, 2022.
15.
GayvertKMMadhukarNSElementoO. A data-driven approach to predicting successes and failures of clinical trials. Cell Chem Biol2016; 23: 1294–1301.
16.
SiahKWKelleyNWBallerstedtS, et al.Predicting drug approvals: the novartis data science and artificial intelligence challenge. Patterns2021; 2: 100312.
17.
WuYLevyMAMicheelCM, et al.Identifying the status of genetic lesions in cancer clinical trial documents using machine learning. BMC Genom2012; 13: 1–9.
18.
GaoJXiaoCGlassLM, et al.Compose: cross-modal pseudo-siamese network for patient trial matching. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, 2020, pp.803–812.
19.
ZhangXXiaoCGlassLM, et al.Deepenroll: patient-trial matching with deep embedding and entailment prediction. In: Proceedings of the web conference 2020, 2020b, pp.1029–1037.
20.
QiYTangQ. Predicting phase 3 clinical trial results by modeling phase 2 clinical trial subject level data using deep learning. In: Machine learning for healthcare conference, 2019, pp.288–303. PMLR.
21.
LoAWSiahKWWongCH. Machine learning with statistical imputation for predicting drug approvals. Available at SSRN 2973611.
22.
LiangJTsouC-H. A novel system for extractive clinical note summarization using EHR data. NAACL HLT 20192019: 46–54.
23.
LinsRDOliveiraHCabralL, et al.The CNN-Corpus: a large textual corpus for single-document extractive summarization. In: Proceedings of the ACM symposium on document engineering 2019, 2019, pp.1–10.
24.
SoseaTZhanHLiJJ, et al.Unsupervised extractive summarization of emotion triggers. In: Proceedings of the 61st annual meeting of the association for computational linguistics (Volume 1: Long Papers), 2023, pp.9550–9569. Association for Computational Linguistics.
25.
AllahyariMPouriyehSAssefiM, et al.Text summarization techniques: a brief survey. Int J Adv Comput Sci Appl2017; 8: 397–405.
26.
ChernIcWangZDasS, et al.Improving factuality of abstractive summarization via contrastive reward learning. In: Proceedings of the 3rd workshop on trustworthy natural language processing (TrustNLP 2023), 2023, pp.55–60.
27.
PuDWangYDembergV. Incorporating distributions of discourse structure for long document abstractive summarization. In: Proceedings of the 61st annual meeting of the association for computational linguistics (Volume 1: Long Papers), 2023, pp.5574–5590. Association for Computational Linguistics.
28.
LewisMLiuYGoyalN, et al.BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Proceedings of the 58th annual meeting of the association for computational linguistics, 2020, pp.7871–7880.
29.
RadfordAWuJChildR, et al.Language models are unsupervised multitask learners. OpenAI Blog2019; 1: 9.
30.
NentidisA. Overview of bioasq 2021: the ninth bioasq challenge on large-scale biomedical semantic indexing and question answering. In: Experimental IR meets multilinguality, multimodality, and interaction: 12th International conference of the CLEF association, CLEF 2021, virtual event, September 21–24, 2021, proceedings, volume 12880, 2021, pp.239. Springer Nature.
31.
WallaceBCSahaSSoboczenskiF, et al.Generating (factual?) narrative summaries of rcts: experiments with neural multi-document summarization. AMIA Summit Transl Sci Proc2021; 2021: 605.
32.
TeamGAnilRBorgeaudS, et al.Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023.
33.
LeeJYDernoncourtF. Sequential short-text classification with recurrent and convolutional neural networks. In: Proceedings of NAACL-HLT, 2016, pp.515–520.
34.
WangJWangZZhangD, et al.Combining knowledge with deep convolutional neural networks for short text classification. In: Proceedings of the 26th international joint conference on artificial intelligence, 2017, pp.2915–2921.
35.
KentonJDM -WCToutanovaLK. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, 2019, pp.4171–4186.
36.
LiuYOttMGoyalN, et al.ROBERTA: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.
37.
SunCQiuXXuY, et al.How to fine-tune bert for text classification? In: Chinese computational linguistics: 18th China national conference, CCL 2019, kunming, China, October 18–20, 2019, proceedings 18, 2019, pp.194–206. Springer.
38.
WangZNgPMaX, et al.Multi-passage bert: A globally normalized bert model for open-domain question answering. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), 2019, pp.5878–5882.
39.
BeltagyIPetersMECohanA. Longformer: the long-document transformer. arXiv preprint arXiv:2004.05150, 2020.
40.
VaswaniAShazeerNParmarN, et al.Attention is all you need. Adv Neural Inf Process Syst2017; 30: 5998–6008.
41.
HePLiuXGaoJ, et al.Deberta: decoding-enhanced bert with disentangled attention. In: International conference on learning representations, 2020.
42.
ZhangJZhaoYSalehM, et al.Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In: International conference on machine learning, 2020a, pp.11328–11339. PMLR.
43.
PaszkeAGrossSMassaF, et al.Pytorch: an imperative style, high-performance deep learning library. Adv Neural Inf Process Syst2019; 32: 8024–8035.
44.
LoshchilovIHutterF. Fixing weight decay regularization in adam, 2018.
