Owing to the complexity of the clinical manifestations in patients with osteonecrosis of the jaw (ONJ), particularly in settings lacking oral and maxillofacial surgery, timely and accurate treatment remains challenging. Here, we developed an integrated vision-language interactive system (namely, the Osteonecrosis of the Jaw Treatment Supporting [ONJ-TS] system), which is designed to provide clinical support for the diagnosis and treatment of ONJ. We retrospectively collected panoramic radiography (PR) images and medical records of 366 patients with ONJ to develop the ONJ-TS system. In the ONJ-TS system, the core Qwen2-VL was fine-tuned for natural language question answering on ONJ treatment. To support the primary functionality of the Qwen2-VL, we trained the SwinTransformer and the U-Net to identify ONJ cases and segment the ONJ regions in the PR images, respectively. Human evaluation of the output responses indicated that the ONJ-TS system exceeded the accuracy levels of junior oral and maxillofacial surgeons (OMFSs) with accuracy scores of 0.9598 (95% confidence interval [CI], 0.9351–0.9845) and 0.8835 (95% CI, 0.8432–0.9238), respectively (P = 0.003). Notably, in comparative analyses with the senior OMFSs (accuracy, 0.9959 [95% CI, 0.8432–0.9238]), the ONJ-TS system demonstrated comparable efficacy in disease management (P < 0.001). Extensive experiments showed that the ONJ-TS system outperformed baseline models in generating accurate responses to treatment support. Given its multifaceted capabilities, the ONJ-TS system holds promise as a digital solution for supporting clinical decisions in managing ONJ and enhancing clinical efficiency and treatment accuracy.
AyersJWPoliakADredzeMLeasECZhuZKelleyJBFaixDJGoodmanAMLonghurstCAHogarthM, et al. 2023. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Intern Med. 183(6):589–596.
2.
BatoolINavedNKazmiSMRUmerF. 2024. Leveraging large language models in the delivery of post-operative dental care: a comparison between an embedded GPT model and ChatGPT. BDJ Open. 10(1):48. doi:10.1038/s41405-024-00226-3
Cerdá-AlberichLSolanaJMallolPRibasGGarcía-JuncoMAlberich-BayarriAMarti-BonmatiL. 2023. ‘MAIC-10 brief quality checklist for publications using artificial intelligence and medical images’, Insights Imaging, 14:11.
5.
ChenC-FRFanQPandaR. 2021. Crossvit: cross-attention multi-scale vision transformer for image classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV); 2021 Oct 10–17; Montreal, QC, Canada. Piscataway (NJ): Institute of Electrical and Electronics Engineers (IEEE). p. 347–356.
6.
Di FedeOCanepaFPanzarellaVMauceriRDel GaizoCBedogniAFuscoVTozzoPPizzoGCampisiG, et al. 2021. The treatment of medication-related osteonecrosis of the jaw (MRONJ): a systematic review with a pooled analysis of only surgery versus combined protocols. Int J Environ Res Public Health. 18(16):8432. doi:10.3390/ijerph18168432
7.
DosovitskiyABeyerLKolesnikovAWeissenbornDZhaiXUnterthinerTDehghaniMMindererMHeigoldGGellyS, et al. 2020. An image is worth 16x16 words: transformers for image recognition at scale. arXiv. [Preprint]. 2010.11929. doi:10.48550/arXiv.2010.11929
8.
Gaêta-AraujoHVanderhaeghenOVasconcelosKFCouckeWCoropciucRPolitisCJacobsR. 2021. Osteomyelitis, osteoradionecrosis, or medication-related osteonecrosis of the jaws? Can CBCT enhance radiographic diagnosis?Oral Dis. 27(2):312–319.
9.
GallifantJAfsharMAmeenSAphinyanaphongsYChenSCacciamaniGDemner-FushmanDDligachDDaneshjouRFernandesC, et al. 2025. The TRIPOD-LLM reporting guideline for studies using large language models. Nat Med. 31(1):60–69.
10.
GohEGalloRHomJStrongEWengYKermanHCoolJAKanjeeZParsonsASAhujaN, et al. 2024. Large language model influence on diagnostic reasoning: a randomized clinical trial. JAMA Netw Open. 7(10):e2440969. doi:10.1001/jamanetworkopen.2024.40969
11.
GürsesBOAlpozEŞenerMÇankayaHBoyacıoğluHGüneriP. 2023. A support vector machine-based algorithm to identify bisphosphonate-related osteonecrosis throughout the mandibular bone by using cone beam computerized tomography images. Dentomaxillofac Radiol. 52(4):20220390. doi:10.1259/dmfr.20220390
12.
HeKZhangXRenSSunJ. 2016. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016 June 27–30; Las Vegas, NV. Piscataway (NJ): Institute of Electrical and Electronics Engineers (IEEE). p. 770–778.
13.
HeimesDMarkNAKuchenRPabstABeckerPKyyakSThiemDGESchulzeRKämmererPW. 2023. Evaluation of medication-related osteonecrosis of the jaw (MRONJ) in terms of staging and treatment strategies by dental students at different educational levels. Medicina (Kaunas). 59(2):252. doi:10.3390/medicina59020252
14.
HowardASandlerMChuGChenL-CChenBTanMWangWZhuYPangR, et al. 2019. Searching for MobileNetV3. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV); 2019 Oct 27–Nov 2; Seoul, South Korea. Piscataway (NJ): Institute of Electrical and Electronics Engineers (IEEE). p. 1314–1324.
15.
HuEJShenYWallisPAllen-ZhuZLiYWangSWangLChenW. 2021. LoRA: low-rank adaptation of large language models. arXiv. [Preprint]. 2106.09685. doi:10.48550/arXiv.2106.09685
16.
HuangHZhengOWangDYinJWangZDingSYinHXuCYangRZhengQ, et al. 2023. ChatGPT for shaping the future of dentistry: the potential of multi-modal large language model. Int J Oral Sci. 15(1):29. doi:10.1038/s41368-023-00239-y
17.
HutchinsonMO’RyanFChavezVLathonPVSanchezGHatcherDCIndresanoATLoJC. 2010. Radiographic findings in bisphosphonate-treated patients with stage 0 disease in the absence of bone exposure. J Oral Maxillofac Surg. 68(9):2232–2240.
18.
IsenseeFJaegerPFKohlSAAPetersenJMaier-HeinKH. 2021. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods. 18(2):203–211.
19.
KoYYYangWFLeungYY. 2024. The role of cone beam computed tomography (CBCT) in the diagnosis and clinical management of medication-related osteonecrosis of the jaw (MRONJ). Diagnostics (Basel). 14(16):1700. doi:10.3390/diagnostics14161700
20.
KojimaYSawadaSSakamotoY. 2024. Medication-related osteonecrosis of the lower jaw without osteolysis on computed tomography images. J Bone Miner Metab. 42(1):27–36.
21.
LiJGuanZWangJCheungCYZhengYLimLLLimCCRuamviboonsukPRamanRCorsinoL, et al. 2024. Integrated image-based deep learning and language models for primary diabetes care. Nat Med. 30(10):2886–2896.
22.
LiuZLinYCaoYHuHWeiYZhangZLinSGuoB. 2021. Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV); 2021 Oct 10–17; Montreal, QC, Canada. Piscataway (NJ): Institute of Electrical and Electronics Engineers (IEEE). p. 9992–10002.
23.
Lončar BrzakBHorvat AleksijevićLVindišEKordićIGranićMVidović JurasDAndabak RoguljA. 2023. Osteonecrosis of the jaw. Dent J (Basel). 11(1):23. doi:10.3390/dj11010023
24.
LongJShelhamerEDarrellT. 2015. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2015 June 7–12; Boston, MA. Piscataway (NJ): Institute of Electrical and Electronics Engineers (IEEE). p. 3431–3440.
25.
NicaDFRiviMRoiCITodeaCDDumaVFSinescuC. 2021. Complementarity of photo-biomodulation, surgical treatment, and antibiotherapy for medication-related osteonecrosis of the jaws (MRONJ). Medicina (Kaunas). 57(2):145. doi:10.3390/medicina57020145
26.
OmiyeJAGuiHRezaeiSJZouJDaneshjouR. 2024. Large language models in medicine: the potentials and pitfalls: a narrative review. Ann Intern Med. 177(2):210–220.
27.
PatelSBLamK. 2023. ChatGPT: the future of discharge summaries?Lancet Digit Health. 5(3):e107–e108. doi:10.1016/s2589-7500(23)00021-3
28.
ReidIRCornishJ. 2011. Epidemiology and pathogenesis of osteonecrosis of the jaw. Nat Rev Rheumatol. 8(2):90–96.
29.
RuggieroSLDodsonTBAghalooTCarlsonERWardBBKademaniD. 2022. American Association of Oral and Maxillofacial Surgeons’ position paper on medication-related osteonecrosis of the jaws-2022 update. J Oral Maxillofac Surg. 80(5):920–943.
30.
ShaoJMaJYuYZhangSWangWLiWWangC. 2024. A multimodal integration pipeline for accurate diagnosis, pathogen identification, and prognosis prediction of pulmonary infections. Innovation (Camb). 5(4):100648. doi:10.1016/j.xinn.2024.100648
31.
ShinJWKimJEHuhKHYiWJHeoMSLeeSSChoiSC. 2024. Radiological manifestations and clinical findings of patients with oncologic and osteoporotic medication-related osteonecrosis of the jaw. Sci Rep. 14(1):8744. doi:10.1038/s41598-024-59500-x
32.
SimonyanKZissermanA. 2014. Very deep convolutional networks for large-scale image recognition. arXiv. [Preprint]. 1409.1556. doi:10.48550/arXiv.1409.1556
33.
SinghalKAziziSTuTMahdaviSSWeiJChungHWScalesNTanwaniACole-LewisHPfohlS, et al. 2023. Large language models encode clinical knowledge. Nature. 620(7972):172–180.
34.
SuyamaKOtsuruMNakamuraNMorishitaKMiyoshiTOmoriKMiuraKISoutomeSHayashidaSRokutandaS, et al. 2024. Bone resection methods in medication-related osteonecrosis of the jaw in the mandible: an investigation of 206 patients undergoing surgical treatment. J Dent Sci. 19(3):1758–1769.
35.
SzegedyCLiuWJiaYSermanetPReedSAnguelovDErhanDVanhouckeVRabinovichA. 2015. Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2015 June 7–12; Boston, MA. Piscataway (NJ): Institute of Electrical and Electronics Engineers (IEEE). p. 1–9.
36.
TejaniASKlontzasMEGattiAAMonganJTMoyLParkSHKahnCEJr; CLAIM 2024 Update Panel. 2024. Checklist for artificial intelligence in medical imaging (CLAIM): 2024 update. Radiol Artif Intell. 6:e240300. doi:10.1148/ryai.240300
37.
WangPBaiSTanSWangSFanZBaiJChenKLiuXWangJGeW, et al. 2024. Qwen2-VL: enhancing vision-language model’s perception of the world at any resolution. arXiv. [Preprint]. 2409.12191. doi:10.48550/arXiv.2409.12191
38.
WilkatMSinghDDLutzIMöllmannHGellrichNCRanaM. 2021. Use and evaluation of a computer-assisted examination method for the diagnosis and analysis of medication-related osteonecrosis of the jaw. Craniomaxillofac Trauma Reconstr. 14(1):36–42.
39.
XieEWangWYuZAnandkumarAAlvarezJMLuoP. 2021. SegFormer: simple and efficient design for semantic segmentation with transformers. In: RanzatoMBeygelzimerADauphinYLiangPSWortman VaughanJ, editors. Advances in neural information processing systems 34, 35th Conference on Neural Information Processing Systems; 2021 Dec 6–14; Virtual. Red Hook (NY): Curran Associates. p. 12077–12090.
40.
ZhangXWuCZhaoZLinWZhangYWangYXieW. 2024. Development of a large-scale medical visual question-answering dataset. Commun Med (Lond). 4(1):277. doi:10.1038/s43856-024-00709-2
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
