Organizations nowadays rely on intensive software systems to support their business operations but vulnerabilities within these systems can cause potential risks for major disruption. AI-based techniques are now widely considered for vulnera-bility identification; however effectiveness heavily relies on the dataset’s size and quality. These techniques often lack contextual information while processing data and pose challenges in resource-constrained environments. AI models are generally black box in nature which creates additional challenges to understand decision making processes. This work proposes a novel hybrid framework using LLM model based on CodeBERT with integration of fine-tuning and Model-Agnostic Meta-Learning for performing effective vulnerability detection. It includes few-shot learning technique for new vulnerability detection tasks while maintaining high performance on known cases. The approach adopts Explainable AI techniques from four dimensions including attention mechanisms, layer-wise analysis, feature contribution, and model confidence scores to explain model decision making. An experiment demonstrates the framework’s effectiveness, show-ing steady decrease in meta-loss from 0.45 to 0.14, accompanied by increase in support accuracy from 85.2% to 92.5%. These findings establish the proposed framework as a robust and interpretable solution for vulnerability detection and management.
HuangQZhaoT. Data collection and labeling techniques for machine learning. arXiv preprint arXiv:2407.12793 2024.
3.
MahyariAA. Harnessing the power of LLMs in source code vulnerability detection. arXiv preprint arXiv:2408.03489 2024.
4.
TanZChenTZhangZ, et al.Sparsity-guided holistic explanation for LLMs with interpretable inference-time intervention. Proceedings of the AAAI Conference on Artificial Intelligence2024; 38: 21619–21627.
5.
LiGWangPLiuJ, et al.Meta In-Context learning makes large language models better zero and Few-Shot relation extractors. arXiv preprint arXiv:2404.17807 2024.
6.
ZhengHShenLTangA, et al.Learn from Model Beyond Fine-Tuning: a survey. arXiv preprint arXiv:2310.08184 2023.
7.
SinhaSYueYSotoV, et al.MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning. arXiv preprint arXiv:2405.11446 2024a.
8.
WangQLaiKHTangC. Solving combinatorial optimization problems over graphs with BERT-based deep reinforcement learning. Inf Sci (Ny)2022; 619: 930–946.
9.
VertselARumiantsauM. Hybrid LLM/Rule-based Approaches to Business Insights Generation from Structured Data. arXiv preprint arXiv:2404.15604 2024.
10.
YangTet al.AutoHMA-LLM: Efficient task coordination and execution in heterogeneous multi-agent systems using hybrid large language models. IEEE Trans Cognit Commun Network2025; 11: 987–998.
11.
HaurognéJBasheerNIslamS. Vulnerability detection using BERT based LLM model with transparency obligation practice towards trustworthy AI. Mach Learn Appl2024; 18: 100598.
12.
HuYZhangG. MAML-BERT in addressing low-resource text classification tasks. Proceedings Volume 13229, Seventh International Conference on Advanced Electronic Materials, Computers, and Software Engineering (AEMCSE 2024)2024; 112: 132293C.
13.
IslamSAbbaAIsmailU, et al.Vulnerability prediction for secure healthcare supply chain service delivery. Integr Comput Aided Eng2022; 29: 389–409.
14.
SilvestriSIslamSAmelinD, et al.Cyber threat assessment and management for securing healthcare ecosystems using natural language processing. International Journal of Information Security2023; 23: 31–50.
15.
SahooSA. Meta-Learning for Large Language Models: Teaching LLMs to Learn New Tasks with Minimal Data. Available at SSRN 4977093 2024.
16.
WuMVuTQuL, et al.The Best of Both Worlds: Bridging Quality and Diversity in Data Selection with Bipartite Graph. arXiv preprint arXiv:2410.12458 2024a.
17.
WuXZhaoHZhuY, et al.Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Era. arXiv preprint arXiv:2403.08946 2024b.
18.
BhattacharjeeAMoraffahRGarlandJ, et al.LLMs as counterfactual explanation modules: Can ChatGPT explain black-box text classifiers? arXiv preprint arXiv:2309.13340 2023.
19.
MavrepisPMakridisGFatourosG, et al.XAI for All: Can Large Language Models Simplify Explainable AI? arXiv preprint arXiv:2401.13110 2024.
20.
NichollsJKuppaALe-KhacN. Enhancing Illicit Activity Detection using XAI: A Multimodal Graph-LLM Framework. arXiv preprint arXiv:2310.13787 2023.
21.
KhediriASlimiHYahiaouiA, et al.Enhancing Machine Learning Model Interpretability in Intrusion Detection Systems through SHAP Explanations and LLM-Generated Descriptions. In: 2024 6th International Conference on Pattern Analysis and Intelligent Systems (PAIS), EL OUED, Algeria, 2024, pp.1–6.
22.
ZhouSAlonUAgarwalS, et al.CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code. arXiv preprint arXiv:2302.05527 2023b.
23.
JinHLiQ. Fine-tuning pre-trained codeBERT for code search in smart contract. Wuhan Univ J Natur Sci2023; 28: 237–245.
24.
ChaiYZhangHShenB, et al.Cross-domain deep code search with meta learning. In: Proceedings of the 44th International Conference on Software Engineering 2022.
25.
CambriaEMalandriLMercorioF, et al.XAI meets LLMs: A Survey of the Relation between Explainable AI and Large Language Models. arXiv preprint arXiv:2407.15248 2024.
26.
BasheerNIslamSAlwaheidiMKS, et al.Adoption of deep-learning models for managing threat in API calls with transparency obligation practice for overall resilience. Sensors2024a; 24: 4859.
VigJ. Visualizing attention in Transformer-Based Language Representation models. arXiv preprint arXiv:1904.02679 2019.
29.
PasadAChouJCLivescuK. Layer-Wise Analysis of a Self-Supervised Speech Representation Model. In: 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Cartagena, Colombia, 2021, pp. 914–921.
30.
ContrerasJWinterfeldAPoppJ, et al.Spectral zones-based SHAP/LIME: Enhancing interpretability in spectral deep learning models through grouped feature analysis. Anal Chem2024; 96: 15588–15597.
31.
BasheerNPranggonoBIslamS, et al.Enhancing Malware Detection Through Machine Learning Using XAI with SHAP Framework. In: IFIP advances in information and communication technology, 2024b, pp. 316–329.
ChenYZhangHYangX, et al.Improving cross-lingual low-resource speech recognition by task-based meta polyLoss. Comput Speech Lang2024; 87: 101648.
34.
SenderaMPrzewiȩżlikowskiMMiksaJ, et al.The general framework for few-shot learning by kernel hyperNetworks. Machine Vision and Applications 2023; 34: 53.
35.
FinnCAbbeelPLevineS. Model-Agnostic Meta-Learning for fast adaptation of deep networks. arXiv preprint arXiv:1703.03400 2017.
36.
MirandaBYuPWangY, et al.The Curse of Low Task Diversity: on the failure of transfer learning to outperform MAML and their empirical equivalence. arXiv preprint arXiv:2208.01545 2022.
37.
LiangfuZYujieXYongbinG, et al.Multiple dependence representation of attention graph convolutional network relation extraction model. IET Cyber-Phys Syst Theory Appl2023; 9: 247–257.
38.
TursunODenmanSSridharanS, et al.Towards Self-Explainability of Deep Neural Networks with Heatmap Captioning and Large-Language Models. arXiv preprint arXiv:2304.02202 2023.
39.
LubranoMBellahsen-HarrarYFickR, et al.Simple and efficient confidence score for grading whole slide images. arXiv preprint arXiv:2303.04604 2023.
40.
ChenYDingZChenX, et al.DiverseVul: A new vulnerable source code dataset for deep learning based vulnerability detection. arXiv preprint arXiv:2304.00409 2023a.
NguyenVNepalSWuT, et al.SAFE: Advancing large language models in leveraging semantic and syntactic relationships for software vulnerability detection. arXiv preprint arXiv:2409.00882 2024.
43.
ChaytiEMJaggiM. A New First-Order Meta-Learning Algorithm with Convergence Guarantees. arXiv preprint arXiv:2409.03682 2024.
44.
WeltevredeMSpaanMTJBöhmerW. The role of diverse replay for generalisation in reinforcement learning. arXiv preprint arXiv:2306.05727 2023.
45.
KumarEAN. Enhancing robustness and generalization in deep learning models for image processing. Power System Technology2023; 47: 278–293.
XiaYShaoHSNMDengX. VulCoB: A CodeBERT-Based System for Source Code Vulnerability Detection. In: GAIIS ’24: Proceedings of the 2024 International Conference on Generative Artificial Intelligence and Information Security, 2024, pp. 249–252.
48.
NiCShenLXuX, et al.Learning-based Models for Vulnerability Detection: an extensive study. arXiv preprint arXiv:2408.07526 2024.
49.
YangYZhouXMaoR, et al.DLAP: A deep learning augmented large language model prompting framework for software vulnerability detection. J Syst Software2024a; 219: 112234.
50.
SultanaSAfreenSEistyNU. Code Vulnerability Detection: A comparative analysis of emerging large language models. arXiv preprint arXiv:2409.10490 2024.
51.
ZhengTLiuHXuH, et al.Few-vulD: A few-shot learning framework for software vulnerability detection. Comput Secur2024; 144: 103992.
52.
ChenYDingZChenX, et al.DiverseVul: A new vulnerable source code dataset for deep learning based vulnerability detection. arXiv preprint arXiv:2304.00409 2023b.
53.
ZhouXZhangTLoD. Large language model for vulnerability detection: Emerging results and future directions. arXiv preprint arXiv:2401.15468 2024.
