IN-LAND: Improving large language models’ performance on abbreviation disambiguation

Abstract

Abbreviation disambiguation (AD) aims to select the most appropriate definition from a set of candidates. Large language models (LLMs) can perform various tasks without specific fine-tuning by using demonstration examples of in-context learning (ICL). However, significant challenges lie in harnessing the capabilities of LLMs for the AD task by designing effective ICL techniques. In this paper, we propose an IN-context-Learning-based AbbreviatioN Disambiguation framework (IN-LAND). IN-LAND introduces a novel mechanism for selecting demonstration examples in AD tasks, evaluating examples based on three criteria: strict abbreviation similarity, loose abbreviation similarity, and sentence semantic similarity. By selecting optimal training samples, this framework aims to improve the inference capabilities of LLMs. Our experimental results demonstrate the efficacy of IN-LAND in French, Legal English, and Malay. Compared to the Random K-shot ICL baseline, our method achieves consistent improvements across the average performance of four different LLMs. The impact is particularly notable in less commonly spoken languages. For instance, in Malay, our method improves the average macro F1 score and accuracy by 7.11% and 9.19%, respectively. In French, the increases are 2.69% for the macro F1 score and 3.17% for accuracy. Similarly, for Legal English, our approach enhances the average accuracy rate by 1.04% while maintaining competitive performance in macro F1 score. These results underscore the robust capabilities of LLMs and their ability to generalize effectively across various languages.