AMELIA: A family of multi-task end-to-end language models for argumentation

Abstract

Argument mining is a subfield of argumentation that aims to automatically extract argumentative structures and their relations from natural language texts. This article investigates how a single large language model can be leveraged to perform one or several argument mining tasks. Our contributions are two-fold. First, we construct a multi-task dataset by surveying and converting 19 well-known argument mining datasets from the literature into a unified format. Second, we explore various training strategies using Meta AI’s Llama-3.1-8B-Instruct model: (1) fine-tuning on individual tasks, (2) fine-tuning jointly on multiple tasks, and (3) merging models fine-tuned separately on individual tasks. Our experiments show that task-specific fine-tuning significantly improves individual performance across all tasks. Moreover, multi-task fine-tuning maintains strong performance without degradation, suggesting effective transfer learning across related tasks. Finally, we demonstrate that model merging offers a viable compromise: it yields competitive performance while mitigating the computational costs associated with full multi-task fine-tuning.

1. Introduction

Argumentation theory aims to model, analyse, and automate argumentative reasoning by relying on formal representations of arguments and their relationships. In his seminal paper, Dung ¹ introduced the concept of abstract argumentation frameworks (AAFs) where arguments are represented as abstract nodes in a graph and attacks between them are directed edges. He then defined various acceptability semantics (such as preferred, complete, and grounded among others) to determine which subsets of arguments (known as extensions) can be collectively accepted. These acceptability semantics (We refer the reader to the guide by Baroni et al. ⁸¹ for a comprehensive tutorial with guided explanations.), offer specific criteria to select acceptable arguments and enable reasoning under uncertainty and inconsistency.^2,3

Building on Dung’s framework, researchers have massively explored extensions of this abstract model with additional expressive components (such as support relations^4,5 or sets of attacking arguments^6,7) or the development of new semantics (e.g. to assess the strength of arguments in a more gradual manner beyond binary acceptability^8,9).

These research efforts enhance the expressivity of the original framework and also contribute to more diverse applicability in real-world domains such as legal reasoning and political analysis. ¹⁰

However, a key limitation of approaches based on Dung’s AAFs lies in their reliance on the assumption that both arguments and their relations are explicitly extracted/specified. While manual formalization may be feasible at small scale in expert-driven systems, the automatic extraction of arguments and their interrelations from large-scale, unstructured data remains an open research challenge. Furthermore, even when arguments are extracted, it has been demonstrated that humans do not always reach a consensus on the directionality of attacks between arguments. ¹¹

In recent years, argument mining (AM), the task of automatically identifying and extracting argumentative structures from natural language texts, has received growing attention in the natural language processing (NLP) community. ¹² In more details, this task involves detecting argumentative components (such as claims and premises) and the relations between them. Given the pervasive role of argumentation in both written and spoken communication, the development of computational models capable of parsing and evaluating arguments has promising applications in downstream tasks such as decision-making, persuasion, fact-checking, and misinformation detection.^13,14

Early approaches to AM relied on models such as support vector machines (SVMs) and recurrent neural networks (RNNs). ¹⁵ However, the development of deep-learning and transformer-based architectures, allowed AM systems to gradually improve their performance. For instance, Mayer et al. ¹⁶ applied transformer-based models to mine arguments in healthcare-related texts, demonstrating that domain-adapted language models can capture argumentation patterns in specialized corpora. Moreover, with the recent development of large language models (LLMs), recent work has demonstrated the potential of prompting or fine-tuning LLMs specifically for argument mining.^17–20 For instance, Cabessa et al. ¹⁷ showed that appropriately fine-tuned LLMs can outperform traditional architectures in extracting argumentative structure from natural language texts and Stahl et al. ²⁰ developed Arginstruct, which fine-tuned LLMs using argumentation instruction tuning to enhance model performance in argumentative tasks.

However, we argue that these previous approaches mainly focused on individual AM tasks (such as argument component or argument relation classification) limiting their applicability to more end-to-end or open-ended argumentative tasks. Furthermore, while instruction tuning methods, such as the one used in Arginstruct, Stahl et al. ²⁰ show promise for improving LLMs’ performance in argumentative contexts, it largely depends on synthetic, LLM-generated instructions as training data, rather than leveraging real-world, human-annotated argumentation data. This may, in turn, limit the generalizability of the models’ argumentative capabilities in real-world settings.

Thus, this work aims to explore the potential of a single LLM to perform one or several argument mining tasks.

In doing this, we explore two central research questions:

RQ1:

“To what extent does fine-tuning improve the performance of an LLM on argument mining tasks?”

RQ2:

“Can an LLM obtain good performance in multiple argument mining tasks? If yes, how?”

Our several contributions aimed at answering these two questions. Namely, we first surveyed and collected 19 real-world AM datasets from the literature and converted them into a standardized format suitable to train LLMs. Next, we fine-tuned and evaluated Meta AI’s Llama-3.1-8B-Instruct LLM on eight identified AM tasks. Lastly, to explore the capabilities of LLMs to perform multiple AM tasks, we explored (1) a model merging approach to combine the previously fine-tuned models and (2) fine-tuning jointly on multiple AM tasks.

Our experiments show that task-specific fine-tuning significantly improves individual performance across all tasks. Moreover, multi-task fine-tuning maintains strong performance without degradation, suggesting effective transfer learning across related tasks. Finally, we demonstrate that model merging offers a viable compromise: it yields competitive performance while mitigating the computational costs associated with full multi-task fine-tuning.

This report is structured as follows. In Section 2, we introduced the background and related work on LMMs and argument mining. In Section 3, we describe the refinement of the existing dataset and the creation of our multi-task dataset. In Section 4, we explain the training regiment for the several LLMs in the context of AM and their evaluations. Finally, we conclude in Section 5.

2. Related work

In this section, we begin with a brief overview of prior work on LLMs and their connection to AM. We then narrow our focus to recent studies that specifically apply LLMs to AM tasks. Finally, we present the various AM datasets available in the literature, which serve as the foundation for our experiments in the following sections.

2.1. Large language models

A language model is a computational system designed to generate or understand human language by estimating the probability of word sequences. This allows the model to predict, complete, or generate coherent text based on a given context. Early language models were purely statistical, relying on methods such as $n$ -gram or skip-gram models. ²¹ Over time, these approaches gave way to neural models, first using RNNs and later advancing to transformer-based architectures, which now form the backbone of modern LLMs. Among transformer-based models, BERT (bidirectional encoder representations from transformers) stands out as a prominent encoder-only model. It introduced masked language modelling and bidirectional attention, achieving state-of-the-art performance on a wide range of natural language understanding tasks through fine-tuning. In 2019, GPT-2 ²² sparked widespread public interest in generative language models. Built on a decoder-only architecture and trained using an autoregressive objective, GPT-2 demonstrated that scaling up both model size and training data could lead to strong performance across diverse tasks, without the need for task-specific supervision.

Nowadays, we call large language models, generative language models with many parameters (usually in billions) and trained on a vast amount of textual data to perform a wide range of natural language tasks. The release of OpenAI’s GPT-3 ²³ further demonstrated that increasing model scale improved zero-shot and few-shot performances. This led to the development of various open-source/closed-sourced models such as Meta AI’s Llama models, ²⁴ Mistral-7B, ²⁵ Alibaba’s Qwen2.5, ²⁶ and OpenAI’s GPT-4 ²⁷ among plenty of others which further improved performance and exhibited improved alignment and reasoning capabilities. Further improvement can be seen in models such as Claude, ²⁸ which integrated architectural changes to support extended thinking and Cogito, ²⁹ which explores human-inspired reasoning architecture. This race to better reasoning has been seen in other models such as DeepSeek-R1 ³⁰ and Deepseek-V3, ³¹ which emphasize reasoning through reinforcement learning training, and Alibaba’s QwQ model, ³² designed for code generation and theorem proving.

As LLMs became more ubiquitous, controlling their performances for specific tasks became essential. On the one hand, research on prompt engineering, such as Chain-of-Thought (CoT) prompting, ³³ which enhances reasoning by guiding an LLM to generate intermediate steps, has been shown to enhance accuracy on reasoning-intensive tasks. This was followed by more structured techniques like Graph-of-Thought (GoT) ³⁴ and Thread-of-Thought (ToT), ³⁵ which aim to improve interpretability and robustness. On the other hand, supervised fine-tuning (SFT) and instruction tuning have also emerged as powerful methods to improve LLM performances. Chung et al. ³⁶ have shown that scaling instruction tuning improves generalization and task transfer, while PEFT (performance effective fine tuning) techniques such as LoRA ³⁷ (and its many variants) achieve performance comparable to full fine-tuning with greatly reduced computational cost.

2.2. AM and LLMs

AM is a field of computational argumentation that aims to automatically identify and structure argumentative discourse in natural language texts. The main AM tasks include the detection of argumentative components (e.g. claims or premises) or the classification of relations between them. ¹² Note that in this work, we also study argument quality assessment, which is one of the major related AM tasks. It consists in assessing the quality of claims and their revisions. ³⁸ Other variants are post quality ³⁹ and overall quality assessment. ⁴⁰

Early work in AM relied on structured prediction methods and neural architecture, such as structured SVMs and RNNs for component identification and relation prediction ¹⁵ and a relation-based approach to capture argumentative structure. ⁴¹ We also note that relation classification can also be improved via the use of argumentation scheme and formal logic. ⁴² More recent approaches leverage transformer-based architecture, which demonstrate strong performance across a range of argument mining tasks, with applications in domains such as healthcare, ¹⁶ political debates analysis, ⁴³ and fallacy detection.^44,45

End-to-end frameworks, which perform multiple AM tasks on natural language texts, have also been explored. For example, Lenz et al. ⁴⁶ introduce a system that automatically transforms natural language text into an argument graph while Morio et al. ⁴⁷ and Schulz et al. ⁴⁸ propose models trained via multi-task learning to perform AM in low-resource settings. To help reduce the need for a large amount of data, cross-corpora and training in low-resource settings have also been studied, with methods using generalizability and transfer learning.^39,49,50

Generative approaches have also been in argumentation as a text-to-text generation task ⁵¹ or to perform structured extraction of complex argumentative structure, like argument quadruplet extraction. ⁵² We refer the interested reader to the seminal paper by Chen et al. ⁵³ on the potential of LLMs in computational argumentation. However, in this work, we restrict the scope of our investigation and purposely do not put our focus on generative tasks.

The use of LLMs in AM has shown progress across core tasks (such as argument component identification, relation classification, fallacy detection and quality assessment). Initial explorations have demonstrated the potential of LLM in capturing argumentative structures, while recent evaluations have assessed their performance across precise sub-tasks.^19,54,55

However, LLMs showed several limitations, such as difficulties in reliably detecting argumentative fallacies in natural settings ⁵⁶ or logical inconsistency in generated fallacy annotation. ⁵⁷ As a solution, fine-tuning and in-context learning have been explored to adapt LLMs to AM. Cabessa et al.^17,58 showed that fine-tuned models improve in the task of argument component classification and relation classification. An instruction-tuned variant such as ArgInstruct Stahl et al., ²⁰ further enhances LLM capabilities by aligning them more closely with the reasoning requirement of argumentative tasks. However, they only use synthetic LLM-generated instructions to perform their instruction tuning. We argue that this may limit the generalizability of the models’ argumentative capabilities in real-world settings. In our approach, we make use of real datasets from the literature which we introduce in the next section.

2.3. Survey of datasets in AM

In the following, we present the 19 datasets collected in the literature, which we will convert and use for fine-tuning LLMs. They are presented in alphabetical order, with illustrative examples provided for only a subset for the sake of brevity.

AbstRCT

Proposed by Mayer et al., ¹⁶ this dataset targets AM in healthcare. It includes 500 medical texts annotated for argumentative component (major claim, claim and premises) and argument relations (attack or support). The dataset highlights challenges like domain-specific vocabulary, evidence scarcity and interpretability.

AQM

Guo et al. ⁵² presented AQM as a dataset for their Argument Quadruplet Extraction (AQE) task, which involves identifying four elements in a statement: the topic, stance, opinion and rationale. The data consist of 34,369 sentences from 801 articles annotated for three argument components: claim, evidence and stance.

ArgSum

Li et al. ⁵⁹ introduced ArgSum as a comprehensive multi-task dataset designed for end-to-end argument summarization and evaluation. It contains user-generated discussions with associated stances, generated summaries, and human or model-generated quality judgments. The dataset supports carrying out tasks such as argument component extraction, stance detection, summarization and summary evaluation. Its structure supports joint training across those tasks, enabling multi-task learning.

ComArg

Boltužić and Šnajder ⁶⁰ developed this dataset by extracting data from online debate forums and social platforms. It includes 2,436 comments annotated for stance and argument recognition. This dataset focuses on short, user-generated responses to controversial questions. This dataset supports tasks such as argument relation classification and stance detection.

CoCoLoFa

Yeh et al. ⁵⁷ introduced CoCoLoFa as a dataset containing 7,706 news comments from 648 news articles annotated for eight common logical fallacies (each comment is annotated with one fallacy among the eight selected fallacies), which were verified with the help of an LLM and human annotators. It covers various fallacy types and is situated in real-world opinionated discussions, such as reader comments on a news article.

Dagstuhl-15512 ArgQuality

Wachsmuth et al. ⁴⁰ proposed this dataset to assess argument quality in natural language across several dimensions such as clarity, cogency, sufficiency and effectiveness. The dataset consists of 320 arguments rated by human annotators over the different quality dimensions. This dataset helps quantify what makes an argument ‘good’ and ‘persuasive’.

FEVER

Thorne et al. ⁶¹ compiled a dataset consisting of 185,445 claims for fact verification by pairing claims with evidence from Wikipedia. Though primarily designed for factual verification, the dataset’s structure overlaps with tasks in AM. Each claim is annotated as ‘supported’, ‘refuted’ or ‘not enough information’ based on the retrieved evidence.

IAM

Cheng et al. ⁶² presented IAM, a large-scale dataset designed for multiple tasks of AM, including argument component identification, relation classification and stance detection. It includes 69,666 sentences spanning multiple domains and having extensive annotations. IAM supports both individual and multi-task learning. It is one of the most comprehensive datasets currently available for training and benchmarking AM models.

IBM claim-polarity

Bar-Haim et al. ⁶³ proposed this dataset of 2,394 claims associated with 55 topics annotated for stance classification. The claims are labelled for polarity (support, oppose and neutral) and contextual dependency.

IBM type

Aharoni et al. ⁶⁴ introduced this dataset with claim and evidence annotations across 33 controversial topics, including 2,883 arguments across 586 documents from Wikipedia. The focus is on distinguishing argument components and associating claims with relevant evidence.

IBM claim

Levy et al. ⁶⁵ constructed this dataset, consisting of 2500 claims, by extracting data from web sources associated with 50 distinct topics. It focused on claim detection and aims to bridge the gap between large, noisy web data and structured argumentative search.

IBM evidence

Shnarch et al. ⁶⁶ presented this dataset of 5,785 sentences annotated for evidence (either supporting or contesting the topic) across 83 topics.

IBM argument

Shnarch et al. ⁶⁷ proposed this dataset to address the challenge of domain adaptation in AM. The datasets enable models to generalize to unfamiliar domains while retaining interpretability. The dataset includes 700 sentences annotated as to whether they contain an argument for the given topic. The sentence annotations span across 20 topics, and the rules provide insight into how argument patterns across multiple domains.

MAFALDA

Helwe et al. ⁴⁵ released MAFALDA as a benchmark dataset for logical fallacy detection and classification, covering over 30 fallacy types. It contains a mix of real and synthetic texts, annotated by experts and LLM-assisted crowd-workers. It includes 200 texts in which sentences have been with one or more fallacies. We give a single entry of this dataset in Example 1.

Example 1

MAFALDA entry identifying two fallacies (false dilemma and hasty generalization) from a post claiming that because a bar in Thurles wasn’t attacked over an ad showing Jesus with a pint, Christians are not as sensitive as Muslims.

Microtext part 1

Peldszus and Stede ⁶⁸ released a structured dataset composed of 112 microtexts, each containing a full argument annotated (claim and premises) and their relations (attack and support). We give a single entry of this dataset in Example 2.

Example 2

Microtext part 1 entry about the topic of waste separation. Five elementary discourse units (edu) are identified and associated with argumentative discourse units (adu). Relations between argumentative discourse units and their relations are specified in $c_1,c_2,c_3$ and $c_4$ . The corresponding micro-level argument graph is represented in Figure 1.

Figure 1.

Representation of the micro-level argument graph on the topic of waste separation from Microtext part 1.

Microtext part 2

Skeppstedt et al. ⁶⁹ extended Microtext part 1 with 171 texts by crowd-sourcing new argumentative texts under controlled prompts. The goal was to enlarge the original dataset while maintaining its clarity and consistency in argument structure.

Nixon-Kennedy debates

Menini et al. ⁴³ curated and annotated the 1960 US Nixon-Kennedy presidential debates with a focus on argumentation strategies in political speech. The dataset includes 1,907 argument pairs covering five topics annotated for argumentative relations and rhetorical patterns, enabling analysis of persuasive techniques and discourse dynamics.

Node

Cabrio and Villata ⁷⁰ introduced the Node dataset, comprising 260 arguments extracted from online and encyclopedic sources. Each argument is annotated with acceptability judgments based on its coherence and logical structure.

Persuasive essays

Stab and Gurevych ⁷¹ provided a dataset containing persuasive essays annotated with argument components (major claim, claim and premise) and their relations (support and attack). It includes 402 essays written by students and annotated by three annotators (two non-expert and one expert annotator), offering a consistent structure and real-world argumentative writing.

Example 3

Corresponding Json entry for the Microtext part 1 entry described in Example 2.

3. A new dataset for multi-task AM

The 19 datasets identified in Section 2.3 cannot be directly used for training as they possess widely different specificities and formats. In this section, we explain our approach to unify (Section 3.1) and exploit these datasets for eight AM tasks we consider (Section 3.2).

3.1. Unifying existing AM datasets

To facilitate the fine-tuning and testing of LLMs (as well as reproducibility), we first unify each of the datasets under a handcrafted jsonl format. This unique format eliminates the need for dataset-specific parsers. During conversion, we also clean the data (e.g. removing unnecessary tags) and add relevant informational keywords. Example 3 shows how the entry from Example 2 is converted into a json entry.

Beyond structural unification, we also harmonize the annotation labels across datasets belonging to the same task. For most tasks, this process only involves minor renamings (e.g. ‘Anecdotal Evidence’ to $\mathtt{A}\mathtt{N}\mathtt{E}\mathtt{C}\mathtt{D}\mathtt{O}\mathtt{T}\mathtt{A}\mathtt{L}$ ) to ensure a single, consistent label per category.

Three tasks required more substantial adjustments based on the definitions of the original labels and the specific phenomena each task aims to detect:

ACC: ‘Major claim’ and ‘Implicit Claim’ were merged into $\mathtt{C}\mathtt{l}\mathtt{a}\mathtt{i}\mathtt{m}$ .

AR: All pro-support variants (‘add’, ‘Pro’, ‘Implicit support’ and ‘sup’) were mapped to $\mathtt{s}\mathtt{u}\mathtt{p}\mathtt{p}\mathtt{o}\mathtt{r}\mathtt{t}$ , while negative counterparts (‘und’, ‘reb’, ‘Partial-attack’, ‘Con’ and ‘Implicit attack’) were unified as $\mathtt{a}\mathtt{t}\mathtt{t}\mathtt{a}\mathtt{c}\mathtt{k}$ . This reflects the task’s focus on identifying only support versus attack relations.

SD: ‘Pro’ and ‘support’ labels were merged into $\mathtt{F}\mathtt{o}\mathtt{r}$ while ‘Con’, ‘contest’ and ‘attack’ were merged into $\mathtt{A}\mathtt{g}\mathtt{a}\mathtt{i}\mathtt{n}\mathtt{s}\mathtt{t}$ .

To preserve the original datasets, these label modifications are not applied during the jsonl conversion itself. Instead, we apply them dynamically when loading the data for training. We encourage the reader to explore the different converted datasets in our GitHub repository: https://github.com/brunoyun/amelia.

We now introduce the argument mining tasks considered in the article.

3.2. AM tasks considered

All the studied tasks (in light red boxes) and associated datasets (in white boxes) are represented in Figure 2. We selected those AM tasks by surveying the literature on AM and restricting ourselves to tasks which can be converted into classification tasks. Note that some datasets (e.g. AQM) are reused for different tasks in different ways.

Figure 2.

Graph of the different argumentation tasks and the associated datasets.

In the rest of this section, we will describe and formalize each of those tasks.

Argument component classification (ACC)

Argumentative discourse units represent the smallest components within a text that contribute to its argumentative structure. ACC is the task of classifying argument component as either ‘premises’ or ‘claim’. This classification task does not address the distinction between argument and non-argumentative materials. We formalized this task as follows:

Input : A topic $t$ and a sentence $s$

Output : $o\in \{\mathtt{C}\mathtt{l}\mathtt{a}\mathtt{i}\mathtt{m},\mathtt{P}\mathtt{r}\mathtt{e}\mathtt{m}\mathtt{i}\mathtt{s}\mathtt{e}\mathtt{s}\}$

We fine-tuned and evaluated this task on datasets including Microtext parts 1 and 2, Persuasive Essays, and AbstRCT. Example 4 shows an example of input/output for the ACC task from Microtext part 1.

Example 4

Consider this example from Microtext part 1.

Input :

$t$ : ‘introduce capital punishment’

$s$ : ‘If Germany were to introduce the death penalty, convicted felons could not be paroled or break out of prison and commit new felonies’

Output : $\mathtt{P}\mathtt{r}\mathtt{e}\mathtt{m}\mathtt{i}\mathtt{s}\mathtt{e}\mathtt{s}$

The sentence $s$ provides a reason supporting the topic $t$ , rather than asserting the main point or an opinion about the topic, thus leading to the classification as ‘Premises’.

Claim detection (CD)

A claim is a statement that asserts something to be true or false. In AM, a claim serves as a central component that forms the basis of reasoning and debate. The CD task aims to identify and extract claims relevant to a given debate’s topic from texts. We formalized this task as follows:

Input : A topic $t$ and sentence $s$

Output : $o\in \{\mathtt{C}\mathtt{l}\mathtt{a}\mathtt{i}\mathtt{m},\mathtt{N}\mathtt{o}\mathtt{n}$ - $\mathtt{C}\mathtt{l}\mathtt{a}\mathtt{i}\mathtt{m}\}$

We fine-tuned and evaluated the CD task on datasets including IAM Claim, IBM Claim, and IBM Argument. Example 5 shows an example of input/output for the CD task from IAM Claim.

Example 5

Consider this example from IAM Claim.

Input :

$t$ : ‘Shouldn’t the sale of violent video games be banned’.

$s$ : ‘It has nothing to do with video games or Paxil, and my son’s no murderer’.

Output : $\mathtt{N}\mathtt{o}\mathtt{n}$ - $\mathtt{C}\mathtt{l}\mathtt{a}\mathtt{i}\mathtt{m}$

Despite being an opinionated sentence, $s$ does not express a claim about the topic.

Evidence detection (ED)

An evidence refers to any information or data that either supports or challenges a claim. In AM, the task of evidence detection focuses on the identification and extraction of relevant pieces of text that help validate or refute claims. We formalized ED as follows:

Input : A topic $t$ , a claim $c$ , and a sentence $s$

Output : $o\in \{\mathtt{E}\mathtt{v}\mathtt{i}\mathtt{d}\mathtt{e}\mathtt{n}\mathtt{c}\mathtt{e},\mathtt{N}\mathtt{o}\mathtt{n}$ - $\mathtt{e}\mathtt{v}\mathtt{i}\mathtt{d}\mathtt{e}\mathtt{n}\mathtt{c}\mathtt{e}\}$

We fine-tuned and evaluated the evidence detection task on the ArgSum Evidence, IAM Evidence, and IBM Evidence datasets. Example 6 shows an example of input/output for the ED task from ArgSum Evidence.

Example 6

Consider this example from ArgSum Evidence.

Input :

$t$ : ‘We should ban private military companies’

$c$ : ‘Private military companies’ main interest is profit’

$s$ : ‘A report by Human Rights Watch in 2006 found that private military companies often prioritize profit over other considerations, including ethical and legal considerations’.

Output : $\mathtt{E}\mathtt{v}\mathtt{i}\mathtt{d}\mathtt{e}\mathtt{n}\mathtt{c}\mathtt{e}$

Here, $s$ cite a report that validate the claim $c$ .

Argument relation (AR) classification

The objective of AR classification is to determine whether a given pair of arguments is connected through an argumentative relationship. Given a pair of arguments (a source argument and a target argument), the task is to classify the relationship from the source argument to the target argument as either ‘attack’, ‘support’ or ‘no relation’. Formally, this task is described as follows:

Input : A topic $t$ , a source argument $a_{src}$ , and a target argument $a_{trg}$

Output : $o\in \{\mathtt{a}\mathtt{t}\mathtt{t}\mathtt{a}\mathtt{c}\mathtt{k},\mathtt{s}\mathtt{u}\mathtt{p}\mathtt{p}\mathtt{o}\mathtt{r}\mathtt{t},\mathtt{n}\mathtt{o}\mathtt{r}\mathtt{e}\mathtt{l}\mathtt{a}\mathtt{t}\mathtt{i}\mathtt{o}\mathtt{n}\}$

We fine-tuned and evaluated the argument relation classification task on datasets including Microtext parts 1 and 2, Persuasive Essays, AbstRCT, Nixon-Kennedy Debates, Node, IBM Claim-polarity and ComArg. Example 7 shows an example of input/output for the AR task from node.

Example 7

Consider this example from Node.

Input :

$t$ : ‘Tablet’

$a_{src}$ : ‘Tablets help students learn more material faster than textbooks. Technology-based instruction can reduce the time students take to reach a learning objective by 30%–80%, according to the US Department of Education and studies by the National Training and Simulation Association’.

$a_{trg}$ : ‘People who read print text comprehend more, remember more, and learn more than those who read digital text’.

Output : $\mathtt{a}\mathtt{t}\mathtt{t}\mathtt{a}\mathtt{c}\mathtt{k}$

$a_{src}$ disagrees with $a_{trg}$ and uses a study to attack $a_{trg}$ .

Evidence type (ET) classification

The ETs refer to the different categories of evidence that can either support or challenge a claim. Common types of evidence include anecdotal, expert opinion, explanation and study. Formally, this task is described as follows:

Input : A topic $t$ , a claim $c$ , and an evidence $e$

Output : $o\in \{\mathtt{N}\mathtt{O}\mathtt{N}\mathtt{E},\mathtt{A}\mathtt{N}\mathtt{E}\mathtt{C}\mathtt{D}\mathtt{O}\mathtt{T}\mathtt{A}\mathtt{L},\mathtt{E}\mathtt{X}\mathtt{P}\mathtt{E}\mathtt{R}\mathtt{T},$ $\mathtt{E}\mathtt{X}\mathtt{P}\mathtt{L}\mathtt{A}\mathtt{N}\mathtt{A}\mathtt{T}\mathtt{I}\mathtt{O}\mathtt{N},$ $\mathtt{S}\mathtt{T}\mathtt{U}\mathtt{D}\mathtt{Y}\}$

This task was fine-tuned and evaluated on datasets including ArgSum Evidence Type, IBM Type and AQM. Example 8 shows an example of input/output for the ET task from AQM.

Example 8

Consider this example from AQM.

Input :

$t$ : ‘Shouldn’t the sale of violent video games be banned’.

$c$ : ‘Conversely, playing violent video games had significantly more hurtful behaviors in children than the children who played prosocial games’.

$e$ : ‘Results indicated that playing prosocial games significantly more helpful behaviors in children than those who played violent video games’.

Ouput : $\mathtt{S}\mathtt{T}\mathtt{U}\mathtt{D}\mathtt{Y}$

Here, the evidence $e$ support the claim $c$ using the result from a study.

Stance detection (SD)

A stance reflects a point of view on a debated subject, expressed as either support or opposed. The task of SD is to determine whether an argument supports or opposes a specific topic. Formally, this task is described as follows:

Input : A topic $t$ and a sentence $s$

Output : $o\in \{\mathtt{F}\mathtt{o}\mathtt{r},\mathtt{A}\mathtt{g}\mathtt{a}\mathtt{i}\mathtt{n}\mathtt{s}\mathtt{t}\}$

We fine-tuned and evaluated the SD task using datasets such as IBM Claim-Polarity, ComARg, IAM Stance, FEVER and AQM. Example 9 shows an example of input/output for the SD task from IAM Stance.

Example 9

Consider this example from IAM Stance.

Input :

$t$ : ‘Should all museums be opened for free’.

$s$ : ‘Free access is essential to provide freedom of cultural and educational opportunity’.

Output : $\mathtt{F}\mathtt{o}\mathtt{r}$

The sentences $s$ agree with the topic, thus leading to the classification $\mathtt{F}\mathtt{o}\mathtt{r}$ .

Fallacies detection (FD)

A fallacy is an argument where the premises do not entail the conclusion. The goal of fallacy detection is to identify whether a given argument contains a fallacy or not. In the latter case, the output is $\mathtt{n}\mathtt{o}\mathtt{n}\mathtt{e}$ . If the argument is identified as fallacious, the task further involves classifying the kind of fallacy it contains. Formally, this task is described as follows:

Input : A sentence $s$

Output : $o\in \{\mathtt{n}\mathtt{o}\mathtt{n}\mathtt{e},\mathtt{a}\mathtt{p}\mathtt{p}\mathtt{e}\mathtt{a}\mathtt{l}\mathtt{t}\mathtt{o}\mathtt{f}\mathtt{e}\mathtt{a}\mathtt{r},$ $\mathtt{h}\mathtt{a}\mathtt{s}\mathtt{t}\mathtt{y}\mathtt{g}\mathtt{e}\mathtt{n}\mathtt{e}\mathtt{r}\mathtt{a}\mathtt{l}\mathtt{i}\mathtt{z}\mathtt{a}\mathtt{t}\mathtt{i}\mathtt{o}\mathtt{n},$ $\mathtt{a}\mathtt{p}\mathtt{p}\mathtt{e}\mathtt{a}\mathtt{l}\mathtt{t}\mathtt{o}\mathtt{w}\mathtt{o}\mathtt{r}\mathtt{s}\mathtt{e}\mathtt{p}\mathtt{r}\mathtt{o}\mathtt{b}\mathtt{l}\mathtt{e}\mathtt{m},$ $\mathtt{a}\mathtt{p}\mathtt{p}\mathtt{e}\mathtt{a}\mathtt{l}\mathtt{t}\mathtt{o}\mathtt{a}\mathtt{u}\mathtt{t}\mathtt{h}\mathtt{o}\mathtt{r}\mathtt{i}\mathtt{t}\mathtt{y},$ $\mathtt{f}\mathtt{a}\mathtt{l}\mathtt{s}\mathtt{e}\mathtt{c}\mathtt{a}\mathtt{u}\mathtt{s}\mathtt{a}\mathtt{l}\mathtt{i}\mathtt{t}\mathtt{y},$ $\mathtt{a}\mathtt{p}\mathtt{p}\mathtt{e}\mathtt{a}\mathtt{l}\mathtt{t}\mathtt{o}\mathtt{t}\mathtt{r}\mathtt{a}\mathtt{d}\mathtt{i}\mathtt{t}\mathtt{i}\mathtt{o}\mathtt{n},$ $\mathtt{a}\mathtt{d}\mathtt{p}\mathtt{o}\mathtt{p}\mathtt{u}\mathtt{l}\mathtt{u}\mathtt{m},$ $\mathtt{g}\mathtt{u}\mathtt{i}\mathtt{l}\mathtt{t}\mathtt{b}\mathtt{y}\mathtt{a}\mathtt{s}\mathtt{s}\mathtt{o}\mathtt{c}\mathtt{i}\mathtt{a}\mathtt{t}\mathtt{i}\mathtt{o}\mathtt{n},$ $\mathtt{c}\mathtt{a}\mathtt{u}\mathtt{s}\mathtt{a}\mathtt{l}\mathtt{o}\mathtt{v}\mathtt{e}\mathtt{r}\mathtt{s}\mathtt{i}\mathtt{m}\mathtt{p}\mathtt{l}\mathtt{i}\mathtt{f}\mathtt{i}\mathtt{c}\mathtt{a}\mathtt{t}\mathtt{i}\mathtt{o}\mathtt{n},$ $\mathtt{f}\mathtt{a}\mathtt{l}\mathtt{s}\mathtt{e}\mathtt{d}\mathtt{i}\mathtt{l}\mathtt{l}\mathtt{e}\mathtt{m}\mathtt{a},$ $\mathtt{a}\mathtt{p}\mathtt{p}\mathtt{e}\mathtt{a}\mathtt{l}\mathtt{t}\mathtt{o}\mathtt{r}\mathtt{i}\mathtt{d}\mathtt{i}\mathtt{c}\mathtt{u}\mathtt{l}\mathtt{e},$ $\mathtt{f}\mathtt{a}\mathtt{l}\mathtt{s}\mathtt{e}\mathtt{a}\mathtt{n}\mathtt{a}\mathtt{l}\mathtt{o}\mathtt{g}\mathtt{y},$ $\mathtt{s}\mathtt{l}\mathtt{i}\mathtt{p}\mathtt{p}\mathtt{e}\mathtt{r}\mathtt{y}\mathtt{s}\mathtt{l}\mathtt{o}\mathtt{p}\mathtt{e},$ $\mathtt{a}\mathtt{p}\mathtt{p}\mathtt{e}\mathtt{a}\mathtt{l}\mathtt{t}\mathtt{o}\mathtt{m}\mathtt{a}\mathtt{j}\mathtt{o}\mathtt{r}\mathtt{i}\mathtt{t}\mathtt{y},$ $\mathtt{a}\mathtt{p}\mathtt{p}\mathtt{e}\mathtt{a}\mathtt{l}\mathtt{t}\mathtt{o}\mathtt{n}\mathtt{a}\mathtt{t}\mathtt{u}\mathtt{r}\mathtt{e},$ $\mathtt{s}\mathtt{t}\mathtt{r}\mathtt{a}\mathtt{w}\mathtt{m}\mathtt{a}\mathtt{n},$ $\mathtt{c}\mathtt{i}\mathtt{r}\mathtt{c}\mathtt{u}\mathtt{l}\mathtt{a}\mathtt{r}\mathtt{r}\mathtt{e}\mathtt{a}\mathtt{s}\mathtt{o}\mathtt{n}\mathtt{i}\mathtt{n}\mathtt{g},$ $\mathtt{e}\mathtt{q}\mathtt{u}\mathtt{i}\mathtt{v}\mathtt{o}\mathtt{c}\mathtt{a}\mathtt{t}\mathtt{i}\mathtt{o}\mathtt{n},$ $\mathtt{a}\mathtt{d}\mathtt{h}\mathtt{o}\mathtt{m}\mathtt{i}\mathtt{n}\mathtt{e}\mathtt{m}\}$

We fine-tuned and evaluated the fallacy detection task on two datasets CoCoLoFa and MAFALDA. Example 10 shows an example of input/output for the FD task from CoCoLoFa. Since MAFALDA includes instances with multiple ground-truth fallacies, we define two sub-tasks, denoted as $F{D}_{Single}$ and $F{D}_{Multi}$ . The first considers cases with a single ground-truth fallacy, while the second accounts for cases with multiple fallacies. In $F{D}_{Multi}$ , a prediction is considered correct if it belongs to the set of ground-truth fallacies. Recall is then computed as the proportion of ground-truth fallacies correctly identified across multiple LLM predictions.

This task is considered more challenging than other AM tasks considered due to the larger variety of fallacy categories (as opposed to many binary/ternary distinctions in the other tasks).

Example 10

Consider this example from CoCoLoFa.

Input :

$s$ : ‘People must come together and take this threat seriously, like this honourable man. Denialism needs to be completely shut down. All the experts agree, climate change is real and is a massive threat to the whole planet. As Obama says, no challenge poses a greater threat to future generations than climate change’.

Output : $\mathtt{a}\mathtt{p}\mathtt{p}\mathtt{e}\mathtt{a}\mathtt{l}\mathtt{t}\mathtt{o}\mathtt{a}\mathtt{u}\mathtt{t}\mathtt{h}\mathtt{o}\mathtt{r}\mathtt{i}\mathtt{t}\mathtt{y}$

The sentence $s$ relies on the opinion of an authority figure who may not have relevant expertise, thus leading to the classification as $\mathtt{a}\mathtt{p}\mathtt{p}\mathtt{e}\mathtt{a}\mathtt{l}\mathtt{t}\mathtt{o}\mathtt{a}\mathtt{u}\mathtt{t}\mathtt{h}\mathtt{o}\mathtt{r}\mathtt{i}\mathtt{t}\mathtt{y}$ .

Argument quality (AQ) assessment

Argument quality refers to how good an argument is; it indicates the degree to which an argument is considered strong and effective. This quality is evaluated based on 15 different quality dimensions: overall quality, local acceptability, appropriateness, arrangement, clarity, cogency, effectiveness, global acceptability, global relevance, global sufficiency, reasonableness, local relevance, credibility, emotional appeal, and sufficiency. The goal of argument quality assessment is to evaluate an argument by rating it as low, average or good across each of the 15 quality dimensions. Formally, this task is described as follows:

Input : A topic $t$ , a stance $s$ , an argument $a$ , and a quality dimension $q$ with its textual definition $d_q$

Output : $o\in \{\mathtt{L}\mathtt{o}\mathtt{w},\mathtt{A}\mathtt{v}\mathtt{e}\mathtt{r}\mathtt{a}\mathtt{g}\mathtt{e},\mathtt{H}\mathtt{i}\mathtt{g}\mathtt{h}\}$

We fine-tuned and evaluated this task using the Dagstuhl-15512 ArgQuality dataset. Example 11 shows an example of input/output for the AQ task from Dagstuhl-15512 ArgQuality Corpus.

Example 11

Consider this example from Dagstuhl-15512 ArgQuality Corpus.

Input :

$t$ : ‘personal-pursuit-or-advancing-the-common-good’.

$s$ : ‘advancing-the-common-good’.

$a$ : ‘I feel like advancing the common good is better than personal pursuit because most people will look out for themselves rather than look out for others. Looking out for yourself before looking out for others won’t do a lot of good in the long run. I believe in the what goes around comes around thing, so when you do something for other people before you try to please yourself, not only does it help them, but something good will also come back to you, helping you better yourself’.

$q$ : ‘clarity’.

$d_q$ : ‘Argumentation has a clear style if it uses correct and widely unambiguous language as well as if it avoids unnecessary complexity and deviation from the issue’.

Output : $\mathtt{A}\mathtt{v}\mathtt{e}\mathtt{r}\mathtt{a}\mathtt{g}\mathtt{e}$

Here, $s$ use ambiguous language and add complexity to the argument by contradicting itself in the first sentences, hence the $\mathtt{A}\mathtt{v}\mathtt{e}\mathtt{r}\mathtt{a}\mathtt{g}\mathtt{e}$ clarity.

3.3. Creating AM task-specific datasets

We created task-specific dataset for each of the eight tasks identified in Section 3.2 by extracting the corresponding input/output from several datasets (as shown in Figure 2) using a specific methodology.

First, we divided each of the 19 datasets into three splits: training ( $60\mathrm{\%}$ ), validation ( $20\mathrm{\%}$ ) and test split ( $20\mathrm{\%}$ ).

Then, we perform a sampling on each of those three split to obtain the data needed to fine-tuned, validate, and test the model for our eight tasks. More precisely, the sampling was performed on each corresponding split of each dataset corresponding to a specific task. Given the disparities in dataset size and class distribution, the sampling procedure was designed to ensure an equal number of instances per class within each task, mitigating class imbalance. Additionally, the sampling preserved the original proportion of each dataset split, maintaining the original proportion of examples contributed by each dataset.

More formally, let us consider a task $t\in \{ADUC,CD,ED,ARC,ET,SD,FD,AQ\}$ with the corresponding set of datasets $D_t$ , for example, $D_{FD}=\{CoCoLoFa,MAFALDA\}$ . For a dataset $d\in D_t$ , we use the notation $Class(d)$ for the set of class labels in the dataset. We write $C{L}_{D_t}=\bigcup _{d\in D_t}Class(d)$ for the set of all class labels within $D_t$ . Moreover, for any dataset $d\in D_t$ and class $c\in Class(d)$ , $NumEl(c,d)$ is the number of instances of class $c$ in $d$ .

To retain the original dataset contributions, we compute for each dataset $d\in D_t$ the sampling ratio $r_{c,d,t}$ for a class label $c\in C{L}_{D_t}$ :

$$r_{c,d,t}=\frac{NumEl(c,d)}{\sum _{d^{\prime }\in D_t}NumEl(c,d^{\prime })}.$$

Subsequently, the number of instances of class $c$ to sample from $d$ is computed as follows:

$$sampl{e}_{c,d,t}=r_{c,d,t}\times \frac{n_{sample}}{|C{l}_{D_t}|}$$

with $n_{sample}$ the number of samples needed.

This sampling method was used to generate $4000$ training samples and $800$ samples for both validation and test sets.

Table 1 shows an example of sampling the IAM Claim (69,666 elements), IBM Claim (2500 elements) and IBM Arguments (700 elements) datasets used for the Claim Detection (CD) task. We can notice that the train, validation, and test datasets are balanced but keep the same proportion of elements from each dataset.

Table 1.

Sampling example for the task of claim detection. The validation and test split have the same number of instances.

Split	Train				Validation & Test
Datasets	IAM Claim	IBM Claim	IBM Arguments	Total	IAM Claim	IBM Claim	IBM Arguments	Total
Claim	$1659$	$252$	$89$	$2000$	$333$	$50$	$18$	$401$
Non-Claim	$1933$	$54$	$13$	$2000$	$387$	$11$	$3$	$401$
Total	$3592$	$306$	$102$	$4000$	$720$	$61$	$21$	$802$

4. LLMs for (multi-task) AM

In this section, we describe the experiments to answer our two research questions and our results.

4.1. Experiment setup

For all of our experiments, we further fine-tuned Llama3.1-8B-Instruct, an already fine-tuned version of the base Meta AI’s Llama 3.1-8B for instruction following. Llama 3.1-8B-Instruct was selected for all experiments because it strikes a strong balance between performance, efficiency, and accessibility. The open-source model is large enough to capture complex reasoning and language patterns, making it suitable for tasks that require nuanced understanding, while still being lightweight enough to run reliably with reasonable computational resources. Its instruction-tuned design ensures consistent and helpful responses across diverse prompts, which was essential for maintaining reproducibility and comparability across experiments. Using a single, well-established model throughout also guarantees methodological consistency and avoids confounding effects from model variation.

Moreover, a uniform prompt format for each task was used for both fine-tuning and inference; this format consists of a task description and an explicit specification of the expected output format. We used the special token <|ANSWER|> to delimitate the LLM’s answer. This choice was inspired by the syntax of special tokens used in the Llama3.1 model’s prompt template. The first part of the prompt describes the objectives and the guidelines of the task, followed by the list of all possible output labels. The second part of the prompt ensures that the generated outputs remain within the predefined labels set for the given task. Example 12 shows the prompt used for the SD task. The prompts for each task are provided in Appendix B.

Example 12

You are an expert in argumentation. Your task is to determine whether the given [SENTENCE] is For or Against. Utilise the [TOPIC] as context to support your decision.

Your answer must be in the following format, with only For or Against in the answer section:

$<|$ ANSWER $|>$ <answer> $<|$ ANSWER $|>$ .

[TOPIC]: <topic>

[SENTENCE]: <sentence>

All experiments were conducted on an Ubuntu machine equipped with an AMD EPYC 7443 24-core CPU, an NVIDIA A40 48 GB GPU, and 65 GB of RAM. All fine-tuning performed in this work spanned two epochs with a batch size of 32. We employed Low-Rank Adaptation (LoRA) ³⁷ with a rank of 16 to mitigate computational costs and memory requirements while ensuring sufficient parameters to accommodate diverse tasks.

As the baselines for our evaluation, we employed the Llama3.1-8B-Instruct model in both zero-shot and few-shot settings, along with a DeBERTa model. ⁷² For the few-shot setting, we provided one example per label of the diverse tasks. This resulted in two examples for ACC, CD, ED and SD, three examples for AR, five for ET, 20 for FD and three examples for each of the quality dimensions for AQ.

4.2. Exploiting model merging for (multi-task) AM with LLMs

In this section, we explain how we exploit model merging to create a multi-task AM LLM. The three-steps pipeline for this model (illustrated as $S{T}_{Merged}$ in Figure 3) is as follows:

We first created a collection of eight models, each fine-tuned for a specific AM task, and released them in two formats: Safetensors and GGUF (GGML Universal File). The Safetensors versions enable accurate reproduction of our evaluation results and support potential retraining, while the GGUF format (a compact binary representation that stores both tensors and metadata in a single file) is optimized for efficient local inference on platforms such as Ollama and LM Studio. Both formats are available on Hugging Face (Our trained LLM collection is accessible at https://huggingface.co/collections/brunoyun/amelia-collection-68518343bf75869b53d0d8bd.).

We categorized the different argumentation tasks into three levels of difficulty: hard, medium, and easy. This categorization was based on the performance of the various fine-tuned models across the tasks (see Table 2 in the subsequent section). Specifically, a task is considered easy if most fine-tuned models – including those not specifically trained for that task – achieve over $60\mathrm{\%}$ performance. Tasks for which model performance predominantly falls between $50\mathrm{\%}$ and $60\mathrm{\%}$ are considered medium, while tasks with lower performance are classified as hard. Accordingly, the challenging tasks include fallacies detection (FD), argument quality (AQ) assessment and evidence type (ET) classification. Intermediate tasks comprise argument component classification (ACC) and argument relation (AR) Classification, whereas the easier tasks are claim detection (CD), evidence detection (ED) and stance detection (SD). Notably, although most fine-tuned models achieve only around $40\mathrm{\%}$ on the AR task, we classify it as medium because merged configurations succeed in obtaining strong performance on this task without retaining a high fraction of task-specific vectors.

Subsequently, we merged the eight finely tuned models using the mergekit library. ⁷³ Although there are numerous merging approaches,^74–77 we evaluated different configurations for two of them: DARE ⁷⁷ and DELLA. ⁷⁴ On the one hand, the former performs a random pruning on the task vectors, followed by a rescaling method to match the performance of the original model. The latter, on the other hand, enhances DARE using magnitude-based adaptive pruning, which assigns higher probabilities to parameters with larger magnitudes, followed by DARE-like rescaling. This method is designed to preserve significant modifications while minimizing interference between task vectors.

Figure 3.

Representation of the merging pipeline. All fine-tuned models are available in our Hugging Face collection.

Table 2.

Performance of the fine-tuned models merged in 16-bit and the GGUF models in 8-bit.

Model	ACC	CD	ED	AR	ET	SD	FD ${}_{Single}$	FD ${}_{Multi}$	AQ
Llama-3.1-8B-Instruct zero-shot	$73.52\mathrm{\%}$	$51.50\mathrm{\%}$	$17.06\mathrm{\%}$	$28.32\mathrm{\%}$	$37.41\mathrm{\%}$	$14.10\mathrm{\%}$	$44.07\mathrm{\%}$	$21.77\mathrm{\%}$	$15.10\mathrm{\%}$
Llama-3.1-8B-Instruct few-shot	$75.47\mathrm{\%}$	$67.83\mathrm{\%}$	$64.20\mathrm{\%}$	$35.97\mathrm{\%}$	$49.31\mathrm{\%}$	$80.00\mathrm{\%}$	$48.50\mathrm{\%}$	$17.25\mathrm{\%}$	$31.83\mathrm{\%}$
DeBERTa	$78.64\mathrm{\%}$	$71.59\mathrm{\%}$	$63.97\mathrm{\%}$	$69.17\mathrm{\%}$	$68.75\mathrm{\%}$	$33.33\mathrm{\%}$	$36.82\mathrm{\%}$	/	$48.00\mathrm{\%}$
Llama-3.1-8B-Instruct FT for ACC	$\underset{\_}{89.61\mathrm{\%}}$	$61.35\mathrm{\%}$	$68.25\mathrm{\%}$	$38.51\mathrm{\%}$	$41.43\mathrm{\%}$	$65.82\mathrm{\%}$	$38.43\mathrm{\%}$	$21.58\mathrm{\%}$	$33.07\mathrm{\%}$
Llama-3.1-8B-Instruct FT for CD	$50.18\mathrm{\%}$	$\mathbf{\text{85.16\%}}$	$68.91\mathrm{\%}$	$38.29\mathrm{\%}$	$33.91\mathrm{\%}$	$66.97\mathrm{\%}$	$38.90\mathrm{\%}$	$22.67\mathrm{\%}$	$31.24\mathrm{\%}$
Llama-3.1-8B-Instruct FT for ED	$63.32\mathrm{\%}$	$74.94\mathrm{\%}$	$\mathbf{\text{78.00\%}}$	$28.60\mathrm{\%}$	$38.67\mathrm{\%}$	$68.42\mathrm{\%}$	$39.65\mathrm{\%}$	$18.47\mathrm{\%}$	$29.01\mathrm{\%}$
Llama-3.1-8B-Instruct FT for AR	$50.81\mathrm{\%}$	$59.98\mathrm{\%}$	$67.00\mathrm{\%}$	$\underset{\_}{87.20\mathrm{\%}}$	$35.07\mathrm{\%}$	$76.00\mathrm{\%}$	$35.14\mathrm{\%}$	$25.86\mathrm{\%}$	$27.97\mathrm{\%}$
Llama-3.1-8B-Instruct FT for ET	$56.10\mathrm{\%}$	$67.08\mathrm{\%}$	$61.45\mathrm{\%}$	$26.88\mathrm{\%}$	$\mathbf{\text{75.22\%}}$	$69.82\mathrm{\%}$	$46.78\mathrm{\%}$	$29.68\mathrm{\%}$	$29.03\mathrm{\%}$
Llama-3.1-8B-Instruct FT for SD	$50.93\mathrm{\%}$	$48.88\mathrm{\%}$	$57.62\mathrm{\%}$	$38.26\mathrm{\%}$	$39.17\mathrm{\%}$	$\underset{\_}{94.63\mathrm{\%}}$	$43.23\mathrm{\%}$	$20.99\mathrm{\%}$	$20.39\mathrm{\%}$
Llama-3.1-8B-Instruct FT for FD	$66.58\mathrm{\%}$	$65.13\mathrm{\%}$	$64.50\mathrm{\%}$	$38.64\mathrm{\%}$	$46.83\mathrm{\%}$	$64.32\mathrm{\%}$	$\mathbf{\text{82.92\%}}$	$50.77\mathrm{\%}$	$41.90\mathrm{\%}$
Llama-3.1-8B-Instruct FT for AQ	$74.46\mathrm{\%}$	$59.73\mathrm{\%}$	$68.00\mathrm{\%}$	$30.86\mathrm{\%}$	$44.06\mathrm{\%}$	$60.43\mathrm{\%}$	$47.98\mathrm{\%}$	$24.31\mathrm{\%}$	$\underset{\_}{69.54\mathrm{\%}}$
GGUF ${}_{ACC}$	$87.73\mathrm{\%}$	$63.59\mathrm{\%}$	$63.75\mathrm{\%}$	$36.31\mathrm{\%}$	$37.98\mathrm{\%}$	$64.63\mathrm{\%}$	$30.19\mathrm{\%}$	$29.27\mathrm{\%}$	$32.94\mathrm{\%}$
GGUF ${}_{CD}$	$54.10\mathrm{\%}$	$81.92\mathrm{\%}$	$60.70\mathrm{\%}$	$36.43\mathrm{\%}$	$31.99\mathrm{\%}$	$63.82\mathrm{\%}$	$30.00\mathrm{\%}$	$31.21\mathrm{\%}$	$33.20\mathrm{\%}$
GGUF ${}_{ED}$	$56.20\mathrm{\%}$	$63.72\mathrm{\%}$	$71.62\mathrm{\%}$	$34.63\mathrm{\%}$	$36.22\mathrm{\%}$	$61.84\mathrm{\%}$	$34.10\mathrm{\%}$	$34.54\mathrm{\%}$	$34.77\mathrm{\%}$
GGUF ${}_{AR}$	$55.19\mathrm{\%}$	$60.25\mathrm{\%}$	$63.70\mathrm{\%}$	$84.57\mathrm{\%}$	$31.71\mathrm{\%}$	$76.50\mathrm{\%}$	$29.94\mathrm{\%}$	$34.18\mathrm{\%}$	$32.15\mathrm{\%}$
GGUF ${}_{ET}$	$58.23\mathrm{\%}$	$64.37\mathrm{\%}$	$58.59\mathrm{\%}$	$29.14\mathrm{\%}$	$72.47\mathrm{\%}$	$68.20\mathrm{\%}$	$39.05\mathrm{\%}$	$32.94\mathrm{\%}$	$31.48\mathrm{\%}$
GGUF ${}_{SD}$	$56.70\mathrm{\%}$	$50.75\mathrm{\%}$	$57.75\mathrm{\%}$	$38.27\mathrm{\%}$	$33.67\mathrm{\%}$	$93.75\mathrm{\%}$	$34.66\mathrm{\%}$	$30.32\mathrm{\%}$	$21.43\mathrm{\%}$
GGUF ${}_{FD}$	$62.20\mathrm{\%}$	$59.91\mathrm{\%}$	$62.88\mathrm{\%}$	$35.51\mathrm{\%}$	$42.52\mathrm{\%}$	$64.68\mathrm{\%}$	$74.08\mathrm{\%}$	$\underset{\_}{62.16\mathrm{\%}}$	$41.69\mathrm{\%}$
GGUF ${}_{AQ}$	$67.08\mathrm{\%}$	$59.73\mathrm{\%}$	$69.50\mathrm{\%}$	$31.17\mathrm{\%}$	$41.31\mathrm{\%}$	$61.16\mathrm{\%}$	$41.86\mathrm{\%}$	$30.02\mathrm{\%}$	$66.53\mathrm{\%}$
Llama-3.1-8B-Instruct FT on Multi-task	$\mathbf{\text{90.74\%}}$	$\underset{\_}{84.71\mathrm{\%}}$	$\underset{\_}{77.75\mathrm{\%}}$	$\mathbf{\text{88.33\%}}$	$\underset{\_}{73.84\mathrm{\%}}$	$\mathbf{\text{95.75\%}}$	$\underset{\_}{82.53\mathrm{\%}}$	$50.22\mathrm{\%}$	$\mathbf{\text{69.80\%}}$
GGUF ${}_{MTFT}$	$87.36\mathrm{\%}$	$81.17\mathrm{\%}$	$73.75\mathrm{\%}$	$83.06\mathrm{\%}$	$71.09\mathrm{\%}$	$94.50\mathrm{\%}$	$79.65\mathrm{\%}$	$\mathbf{\text{65.32\%}}$	$63.14\mathrm{\%}$
Merged Model	$78.72\mathrm{\%}$	$70.69\mathrm{\%}$	$69.62\mathrm{\%}$	$72.52\mathrm{\%}$	$54.60\mathrm{\%}$	$77.04\mathrm{\%}$	$57.00\mathrm{\%}$	$35.03\mathrm{\%}$	$57.52\mathrm{\%}$
GGUF ${}_{Merged}$	$65.95\mathrm{\%}$	$65.83\mathrm{\%}$	$62.13\mathrm{\%}$	$62.93\mathrm{\%}$	$49.06\mathrm{\%}$	$74.38\mathrm{\%}$	$50.04\mathrm{\%}$	$40.75\mathrm{\%}$	$44.97\mathrm{\%}$

The cells with red backgrounds indicate the hard tasks, orange ones indicate the medium tasks and green ones indicate the easy tasks.

ACC = argument component classification; CD = claim detection; ED = evidence detection; AR = argument relation classification; ET= evidence type classification; SD = stance detection; FD = fallacies detection; AQ = argument quality assessment.

We adjusted the following three parameters: the density $\rho$ which represents the fraction of weight to retain in difference to the base model, the weight $w$ of each model task vectors which represents the probability of each model task vector to be used during merging and the $ϵ$ parameters (used only with the DELLA method) which define the half-width range for the keep probabilities. Keep probabilities for parameters in a task vector will range between $\rho -ϵ>0$ and $\rho +ϵ<1$ . The configurations for the DELLA method were formulated based on the outcomes of the diverse DARE configurations. The intuition was that pruning the task vectors’ parameters in accordance with their respective importance would enhance the performance of the merging process. Table 3 summarizes the merging configurations investigated and Table 4 shows the performance of the merging methods across the eight different AM tasks.

Table 3.

Parameters for the diverse merging configurations.

	$\rho$			$ϵ$			w
Merge configuration	Hard	Medium	Easy	Hard	Medium	Easy	Hard	Medium	Easy
DARE I	$0.5$	$0.5$	$0.5$		–		$0.125$	$0.125$	$0.125$
DARE II	$0.7$	$0.7$	$0.7$		–		$0.125$	$0.125$	$0.125$
DARE III	$0.85$	$0.8$	$0.5$		–		$0.125$	$0.125$	$0.125$
DARE IV	$0.8$	$0.8$	$0.8$		–		$0.2$	$0.15$	$0.03$
DARE V	$0.85$	$0.8$	$0.5$		–		$0.2$	$0.15$	$0.03$
DELLA I	$0.85$	$0.8$	$0.5$	$0.1$	$0.1$	$0.4$	$0.125$	$0.125$	$0.125$
DELLA II	$0.9$	$0.7$	$0.5$	$0.1$	$0.15$	$0.4$	$0.2$	$0.15$	$0.03$
DELLA III	$0.9$	$0.85$	$0.8$	$0.05$	$0.1$	$0.1$	$0.2$	$0.15$	$0.03$

The cells with red backgrounds indicate the hard tasks, orange ones indicate the medium tasks and green ones indicate the easy tasks. Within each configuration, tasks with the same difficulty were assigned the same hyperparameter value.

Table 4.

Performance (F1 score) of the merge configurations.

Merge Configuration	ACC	CD	ED	AR	ET	SD	FD ${}_{Single}$	FD ${}_{Multi}$	AQ	Mean $F1$
DARE I	$68.71\mathrm{\%}$	$73.79\mathrm{\%}$	$\mathbf{\text{71.88\%}}$	$58.85\mathrm{\%}$	$49.19\mathrm{\%}$	$84.06\mathrm{\%}$	$55.47\mathrm{\%}$	$30.32\mathrm{\%}$	$42.35\mathrm{\%}$	$59.40\mathrm{\%}$
DARE II	$68.96\mathrm{\%}$	$74.68\mathrm{\%}$	$71.00\mathrm{\%}$	$59.22\mathrm{\%}$	$49.94\mathrm{\%}$	$83.98\mathrm{\%}$	$53.21\mathrm{\%}$	$30.13\mathrm{\%}$	$41.56\mathrm{\%}$	$59.18\mathrm{\%}$
DARE III	$73.09\mathrm{\%}$	$74.31\mathrm{\%}$	$71.75\mathrm{\%}$	$59.60\mathrm{\%}$	$49.69\mathrm{\%}$	$84.35\mathrm{\%}$	$55.52\mathrm{\%}$	$35.03\mathrm{\%}$	$41.30\mathrm{\%}$	$60.51\mathrm{\%}$
DARE IV	$\mathbf{\text{78.84\%}}$	$69.70\mathrm{\%}$	$69.50\mathrm{\%}$	$68.75\mathrm{\%}$	$54.44\mathrm{\%}$	$74.65\mathrm{\%}$	$57.58\mathrm{\%}$	$35.04\mathrm{\%}$	$51.11\mathrm{\%}$	$62.17\mathrm{\%}$
DARE V	$77.71\mathrm{\%}$	$70.45\mathrm{\%}$	$70.00\mathrm{\%}$	$67.75\mathrm{\%}$	$54.19\mathrm{\%}$	$75.68\mathrm{\%}$	$57.00\mathrm{\%}$	$31.55\mathrm{\%}$	$51.50\mathrm{\%}$	$61.75\mathrm{\%}$
DELLA I	$74.71\mathrm{\%}$	$\mathbf{\text{74.68\%}}$	$71.12\mathrm{\%}$	$63.86\mathrm{\%}$	$50.56\mathrm{\%}$	$\mathbf{\text{84.83\%}}$	$57.19\mathrm{\%}$	$\mathbf{\text{35.24\%}}$	$44.84\mathrm{\%}$	$61.89\mathrm{\%}$
DELLA II	$78.72\mathrm{\%}$	$70.69\mathrm{\%}$	$69.62\mathrm{\%}$	$\mathbf{\text{72.52\%}}$	$54.60\mathrm{\%}$	$77.04\mathrm{\%}$	$57.00\mathrm{\%}$	$35.03\mathrm{\%}$	$\mathbf{\text{57.52\%}}$	$\mathbf{\text{63.64\%}}$
DELLA III	$78.60\mathrm{\%}$	$71.70\mathrm{\%}$	$69.37\mathrm{\%}$	$70.64\mathrm{\%}$	$\mathbf{\text{54.94\%}}$	$77.20\mathrm{\%}$	$\mathbf{\text{58.54\%}}$	$35.03\mathrm{\%}$	$55.16\mathrm{\%}$	$63.46\mathrm{\%}$

The cells with red backgrounds indicate the hard tasks, orange ones indicate the medium tasks and green ones indicate the easy tasks.

ACC = argument component classification; CD = claim detection; ED = evidence detection; AR = argument relation classification; ET= evidence type classification; SD = stance detection; FD = fallacies detection; AQ = argument quality assessment.

We make the following observations:

• The fine-tuned models on individual tasks consistently outperform the base model in both zero-shot and few-shot settings, as well as the DeBERTa baseline. Notably, the fine-tuned model for the ACC task achieves an F1 score of $89.61\mathrm{\%}$ , a substantial improvement over the zero-shot ( $73.52\mathrm{\%}$ ) and few-shot ( $75.46\mathrm{\%}$ ) baselines. Similar gains are observed across all tasks, underscoring the effectiveness of task-specific fine-tuning in enhancing model performance. From Table 2, we can see that the merged model has an average gain of efficiency of $\uparrow 29.98\mathrm{\%}$ (resp. $\uparrow 11.38\mathrm{\%}$ ) compared to the base model in zero-shot (resp. few-shot) prompting.

• DARE IV and DELLA II achieve high mean F1 scores (61.75% and 63.64%, respectively). In particular, DELLA II offers the best overall trade-off, with superior performance across multiple categories, including the challenging tasks such as AR and AQ tasks. This suggests that the introduction of non-uniform hyper-parameter values (especially different weights $w$ ) is beneficial when dealing with heterogeneous task difficulty.

• DARE I and II, where all tasks share identical hyperparameters, tend to underperform in terms of mean F1 (59.40% and 59.18%). While these models perform consistently, they fail to exploit task-specific adjustments, particularly struggling on harder tasks such as AR, AQ and ET. This supports the intuition that hard tasks require stronger density and adjusted weight to avoid being overshadowed during merging.

• DELLA variants outperform most DARE configurations in mean F1 scores. DELLA I and DELLA III perform comparably, but DELLA II benefits the most from balancing relatively high $\rho$ values with selective $ϵ$ values and non-uniform $w$ , achieving the best mean F1. This suggests that carefully tuning can improve generalization on difficult tasks.

Overall, the DELLA merging method, especially DELLA II and III, are more effective than the DARE configurations. They maintain better performance over different tasks, suggesting that these two configurations are better suited for building a multi-task models for argument mining. When comparing the different models in Table 2, ‘Merged Model’ refers to our best merged model, that is, DELLA II.

4.3. Fine-tuning a LLM for multi-task AM

We also fine-tuned the Llama 3.1-8B-Instruct model on our multi-task dataset (composed of the training split of all eight AM tasks). This fine-tuned model used LoRA with the same parameters as the task-specific fine-tuning (see Section 4.1). This model is represented by $S{T}_{MTFT}$ in Figure 3. For the evaluation, we tested the fine-tuned model (referred to as ‘Llama-3.1-8B-Instruct FT on Multi-task’ in Table 2) on each task’s test dataset separately. Figure 4 shows a representation of the multi-task fine-tuning.

Figure 4.

Representation of the multi-task fine-tuning.

This multi-task fine-tuned model achieves the best overall performance, outperforming all other trained models on most tasks. It obtains state-of-the-art results on ACC ( $90.74\mathrm{\%}$ ), AR ( $88.33\mathrm{\%}$ ), SD ( $95.75\mathrm{\%}$ ) and AQ ( $69.80\mathrm{\%}$ ), while maintaining comparable performance to the models fine-tuned on individual AM tasks. These results suggest not only an effective transfer of knowledge across tasks – particularly among those with structural similarities such as ACC, CD, ET, and SD – but also that no noticeable conflict arises during joint training, allowing the model to benefit from shared representations without degradation on any individual task.

While the multi-task fine-tuned model achieves superior overall performance compared to the merged model, the latter also offers notable advantages. Model merging provides a flexible and lightweight alternative that does not require joint training on all tasks, making it especially useful when task-specific data is scarce, computational resources are limited, or continual updates are needed. In addition, merging preserves the strengths of task-specialized models without the risk of overfitting to a combined dataset, and allows new tasks to be incorporated incrementally without re-training the entire system. Thus, although multi-task fine-tuning yields the best aggregate results, model merging remains a valuable approach in scenarios where efficiency, modularity, and adaptability are prioritized.

5. Conclusion

In this work, we introduced AMELIA, a family of (multi-task) end-to-end language models for AM. Our contributions are three-fold. First, we consolidated and unified 19 widely used AM datasets into a common format, thereby providing a large, standardized resource that enables reproducibility and facilitates the application of LLMs to diverse argumentative tasks. Second, we conducted an extensive evaluation of fine-tuning strategies using Meta AI’s Llama-3.1-8B-Instruct model, demonstrating that task-specific fine-tuning substantially improves performance across all tasks, while multi-task fine-tuning preserves strong results without degradation, thus confirming the potential of transfer learning across closely related tasks. Third, we explored model merging techniques as a resource-efficient alternative, showing that methods such as DELLA can yield competitive results while maintaining modularity and adaptability.

Our experiments highlight several important insights. Multi-task fine-tuning consistently outperforms both zero-shot and few-shot baselines as well as traditional architectures, establishing a new state-of-the-art on multiple tasks. At the same time, model merging offers a practical compromise when computational or data constraints prevent joint training, making it a promising strategy for scalable deployment. Together, these findings underline the flexibility of LLMs for AM and provide evidence that both fine-tuning and merging can be leveraged to address different application scenarios.

Looking ahead, several promising research directions emerge. First, while our study focused primarily on classification-based tasks, extending this framework to generative AM tasks – such as automatic argument graph construction or debate summarization – would substantially broaden its scope. Pursuing this avenue would require the development of more sophisticated evaluation metrics beyond conventional NLP measures (e.g. ROUGE, ⁷⁸ METEOR, ⁷⁹ or BERTScore ⁸⁰ ) and the integration of human or LLM-based evaluations via online platforms to better capture argumentative dimensions that may escape automatic scoring. Second, incorporating explainability mechanisms and formal argumentation semantics into LLM-based systems would help bridge the gap between high predictive performance and interpretability. This step is critical for deployment in sensitive domains such as law, healthcare, and policy-making, where transparency and accountability are essential. Third, to further enhance performance, we plan to explore hyperparameter optimization and prompt engineering. In particular, we will experiment with varying the LoRA rank parameter $r$ (e.g. 32, 64 and 128) to identify the most effective configurations across tasks. Additionally, we will investigate optimized prompts in both zero-shot and few-shot settings, with the goal of designing task-specific prompts that maximize model performance. Finally, we intend to extend our approach to other LLM architectures, thereby enabling a more comprehensive comparison across model families. This includes evaluating instruction-tuned models beyond Llama and systematically analysing performance under consistent experimental settings, in order to better characterize model-specific strengths and limitations in addressing diverse AM tasks.

Our work demonstrates that LLMs, when carefully adapted through fine-tuning and merging strategies, offer a powerful foundation for advancing AM. By releasing the AMELIA models and datasets publicly, we aim to provide the community with both methodological insights and practical tools to accelerate research in computational argumentation.