Explaining transformer-based image captioning models: An empirical analysis

Abstract

Image Captioning is the task of translating an input image into a textual description. As such, it connects Vision and Language in a generative fashion, with applications that range from multi-modal search engines to help visually impaired people. Although recent years have witnessed an increase in accuracy in such models, this has also brought increasing complexity and challenges in interpretability and visualization. In this work, we focus on Transformer-based image captioning models and provide qualitative and quantitative tools to increase interpretability and assess the grounding and temporal alignment capabilities of such models. Firstly, we employ attribution methods to visualize what the model concentrates on in the input image, at each step of the generation. Further, we propose metrics to evaluate the temporal alignment between model predictions and attribution scores, which allows measuring the grounding capabilities of the model and spot hallucination flaws. Experiments are conducted on three different Transformer-based architectures, employing both traditional and Vision Transformer-based visual features.

1. Introduction

In the last few years, the integration of vision and language has witnessed a relevant research effort which has resulted in the development of effective algorithms working at the intersection of the Computer Vision and Natural Language Processing domains, with applications such as image and video captioning [2,14,24,27], multi-modal retrieval [13,18,34,51], visual question answering [63,68,85], and embodied AI [3,29,32]. While the performance of these models is constantly increasing and their pervasivity intensifies, the need of explaining models’ predictions and quantifying their grounding capabilities is becoming even more fundamental.

In this work, we focus on the image captioning task, which requires an algorithm to describe an input image in natural language. Recent advancements in the field have brought architectural innovations and growth in model size, at the price of lowering the degree of interpretability of the model [24,36]. As a matter of fact, early captioning approaches were based on single-layer additive attention distributions over input regions, which enabled straightforward interpretability of the behavior of the model with respect to the visual input. State-of-the-art models, on the contrary, are based on the Transformer [71] paradigm, which entails multiple layers and multi-head attention, making interpretability and visualization more complex, as the function that connects visual elements to the language model is now highly non-linear. Moreover, the average model size of recent proposals is constantly increasing [62], adding even more challenges to interpretability.

In this paper, we consider a class of recently proposed captioning models, all based on the integration of a fully-attentive encoder with a fully-attentive decoder, in conjunction with techniques for handling a-priori semantic knowledge and multi-level visual features. The reason behind the choice of this class of model is due to their performances in multi-modal tasks [14,75], which at present seem to perform favorably when compared to other Transformer-based architectural choices [36,83]. We firstly verify and foster the interpretability of such models by developing solutions for visualizing what the model concentrates on in the input image, at each step of the generation. Being the model a non-linear function, we consider attribution methods based on gradient computation, which allow the generation of an attribution map that highlights portions of the input image on which the model has concentrated while generating each word of the output caption. Furthermore, on the basis of these visualizations, we quantitatively evaluate the temporal alignment between model predictions and attribution scores – a procedure that allows to quantitatively identify pitfalls in the grounding capabilities of the model and to discover hallucination flaws, in addition to measuring the memorization capabilities of the model. To this aim, we propose a new alignment and grounding metric which does not require access to ground-truth data.

Experimental evaluations are conducted on both the COCO dataset [38] for image captioning, and on the ACVR Robotic Vision Challenge dataset [21] which contains simulated data from a domestic robot scenario. In addition to considering different model architectures, we also consider different image encoding strategies: the well-known usage of a pre-trained object detector to identify relevant regions [2] and the emerging Vision Transformer [17]. Experimental results will shed light on the grounding capabilities of state-of-the-art models and on the challenges posed by the adoption of the Vision Transformer model.

To sum up, our main contributions are as follows:

We consider different encoder-decoder architectures for image captioning and evaluate their interpretability by developing solutions for visualizing what the model concentrates on the input image at each step of the generation.

We propose an alignment and grounding metric which evaluates the temporal alignment between model predictions and attribution scores, thus identifying defects in the grounding capabilities of the model.

We conduct extensive experiments on the COCO dataset and the ACVR Robotic Vision Challenge dataset, considering different model architectures, visual encoding strategies, and attribution methods.

2. Related work

2.1. Image captioning

Before the advent of deep learning, traditional image captioning approaches were based on the generation of simple template sentences, which were later filled by the output of an object detector or an attribute predictor [60,80]. With the surge of deep neural networks, captioning has started to employ RNNs as language models and the output of one or more layers of a CNN was employed to encode visual information and to condition the generation of language [16,26,33,56,73]. On the training side, initial methods were based on a time-wise cross-entropy training. A notable achievement has then been made with the introduction of the REINFORCE algorithm, which enabled the use of non-differentiable caption metrics as optimization objectives [39,54,56]. On the image encoding side, instead, additive attention mechanisms have been adopted to incorporate spatial knowledge, initially from a grid of CNN features [42,77,82], and then using image regions extracted with an object detector [2,43,48]. To further improve the encoding of objects and their relationships, graph convolution neural networks have been employed as well [78,81], to integrate semantic and spatial relationships between objects or to encode scene graphs.

After the emergence of convolutional language models, which have been explored for captioning as well [4], new fully-attentive paradigms [15,65,71] have been proposed and achieved state-of-the-art results in machine translation and language understanding tasks. Likewise, recent approaches have investigated the application of the Transformer model [71] to the image captioning task [8,35,79], also proposing variants or modifications of the self-attention operator [10,14,20,23,24,46]. Transformer-like architectures can also be applied directly on image patches, thus excluding or limiting the usage of the convolutional operator [17,70]. On this line, Liu et al. [40] devised the first convolution-free architecture for image captioning. Specifically, a pre-trained Vision Transformer network (i.e. ViT [17]) is adopted as encoder and a standard Transformer decoder is employed to generate captions.

Other works using self-attention to encode visual features achieved remarkable performance also thanks to vision-and-language pre-training [41,68] and early-fusion strategies [36,85]. For example, following the BERT architecture [15], Zhou et al. [85] devised a single stream of Transformer layers, where region and word tokens are early fused together into a unique flow. This model is first pre-trained on large amounts of image-caption pairs to perform both bidirectional and sequence-to-sequence prediction tasks and then finetuned for image captioning. Li et al. [36] proposed OSCAR, a BERT-like architecture that also includes objects tags, extracted from an object detector and concatenated with the image regions and word embeddings fed to the model. They also performed a large-scale pre-train with 6.5 million image-text pairs, with a masked token loss similar to the BERT mask language loss and a contrastive loss for distinguishing aligned words-tags-regions triples from polluted ones. On the same line, Zhang et al. [83] proposed VinVL, built on top of OSCAR.

A recent and related research line is that of employing large-scale pre-training for obtaining better image representations, either in a discriminative or generative fashion. For instance, in the CLIP model [51] the pre-training task of predicting which caption goes with which image is employed as an efficient and scalable way to learn state-of-the-art visual representations suitable for multi-modal tasks, while in DALL-E [53] a transformer is trained to autoregressively model text and image tokens as a single stream of data, and achieves competitive performances with previous domain-specific models when evaluated in a zero-shot fashion for image generation.

2.2. Explainability and visualization in vision-and-language

While the performance of captioning algorithms has been increasing in the last few years, and while these models are approaching the level of quality required to be run in production, providing effective visualizations of what the model is doing, and explanations to why it fails, is still under-investigated [50,57,59,67]. It shall be noted, in this regard, that early captioning models based on additive attention were easy to be visualized – as their attentive distribution was a single-layer weighted summation of visual features. In the case of modern captioning models, instead, each head of each encoder/decoder layer takes an attentive distribution, thus making visualization less intuitive and straightforward. A solution that is becoming quite popular is that of employing an attribution method, which allows attributing predictions to visual features even in presence of significant non-linearities. For instance, [14,25] used Integrated Gradients [67] to provide region-level attention visualization. Sun et al. [66], instead, develops better visualization maps by employing layer-wise relevance propagation and gradient-based explanation methods, tailored to image captioning models. Following this line, in this work, we visualize attentional states by employing different attribution methods.

A related research line is that of grounded description, which aims at ensuring that objects mentioned in the output captions match with the attentive distribution of the model, thus avoiding hallucination and improving localization [9,44,84]. Some works also integrated saliency information (i.e. what do humans pay more attention to in a scene [11,74]) in captioning generation. This idea was first proposed by Sugano and Bulling [64] who exploited human eye-fixation information for image captioning by including normalized fixation histograms over the image as an input to the soft-attention module of [77] and weighing the attended image regions based on whether these are fixated or not. Subsequent works on this line [12,52,69] used predicted saliency information in place of eye-fixation information.

3. Transformer-based image captioning

Regardless of the variety of methodologies and architectures which have been proposed in the last few years, all image captioning models can be logically decomposed into a visual encoder module, in charge of processing visual features, and a language model, in charge of generating the actual caption [62]. In this work we focus on Transformer-based captioning models, hence both the visual encoding step and the language generation step are carried out employing Transformer-like layers and attention operations.

Following the original Transformer architecture [71], multi-modal connections between the two modules are based on the application of the cross-attention operator, thus leaving self-attentive connections between modalities separated from each other. An alternative to this choice which is worth mentioning is that of employing a joint self-attentive encoding on both modalities (e.g., by concatenating visual features and words, and feeding the result to a Transformer encoder) [36,83]. While this early-fusion strategy has become particularly popular in the context of Vision-and-Language pre-training [76], as it allows the application of contrastive-like pre-training losses, we leave it apart in favor of a separate encoding of the modalities as this is still the dominant approach in image captioning when pre-training is not employed [14,24]. Further, a separate encoding of the two modalities has been demonstrated to perform favorably also when large-scale pre-training is employed [75]. In this choice, we are further motivated by preliminary experiments that suggest that an early-fusion strategy does not bring significant performance improvements when used in a fully supervised training setting.

3.1. Image encoding

Following recent advancements in the field, we consider two image encoding approaches: that of extracting regions from the input image through an object detector pre-trained to detect objects and attributes, and that of employing grid features extracted from a Vision Transformer model [17]. While the first approach has become the default choice in the last few years after the emergence of the Up-Down model [2], the Vision Transformer [17] is quickly becoming a compelling alternative, which we explore in this paper. To enrich the quality of the visual features, we employ a Vision Transformer model which is pre-trained for multi-modal retrieval [51]. To the best of our knowledge, the application of Vision Transformers in an image captioning model has been investigated by Liu et al. [40], using the original model trained for image classification, and by Shen et al. [58], which adopted a pre-training based on multi-modal retrieval as well.

Fig. 1.

Structure of the image captioning models considered in our analysis: (a) a standard fully-attentive encoder-decoder captioner; (b) an encoder with additional memory slots to encode a-priori information; (c) a decoder with meshed connectivity which weights contributions from all encoder layers.

Image region features. In this approach, Faster R-CNN [55] is adopted to detect objects in the input image in two stages: the first, called Region Proposal Network, produces object proposals rolling over intermediate features of a CNN; the second operates a pooling of the region of interest to extract a feature vector for each proposal. One of the key elements of this approach resides in its pre-training strategy, where an auxiliary training loss is added for learning to predict attribute classes alongside object classes on the Visual Genome dataset [30]. This allows the model to predict a dense and rich set of detections, including both salient object and contextual regions and favors the learning of better feature representations. In particular, we employ a Faster R-CNN [55] model with a ResNet-101 [22] backbone, finetuned on the Visual Genome dataset [30]. This feature extraction process results in a variable number of detections for an input image, each described by a 2048-d feature descriptor.

Vision transformer (ViT). Transformer-like architectures can be applied directly on image patches, thus excluding or limiting the usage of the convolutional operator [17,70]. The image encoder we employ in this case is a ViT-B/32 which closely follows the original implementation [17] with only the minor modification of adding an additional layer normalization to the combined patch and position embeddings before the transformer and use a slightly different initialization scheme. The encoder is pre-trained on with a symmetric cross-entropy loss for multi-modal retrieval, on a private large-scale dataset [51]. In this case, the image encoding process results in a set of 50 768-d feature vectors, corresponding to either one of the 49 $32\times 32$ patches in which the input image is divided, or to the extra “classification token” of the architecture.

3.2. Transformer layers

Regardless of the choice, both image encoding methodologies result in a set of feature vectors describing the input image. These are then fed to the encoder part of the architecture (Fig. 1). Both encoder and decoder consist of a stack of Transformer layers that act, indeed, on image regions and words. In the following, we revise their fundamental features. Each encoder layer consists of a self-attention and feed-forward layer, while each decoder layer is a stack of one self-attentive and one cross-attentive layer, plus a feed-forward layer. Both attention layers and feed-forward layers are encapsulated into “add-norm” operations, described in the following.

Multi-head attention. The core component of both self-attention and cross-attention layers is an attention mechanism [6] with multiple heads with different learned weights. Attention is applied using scaled dot-products as similarity measure [71] while keys, queries, and values are computed through linear transformations.

Formally, given two sequences ${\mathbf{x}}_1\mathrm{,}{\mathbf{x}}_2\mathrm{,}\dots \mathrm{,}{\mathbf{x}}_{\mathit{N}}$ and ${\hat{\mathbf{x}}}_1\mathrm{,}{\hat{\mathbf{x}}}_2\mathrm{,}\dots \mathrm{,}{\hat{\mathbf{x}}}_{\mathit{M}}$ of d-dimensional input vectors, each head applies two linear transformations to the first sequence to form key and value vectors: $$\begin{array}{}\text{(1)}& {\mathbf{k}}_{\mathit{t}}={\mathbf{W}}_{\mathit{k}}{\mathbf{x}}_{\mathit{t}}\mathrm{,}{\mathbf{v}}_{\mathit{t}}={\mathbf{W}}_{\mathit{v}}{\mathbf{x}}_{\mathit{t}}\mathrm{,}\end{array}$$ where ${\mathbf{W}}_{\mathit{k}}$ and ${\mathbf{W}}_{\mathit{v}}$ are the key and value transformation matrices, with size ${\mathit{d}}_{\mathit{h}}\times \mathit{d}$ , where ${\mathit{d}}_{\mathit{h}}=\mathit{d}\mathrm{/}\mathit{H}$ is the dimensionality of a single head, and H is the number of heads. Analogously, a linear transformation is applied to the second sequence to obtain query vectors: $$\begin{array}{}\text{(2)}& {\mathbf{q}}_{\mathit{t}}={\mathbf{W}}_{\mathit{q}}{\hat{\mathbf{x}}}_{\mathit{t}}\mathrm{,}\end{array}$$ where ${\mathbf{W}}_{\mathit{q}}$ has the same size of ${\mathbf{W}}_{\mathit{k}}$ and ${\mathbf{W}}_{\mathit{v}}$ . Query vectors are used to compute a similarity score with key vectors and generate a weight distribution over values. The similarity score between a query ${\mathbf{q}}_{\mathit{t}}$ and a key ${\mathbf{k}}_{\mathit{c}}$ is computed as a scaled dot-product between the two vectors, i.e. $\mathrm{(}{\mathbf{q}}_{\mathit{t}}^{⊺}{\mathbf{v}}_{\mathit{c}}\mathrm{)}\mathrm{/}\sqrt{{\mathit{d}}_{\mathit{h}}}$ . Each head then produces a vector by averaging the values ${\{{\mathbf{v}}_{\mathit{c}}\}}_{\mathit{c}}$ with the weights defined by an attentive distribution over the similarity scores: $$\begin{array}{}\text{(3)}& \hfill & {\mathbf{y}}_{\mathit{t}}={\displaystyle \sum }_{\mathit{c}=1}^{\mathit{N}}{\mathit{\alpha }}_{\mathit{t}\mathit{c}}{\mathbf{v}}_{\mathit{c}}\mathrm{,}\text{where}\text{(4)}& \hfill & {\mathit{\alpha }}_{\mathit{t}\mathit{c}}={\displaystyle \frac{exp\mathrm{(}{\mathbf{q}}_{\mathit{t}}^{⊺}{\mathbf{v}}_{\mathit{c}}\mathrm{/}\sqrt{{\mathit{d}}_{\mathit{h}}}\mathrm{)}}{{\sum }_{\mathit{i}}exp\mathrm{(}{\mathbf{q}}_{\mathit{t}}^{⊺}{\mathbf{v}}_{\mathit{i}}\mathrm{/}\sqrt{{\mathit{d}}_{\mathit{h}}}\mathrm{)}}}.\end{array}$$ Results from different heads are then concatenated and projected to a vector with dimensionality d through a final linear transformation.

In the encoding stage, the sequence of visual features is used to infer queries, keys, and values, thus creating a self-attention pattern in which pairwise visual relationships are modeled. In the decoder, we instead apply both a cross-attention and a masked self-attention pattern. In the former, the sequence of words is used to infer queries, and visual elements are used as keys and values. In the latter, the left-hand side of the textual sequence is used to generate keys and values for each element of the sequence, thus enforcing causality in the generation.

Position-wise feed-forward layers. The second component of a Transformer layer is a fully-connected forward layer which is applied time-wise over the input sequence. This consists of two affine transformations with a single non-linearity, $$\begin{array}{}\text{(5)}& \mathtt{\text{FF}}\mathrm{(}{\mathbf{x}}_{\mathit{t}}\mathrm{)}=\mathbf{U}\mathit{\sigma }\mathrm{(}\mathbf{V}{\mathbf{x}}_{\mathit{t}}+\mathbf{b}\mathrm{)}+\mathbf{c}\mathrm{,}\end{array}$$ where $\mathit{\sigma }\mathrm{(}\mathit{x}\mathrm{)}=max\mathrm{(}\mathit{x}\mathrm{,}0\mathrm{)}$ is the ReLU activation function, and V and U are learnable weight matrices, respectively with sizes $\mathit{d}\times {\mathit{d}}_{\mathit{f}}$ and ${\mathit{d}}_{\mathit{f}}\times \mathit{d}$ ; b and c are bias terms. The size of the hidden layer ${\mathit{d}}_{\mathit{f}}$ is usually chosen to be larger than d, e.g. four times d in most implementations [71].

Skip connection and layer normalization. Each sublayer (attention or position-wise feed-forward) is encapsulated within a residual connection [22] and layer normalization [5]. This “add-norm” operation is defined as $$\begin{array}{}\text{(6)}& \mathtt{\text{AddNorm}}\mathrm{(}{\mathbf{x}}_{\mathit{t}}\mathrm{)}=\mathtt{\text{LayerNorm}}\mathrm{(}{\mathbf{x}}_{\mathit{t}}+\mathit{f}\mathrm{(}{\mathbf{x}}_{\mathit{t}}\mathrm{)}\mathrm{)}\mathrm{,}\end{array}$$ where f indicates either an attention layer or a position-wise feed-forward layer.

3.3. Fully-attentive encoder

Given a set of visual features extracted from an input image, attention can be used to obtain a permutation invariant encoding through self-attention operations. As noted in [10,14], attentive weights depend solely on the pairwise similarities between linear projections of the input set itself. Therefore, a sequence of Transformer layers can naturally encode the pairwise relationships between visual features. Because everything depends solely on pairwise similarities, though, self-attention cannot model a priori knowledge on relationships between visual features. For example, given a visual feature encoding a glass of fruit juice and one encoding a plate with a croissant, it would be difficult to infer the concept of breakfast, which could, instead, be easily inferred using a priori knowledge on relationships.

Memory-augmented attention. In this paper, we consider both an encoder made of original Transformer layers [71], and the memory-augmented encoder proposed in [10,14], which overcomes the aforementioned limitation. In the latter case, the set of keys and values used for self-attention is extended with additional “slots” which can encode a priori information – and which are implemented as plain learnable vectors which can be directly updated via SGD. Formally, given the set of visual features X expressed in matrix form, the operator is defined as: $$\begin{array}{}\text{(7)}& \begin{array}{l}\tilde{\mathbf{X}}=\mathsf{Attention}\mathrm{(}{\mathit{W}}_{\mathit{q}}\mathbf{X}\mathrm{,}\mathbf{K}\mathrm{,}\mathbf{V}\mathrm{)}\\ \mathbf{K}=[{\mathit{W}}_{\mathit{k}}\mathbf{X}\mathrm{,}{\mathbf{M}}_{\mathit{k}}]\\ \mathbf{V}=[{\mathit{W}}_{\mathit{v}}\mathbf{X}\mathrm{,}{\mathbf{M}}_{\mathit{v}}]\mathrm{,}\end{array}\end{array}$$ where $\mathsf{Attention}$ indicates the classic attention operator, ${\mathbf{M}}_{\mathit{k}}$ and ${\mathbf{M}}_{\mathit{v}}$ are learnable matrices with ${\mathit{n}}_{\mathit{m}}$ rows, and $[·\mathrm{,}·]$ indicates concatenation. Intuitively, by adding learnable keys and values, through attention it will be possible to retrieve learned knowledge which is not already embedded in X.

Just like the self-attention operator, memory-augmented attention can be applied in a multi-head fashion. In this case, the memory-augmented attention operation is repeated h times, using different projection matrices ${\mathit{W}}_{\mathit{q}}$ , ${\mathit{W}}_{\mathit{k}}$ , ${\mathit{W}}_{\mathit{v}}$ , and different learnable memory slots ${\mathbf{M}}_{\mathit{k}}$ , ${\mathbf{M}}_{\mathit{v}}$ for each head. Then, we concatenate the results from different heads and apply a linear projection.

Full encoder. Multiple encoding layers are stacked in sequence so that the i-th layer consumes the output set computed by layer $\mathit{i}-1$ . This amounts to creating multi-level encodings of the relationships between visual features, in which higher encoding layers can exploit and refine relationships already identified by previous layers, eventually using a priori knowledge, in the case of the memory-augmented attention. A stack of N encoding layers will therefore produce a multi-level output $\tilde{\mathcal{X}}=\mathrm{(}{\tilde{\mathbf{X}}}^1\mathrm{,}\dots \mathrm{,}{\tilde{\mathbf{X}}}^{\mathit{N}}\mathrm{)}$ , obtained from the outputs of each encoding layer.

3.4. Fully-attentive decoder

The language model we employ is composed of a stack of decoder layers, each performing self-attention and cross-attention operations. As mentioned, each cross-attention layer uses the outputs from the decoder to infer keys and values, while self-attention layers rely exclusively on the input sequence of the decoder. However, keys and values are masked so that each query can only attend to keys obtained from previous words, i.e. the set of keys and values for query ${\mathbf{q}}_{\mathit{t}}$ are, respectively, ${\{{\mathbf{k}}_{\mathit{i}}\}}_{\mathit{i}⩽\mathit{t}}$ and ${\{{\mathbf{v}}_{\mathit{i}}\}}_{\mathit{i}⩽\mathit{t}}$ . We consider two variants of the decoder: the standard decoder, which follows the implementation of [71], and the meshed decoder, proposed in [14].

Standard decoder. In the standard decoder, only the activations from the last encoding layer, i.e. ${\tilde{\mathbf{X}}}^{\mathit{N}}$ , are employed to perform cross-attention in all decoding layers.

Meshed decoder. In the meshed decoder, we instead employ all the intermediate activations of the encoder, i.e. $\mathrm{(}{\tilde{\mathbf{X}}}^1\mathrm{,}\dots \mathrm{,}{\tilde{\mathbf{X}}}^{\mathit{N}}\mathrm{)}$ . Given an input sequence of word vectors Y, and outputs from all encoding layers $\tilde{\mathcal{X}}$ , the Meshed Attention operator connects Y to all elements in $\tilde{\mathcal{X}}$ through gated cross-attentions. These multi-level contributions are then summed together after being modulated. Formally, the operator is defined as $$\begin{array}{}\text{(8)}& \mathsf{MeshedAttention}\mathrm{(}\tilde{\mathcal{X}}\mathrm{,}\mathbf{Y}\mathrm{)}={\displaystyle \sum }_{\mathit{i}=1}^{\mathit{N}}{\mathit{\alpha }}_{\mathit{i}}\odot \mathsf{CrossAttention}\mathrm{(}{\tilde{\mathbf{X}}}^{\mathit{i}}\mathrm{,}\mathbf{Y}\mathrm{)}\mathrm{,}\end{array}$$ where $\mathsf{CrossAttention}\mathrm{(}·\mathrm{,}·\mathrm{)}$ stands for the encoder-decoder cross-attention, computed using queries from the decoder and keys and values from the encoder, and ${\mathit{\alpha }}_{\mathit{i}}$ is a matrix of weights having the same size as the cross-attention results. Weights in ${\mathit{\alpha }}_{\mathit{i}}$ modulate both the single contribution of each encoding layer, and the relative importance between different layers. These are computed by measuring the relevance between the result of the cross-attention computed with each encoding layer and the input query, as follows: $$\begin{array}{}\text{(9)}& {\mathit{\alpha }}_{\mathit{i}}=\mathit{\sigma }\mathrm{\left(}{\mathit{W}}_{\mathit{i}}\right[\mathbf{Y}\mathrm{,}\mathsf{CrossAttention}\mathrm{(}{\tilde{\mathbf{X}}}^{\mathit{i}}\mathrm{,}\mathbf{Y}\mathrm{)}]+{\mathit{b}}_{\mathit{i}}\mathrm{)}\mathrm{,}\end{array}$$ where $[·\mathrm{,}·]$ indicates concatenation, σ is the sigmoid activation, ${\mathit{W}}_{\mathit{i}}$ is a $2\mathit{d}\times \mathit{d}$ weight matrix, and ${\mathit{b}}_{\mathit{i}}$ is a learnable bias vector.

3.5. Training strategy

At training time, the input of the encoder is the ground-truth sentence $\{\text{BOS}\mathrm{,}{\mathit{w}}_1\mathrm{,}{\mathit{w}}_2\mathrm{,}\dots \mathrm{,}{\mathit{w}}_{\mathit{n}}\}$ , and the model is trained with a cross-entropy loss to predict the shifted ground-truth sequence, i.e. $\{{\mathit{w}}_1\mathrm{,}{\mathit{w}}_2\mathrm{,}\dots \mathrm{,}{\mathit{w}}_{\mathit{n}}\mathrm{,}\text{EOS}\}$ , where $\text{BOS}$ and $\text{EOS}$ are special tokens to indicate the start and the end of the caption.

While at training time the model jointly predicts all output tokens, the generation process at prediction time is sequential. At each iteration, the model is given as input the partially decoded sequence; it then samples the next input token from its output probability distribution, until a $\text{EOS}$ marker is generated.

Following previous works [2,54,56], after a pre-training step using cross-entropy, we further optimize the sequence generation using Reinforcement Learning. Specifically, we employ a variant of the self-critical sequence training approach [56] which applies the REINFORCE algorithm on sequences sampled using Beam Search [2]. Further, we baseline the reward using the mean of the rewards rather than greedy decoding as done in [2,56].

Specifically, given the output of the decoder we sample the top-k words from the decoder probability distribution at each timestep, and always maintain the top-k sequences with highest probability. We then compute the reward of each sentence ${\mathbf{w}}^{\mathit{i}}$ and backpropagate with respect to it. The final gradient expression for one sample is thus: $$\begin{array}{}\text{(10)}& {\nabla }_{\mathit{\theta }}\mathit{L}\mathrm{(}\mathit{\theta }\mathrm{)}=-{\displaystyle \frac{1}{\mathit{k}}}{\displaystyle \sum }_{\mathit{i}=1}^{\mathit{k}}\mathrm{\left(}\mathrm{\right(}\mathit{r}\mathrm{\left(}{\mathbf{w}}^{\mathit{i}}\mathrm{\right)}-\mathit{b}\mathrm{)}{\nabla }_{\mathit{\theta }}log\mathit{p}\mathrm{(}{\mathbf{w}}^{\mathit{i}}\mathrm{\left)}\mathrm{\right)}\end{array}$$ where $\mathit{b}=\mathrm{(}{\sum }_{\mathit{i}}\mathit{r}\mathrm{(}{\mathbf{w}}^{\mathit{i}}\mathrm{)}\mathrm{)}\mathrm{/}\mathit{k}$ is the baseline, computed as the mean of the rewards obtained by the sampled sequences. To reward the overall quality of the generated caption, we use image captioning metrics as a reward. Following previous works [2], we employ the CIDEr metric (specifically, the CIDEr-D score) which has been shown to correlate better with human judgment [72].

4. Providing explanations

Given the predictions by a captioning model, we develop a methodology for explaining what the model attends in the input image and quantitatively measure its grounding capabilities and the degree of alignment between its predictions and the visual inputs it attends. To this aim, we first extract attention visualizations over input visual elements, as a proxy of what the model attends at each timestep during the generation. These, depending on the model, can be either object detections or raw pixels.1

¹

During the rest of this section, we will employ the term “visual elements” to interchangeably indicate image pixels or regions.

Being the model a highly non-linear function, as anticipated we resort to the usage of an attribution method [59,61,67] to calculate a score map over each visual element. The score map encodes the contribution of the visual element to each predicted word: by aggregating scores on visual elements over object regions, the grounding and alignment capabilities of the model can be evaluated by considering the intersection between attended regions and predicted words.

4.1. Attribution methods

Given a predicted sentence $\{{\mathit{w}}_0\mathrm{,}{\mathit{w}}_1\mathrm{,}\dots \mathrm{,}{\mathit{w}}_{\mathit{T}}\}$ , for each predicted word we compute an attribution function ${\mathit{A}}_{\mathit{t}}\mathrm{(}{\mathit{I}}_0\mathrm{)}$ that ranks the visual elements of the input image ${\mathit{I}}_0$ based on their influence on the score of ${\mathit{w}}_{\mathit{t}}$ . On this line, we employ different attribution methods based on gradient computation that are applicable, namely Saliency, Guided Backpropagation, and Integrated Gradients.

Saliency. Originally proposed in [59], it approximates a non-linear captioning model in the neighborhood of ${\mathit{I}}_0$ with a first-order Taylor expansion. Formally, it expresses the score for ${\mathit{w}}_{\mathit{t}}$ produced by the model as: $$\begin{array}{}\text{(11)}& {\mathit{S}}_{\mathit{w}}\mathrm{(}\mathit{I}\mathrm{)}\approx {\displaystyle \frac{\partial {\mathit{S}}_{\mathit{w}}}{\partial \mathit{I}}}{|}_{{\mathit{I}}_0}\mathit{I}+\mathit{b}\mathrm{,}\end{array}$$ where b is a bias. Under this approximation, the magnitude of the partial derivative of the output of the model with respect to the input can be used to identify which input elements need to be changed to affect the output the most. The attribution score, in this case, is thus simply given by the value of the partial derivative reported in Eq. (11).

Guided backprop. Guided backpropagation [61] computes the gradient of the target word with respect to the input, as the Saliency approach does. In this case, though, gradients of ReLU functions are overridden so that only non-negative gradients are backpropagated.

Integrated gradients. Integrated Gradients [67] is an axiomatic model that works by approximating the integral of gradients of the model output with respect to the input, along the path from a given baseline to the input itself. Formally, the attribution function is defined as follows in this case: $$\begin{array}{}\text{(12)}& {\mathit{A}}_{\mathit{t}}\mathrm{(}{\mathit{I}}_0\mathrm{)}=\mathrm{(}{\mathit{I}}_{\mathit{b}}-{\mathit{I}}_0\mathrm{)}·{{\displaystyle \int }}_{\mathit{\alpha }=0}^1{\displaystyle \frac{\partial {\mathit{S}}_{\mathit{w}}}{\partial \mathit{I}}}{|}_{{\mathit{I}}_{\mathit{b}}+\mathit{\alpha }·\mathrm{(}{\mathit{I}}_{\mathit{b}}-{\mathit{I}}_0\mathrm{)}}\mathit{d}\mathit{\alpha }\mathrm{,}\end{array}$$ where ${\mathit{I}}_{\mathit{b}}$ is a baseline input image, which is our case is a zero tensor.

4.2. Evaluating grounding capabilities

Once the attribution map has been computed, we aggregate the scores over visual regions, as computed from a Faster-RCNN model pre-trained on Visual Genome [2]. For the sake of interpretability and to enhance visualization, we normalize the scores between 0 and 1 applying a contrast stretching-like normalization [19]. At each timestep of the caption, we then identify the region with the highest attribution score. To measure the grounding and alignment capabilities of the model, we check whether each noun that has been predicted by the model is semantically similar to a region which the model has attended, either at the same timestep or in previous ones.

Our grounding score, given a temporal margin δ, can be formally defined as follows: $$\begin{array}{}\text{(13)}& \mathsf{\text{GroundingScore}}\mathrm{(}\mathit{\delta }\mathrm{)}={\displaystyle \sum }_{\mathit{t}\in \mathcal{N}}\underset{\mathit{\tau }\in [\mathit{t}-\mathit{\tau }\mathrm{,}\mathit{t}]}{max}sim\mathrm{(}{\mathit{w}}_{\mathit{t}}\mathrm{,}{\mathit{c}}_{\mathit{\tau }}\mathrm{)}\mathrm{,}\end{array}$$ where sim indicates the GloVe [49] similarity, ${\mathit{w}}_{\mathit{t}}$ is the t-th word predicted by the captioner, and ${\mathit{c}}_{\mathit{\tau }}$ is the semantic class of the region with the highest attribution score at timestep τ. $\mathcal{N}$ is the set of timesteps in which a noun is predicted by the captioner. The grounding score is then averaged over the entire dataset.

As it can be observed, the proposed score increases monotonically when increasing δ. A model with good alignment capabilities, though, will have high score values starting from low values of δ, meaning that it attends correct regions right before producing the corresponding words. A model with low alignment capabilities, but with good memory properties, will instead reach high score values for higher δ values, meaning that it attends and memorizes regions well before generating the corresponding words. On the contrary, a model with low grounding capabilities (or with hallucination flaws) will have low score values on average, regardless of δ.

5. Experimental evaluation

5.1. Datasets

To train and test the considered Transformer-based captioning models, we employ the Microsoft COCO dataset [38], which contains $123287$ images labeled with 5 captions each. We employ the data splits defined in [27], where $5000$ images are used for validation, $5000$ images for testing, and the rest for training. Further, we also validate our approach and the proposed metric on images taken from a robot-centric point of view. To this end, we employ the ACVR Robotic Vision Challenge dataset [21] which contains simulated data from a domestic robot scenario. The dataset contains scenes with cluttered surfaces, and day and night lighting conditions. Authors have simulated domestic service robots of multiple sizes, resulting in sequences with three different camera heights above the ground plane. We employ the validation set of this dataset that consists of over $21000$ images in four simulated indoor video sequences.

5.2. Captioning metrics

We employ popular captioning metrics to evaluate both fluency and semantic correctness: BLEU [47], $\text{ROUGE}$ [37], METEOR [7], CIDEr [72], and SPICE [1]. BLEU is a form of precision of word n-grams between predicted and ground-truth sentences. As done in previous works, we evaluate our predictions with BLEU using n-grams of length 1 and 4 (i.e. B-1 and B-4). $\text{ROUGE}$ (R) computes an F-measure with a recall bias using the longest common sub-sequence technique. METEOR (M), instead, scores captions by aligning them to one or more ground-truths. Alignments are based on exact, stem, synonym, and paraphrase matches between words and phrases. CIDEr (C) computes the average cosine similarity between n-grams found in the generated caption and those found in reference sentences, weighting them using TF-IDF. SPICE (S), finally, considers matching tuples extracted from the candidate and the reference scene graphs, thus favoring the semantic content rather than the fluency of generated captions. While it has been shown experimentally that BLEU and ROUGE have a lower correlation with human judgments than the other metrics [72], the common practice in the image captioning literature is to report all the mentioned metrics. To ensure a fair evaluation, we use the Microsoft COCO evaluation toolkit to compute all scores.

In addition, we also evaluate the capability of the captioning approaches to name objects on the scene. To evaluate the alignment between predicted and ground-truth nouns, we employ an alignment score [9] based on the Needleman-Wunsch algorithm [45]. Specifically, given a predicted caption $\mathit{y}$ and its target counterpart ${\mathit{y}}^{\ast }$ , we extract all nouns from both sentences and evaluate the alignment between them. We use the following scoring system: the reward for matching two nouns is equal to the cosine similarity between their word embeddings; a gap gets a negative reward equal to the minimum similarity value, i.e. $-1$ . Once the optimal alignment is computed, we normalize its score, $al\mathrm{(}\mathit{y}\mathrm{,}{\mathit{y}}^{\ast }\mathrm{)}$ , with respect to the length of the sequences. The alignment score is thus defined as: $$\begin{array}{}\text{(14)}& NW\mathrm{(}\mathit{y}\mathrm{,}{\mathit{y}}^{\ast }\mathrm{)}={\displaystyle \frac{al\mathrm{(}\mathit{y}\mathrm{,}{\mathit{y}}^{\ast }\mathrm{)}}{max\mathrm{(}\mathrm{\#}\mathit{y}\mathrm{,}\mathrm{\#}{\mathit{y}}^{\ast }\mathrm{)}}}\end{array}$$ where $\mathrm{\#}\mathit{y}$ and $\mathrm{\#}{\mathit{y}}^{\ast }$ represent the number of nouns contained in $\mathit{y}$ and ${\mathit{y}}^{\ast }$ , respectively. Notice that $NW\mathrm{(}·\mathrm{,}·\mathrm{)}\in [-1\mathrm{,}1]$ .

To assess instead how the predicted caption covers all the objects without considering the order in which they are named in the caption, we employ a soft coverage measure between the ground-truth set of object classes and the set of names in the caption [10]. In this case, we first compute the optimal assignment between predicted and ground-truth nouns using distances between word vectors and the Hungarian algorithm [31]. We then define an intersection score between the two sets as the sum of assignment profits. The coverage measure is computed as the ratio of the intersection score and the number of ground-truth nouns: $$\begin{array}{}\text{(15)}& Cov\mathrm{(}\mathit{y}\mathrm{,}{\mathit{y}}^{\ast }\mathrm{)}={\displaystyle \frac{\text{I}\mathrm{(}\mathit{y}\mathrm{,}{\mathit{y}}^{\ast }\mathrm{)}}{\mathrm{\#}{\mathit{y}}^{\ast }}}\mathrm{,}\end{array}$$ where $\text{I}\mathrm{(}·\mathrm{,}·\mathrm{)}$ is the intersection score, and the # operator represents the cardinality of the set of nouns.

For both NW and Coverage metrics, we employ GloVe word embeddings [49] to compute the similarity between predicted and ground-truth nouns.

Table 1

Captioning performance of the considered models, on the COCO Karpathy splits

Method	Validation Set								Test Set
	B-1	B-4	M	R	C	S	NW	Cov	B-1	B-4	M	R	C	S	NW	Cov
Image Region Features
Transformer	79.4	36.7	27.7	57.1	122.7	20.7	0.240	74.1	79.4	36.4	27.7	56.8	123.7	20.9	0.242	74.5
SMArT [10]	80.5	38.3	29.0	58.4	129.3	22.3	0.258	76.1	80.4	38.1	28.8	58.2	129.7	22.2	0.256	76.2
${\mathcal{M}}^2$ Transformer [14]	80.7	38.3	28.9	58.4	129.0	22.4	0.254	75.9	80.8	39.1	29.2	58.6	131.2	22.6	0.256	76.0
Vision Transformer (ViT)
Transformer	81.0	38.9	28.9	58.6	128.7	22.6	0.261	77.6	80.5	38.5	28.8	58.5	128.7	22.6	0.258	77.4
SMArT [10]	81.1	39.0	29.1	58.6	129.4	22.9	0.263	77.1	81.1	38.9	29.0	58.6	130.7	23.0	0.263	77.5
${\mathcal{M}}^2$ Transformer [14]	81.2	39.2	29.2	58.7	130.3	23.0	0.261	77.7	80.8	38.7	29.2	58.7	130.9	23.0	0.263	77.9

5.3. Implementation details

In our experiments, we train and test three different captioning models. For convenience and for coherency with the original papers in which these approaches have been firstly proposed, we refer to a captioner that follows the structure of the original Transformer [71] with three layers as “Transformer”, to a captioner that employs memory-augmented attention and two layers as “SMArT” [10], and to a captioner that employs both memory-augmented attention and a mesh-like connectivity, with three layers as “ ${\mathcal{M}}^2$ Transformer” [14]. As previously mentioned, we train two versions of each captioning model: one using image region features extracted from Faster R-CNN trained on Visual Genome [2,55] and the other employing the Vision Transformer [17] as feature extractor. For all captioning models, we set the dimensionality d of all layers is to 512 and use a number of heads H equal to 8. The dimensionality of the inner feed-forward layer ${\mathit{d}}_{\mathit{f}}$ is 2048. We use dropout with keep probability 0.9 after each attention layer and after position-wise feed-forward layers. Input words are represented with one-hot vectors and then linearly projected to the input dimensionality of the model, d. We also employ sinusoidal positional encodings [71] to represent word positions inside the sequence and sum the two embeddings before the first encoding layer.

All models are trained using Adam [28] as optimizer with ${\mathit{\beta }}_1=0.9$ and ${\mathit{\beta }}_2=0.98$ . The learning rate is varied during training using the strategy presented in [71], i.e. according to the formula: ${\mathit{d}}^{-0.5}·min\mathrm{(}{\mathit{s}}^{-0.5}\mathrm{,}\mathit{s}·{\mathit{w}}^{-0.5}\mathrm{)}$ , where s is the current optimization step and w is a warmup parameter, set to $10000$ in all our experiments. After the pre-training with cross-entropy loss, we finetune the models with a fixed learning rate of $5\times {10}^{-6}$ .

5.4. Captioning performance

We first report the captioning performance of the aforementioned models, when using image region features and when employing the Vision Transformer as visual backbone. Table 1 shows the captioning performance of all models on the COCO validation and test splits defined in [27], in terms of standard captioning metrics, NW, and Coverage scores. As it can be noticed, the pre-trained Vision Transformer is competitive with the more standard practice of using image region features in terms of standard captioning metrics, as it advances the performances on the COCO validation set while providing similar performances on the test set. We notice, further, that it brings an improvement in terms of NW and Coverage with respect to the usage of image region features, thus confirming that it can be employed as a proper visual backbone for the generation of quality captions.

To analyze the computational demands of the three considered captioning approaches, we report in Fig. 2 the mean and standard deviation of prediction times as a function of the batch size. In this analysis, we only evaluate the prediction times of the three captioning architectures without considering the feature extraction phase. For a fair evaluation, we run all approaches on the same workstation and on the same GPU, i.e. an NVIDIA 2080Ti. As it can be observed, the SMArT architecture achieves a better efficiency than a traditional Transformer with three layers, thanks to its reduced number of layers (i.e. 2). Finally, the ${\mathcal{M}}^2$ Transformer model has the highest prediction time, because of the meshed connection between encoder and decoder layers.

Fig. 2.

Prediction times of the considered models when varying the batch size.

Fig. 3.

Qualitative examples of the $\mathsf{\text{GroundingScore}}$ metric, on captions generated by the SMArT model on COCO images. The metric is reported as an array, containing the values of $\mathsf{\text{GroundingScore}}$ at $\mathit{\delta }\in \{0\mathrm{,}1\mathrm{,}3\mathrm{,}5\mathrm{,}\infty \}$ . At each timestep, the detection with the highest attribution score is reported. Correctly grounded nouns, at $\mathit{\delta }=0$ , are reported in green, while incorrectly grounded nouns are reported in red.

Fig. 4.

Qualitative examples of the $\mathsf{\text{GroundingScore}}$ metric, on captions generated by the ${\mathcal{M}}^2$ Transformer model on COCO images. The metric is reported as an array, containing the values of $\mathsf{\text{GroundingScore}}$ at $\mathit{\delta }\in \{0\mathrm{,}1\mathrm{,}3\mathrm{,}5\mathrm{,}\infty \}$ . At each timestep, the detection with the highest attribution score is reported. Correctly grounded nouns, at $\mathit{\delta }=0$ , are reported in green, while incorrectly grounded nouns are reported in red.

5.5. Grounding and alignment evaluation

Qualitative examples. We now turn to the evaluation of the grounding and alignment capabilities of the models, using the proposed $\mathsf{\text{GroundingScore}}$ . Figure 3 reports qualitative examples of the metric, computed over sample images and captions generated by the SMArT model. For visualization purposes, the metric is reported as an array, containing the values of $\mathsf{\text{GroundingScore}}$ at $\mathit{\delta }\in \{0\mathrm{,}1\mathrm{,}3\mathrm{,}5\mathrm{,}\infty \}$ . As it can be seen, the metric effectively captures the alignment and grounding capabilities of the model. In the first row, the network concentrates on the right object class while generating all nouns of the caption (“man”, “motorcycle”, “cow”, “road”), thus getting a 100% $\mathsf{\text{GroundingScore}}$ at $\mathit{\delta }=0$ . In the second case, instead, it concentrates on the skis region one step before generating “skis”: because of this slight temporal misalignment, its $\mathsf{\text{GroundingScore}}$ at $\mathit{\delta }=0$ is 88.8%, while its $\mathsf{\text{GroundingScore}}$ at $\mathit{\delta }=1$ becomes 100%. In the last sample, finally, the $\mathsf{\text{GroundingScore}}$ never reaches 100%, as the network hallucinates the concept of “city” while just looking at a detection containing a building, and never attends a detection containing the whole skyline. Similarly, in Fig. 4 we report qualitative examples on captions generated by the ${\mathcal{M}}^2$ Transformer model.

Quantitative evaluation. Tables 2 and 3 reports the $\mathsf{\text{GroundingScore}}$ obtained for different values of the temporal margin δ on the COCO validation and test sets, respectively. In the same tables, we report the scores obtained by applying all the aforementioned attribution methods, i.e. Saliency, Guided Backpropagation, and Integrated Gradients.

Table 2

Grounding and alignment performance of the considered models on the COCO validation set, expressed in terms of $\mathsf{\text{GroundingScore}}$

Method	Saliency					Guided Backprop					Integrated Gradients
	$\mathit{\delta }=0$	$\mathit{\delta }=1$	$\mathit{\delta }=3$	$\mathit{\delta }=5$	$\mathit{\delta }=\infty$	$\mathit{\delta }=0$	$\mathit{\delta }=1$	$\mathit{\delta }=3$	$\mathit{\delta }=5$	$\mathit{\delta }=\infty$	$\mathit{\delta }=0$	$\mathit{\delta }=1$	$\mathit{\delta }=3$	$\mathit{\delta }=5$	$\mathit{\delta }=\infty$
Image Region Features
Transformer	75.15	78.59	80.28	80.84	81.20	67.42	74.23	78.37	80.00	80.78	79.26	82.76	84.81	85.43	85.78
SMArT [10]	80.37	83.76	85.68	86.15	86.36	72.23	78.39	81.75	82.87	83.51	81.30	85.36	87.47	87.92	88.12
${\mathcal{M}}^2$ Transformer [14]	80.68	84.01	85.85	86.26	86.45	71.43	77.98	81.53	82.68	83.28	75.16	80.90	83.83	84.71	85.20
Vision Transformer (ViT)
Transformer	69.94	70.79	71.44	71.67	71.83	67.19	73.59	77.12	78.48	79.30	69.02	75.38	78.90	80.14	80.95
SMArT [10]	70.08	70.83	71.40	71.62	71.75	67.39	74.21	77.62	78.93	79.78	69.42	75.85	79.27	80.52	81.25
${\mathcal{M}}^2$ Transformer [14]	70.13	71.02	71.62	71.84	71.97	67.85	74.31	77.87	79.10	79.93	69.56	75.90	79.48	80.70	81.46

Table 3

Grounding and alignment performance of the considered models on the COCO test set, expressed in terms of $\mathsf{\text{GroundingScore}}$

Method	Saliency					Guided Backprop					Integrated Gradients
	$\mathit{\delta }=0$	$\mathit{\delta }=1$	$\mathit{\delta }=3$	$\mathit{\delta }=5$	$\mathit{\delta }=\infty$	$\mathit{\delta }=0$	$\mathit{\delta }=1$	$\mathit{\delta }=3$	$\mathit{\delta }=5$	$\mathit{\delta }=\infty$	$\mathit{\delta }=0$	$\mathit{\delta }=1$	$\mathit{\delta }=3$	$\mathit{\delta }=5$	$\mathit{\delta }=\infty$
Image Region Features
Transformer	74.95	78.18	80.06	80.56	80.84	67.02	74.08	78.25	79.88	80.61	79.22	82.75	84.86	85.43	85.77
SMArT [10]	79.83	83.45	85.57	85.98	86.17	72.28	78.15	81.60	82.79	83.37	80.66	84.98	87.20	87.69	87.89
${\mathcal{M}}^2$ Transformer [14]	80.23	83.70	85.74	86.12	86.28	71.29	78.07	81.60	82.67	83.24	74.98	80.60	83.69	84.54	84.98
Vision Transformer (ViT)
Transformer	69.89	70.69	71.34	71.56	71.68	67.14	73.54	77.04	78.43	79.18	69.11	75.55	78.93	80.20	80.94
SMArT [10]	69.96	70.81	71.41	71.59	71.72	67.31	74.12	77.65	79.01	79.83	69.85	75.73	79.24	80.43	81.09
${\mathcal{M}}^2$ Transformer [14]	69.99	70.85	71.37	71.62	71.76	67.54	74.12	77.81	79.24	80.06	69.86	76.01	79.55	80.72	81.31

Starting from the models employing regions as image descriptors, we notice that both SMArT and ${\mathcal{M}}^2$ Transformer perform favorably in terms of $\mathsf{\text{GroundingScore}}$ with respect to the original Transformer, thus confirming that the usage of memory vector and mesh-like connectivity does not only increases quality metrics, but also the grounding and alignment capability of the model. According to all attribution methods, both models start with high scores at $\mathit{\delta }=0$ , thus showing high grounding capabilities and synchronization with respect to attended objects while generating the output caption. For $\mathit{\delta }=\infty$ (i.e. when considering an infinite temporal window), they both reach grounding scores between 80 and 88%, depending on the attribution method of choice and the split.

Turning then to the evaluation of the Vision Transformer as visual backbone, we instead notice that it generally lowers the grounding capabilities of the models. The maximum grounding score at $\mathit{\delta }=0$ , for instance, is 70.13, thus significantly lower than the average score obtained when employing region features. Also, we notice that the relative difference between the grounding score at $\mathit{\delta }=\infty$ and $\mathit{\delta }=0$ reduces, indicating that the Vision Transformer fosters hallucination and incorrect localization rather than generating synchronization flaws.

Finally, in Table 4 we report the same analysis on the ACVR dataset, which features simulated data and a robot-centric point of view. As it can be seen, there is no reduction in grounding capabilities, as testified by the values reported according to all attribution methods. Instead, we observe a slight increase of the grounding metric at $\mathit{\delta }=0$ , which underlines that the models can ground objects correctly also in this simulated setting.

Table 4

Grounding and alignment performance of the considered models on the ACRV dataset, expressed in terms of $\mathsf{\text{GroundingScore}}$

Method	Saliency					Guided Backprop					Integrated Gradients
	$\mathit{\delta }=0$	$\mathit{\delta }=1$	$\mathit{\delta }=3$	$\mathit{\delta }=5$	$\mathit{\delta }=\infty$	$\mathit{\delta }=0$	$\mathit{\delta }=1$	$\mathit{\delta }=3$	$\mathit{\delta }=5$	$\mathit{\delta }=\infty$	$\mathit{\delta }=0$	$\mathit{\delta }=1$	$\mathit{\delta }=3$	$\mathit{\delta }=5$	$\mathit{\delta }=\infty$
Image Region Features
Transformer	74.91	78.39	79.99	80.40	80.62	68.02	76.12	79.83	81.02	81.47	76.91	81.84	83.92	84.44	84.66
SMArT [10]	80.56	84.29	85.92	86.41	86.57	74.80	80.67	84.02	84.95	85.37	79.42	86.41	88.13	88.83	89.08
${\mathcal{M}}^2$ Transformer [14]	82.13	85.77	87.33	87.81	88.00	72.36	78.84	82.46	83.44	83.86	79.52	84.53	86.38	87.03	87.40
Vision Transformer (ViT)
Transformer	71.03	72.90	75.28	77.20	80.00	72.72	75.83	77.76	78.88	79.92	71.66	74.74	77.46	78.84	79.95
SMArT [10]	71.14	72.91	75.36	77.21	79.91	72.26	75.47	77.51	78.65	79.65	71.51	74.41	76.91	71.92	79.54
${\mathcal{M}}^2$ Transformer [14]	71.13	72.86	75.77	77.29	80.00	72.83	75.94	77.96	78.88	80.01	71.73	74.69	77.15	78.21	79.40

Fig. 5.

Qualitative examples of the $\mathsf{\text{GroundingScore}}$ metric, on captions generated on the ACVR dataset. The metric is reported as an array, containing the values of $\mathsf{\text{GroundingScore}}$ at $\mathit{\delta }\in \{0\mathrm{,}1\mathrm{,}3\mathrm{,}5\mathrm{,}\infty \}$ . At each timestep, the detection with the highest attribution score is reported. Correctly grounded nouns, at $\mathit{\delta }=0$ , are reported in green, while incorrectly grounded nouns are reported in red.

6. Conclusion

We have presented a visualization and grounding methodology for Transformer-based image captioning algorithms. Our work takes inspiration from the increasing need of explaining models’ predictions and quantify their grounding capabilities. Given the highly non-linear nature of state-of-the-art captioners, we employed different attribution methods to visualize what the model concentrates on at each step of the generation. Further, we have proposed a metric that quantifies the grounding and temporal alignment capabilities of a model, and which can be used to measure hallucination or synchronization flaws. Experiments have been conducted on the COCO and ACVR datasets, employing three Transformer-based architectures.