Stable Diffusion is a widely used text-to-image generation model. However, its outputs are highly sensitive to hyperparameters settings and often suffer issues such as semantic drift, subject misalignment and detail loss. Traditional methods rely on manually adjusting hyperparameters to alter the attention distribution and thus improve the quality of generated images, which is time-consuming and lacks precision. Therefore, we propose an attention-guided visual diagnostic system named DAttnVis, which is designed to assist users in understanding the complex inference process of the Stable Diffusion model and optimizing its parameters. The core idea is to transform high-dimensional attention signals into comparable diagnostic representations across layers using a quantifiable metric—the Attention Concentration Index (ACI). Additionally, an anomaly detection method based on Median Absolute Deviation (MAD) is proposed to accurately identify abnormal attention layers. By linking multiple views, including UNet attention flow, diagnosis and guidance, cross-attention, and historical comparison, DAttnVis constructs a comprehensive diagnostic workflow that covers global screening, structural drilling-down, semantic tracing, and result verification. Quantitative evaluation experiments, case studies and user studies demonstrate that DAttnVis can effectively reduce trial-and-error costs and debugging burdens in the model tuning process, while improving the accuracy of anomalous structure localization and key prompt attribution.
YangLZhangZSongY, et al. Diffusion models: a comprehensive survey of methods and applications. ACM Comput Surv2024; 56: 1–39.
2.
ZhangCWuQGambardellaCC, et al. Taming stable diffusion for text to 360° Panorama image generation. In: 2024 IEEE/CVF conference on computer vision and pattern recognition (CVPR), Seattle, WA, 17–21 June 2024, pp. 6347–6357. IEEE.
3.
Mahdavi GoloujehASullivanAMagerkoB.Is it AI or is it me? Understanding users’ prompt journey with text-to-image generative AI tools. In: CHI conference on human factors in computing systems (ed. O’BrienA), Hawaii, 11–16 May 2024, paper no. 123, pp. 1–13. ACM.
4.
LiWWangXLiW, et al. A survey of automatic prompt engineering: an optimization perspective. arXiv preprint, arXiv:2502.11560. Epub ahead of print 2025.
5.
BradeSWangBSousaM, et al. Promptify: text-to-image generation through interactive prompt exploration with large language models. In: ACM symposium on user interface software and technology (ed. O’BrienA), San Francisco, CA, 29 October–1 November 2023, paper no. 456, pp. 1–14. ACM.
6.
HouRChangHMaB, et al. Cross attention network for few-shot classification. Proceedings of the 33rd International Conference on Neural Information Processing Systems, 2019, pp. 4003–4014.
7.
CroitoruFAHondruVIonescuRT, et al. Diffusion models in vision: a survey. IEEE Trans Pattern Anal Mach Intell2023; 45: 10850–10869.
8.
ChenYWeiWWangL, et al. Where do they go next? Causal inference-based prediction and visual analysis of graduates’ first destination. J Vis2024; 27: 885–908.
9.
ParkJHJuYJLeeSW.Explaining generative diffusion models via visual analysis for interpretable decision-making process. Expert Syst Appl2024; 248: 123231.
10.
BoenischFDziedzicAHintersdorfD, et al. Finding nemo: localizing neurons responsible for memorization in diffusion models. Adv Neural Inf Process Syst2024; 37: 88236–88278.
11.
ReedSAkataZYanX, et al. Generative adversarial text to image synthesis. In: International conference on machine learning, New York, 19–24 June 2016, pp. 1060–1069.
12.
XuTZhangPHuangQ, et al. Attngan: Fine-grained text to image generation with attentional generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, 18–22 June 2018, pp. 1316–1324. IEEE.
RombachRBlattmannALorenzD, et al. High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, New Orleans, 19–24 June 2022, pp. 10684–10695. IEEE.
19.
PodellDEnglishZLaceyK, et al. SDXL: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint, arXiv:2307.01952. Epub ahead of print 2023.
20.
PeeblesWXieS.Scalable diffusion models with transformers. In: Proceedings of the IEEE/CVF international conference on computer vision, Paris, France, 2–6 October 2023, pp. 4195–4205. IEEE.
21.
BrauwersGFrasincarF.A general survey on attention mechanisms in deep learning. IEEE Trans Knowl Data Eng2023; 35: 3279–3298.
22.
LinHChengXWuX, et al. CAT: cross attention in vision transformer. In: 2022 IEEE international conference on multimedia and expo (ICME), Taipei, Taiwan, 18–22 July 2022, pp. 1–6. IEEE.
23.
JinJChenXMengY, et al. Deep blind chest X-Ray image quality assessment with region-of-interest-guided attention. IEEE MultiMedia2023; 30: 36–46.
24.
HaoJChenYLvC, et al. A visual analytics framework for unveiling multi-scale patch decomposition in transformer-based time-series forecasting. J Vis2026; 29: 137–152.
25.
JainSWallaceBC.Attention is not explanation. arXiv preprint, arXiv:1902.10186. Epub ahead of print 2019.
26.
WiegreffeSPinterY.Attention is not not explanation. arXiv preprint, arXiv:1908.04626. Epub ahead of print 2019.
27.
WuCChenYDongY, et al. VizOPTICS: Getting insights into OPTICS via interactive visual analysis. Comput Electr Eng2023; 107: 108624.
28.
ChenYZhangQGuanZ, et al. GEMvis: a visual analysis method for the comparison and refinement of graph embedding models. Vis Comput2022; 38: 3449–3462.
29.
HanZChenFZhangH, et al. An attention-based representation learning model for multiple relational knowledge graph. Expert Syst2023; 40: e13234.
30.
YuCLiSFengW, et al. SACA-fusion: a low-light fusion architecture of infrared and visible images based on self- and cross-attention. Vis Comput2024; 40: 3347–3356.
31.
ZhuXLiLLiuJ, et al. Image captioning with triple-attention and stack parallel LSTM. Neurocomputing2018; 319: 55–65.
32.
AbnarSZuidemaW.Quantifying attention flow in transformers. arXiv preprint, arXiv:2005.00928. Epub ahead of print 2020.
33.
VigJ.BertViz: A tool for visualizing multihead self-attention in the BERT model. In: ICLR workshop: debugging machine learning models (ed. O’BrienA), New Orleans, 6–9 May 2019, paper no. 3.
34.
HooverBStrobeltHGehrmannS.exBERT: a visual analysis tool to explore learned representations in transformer models. In: Proceedings of the 58th annual meeting of the association for computational linguistics: system demonstrations, Online, 5–10 July 2020, pp. 187–196.
35.
AlammarJ.Ecco: an open source library for the explainability of transformer language models. In: Proceedings of the 59th annual meeting of the Association for Computational Linguistics and the 11th international joint conference on natural language processing: system demonstrations, Online, 1–6 August 2021, pp. 249–257.
36.
AflaloEDuMTsengSY, et al. Vl-interpret: an interactive visualization tool for interpreting vision-language transformers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, New Orleans, 19–24 June 2022, pp. 21406–21415. IEEE.
37.
TangRLiuLPandeyA, et al. What the DAAM: interpreting stable diffusion using cross attention. In: Proceedings of the 61st annual meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Toronto, 9–14 July 2023, pp. 5644–5659.
38.
HertzAMokadyRTenenbaumJ, et al. Prompt-to-prompt image editing with cross attention control. arXiv preprint, arXiv:2208.01626. Epub ahead of print 2022.
39.
CheferHAlalufYVinkerY, et al. Attend-and-excite: attention-based semantic guidance for text-to-image diffusion models. ACM Trans Graph2023; 42: 1–10.
40.
ParkJKoJByunD, et al. Cross-attention head position patterns can align with human visual concepts in text-to-image generative models. In: The thirteenth international conference on learning representations, Vienna, Austria, 7–11 May 2024.
41.
GeTLuYLuK, et al. VEGA: visual comparison of phylogenetic trees for evolutionary genome analysis (ChinaVis 2019). J Vis2020; 23: 523–537.
42.
LuKChenYLuoZ, et al. Visual comparison of hierarchies with inherited attributes: application to multi-regional MRL standards in food safety. J Vis2025; 28: 553–568.
43.
CheferHGurSWolfL.Transformer interpretability beyond attention visualization. In: 2021 IEEE/CVF conference on computer vision and pattern recognition (CVPR), Nashville, TN, 20–25 June 2021, pp. 782–791. IEEE.
44.
ChenCFRFanQPandaR.Crossvit: cross-attention multi-scale vision transformer for image classification. In: Proceedings of the IEEE/CVF international conference on computer vision, Online, 11–17 October 2021, pp. 357–366. IEEE.
45.
LiYWangJAboagyeP, et al. Visual analytics for efficient image exploration and user-guided image captioning. IEEE Trans Vis Comput Graph2024; 30: 2875–2887.
46.
JaunetTKervadecCVuillemotR, et al. VisQA: X-raying vision and language reasoning in transformers. IEEE Trans Vis Comput Graph2022; 28: 976–986.
47.
LiuBWangCCaoT, et al. Towards understanding cross and self-attention in stable diffusion for text-guided image editing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Seattle, WA, 17–21 June 2024, pp. 7817–7826. IEEE.
48.
LeeSHooverBStrobeltH, et al. Diffusion explainer: Visual explanation for text-to-image stable diffusion. In: 2024 IEEE visualization and visual analytics (VIS), Florida, 13–18 October 2024, pp. 96–100. IEEE.
49.
FengYWangXWongKK, et al. Promptmagician: interactive prompt engineering for text-to-image creation. IEEE Trans Vis Comput Graph2024; 30: 295–305.
50.
GuoYShaoHLiuC, et al. Prompthis: visualizing the process and influence of prompt editing during text-to-image creation. IEEE Trans Vis Comput Graph2025; 31: 4547–4559. https://doi.org/10.1109/TVCG.2024.3408255
AumanCBhatiDArquillaK, et al. Integrating layer-wise relevance propagation with stable diffusion for enhanced interpretability. Report, University of California, 2025.
53.
MotamedSVan GansbekeWVan GoolL.Investigating the effectiveness of cross-attention to unlock zero-shot editing of text-to-video diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Seattle, 16–22 June 2024, pp. 7406–7415. IEEE.
54.
IglewiczBHoaglinDC (eds). How to detect and handle outliers. 1st ed.Quality Press, 1993.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
