How a text-to-image model adds meaning beyond minimal prompts: An SFL-informed intersemiotic analysis of visual-verbal relations

Abstract

AI-generated images now function as routine multimodal texts, yet we know little about how they construct meaning beyond minimal prompts. Using an SFL-informed multimodal framework, this study analyses intersemiotic expansion, projection, and affect in a purpose-built corpus of 120 text-to-image outputs produced with a strictly controlled two-line prompt. Four subject types differing in animacy and conventional agency were crossed with six minimally specified predicates spanning material, mental, verbal, and projective processes (five independent generations each). Coding shows that divergence from prompts follows stable semiotic patterns rather than random variation. Action-oriented predicates strongly license specification and addition, whereas predicates foregrounding internal states rely more heavily on projection and selective specification. Subject type modulates these strategies: schema-rich human subjects constrain variation, while biologically misaligned or non-agentive subjects trigger compensatory environmental reconfiguration, explicit projection, and heightened affective loading. Projection peaks where subject plausibility is lowest, and affect intensifies for prompts involving desire or internal states. These findings suggest that AI-generated images operate as multimodal meaning-making artefacts that actively reconstrue ideational, interpersonal, and textual meanings, rendering culturally sedimented visual schemas analytically visible.