Adversarial Prompt Injection Attack on Multimodal Large Language Models

arXiv cs.CV / 4/1/2026

📰 NewsSignals & Early TrendsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper investigates a new class of adversarial prompt injection that targets multimodal large language models by embedding malicious instructions in the visual modality.
It proposes a method that adaptively embeds a malicious prompt into an input image using a bounded text overlay, while iteratively optimizing imperceptible visual perturbations to match internal feature representations of malicious visual/textual targets.
The visual target is constructed as a text-rendered image and progressively refined during optimization to improve semantic fidelity and transferability across models.
Experiments across two multimodal understanding tasks and multiple closed-source MLLMs show the proposed approach outperforms existing prompt-injection techniques that mainly rely on textual or human-observable visual prompts.

Abstract

Although multimodal large language models (MLLMs) are increasingly deployed in real-world applications, their instruction-following behavior leaves them vulnerable to prompt injection attacks. Existing prompt injection methods predominantly rely on textual prompts or perceptible visual prompts that are observable by human users. In this work, we study imperceptible visual prompt injection against powerful closed-source MLLMs, where adversarial instructions are embedded in the visual modality. Our method adaptively embeds the malicious prompt into the input image via a bounded text overlay to provide semantic guidance. Meanwhile, the imperceptible visual perturbation is iteratively optimized to align the feature representation of the attacked image with those of the malicious visual and textual targets at both coarse- and fine-grained levels. Specifically, the visual target is instantiated as a text-rendered image and progressively refined during optimization to more faithfully represent the desired semantics and improve transferability. Extensive experiments on two multimodal understanding tasks across multiple closed-source MLLMs demonstrate the superior performance of our approach compared to existing methods.