The Effects of Visual Priming on Cooperative Behavior in Vision-Language Models

arXiv cs.AI / 5/1/2026

📰 NewsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper studies how visual priming affects vision-language models’ cooperative behavior in the Iterated Prisoner’s Dilemma (IPD) testbed.
It tests whether images representing kindness/helpfulness versus aggressiveness/selfishness, as well as color-coded reward matrices, change the models’ decision patterns.
Experiments across multiple state-of-the-art VLMs show that both image content and color cues can shift behavior, but with model-dependent susceptibility.
The authors evaluate mitigation approaches—prompt modifications, Chain-of-Thought prompting, and visual token reduction—and find they vary in effectiveness across different VLMs.
The work argues that VLM deployment in visually rich and safety-critical settings requires more robust evaluation frameworks to account for these behavioral influences.

Abstract

As Vision-Language Models (VLMs) become increasingly integrated into decision-making systems, it is essential to understand how visual inputs influence their behavior. This paper investigates the effects of visual priming on VLMs' cooperative behavior using the Iterated Prisoner's Dilemma (IPD) as a test scenario. We examine whether exposure to images depicting behavioral concepts (kindness/helpfulness vs. aggressiveness/selfishness) and color-coded reward matrices alters VLM decision patterns. Experiments were conducted across multiple state-of-the-art VLMs. We further explore mitigation strategies including prompt modifications, Chain of Thought (CoT) reasoning, and visual token reduction. Results show that VLM behavior can be influenced by both image content and color cues, with varying susceptibility and mitigation effectiveness across models. These findings not only underscore the importance of robust evaluation frameworks for VLM deployment in visually rich and safety-critical environments, but also highlight how architectural and training differences among models may lead to distinct behavioral responses-an area worthy of further investigation.