Perturbation Probing: A Two-Pass-per-Prompt Diagnostic for FFN Behavioral Circuits in Aligned LLMs

arXiv cs.CL / 5/1/2026

📰 NewsDeveloper Stack & InfrastructureIdeas & Deep AnalysisModels & Research

共有:

Key Points

Perturbation probing is introduced as a two-forward-pass, no-backprop method to generate causal hypotheses about FFN neuron circuits in aligned LLMs.
The approach identifies two major circuit structures across multiple models and architectures: opposition circuits (linked to RLHF suppressing pre-training tendencies) and routing circuits (linked to pre-training behaviors distributed through attention).
For safety refusal behavior, roughly 50 neurons control the refusal template; ablating them changes about 80% of response formats on 520 AdvBench prompts while keeping harmful compliance near zero.
For language selection, directional intervention can switch English-to-Chinese on 99.1% of 580 benchmark prompts in a subset of models, but the method fails broadly, defining practical limits of directional steering.
A key metric, the FFN-to-skip signal ratio computed from the same two passes, distinguishes circuit types and helps predict when the intervention will work; circuit topology also varies by architecture (e.g., Qwen vs. Gemma).

Abstract

Perturbation probing generates task-specific causal hypotheses for FFN neurons in large language models using two forward passes per prompt and no backpropagation, followed by a one-time intervention sweep of about 150 passes amortized across all identified neurons. Across eight behavioral circuits, 13 models, and four architecture families, we identify two circuit structures that organize LLM behavior. Opposition circuits appear when RLHF suppresses a pre-training tendency. In safety refusal, about 50 neurons, or 0.014 percent of all neurons, control the refusal template; ablating them changes 80 percent of response formats on 520 AdvBench prompts while producing near-zero harmful compliance, 3 of 520 cases, all with disclaimers. Routing circuits appear for pre-training behaviors distributed through attention. For language selection, residual-stream direction injection switches English to Chinese output on 99.1 percent of 580 benchmark prompts in the 3 of 19 tested models that satisfy three observed conditions: bilingual training, FFN-to-skip signal ratio between 0.3 and 1.1, and linear representability. The same intervention fails on the other 16 models and on math, code, and factual circuits, defining the limits of directional steering. The FFN-to-skip signal ratio, computed from the same two forward passes, distinguishes the two structures and predicts the appropriate intervention. Circuit topology varies by architecture, from Qwen's concentrated FFN bottleneck to Gemma's normalization-shielded circuit. In Qwen3.5-2B, ablating 20 neurons eliminates multi-turn sycophantic capitulation, while amplifying 10 related neurons improves factual correction from 52 percent to 88 percent on 200 TruthfulQA prompts. These results show that perturbation probing offers mechanistic insight into RLHF-organized behavior and a practical toolkit for precision template-layer editing.