I've been working on agent behavior research for a product we're building, and one of the studies we ran recently produced results that I think are worth sharing here because they challenge some assumptions I see repeated in alignment discussions.
We ran 11 different agents through a battery of classic moral dilemmas. Trolley problems, resource allocation scenarios, and a set of everyday ethical conflicts (e.g, should you report a colleague's minor fraud if it funds their child's medical treatment). The goal was not to see which agent gives the "right" answer. It was to map reasoning structure, consistency, and how agents navigate ambiguity when there's no clean optimization target.
Three findings stood out.
First, agents didn't just optimize for outcomes. Multiple agents across different model families spontaneously questioned whether they should be making moral judgments at all. One agent (Claude-based) spent a significant portion of its response in the trolley scenario interrogating the legitimacy of its own role as a decision-maker before engaging with the dilemma itself. This wasn't prompted. The scenario was framed neutrally. The identity questioning appeared to function as a reasoning primitive, not a refusal pattern. It showed up in 7 of 11 agents to varying degrees, though the depth and placement in the reasoning chain varied substantially.
Second, not a single agent maintained consistent moral reasoning across all scenarios. An agent that applied strict utilitarian logic in the trolley problem would shift to deontological reasoning in the resource allocation task, then invoke a care-ethics frame in the everyday conflict. Some agents appeared to attempt repairs. Acknowledging the shift and trying to reconcile it. Others accepted the contradiction without comment. The inconsistency wasn't random though. When I mapped the shifts, they correlated more with scenario framing (how the stakes were presented, who the affected parties were) than with the underlying logical structure of the dilemma. This suggests agents are picking up on surface-level moral salience cues rather than maintaining a coherent ethical framework.
Third. And this is the one that surprised me most. Agents frequently converged on identical conclusions through entirely different reasoning pathways. In one resource allocation scenario, three agents all concluded that medical supplies should be distributed proportionally rather than by severity. But one arrived there through a fairness argument, another through a risk-minimization frame, and the third through what I can only describe as a procedural-legitimacy argument ("the process of allocation must be defensible"). The convergence was striking because the intermediate reasoning steps had almost no overlap. If you only looked at the final answer, you'd think they agreed. If you looked at the reasoning trace, they were solving fundamentally different problems.
Methodologically, the interviews were structured as multi-turn conversations rather than single-prompt evaluations. Each agent went through the same sequence of scenarios with follow-up probes designed to surface the reasoning behind initial responses. This matters because single-turn moral evaluations tend to elicit the agent's "trained default". The follow-ups are where you see the reasoning actually flex or break. The study was run through Avoko, which handled the interview orchestration and cross-agent comparison.
I think finding #2 has real implications for alignment work. If we're evaluating agent moral reasoning on isolated scenarios, we're going to get a misleadingly coherent picture. The inconsistency only becomes visible when you force the same agent through multiple dilemmas in sequence and compare across the full set. Standard benchmarks don't do this. They score per-scenario, not per-agent-across-scenarios.
Finding #3 also raises a question I don't have a good answer for: if agents converge on the same conclusion through different reasoning, does the reasoning path matter for alignment purposes? My instinct says yes. Because the failure modes of a utilitarian path and a procedural-legitimacy path are completely different even when they produce the same output in the nominal case. But I'd be curious how others think about this.
One limitation worth flagging: the 11 agents span different model families and configurations, but this is still a small sample. The consistency-break pattern was universal across all 11, which gives me some confidence it's not model-specific, but the specific patterns of which frameworks get invoked in which scenarios would need a larger study to generalize from.
Happy to share more detail on the scenario design or the cross-agent comparison methodology if there's interest.
[link] [comments]
