Modeling Clinical Concern Trajectories in Language Model Agents

arXiv cs.AI / 5/1/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper examines how LLM agents in clinical settings can abruptly escalate risk due to threshold-driven behavior, which obscures earlier warning signals.
It proposes a lightweight agent architecture that integrates a memoryless clinical risk encoder over time using first- and second-order state dynamics to generate a continuous escalation-pressure signal.
Experiments in synthetic ward scenarios show that purely stateless agents produce sharp “escalation cliffs,” while second-order dynamics yield smoother, more anticipatory concern trajectories even when escalation timing is similar.
The resulting trajectories provide sustained pre-escalation unease that supports human-in-the-loop monitoring and potentially more informed clinical interventions.
The authors argue that explicit state dynamics improve the clinical interpretability of LLM agents by making it visible how long concern has been increasing, not only when thresholds are crossed.

Abstract

Large language model (LLM) agents deployed in clinical settings often exhibit abrupt, threshold-driven behavior, offering little visibility into accumulating risk prior to escalation. In real-world care, however, clinicians act on gradually rising concern rather than instantaneous triggers. We study whether explicit state dynamics can expose such pre-escalation signals without delegating clinical authority to the agent. We introduce a lightweight agent architecture in which a memoryless clinical risk encoder is integrated over time using first- and second-order dynamics to produce a continuous escalation pressure signal. Across synthetic ward scenarios, stateless agents exhibit sharp escalation cliffs, while second-order dynamics produce smooth, anticipatory concern trajectories despite similar escalation timing. These trajectories surface sustained unease prior to escalation, enabling human-in-the-loop monitoring and more informed intervention. Our results suggest that explicit state dynamics can make LLM agents more clinically legible by revealing how long concern has been rising, not just when thresholds are crossed.