Governing What You Cannot Observe: Adaptive Runtime Governance for Autonomous AI Agents

arXiv cs.AI / 4/28/2026

📰 NewsDeveloper Stack & InfrastructureIdeas & Deep AnalysisModels & Research

Key Points

  • The paper argues that autonomous AI agents can become unsafe even without any code changes due to behavioral drift, adapting adversaries, and shifting decision patterns.
  • It introduces the “Informational Viability Principle,” which frames runtime governance as estimating an upper bound on unobserved risk and allowing actions only when the agent’s safety capacity exceeds that bound by a margin.
  • The authors propose an “Agent Viability Framework” based on viability theory, specifying three properties—monitoring, anticipation, and monotonic restriction—that are necessary and jointly sufficient for covering documented agent failure modes.
  • They instantiate the framework with a system called RiskGate, combining statistical risk estimators, a fail-secure monotonic control pipeline, and a closed-loop Autopilot with a kill switch as a last resort.
  • A scalar Viability Index and a first-order prediction of the turning point are used to shift governance from reactive enforcement toward predictive regulation.

Abstract

Autonomous AI agents can remain fully authorized and still become unsafe as behavior drifts, adversaries adapt, and decision patterns shift without any code change. We propose the \textbf{Informational Viability Principle}: governing an agent reduces to estimating a bound on unobserved risk \hat{B}(x) = U(x) + SB(x) + RG(x) and allowing an action only when its capacity S(x) exceeds \hat{B}(x) by a safety margin. The \textbf{Agent Viability Framework}, grounded in Aubin's viability theory, establishes three properties -- monitoring (P1), anticipation (P2), and monotonic restriction (P3) -- as individually necessary and collectively sufficient for documented failure modes. \textbf{RiskGate} instantiates the framework with dedicated statistical estimators (KL divergence, segment-vs-rest z-tests, sequential pattern matching), a fail-secure monotonic pipeline, and a closed-loop Autopilot formalised as an instance of Aubin's regulation map with kill-switch-as-last-resort; a scalar Viability Index VI(t) \in [-1,+1] with first-order t^* prediction transforms governance from reactive to predictive. Contributions are the theoretical framework, the reference implementation, and analytical coverage against published agent-failure taxonomies; quantitative empirical evaluation is scoped as follow-up work.