AI models fail at robot control without human-designed building blocks but agentic scaffolding closes the gap

THE DECODER / 4/2/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

Key Points

  • Nvidia, UC Berkeley, and Stanford propose a framework that systematically evaluates how well AI models can control robots using code-based setups.
  • The study finds that without human-designed abstractions or building blocks, even leading AI models struggle to achieve reliable robot control.
  • The gap can be substantially reduced by “agentic scaffolding,” particularly by applying targeted test-time compute scaling during execution.
  • Overall, the results suggest that combining AI with structured tooling/abstractions may be crucial for robust real-world robot control rather than relying on raw model capability alone.

A new framework from Nvidia, UC Berkeley, and Stanford systematically tests how well AI models can control robots through code. The findings: without human-designed abstractions, even top models fail, but methods like targeted test-time compute scaling closes the gap.

The article AI models fail at robot control without human-designed building blocks but agentic scaffolding closes the gap appeared first on The Decoder.