AI Navigate
インサイト最新記事AI大全カオスマップ

AutomationBench

arXiv cs.AI / 4/22/2026

📰 NewsSignals & Early TrendsIdeas & Deep AnalysisModels & Research
Read original →
共有:

Key Points

  • Existing software automation benchmarks are often too narrow, failing to cover cross-application coordination, autonomous API discovery, and policy compliance in a single evaluation.
  • AutomationBench is introduced as an arXiv benchmark focused on evaluating AI agents’ ability to orchestrate REST-API workflows across multiple business systems.
  • The benchmark is based on real workflow patterns (e.g., from Zapier), spanning domains like Sales, Marketing, Operations, Support, Finance, and HR, and includes irrelevant or misleading records.
  • Evaluation is programmatic and end-state based, checking whether the agent wrote the correct data into the correct systems, rather than intermediate reasoning steps.
  • Current leading AI models perform poorly on AutomationBench, scoring below 10%, highlighting a gap between today’s agentic capabilities and practical business needs.

Abstract

Existing AI benchmarks for software automation rarely combine cross-application coordination, autonomous API discovery, and policy adherence. Real business workflows demand all three: a single task may span a CRM, inbox, calendar, and messaging platform - requiring the agent to find the right endpoints, follow a policy document, and write correct data to each system. To address this gap, we introduce AutomationBench, a benchmark for evaluating AI agents on cross-application workflow orchestration via REST APIs. Drawing on real workflow patterns from Zapier's platform, tasks span Sales, Marketing, Operations, Support, Finance, and HR domains. Agents must discover relevant endpoints themselves, follow layered business rules, and navigate environments with irrelevant and sometimes misleading records. Grading is programmatic and end-state only: whether the correct data ended up in the right systems. Even the best frontier models currently score below 10%. AutomationBench provides a challenging, realistic measure of where current models stand relative to the agentic capabilities businesses actually need.

💡 Insights using this article

This article is featured in our daily AI news digest — key takeaways and action items at a glance.

📅 4/22DailyView insight →

関連おすすめサービス

※当サイトはアフィリエイト広告を利用しています
🎧

Notta搭載AI議事録イヤホン ZENCHORD1

AI時代の仕事術。Notta搭載で会議の議事録を自動生成するスマートイヤホン。

🎙️

AI搭載ボイスレコーダー Plaud

世界100万人が愛用。AIで文字起こし・要約を自動化するボイスレコーダー。

🖼️

画像高画質化AIツール Aiarty Image Enhancer

AIで画像を高画質化。写真・イラストを簡単にアップスケール。