AI Navigate

Improving LLM Performance Through Black-Box Online Tuning: A Case for Adding System Specs to Factsheets for Trusted AI

arXiv cs.AI / 3/13/2026

📰 NewsDeveloper Stack & InfrastructureIdeas & Deep AnalysisModels & Research
Read original →
共有:

Key Points

  • The paper proposes a black-box online controller that uses only end-to-end measurements over short segments and hill climbing to maximize goodput for LLM serving without internal instrumentation.
  • It provides empirical evidence that this black-box online approach is well-founded and effective in practice.
  • The method is demonstrated on LLM serving as a concrete example and shows potential throughput gains while meeting service-level objectives.
  • The authors argue for integrating system performance and sustainability metrics into Factsheets to help organizations adopting AI systems.

Abstract

In this paper, we present a novel black-box online controller that uses only end-to-end measurements over short segments, without internal instrumentation, and hill climbing to maximize goodput, defined as the throughput of requests that satisfy the service-level objective. We provide empirical evidence that this design is well-founded. Using this advance in LLM serving as a concrete example, we then discuss the importance of integrating system performance and sustainability metrics into Factsheets for organizations adopting AI systems.

関連おすすめサービス

※当サイトはアフィリエイト広告を利用しています
🎧

Notta搭載AI議事録イヤホン ZENCHORD1

AI時代の仕事術。Notta搭載で会議の議事録を自動生成するスマートイヤホン。

🎙️

AI搭載ボイスレコーダー Plaud

世界100万人が愛用。AIで文字起こし・要約を自動化するボイスレコーダー。

🖼️

画像高画質化AIツール Aiarty Image Enhancer

AIで画像を高画質化。写真・イラストを簡単にアップスケール。