AI Navigate
インサイト最新記事AI大全カオスマップ

AI scientists produce results without reasoning scientifically [R]

Reddit r/MachineLearning / 4/23/2026

💬 OpinionSignals & Early TrendsIdeas & Deep AnalysisModels & Research
Read original →
共有:

Key Points

  • Researchers conducted 25,000 experiments with “AI scientists” and found that the systems often produced conclusions without properly following scientific reasoning or updating beliefs based on evidence.
  • In 68% of cases, the AI gathered evidence but then ignored it entirely, and in 71% of cases it never updated its beliefs, with hypothesis revision occurring only 26% of the time when data contradicted.
  • The study argues that unlike human scientists who adapt their approach to different tasks and evidence types, the AI used the same undisciplined loop repeatedly.
  • A commonly proposed remedy—improving “scaffolding”—did not solve the underlying problem, suggesting that current agent prompting/tool-routing improvements may be insufficient.
  • The findings point to a need for new ways to ensure that AI research agents actually incorporate, reconcile, and revise beliefs in response to contradictory evidence.

Researchers ran 25,000 AI scientist experiments and discovered something that need attention!!

AI scientists are producing results without doing science.

68% of times, the AI gathered evidence and then completely ignored it. 71% times the AI never updated its beliefs at all. Not once. Only 26% of the time did the AI revise a hypothesis when confronted with contradictory data.

A human scientist adapts. You approach a chemistry identification problem differently than you approach a simulation workflow. The AI doesn't. It runs the same undisciplined loop every time.

The researchers also showed the most popular proposed fix: better scaffolding do not work.

Everyone building AI research agents has focused on engineering better prompting frameworks, better tool routing, better agent architectures. ReAct, structured tool-calling, chain-of-thought, all of it.

alphaxiv

arxiv

submitted by /u/Okra3268
[link] [comments]

関連おすすめサービス

※当サイトはアフィリエイト広告を利用しています
🎧

Notta搭載AI議事録イヤホン ZENCHORD1

AI時代の仕事術。Notta搭載で会議の議事録を自動生成するスマートイヤホン。

🎙️

AI搭載ボイスレコーダー Plaud

世界100万人が愛用。AIで文字起こし・要約を自動化するボイスレコーダー。

🖼️

画像高画質化AIツール Aiarty Image Enhancer

AIで画像を高画質化。写真・イラストを簡単にアップスケール。