文章背景与核心概要
大型语言模型驱动的自主研究循环往往会遭遇“漂移（drift）”现象——它们倾向于通过局部的微调来渐进式优化目标指标，而非去检验那些启发实验的底层假设。为了解决这一问题，本文作者引入了一个结构稳健的AI科学家，专门用于在仿真环境中研究四足机器人导航策略的泛化能力。
在扩展Karpathy提出的 autoresearch 范式的基础上，该研究循环引入了三大核心创新：1. 不可变的实验卡片（Immutable Experiment Card），通过固定模式将每次迭代的预测与其结果配对，防止对被证伪的假设进行追溯性篡改；2. 职责受限的专用子代理；3. 堪布（Kkanbu），一个作为类型化知识Graph维护用户研究品味、并作为唯一允许做出主观判断的偏好预言机（preference oracle）。
通过在十一个研究流（分别包含和不包含 kkanbu）中运行相同的循环，研究人员隔离出了该预言机的影响。有趣的是，两组均未发生漂移，两组都成功证伪了大约75%的自身假设，并且没有预言机的一组最终产出了训练效果最好的策略。预言机的作用并非提高原始得分，而是引导研究方向：它独家开创了测试时适应（test-time adaptation），在其所属组别中设计出了获胜方案，并将关键经验跨研究流传递（避免了无预言机组反复重新推导这些经验）。最终，坚固的结构支架保证了研究循环的诚实性，而 kkanbu 则赋予了它寻找正确方向的主观“品味”。
An AI Scientist that Doesn't Drift: Taste, Structure, and Falsifiable Findings in a Quadruped Navigation Research Loop
arXiv: 2608.07542 [cs.AI]
Submitted on: 30 July 2026
Authors: Yiwen Zhang, Eloise Zeng, Jaeha Lee, Tony Yue Yu
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multiagent Systems (cs.MA); Robotics (cs.RO)
arXiv: 2608.07542 [cs.AI]
Submitted on: 30 July 2026
Authors: Yiwen Zhang, Eloise Zeng, Jaeha Lee, Tony Yue Yu
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multiagent Systems (cs.MA); Robotics (cs.RO)
Summary
Autonomous research loops powered by large language models often experience "drift"—gradually optimizing targeted metrics through local refinements rather than testing the underlying hypotheses that inspired the experiments.
Autonomous research loops powered by large language models often experience "drift"—gradually optimizing targeted metrics through local refinements rather than testing the underlying hypotheses that inspired the experiments.
To combat this, the authors introduce a structurally robust AI Scientist designed to study generalization in quadruped robot navigation policies within simulation. Expanding on Karpathy’s autoresearch paradigm, the loop incorporates three key innovations: 1. An Immutable Experiment Card: Pairs each iteration's prediction with its outcome under a fixed schema, preventing retroactive altering (retconning) of falsified hypotheses. 2. Specialized Subagents: Restricted strictly to mechanical roles. 3. Kkanbu: A preference oracle that maintains the user's research taste as a typed knowledge graph and acts as the sole component permitted to make subjective judgments.
To combat this, the authors introduce a structurally robust AI Scientist designed to study generalization in quadruped robot navigation policies within simulation. Expanding on Karpathy’s autoresearch paradigm, the loop incorporates three key innovations: 1. An Immutable Experiment Card: Pairs each iteration's prediction with its outcome under a fixed schema, preventing retroactive altering (retconning) of falsified hypotheses. 2. Specialized Subagents: Restricted strictly to mechanical roles. 3. Kkanbu: A preference oracle that maintains the user's research taste as a typed knowledge graph and acts as the sole component permitted to make subjective judgments.
By running identical loops across eleven research streams—both with and without kkanbu—the researchers isolated the oracle's impact. Interestingly, neither arm drifted, with both arms successfully falsifying roughly 75% of their own hypotheses, and the oracle-less arm ultimately producing the best-trained policy.
By running identical loops across eleven research streams—both with and without kkanbu—the researchers isolated the oracle's impact. Interestingly, neither arm drifted, with both arms successfully falsifying roughly 75% of their own hypotheses, and the oracle-less arm ultimately producing the best-trained policy.
Instead of boosting raw scores, the oracle steered direction: it uniquely pioneered test-time adaptation, authored the winning designs where its arm led, and carried vital lessons across streams that the oracle-less arm repeatedly had to re-derive. Ultimately, the rigid structural scaffold keeps the loop honest, while kkanbu provides the subjective "taste" of where it should look.
Instead of boosting raw scores, the oracle steered direction: it uniquely pioneered test-time adaptation, authored the winning designs where its arm led, and carried vital lessons across streams that the oracle-less arm repeatedly had to re-derive. Ultimately, the rigid structural scaffold keeps the loop honest, while kkanbu provides the subjective "taste" of where it should look.