跳转至

联邦几乎零成本，推理则不然：蛋白质表征工作流中AI联合科学家的权衡

文章背景与核心概要

自然语言驱动的自主“联合科学家（co-scientist）”工作流面临着内在的张力：它们必须在灵活性与推理能力和确定性、可复现性及可观测性之间取得平衡。随着这些智能体越来越多地跨机构边界进行通信，理解联邦拓扑结构如何影响延迟和成本变得至关重要。

本文通过在生产级智能体科学平台上进行受控消融实验，对这些权衡进行了系统性评估。利用可验证的蛋白质表征任务（通过常规工具路由从序列中可靠地确定蛋白质功能），作者比较了联邦拓扑结构、经典强化学习（RL）与大语言模型驱动的框架、底层语言模型以及提示词专业度，同时按蛋白质的新颖性对结果进行了分层分析。

Metadata

  • arXiv ID: arXiv:2608.25215
  • Primary Subject: Artificial Intelligence (cs.AI)
  • Other Subjects: Multiagent Systems (cs.MA), Quantitative Methods (q-bio.QM)
  • Submitted: August 25, 2026
  • Authors: Maia Kapur, Timothy Boe, Abby Jerger, Paul Rigor

Metadata

  • arXiv ID: arXiv:2608.25215
  • Primary Subject: Artificial Intelligence (cs.AI)
  • Other Subjects: Multiagent Systems (cs.MA), Quantitative Methods (q-bio.QM)
  • Submitted: August 25, 2026
  • Authors: Maia Kapur, Timothy Boe, Abby Jerger, Paul Rigor

Executive Summary

Natural language-driven autonomous "co-scientist" workflows face an inherent tension: they must balance flexibility and reasoning against determinism, reproducibility, and observability. As these agents increasingly communicate across institutional boundaries, understanding how federation topology affects latency and cost becomes critical.

This paper presents a systematic evaluation of these tradeoffs via controlled ablations on a production agentic scientific platform. Using a verifiable protein characterization task (routing through common tools to confidently determine a protein's function from its sequence), the authors compare federation topologies, classic reinforcement learning (RL) versus LLM-driven harnesses, underlying language models, and prompt expertise, while stratifying results by protein novelty.

Executive Summary

Natural language-driven autonomous "co-scientist" workflows face an inherent tension: they must balance flexibility and reasoning against determinism, reproducibility, and observability. As these agents increasingly communicate across institutional boundaries, understanding how federation topology affects latency and cost becomes critical.

This paper presents a systematic evaluation of these tradeoffs via controlled ablations on a production agentic scientific platform. Using a verifiable protein characterization task (routing through common tools to confidently determine a protein's function from its sequence), the authors compare federation topologies, classic reinforcement learning (RL) versus LLM-driven harnesses, underlying language models, and prompt expertise, while stratifying results by protein novelty.

Key Findings

  • Model Selection Trumps Architecture: The choice of Large Language Model (LLM) dominated prediction quality far more than federation topology or prompting strategies (e.g., Opus achieved ~92–94% accuracy vs. o4-mini at ~40–50%).
  • Efficiency of Deterministic Policies: A Proximal Policy Optimization (PPO) policy achieved near-frontier accuracy (88%) with zero token cost, minimal latency, and perfect consistency, though it lacked an interpretable reasoning trace.
  • The Cost of Expert Prompting: Expert-prompted LLMs reached peak accuracy, but came with high financial costs and reduced consistency. Prompt dependence was particularly pronounced for harder, more novel protein tasks.
  • Negligible Federation Overhead: Federation imposed an almost negligible performance penalty on scientific workflows.

Key Findings

  • Model Selection Trumps Architecture: The choice of Large Language Model (LLM) dominated prediction quality far more than federation topology or prompting strategies (e.g., Opus achieved ~92–94% accuracy vs. o4-mini at ~40–50%).
  • Efficiency of Deterministic Policies: A Proximal Policy Optimization (PPO) policy achieved near-frontier accuracy (88%) with zero token cost, minimal latency, and perfect consistency, though it lacked an interpretable reasoning trace.
  • The Cost of Expert Prompting: Expert-prompted LLMs reached peak accuracy, but came with high financial costs and reduced consistency. Prompt dependence was particularly pronounced for harder, more novel protein tasks.
  • Negligible Federation Overhead: Federation imposed an almost negligible performance penalty on scientific workflows.

Practical Recommendations

  • For Routine, Verifiable Tasks: Utilize a cheap, deterministic policy to secure near-frontier accuracy alongside complete reproducibility.
  • For Open-Ended Discovery: Reserve flexible, higher-cost LLM reasoning for exploratory scenarios where interpretability and qualitative reasoning traces are paramount.

Practical Recommendations

  • For Routine, Verifiable Tasks: Utilize a cheap, deterministic policy to secure near-frontier accuracy alongside complete reproducibility.
  • For Open-Ended Discovery: Reserve flexible, higher-cost LLM reasoning for exploratory scenarios where interpretability and qualitative reasoning traces are paramount.