跳转至

大语言模型中的长视界状态追踪：通过深度依赖工具调用链执行MD5

文章背景与核心概要

在大语言模型（LLM）的评测中，长视界（Long-Horizon）任务常常会导致灾难性的错误级联，因为每个步骤都严格依赖于前一步。传统的智能体基准测试很难精确定位这些失败的根本原因，因为它们经常将状态追踪的困难与指令解释混淆，并且容易受到诸如幻觉生成的最终答案等捷径的影响。

本文引入了一个严格的、纯记账式的评估框架：提示LLM通过一个包含196次依赖工具调用（跨越64轮）的序列，从头计算MD5密码散列（RFC 1321），同时在其上下文中维护四个32位字 \((a, b, c, d)\)。研究发现，通过在上下文中保留推理历史以及对启用思考功能的工人进行投票，像 gpt-oss-120b 这样的小激活参数混合专家模型能够在温度为0的条件下成功跨越所有196次调用并返回正确的摘要。该研究清晰地剥离了指令干扰，为评估LLM在长步骤下的精确状态保持能力提供了黄金标准。

长期状态追踪：大语言模型中的长视界状态追踪：通过深度依赖工具调用链执行MD5 (Long-Horizon State Tracking in LLMs: Executing MD5 through a Deep Sequence of Dependent Tool Calls)

license icon 查看许可证

元数据 (Metadata)

  • arXiv ID: arXiv:2609.00012 [cs.AI]
  • 学科分类 (Subjects): 人工智能 (cs.AI); 密码学与安全 (cs.CR); 多智能体系统 (cs.MA)
  • 作者 (Authors): Dheeraj Mohandas Pai, Lu Xian
  • 提交时间 (Submitted): 2026年8月2日
  • 篇幅与图表 (Comments): 6页，2张图表
  • DOI: 10.48550/arXiv.2609.00012

执行摘要 (Executive Summary)

Long-horizon tasks often cause catastrophic error cascades in Large Language Models (LLMs) because each step depends strictly on the previous one. Traditional agentic benchmarks struggle to pinpoint the root causes of these failures, as they frequently conflate state-tracking difficulties with instruction interpretation and remain vulnerable to shortcuts like hallucinated final answers.

长视界任务往往会在大语言模型（LLM）中引起灾难性的错误级联，因为每一步都严格依赖于前一步。传统的智能体基准测试难以精确定位这些失败的根本原因，因为它们经常将状态追踪的困难与指令解释混淆，并且容易受到诸如幻觉生成的最终答案等捷径的影响。

This paper introduces a rigorous, pure-bookkeeping evaluation framework: prompting an LLM to compute an MD5 cryptographic hash from scratch (RFC 1321) through a sequence of \(196\) dependent tool calls spanning \(64\) rounds, while carrying four \(32\)-bit words \((a, b, c, d)\) in its context.

本文引入了一个严谨的、纯记账式的评估框架：提示LLM通过包含 \(196\) 次依赖工具调用（跨越 \(64\) 轮）的序列从头计算 MD5 密码散列（RFC 1321），同时在其上下文中携带四个 \(32\) 位字 \((a, b, c, d)\)。

核心发现 (Key Findings)

  • 状态保持： gpt-oss-120b（一个每个token具有约 5.5B 激活参数的混合专家模型）在温度为 \(0\) 的条件下，成功在所有 \(196\) 次调用中保持了完整状态，并在大多数已完成的运行中返回了正确的摘要。

    State Preservation: gpt-oss-120b (a Mixture-of-Experts model with \(\sim\)5.5B active parameters per token) successfully carries the full state across all \(196\) calls at temperature \(0\) and returns the correct digest on a majority of completed runs.

  • 架构鲁棒性： 即使将每个基础工具替换为第二个LLM——从而创建一个完全从头计算散列而没有精确算术预言机（oracle）的驱动-工作者团队——系统仍然能够正常运行。

    Architecture Robustness: Even when replacing every primitive tool with a second LLM—creating a driver-worker team that computes the hash completely from scratch without an exact-arithmetic oracle—the system remains functional.

  • 核心成功因素： 两个关键要素在不改变模型权重的情况下确保了性能：

  • 在每一轮的上下文中保留模型自己的推理历史。
  • 对启用思考功能的工作者进行投票，以消除微小的模运算错误。

    Core Success Factors: Two key ingredients ensure performance without altering model weights: 1. Retaining the model's own reasoning history in context at each turn. 2. Voting over a thinking-enabled worker to eliminate minor modular-arithmetic errors.

摘要 (Abstract)

Long-horizon tasks remain uncommon in large language model (LLM) evaluation, and for a reason: when each step depends on the last, per-step accuracy that looks excellent in isolation decays catastrophically, as errors cascade and the end-to-end failure probability grows sharply with length. Existing agentic benchmarks report end-to-end success but confound this state-tracking difficulty with instruction interpretation, give no control group that isolates it, and are vulnerable to shortcuts such as a hallucinated final answer, so they cannot say why a long run fails. Whether an LLM can carry exact intermediate state across many tool calls at all is itself not well established. We test this cleanly by having the model compute a cryptographic hash, MD5, step by step: a sequence of \(196\) dependent tool calls over \(64\) rounds while it carries four \(32\)-bit words \((a,b,c,d)\) in its own context from one call to the next. Interpretation is trivial and, because we implement MD5 from scratch (RFC 1321), we align every call to the ground-truth trace and check the digest to the bit, so any failure is pure bookkeeping. gpt-oss-120b, a mixture-of-experts model with only \(\sim\)5.5B active parameters per token, at temperature \(0\) with a short fixed prompt, carries the full state across all \(196\) calls and returns the correct digest on a majority of completed runs. In the strongest setting we replace every primitive tool with a second LLM, so a driver and a worker compute the whole hash from scratch with no exact-arithmetic oracle in the loop. Two ingredients decide success and neither changes the weights: keeping the model's own reasoning in its context each turn, and voting over a thinking-enabled worker to remove its modular-arithmetic slips. We localize the residual failures by origin, separating state-tracking from arithmetic and from serving.

大语言模型（LLM）的评测中很少涉及长视界任务，这是有原因的：当每一步都依赖于前一步时，孤立看表现极佳的单步准确率会发生灾难性的衰减，因为错误会级联，端到端的失败概率会随着长度的增加而急剧上升。现有的智能体基准测试报告了端到端的成功率，但将这种状态追踪的困难与指令解释混淆了，没有提供能将其隔离的对照组，并且容易受到诸如幻觉生成的最终答案等捷径的影响，因此无法解释为什么长流程会失败。LLM是否完全能够在多次工具调用中携带精确的中间状态，本身也尚未得到充分证实。我们通过让模型一步步计算密码散列（MD5）来进行清晰的测试：这是一个包含 \(196\) 次依赖工具调用、跨越 \(64\) 轮的序列，在此期间模型在自己的上下文中将四个 \(32\) 位字 \((a,b,c,d)\) 从一次调用传递到下一次调用。指令解释变得微不足道，并且由于我们从头实现了 MD5（RFC 1321），我们将每一次调用与基准真相（ground-truth）轨迹对齐，并对摘要进行比特级的检查，因此任何失败都纯粹是记账问题。gpt-oss-120b 是一个每个token仅有约 5.5B 激活参数的混合专家模型，在温度为 \(0\) 和一个简短固定提示词的条件下，能够在所有 \(196\) 次调用中维持完整状态，并在大多数已完成的运行中返回正确的摘要。在最强设置中，我们将每个基础工具替换为第二个 LLM，从而使驱动者和工作者从头计算整个散列，循环中没有精确的算术预言机。有两个要素决定了成功，且都不需要改变权重：在每一轮的上下文中保留模型自己的推理，以及对启用思考功能的工作者进行投票以消除其模运算失误。我们按来源定位了剩余的失败，将状态追踪与算术和模型服务区分开来。