任务的算力天花板：Transformer 究竟何时能脱离思维链取得成功？

Ceiling of a Task: When Can a Transformer Succeed Without Its Chain of Thought?

arXiv ID: 2609.33134 [cs.AI]
Submitted: September 27, 2026
Authors: Jiashu He, Jinxuan Fan, Xiao Xiao, Radu Marculescu, Alejandro Ribeiro

文章背景与核心概要

近年来，以 OpenAI o1 为代表的前沿推理模型在给出最终答案前，总会生成冗长而详尽的“思维链” (Chain of Thought, CoT) 。然而学术界与工业界一直存在激烈争论：这些看似深思熟虑的中间步骤，究竟是在切实执行复杂的深层计算，还是仅仅为了迎合人类思考习惯的表面修饰？来自宾夕法尼亚大学等机构的研究团队 (Jiashu He, Jinxuan Fan, Xiao Xiao, Radu Marculescu, Alejandro Ribeiro) 从电路复杂度的视角切入，将 Transformer 抽象为浅层电路，并提出了“任务天花板” (Ceiling of a Task) 这一全新概念。该研究通过严谨的数学证明与实证测试表明，对于具有强串行依赖特性的复杂任务，一旦剔除或破坏思维链，模型准确率就会不可逆地跌落至任务的理论天花板乃至纯粹的随机猜测水平。这项工作不仅为理解思维链的必要性提供了坚实的理论基石，也清晰划定了 Transformer 在不借助思维链时所能触及的能力边界。

📌 执行摘要

📌 Executive Summary

本文深入探讨了推理模型在输出前生成的长长“思维链” (Chain of Thought, CoT) ，究竟是在切实分步执行真正的计算工作，还是仅仅流于形式的表面装饰。通过将 Transformer 抽象为理论计算机科学中的“浅层电路”，作者团队提出了 任务天花板 (task's ceiling) 这一核心概念——即浅层计算电路在固定次数的前向传播中，所能达到的理论最高准确率极限。

This paper investigates whether the "chains of thought" generated by reasoning models perform genuine computational work or merely serve as decorative outputs. By modeling transformers as shallow circuits, the authors introduce the concept of a task's ceiling—the upper limit of accuracy achievable by a shallow circuit in a constant number of passes.

通过严谨的理论证明，并结合在有限群字问题 (finite group word problems) 以及 MATH-500、AIME 等前沿数学基准上的实证评测，研究团队证实：一旦人为抹除或破坏思维链，模型的解题准确率便会如理论预言一般迅速跌落，其性能极限被牢牢锁死在任务自身的结构天花板之下 (而在前后步骤具有严格依赖关系的最大串行任务中，表现更是会彻底崩塌至随机瞎猜水平) 。归根结底，“任务天花板”这把理论标尺，清晰而有力地界定了 Transformer 何时能够脱离思维链自如应对，何时又必须高度依赖思维链才能取得突破。

Through theoretical proofs and empirical evaluations on finite group word problems, MATH-500, and AIME datasets, the study demonstrates that removing or corrupting the chain of thought predictably degrades model performance down to the task's structural ceiling (or to random chance for maximally serial tasks). Ultimately, the "ceiling of a task" successfully delineates when a transformer can—and cannot—succeed without its chain of thought.

📑 论文摘要

📑 Abstract

推理模型在给出最终答案之前往往会生成长长的思维链，但这些中间内容究竟是在进行货真价实的计算，还是很大程度上只是流于形式的表面装饰，学术界对此一直存有争议。我们通过将 Transformer 视为计算理论中的“浅层电路” (shallow circuit) ，对这一核心问题展开了系统研究。

Reasoning models generate long chains of thought before they answer, yet it is debated whether the content of these chains does real computational work or is largely decorative. We study this question by viewing a transformer as a shallow circuit.

在固定网络层数的 Transformer 中，单次前向传播的计算深度是恒定的常数，因此只要运行模型的次数是固定的常数，整个计算过程在本质上就是一个浅层电路。我们将浅层电路在某一任务上所能达到的极限准确率称为该任务的 天花板 (ceiling) ；如果一个任务的理论天花板小于 1 (即无法被浅层电路完美解决) ，那么该任务便被定义为 串行任务 (serial task) 。

One forward pass through a fixed number of layers has constant depth, so any procedure that runs the model a constant number of times is a shallow circuit. We call the best accuracy that a shallow circuit can reach on a task the ceiling of the task, and a task is serial if its ceiling lies below one.

针对串行任务，无论模型采用了何种训练方式，我们严格证明了对所有 Transformer 均普适成立的三大核心结论：

1. 必要性 (Necessity)：将思维链替换为任何不依赖其具体内容的形式 (例如无意义的填充 Token 或对题目的单纯复述) ，都会导致模型准确率直线下跌至任务天花板；而在最大串行任务中，准确率更是会跌至随机猜测水平。
2. 深度门槛 (Depth)：对于准确率成功突破天花板的模型，绝不可能存在任何浅层计算能够生成其思维链，哪怕仅仅是生成大致的近似文本也无法实现。
3. 局部性 (Locality)：一旦思维链生成完毕，模型只需经过一次浅层前向计算便能直接得出最终答案，这证明所有复杂的串行推理实质上全都发生在思维链的推导过程之中。

We prove three results on serial tasks that hold for every transformer, no matter how it was trained: 1. Necessity: Replacing the chain by anything that does not depend on its content, such as filler tokens or a restatement of the question, drives the accuracy down to the ceiling, and on a maximally serial task down to chance. 2. Depth: No shallow computation can write the chain of a model whose accuracy exceeds the ceiling, not even approximately. 3. Locality: The answer is one shallow pass away from the finished chain, so all of the serial reasoning happens in the chain.

在理论天花板已知的有限群字问题 (word problems of finite groups) 上，无论是否借助强化学习 (reinforcement learning, RL) ，从零训练的小型 Transformer 都精准复现了理论预测的数值：经过思维链训练的模型能够完美应对任意长度的输入，但一旦将思维链擦除，准确率立即滑落至随机瞎猜水平；未经过思维链训练的模型，其准确率随着输入长度的增加不可阻挡地跌向任务天花板；而主流开源权重推理模型在面对以自然语言表述的相同问题时，一旦失去思维链，其性能同样跌回基准线。

On word problems of finite groups, whose ceilings are known, small transformers trained from scratch, with or without reinforcement learning, attain the predicted numbers: chain-trained models solve every input length and fall to chance when the chain is erased, chainless models collapse to the ceiling as the input length grows, and open-weight reasoning models given the same problem in words return to the baseline without their chain.

在颇具挑战性的 MATH-500 和 AIME 高难度数学基准上，直接抹去思维链会导致开源推理模型的准确率剧烈暴跌 0.52 至 0.82；实验还揭示了一个关键细节：将思维链中的句子次序打乱 (sentence shuffle) 对准确率几乎毫无影响，但一旦进行细粒度的 Token 级随机打乱 (token shuffle) ，其破坏性便与彻底抹除思维链完全一样。对于使用 GRPO 算法 (无论配合真实奖励还是随机奖励) 训练出的模型检查点，上述规律同样完全成立。由此可见，“任务天花板”这一理论工具，确切解答了 Transformer 何时能够脱离思维链取得成功。

On MATH-500 and AIME, erasing the chain costs open reasoning models 0.52 to 0.82 accuracy, a sentence shuffle is harmless, and a token shuffle is as harmful as erasing; the same holds for checkpoints trained by GRPO with a correct or a random reward. The ceiling of a task therefore answers when a transformer can succeed without its chain of thought.

📋 论文元数据与更多信息

📋 Metadata & Additional Information

Field	Details
Subjects	Artificial Intelligence (cs.AI)
Cite as	arXiv:2609.33134 [cs.AI]
DOI	10.48550/arXiv.2609.33134
License	Creative Commons Attribution 4.0 International
Full-Text Links	View PDF | HTML Version | TeX Source