跳转至

野外测试时扩展：为什么“开发”而非“探索”才是瓶颈

文章背景与核心概要

测试时扩展（TTS）在数学和代码等验证相对简单的领域中，对于提升大语言模型输出质量表现出了极高的效能。然而，这项最新研究首次在开放式生成基准（涵盖医学、法律、金融、通用对话和创意写作）上，对五大类TTS方法进行了计算量归一化的系统性比较。

通过将推理计算预算分解为“探索”（exploration）与“开发”（exploitation），作者揭示了一个鲜明的对比：探索阶段表现优异，生成更大的候选池能随着计算量的增加持续产出高质量的输出；然而开发阶段却遭遇了严重瓶颈。当前的奖励模型与真实质量的相关性极低（\(\rho_v \approx 0.12\)），树搜索会导致候选多样性崩溃，而迭代改进（Refinement）带来的收益也不稳定。

最终，该研究得出结论：候选池本身很少是瓶颈——如何从中进行有效选择才是。

📌 Summary

Test-time scaling (TTS) has proven exceptionally effective for improving language model outputs in domains where verification is straightforward, such as mathematics and coding. However, this study presents the first compute-normalised comparison of five TTS families across open-ended generation benchmarks (medicine, law, finance, general chat, and creative writing).

By breaking down inference compute budgets into exploration and exploitation, the authors reveal a stark contrast: * Exploration works: Generating larger candidate pools consistently yields better maximum-quality outputs as compute increases. * Exploitation fails: Selecting the best output from the pool is severely bottlenecked. Current reward models correlate poorly (\(\rho_v \approx 0.12\)) with true quality, tree search collapses candidate diversity, and refinement yields inconsistent gains.

Ultimately, the candidate pool is rarely the bottleneck—choosing effectively from it is.

测试时扩展（TTS）已被证明在验证直观的领域（如数学和编程）中，对于改善语言模型输出极其有效。然而，本研究首次在医学、法律、金融、通用对话和创意写作等开放式生成基准上，对五大类TTS方法进行了计算量归一化的比较。

通过将推理计算预算分解为探索（exploration）和开发（exploitation），作者揭示了一个鲜明的对比： * 探索有效： 随着计算量增加，生成更大的候选池能够持续产出更高最高质量的输出。 * 开发失效： 从候选池中选择最佳输出遇到了严重瓶颈。当前奖励模型与真实质量的相关性极低（\(\rho_v \approx 0.12\)），树搜索导致候选多样性崩溃，而迭代改进也仅带来了不稳定的收益。

归根结底，候选池本身很少是瓶颈——从中进行有效选择才是。

📑 Abstract

Test-time scaling (TTS) improves language model outputs by spending additional inference compute - generating multiple candidates, searching over partial sequences, or iteratively refining drafts. These techniques yield large gains on mathematics and code, but have been developed and stress-tested almost exclusively on tasks where verification is straightforward. We conduct the first compute-normalised comparison of five TTS families across five open-ended generation benchmarks spanning medicine, law, finance, general chat, and creative writing - grounded in a unified framework that decomposes the effectiveness of each method's token budget into exploration and exploitation. The answer depends on which side of that decomposition you examine. Scaling exploration works: the best candidate in the pool improves steadily with compute across all settings. What breaks is exploitation - the step that converts a rich candidate pool into a final output. With state-of-the-art generators, reward models correlate at only \(\rho_v \approx 0.12\) with true quality, rendering selection near-random regardless of budget. Tree search amplifies this failure through diversity collapse. Refinement helps on one of five benchmarks; its apparent gains elsewhere are confounded. Only synthesis across candidates (Fusion) consistently improves over single-sample baselines, yet still recovers only ~40% of available quality. The candidate pool is not the bottleneck - choosing from it is.

Test-time scaling (TTS) improves language model outputs by spending additional inference compute - generating multiple candidates, searching over partial sequences, or iteratively refining drafts. These techniques yield large gains on mathematics and code, but have been developed and stress-tested almost exclusively on tasks where verification is straightforward. We conduct the first compute-normalised comparison of five TTS families across five open-ended generation benchmarks spanning medicine, law, finance, general chat, and creative writing - grounded in a unified framework that decomposes the effectiveness of each method's token budget into exploration and exploitation. The answer depends on which side of that decomposition you examine. Scaling exploration works: the best candidate in the pool improves steadily with compute across all settings. What breaks is exploitation - the step that converts a rich candidate pool into a final output. With state-of-the-art generators, reward models correlate at only \(\rho_v \approx 0.12\) with true quality, rendering selection near-random regardless of budget. Tree search amplifies this failure through diversity collapse. Refinement helps on one of five benchmarks; its apparent gains elsewhere are confounded. Only synthesis across candidates (Fusion) consistently improves over single-sample baselines, yet still recovers only ~40% of available quality. The candidate pool is not the bottleneck - choosing from it is.

测试时扩展（TTS）通过消耗额外的推理计算来改进语言模型的输出——例如生成多个候选、在部分序列上进行搜索，或迭代求精草稿。这些技术在数学和代码领域取得了巨大收益，但几乎全是在验证过程简单的任务上开发和进行压力测试的。我们基于一个统一的框架，将每种方法的Token预算效能分解为“探索”与“开发”，首次在涵盖医学、法律、金融、通用对话和创意写作的五个开放式生成基准上，对五大类TTS方法进行了计算量归一化的比较。结果取决于你审视的是分解后的哪一侧。扩展探索是行之有效的：在所有设置下，候选池中的最佳候选者随着计算量的增加而稳步提升。出问题的是“开发”环节——即把丰富的候选池转化为最终输出的步骤。在使用先进生成器的情况下，奖励模型与真实质量的相关性仅为 \(\rho_v \approx 0.12\)，这导致无论预算多少，选择过程都几近随机。树搜索通过多样性崩溃放大了这种失效。迭代精炼仅在五个基准之一上起作用；其在其他地方表现出的增益实则受到混淆因素影响。只有跨候选的综合（Fusion）能够始终如一地优于单样本基准，但也只能恢复约 40% 的可用质量。候选池并非瓶颈所在——从中做出正确选择才是。