跳转至

指令复制作为推理时控制基元

文章背景与核心概要

本文研究了指令复制（instruction duplication）这一极简的黑盒推理时控制机制，旨在提升大型语言模型对程序性指令的遵循能力。该方法无需对模型进行任何重新训练或修改解码过程，仅通过简单重复程序性指令，便能显著影响模型的性能、可靠性以及下游系统的实际效能。

在实验评估中，研究人员跨越了七个经过指令微调的模型、300道医学多选题、八种放置条件以及16,800次计划生成进行了测试。结果表明，将指令复制次数从一次增加到两次，能够将“All-8”诊断指标（即通过所有八项可观测测试的响应比例）从 90.22% 提升至 93.17%（提高了 2.95 个百分点），消除了单次复制后剩余失败案例中的 30.2%。此外，该方法在答案工程（Answer Engineering, AE）等下游系统应用中展现出了显著的实用价值，例如将特定医疗终端的系统级 AE 性能从 84.2% 大幅提升至 97.1%。这证明了指令复制作为一种低复杂度、对位置敏感的控制手段，在与消费外显轨迹的下游系统结合时能够释放巨大的实际效能。

执行摘要

本研究探讨了指令复制，这是一种极简的黑盒推理时控制机制，旨在提高大语言模型对程序性指令的遵循能力。无需任何模型重新训练或解码修改，只需简单重复程序性指令，就会影响模型的性能、可靠性和下游系统的有效性。

This paper investigates instruction duplication, a minimal black-box inference-time control mechanism designed to improve procedural instruction following in large language models. Without requiring any model retraining or decoding changes, simply repeating a procedural instruction impacts model performance, reliability, and downstream system effectiveness.

核心发现

  • 改进的确定的诊断响应： 将指令副本从一个增加到两个，使得“All-8”诊断指标（在七个指令微调模型中通过所有八个可观测测试的响应）从 90.22% 提高到 93.17%（提高了 2.95 个百分点），消除了剩余失败案例的 30.2%。
  • 信息检索指标： 临时（Pre-provisional）TF-IDF 召回率从 73.44% 提高到 74.81%（\(p < 0.001\)），而最终答案准确率保持在 60.21% 的稳定水平。
  • 下游实际影响： 在答案工程（AE）中——其中显式轨迹状态决定了局部修复——尾部重复显著提升了性能终点，例如将特定医疗终点的纯系统 AE 从 84.2% 提高到 97.1%。

Key Findings

  • Improved Deterministic Diagnostic Responses: Moving from one to two instruction copies raised the "All-8" diagnostic metric (responses passing all eight observable tests across seven instruction-tuned models) from 90.22% to 93.17% (+2.95 percentage points), eliminating 30.2% of the remaining failures.
  • Information Retrieval Metrics: Pre-provisional TF-IDF recall increased from 73.44% to 74.81% (\(p < 0.001\)), while final-answer accuracy remained stable at 60.21%.
  • Downstream Practical Impact: In Answer Engineering (AE)—where explicit trajectory states dictate local repair—trailing duplicates significantly boosted performance endpoints, such as raising system-only AE from 84.2% to 97.1% for specific medical endpoints.

元数据与参考信息

Metadata & Reference Information

摘要

遵循程序性指令是可控语言模型系统的基本要求，特别是当生成的轨迹在下游被检查或修复时。我们引入了指令复制，这是一种极简的黑盒推理时控制方法，仅重复程序性指令，而无需重新训练或更改解码方式。

Procedural instruction following is a basic requirement for controllable language-model systems, especially when generated trajectories are inspected or repaired downstream. We introduce instruction duplication, a minimal black-box inference-time control that repeats only the procedural instruction, without retraining or decoding changes.

在跨越七个指令微调模型、300道医学多选题、八种放置条件以及16,800次计划生成的实验中，将副本数量从一个增加到两个，使得确定性的 All-8 诊断指标（通过所有八个可观测测试的响应）从 90.22% 提高到 93.17%（+2.95 个百分点），消除了单次复制后剩余失败中的 30.2%。

Across seven instruction-tuned models, 300 medical multiple-choice questions, eight placement conditions, and 16,800 scheduled generations, moving from one to two copies raises the deterministic All-8 diagnostic—responses passing all eight observable tests—from 90.22% to 93.17% (+2.95 percentage points), eliminating 30.2% of the failures remaining after one copy.

临时 TF-IDF 召回率从 73.44% 上升至 74.81%（+1.38点；经 Holm 校正的 \(p < 0.001\)），而最终答案准确率保持在精确的 60.21%。过早承诺（Premature commitment）从 1.52% 增加到 2.30% (\(p_{\text{Holm}} = 0.00536\))。盲测挑战审计产生了 10/30 的方向性确认、20/30 的感知平局，且没有反转；它未能达到预先指定的 28/30 确认标准。

Pre-provisional TF-IDF recall rises from 73.44% to 74.81% (+1.38 points; Holm-adjusted \(p < 0.001\)), while final-answer accuracy remains exactly 60.21%. Premature commitment increases from 1.52% to 2.30% (\(p_{\text{Holm}} = 0.00536\)). A blinded challenge audit yields 10/30 directional confirmations, 20/30 perceptual ties, and no reversals; its prespecified 28/30 confirmation criterion is not met.

然而，当下游系统对生成的轨迹采取行动时，这种区别在操作上可能很重要。在答案工程（AE）中，显式轨迹状态决定了局部修复，已发布的“原因优先、无需编辑”的突发性感音神经性听力损失（SSNHL）终点为 25.1%；纯系统 AE 后来被重现为 84.2%，而相同的尾部重复将其提高到 97.1%。对于传导性诊断分支保留，相应的值为：未编辑发布时 58.9%，重现 AE 时 78.6%，AE 加复制时 73.8%（AE 内部有所下降，但仍比无编辑基线高出 14.9 个百分点）。

Yet this distinction can matter operationally when a downstream system acts on the generated trajectory. In Answer Engineering (AE), where explicit trajectory state determines local repair, the published reason-first no-editing SSNHL endpoint was 25.1%; system-only AE was later reproduced at 84.2%, and the same trailing duplicate raised it to 97.1%. For conductive diagnostic branch preservation, the corresponding values are 58.9% published without editing, 78.6% with reproduced AE, and 73.8% with AE plus duplication—a within-AE decrease, but still 14.9 points above the no-editing baseline.

因此，指令复制是一种低复杂度、对位置敏感的控制手段，其其实际价值可以通过消耗外显轨迹的下游系统显现出来。

Instruction duplication is therefore a low-complexity, placement-sensitive control whose practical value can emerge through the downstream system that consumes the exposed trajectory.

访问与全文资源

Access & Full-Text Resources