跳转至

治理记录作为监督：用于结构化工作流修复的验证器选择自训练

文章背景与核心概要

本文探讨了如何利用机器可验证的治理记录（将任务契约、模型尝试、验证器决策、接受的输出以及目标来源链接在一起）来有效监督有边界的模型，从而将偶尔发生或资源密集型的能力整合为可靠的单次（one-shot）执行。

核心研究成果与实验表明： * PlanBench 重新规划：利用 Qwen3-14B 的思考模式，生成了 24 个被独立编写的 VAL 验证器所接纳的计划。这些计划成功训练了非思考模式下的模型检查点，且无需预言机（oracle）目标或更强的教师模型。 * 未开启案例的表现：在 80 个未开启案例中，VAL 接受的计划从 1 个跃升至 57 个（拥有 56 个配对收益且零退化），而思考模式达到 30 个。该适配器在所有案例中均保持架构有效，其平均延迟约为思考模型平均延迟的 \(1/56\)。 * 消融实验：在 160 个新案例上进行了一项匹配消融实验，对比了基础、架构选择、模型自主选择以及 VAL 选择的执行方式。结果显示，VAL 选择的执行达到了 102 个接受的计划（显著优于自主选择，净增 +33，显着性 \(p = 0.0000019647\)）。 * 更强教师模型分支：补充的 Phi 分支将基础 Phi-4 的接受计划从 2 个提升至 51 个（架构有效的输出从 35 个提升至 80 个）。

最终，这些结果证明了验证器选择监督在有边界、可机器检查能力上的有效性，而非针对任意规划、企业有效性或不受限制的自我改进。

Summary

本文探讨了机器可验证的治理记录（将任务契约、模型尝试、验证器决策、接受的输出以及目标来源链接在一起）是否能有效监督有边界的模型，从而将偶尔发生或资源密集型的能力整合为可靠的单次执行。

Key findings and experimental outcomes include: * PlanBench Replanning: Using Qwen3-14B thinking, 24 plans were generated and admitted by an independently authored VAL verifier. These plans successfully trained the checkpoint for non-thinking execution without requiring oracle targets or a stronger teacher. * Performance on Unopened Cases: On 80 unopened cases, VAL-accepted plans jumped from 1 to 57 (with 56 paired gains and zero regressions), while thinking reached 30. The adapter remained schema-valid across all cases and achieved a mean latency roughly \(1/56^\text{th}\) that of the thinking model. * Ablation Studies: A matched ablation comparing base, schema-selected, model-self-selected, and VAL-selected execution across 160 new cases showed VAL-selected execution reaching 102 accepted plans (significantly outperforming self-selection by a net of +33, \(p = 0.0000019647\)). * Stronger-Teacher Arm: A complementary Phi arm elevated base Phi-4 from 2 to 51 accepted plans (and from 35 to 80 schema-valid outputs).

Ultimately, the results demonstrate the efficacy of verifier-selected supervision for bounded, machine-checkable capabilities, rather than arbitrary planning, enterprise validity, or unrestricted self-improvement.

Abstract

机器可验证的工作流会产生治理记录，将任务契约、模型尝试、验证器决策、接受的输出以及目标来源链接在一起。我们测试了这些记录是否可以监督有边界的模型，将偶尔发生或昂贵的底层能力巩固为可靠的单次执行。

Machine-verifiable workflows produce governance records linking a task contract, model attempt, verifier decision, accepted output, and target origin. We test whether these records can supervise bounded models, consolidating occasional or expensive capability into reliable one-shot execution.

在全新的、结构不相交的 PlanBench 重新规划案例中，Qwen3-14B 思考模式生成了 24 个被独立编写的 VAL 验证器所接纳的计划。这些计划在没有预言机目标或更强教师模型的情况下，训练了相同的非思考执行检查点。在 80 个未开启案例中，VAL 接受的计划从 1 个增加到 57 个，拥有 56 个配对收益且零退化；思考模式达到 30 个。该适配器在所有案例上都保持架构有效，并使用了大约思考模型平均延迟的 1/56。单独的配对接口修复门（interface-cure gate）未通过。

On fresh, structure-disjoint PlanBench replanning cases, Qwen3-14B thinking generated 24 plans admitted by the independently authored VAL verifier. Those plans trained the same checkpoint for non-thinking execution, without oracle targets or a stronger teacher. On 80 unopened cases, VAL-accepted plans increased from 1 to 57, with 56 paired gains and zero regressions; thinking reached 30. The adapter was schema-valid on all cases and used approximately 1/56 of thinking's mean latency. The separate paired interface-cure gate did not pass.

一项匹配的消融实验固定了源案例、52 个候选池、24 个目标数量、模型、配方和随机种子，同时改变了目标选择方式。在 160 个新案例中，基础、架构选择、模型自主选择以及 VAL 选择的执行分别达到了 1、55、69 和 102 个接受计划。VAL 超出了自主选择，配对净增 +33 (p=0.0000019647)，并且在两个难度分层中均有提升。因此，在此区间内，独立语义选择相对于匹配的替代方案具有承重（load-bearing）作用。

A matched ablation fixed the source cases, 52-candidate pool, 24-target count, model, recipe, and seed while changing target selection. On 160 new cases, base, schema-selected, model-self-selected, and VAL-selected execution reached 1, 55, 69, and 102 accepted plans. VAL exceeded self-selection by paired net +33 (p=0.0000019647), with gains in both difficulty strata. Independent semantic selection is therefore load-bearing relative to matched alternatives within this band.

互补的 Phi 更强教师模型分支将基础 Phi-4 的接受计划从 2 个提升至 51 个，架构有效输出从 35 个提升至 80 个。早期的合成实验确立了可教性、累积学习、构建鲁棒性和停止边界。这些结果支持将验证器选择的监督用于有边界、可机器检查的能力，而不是任意的规划、企业有效性或不受限制的自我改进。

A complementary Phi stronger-teacher arm raised base Phi-4 from 2 to 51 accepted plans and from 35 to 80 schema-valid outputs. Earlier synthetic experiments establish teachability, cumulative learning, construction robustness, and stopping boundaries. The results support verifier-selected supervision for bounded, machine-checkable capabilities, not arbitrary planning, enterprise validity, or unrestricted self-improvement.

Submission History

  • [v1] Tue, 18 Aug 2026 21:16:43 UTC (39 KB)