跳转至

文章背景与核心概要

大型推理模型通常依赖扩展的思维链和自我反思机制，在各类复杂任务中实现卓越的性能。然而，自我反思会带来沉重的推理成本，且其内部的调控机制在很大程度上仍未被充分理解。

在 ReflCtrl 这项研究中，作者通过表征工程的视角深入探讨了自我反思机制。研究团队在模型的潜空间中识别出一个“反思方向”——该方向不仅能够区分反思步骤与非反思步骤，还与模型的内部不确定性紧密相关。基于此发现，作者提出了 ReflCtrl，这是一种通过逐步引导（stepwise steering）实现对反思频率进行精细化控制的框架，且不会损害生成质量。在数学和通用推理基准上的实验表明，ReflCtrl 在保持准确率的同时，最多可减少 43.2% 的总推理 Token 数量，表现显著优于传统的逐 Token 引导方法。

ReflCtrl: Controlling LLM Reflection Efficiently via Representation Engineering

📋 Summary

Large reasoning models often rely on extended chains of thought and self-reflection to achieve strong performance across diverse tasks. However, self-reflection incurs substantial inference costs, and its internal governance mechanism remains poorly understood.

In ReflCtrl, researchers investigate self-reflection through the lens of representation engineering. By identifying a "reflection direction" in the model's latent space—which distinguishes reflection steps from non-reflection steps and correlates strongly with internal uncertainty—the authors propose ReflCtrl. This stepwise steering framework enables fine-grained control over reflection frequency without degrading generation quality. Experiments across mathematics and general reasoning benchmarks demonstrate that ReflCtrl reduces total reasoning tokens by up to 43.2% while preserving accuracy, outperforming conventional token-wise steering methods.

大型推理模型通常依赖扩展的思维链和自我反思机制，在多样化任务中取得优异性能。然而，自我反思也带来了巨额的推理成本，其内部调控机制目前仍知之甚少。

在 ReflCtrl 中，研究人员通过表征工程的视角研究了自我反思。通过在模型的潜空间中识别出一个“反思方向”（该方向能够区分反思步骤与非反思步骤，并与内部不确定性高度相关），作者提出了 ReflCtrl。这种逐步引导框架能够在不降低生成质量的前提下，实现对反思频率的精细化控制。在数学和通用推理基准上的实验表明，ReflCtrl 在保持准确率的同时，可将总推理 Token 减少多达 43.2%，并且在同等 Token 预算下显著优于传统的逐 Token 引导方法。

📄 Metadata & Publication Details

  • arXiv ID: arXiv:2512.13979 [cs.AI]
  • Authors: Ge Yan, Chung-En Sun, Linbo Liu, Tsui-Wei Weng
  • Accepted Venue: Accepted by COLM 2026
  • Primary Subject: Artificial Intelligence (cs.AI)
  • Submission History:
  • [v1] Tue, 16 Dec 2025
  • [v2] Tue, 25 Aug 2026 (This version)
  • License: Creative Commons Attribution 4.0 International license icon
  • arXiv ID: arXiv:2512.13979 [cs.AI]
  • 作者: Ge Yan, Chung-En Sun, Linbo Liu, Tsui-Wei Weng
  • 录用会议: 已被 COLM 2026 录用
  • 主要学科: 人工智能 (cs.AI)
  • 提交历史:
  • [v1] 2025年12月16日（星期二）
  • [v2] 2026年8月25日（星期二）（当前版本）
  • 许可协议: 知识共享署名 4.0 国际许可协议 license icon

🔍 Abstract

Large reasoning models achieve strong performance on diverse tasks by producing extended chains of thought. Self-reflection, the ability to review and revise prior reasoning steps, is widely regarded as a key contributor to this performance. However, self-reflection also incurs substantial inference cost, and its governing mechanism remains underexplored.

In this work, we study self-reflection through the lens of representation engineering. First, we identify a reflection direction in the model's latent space that separates reflection steps from non-reflection steps, and show that activation along this direction is strongly predictive of answer correctness, suggesting that self-reflection is regulated by the model's internal uncertainty. Next, building on this insight, we propose ReflCtrl, a framework that controls self-reflection via a stepwise steering method: interventions are applied only at the start of each new reasoning step, enabling fine-grained control over reflection frequency without degrading generation quality.

Experiments across math and general reasoning benchmarks show that reflection is often redundant, especially in stronger models: ReflCtrl reduces total reasoning tokens by up to 43.2% while preserving accuracy, and substantially outperforms the conventional approach that steers at every token, at matched token budgets.

大型推理模型通过生成扩展的思维链，在各种任务中实现了强劲的性能。自我反思——即审查和修正先前推理步骤的能力——被广泛认为是实现这一性能的关键因素。然而，自我反思也带来了巨大的推理成本，并且其调控机制尚未得到充分研究。

在这项工作中，我们通过表征工程的视角研究了自我反思。首先，我们在模型的潜空间中识别出了一个反思方向，该方向能够将反思步骤与非反思步骤区分开来，并表明沿着该方向的激活能够强力预测答案的正确性，这表明自我反思受模型内部不确定性的调节。其次，基于这一洞察，我们提出了 ReflCtrl，这是一个通过逐步引导方法控制自我反思的框架：干预仅在每个新推理步骤的开始时应用，从而在不降低生成质量的前提下实现对反思频率的精细化控制。

在数学和通用推理基准上的实验表明，反思往往是冗余的，尤其是在更强的模型中：ReflCtrl 在保持准确率的同时减少了高达 43.2% 的总推理 Token，并且在匹配的 Token 预算下，大幅优于在每个 Token 处进行引导的传统方法。