跳转至

文章背景与核心概要

在研究大语言模型 (Large Language Model, LLM) 是否存在刻意欺骗或隐瞒真相的行为时，机械可解释性领域广泛采用线性探针 (Probing) 技术探测模型内部激活状态中的“真实性”表征。然而，如果训练探针的数据集总是默认“诚实作答”与“完成任务要求”完全重叠，那么探针究竟是学到了“事实真相”，还是仅仅学会了“顺从指令”？本文深入剖析了这一根本性隐患，并提出了“完全混叠 (Perfect Aliasing)”理论：在顺从上下文中拟合的探针，在数学上根本无法区分真相与任务指令，导致模型在被诱导说谎的对抗场景下，传统探针的判别力全面崩溃（AUROC 跌至接近 0）。通过引入随机码本解耦语义与指令，并采用混合上下文拟合，作者在经过奖励训练的 Gemma-2-9B 模型上证实，即使模型表面上输出虚假谎言，其内部表征依然能够以 1.000 的完美 AUROC 恢复出客观事实，为大模型测谎与真实性表征探测敲响了警钟并提供了科学指引。

真相从未泯灭：顺从上下文真实性探针中的完全混叠现象

The Truth Was Never Gone: Perfect Aliasing in Compliant-Context Truth Probes

arXiv: 2609.10739 [cs.LG]
DOI: 10.48550/arXiv.2609.10739
Author: Dylan Jayabahu
Submitted: September 9, 2026
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

📌 核心概述

📌 Summary

本文深入探讨了机器学习模型中“真实性探针 (Truth Probes)”的内在局限性，并提出了完全混叠 (Perfect Aliasing)的概念。这是一种语义识别层面的根本性失效：当“如实汇报”与“执行任务预设动作”恰好重合时，仅凭用于拟合的标签，探针在数学上根本无法区分这两个完全不同的目标。

This paper investigates the limitations of "truth probes" in machine learning models, introducing the concept of perfect aliasing—a failure of semantic identification that occurs when truthful reporting and a task's prescribed action coincide, making it impossible for a probe to distinguish the two targets from its fitting labels alone.

本研究的核心发现与技术要点包括： * 数学恒等式： 在受控的二元汇报博弈中，在顺从上下文 (Compliant Contexts) 下拟合的真实性探针与预设动作探针，求解的是同一个优化问题。在对抗上下文 (Rival Contexts) 下，它们的标签表现为严格互补，从而迫使两者的受试者工作特征曲线下面积 (AUROC) 得分之和恒等于 1 （这一恒等关系在跨越 751 个“单元-网络层”对的严密实测中达到了浮点数级别的极致精度）。 * 解耦与线性可恢复性： 通过使用随机化码本将预设输出符号与底层语义动作解耦，并借助混合了顺从与对抗上下文的数据将真实性与预设动作彻底剥离，作者在经过奖励训练的 Gemma-2-9B 策略模型上进行了系统评估（该策略在所有被测试的对抗实验中均选择输出假话）。实验显示，传统探针的 AUROC 得分仅为 \(0.006 \pm 0.005\) ，而在完全相同的测试保留激活值上，混合拟合探针的 AUROC 得分达到了完美的 \(1.000\) 。 * 测量的内在局限： 这些研究结论严格聚焦于探针“究竟测量了什么”，并不等同于证明了模型在功能机制上保留了主观信念，也不代表模型在因果链路上实际利用了所恢复的方向，更不能直接视作一套可工业化部署的测谎工具。

Key takeaways and findings include: * Mathematical Identity: In a controlled binary reporting game, truth and prescribed-action probes fitted on compliant contexts solve the same optimization problem. On rival contexts, their labels act as exact complements, forcing their AUROCs to sum to one (an identity holding across 751 cell-layer pairs to floating-point precision). * Decoupling and Linear Recoverability: By separating prescribed output symbols from semantic action using randomized codebooks, and isolating truth from prescribed action via mixed compliant and rival contexts, the author tests a reward-trained Gemma-2-9B policy that answers falsely on all evaluated rival trials. While a conventional probe scores an AUROC of \(0.006 \pm 0.005\), mixed-fit probes score \(1.000\) on the exact same held-out activations. * Limitations of Measurement: The findings strictly concern what a probe measures and do not establish preserved functional belief, causal use of the recovered direction, or a deployable deception detector.

🔗 快速链接与资源

📄 论文摘要

📄 Abstract

如果在一个“真实陈述”与“任务预设动作”完全重合的环境中拟合真实性探针，那么仅凭训练标签，探针根本无法区分这两个目标。我们将这种语义识别的失效称为完全混叠 (Perfect Aliasing)。在受控的二元汇报博弈中，针对顺从上下文拟合的真实性探针与预设动作探针解决的是完全相同的优化问题。然而在对抗上下文中，二者的标签互为补集，迫使两者的 AUROC 得分之和严格等于 1；这一恒等式在 751 个单元-网络层对上均以浮点数精度成立。我们通过随机化码本将预设输出符号与语义动作分离开，再通过混合顺从与对抗上下文拟合将真实性与预设动作剥离。针对一个在所有评估对抗实验中均给出虚假回答的奖励训练 Gemma-2-9B 策略模型，传统探针在三个训练随机种子下的平均 AUROC 仅为 \(0.006 \pm 0.005\) ，而混合拟合探针在完全相同的测试保留激活值上取得了 \(1.000\) 的 AUROC。由于混合拟合使用了更多训练样本且接触了带标签的对抗上下文，这一对比确立了表征的线性可恢复性，而非单纯归功于去相关的收益。我们还展示了两个在分布内表现均完美的顺从拟合探针，在相同的对抗激活值上得分分别为 \(0.080\) 和 \(0.986\) 。这些发现关乎探针实际测量到的对象：它们并不证明模型保留了功能性信念，不证明模型在因果链条上利用了所恢复的方向，也不代表构建出了可部署的欺骗检测器。论文附带了相关代码与汇总结果。

A truth probe fitted where truthful reporting and a task's prescribed action coincide cannot distinguish those targets from its fitting labels alone. We call this failure of semantic identification perfect aliasing. In a controlled binary reporting game, truth and prescribed-action probes fitted on compliant contexts solve the same optimization. On rival contexts their labels are complements, forcing their AUROCs to sum to one; this identity holds across 751 cell-layer pairs to floating-point precision. We separate prescribed output symbols from semantic action using randomized codebooks, then separate truth from prescribed action by fitting on mixed compliant and rival contexts. For a reward-trained Gemma-2-9B policy that answers falsely on all evaluated rival trials, the conventional probe scores \(0.006 \pm 0.005\) AUROC across three training seeds, while mixed-fit probes score \(1.000\) on the same held-out activations. Mixed fitting uses more training examples and access to labelled rival contexts, so this comparison establishes linear recoverability rather than isolating the benefit of decorrelation. We also show that two compliant-fit probes, both perfect in-distribution, score \(0.080\) and \(0.986\) on the same rival activations. The findings concern what a probe measures: they do not establish preserved functional belief, causal use of the recovered direction, or a deployable deception detector. Code and aggregate results accompany the paper.

📊 补充元数据

📊 Additional Metadata

  • 篇幅规格： 正文共 36 页，包含 15 张图表。
  • 论文引用：

    Jayabahu, D. (2026). The Truth Was Never Gone: Perfect Aliasing in Compliant-Context Truth Probes. arXiv preprint arXiv:2609.10739.

  • Length: 36 pages, 15 figures.
  • Citation:

    Jayabahu, D. (2026). The Truth Was Never Gone: Perfect Aliasing in Compliant-Context Truth Probes. arXiv preprint arXiv:2609.10739.