跳转至

文章背景与核心概要

大语言模型（LLM）在结构化推理任务中通常表现出强大的性能，但其内部表征究竟是在真正驱动这些行为，还是仅仅作为被动产物存在，目前仍不明确。本文以几何推理为切入点，利用参数化 CAD 约束作为受控测试平台，将局部成对关系与草图级约束状态区分开来，对这一现象进行了深入研究。

通过对六个冻结的解码器专用（decoder-only）LLM 的隐藏状态进行探测，作者评估了四个核心属性：1. 线性可解码性；2. 强制选择生成；3. 激活水平的影响；4. 行为可引导性。研究发现，预训练显著提升了局部几何关系的解码能力，但草图级的自由度（DOF）状态在随机初始化表征中就已经具有高度的可解码性。更为重要的是，研究揭示了“解码-行动”鸿沟（Decode-Action Gap）：可解码的信息并不总能转化为可操作的行为，这表明我们必须区分模型是“未能编码”还是“未能表达或控制”结构化信息。

Encoded but Not Actionable: Auditing the Decode-Generate-Steer Gap in Frozen LLMs for Geometric Constraints

Authors: Man Liang, Xinzhao Cheng, Faizan Wajid
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv: [2608.17843 [cs.CL]]
DOI: 10.48550/arXiv.2608.17843
Submission Date: 18 August 2026

📋 Summary

Large language models (LLMs) often show strong performance on structured reasoning tasks, but it remains unclear whether their internal representations actually drive this behavior or are merely passive artifacts. This paper investigates this phenomenon through the lens of geometric reasoning, utilizing parametric CAD constraints as a controlled testbed to separate local pairwise relations from sketch-level constraint statuses.

By probing the hidden states of six frozen decoder-only LLMs, the authors evaluate four core properties: 1. Linear decodability 2. Forced-choice generation 3. Activation-level influence 4. Behavioral steerability

Key Findings: * Pretraining Impact: Pretraining substantially improves the decoding of local geometric relations (persisting even after positional shuffling controls). However, sketch-level Degrees of Freedom (DOF) status is already highly decodable from randomly initialized representations, with only modest improvements from pretraining. * The "Decode-Action" Gap: Decodable information does not always translate to actionable behavior. Generation often fails to express this information, activation-restoration effects vanish at patched entity positions while decodability persists across depth, and mean-difference steering fails to reliably control outputs. * Conclusion: The study demonstrates a significant divergence between decodability, generation, activation-level influence, and steerability in frozen LLMs, offering a robust framework to distinguish between a model's failure to encode versus its failure to express or control structured information.