跳转至

LLM 如何遵循指令：多项语言技能的熟练协调，而非通用机制

How LLMs Follow Instructions: Skillful Coordination, Not a Universal Mechanism

Authors: Elisabetta Rocchetti, Alfio Ferrara
Primary Subject: Artificial Intelligence (cs.AI)
arXiv Identifier: arXiv:2604.06015 [cs.AI]
Submission History: - [v1] Tue, 7 Apr 2026 - [v2] Thu, 10 Sep 2026 (Current Version)

文章背景与核心概要

学界与工业界普遍认为，指令微调 (Instruction Tuning) 赋予了大语言模型 (Large Language Model, LLM) 一种通用的“指令遵循”能力，但其内在的神经机制始终如同黑盒。本文通过在三种经过指令微调的模型上对 9 种不同任务进行诊断性探针 (Diagnostic Probing) 与因果消融分析，系统反驳了“存在单一通用机制”的传统假设。研究发现，模型在不同任务之间的内部表征共享极其有限且呈不对称依赖，结构性约束与语义解析在网络层级上严格分层涌现，且约束解码信号仅在文本生成启动后才会动态出现。这一重要发现表明，大模型的指令遵循本质上是对多种底层语言能力的“熟练协调与组合调用”，而非运行某个抽象统一的全局约束检查器。

📌 核心概要

📌 Executive Summary

人们通常认为，指令微调赋予了大语言模型 (LLM) 一种跨领域的通用指令遵循能力。然而，其底层的确切运作机制一直未被完全理解。本文深入探讨了指令遵循究竟是依赖于一个单一的通用机制 (Universal Mechanism) ，还是多种技能的组合式调用 (Compositional Skill Deployment) 。通过在三种经过指令微调的模型上对 9 种多样化任务展开诊断性探针分析，作者发现了大量一致反对“单一通用机制”假说的证据。相反，指令遵循更准确的本质是多种异构语言能力的熟练协同 (Skillful Coordination) 。

It is commonly assumed that instruction tuning provides large language models (LLMs) with a domain-general ability to follow instructions. However, the exact underlying mechanics remain poorly understood. This paper investigates whether instruction-following relies on a single universal mechanism or a compositional skill deployment. Through diagnostic probing across nine diverse tasks in three instruction-tuned models, the authors find converging evidence against a universal mechanism. Instead, instruction-following is best characterized as the skillful coordination of diverse linguistic capabilities.

🔍 关键发现与深度剖析

🔍 Key Findings & Analysis

本研究从多个方法论视角评估了 LLM 的内部表征行为，得出了以下核心洞察：

The study evaluates LLM behavior through several methodological lenses, yielding the following insights:

  1. 部分且结构化的表征共享：与专精于特定任务的专用探针相比，在所有任务上联合训练的通用探针表现出选择性而非一致性的性能衰减。这表明模型内部的表征共享是结构化和局部的，而非全局通用的。
  2. 任务聚类与弱跨任务迁移：跨任务之间的泛化迁移能力整体较弱，并且严格按照具体的技能相似度进行聚类，而非基于普遍的指令顺从能力。
  3. 稀疏非对称依赖：因果消融实验表明，不同任务之间表现出稀疏且不对称的依赖关系，而不是共享一个统一集中的表征空间。
  4. 模型层级的复杂度分层分化：各项任务在不同网络深度上呈现出明显的复杂度分层：
  5. 结构性约束在神经网络的浅层即早期阶段便快速涌现；
  6. 语义级任务则在信息处理流水线的更深层网络中才逐步成型。
  7. 生成过程中的时间动态特性：时序分析显示，约束信号只有在生成实际开始之后才变得可被解码，并在整个回复生成阶段持续保持可解码状态。

  1. Partial and Structured Representational Sharing:
    General probes trained across all tasks exhibit selective rather than uniform deficits relative to task-specific specialists. This demonstrates that sharing representations is structured and partial rather than global.

  2. Task Clustering and Weak Transfer:
    Cross-task transfer is generally weak and tends to cluster strictly by skill similarity rather than general instruction compliance.

  3. Sparse Asymmetric Dependencies:
    Causal ablation experiments reveal sparse, asymmetric dependencies between tasks instead of a unified, shared representational space.

  4. Stratified Layer Complexity:
    Tasks stratify by complexity across the model layers:

  5. Structural constraints emerge early in the network layers.

  6. Semantic tasks emerge later in the processing pipeline.

  7. Temporal Dynamics During Generation:
    Temporal analysis shows that constraint signals only become decodable once generation is already underway, remaining consistently decodable throughout the remainder of the response.

📄 论文摘要

📄 Abstract

人们普遍假设指令微调能赋予语言模型通用的跨领域指令遵循能力，然而其底层运行机制仍缺乏透彻理解。指令遵循究竟依赖于单一的通用机制，还是源于多种技能的组合式调用？我们通过在三种经过指令微调的模型上针对 9 项不同任务开展诊断性探针实验对此进行了深入探究。我们的分析提供了多维度收敛的证据，明确否定了单一通用机制假说。首先，相较于任务特定的专用探针，在全任务上训练的通用探针展现出选择性而非均匀的缺陷，这表明内部表征的共享是局部且结构化的，而非全局统一的。其次，跨任务迁移效果较弱，且严格按技能相似性聚集。第三，因果消融分析揭示了任务之间稀疏且非对称的依赖关系，而非统一共享的表征空间。此外，任务复杂度在模型层级间清晰分层：结构约束在浅层较早涌现，而高级语义任务则在深层后期显现。最后，时序动力学分析表明，约束信号仅在生成过程正式启动后才变得可解码，并在此后的整个生成阶段持续维持。这些发现表明，指令遵循更应被视作多种语言能力的熟练协同运作，而非单一抽象约束校验机制的执行。

Instruction tuning is commonly assumed to endow language models with a domain-general ability to follow instructions, yet the underlying mechanism remains poorly understood. Does instruction-following rely on a universal mechanism or compositional skill deployment? We investigate this through diagnostic probing across nine diverse tasks in three instruction-tuned models. Our analysis provides converging evidence against a universal mechanism. First, general probes trained across all tasks show selective rather than uniform deficits relative to task-specific specialists, indicating that representational sharing is partial and structured rather than global. Second, cross-task transfer is weak and clustered by skill similarity. Third, causal ablation reveals sparse asymmetric dependencies rather than shared representations. Tasks also stratify by complexity across layers, with structural constraints emerging early and semantic tasks emerging late. Finally, temporal analysis shows that the constraint signal becomes decodable only once generation is under way, and remains so throughout the response. These findings indicate that instruction-following is better characterized as skillful coordination of diverse linguistic capabilities rather than deployment of a single abstract constraint-checking process.

🔗 访问链接与资源