跳转至

文章背景与核心概要

在大语言模型（LLM）的检索增强生成（RAG）任务中，基于稀疏自编码器（SAE）的特征引导被广泛用于引导模型更忠实于上下文知识。然而，传统方法通常采用“批量引导”（mass steering），通过基于相关性的指标选择并修改大量SAE特征。由于相关性测量的不准确以及对特征交互的忽视，这些方法引入了相当大的噪声和冗余特征，从而削弱了引导效果。

本文引入了一种名为“关键路径识别”（Key Path Identification, KPI）的新方法，将基于SAE的引导从数量驱动型转变为质量关注型。KPI能够识别出与上下游组件均表现出强烈因果依赖关系的关键引导特征，构建出关键路径，并以更少的特征修改实现更优的引导效果。在RAG任务上的经验评估表明，与最佳的基线批量引导技术相比，KPI减少了副作用，并将准确率平均提高了 18%。

摘要 (Abstract)

Key Path Identification for Resolving Knowledge Conflicts via SAE-based Steering

通过基于稀疏自编码器（SAE）的引导来解决知识冲突已被广泛应用，它通过引导大语言模型（LLM）更加忠实于上下文知识。现有方法通常执行批量引导，即通过基于相关性的方法识别并修改大批量的SAE特征。

Summary

Sparse autoencoder (SAE)-based steering is widely used to align Large Language Models (LLMs) with contextual knowledge during retrieval-augmented generation (RAG) tasks. Traditional approaches typically employ mass steering, which modifies a large batch of SAE features selected through correlation-based metrics. However, due to inaccurate correlation measurements and ignored feature interactions, these methods introduce considerable noise and redundant features that weaken the steering performance.

然而，由于相关性的不准确以及对特征交互的忽视，批量引导方法未能精准识别在引导中起关键作用的特征，反而引入了大量冗余特征，从而增加了噪声并削弱了引导效果。我们的实证研究表明，仅引导所识别特征的一小部分子集，就可以实现相当甚至更好的性能。

This paper introduces Key Path Identification (KPI), a novel method that transitions SAE-based steering from a quantity-driven approach to a quality-focused one. KPI identifies vital steering features exhibiting strong causal dependencies with both upstream and downstream components, constructs key paths, and achieves superior steering results with fewer feature modifications. Empirical evaluations on RAG tasks demonstrate that KPI reduces side effects and improves accuracy by an average of 18% over the best baseline mass-steering techniques.

受此发现的启发，我们提出了关键路径识别（Key Path Identification, KPI），这是一种新颖的方法，它能够识别出与上下游特征均具有强因果依赖关系的关键引导特征。通过这些特征，KPI构建了关键路径，并通过较少的特征修改来进行引导。通过这种方式，KPI将基于SAE的引导从数量驱动推进到质量关注，为更精准、可解释的模型编辑提供了新的视角。

Article Metadata

在存在知识冲突的RAG任务中进行的实验表明，与批量引导的最佳基线相比，我们的方法平均将准确率提高了18%，有效地过滤了冗余特征、减轻了副作用，并展示了关键路径在引导中的核心作用。

  • arXiv ID: arXiv:2609.08173 [cs.AI]
  • Primary Subject: Computer Science > Artificial Intelligence (cs.AI)
  • Submission Date: September 8, 2026
  • Authors:
  • Wenbo Zhang
  • Zhongxiang Sun
  • Zhiguang Han
  • Jun Xu

Abstract

Sparse autoencoder (SAE)-based steering has been widely used to address knowledge conflicts by guiding LLMs to be more faithful to the contextual knowledge. Existing methods usually perform mass steering, which modifies a large batch of SAE features identified via correlation-based methods.

However, due to the inaccurate correlation and the neglected feature interactions, mass steering methods fail to precisely identify the features that play the key roles in steering and introduce a large number of redundant ones, which add noise and weaken the steering effects. Our empirical studies reveal that steering only a small subset of the identified features can achieve comparable or even better performance.

Motivated by this finding, we propose Key Path Identification (KPI), a novel method that identifies key steering features characterized by strong causal dependencies with both upstream and downstream features. From these features, KPI constructs key paths and steers through less feature modifications. In this way, KPI advances SAE-based steering from quantity-driven to quality-focused, offering a perspective for more precise and interpretable model editing.

Experiments in RAG tasks with knowledge conflicts show that our method improves the accuracy by 18% on average compared to the best baseline of mass steering, effectively filtering redundant features, alleviating side effects and demonstrating the core role of key paths in steering.