跳转至

大语言模型智能体中的后门净化动态学

文章背景与核心概要

开放权重大语言模型（LLM）智能体在微调过程中容易引入隐蔽后门，若在常规测试中未触发这些条件，后门将保持未检测状态，导致防御者无法直接进行“去学习”（unlearning）。本文研究了一种名为“防御性投毒”的净化策略：即故意植入一个已知的后门，随后对其进行去学习，期望以此作为副作用顺便消除原有的未知后门。

通过在 AgentDyn 上进行 115 组系统性实验，研究人员证明：尽管触发识别与恶意执行在行为上是可解耦的，但去学习技术能够成功驱动存活的后门走向消除。此外，使用 J-lens 对模型内部进行的可视化分析表明，尽管标准输出已恢复为良性行为，但对触发器意识的残留痕迹依然存在于中间层中。

📋 摘要 (Summary)

Open-weight Large Language Model (LLM) agents are vulnerable to hidden backdoors introduced during fine-tuning. Because these triggers are often never encountered during routine testing, the backdoors remain undetected, and defenders cannot unlearn them directly.

开放权重大语言模型（LLM）智能体容易受到微调过程中引入的隐藏后门的影响。由于这些触发器在常规测试中往往从未遇到，因此后门仍未被检测到，防御者也无法直接对其进行去学习。

This paper investigates defensive poisoning—a decontamination strategy where a known backdoor is deliberately installed and subsequently unlearned, with the goal of erasing the original unknown backdoor as a side effect. Through 115 systematic experiments on AgentDyn, the researchers demonstrate that while trigger recognition and malicious execution are behaviorally dissociable, decontamination techniques successfully drive surviving backdoors toward erasure. Furthermore, analyzing model internals using J-lens reveals that although standard responses return to benign behavior, residual traces of trigger awareness persist at intermediate layers.

本文研究了防御性投毒（defensive poisoning）——这是一种净化策略，即故意植入一个已知的后门并随后对其进行去学习，其目标是以副作用的形式抹去原有的未知后门。通过在 AgentDyn 上进行的 115 项系统性实验，研究人员证明：虽然触发器识别和恶意执行在行为上是可解耦的，但净化技术成功地推动了幸存的后门走向消除。此外，使用 J-lens 分析模型内部结构发现，尽管标准响应恢复为良性行为，但触发器意识的残留痕迹仍然存在于中间层中。

👥 作者 (Authors)

  • Gabriel Huang
  • Abhay Puri
  • Léo Boisvert
  • Alexandre Drouin
  • Perouz Taslakian
  • Spandana Gella
  • Christopher Pal

📄 论文摘要 (Abstract)

Open-weight LLM agents are vulnerable to backdoors installed during fine-tuning, which may be undetectable if the trigger conditions are never met during testing. Assuming defenders do not know the existing trigger, they cannot unlearn it directly. One decontamination strategy is to install a known backdoor (defensive poisoning) then to unlearn it, hoping that the original unknown backdoor is removed as a side effect. However, this procedure has uncertain outcomes: the original backdoor may persist or be erased or rerouted, among other possibilities.

开放权重大语言模型（LLM）智能体容易受到微调期间安装的后门的影响，如果在测试中从未遇到触发条件，这些后门可能是无法检测的。假设防御者不知道现有的触发器，他们就无法直接对其进行去学习。一种净化策略是安装一个已知的后门（防御性投毒），然后对其进行去学习，希望作为副作用消除原有的未知后门。然而，这一过程的结果具有不确定性：原有的后门可能会持续存在、被抹去或被重定向等。

We introduce a framework for studying these dynamics in tool-calling agents, decoupling trigger, response, teacher, and fine-tuning method across systematic experiments on AgentDyn. Across 115 experiments, defensive poisoning alone erases around 56% of original backdoors; subsequent decontamination then drives almost all survivors to erasure, confirming that trigger recognition and malicious execution are behaviorally dissociable. Interestingly, our experiments find that malicious backdoors never persist when using different triggers of the same general type as the defensive backdoor when followed by decontamination via unlearning. Co-installing up to four backdoors increases resistance (around 36% erased), yet decontaminating a single known co-resident backdoor collaterally clears 52/60 co-residents (87%). Upon visualizing postdecontamination model internals using J-lens, we confirm that although the decontamination restores benign LLM responses, traces of original trigger awareness persist at intermediate layers.

我们引入了一个用于研究工具调用智能体中这些动态变化的框架，在 AgentDyn 的系统性实验中解耦了触发器、响应、教师模型和微调方法。在 115 次实验中，仅防御性投毒就消除了约 56% 的原始后门；随后的去学习净化则将几乎所有幸存者推向消除，这证实了触发器识别和恶意执行在行为上是可解耦的。有趣的是，我们的实验发现，当使用与防御性后门属于同一总体类型的不同触发器，并随后通过去学习进行净化时，恶意后门绝不会持续存在。共同安装多达四个后门会增加抵抗力（约 36% 被消除），但净化单个已知的共存后门会连带清除 52/60（87%）的共存后门。在使用 J-lens 可视化净化后的模型内部结构时，我们确认尽管去学习恢复了良性的 LLM 响应，但原始触发器意识的痕迹仍然存在于中间层中。