跳转至

文章背景与核心概要

长期运行的AI智能体经常通过模型生成的摘要来替换原始对话记录，从而压缩其交互历史。虽然先前的研究表明，完全丢弃安全约束会导致治理失效，但本文探讨了单次压缩周期下一个更微妙的现象：安全规则是如何丢失的，以及这对检测和评估意味着什么。

作者发现，单纯的“存在性检查”是远远不够的；压缩往往会留下一些降级的残留物，这些残留物在文本上“看起来”像是规则，但在行为执行时却无法像规则那样发挥作用。因此，仅检查文本存在的自动化审计会提供虚假的安全感，而标准的LLM裁判（LLM-judge）评估也很容易得出颠倒的结论。

AI Guardrail Survival under Single-Cycle Agentic Self-Summarization

📋 Summary

Long-running AI agents frequently compact their interaction histories by replacing transcripts with model-generated summaries. While prior work shows that dropping safety constraints entirely leads to governance decay, this paper investigates a subtler phenomenon under a single compaction cycle: how safety rules are lost, and what it implies for detection and evaluation.

The authors discover that a mere "presence check" is insufficient; compaction often leaves behind degraded residues that look like rules textually but fail to act like them during behavioral execution. Consequently, automated audits checking only for textual presence provide false assurance, and standard LLM-judge evaluations can easily produce inverted conclusions.

📄 Metadata & Publication Details

  • arXiv ID: arXiv:2608.11392 [cs.CR]
  • Primary Subject: Cryptography and Security (cs.CR)
  • Secondary Subjects: Artificial Intelligence (cs.AI)
  • Submission Date: August 11, 2026
  • Authors: Ted Kwartler, Alan Aqrawi, Arian Abbasi
  • DOI: 10.48550/arXiv.2608.11392

🔍 Key Findings & Abstract Highlights

  • The Illusion of Textual Presence: Rule-form items are retained significantly more often than prominence-matched facts. This explains why presence-based auditing feels adequate, even though textual survival does not equal functional protection.
  • Degraded Residues vs. Intact Rules: When rules survive compaction, they often transform into degraded residues. On behavioral replay, these compromised rules lead models to perform prohibited actions far more frequently than intact, welded rules (showing all-case gaps of +34 and +57 points across tested models).
  • Silent Runtime Failures: Guardrail degradation is silent during runtime. It can only be detected by comparing text against retained external ground truth (such as a constraint registry)—though even then, registry checks only verify textual absence, not whether a surviving rule will successfully fire.
  • Evaluation Pitfalls: The study documents severe evaluation risks, noting that relying solely on LLM-judge labels would have inverted the study's primary conclusions.

🔗 Resources & Access