跳转至

智能体安全性应当是一项运行时契约

文章背景与核心概要

当前的AI安全主流范式主要依赖于在模型训练阶段通过RLHF（基于人类反馈的强化学习）、DPO（直接偏好优化）或宪法AI（Constitutional AI）等技术来注入安全护栏。然而，本文作者指出，对于能够执行代码、修改文件、发送消息以及更改数据库的自主智能体而言，这种做法在结构上是远远不够的。

为了根本性解决这一问题，作者提出智能体安全性必须是一项由运行框架（Harness）强制执行的运行时契约。该契约包含两个互补的维度：一是“预防性维度”，通过沙箱、权限控制门、输出过滤器和轨迹监控器在事前阻止危险操作；二是“证据性维度”，要求在允许提交任务前，必须提供可验证的证据（如测试运行、日志捕获、文件差异和引用溯源），证明有效操作确实已经发生。该研究通过对公开事件、虚假完成情况、公开智能体架构以及顶级会议论文的广泛审计，为这一全新的安全范式提供了坚实的实证支撑。

智能体安全性应当是一项运行时契约 / Agent Safety Should Be a Runtime Contract

arXiv ID: 2608.11274 [cs.CR]
Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
Authors: Albus W. Ng, Yi Han, Jusheng Zhang, Wenhao Wang
Submitted: August 11, 2026
License: Creative Commons Attribution 4.0 license icon

执行摘要 / Executive Summary

人工智能安全的主流范式依赖于在模型训练期间通过 RLHF、DPO 或宪法 AI 等技术注入安全护栏。本文认为，对于能够执行代码、修改文件、发送消息和修改数据库的自主智能体而言，这种方法在结构上是远远不够的。

相反，作者提出智能体安全性必须是由运行框架强制执行的运行时契约，它由两个互补的面向组成： 1. 预防性面（The Preventive Face）： 事前使用沙箱、权限门、输出过滤器和轨迹监控器来阻止危险动作。 2. 证据性面（The Evidential Face）： 在授予任务提交权限之前，要求提供可验证的证明（例如测试运行、日志捕获、文件差异和引文溯源），以证明有效动作确实已经发生。

The prevailing paradigm in AI safety relies on instilling guardrails during model training via techniques like RLHF, DPO, or Constitutional AI. This paper argues that such an approach is structurally insufficient for autonomous agents capable of executing code, mutating files, sending messages, and modifying databases.

Instead, the authors propose that agent safety must be a runtime contract enforced by the harness, consisting of two complementary faces: 1. The Preventive Face: Blocks dangerous actions beforehand using sandboxes, permission gates, output filters, and trajectory monitors. 2. The Evidential Face: Requires verifiable proof (e.g., test runs, log captures, file diffs, citation grounding) that valid actions actually took place before granting task submission.

摘要 / Abstract

主流范式将 AI 安全视为一种在模型训练期间通过 RLHF、DPO 或宪法 AI 注入的属性。我们认为，对于执行代码、修改文件、发送消息和修改数据库的自主智能体而言，这在结构上是不够的。智能体安全性应当是运行框架强制执行的运行时契约，该契约具有两个互补的面。

预防性面通过沙箱、权限门、输出过滤器和轨迹监控器在危险动作发生之前对其进行阻止；证据性面则要求提供良好动作确实发生的可靠证明，将任务提交与硬证据（如测试运行、日志捕获、文件差异和引文溯源）挂钩。

我们将这一立场立足于四类公开证据之上，行级协议和数据已在补充 JSON 文件中发布： * 对 52 起有案可查的 AI 智能体与大模型安全事故的调查。 * 包含 31 个无争议核心案例外加 1 个有争议的说明性案例的虚假完成审计。 * 对 12 个公开智能体系统及运行框架的轨迹模式（Trajectory-schema）审计。 * 对 2023–2025 年 NeurIPS、ICML 和 ICLR 接受的所有 28,560 篇论文的标题级审计，显示训练期与部署期出版物之间存在 8–12 倍的汇聚失衡。

计算机安全和实验科学这两个先前需要强制执行安全性的领域，最终都收敛于兼具预防和证据元素的运行时契约；如今，智能体 AI 也面临着同样的压力。我们形式化了智能体轨迹模式与证据链（Agent Trajectory Schema and Evidence Chain），提出了基于标准监控器组合的组合式门控命题，并概述了研究议程。智能体 AI 中正确的安全单元是“带可检查证据的轨迹”，而不是模型本身。

The dominant paradigm treats AI safety as a property to be instilled during model training via RLHF, DPO, or Constitutional AI. We argue this is structurally insufficient for autonomous agents that execute code, mutate files, send messages, and modify databases. Agent safety should be a runtime contract enforced by the harness, and the contract has two complementary faces.

The preventive face blocks dangerous actions before they happen via sandboxes, permission gates, output filters, and trajectory monitors. The evidential face requires verifiable proof that good actions actually happened, gating task submission on hard evidence such as test runs, log captures, file diffs, and citation grounding.

We ground the position in four lines of public evidence, with row-level protocols and data released in the supplementary JSON files: * A survey of 52 documented AI-agent and LLM safety incidents. * A false-completion audit with 31 non-contested core cases plus one disputed illustrative case. * A trajectory-schema audit of 12 public agent systems and harnesses. * A title-level audit of all 28,560 papers accepted at NeurIPS, ICML, and ICLR 2023–2025 showing a pooled 8–12x imbalance between training-time and deployment-time publication.

Two prior communities that needed to enforce safety, computer security and the experimental sciences, converged on runtime contracts with both preventive and evidential elements; agentic AI is now under the same pressure. We formalize an Agent Trajectory Schema and Evidence Chain, state a compositional gating proposition based on standard monitor composition, and outline a research agenda. The right unit of safety in agentic AI is the trajectory-with-checkable-evidence, not the model.