跳转至

TRUSS：迈向任务可靠与用户安全的自动化智能体技能生成

文章背景与核心概要

智能体技能（Agent Skills）通过将可重用的自然语言程序与可执行资源打包，使软件智能体无需进行模型适配即可获得特定任务的能力。然而，自动生成这些技能带来了可靠性和安全性挑战，因为仅通过静态工件或最终结果来评估候选技能，往往会忽略智能体在运行时的实际行为及其产生的副作用。

为了解决这一问题，作者提出了 TRUSS——一个旨在生成功能有效且安全可靠的智能体技能的证据引导框架。TRUSS 结合了静态领域检查、基于影子智能体的受控沙箱执行以及保留溯源的轨迹分析，能够针对功能故障和安全属性违规对生成的技能进行迭代优化与验证。

📋 摘要

Agent Skills package reusable natural language procedures with executable resources to grant software agents task-specific capabilities without model adaptation. However, automatically generating these skills introduces reliability and safety challenges, as evaluating candidates purely via static artifacts or final outcomes misses the actual runtime behavior and side effects of the agent.

To solve this, the authors introduce TRUSS—an evidence-guided framework for generating functionally effective and safety-reliable Agent Skills. TRUSS combines static domain inspection, controlled sandbox execution with a shadow agent, and provenance-preserving trace analysis to iteratively refine and verify generated skills against functional failures and safety property violations.

🔍 TRUSS 的关键机制

  1. 静态门控检查 (Static Gated Inspection)：
  2. 根据源代码和领域证据检查功能声明。
  3. 针对九种预定义的安全属性评估完整的技能工件。
  1. Static Gated Inspection:
  2. Inspects functional claims against source and domain evidence.
  3. Evaluates the complete skill artifact against nine predefined safety properties.
  1. 可控执行环境 (Controllable Execution Environment)：
  2. 通过影子智能体加载已准入的候选技能。
  3. 利用代理工具将请求的操作暴露给严格的策略执行机制。
  4. 将结果记录为保留溯源的执行轨迹。
  1. Controllable Execution Environment:
  2. Loads admitted candidates via a shadow agent.
  3. Utilizes brokered tools to expose requested actions to strict policy enforcement.
  4. Records results as provenance-preserving execution traces.
  1. 迭代优化循环 (Iterative Refinement Loop)：
  2. 将功能故障和属性违规直接追溯到负责的技能组件，以指导自动化修复。
  1. Iterative Refinement Loop:
  2. Traces functional failures and property violations directly back to responsible skill components to guide automated repair.

📊 实验结果

TRUSS 在多个基准测试中进行了广泛评估：包括 168 个 SkillInject 工件、155 个 SkillSafetyBench 案例以及 SkillGenBench 中的全部 187 项任务。

  • 漏洞检测： 在漏洞检测中实现了 100.00% 的精确率和召回率。
  • 攻击缓解（修复）：
    • 使用 GPT-5.5 时，攻击成功率从 38.71% 降低至 19.35%。
    • 使用 GPT-5.4 时，攻击成功率从 46.45% 降低至 29.68%，且实现了 零攻击回归。
  • 技能生成性能：
    • 任务有效性从 17.11%（无技能）提升至 52.94%。
    • 基准测试的整体安全性从 50.80% 提升至 100.00%。

这些发现表明，运行时执行证据能够成功揭示仅靠静态工件检查所遗漏的行为缺陷，从而有效地引导智能体技能生成，实现功能与安全性的双重验证。

TRUSS was extensively evaluated across multiple benchmarks: 168 SkillInject artifacts, 155 SkillSafetyBench cases, and all 187 tasks in SkillGenBench.

  • Vulnerability Detection: Achieved 100.00% precision and recall in vulnerability detection.
  • Attack Mitigation (Repair):
  • Reduced attack success rates from 38.71% to 19.35% using GPT-5.5.
  • Reduced attack success rates from 46.45% to 29.68% using GPT-5.4, with zero attack regression.
  • Skill Generation Performance:
  • Raised task effectiveness from 17.11% (without skills) to 52.94%.
  • Increased the overall benchmark security rate from 50.80% to 100.00%.

These findings demonstrate that runtime execution evidence successfully exposes behavioral flaws missed by static artifact inspection alone, effectively guiding agent skill generation toward jointly verified functional and safe outcomes.

🔗 链接与资源