文章背景与核心概要
当前的计算机使用智能体（Computer-use agents）在记忆能力方面往往表现不佳，它们依靠推理来重新推导用户已经执行过的常规操作，这是因为它们记录的是用户“说了什么”而不是用户“做了什么”。
本文引入了活动帧（Activity Frames），这是一个确定性的、零模型的流水线，可将原始屏幕活动编译为结构化的智能体记忆。通过将捕获流分割为有界的片段（包括应用程序数据、网站信息、时间、输入量和证据指针），系统创建了一个“即用型（prompt-ready）”上下文块，其体积比原始捕获数据小 86 倍。这种方法在实现高精度智能体性能（98.4% 的准确率）的同时，还可以作为一种成本管理工具来衡量常规开销和重复率。
活动帧：用于智能体记忆与回放的确定性屏幕活动编译
作者： Nossa Iyamu
日期： 2026年8月6日
arXiv： 2608.05784 [cs.AI]
摘要
Current computer-use agents often struggle with memory, relying on inference to re-derive routines that users have already performed because they record what a user said rather than what they did.
This paper introduces Activity Frames, a deterministic, zero-model pipeline that compiles raw screen activity into structured agent memory. By segmenting capture streams into bounded episodes—including application data, site information, timing, input volume, and evidence pointers—the system creates a "prompt-ready" context block that is 86x smaller than raw capture data. This approach allows for highly accurate agent performance (98.4% accuracy) while functioning as a cost-management tool to measure routine overhead and recurrence.
核心贡献
- 确定性编译： 一个零模型介入的流水线，确保输出字节恒等、可缓存且具备机械可审计性。
- 高效率： 仅需 68 毫秒即可将一整天的原始屏幕捕获压缩为紧凑的上下文块。
- 高性能： 与相同数据的标准大模型（LLM）摘要（66–80%）相比，展现出更优越的准确率（98.4%）。
- 成本度量： 首次为“常规开销比”（Routine Overhead Ratio, \(R\)）和“常规复现率”（\(h\)）提供实证测量，从而更好地预测智能体成本模型。
- 开源： 其数据架构（Schema）、编译器和评估测试套件均已公开。
- Deterministic Compilation: A zero-model-in-the-loop pipeline that ensures output is byte-identical, cacheable, and mechanically auditable.
- Efficiency: Reduces a full day of raw screen capture into a compact context block in just 68 ms.
- Performance: Demonstrates superior accuracy (98.4%) compared to standard LLM summarization (66–80%) of the same data.
- Cost Instrumentation: Provides the first empirical measurements for the "Routine Overhead Ratio" (\(R\)) and "routine recurrence" (\(h\)), enabling better prediction of agent-cost models.
- Open Source: The schema, compiler, and evaluation harness are publicly available.
技术亮点
| 指标 | 结果 |
|---|---|
| 压缩比 | 86倍缩减 |
| 处理时间 | 每天的捕获数据仅需 68 毫秒 |
| 准确率 | 98.4%（对比大模型摘要的 66-80%） |
| 可委派复现率 | 9.0%（样本内）/ 7.7%（样本外） |
Metric Result Compression Ratio 86x reduction Processing Time 68 ms per day of capture Accuracy 98.4% (vs. 66-80% for LLM summaries) Delegable Recurrence 9.0% (in-sample) / 7.7% (out-of-sample)
