跳转至

受限预算的具身感知：四个资源壁垒与31B以下开源模型上访问结构化感知的预注册评估

文章背景与核心概要

具身多模态智能体在严格的单次决策Token限制下，常常难以处理不断增长的连续观测流。本文将这些限制形式化为四个截然不同的资源壁垒，并推出了访问结构化感知（Access-Structured Perception, ASP），这是一个专为冻结多模态模型设计的免训练封装器。

该研究使用合成的长视野基准测试 SEW-Bench，对七个开源权重模型（参数量从 3B 到 31B 不等）进行了评估。研究结果表明，在严苛的预算下，查询条件下的访问机制远比单纯的参数规模扩展或上下文扩展更为关键。然而，研究也发现，在线提示词压缩（online prompted compression）无法证明其计算开销的合理性，最终触发了特定的预注册证伪标准。

📋 摘要

📋 Summary

Embodied multimodal agents frequently struggle to process continuously growing observation streams within strict per-decision token limitations. This research formalizes these limitations into four distinct resource walls and introduces Access-Structured Perception (ASP), a training-free wrapper designed for frozen multimodal models.

Evaluated across seven open-weight models (ranging from 3B to 31B parameters) using the synthetic long-horizon benchmark SEW-Bench, the findings demonstrate that query-conditioned access is far more critical under tight budgets than raw parameter scaling or context expansion. However, the study also reveals that online prompted compression fails to justify its computational overhead, ultimately triggering specific pre-registered falsification criteria.

🧱 四个资源壁垒

🧱 The Four Resource Walls

作者通过四个基础壁垒，将预算受限的具身感知的局限性进行了形式化定义： 1. 感知香农壁垒（Perceptual Shannon Wall）： 限制了活动状态表示的容量。 2. 视野壁垒（Horizon Wall）： 限制了随时间推移进行与查询无关的帧选择。 3. 轮次壁垒（Round Wall）： 限制了非自适应的检索机制。 4. 条件组合壁垒（Conditional Composition Wall）： 强加了固定深度的推理限制。

The authors formalize the constraints of budget-limited embodied perception through four fundamental barriers: 1. Perceptual Shannon Wall: Bounds the capacity of the active state representation. 2. Horizon Wall: Limits query-independent frame selection over time. 3. Round Wall: Restricts non-adaptive retrieval mechanics. 4. Conditional Composition Wall: Imposes fixed-depth inference limits.

⚙️ 方法：访问结构化感知 (ASP)

⚙️ Approach: Access-Structured Perception (ASP)

为了克服这些壁垒，作者提出了 ASP，这是一种专为冻结的多模态模型定制的免训练封装器，包含三个核心组件： * 一个有上限的结构化状态表示。 * 一个逐字的（verbatim）情景索引。 * 查询条件下的预算分配，并结合了迭代访问机制。

To overcome these barriers, the authors propose ASP, a training-free wrapper tailored for frozen multimodal models comprising three core components: * A capped structured state representation. * A verbatim episodic index. * Query-conditioned budget allocation coupled with iterative access mechanisms.

📊 关键发现与结果

📊 Key Findings & Results

  • 情景检索性能： 在受限的 4,096 Token 决策预算下，ASP 实现了 75% 至 94% 的情景检索准确率，显著优于同等预算下与查询无关的采样（仅得 3% 至 19%）。
  • 预算重新分配： 在评估的每个主干模型中，动态重新分配预算证明比将基线采样预算扩大四倍更有效。
  • 架构证伪： 尽管检索性能强劲，但在完整的三个组件架构中，通道对偶性（channel duality）并未得到验证：
  • 移除压缩状态实际上将旗舰平均性能从 35.4 提高到了 58.0。
  • 在任何模型主干上，ASP 的表现均未超过纯逐字（verbatim-only）基线。
  • 四个预注册证伪标准中有两个被触发。
  • Episodic Retrieval Performance: Under a constrained 4,096-token decision budget, ASP achieves an episodic retrieval accuracy of 75% to 94%, significantly outperforming equal-budget query-independent sampling (which scores only 3% to 19%).
  • Budget Reallocation: Dynamically reallocating the budget proves more effective than quadrupling the baseline sampling budget across every evaluated backbone model.
  • Architecture Falsification: Despite strong retrieval performance, channel duality is not validated in the full three-component architecture:
  • Removing the compressive state actually raises the flagship mean performance from 35.4 to 58.0.
  • ASP does not outperform the verbatim-only baseline on any model backbone.
  • Two out of four pre-registered falsification criteria fire.

🔗 链接与资源