跳转至

文章背景与核心概要

随着可穿戴设备和第一视角（Egocentric）摄像头的普及，长时序第一视角记忆系统能够将连续的第一人称音视频转化为可搜索的人类历史体验档案。然而，现有系统面临着两大核心瓶颈：一是基于缺乏上下文的字幕生成的索引在智能体检索时往往会失效；二是标准的检索机制往往忽略了用户问题中所隐含的时间背景或时间线相关性。

为了克服这些限制，本文作者推出了 EgoCITE（Egocentric Context-augmented Indexing and Time-aware Evidence retrieval，第一视角上下文增强索引与时间感知证据检索），这是一个专门针对长时序第一视角问答（QA）量身定制的智能体记忆框架。与基线记忆系统相比，EgoCITE 将回答准确率提升了 4.4% 至 14.2%，同时在计算成本上比长上下文大语言模型（LLM）智能体降低了 \(36\times\)。

EgoCITE: Context-Augmented Indexing and Time-Aware Retrieval for Long-Horizon Egocentric Memory

arXiv ID: arXiv:2608.12627 [cs.CV]
Authors: Le Zhang, Hao Chen, Vlad Roznyatovskiy, Jianzhong Zhang, Ke Sun
Primary Subject: Computer Vision and Pattern Recognition (cs.CV)
Submission History: Submitted August 12, 2026; revised August 18, 2026 (v3).

arXiv ID: arXiv:2608.12627 [cs.CV]
Authors: Le Zhang, Hao Chen, Vlad Roznyatovskiy, Jianzhong Zhang, Ke Sun
Primary Subject: Computer Vision and Pattern Recognition (cs.CV)
Submission History: Submitted August 12, 2026; revised August 18, 2026 (v3).

📌 Executive Summary

Long-horizon egocentric memory systems convert continuous first-person video and audio into a searchable archive of past human experiences. However, current systems suffer from two primary bottlenecks: 1. Unreliable Indices: Indices derived from context-poor captions fail during agentic search. 2. Ignored Temporal Intent: Standard retrieval mechanisms overlook the temporal context or timeline relevance implied by a user's question.

To overcome these limitations, the authors introduce EgoCITE (Egocentric Context-augmented Indexing and Time-aware Evidence retrieval), an agentic memory framework tailored for long-horizon egocentric Question Answering (QA). EgoCITE improves answer accuracy over baseline memory systems by 4.4% to 14.2% while operating at \(36\times\) lower computational cost than long-context Large Language Model (LLM) agents.

📌 执行摘要

长时序第一视角记忆系统将连续的第一人称视频和音频转换为可搜索的过去人类经验档案。然而，当前系统面临两个主要瓶颈： 1. 不可靠的索引： 从缺乏上下文的字幕中提取的索引在智能体检索过程中往往会失效。 2. 忽视时间意图： 标准的检索机制忽略了用户问题中隐含的时间背景或时间线相关性。

为了克服这些局限性，作者推出了 EgoCITE（Egocentric Context-augmented Indexing and Time-aware Evidence retrieval，第一视角上下文增强索引与时间感知证据检索），这是一个专为长时序第一视角问答（QA）量身定制的智能体记忆框架。EgoCITE 比基线记忆系统的回答准确率提高了 4.4% 到 14.2%，同时运行计算成本比长上下文大语言模型（LLM）智能体低 \(36\times\)。

🏗️ Core Components of EgoCITE

EgoCITE is structured around three modular components designed to streamline both storage and retrieval:

  1. EgoScheme (Context-Augmented Indexing): Utilizes local multimodal context to transform fragmentary, ambiguous video captions and speech transcripts into self-contained, atomic memory indices.
  2. EgoIndex (Multi-View Organization): Systematically organizes complementary modalities—including actions, activities, utterances, and conversations—into multi-view memory indices operating across various levels of granularity.
  3. EgoRetrv (Time-Aware Retrieval & Curation): Integrates standard semantic search with question-conditioned temporal relevance scoring alongside automated curation of retrieved evidence.

🏗️ EgoCITE 的核心组件

EgoCITE 围绕三个模块化组件构建，旨在简化存储和检索流程：

  1. EgoScheme（上下文增强索引）： 利用局部多模态上下文，将碎片化、模糊的视频字幕和语音转录转化为自包含的原子级记忆索引。
  2. EgoIndex（多视角组织）： 系统性地将互补模态（包括动作、活动、言语和对话）组织成在不同粒度级别上运行的多视角记忆索引。
  3. EgoRetrv（时间感知检索与策展）： 将标准语义搜索与受问题条件约束的时间相关性评分相结合，同时对检索到的证据进行自动策展。

📊 Evaluation & Performance

EgoCITE was evaluated across three prominent benchmarks: EgoLifeQA, EgoMem, and EgoR1-Bench.

  • Metric Focus: Answer accuracy and target-event retrieval alignment.
  • Key Results:
  • Outperformed existing agentic memory baselines by 4.4% – 14.2% in accuracy.
  • Achieved a \(36\times\) reduction in computational cost compared to conventional long-context LLM agent approaches.

📊 评估与性能

EgoCITE 在三个著名的基准测试中进行了评估：EgoLifeQA、EgoMem 和 EgoR1-Bench。

  • 指标重点： 回答准确率和目标事件检索对齐度。
  • 关键结果：
  • 准确率比现有的智能体记忆基准高出 4.4% – 14.2%。
  • 与常规的长上下文 LLM 智能体方法相比，计算成本降低了 \(36\times\)。

🔗 链接与资源