跳转至

当用户不再主动提问：对话式智能体中上下文驱动记忆检索的基准评测

文章背景与核心概要

随着大语言模型（LLM）被广泛部署为长周期（long-horizon）对话智能体，高效的记忆系统变得愈发关键。然而，现有的基准测试主要通过孤立的问答（Q&A）式探测来评估记忆，而非模拟自然、原生的对话使用场景。

本文引入了 LOCOMO-CONV——一个由 LoCoMo 衍生而来的新型对话式记忆基准，它通过四种不同的查询风格（对话式、隐式、反事实和组合式）来评估智能体。通过对五个具有代表性的记忆系统在检索召回率和端到端响应质量两方面进行评估，研究表明：对话式的框架暴露出了标准问答基准所忽视的大量检索盲区，尤其是在隐式和组合式查询中。此外，作者发现强大的检索能力并不总能转化为优质的响应质量；隐式查询往往表现出“静默基础”（silent grounding），即记忆在没有显式浮现原始黄金事实（gold fact）的情况下，成功改善了上下文的基础性。为了推动该领域的发展，作者发布了辅助性的 supportive_memory 标注，用于捕获超越标准黄金证据的有用对话上下文。

摘要 (Summary)

As Large Language Models (LLMs) are deployed as long-horizon conversational agents, effective memory systems have become increasingly critical. However, existing benchmarks primarily evaluate memory through isolated Q&A-style probing rather than natural, in-situ conversational usage.

This paper introduces LOCOMO-CONV, a novel conversational memory benchmark derived from LoCoMo that evaluates agents across four distinct query styles: dialog, implicit, counterfactual, and composed. Evaluating five representative memory systems on both retrieval recall and end-to-end response quality reveals that conversational framing exposes substantial retrieval gaps overlooked by standard Q&A benchmarks—particularly on implicit and composed queries. Furthermore, the authors find that strong retrieval does not always translate into superior response quality, and that implicit queries often exhibit "silent grounding," where memory successfully improves contextual grounding without explicitly surfacing the original gold fact. To advance the field, the authors release auxiliary supportive_memory annotations to capture useful conversational context beyond standard gold evidence.

随着大语言模型（LLM）被部署为长周期对话智能体，构建高效的记忆系统变得越来越关键。然而，现有的基准测试主要通过孤立的问答式探测来评估记忆，而不是原生的对话使用场景。本文引入了 LOCOMO-CONV，这是一个从 LoCoMo 衍生而来的新型对话式记忆基准，评估智能体在四种不同查询风格下的表现：对话式、隐式、反事实和组合式。通过对五个具有代表性的记忆系统在检索召回率和端到端响应质量上的评估，结果表明，对话式框架暴露出了标准问答基准所忽视的大量检索盲区，尤其是在隐式和组合式查询中，多面查询重写（multi-facet query rewriting）虽然能缩小原始轮次记忆（raw-turn memory）的差距，但对抽象记忆（abstractive memory）无效。此外，我们发现强大的检索并不能完全转化为响应质量，且隐式查询表现出“静默基础”现象，此时记忆在未显式呈现黄金事实的情况下提升了上下文接地性。这些结果表明基于推理的记忆精细化是一个有前景的方向，我们同时发布了辅助性的 supportive_memory 标注，用以捕获超越原始黄金证据的对话有用上下文。

论文元数据 (Paper Metadata)

摘要原文 (Abstract)

Large language models (LLMs) are increasingly deployed as long-horizon conversational agents, motivating growing interest in memory systems. However, existing benchmarks primarily evaluate memory through QA-style probing rather than in-situ conversational usage. We introduce LOCOMO-CONV, a conversational memory benchmark derived from LoCoMo with four query styles: dialog, implicit, counterfactual, and composed. Across five representative memory systems, we evaluate both retrieval recall and end-to-end response quality. Our experiments show that conversational framing exposes substantial retrieval gaps overlooked by QA benchmarks, especially on implicit and composed queries, which multi-facet query rewriting narrows for raw-turn memory but not abstractive memory. We further find that strong retrieval does not fully translate into response quality, and that implicit queries exhibit silent grounding, where memory improves contextual grounding without explicitly surfacing the gold fact. These results point to reasoning-based memory elaboration as a promising direction, and we release auxiliary supportive_memory annotations capturing conversationally useful context beyond the original gold evidence.

大语言模型（LLM）正日益被广泛部署为长周期对话智能体，这激发了人们对记忆系统不断增长的兴趣。然而，现有的基准测试主要通过问答式探测来评估记忆，而非原生的对话使用场景。我们引入了 LOCOMO-CONV，这是一个源自 LoCoMo 的对话式记忆基准，具有四种查询风格：对话式、隐式、反事实和组合式。在五个具有代表性的记忆系统中，我们评估了检索召回率和端到端响应质量。我们的实验表明，对话式框架暴露了被问答基准所忽视的大量检索盲区，特别是在隐式和组合式查询中，多面查询重写缩小了原始轮次记忆的差距，但对抽象记忆无效。我们进一步发现，强大的检索并不能完全转化为响应质量，并且隐式查询表现出静默基础，即记忆在未显式浮现黄金事实的情况下改善了上下文接地性。这些结果指向了基于推理的记忆细化这一有前景的方向，并且我们发布了辅助性的 supportive_memory 标注，用以捕获超越原始黄金证据的对话有用上下文。

访问与全文链接 (Access and Full-Text Links)

引用工具与参考文献 (Citation Tools & References)

You can explore references, citations, and auxiliary tools for this paper via the following platforms: * Google 学术 * Semantic Scholar * NASA ADS

你可以通过以下平台探索本文的参考文献、引用及辅助工具： * Google Scholar * Semantic Scholar * NASA ADS