跳转至

文章背景与核心概要

传统的信息检索通常依赖“索引-检索-排序”的流水线，而生成式信息检索（GIR）则通过直接生成相关条目的标识符，提供了一种极具吸引力的替代方案。然而，现有的GIR方法面临着前缀级解码错误、缺乏多模态支持以及检索准确率落后于稠密向量方法等局限性。

为了克服这些挑战，本文作者提出了 DrIG，这是一个利用双角色标识符（Dual-role Identifiers）的通用多模态检索新型生成式框架。通过为每个候选条目分配一个残差量化的标识符，该标识符既可以按顺序串行执行（用于自回归解码），也可以作为一个无序集合（提供与前缀无关的相关性先验），从而有效地指导受限束搜索（constrained beam search）并摆脱局部最优。在 M-BEIR 和文本到图像评估数据集上的实验表明，DrIG 的性能超越了当前最先进的生成式基线，同时在效率与有效性之间取得了极具竞争力的平衡。

Generative Universal Multimodal Retrieval with Dual-role Identifiers

arXiv: 2608.12987 [cs.IR]
DOI: 10.48550/arXiv.2608.12987
Authors: Kaipeng Li, Haitao Yu, Xuanchen Zhou
Submitted: August 13, 2026 (Under review)
Subjects: Information Retrieval (cs.IR), Artificial Intelligence (cs.AI)

📌 Summary

生成式信息检索（GIR）通过训练生成器直接产生相关条目的标识符，已成为替代传统“索引-检索-排序”流水线的一种引人注目的方案。尽管前景广阔，但仍存在许多公开的挑战： 1. 受约束的从左到右解码容易受到前缀级错误和局部最优的影响。 2. 以往的大多数 GIR 研究在很大程度上仍属于单模态，对跨文本、图像以及图文混合条目的指令感知检索（instruction-aware retrieval）研究尚不充分。 3. 尽管基于离散标识符的 GIR 具有更高的效率，但其检索准确率仍然落后于尖端的基于稠密向量的检索方法。

为了解决这些挑战，我们提出了 DrIG，这是一个用于通用多模态检索的新型生成式框架，具有双角色标识符（Dual-role Identifiers），支持跨多种模态和领域的各种检索任务。每个候选条目被分配一个单一的残差量化标识符，该标识符具有两个互补的角色： * 顺序角色（Sequential Role）： 标识符以自回归方式解码，其中第一个token显式建模模态，其余token捕获渐进的精细语义。 * 基于集合的角色（Set-based Role）： 相同的token被重新解释为一个无序集合，以提供与前缀无关的相关性先验，从而指导受限束搜索并减轻局部最优错误。

关键实验发现

  • DrIG 在各种任务中始终优于最先进的生成式多模态基线。混合重排在与强稠密检索器的对比中取得了良好的效率-有效性权衡。
  • 消融和扩展分析揭示了基础大模态模型（LMM）、束大小（beam size）、重排深度和融合策略如何影响检索性能，为系统设计提供了实践指导。

Generative Information Retrieval (GIR) bypasses traditional index-retrieve-then-rank pipelines by directly generating relevant item identifiers. However, current GIR methods suffer from prefix-level decoding errors, a lack of multimodality, and lower retrieval accuracy compared to dense vector methods.

To address these limitations, the authors introduce DrIG, a novel generative framework for universal multimodal retrieval that utilizes Dual-role Identifiers. By assigning each candidate a residual-quantized identifier that functions sequentially (for autoregressive decoding) and as an unordered set (providing a prefix-independent relevance prior), DrIG effectively guides constrained beam search and overcomes local optima. Experiments on the M-BEIR and text-to-image evaluation datasets demonstrate that DrIG outperforms state-of-the-art generative baselines while achieving a competitive efficiency-effectiveness trade-off.

📝 Abstract

Generative information retrieval (GIR) has emerged as a compelling alternative to the conventional index-retrieve-then-rank retrieval pipeline by training a generator to produce the identifiers of relevant items directly. Despite its promise, a number of open challenges still remain: 1. Constrained left-to-right decoding is vulnerable to prefix-level errors and local optima. 2. Most prior GIR research remains largely unimodal, leaving instruction-aware retrieval across text, image, and mixed image-text items underexplored. 3. Although discrete identifier-based GIR offers higher efficiency, its retrieval accuracy still lags behind that of cutting-edge dense-vector-based retrieval methods.

Motivated by these challenges, we propose DrIG, a novel Generative framework for universal multimodal retrieval featuring Dual-role Identifiers, which supports diverse retrieval tasks across multiple modalities and domains. Each candidate is assigned a single residual-quantized identifier that serves two complementary roles: * Sequential Role: The identifier is decoded autoregressively, where the first token explicitly models modality and the remaining tokens capture progressively finer semantics. * Set-based Role: The same tokens are reinterpreted as an unordered set to provide a prefix-independent relevance prior, which guides constrained beam search and alleviates local-optimum errors.

Key Experimental Findings

  • DrIG consistently outperforms state-of-the-art generative multimodal baselines across diverse tasks. Hybrid reranking achieves a favorable efficiency-effectiveness trade-off against strong dense retrievers.
  • Ablation and scaling analyses reveal how the base Large Multimodal Model (LMM), beam size, reranking depth, and fusion strategy affect retrieval performance, offering practical guidance for system design.