跳转至

文章背景与核心概要

通用多模态检索旨在同时支持高效的大规模匹配和精确的语义推理。传统的基于多模态大语言模型（MLLM）的嵌入技术依赖于隐藏状态，而思维链（CoT）推理则通过将中间证据编码到表示空间中来增强嵌入。然而，现有的CoT方法在隔离状态下分析查询和候选对象，无法有效区分正样本匹配和易混淆的难负样本。

为了克服这些局限性，作者推出了 UMER，这是一个统一的框架，它通过面向对感知的判别推理（Pair-Aware Discriminative Reasoning），在一个单一的MLLM中将对比嵌入与显式的成对相关性判断（判别排序）结合起来。此外，互补的互蒸馏策略在嵌入函数和排序函数之间传递可靠的偏好，从而在MMEB-V2基准测试上实现SOTA（最先进）的性能，并支持预算可调的推理。

UMER: Unifying Embedding and Ranking via Pair-Aware Discriminative Reasoning for Universal Multimodal Retrieval

arXiv: 2608.18504 [cs.AI]
Submitted on: August 19, 2026
Authors: Libiao Chen, Xiyang Liu, Yanheng Wei, Tao Wang, Zhenyu Tang

📌 Executive Summary

📌 Executive Summary

Universal multimodal retrieval requires both efficient large-scale matching and precise semantic reasoning. Traditional Multimodal Large Language Model (MLLM) embedding techniques rely on hidden states, while Chain-of-Thought (CoT) reasoning embeds intermediate evidence into the representation space. However, existing CoT methods analyze queries and candidates in isolation, failing to effectively differentiate between positive matches and confusable hard negatives.

To overcome these limitations, the authors introduce UMER, a unified framework that combines contrastive embeddings with explicit pairwise relevance judgment (discriminative ranking) inside a single MLLM via Pair-Aware Discriminative Reasoning. Additionally, a complementary mutual distillation strategy transfers reliable preferences between the embedding and ranking functions, enabling state-of-the-art performance on the MMEB-V2 benchmark alongside budget-adjustable inference.

📑 Abstract

通用多模态检索旨在支持多样化的指令感知检索任务，这既需要高效的语料库规模匹配，又需要细粒度的语义推理。最近基于 MLLM 的嵌入方法通常从隐藏状态推导表示，而思维链（CoT）推理正成为增强嵌入的一种有前景的策略，它通过将中间语义证据编码到表示空间中来实现。

📑 Abstract

Universal multimodal retrieval aims to support diverse instruction-aware retrieval tasks, demanding both efficient corpus-scale matching and fine-grained semantic reasoning. Recent MLLM-based embedding methods typically derive representations from hidden states, while Chain-of-Thought (CoT) reasoning is emerging as a promising strategy for embedding enhancement by encoding intermediate semantic evidence into the representation space.

然而，现有的 CoT 方法通常对查询和候选对象进行孤立的单项式（item-wise）推理，无法提供明确的证据来区分正样本和语义上易混淆的难负样本。此外，对比嵌入虽然能捕获全局相似性，但在需要答案验证、类别判断或细粒度推理的元任务（meta-tasks）上表现力不足。

However, existing CoT methods typically use item-wise reasoning over queries and candidates in isolation, providing no explicit evidence to distinguish a positive from a semantically confusable hard negative. Moreover, contrastive embeddings capture global similarity but struggle with meta-tasks requiring answer verification, category judgment or fine-grained reasoning.

在本文中，我们提出了 UMER，这是一个用于通用多模态检索的统一多模态嵌入与排序框架。UMER 用面向对感知的判别推理替代了单项式反思，该推理通过比较查询-候选对来识别指令相关的匹配与差异证据。UMER 在一个单一的 MLLM 中联合学习用于高效全局匹配的对比嵌入和用于显式成对相关性判断的判别排序。互补的互蒸馏策略进一步在嵌入和排序函数之间传递可靠的成对偏好。在 MMEB-V2 基准测试中，UMER 在同等实验设置下取得了最先进的性能，同时支持预算可调的推理。

In this paper, we propose UMER, a Unified Multimodal Embedding and Ranking framework for universal multimodal retrieval. UMER replaces item-wise reflection with Pair-Aware Discriminative Reasoning, which compares query--candidate pairs to identify instruction-relevant matching and discrepancy evidence. UMER jointly learns contrastive embeddings for efficient global matching and discriminative ranking for explicit pairwise relevance judgment within a single MLLM. A complementary mutual distillation strategy further transfers reliable pairwise preferences between the embedding and ranking functions. On the MMEB-V2 benchmark, UMER achieves state-of-the-art performance under comparable experimental settings while supporting budget-adjustable inference.