跳转至

文章背景与核心概要

传统的视频虚拟试穿（VVT）技术通常依赖于复杂且容易出错的多阶段预处理步骤，例如人体解析、姿态估计和服装几何变形等。这些显式几何先验中的微小误差往往会在生成结果中不断累积并导致最终画面失真。为了解决这一行业痛点，本文介绍了 UniVVT 框架，它彻底摆脱了传统方法对推理阶段预处理的依赖，将视频虚拟试穿重新定义为“语义条件驱动的视频生成”任务。

UniVVT 的核心技术在于引入了基于多模态大语言模型（MLLM）的场景-任务感知器（Scene-Task Perceiver），能够将源视频、目标服装和任务指令共同编码为紧凑的潜在特征标记。随后，通过轻量级语义桥接器将这些标记与基于扩散模型的视频生成器进行对齐，并结合三阶段渐进式训练策略，实现了高保真的服装迁移效果和在各大基准测试中的超强鲁棒性。该研究为端到端视觉生成任务提供了全新的设计范式。

UniVVT: A Unified End-to-End Framework for High-Fidelity Video Virtual Try-on

📋 Summary

UniVVT is a novel, unified end-to-end framework for Video Virtual Try-On (VVT) that eliminates the need for traditional, error-prone preprocessing steps like human parsing, pose estimation, and garment warping during inference. Instead of treating VVT as mask-conditioned video inpainting, UniVVT reframes it as semantically conditioned video generation.

Using a Multimodal Large Language Model (MLLM)-based scene-task perceiver, the framework jointly encodes the source video, target garment, and task instruction into compact latent tokens. A lightweight semantic bridge aligns these tokens with a diffusion-based video generator, and a three-stage progressive training strategy ensures high-fidelity garment transfer and robust performance across benchmarks.

📌 Document Overview

  • arXiv ID: arXiv:2608.05745 [cs.CV] (Submitted on August 6, 2026)
  • Primary Subject: Computer Vision and Pattern Recognition (cs.CV)
  • Secondary Subject: Artificial Intelligence (cs.AI)
  • Authors: Yushe Cao, Shikun Feng, Fei Shen, Haikuo Peng, Jianqiang Xia, Yiheng Zhu, Dianxi Shi, and Chun Yu
  • Metrics: 17 pages, 21 figures

🔬 Abstract & Key Innovations

Traditional VVT pipelines rely on a multi-stage design that complicates deployment and allows errors in explicit geometric priors (such as masks and poses) to propagate irreversibly into the generated output.

UniVVT solves this through: 1. Scene-Task Perceiver: Built on a Multimodal Large Language Model to jointly encode the source video, target garment, and task instruction into task-aware latent tokens. 2. Semantic Bridge: A lightweight component that aligns the latent tokens with the conditioning space of a diffusion-based video generator. 3. Three-Stage Progressive Training Strategy: Comprises semantic alignment, joint task adaptation, and flexible-resolution refinement to robustly couple heterogeneous components.