跳转至

从语料库到协同进化能力：面向通用图像生成的以能力为中心的数据设计

文章背景与核心概要

尽管大规模图像生成领域在数据规模、质量优化、重采样及重标注方面取得了显著进展，但传统的训练流程通常将特定任务的数据集孤立地进行优化。本文指出，当前的核心挑战不仅在于如何策划单个任务的语料库，更在于如何根据生成能力之间的内在依赖关系，对异构监督信号进行战略性组织。

为了解决这一问题，作者提出了一种“以能力为驱动的数据基础设施”，将特定能力的监督构建与能力对齐的课程调度相结合。该框架通过三个互操作的数据引擎，建立了文本-图像对齐、图像间转换以及图像-知识关联的互补监督关系。此外，通过多阶段课程学习，模型能够按照能力获取的依赖顺序，逐步演进任务组合、视觉概念分布及数据质量。

该研究最终构建了一个包含4.4亿张图像的T2I语料库、1.2亿对编辑数据以及超过2700万对图像-实体数据。基于此基础设施，作者从零开始训练了3B和6B参数规模的多模态扩散模型。在CPI-Bench及定性评估中的表现证明，该方法在视觉覆盖范围、渲染多样性以及生成能力迁移方面均展现出卓越的性能。

📑 摘要

Large-scale image generation has benefited from advances in data scale, quality, rebalancing, and recaptioning, yet conventional pipelines typically optimize task-specific datasets in isolation. A central challenge is not only how to curate each task-specific corpus, but also how to organize heterogeneous supervision according to the dependencies among generative capabilities.

We present a capability-driven data infrastructure that couples capability-specific supervision construction with capability-aligned curriculum scheduling. Its three specialized yet interoperable data engines build complementary relational supervision for text-image grounding, inter-image transformation, and image-knowledge association, while caption experts align T2I and editing supervision across tasks and granularities. A multi-stage curriculum jointly evolves task composition, visual-concept distribution, data quality, and image resolution along the dependency order of capability acquisition, with capability-aware evaluation closing the loop through targeted retrieval, expert construction, and gap-aware resampling.

At scale, the framework curates a 440M-image T2I corpus, 120M editing pairs, and over 27M image-entity pairs. With this infrastructure, we train multimodal diffusion models at two scales from scratch, with 3B and 6B sizes respectively. We conduct quantitative evaluation on CPI-Bench, along with qualitative evaluations across diverse text-to-image and editing scenarios. Experimental results present broad visual coverage, versatile rendering, and effective transfer across generative capabilities.

📊 附加元数据

  • 主要学科: 计算机视觉与模式识别 (cs.CV)
  • 次要学科: 人工智能 (cs.AI)
  • 备注: 19页，10张图表