跳转至

用于视觉表征学习的双曲层次聚类

文章背景与核心概要

现代视觉骨干网络（如 Vision Transformers）主要依赖卷积、注意力机制或 MLP 等“Token Mixer”组件来实现图像块之间的信息交互。尽管这些方法在精度与计算成本之间取得了平衡，但它们大多表现为缺乏可解释性的“黑盒”模型，难以洞察其内部的编码逻辑。

为了解决这一局限性，本文提出了一种基于层次聚类范式的透明 Token Mixer——ClusterMixer。该方法通过在双曲空间（Hyperbolic Space）中执行聚类，能够以极低的失真度自然地捕捉视觉数据中固有的树状结构。基于此，作者构建了全新的骨干网络架构 HCFormer。实验表明，HCFormer 在图像分类、目标检测、实例分割和语义分割等多种视觉任务中均优于传统架构，为构建可解释的视觉骨干网络提供了新的研究范式。

📌 总结

现代视觉骨干网络（如 Vision Transformers）严重依赖 Token Mixer——即卷积、注意力机制或 MLP 等组件，以促进图像块之间的信息交换。虽然这些方法成功地平衡了精度和计算成本，但它们在很大程度上作为缺乏可解释性的“黑盒”运行。

Modern vision backbones (such as Vision Transformers) heavily rely on token mixers—components like convolutions, attention mechanisms, or MLPs—to facilitate information exchange between image patches. While these methods successfully balance accuracy and computational cost, they operate largely as opaque "black boxes" lacking interpretability.

为了克服这一局限，本文引入了 ClusterMixer，这是一种基于层次聚类范式、设计透明的 Token Mixer。通过在双曲空间中进行聚类（该空间能以最小的失真自然地容纳树状视觉数据结构），作者提出了 HCFormer，这是一种新颖的骨干网络架构。HCFormer 将 ClusterMixer 与专门的聚类策略相结合，在包括图像分类、目标检测、实例分割和语义分割在内的广泛计算机视觉任务中，表现优于传统的同类模型。

To overcome this limitation, this paper introduces ClusterMixer, a transparent, design-driven token mixer rooted in a hierarchical clustering paradigm. By performing clustering in hyperbolic space—which naturally accommodates tree-like visual data structures with minimal distortion—the authors propose HCFormer, a novel backbone architecture. HCFormer integrates ClusterMixer with specialized clustering strategies, outperforming traditional counterparts across a wide array of computer vision tasks, including image classification, object detection, instance segmentation, and semantic segmentation.

📖 摘要

我们通过重新审视机器学习中最经典的方法之一——聚类，对视觉骨干网络中的 Token Mixer 进行了研究。有效的 Token Mixer 是现代视觉骨干网络（如 Vision Transformers）的基本组件，它促进了图像块之间的信息交换。

We investigate the token mixer in vision backbones by revisiting clustering, one of the most classic approaches in machine learning. An effective token mixer is a fundamental component of modern vision backbones like vision Transformers, facilitating information exchange between image patches.

主流的 Token Mixer 依赖于卷积、注意力机制、MLP 或它们的混合体，主要关注精度与计算成本之间的权衡。然而，这些方法的一个显著缺点是其黑盒性质；它们的编码过程是不透明的，且缺乏可解释性。

Mainstream token mixers, which rely on convolution, attention, MLP, or their hybrids, primarily focus on navigating the trade-off between accuracy and computational cost. However, a significant drawback of these methods is their black-box nature; their encoding process is opaque and lacks interpretability.

为了摆脱这些不透明的设计，我们引入了 ClusterMixer，这是一种基于聚类范式且在设计上具有可解释性的透明 Token Mixer。ClusterMixer 通过层次聚类机制明确地制定了 Token 混合过程。为了建模视觉数据中固有的自然树状关系，聚类是在双曲空间中进行的，该空间非常适合以低失真嵌入层次结构。

Diverging from these opaque designs, we introduce ClusterMixer, a transparent token mixer that is grounded in a clustering paradigm and interpretable by design. ClusterMixer explicitly formulates the token mixing process through a hierarchical clustering mechanism. To model the natural, tree-like relationships inherent in visual data, the clustering is performed in hyperbolic space, which is well-suited for embedding hierarchies with low distortion.

基于这一创新，我们提出了 HCFormer，这是一种新的骨干网络架构，它将 ClusterMixer 与一系列精心设计的聚类策略相结合，以确保在各项任务中实现稳健的性能。大量的实验表明，HCFormer 在包括图像分类、目标检测、实例分割和语义分割在内的多种任务中始终优于同类模型。考虑到其透明度和有效性，我们希望 HCFormer 能够推动向可解释骨干网络范式的转变。

Building on this innovation, we present HCFormer, a new backbone architecture that integrates ClusterMixer with a series of meticulously designed clustering strategies to ensure robust performance across tasks. Extensive experiments demonstrate that HCFormer consistently outperforms its counterparts across diverse tasks, including image classification, object detection, instance segmentation, and semantic segmentation. Considering its transparency and efficacy, we hope HCFormer can facilitate a paradigm shift toward interpretable backbones.

📋 元数据与资源