跳转至

停止全精度索引：重新审视向量嵌入的聚类方法

文章背景与核心概要

在当前的向量数据库与大模型检索增强生成（RAG）系统中，向量嵌入的高效索引和检索至关重要。传统的聚类方法（如基于 \(k\)-means 的粗量化索引）通常依赖于高维的全精度浮点数向量，这带来了巨大的存储开销和计算瓶颈。

本文重新审视了向量检索中的三项核心技术——降维、量化和维度裁剪，并提出了一种创新的索引构建管线：在聚类之前应用这些压缩技术。研究结果颠覆了传统认知，证明了聚类过程完全无需使用全精度向量；即便是极端压缩的 1-bit 编码也能实现近乎最优的聚类质量（与理想状态误差在 1% 以内），同时将存储需求锐减 60 倍并显著提升整体性能。

This paper investigates the optimization of vector embedding indexing through clustering by revisiting three core techniques: dimensionality reduction, quantization, and dimension pruning. The authors propose an indexing pipeline where these techniques are applied before clustering. Their findings demonstrate that using full-precision vectors for clustering is unnecessary; even 1-bit codes achieve near-optimal clustering quality (within 1% of ideal) while drastically reducing storage requirements by 60× and providing significant performance improvements.

📄 论文元数据 (Paper Metadata)

  • arXiv ID: arXiv:2608.14648 [cs.DB]
  • Title: Stop Indexing at Full Precision: Revisiting Clustering for Vector Embeddings
  • Authors: Leonardo Kuffo, Peter Boncz
  • Submitted On: 30 July 2026
  • Primary Subject: Databases (cs.DB)
  • Secondary Subjects: Artificial Intelligence (cs.AI), Machine Learning (cs.LG)
  • Venue: VLDB 2026 Workshop: The 2nd Workshop on Vector Databases
  • DOI: 10.48550/arXiv.2608.14648
  • arXiv ID: arXiv:2608.14648 [cs.DB]
  • Title: Stop Indexing at Full Precision: Revisiting Clustering for Vector Embeddings
  • Authors: Leonardo Kuffo, Peter Boncz
  • Submitted On: 30 July 2026
  • Primary Subject: Databases (cs.DB)
  • Secondary Subjects: Artificial Intelligence (cs.AI), Machine Learning (cs.LG)
  • Venue: VLDB 2026 Workshop: The 2nd Workshop on Vector Databases
  • DOI: 10.48550/arXiv.2608.14648

🔍 摘要 (Abstract)

在这项研究中，我们重新审视了向量搜索中广泛使用的三种技术，并利用它们通过聚类来优化向量嵌入索引：降维、量化和维度裁剪。我们提出了一种索引管线，在此管线中这些技术在聚类之前应用，并重点研究它们对存储占用、聚类时间以及向量搜索任务所得质心质量的影响。我们的结果表明，使用全精度向量进行聚类是过度的，因为即使是 1-bit 编码也可以实现近乎最优的聚类质量（与理想情况相差在 1% 以内），同时将存储需求减少 60 倍，并带来极具吸引力的性能提升。

In this study, we revisit three widely used techniques in vector search and utilize them to optimize vector embedding indexing through clustering: dimensionality reduction, quantization, and dimension pruning. We propose an indexing pipeline in which these techniques are applied before clustering, and we focus on how they affect storage footprint, clustering time, and the quality of the resulting centroids for vector search tasks. Our results reveal that using full-precision vectors for clustering is excessive, as even 1-bit codes can achieve near-optimal clustering quality (within 1% of ideal) while reducing storage requirements by 60x and delivering attractive performance gains.

📚 参考文献与外部工具 (References & External Tools)