VQ-bench：可组合式矢量量化框架与评测基准

VQ-bench: a Composable Vector Quantization Framework

Authors: Ashwin Padaki, Amir Ingber, Edo Liberty
Published: September 17, 2026 | Categories: Engineering, Research
Links: Website | GitHub Repository | Paper | Talk Slides | VecDB@VLDB 2026

文章背景与核心概要

在人工智能与现代信息检索领域，高维向量是向量数据库和各类大语言模型 (Large Language Model, LLM) 运转的核心数据载体，但全精度的海量向量存储与检索面临着极高的硬件成本。矢量量化 (Vector Quantization, VQ) 技术通过将高维连续向量压缩为紧凑的离散编码，成为大幅削减内存占用并加速相似度计算的关键支柱。然而，学术界与工业界虽然涌现出繁多的量化算法，却因评估指标各异、测试数据集不同以及底层硬件优化不一致，长期缺乏公平可比的标准评测。为此，Pinecone 正式推出了开源框架与基准评测套件 VQ-bench，创新性地将绝大多数复杂量化器解构为由少量核心原语模块组装而成的流水线。VQ-bench 不仅为经典与前沿量化算法提供了统一严谨的横向对比舞台，更让开发者能够像搭积木一样自由组合、快速验证新型量化策略，为向量检索生态注入了强大的模块化基础设施。

核心内容摘要

Summary

对于向量数据库和大语言模型 (Large Language Model, LLM) 而言，矢量量化 (Vector Quantization, VQ) 是一项不可或缺的关键技术，能够大幅压缩高维向量的存储空间。然而，学术界近年来发表了浩如烟海的量化器算法，由于评测指标各异、测试数据集不统一以及针对不同硬件的底层优化大相径庭，想要对它们进行公平公正的横向对比，一直是一大行业痛点。

Vector quantization (VQ) is crucial for reducing the storage size of high-dimensional vectors in vector databases and large language models (LLMs). However, evaluating and comparing the sheer volume of published quantizers has historically been challenging due to inconsistent metrics, datasets, and hardware optimizations.

为了攻克这一难题，Pinecone 推出了开源框架与基准评测套件 VQ-bench。它的精妙之处在于，将绝大多数主流量化器抽象为由少量基础“原语”组合而成的模块化“流水线”。本文将深入探讨 VQ-bench 如何实现量化器评测的标准化体系，详细拆解备受关注的 E-RaBitQ 等经典架构，并重点展示在 ArXiv 和 Yahoo 等公开数据集上的核心性能对比结果。

To solve this, Pinecone introduces VQ-bench, an open-source framework and benchmarking suite that models most published quantizers as modular "pipelines" built from a small set of primitive operations. This post outlines how VQ-bench standardizes quantizer evaluation, breaks down popular architectures like E-RaBitQ, and highlights key performance comparisons across datasets like ArXiv and Yahoo.

引言

Introduction

对于任何向量数据库来说，在对向量进行相似度检索之前，必须先将它们妥善存储在系统中。然而，以全精度直接保存海量的高维向量，硬件开销极为昂贵。**矢量量化 (Vector Quantization, VQ) ** 技术能够显著减少存储单个向量所需的比特位 (Bit) 数，因此成为了构建与运维高效向量数据库的核心技术基石。

Before a vector database can search vectors, it has to store them. But storing high-dimensional vectors at full precision is quite expensive. Vector quantization (VQ) reduces the number of bits needed to store a vector, making it a critical part of maintaining a vector database.

正因为 VQ 在向量数据库与大模型体系中扮演着举足轻重的角色，每年学术界都会涌现出海量的相关研究论文。早在打造首批系统原型时，Pinecone 就已经深入应用了量化技术。不过精益求精的追求永无止境，于是我们着手对近年来的前沿研究成果展开全面调研与基准测试。在这个过程中，市面上数量极其庞大的量化器让我们眼花缭乱；更棘手的是，几乎每篇论文的评测标准都不尽相同——各家使用的测试数据集互不一致，衡量的性能指标五花八门，底层更针对不同的硬件架构做了各自定制的加速优化。我们始终无法在业内找到任何能够对主流顶尖量化算法进行系统性、横向公平对比的研究工作。

Because VQ is so important (to both vector databases and LLMs), many research papers are published on the topic every year. Pinecone has been using quantization since its first prototypes. But we can always do better, so we set out to survey and benchmark newer results. We were pretty overwhelmed by just how many quantizers are out there. To make matters worse, every paper seemed to evaluate performance differently, measuring different metrics on different datasets and optimizing for different hardware. We were unable to find any systematic attempt to evaluate the leading methods against one another.

显然，想要从零开始百分之百忠实复现几十种量化器，本身就是一项巨大的挑战。幸运的是，随着我们对前沿文献的逐步深挖，一条清晰的规律浮出了水面：许多新发表的量化器本质上只是对现有算法的微小变体。事实上，绝大部分量化器都是基于一组数量相当有限的基础运算单元构建起来的。这激发了我们的灵感：如果我们把这些核心基础单元打包成一个开源原语库，让构建量化器变得如同照着菜单配菜一样简单——只需选定使用哪些原语并排定先后执行顺序，那会怎样？如此一来，我们便能在一个完全统一、透明且可复现的基准线上公平评测所有算法；同时，这也为研究人员探索现有量化器的全新变体、甚至是彻底发明全新的算法，铺平了实验道路。

Of course, faithfully implementing dozens of quantizers from scratch comes with its own challenges. Luckily, as we dug deeper into the literature, we began to notice a pattern. Many published quantizers are actually just slight variations of existing ones. In fact, most of them are built from a relatively small set of primitive operations. That gave us an idea: what if we published an open-source library of these core primitives, where building a quantizer was as easy as writing a recipe of which primitives to use and in what order? Then, we would be able to evaluate all of these quantizers in a fair and reproducible way. It would also make it easier to experiment with new variations of existing quantizers or invent new ones altogether.

这就是 VQ-bench 项目的缘起。伴随着这篇博文，我们非常自豪地正式向社区开源发布 VQ-bench，包含以下核心资源：

This was the start of the VQ-bench project. With this post, we're excited to share VQ-bench with the public, including:

• 公开的 官方网站 (website) ：持续更新并展示主流热门量化器的最新基准评测数据；
• 开源 GitHub 仓库：欢迎社区贡献您自己的量化器实现与运算原语；
• 专题学术 论文 (paper) ：发表于 VecDB@VLDB 2026 研讨会，并附带完整的 演讲幻灯片 (talk slides) 。

• A public website with a running benchmark of popular quantizers
• A GitHub repo where you can contribute your own quantizers and primitives
• A paper on VQ-bench (presented at VecDB@VLDB 2026), along with the talk slides.

请注意，目前发布的仅是 VQ-bench 的首个迭代版本；我们非常期待来自社区的宝贵反馈、勘误建议与代码贡献，后续我们也会持续纳入更多前沿量化器。

Note that this is just the first iteration of VQ-bench; we encourage feedback, corrections, and contributions, and we will add more quantizers over time.

量化器核心接口

Quantizers

通俗来说，**量化器 (Quantizer) ** 本质上是任何能够接收一组向量、将其压缩编码并在后续按需还原出目标信息的计算实体。在 VQ-bench 体系中，一个标准的量化器必须实现以下四种核心方法：

A quantizer is anything that can take a set of vectors, compress them, and recover desired information later on. In VQ-bench, a quantizer must implement four methods:

Method	Function
fit	given a sample of vectors (and optionally queries), learn a model
encode	given the model and a set of vectors, return per-vector codes
reconstruct	given the model and the code for vector x, reconstruct it
score	given the model, a query vector q, and the code for x, estimate the dot-product score ⟨q, x⟩

基础运算原语

Primitives

在学术界的研究中，量化器极少是从零孤立构建的，它们大都是由一组基础运算单元拼接组装而成的。在 VQ-bench 中，这套核心运算单元被形式化定义为原语 (Primitives) 。一个原语除了实现上述普通量化器具备的四种方法外，还额外实现了两个核心方法，用于精确定义数据如何流转传递给下一个处理阶段：

Quantizers are rarely built from scratch. In the literature, they are assembled from a small set of basic operations, which VQ-bench formalizes as primitives. A primitive implements the same four methods as any other quantizer, plus two more that specify exactly how it hands data to the next stage:

Method	Function
apply	given the model, transform the vectors into what the next stage should see
apply_queries	given the model, transform the queries into what the next stage should see

此外，原语的 reconstruct 与 score 方法分别将下游下一阶段输出的重构向量与得分估计值作为输入，进而完成逆向的数据折叠与聚合。

A primitive's reconstruct and score methods also take as input the next stage's reconstruction and score estimate, respectively.

这样一来，每个原语总共实现了六个核心方法。额外增加的两个方法构成了一套级联契约 (Chaining Contract) ，正是它们的存在，才使得各个原语能够像积木一样顺畅组合串联，这正是下一章节要探讨的核心。

That makes six methods in total. The extra two are the chaining contract: they are what let primitives be composed, which is the subject of the next section.

VQ-bench 将所有原语划分为三大主要类别：

VQ-bench implements three groups of primitives.

• 调节器 (Conditioners) ：负责对输入数据进行预处理变换，并将处理后的数据传递给下游模块，例如去均值中心化 (Center) 、归一化 (Normalize) 、主成分分析 (PCA) 、随机旋转 (RandomRotate) 等；
• 舍入器 (Rounders) ：负责将连续向量投影映射到有限码本 (Codebook) 中，并将逼近后的残差 (residual) 继续向下游传递，例如无符号整数映射 (CastUint) 、角度映射 (CastAngular) 、正态分布映射 (CastNormal) 、K 均值聚类 (KMeans) 等；
• 切分器 (Splitters) ：负责将高维向量拆分为多个子向量切片，并为每个切片分配各自独立的原语处理链，例如子空间分段 (Segment) 等。

• Conditioners transform the data and pass it downstream (Center, Normalize, PCA, RandomRotate, ...).
• Rounders cast each vector to a finite codebook, passing the residual downstream (CastUint, CastAngular, CastNormal, KMeans, ...).
• Splitters split the vectors and quantize each part with its own chain of primitives (Segment).

模块化流水线架构

Pipelines

所谓的流水线 (Pipeline) ，是指将两个或多个原语以链表形式级联组合而成的特殊量化器。压缩向量的过程就像沿着流水线正向穿行，而重构还原向量（或计算检索得分）的过程则是反向逆流而上。

A pipeline is a special type of quantizer given by composing two or more primitives in a chain. Compressing a vector walks it forward through the chain, and recovering a vector (or its score) walks it backward.

• 正向传递 (Forward pass) ：fit 与 encode 遵循相同的正向流转逻辑。在每个处理阶段，它们完成当前阶段的既定任务（训练拟合模型或生成编码）。随后调用 apply 方法对向量进行变换并递归传递给下一阶段。流程结束时，fit 会将各阶段的模型参数拼接合并，encode 则将各阶段生成的编码有序串联。
• 反向传递 (Backward pass) ：reconstruct 从流水线的最后一个阶段开始逆向执行。处于上层的每个阶段依次将自身对应的变换“撤销并叠加”回去（例如重新加上均值中心向量、做逆向旋转等），最终第一阶段就能还原出对原始向量的高精度近似。
• 得分计算 (score)：整体执行逻辑与反向传递类似，唯一区别在于每个阶段都需要看到与当前阶段视角完全一致的查询向量。因此，它首先通过 apply_queries 将查询向量单向正向推导一遍，随后再在计算得分时执行反向聚合。

• The forward pass: fit and encode follow the same path. At each stage, they perform that stage's job (learning the model / computing the codes). Then, they call apply to transform the vectors to the next stage and recurse. At the end, fit concatenates each stage's model and encode concatenates each stage's codes.
• The backward pass: reconstruct starts at the last stage. Each stage above it folds its own contribution back in (e.g., adding back the mean, undoing a rotation, etc.) until the first stage has an approximation of the original vector.
• score works the same way, except every stage needs the query as it saw the data. So, it begins by walking just the query forward with apply_queries. Then, it performs the backward pass on the score.

当然，量化器并非必须采用流水线架构。只要一个算法完整实现了上述四个核心方法，就可以作为合法的量化器接入体系，我们的接口充分包容了各类非流水线结构的算法设计。不过，文献中绝大多数已发表的量化方案都可以极其优雅地表达为原语流水线，这也正是这种解构设计能够作为统一基石的巨大价值所在。

A quantizer does not have to be a pipeline. Anything that implements the four methods qualifies, and the interface leaves room for methods that are built some other way. But most published quantizers can be expressed as pipelines of primitives, which is what makes the decomposition worth building on.

以近年来备受关注的前沿量化算法 E-RaBitQ 为例（在我们的系列基准实验中，它的表现极为出色）。E-RaBitQ 的核心处理管线仅仅由四个原语串联而成：

For example, E-RaBitQ is a popular quantizer (which we found to be quite performant in our experiments). The E-RaBitQ pipeline consists of four primitives:

1. 中心化 (Center) ：从每个向量中减去整个数据集的均值向量；
2. 归一化 (Normalize) ：将每个向量缩放调整为单位范数（模长为 1）；
3. 随机旋转 (Random Rotation) ：对每个向量施加一个随机正交旋转（或随机阿达马旋转，Random Hadamard Rotation）；
4. 角度映射 (Angular Cast) ：在超球面上根据角度将每个向量映射到离它最近的 \(b\) 位 (\(b\)-bit) 整数网格点上。

1. Center: subtract the average dataset vector from each vector
2. Normalize: scale each vector to unit norm
3. Random Rotation: apply a random orthogonal (or random Hadamard) rotation to each vector
4. Angular Cast: snap each vector to a -bit integer grid by rounding to the nearest grid point in angle.

下图直观展示了这一流水线的数据流向，下方表格则详细列出了各个原语在此流水线中所执行的具体函数逻辑：

A diagram of this pipeline and table for the primitive functions are given below.

The E-RaBitQ pipeline.

	Center	Normalize	Random Rotation	Angular Cast
fit	mean dataset vector μ	none	rotation seed	none
encode	none	the norm ‖x‖	none	grid(x) and cos(x, grid(x)) — b bits per dimension and one scalar
apply	x → x − μ	x → x / ‖x‖	x → Rx	x → x − ĝ, where ĝ = grid(x) / ‖grid(x)‖
apply_queries	identity	identity	q → Rq	identity
reconstruct	y → y + μ	y → ‖x‖ · y	y → Rᵀy	y → y + ĝ
score	s → s + ⟨q, μ⟩	s → ‖x‖ · s	s → s, since the query was rotated too	s → s + ⟨q, ĝ⟩ / cos(x, grid(x))

实验评测与结果分析

Experimental Results

我们在来自 VIBE 基准的 5 个公开数据集上，全面评测了包含 14 种经典及前沿算法的量化器套件。每个数据集均包含用于量化编码的基础向量集以及用于相似度打分的检索查询集。在此，我们重点展示其中两个典型数据集的评测结果：ArXiv（包含 1,344,643 个 768 维向量）和 Yahoo（包含 677,305 个 384 维向量）。更详尽完整的评测数据与交互图表均可在 官方网站 (website) 上公开查阅。

We evaluated a suite of 14 quantizers on 5 datasets from VIBE. Each dataset consists of vectors to encode and queries to score. Below, we present some results for two of the datasets: ArXiv (1,344,643 vectors in 768 dimensions) and Yahoo (677,305 vectors in 384 dimensions). You can view the full results on the website.

重构误差 (Reconstruction Error)

Reconstruction Error

**重构均方误差 (Reconstruction MSE) ** 是矢量量化领域的传统评估指标，在诸如大语言模型权重压缩等场景中至关重要。其具体测量方法为：首先从数据集中随机采样 1000 个测试向量 \(x\)；接着利用待测算法生成对应的重构向量 \(\hat{x}\)，并计算所有测试样本的均方误差平均值 \(\|x - \hat{x}\|^2\)。

Reconstruction MSE is the traditional metric for VQ, and it's important for applications like LLM weight compression. To measure it, we sample 1000 random dataset vectors . A quantizer reconstructs and we measure the average value of .

ArXiv

Yahoo

召回率评测 (Recall)

Recall

而在向量数据库与信息检索实际落地场景中，更贴近业务核心表现的指标则是召回率 (Recall) ，尤其在重排 (Reranking) 阶段中起着决定性作用。为了精确评估该指标，对于每个查询向量，我们首先计算出数据集中真实点积得分最高的前 1000 个目标向量；随后利用量化器快速预估这 1000 个向量的相似度得分，并统计预估得分最高的 Top-10 向量命中真实 Top-10 向量的比例，最终计算所有查询的平均值 (Recall@10)。

For vector databases, a more relevant metric is recall, specifically for reranking. To measure it, we take each query and compute the 1000 dataset vectors of maximum dot-product. A quantizer estimates these 1000 scores, and we measure what fraction of the estimated top-10 were contained in the true top-10 (averaging this fraction over all queries).

ArXiv

Yahoo

编码耗时对比 (Encode Time)

Encode Time

除了量化精度与检索质量，我们还详细记录了量化器对全量数据集进行压缩编码所需的总耗时。在测试实现中，编码过程采用分块流式处理并依托多线程进行并行加速。基准硬件环境配置为搭载 16GB 内存的 Apple M2 Pro 芯片，并在 6 个计算线程下测得。

We also measure how long it takes to encode the entire dataset. Note that encoding is done in chunks and accelerated via multithreading. These results were obtained on an Apple M2 Pro with 16GB RAM using 6 threads.

ArXiv

Yahoo

评测结论与洞察

Discussion

综合上述实验数据，我们可以总结出几条非常明确的技术趋势：乘积量化 (PQ) 与最优乘积量化 (OPQ) 在重构均方误差 (MSE) 方面始终保持着最低的领先水平；在检索召回率方面，EDEN 与 E-RaBitQ 的表现旗鼓相当，尤其在分配较高比特预算时优势更加突出；此外，EDEN 的编码处理速度远快于 PQ、OPQ 和 E-RaBitQ，兼顾了极高的压缩吞吐量与出色的精度表现，因而成为大多数实际工程量化场景中极具吸引力的理想选择。

Overall, we can see some clear trends. PQ and OPQ consistently have the lowest reconstruction MSE. EDEN and E-RaBitQ are comparable in terms of recall, especially at higher bit budgets. EDEN is also much faster to encode than PQ, OPQ, and E-RaBitQ, making it a good candidate for most quantization applications.

参与贡献与社区共建

Contribute

我们在最初架构设计时就将可扩展性作为核心考量，开源 GitHub 仓库 非常欢迎两类社区贡献：

We built VQ-bench to be extended, and the repo takes two kinds of contributions.

• 构想出了全新的量化器？ 通常只需编写几行简明代码即可完成实现。正如前文展示的 E-RaBitQ 管线，其源码仅仅是把 四个原语排成一个列表；而学术界发表的众多量化器，本质上也只是对库中已有原语进行重新排列组合而已；
• 发明了全新的运算原语？ 只需完整实现前文提到的六个接口方法，新原语便能立刻与目录中的所有现有原语无缝混搭组合。所有现有的以及未来尚未编写的处理流水线，均可自由调用它。

• Got a new quantizer? Usually just a few lines of code. The E-RaBitQ pipeline above is four primitives in a list, and many published quantizers are a similar reordering of primitives the library already ships.
• Got a new primitive? Implement the six methods above and it composes with every other primitive in the catalog. Every pipeline can use it, including the ones nobody has written yet.

无论你选择哪种贡献方式，都能直接开箱即用整套严谨的自动化评测体系。只需配置一份简短的参数文件，测试框架便会针对全套评测套件运行你的算法，并在完全统一的标准下测量各项关键指标：Recall@k 召回率、重构与得分估计误差、偏差分布、Softmax KL 散度与总变差、每维度占用比特数，以及编码、打分与重构的时空成本。随着我们持续扩充数据集和度量维度，基准评测的覆盖范围正在不断扩大。我们会定期刷新并更新公开发布的基准评测排行榜，届时社区新增的优秀方法也将被同步纳入展示。

Either way, you get the evaluation harness. A short config runs your method over the whole suite, measured exactly the way every other method is measured: recall@k, reconstruction and score error, bias, softmax KL and total variation, size in bits per dimension, and encode, score, and reconstruction cost. Both lists keep growing as we add datasets and metrics. We refresh the published benchmark on a regular cadence, and new methods are folded in then.

我们也热切期待来自原作者与社区的勘误。如果我们对您提出的量化算法理解有偏或实现有误，抑或是遗漏了值得纳入的重要方法，欢迎随时在 GitHub 提交 Issue 与我们深入交流！

We also want corrections. If we implemented your quantizer wrong, or we missed a method worth including, open an issue and tell us.