跳转至

文章背景与核心概要

开源大语言模型（LLM）通常通过复杂的多阶段训练流程构建，这在其来源、所有权和演变过程之间产生了错综复杂的关系。为了解决模型溯源、治理和供应链完整性问题，本文引入了LLM“生物特征”的概念，探讨模型是否仅在权重空间中就展现出内在指纹（无需访问输入数据）。

为了解决这一血缘甄别问题，研究人员提出了一个统一的几何指纹框架，从两个互补的视角分析权重矩阵：一是通过奇异值分布捕获以编码全局幅度模式的“谱能量”；二是通过子空间偏差量化以捕获定向几何的“子空间对齐”。在超过110组不同的开源大语言模型对上进行的测试表明，谱能量能够可靠地区分独立训练的模型和模型家族，而子空间对齐则能够对紧密相关的模型（如具有不同数据集规模或经过后训练调整的模型）进行细粒度区分。

Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Space

Summary

Open-weight large language models (LLMs) are often built through complex, multi-stage training pipelines, creating intricate relationships regarding their origin, ownership, and evolution. This paper introduces the concept of LLM "biometrics" to determine whether models exhibit intrinsic fingerprints directly within weight space—without requiring access to input data.

The authors propose a unified geometric fingerprinting framework to solve this lineage discrimination problem, analyzing weight matrices through two complementary perspectives: 1. Spectral energy (captured via singular value distributions to encode global magnitude patterns). 2. Subspace alignment (quantified via subspace deviations to capture directional geometry).

Testing on over 110 diverse open-weight LLM pairs, the research demonstrates that spectral energy reliably separates independently trained models and model families, while subspace alignment permits fine-grained differentiation among closely related models (such as those with varying dataset scales or post-training adjustments).

Paper Metadata

论文元数据

  • arXiv Identifier: arXiv:2608.07786 [cs.AI]
  • Authors: Yiwei Chen, Bingqi Shang, Sijia Liu
    • Authors: Yiwei Chen, Bingqi Shang, Sijia Liu
  • Submitted: August 7, 2026
    • Submitted: August 7, 2026
  • Accepted to: COLM 2026
    • Accepted to: COLM 2026
  • Primary Subject: Artificial Intelligence (cs.AI)
    • Primary Subject: Artificial Intelligence (cs.AI)
  • Secondary Subject: Machine Learning (cs.LG)
    • Secondary Subject: Machine Learning (cs.LG)
  • DOI: 10.48550/arXiv.2608.07786

Abstract

摘要

开源权重的大语言模型（LLM）越来越多地通过复杂的多阶段流水线进行开发，从而产生复杂的血缘关系，反映出模型的来源、所有权和演变过程。理解这些关系对于模型溯源、治理以及供应链完整性至关重要。在这项工作中，我们研究了LLM“生物特征”（类似于人类生物特征）的概念，探讨LLM是否仅凭权重空间中的内在指纹（无需访问输入数据）就能揭示其来源和血缘。我们将此表述为一个血缘甄别问题，以区分独立来源、同系列和共享基础的模型。为了表征这些关系，我们提出了一个统一的几何指纹框架，从两个互补的视角分析权重矩阵：（i）谱能量，通过奇异值分布捕获以编码全局幅度模式；（ii）子空间对齐，通过子空间偏差量化以捕获定向几何。我们的分析揭示了权重空间中结构相似性的清晰层级：谱能量能够可靠地区分独立训练的模型和不同的模型家族，而子空间对齐则能够对紧密相关的模型（包括数据集规模的变化和后训练程序）进行细粒度甄别。在超过110个不同的开源LLM模型对上进行的广泛实验表明，权重空间几何为模型血缘提供了稳健且可解释的信号，从而实现了共享基础模型内部的粗粒度机制分离和细粒度甄别。

Open-weight large language models (LLMs) are increasingly developed through complex, multi-stage pipelines, leading to intricate lineage relationships that reflect model origin, ownership, and evolution. Understanding these relationships is important for model provenance, governance, and supply-chain integrity. In this work, we investigate the notion of LLM "biometrics" (analogous to human biometrics) to ask whether LLMs exhibit intrinsic fingerprints in weight space alone, without access to input data, that reveal their origin and lineage. We formulate this as a lineage discrimination problem, distinguishing among independent-origin, same-series, and shared-base models. To characterize these relationships, we propose a unified geometric fingerprinting framework that analyzes weight matrices from two complementary perspectives: (i) spectral energy, captured by singular value distributions to encode global magnitude patterns, and (ii) subspace alignment, quantified via subspace deviations to capture directional geometry. Our analysis uncovers a clear hierarchy of structural similarity in weight space: spectral energy reliably distinguishes independently trained models and different model families, while subspace alignment enables fine-grained discrimination among closely related models, including variations in dataset scale and post-training procedures. Extensive experiments on over 110 diverse open-weight LLM pairs demonstrate that weight-space geometry provides a robust and interpretable signal for model lineage, enabling coarse-grained regime separation and fine-grained discrimination within shared-base models.

链接与资源