ATBInsight

如何精准捕获代码缺陷：基于 Rust 正则引擎的高效模糊测试实战

Mon, 21 Sep 2026 00:00:00 GMT

在复杂系统与底层基础库的开发中，传统的单元测试往往只能覆盖开发者预想到的常规路径，极易漏掉隐蔽的边界状态与特征交互缺陷。本文源自知名技术讨论社区 Lobste.rs 上关于“测试有效性”的探讨，作者以 Rust 生态核心的 `regex` 正则引擎历史真实 Bug 为例，展示了如何脱离臃肿庞杂的外部测试框架，仅凭一个轻量级的伪随机数生成器 (PRNG) 与群集测试 (Swarm Testing) 策略，高效生成极短却极具杀伤力的边缘用例。通过将待测引擎与标准实现 (Oracle) 交叉比对，这种轻巧敏捷的方法不仅执行速度极快，还能在极短时间内精准揪出传统测试难以触及的深层逻辑漏洞。这种实战经验表明，高质效的测试并不依赖海量算力与庞大测试数据，而是取决于精巧的测试架构设计与特征组合策略。 --- * **发布日期：** 2026 年 9 月 19 日 * **原文出处：** [Lobste.rs 讨论区](https://lobste.rs/s/mkv2pl/unit_tests_mark_territory_more_than#c_5nimhl) > * **Published on:** Sep 19, 2026 > * **Source:** [Lobste.rs Discussion](https://lobste.rs/s/mkv2pl/unit_tests_mark_territory_more_than#c_5nimhl) --- ## 📌 核心概述 > ## 📌 Summary 本文通过为 Rust 语言的核心正则库 `regex` 量身打造一个轻量级模糊测试器 (Fuzzer) ，深入探讨了生成式随机测试 (Generative Testing) 相比于传统单元测试的强大优势。作者并未依赖笨重庞大的测试框架，而是展示了如何仅凭一个极简的伪随机数生成器 (PRNG) ，搭配权威比对基准 (Oracle) ——直接将 `regex` 与轻量版 `regex_lite` 的运行结果进行交叉比对——就能精准揪出那些隐藏极深的代码缺陷。 > This article explores the effectiveness of generative (randomized) testing compared to traditional unit tests by building a custom fuzzer for the Rust `regex` crate. Rather than relying on complex frameworks, the author demonstrates how a lightweight, pseudo-random number generator (PRNG) coupled with an oracle (comparing `regex` against `regex_lite`) can successfully uncover hidden bugs. 文中所阐述的核心原则包括：采用 **群集测试 (Swarm Testing)** 策略 (随机分配特性分布与字符集) 、专注于挖掘 **短小而刁钻的边界极端用例** 而非盲目堆砌海量数据负载，以及将漏掉的 Bug 视作测试框架本身的设计不足并不断反哺完善。 > Key principles covered include **swarm testing** (randomizing feature distributions and alphabets), targeting **small and tricky edge cases** rather than massive payloads, and treating missed bugs as failures in the testing harness itself. --- ## 缺陷溯源：一个真实的正则表达式 Bug > ## The Bug 对于正则表达式 `".abb|b"` 和输入字符串 `"zabb"`，旧版本的 `regex` crate 出现了一个离奇的错误：它返回的第一个匹配项居然是 `"b"`，而不是理应匹配的完整字符串 `"zabb"`： > For the `".abb|b"` regex and `"zabb"` input, an older version of the `regex` crate incorrectly returned `b` as the first match instead of the entire `zabb` string: ```rust use regex; fn main() { let r = regex::Regex::new(".abb|b").unwrap(); let m = r.find("zabb").unwrap(); // Fails with regex-automata=0.4.15: assert_eq!(m.as_str(), "zabb") } ``` 我们该如何主动发现这类诡异的缺陷呢？正则表达式引擎属于纯粹的算法实现，这使它们成为了应用生成式测试的绝佳温床。而在算法测试领域，最强大、最直接的手段莫过于将待测系统的输出与已知正确的权威答案 (即测试预言机 Oracle) 进行比对。虽然原始评论中提到其测试工具当时缺乏可用的 Oracle，但在构建稳健系统时，我们在架构设计之初就应当把 Oracle 纳入考量。幸运的是，在针对 `regex` 库的案例中，我们完全可以直接拉来 `regex_lite` 库作为权威参考，将两者的输出进行全自动交叉验证。 > How do we find bugs like this? Regular expression engines are pure algorithms, making them prime candidates for generative testing. The most powerful technique for testing algorithms is to compare them against a known correct answer (an oracle). While the original comment noted their fuzzer lacked an oracle, robust systems should be designed with oracles in mind. Fortunately, for our `regex` case study, we can simply cross-check outputs against the `regex_lite` crate. --- ## 生成测试字符串 > ## Generating a String 我们可以借助一个极简的伪随机数生成器 (PRNG) ，搭建一个纯粹的随机字符串生成工具： > We can build a simple random string generator using a pseudo-random number generator: ```rust use fastrand::Rng; ``` 在进行随机化测试时，人们往往有一种下意识的直觉：疯狂生成海量的数据载荷 (例如动辄 5 GiB 的超大输入) 。然而在现实中，软件缺陷通常孕育自各种特性之间微妙而精细的小规模交互，而非单纯因为数据量巨大。 > When performing randomized testing, the instinct is often to generate massive payloads (e.g., 5 GiBs of input). However, bugs usually stem from small, intricate interactions between features. 我们的生成策略遵循两步走方案： > Our strategy follows a two-step approach: 1. **确立基准字符集**：从现有的单元测试中提取并固定基础字符表； 2. **动态子集抽样**：在每一次迭代中，随机挑选该字符表的一个 **子集** (即群集测试 Swarm Testing 的精髓) ，随后仅使用这些被挑中的字符生成随机长度的测试字符串。 > 1. Fix a base alphabet derived from unit tests. > 2. For each iteration, pick a random *subset* of that alphabet (**swarm testing**), then generate a string of random length using only those characters. 为了把运行性能压榨到极致，我们在各轮迭代之间通过预分配策略循环复用内存缓冲区，彻底避免频繁的动态内存申请： > To maximize performance, we reuse memory across iterations via static allocation: ```rust use fastrand::Rng; fn main() { let mut rng = Rng::new(); // Re-use the same memory for all tests. let mut text_alphabet: Vec = vec![]; let mut text: Vec = vec![]; for _ in 0..1_000_000 { // It's unlikely that a counter example with // 7 different letters exists, while there // isn't one with just 6. alphabet_swarm(&mut rng, b"abcdef", &mut text_alphabet); let text = gen_string(&mut rng, &text_alphabet, &mut text); } } fn alphabet_swarm<'a>( rng: &mut Rng, all: &[u8], pick: &'a mut Vec, ) { pick.clear(); pick.extend(all); rng.shuffle(pick); let count = rng.usize(1..=pick.len()); pick.truncate(count); } fn gen_string<'a>( rng: &mut Rng, alphabet: &[u8], result: &'a mut Vec, ) -> &'a str { result.clear(); // Again, this is a short string. // Longer failures are not likely. let count = rng.usize(0..8); for _ in 0..count { result.push(alphabet[rng.usize(0..alphabet.len())]); } str::from_utf8(result).unwrap() } ``` --- ## 设定正则语法特性的分布权重 > ## Generating a Regex Distribution 在生成正则表达式时，我们同样贯彻这种群集测试思路： > We apply the same swarm-testing strategy to generate regular expressions: * 随机激活正则语法语义特性的某一个子集； * 随机决定生成表达式的长度规模； * 全程复用内存缓冲区。 > * Pick a subset of active regex features. > * Pick sizes at random. > * Re-use memory buffers. 首先，我们为不同的正则表达式语法特性定义离散权重，而不是非开即关的布尔开关： > First, we define weights for different regex features rather than using binary toggles: ```rust #[derive(Default, Debug)] struct ReOptions { alt: u16, // | rep: u16, // * any: u16, // . lit: u16, // 'a' sum: u16, alphabet: Vec, } ``` 接下来，我们实现群集抽样逻辑，在运行期间动态随机配置各项语法特性的权重与字符集： > Next, we implement the swarm logic to dynamically configure weights and alphabets: ```rust impl ReOptions { fn swarm(&mut self, rng: &mut Rng, alphabet_full: &[u8]) { // We _still_ want to enable a few features at a time. self.alt = if rng.bool() { 0 } else { rng.u16(0..100) }; self.rep = if rng.bool() { 0 } else { rng.u16(0..100) }; self.any = if rng.bool() { 0 } else { rng.u16(0..100) }; self.lit = rng.u16(1..100); self.sum = self.alt + self.rep + self.any + self.lit; assert!(self.sum > 0); alphabet_swarm(rng, alphabet_full, &mut self.alphabet); } } ``` --- ## 递归生成正则表达式 > ## Generating a Regex 我们采用递归方式构造复杂的正则表达式语法树，通过向下传递输出缓冲区以及一个控制表达式规模的 `size` 参数来精确约束生成深度： > We construct regular expressions recursively, passing down output buffers and a `size` parameter to control expression length: ```rust fn gen_re( rng: &mut Rng, options: &ReOptions, result: &mut Vec, ) { result.clear(); let size = rng.u8(0..8); gen_re_rec(rng, options, result, size); } fn gen_re_rec( rng: &mut Rng, options: &ReOptions, result: &mut Vec, size: u8, ) { if size == 0 { return; // Base case, empty regex. } // Pick one of the features, according to weights. let mut p = rng.u16(0..options.sum); if p < options.alt { // Alternation distributes the size // among the two children. let size_left = rng.u8(0..=size - 1); let size_right = size - size_left - 1; assert!(size == size_left + 1 + size_right); result.push(b'('); gen_re_rec(rng, options, result, size_left); result.extend(b")|("); gen_re_rec(rng, options, result, size_right); result.push(b')'); return; } p -= options.alt; if p < options.rep { result.push(b'('); gen_re_rec(rng, options, result, size - 1); result.extend(b")*"); return; } p -= options.rep; if p < options.any { gen_re_rec(rng, options, result, size - 1); result.push(b'.'); return; } p -= options.any; if p < options.lit { gen_re_rec(rng, options, result, size - 1); let index = rng.usize(0..options.alphabet.len()); let lit = options.alphabet[index]; result.push(lit); return; } unreachable!(); } ``` --- ## 缺陷搜索主循环 > ## Search Loop 考虑到编译正则表达式本身涉及状态机构建，属于计算密集型操作，我们可以将每次编译出的一对正则对象保留下来，复用于数千个随机输入字符串的匹配验证： > Because compiling regular expressions is computationally expensive, we can test multiple input strings against a single compiled regex pair: ```rust fn main() { let mut rng = Rng::new(); let mut options = ReOptions::default(); let mut text_alphabet: Vec = vec![]; let mut text: Vec = vec![]; let mut re: Vec = vec![]; let mut test_count: u32 = 0; for _ in 0..1_000_000 { options.swarm(&mut rng, b"abcdef"); alphabet_swarm(&mut rng, b"abcdefx", &mut text_alphabet); gen_re(&mut rng, &options, &mut re); let re = str::from_utf8(&re).unwrap(); let r1 = regex::Regex::new(re).unwrap(); let r2 = regex_lite::Regex::new(re).unwrap(); for _ in 0..1000 { test_count += 1; let text = gen_string(&mut rng, &text_alphabet, &mut text); let m1 = r1.find(text) .map_or("not found", |it| it.as_str()); let m2 = r2.find(text) .map_or("not found", |it| it.as_str()); if m1 != m2 { eprintln!("err re={re} text={text} m1={m1} m2={m2}"); return; } if test_count % 500_000 == 0 { eprintln!("ok re={re} text={text}"); } } } } ```

PrismML 发布三值大模型 Ternary Bonsai 2 27B：仅 5.9GB 体积保留 98.2% 顶尖性能

Sun, 20 Sep 2026 00:00:00 GMT

近年来，如何在保持前沿大语言模型 (Large Language Model, LLM) 强大能力的同时，将其体积极致压缩以便在消费级设备上流畅运行，始终是 AI 领域的核心技术挑战之一。PrismML 团队近期正式开源了基于 Qwen3.8 27B 深度优化的三值大模型——**Ternary Bonsai 2 27B**。该模型采用宽松友好的 Apache 2.0 协议，创新性地将权重限制为仅包含 -1、0、+1 的三值离散表示，成功把原本高达 53.80 GB 的庞大模型压缩至惊人的 5.93 GB (平均每个权重仅占 1.72 比特) ，并在 20 项涵盖推理、数学与代码的基准测试中平均保留了原模型 98.2% 的顶尖性能。凭借对文本与图像的多模态理解能力以及 262K 的超长上下文窗口支持，该模型不仅可以在普通 16 GB 笔记本或单张 24 GB 消费级显卡上高吞吐流畅运行，更标志着百亿级端侧大模型向极致能效与平民化部署迈出了里程碑式的一步。 --- ## 核心概述 > ## Summary [PrismML](https://prismml.com/) 正式推出了 [Ternary Bonsai 2 27B](https://prismml.com/news/bonsai-2-27b)，这是基于 [Qwen3.8 27B](https://huggingface.co/Qwen/Qwen3.8-27B) 打造的高能效三值权重衍生模型。该模型采用 Apache 2.0 开源协议，将原本高达 53.80 GB 的半精度 (FP16) 原始模型彻底瘦身至仅 5.93 GB，并在 20 项基准测试中令人惊叹地保留了父模型 98.2% 的平均性能。Ternary Bonsai 2 27B 不仅支持文本与图像理解，还具备 262K Token 的超长上下文窗口；借助 PrismML 提供的定制化运行时，它能够在普通的 16 GB 笔记本电脑或单张 24 GB 显存显卡上轻松完成本地部署。 > [PrismML](https://prismml.com/) has introduced [Ternary Bonsai 2 27B](https://prismml.com/news/bonsai-2-27b), a highly efficient ternary-weight derivative of [Qwen3.8 27B](https://huggingface.co/Qwen/Qwen3.8-27B). Compressing the original 53.80 GB (FP16) model down to a mere 5.93 GB, this Apache 2.0-licensed model preserves an impressive 98.2% of its parent model's average performance across 20 benchmarks. Supporting text, images, and a 262K-token context window, Ternary Bonsai 2 27B can be deployed on a standard 16 GB laptop or a single 24 GB GPU using PrismML’s custom runtimes. --- ## 什么是 Ternary Bonsai 2 27B？ > ## What is Ternary Bonsai 2 27B? 该模型完整继承了 Qwen3.8 27B 的基础架构，总参数量达 273.6 亿 (27.36B) ，具体分布如下： > The model preserves the foundational architecture of the Qwen3.8 27B with its 27.36 billion parameters, distributed as follows: * **语言主干网络 (Language Backbone) ：** 243.5 亿 (24.35B) 参数 (采用混合注意力机制：约 75% 线性注意力层和 25% 全注意力层) ； * **嵌入层与语言模型头 (Embeddings & LM Head) ：** 25.4 亿 (2.54B) 参数； * **视觉塔 (Vision Tower) ：** 4.7 亿 (0.47B) 参数 (以独立的 0.63 GB GGUF 文件分发，仅在处理图像时载入) 。 > * **Language Backbone:** 24.35B parameters (utilizing hybrid attention: ~75% linear-attention and 25% full-attention layers). > * **Embeddings & LM Head:** 2.54B parameters. > * **Vision Tower:** 0.47B parameters (shipped as a separate 0.63 GB GGUF file, loaded only when processing images). 三值权重广泛应用于嵌入层、注意力投影矩阵、MLP 投影层以及语言模型头 (LM Head) 。全模型仅有 2620 万 (26.2M) 参数 (占比仅 0.0976%) 依然保留在较高精度，这些高精度参数主要集中在循环状态路径与归一化权重上。 > Ternary weights are applied across embeddings, attention projections, MLP projections, and the LM head. Only 26.2M parameters (0.0976%)—specifically the recurrent state path and normalization weights—remain in higher precision. ## 三值量化格式是如何工作的？ > ## How Does the Ternary Format Work? 该模型的权重被严格限制在三个离散值之间：**-1、0 或 +1**。 > The model utilizes weights restricted to three values: **-1, 0, or +1**. * **比特分配 (Bit Allocation) ：** 每 128 个权重组成一组，共享一个 FP16 缩放因子 (Scale) 。单个三进制数值携带 $\log_2(3)$ (约 1.585) 比特的信息量。结合缩放因子开销与保留的高精度张量，模型的实际运行开销为**每个权重 1.72 比特 (1.72 bits per weight)**。 * **打包方案 (Packings) ：** 为了在实际算子内核上获得极致性能，[技术白皮书](https://github.com/PrismML-Eng/Bonsai-demo/blob/main/bonsai-2-27b-whitepaper.pdf) 详述了两种 GGUF 打包格式： * `PTQ1_0`：以每个权重 1.76 比特的密度紧凑打包三进制位 (Trit) (模型总大小 5.93 GB) ； * `PQ2_0`：将每个三进制位存入一个 2 比特的槽位中 (模型总大小 7.25 GB) ，能够实现更迅速的解包速度。 * **基底旋转 (Rotated Basis) ：** 遵循 [SpinQuant](https://arxiv.org/abs/2405.16406) 的技术方案，PrismML 在三值分配前引入了分块阿达马旋转 (Hadamard Rotation，分块大小为 1,024) ，并在运行时对激活值应用相应的数学变换。 > * **Bit Allocation:** Every group of 128 weights shares a single FP16 scale. A ternary value carries $\log_2(3)$ (approx. 1.585) bits. Combined with the scale overhead and high-precision tensors, the model operates at **1.72 bits per weight**. > * **Packings:** To optimize performance on real kernels, the [whitepaper](https://github.com/PrismML-Eng/Bonsai-demo/blob/main/bonsai-2-27b-whitepaper.pdf) details two GGUF packings: > * `PTQ1_0`: Packs trits densely at 1.76 bits per weight (5.93 GB total size). > * `PQ2_0`: Stores each trit in a 2-bit slot (7.25 GB total size), offering faster unpacking. > * **Rotated Basis:** Following [SpinQuant](https://arxiv.org/abs/2405.16406) methodology, PrismML applies a blockwise Hadamard rotation (block size 1,024) before ternary assignment, with corresponding transformations applied to activations during runtime. ## 与 Qwen3.8 27B 的基准测试对比如何？ > ## How Does It Score Against Qwen3.8 27B? 评估测试由 PrismML 在思考模式下开展，测试环境基于 NVIDIA H100 GPU，并结合了 EvalScope 与 vLLM 推理框架： > Evaluations were conducted by PrismML in thinking mode using EvalScope and vLLM on H100 GPUs: | 能力维度 | Qwen3.6 27B | Qwen3.8 27B | Ternary Bonsai 2 27B | 性能保持率 | | :--- | :---: | :---: | :---: | :---: | | 知识与推理能力 | 84.71 | 86.66 | 83.95 | 96.9% | | 数学计算能力 | 94.64 | 97.06 | 96.57 | 99.5% | | 代码编写能力 | 82.57 | 82.17 | 81.58 | 99.3% | | 智能体与工具调用能力 | 80.05 | 79.74 | 77.57 | 97.3% | | 指令遵循能力 | 74.53 | 81.25 | 82.66 | 101.7% | | 视觉多模态能力 | 79.82 | 81.64 | 78.59 | 96.3% | | **综合评分 (20 项基准测试) ** | **83.6** | **85.4** | **83.9** | **98.2%** | > | Capability | Qwen3.6 27B | Qwen3.8 27B | Ternary Bonsai 2 27B | Retention | > | :--- | :---: | :---: | :---: | :---: | > | Knowledge and reasoning | 84.71 | 86.66 | 83.95 | 96.9% | > | Math | 94.64 | 97.06 | 96.57 | 99.5% | > | Coding | 82.57 | 82.17 | 81.58 | 99.3% | > | Agentic and tool calling | 80.05 | 79.74 | 77.57 | 97.3% | > | Instruction following | 74.53 | 81.25 | 82.66 | 101.7% | > | Vision | 79.82 | 81.64 | 78.59 | 96.3% | > | **Overall (20 benchmarks)** | **83.6** | **85.4** | **83.9** | **98.2%** | 相较于体积为 7.3 GB 的传统量化构建版本 (例如平均得分仅 75.2 的 `IQ2_XXS`) ，Bonsai 2 展现出了飞跃式的提升——在 AIME26 数学竞赛测试中取得 95.83 分 (对比后者的 78.6 分) ，在 LiveCodeBench v6 编程基准测试中取得 90.07 分 (对比后者的 70.05 分) 。 > Compared to conventional quantization builds like `IQ2_XXS` (averaging 75.2 at 7.3 GB), Bonsai 2 demonstrates dramatic improvements—scoring 95.83 on AIME26 (vs. 78.6) and 90.07 on LiveCodeBench v6 (vs. 70.05). ## 哪些能力依然存在质量损耗？ > ## Where Does It Still Lose Quality? 尽管 98.2% 的综合平均保留率非常优异，但在特定的长流程复杂任务中，模型性能的下滑依然相对明显： > While the 98.2% overall average is robust, quality drops are more pronounced in specific long-horizon workflows: * **智能体工作负载 (Agentic Workloads) ：** 在 [Terminal-Bench 2.1](https://www.tbench.ai/news/terminal-bench-2-1) 终端基准测试中，Bonsai 2 得分为 52.8 分 (Qwen3.8 原型为 69.7 分) ；在 [SWE-bench Verified](https://openai.com/index/introducing-swe-bench-verified/) 软件工程智能体基准测试中，其得分为 60.8 分 (对比全精度的 80.6 分，性能保持率约为 75%) 。 * **推理算力投入 (Reasoning Effort) ：** 中等思考预算 (Medium-effort) 下的平均得分为 79.3 分 (对比 FP16 基准的 82.6 分) ，且当前版本暂不支持低思考预算 (Low-effort) 执行。*(注：所有评估数据均源自 PrismML 官方公布，尚待第三方独立复现与验证。)* > * **Agentic Workloads:** On [Terminal-Bench 2.1](https://www.tbench.ai/news/terminal-bench-2-1), Bonsai 2 scores 52.8 compared to Qwen3.8's 69.7. On [SWE-bench Verified](https://openai.com/index/introducing-swe-bench-verified/), it scores 60.8 against 80.6 (~75% retention). > * **Reasoning Effort:** Medium-effort execution averages 79.3 (compared to 82.6 for the FP16 baseline), and low-effort execution is unsupported. *(Note: All evaluation data stems from PrismML and awaits independent verification.)* ## 在真实硬件上的运行速度如何？ > ## How Fast Is It on Real Hardware? 基于 PrismML 自研算子内核在批大小为 1 (Batch Size 1) 解码时的实测性能 (2026 年 9 月测试数据) ： > Benchmarked at batch size 1 decode on custom PrismML kernels (September 2026): * **硬件实测速度：** * **RTX 5090：** 142.5 Token/秒 (每生成一个 Token 仅耗电 0.582 毫瓦时/mWh) ； * **RTX 4090：** 96.7 Token/秒 (采用 `PTQ1_0` 格式) ； * **NVIDIA L4 (72W) ：** 32.1 Token/秒； * **Apple M5 Max：** 46.8 Token/秒； * **Apple M5 Pro：** 27.7 Token/秒。 * **打包格式效率对比：** `PTQ1_0` 在 Ada 架构显卡与 L4 上表现最佳；而在 Blackwell、Hopper、Ampere 以及 Apple 芯片架构上，`PQ2_0` 的运行速度更快。PrismML 同时表示，相较于全精度 8B 模型，该模型的整体能效比提升了 40%。 > * **Hardware Speeds:** > * **RTX 5090:** 142.5 tokens/sec (0.582 mWh per token). > * **RTX 4090:** 96.7 tokens/sec (using `PTQ1_0`). > * **NVIDIA L4 (72W):** 32.1 tokens/sec. > * **Apple M5 Max:** 46.8 tokens/sec. > * **Apple M5 Pro:** 27.7 tokens/sec. > * **Packing Efficiency:** `PTQ1_0` excels on Ada-generation cards and the L4, whereas `PQ2_0` is faster on Blackwell, Hopper, Ampere, and Apple silicon architectures. PrismML also claims a 40% improvement in energy efficiency over full-precision 8B models. ## 如何在本地运行？ > ## How Do You Run It? * **GGUF 格式运行：** 需要使用 PrismML 专门优化的 [llama.cpp 分支仓库](https://github.com/PrismML-Eng/llama.cpp) (原生 llama.cpp 目前无法识别 `PTQ1_0` 与 `PQ2_0` 格式) 。参照 [Bonsai-demo 代码仓库](https://github.com/PrismML-Eng/Bonsai-demo/) 的使用说明，运行 `./setup.sh` 脚本，随后执行 `./scripts/start_llama_server.sh`，即可在本地 `localhost:8080` 启动推理服务。 * **Apple Silicon 平台：** 可直接选用针对 Mac 优化的 [MLX 整合包](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-mlx-2bit) 及其附带的模型加载器。 * **浏览器端体验：** 可以访问 [WebGPU 在线演示](https://huggingface.co/spaces/webml-community/ternary-bonsai-2-webgpu-kernels) ，在现代网络浏览器中直接体验端侧免安装运行。 > * **GGUF Format:** Requires PrismML’s [llama.cpp fork](https://github.com/PrismML-Eng/llama.cpp) (stock llama.cpp rejects `PTQ1_0` and `PQ2_0` formats). Follow instructions in the [Bonsai-demo repo](https://github.com/PrismML-Eng/Bonsai-demo/) by running `./setup.sh` followed by `./scripts/start_llama_server.sh` to launch a local server at `localhost:8080`. > * **Apple Silicon:** Use the [MLX pack](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-mlx-2bit) with its bundled loader. > * **Browser:** Explore the [WebGPU demo](https://huggingface.co/spaces/webml-community/ternary-bonsai-2-webgpu-kernels) to run the model directly inside a web browser. ## 核心要点速览 > ## Key Takeaways * **极致压缩：** 模型最终体积仅为 5.93 GB，比 53.80 GB 的 FP16 全精度基准缩小了约 9.1 倍； * **极高保留率：** 在 20 项基准测试中平均得分达到 83.9 分，保留了原版 Qwen3.8 27B 模型 98.2% 的顶尖性能； * **极速推理：** 在 RTX 5090 显卡上能够达到每秒 142.5 Token，在 Apple M5 Max 芯片上可达到每秒 46.8 Token； * **长流程智能体局限：** 在 SWE-bench 等长跨度复杂任务中，性能保持率下降至全精度的约 75%； * **需专属运行时：** 当前运行必须依赖 PrismML 定制适配的 llama.cpp 分支或 MLX 运行时。 > * **Massive Reduction:** 5.93 GB model size, operating ~9.1x smaller than the 53.80 GB FP16 baseline. > * **High Retention:** Achieves an 83.9 average across 20 benchmarks, retaining 98.2% of the parent Qwen3.8 27B model's performance. > * **Blazing Fast:** Delivers 142.5 tokens per second on an RTX 5090 and 46.8 tokens per second on an M5 Max. > * **Agentic Limitations:** Long-horizon tasks like SWE-bench drop to roughly 75% retention compared to full precision. > * **Custom Runtimes Required:** Requires PrismML's specialized llama.cpp fork or MLX runtime to execute. ---

Linkup 发布 SPARSEUP：仅 149M 参数的开源最强稀疏嵌入模型

Sun, 20 Sep 2026 00:00:00 GMT

在信息检索与检索增强生成 (RAG) 领域，稠密向量模型虽然应用广泛，但在生僻专有名词匹配和可解释性上存在天然局限，而传统的稀疏模型又往往面临词表膨胀和高计算开销的挑战。为此，Linkup Research 正式发布了开源稀疏嵌入模型 (Sparse Embedding Model) —— **SPARSEUP**。该模型基于 149M 参数的 ModernBERT 架构打造，采用 Apache 2.0 许可证完全开源，在 BEIR-13 基准测试中斩获 56.4 的平均 nDCG@10 得分，成为目前 150M 参数以下公开可用的最强词表稀疏编码器。通过 Logit 平移、单位置 Top-k 截断和大小写折叠等三项精巧架构优化，SPARSEUP 巧妙破解了传统 SPLADE 模型停用词泛滥的难题，并结合 Seismic 倒排索引实现了约 380 微秒的单线程极速检索。这一成果不仅补齐了现代信息检索中轻量级稀疏范式的技术拼图，更为开发者构建高效率、强可解释性的混合检索系统提供了全新利器。 --- ## 核心概述 > ## Executive Summary Linkup Research 正式开源了基于机器学习的稀疏嵌入模型 (Sparse Embedding Model) —— **SPARSEUP**。该模型基于 149M 参数的 ModernBERT 骨干网络构建，并以宽松的 Apache 2.0 许可证公开发布。SPARSEUP 在知名的 BEIR-13 基准测试 (Benchmark) 中取得了 56.4 的平均 nDCG@10 得分，被 Linkup 定位为当前 150M 参数量级以下公开可用的最强基于词表的稀疏编码器。开发者可以通过 Hugging Face，借助 Transformers 或 Sentence Transformers 库轻松部署 SPARSEUP。该模型恰好补齐了此前 LightOn 模型套件的技术拼图，在保持极高检索效率与人类直观可读权重的特性的同时，为稠密检索 (Dense Retrieval) 和晚期交互检索 (Late-Interaction Retrieval) 提供了极具竞争力的稀疏检索替代方案。 > Linkup Research has released **SPARSEUP**, an open-source learned sparse embedding model built on a 149M-parameter ModernBERT backbone and distributed under the Apache 2.0 license. Achieving a 56.4 average nDCG@10 on BEIR-13, Linkup positions SPARSEUP as the strongest public vocabulary-based sparse encoder under 150M parameters. Deployable via Hugging Face using Transformers or Sentence Transformers, SPARSEUP bridges the gap in LightOn's recent model suite, offering a viable sparse alternative to dense and late-interaction retrieval styles while maintaining high efficiency and human-readable weights. --- ## 为什么需要稀疏模型？为什么是现在？ > ## Why a Sparse Model, and Why Now? 目前绝大多数开源检索模型都依赖于稠密表征 (Dense Representations)，即为每段文本生成一个固定维度的稠密向量。与此不同的是，稀疏模型直接在预定义的整个词表上输出权重分布，其中的每一个维度都精确对应一个真实的 Token。这种设计的精妙之处在于，生成的稀疏向量能够无缝对接到传统的倒排索引 (Inverted Indexes) 搜索引擎中；同时，由于每个激活维度都对应具体词汇，权重具备天然的人类可读性，并且在命中冷门、罕见的生僻词与专有名词时表现尤为强悍。 > Most open retrieval models rely on dense representations (one vector per text). In contrast, sparse models output weights over a vocabulary, where each dimension maps to a real token. This design allows vectors to integrate smoothly into inverted indexes while remaining human-readable and highly effective at matching rare words. SPARSEUP 的研发直接受到了 LightOn 近期发布 [DenseOn and LateOn](https://huggingface.co/papers/2607.27178) 的启发。LightOn 当时开源了全套训练数据、训练秘方 (Recipe)、一个稠密模型以及一个晚期交互模型。而 SPARSEUP 采用相同的骨干网络家族与微调 (Fine-Tuning) 数据，恰好补全了缺失的“稀疏模型”版块，从而让开发者能够站在完全平等的基准线上，横向对比这三种主流的信息检索范式。 > The development of SPARSEUP was motivated by LightOn’s release of [DenseOn and LateOn](https://huggingface.co/papers/2607.27178), which provided open data, a training recipe, a dense model, and a late-interaction model. SPARSEUP fills the missing sparse slot by utilizing the same backbone family and fine-tuning data, allowing developers to compare all three retrieval paradigms side by side. --- ## SPARSEUP 是如何构建的 > ## How SPARSEUP is Built SPARSEUP 的训练起点是 [LateOn-unsupervised](https://huggingface.co/lightonai/LateOn-unsupervised) 检查点，但该检查点并未包含掩码语言模型 (Masked Language Modeling, MLM) 头。Linkup 团队将 ModernBERT 原生的 MLM 预测头重新“嫁接”回模型中，并纯粹采用对比学习 (Contrastive Learning) 方法，基于 [LightOn 的微调混合数据集 (LightOn’s fine-tuning mixture)](https://huggingface.co/datasets/lightonai/embeddings-fine-tuning) 展开微调。 > Training begins from the [LateOn-unsupervised](https://huggingface.co/lightonai/LateOn-unsupervised) checkpoint, which lacked an MLM head. The Linkup team grafted ModernBERT’s original MLM head back onto the model and performed fine-tuning using [LightOn’s fine-tuning mixture](https://huggingface.co/datasets/lightonai/embeddings-fine-tuning) with contrastive learning exclusively. * **训练策略：** 每个查询 (Query) 都从 50 个候选池中采样配对 7 个难负例 (Hard Negatives)，同时结合批次内负例 (In-batch Negatives) 共同训练。整个训练流程摒弃了复杂的交叉编码器蒸馏 (Cross-Encoder Distillation)，单张 H100 GPU 即可高效完成训练。 * **突破原生 SPLADE 的局限：** 若在该骨干网络上直接应用标准的 SPLADE 方案，会导致输出表征过于稠密，充斥着大量无意义的停用词 (Stopwords)。Linkup 通过三项核心架构调整完美化解了这一难题： 1. **Logit 平移 (Logit shifting)：** 编码器通过计算 `log(1 + ReLU(x - 15))`，巧妙抵消了 ModernBERT MLM 原始 Logit 在初始化阶段过大从而导致对数函数饱和的问题。 2. **逐位置 Top-k 截断 (Per-position top-k)：** 每个输入的 Token 在进入最大池化 (Max Pooling) 之前，仅保留权重最高的前 12 个词表维度。这种设计限制了每个 Token 的词汇扩展幅度，而非死板地限制最终向量的总长度。 3. **大小写折叠 (Case folding)：** 将字节级 BPE (Byte-level BPE) 中形如 `heat`、`Heat`、`Ġheat` 和 `ĠHeat` 等同义变体统一折叠归并到单一 ID 下，并保留其中的最大权重，从而将词表输出维度从约 50k 显著压缩至约 34k。 > * **Training Strategy:** Each query is paired with 7 hard negatives sampled from a pool of 50, alongside in-batch negatives. The process excludes cross-encoder distillation and fits efficiently on a single H100 GPU. > * **Overcoming Vanilla SPLADE Limitations:** A standard SPLADE implementation on this backbone resulted in overly dense token bags saturated with stopwords. Linkup resolved this with three core architectural adjustments: > 1. **Logit shifting:** The encoder computes `log(1 + ReLU(x - 15))` to counteract ModernBERT MLM logits that otherwise saturate the log function at initialization. > 2. **Per-position top-k:** Each input token retains only its 12 strongest vocabulary dimensions prior to max pooling, capping expansion per token rather than total vector size. > 3. **Case folding:** Byte-level BPE variants like `heat`, `Heat`, `Ġheat`, and `ĠHeat` are folded onto a single ID while preserving the largest weight, reducing output dimensions from ~50k to ~34k. 在输入格式上，查询和文档分别添加 `[Q]` 与 `[D]` 前缀作为标识，检索得分通过点积 (Dot Product) 计算。在评估基准中，查询的最大长度设为 128 个 Token，文档的最大长度设为 512 个 Token。 > Queries and documents use `[Q]` and `[D]` prefixes respectively, scored via dot product. Evaluation max lengths are set to 128 tokens for queries and 512 for documents. --- ## 基准测试结果 > ## Benchmark Results 根据官方 [模型卡片 (Model Card)](https://huggingface.co/Linkup-Platform/linkup-sparseup-embed-v1) 披露的数据，在 BEIR-13 基准测试 (采用 nDCG@10 指标，且不包含 MS MARCO) 中，SPARSEUP 与其他主流稀疏编码器的对比表现如下： > Evaluated against other sparse encoders on BEIR-13 (nDCG@10, excluding MS MARCO) per the [model card](https://huggingface.co/Linkup-Platform/linkup-sparseup-embed-v1): | 模型 | BEIR-13 平均得分 | | :--- | :--- | | **SPARSEUP** | **56.4** | | [opensearch-neural-sparse-encoding-doc-v3-gte](https://huggingface.co/opensearch-project/opensearch-neural-sparse-encoding-doc-v3-gte) | 54.6 | | opensearch-neural-sparse-encoding-v1 | 52.44 | | ModernBERT-VT | 52.4 | | splade-v3 | 51.7 | | [granite-embedding-30m-sparse](https://huggingface.co/ibm-granite/granite-embedding-30m-sparse) | 50.6 | | LACONIC-1B *(10 亿参数，属于不同规模量级)* | 58.7 | > | Model | BEIR-13 avg | > | :--- | :--- | > | **SPARSEUP** | **56.4** | > | [opensearch-neural-sparse-encoding-doc-v3-gte](https://huggingface.co/opensearch-project/opensearch-neural-sparse-encoding-doc-v3-gte) | 54.6 | > | opensearch-neural-sparse-encoding-v1 | 52.44 | > | ModernBERT-VT | 52.4 | > | splade-v3 | 51.7 | > | [granite-embedding-30m-sparse](https://huggingface.co/ibm-granite/granite-embedding-30m-sparse) | 50.6 | > | LACONIC-1B *(1B parameters, different size class)* | 58.7 |

我是如何借助 AI 与 Lean 证明康威50年猜想的：Dan Abramov 的一月通关实录

Sat, 19 Sep 2026 00:00:00 GMT

本文记录了著名技术开发者 Dan Abramov 作为一名自称的“数学小白”，如何仅凭前沿大语言模型 (Claude 与 ChatGPT) 以及交互式定理证明工具 Lean 4，历时一个月攻克传奇数学家 John Conway 悬置整整 50 年的“全数整数细分猜想” (Omnific Integer Refinement Conjecture) 的奇幻实录。在经历初期 AI 严重胡言乱语、“黑话”满天飞与伪证明破产的泥潭后，作者开创性地搭建了包含项目经理、红队攻防、文献逆向与形式化编译的多智能体协同实验室架构。通过严格的代码隔离、定理反演与编译器内核硬校验，最终成功编译出了一份经 Lean 4 内核无额外公理检验通过的完整证明证书。本文兼具幽默坦诚的个人叙事与前沿形式化验证的深度思考，展现了生成式 AI 赋能前沿跨界探索的巨大潜能。 --- ## 核心概要 > # Summary 一名自称“数学小白”的业余爱好者，耗时整整一个月，完全凭借前沿大语言模型 (Large Language Model, LLM) —— Claude 与 ChatGPT，以及形式化定理证明工具 Lean 4，向数学界的公开未解难题发起了冲击：试图证明 John Conway 在 50 年前提出、关于超现实数 (Surreal Numbers) 的**全数整数细分猜想 (Omnific Integer Refinement Conjecture)**。尽管起初遭遇了严重的 AI 幻觉（充斥着大量虚构行话的“AI 垃圾内容”），作者随后精心打造了一套多智能体协同实验室工作流，结合严格的 Lean 4 形式化验证、独立代码文件沙箱隔离与层层递进的红队审计机制。最终，整套系统成功编译出了一份经 Lean 4 编译器内核完备校验的康威猜想证明证书，并向全球数学界同行公开邀请形式化证伪。 > An amateur mathematician ("math noob") spends a month using AI frontier models (Claude and ChatGPT) and the Lean theorem prover to attempt an open mathematical problem: proving John Conway’s 50-year-old **omnific integer refinement conjecture** regarding surreal numbers. Despite initial struggles with AI hallucinations ("slop" and made-up terminology), the author builds a multi-agent laboratory workflow combined with rigorous Lean formalization, independent file isolation, and step-by-step audits. Ultimately, the system successfully compiles a kernel-checked proof certificate of Conway's conjecture in Lean, inviting formal peer refutation. --- ## 我是如何“凭直觉与感觉”搞定康威猜想证明的 > ## How I Vibed a Proof of Conway’s Conjecture **日期：** 2026年9月18日 **作者：** Dan Abramov ([Ko-fi](https://ko-fi.com/gaearon) | [GitHub 源码仓库](https://github.com/gaearon/conway-refinement)) > **Date:** September 18, 2026 > **Author:** Dan Abramov ([Ko-fi](https://ko-fi.com/gaearon) | [GitHub Source](https://github.com/gaearon/conway-refinement)) 几个月前，AI 在数学领域取得突破的新闻开始频频登上各大媒体头条。[“搞个大突破” (Do a breakthrough)](https://www.theargumentmag.com/p/computer-do-a-breakthrough-no-mistakes) 甚至成了 Twitter 上的热门技术热梗。自然而然地，我也按捺不住好奇心：像我这样一个连高等微积分题解都要到处找答案的[数学小白](https://github.com/gaearon/analysis-solutions)，是不是也能随便找一个悬而未决的数学开放难题，然后让前沿模型帮我把它解出来？ > A few months ago, AI math results started making headlines. [“Do a breakthrough”](https://www.theargumentmag.com/p/computer-do-a-breakthrough-no-mistakes) became a Twitter meme. Naturally, I became curious whether I, too, a [math noob](https://github.com/gaearon/analysis-solutions), could find some open mathematical problem and then have a frontier model solve it. 这花光了我整整一个月的全部业余时间，并烧掉了[天文数字般的 Token](#how-many-tokens)；但我想，我已经成功拿到了由 Lean 4 严格检验通过的[数学证明](https://github.com/gaearon/conway-refinement)，攻克了 John Conway 在 50 年前留下的这一著名猜想： > It took me an entire month of my free time and a [boatload](#how-many-tokens) of tokens, but I believe I’ve obtained a Lean [proof](https://github.com/gaearon/conway-refinement) of this conjecture posed by John Conway 50 years ago: ![Conjecture: Omnific integers have a refinement property...](./images/conway.jpg) Conway 的细分猜想断言：全数整数具备优良的因数细分性质——如果四个全数整数满足 $ab = cd$，那么必然存在另外四个整数 $e, f, g, h$，使得 $a = ef$、$b = gh$、$c = eg$ 且 $d = fh$。 > Conway’s refinement conjecture claims that omnific integers have a refinement property: if *ab = cd*, there are integers *e*, *f*, *g*, *h* with *a = ef*, *b = gh*, *c = eg*, *d = fh*. 需要坦诚说明的是，我的证明目前**尚未**经过人类专业数学家的独立同行评审。不过，我掌握着[相当扎实的依据](https://github.com/gaearon/conway-refinement#why-i-think-its-correct)相信这一证明是坚实成立的，并且我非常真诚地欢迎任何人前来挑错和证伪。 > My proof has *not* been independently verified by mathematicians. However, I have [decent reasons](https://github.com/gaearon/conway-refinement#why-i-think-its-correct) to believe the proof is correct, and I genuinely invite a refutation. 目前，该证明已经完全通过了[来自 Palomar 注册表](https://palomar-registry.org/entry?id=PALOMAR-2026-09-03-000002&version=1)的严格机械化检查，几位精通 Lean 4 和该领域的专家也确认[形式化命题本身的陈述](https://github.com/gaearon/conway-refinement/blob/264445c93b78554c408e99e4e7f663693b4e91ab/ConwayRefinement/Standalone/Mathlib/InlineConwayRefinement.lean#L246-L257)完全准确。因此，只要我的证明代码没有意外触发 Lean 4 底层内核的致命 Bug，那么它大概率也是货真价实成立的。 > The proof has passed the mechanical checks [from the Palomar registry](https://palomar-registry.org/entry?id=PALOMAR-2026-09-03-000002&version=1), and a few people familiar with both Lean and the field said that [the statement](https://github.com/gaearon/conway-refinement/blob/264445c93b78554c408e99e4e7f663693b4e91ab/ConwayRefinement/Standalone/Mathlib/InlineConwayRefinement.lean#L246-L257) seems correct. So, assuming my proof doesn’t rely on a Lean kernel bug, it’s likely to be legit too. 在这篇长文中，我将详细记录我的探索路径，以及一路上所踩过的坑和收获的认知。 > In this post, I’ll describe my approach, and some things I learned along the way. --- ## 第一天 > ## First Day 我一直觉得，“在完全不懂数学本质的情况下把它解出来”这个念头既荒谬又离奇，但恰恰是这种荒诞感，让这件事显得格外诱人。 > I thought the idea of “solving” a math problem without understanding its substance is rather absurd, which of course made it all the more appealing. 不过，我可不想随随便便找个平庸的题做；我想要的是那种能够真正牵引我、让我心潮澎湃的问题。 > However, I didn’t just want *any* result; I wanted something that pulls me.

当智能体执行提交：认知可串行化保障复杂系统一致性

Sat, 19 Sep 2026 00:00:00 GMT

随着自主 AI 智能体 (AI Agent) 逐步接管复杂的生产与业务流程，它们不再仅仅是回答问题的交互助手，而是开始主动对系统状态执行写操作并调用各类外部服务。然而，智能体在漫长推理过程中所依赖的前提依据——包括数据库状态、检索知识、系统策略和委派权限——往往具有随机性且可能在后台被并发修改，使得传统数据库的事务隔离机制难以保证决策与提交动作之间的一致性。为了解决这一制约智能体安全落地的关键瓶颈，本文提出了“认知可串行化 (Cognitive Serializability) ”理论与 TCT 框架。该方案在数学上形式化了认知推导的一致性边界，并在工程实测中以仅 3.22 毫秒的极低额外开销拦截了全部并发异常，为构建高可靠、强一致的智能体自治系统奠定了坚实的理论与工程基石。 --- # 当智能体执行提交：认知可串行化保障复杂系统一致性 > # When AI Agents Commit: Cognitive Serializability Across Data, Evidence, Policy, and Authority **作者：** Jun He, Deying Yu **提交时间：** 2026年7月28日 **主要领域：** 人工智能 (`cs.AI`) **次要领域：** 分布式、并行与集群计算 (`cs.DC`) **arXiv 标识符：** [arXiv:2609.20261 [cs.AI]](https://arxiv.org/abs/2609.20261) **DOI：** [10.48550/arXiv.2609.20261](https://doi.org/10.48550/arXiv.2609.20261) > **Authors:** Jun He, Deying Yu > **Submitted:** 28 July 2026 > **Primary Subject:** Artificial Intelligence (`cs.AI`) > **Secondary Subjects:** Distributed, Parallel, and Cluster Computing (`cs.DC`) > **arXiv Identifier:** [arXiv:2609.20261 [cs.AI]](https://arxiv.org/abs/2609.20261) > **DOI:** [10.48550/arXiv.2609.20261](https://doi.org/10.48550/arXiv.2609.20261) --- ## 核心概述在现代分布式系统中，自主 AI 智能体往往需要结合各类输入来动态推导出对系统状态的修改动作——这些输入涵盖数据库读取、检索召回的背景证据、业务治理策略、模型自身的置信度以及外部委派的执行权限。然而，由于这些输入具有概率随机性，并且极易在智能体长时推理的“思考”期间在后台发生变动，传统的数据库隔离级别 (仅仅对最终提交的事务进行物理排序) 以及常规的智能体事务处理机制往往难以应对。除非系统契约能巨细靡遗地把所有推导前提都显式转化为谓词条件，否则它们根本无法保证智能体最终提交的修改动作与其推导依据在逻辑上达成统一且有效的严格一致。为了彻底解决这一痛点，本文正式提出了**认知可串行化 (Cognitive Serializability) **理论与一整套系统框架 (TCT) 。该框架在丝毫不牺牲系统吞吐与响应性能的前提下，能够为复杂智能体系统提供零误差的正确性保障以及确定的向前推进能力。 > ## Summary > > Autonomous AI agents dynamically derive state mutations from a wide range of inputs—including database reads, retrieved evidence, governance policies, internal beliefs, and delegated authority. Because these inputs are stochastic and often change while reasoning is actively in progress, traditional database isolation (which orders submitted transactions) and standard agentic transaction processing fall short. They fail to establish a unified valid point of agreement between a mutation and its underlying derivation inputs unless the contract explicitly represents all relevant predicates. > > This paper introduces **Cognitive Serializability** and a comprehensive framework (**TCT**) to guarantee zero-error soundness and positive progress in agentic systems without sacrificing performance. --- ## 核心概念与运行机制 * **类型化依赖 Token (Typed Dependency Tokens) ：** 清晰区分数据的“内容完整性”与规则的“通用适用性”。 * **可信中介机制 (Trusted Mediation) ：** 精准捕获并记录在智能体推理过程中所使用的全部确切数值。 * **严格认知可串行化 (Strict Cognitive Serializability) ：** 确保所有已提交的执行效果都遵循严格的串行顺序，并在逻辑上等价于所有暴露给推导过程的前提数值全程未发生任何改变。系统防护栅栏 (Fences) 将持续生效，直至运行时事件最终完成封装持久化域的封口确认。 * **效果兼容型认知准入 (Effect-Compatible Cognitive Admission，弱化版) ：** 依据并发持有的最新依赖向量与当前最新策略，对修改效果重新进行合规校验，从而巧妙避开了对原始随机推导过程进行强行串行化的巨大开销。 * **TCT 框架 (The TCT Framework) ：** 深度融合以下八大核心组件以强制执行系统一致性： 1. 不可变的版本化可执行定义 2. 注册表派生的授权计划 3. 密封信封机制 4. 守卫先行提交事务 5. 密封后与信封及见证绑定的授权授予 6. 协同提交凭据回执 7. 幂等的授权终结确认 8. 回执驱动的认知状态对齐 > ## Key Concepts & Mechanisms > > * **Typed Dependency Tokens:** Distinguish between content integrity and general applicability. > * **Trusted Mediation:** Captures the exact values exposed to an agent's reasoning process. > * **Cognitive Serializability (Strict):** Ensures that committed effects follow a strict serial order alongside a logical event where every value exposed to derivation remains entirely unchanged. Fences persist until the runtime event finalizes the sealed durability domain. > * **Effect-Compatible Cognitive Admission (Weaker):** Recertifies an effect against a concurrently held current dependency vector and current policy, bypassing the need to serialize the original stochastic derivation. > * **The TCT Framework:** Combines several core components to enforce consistency: > 1. Immutable versioned executable definitions > 2. Registry-derived authority plans > 3. Sealed envelopes > 4. Guard-first commit transactions > 5. Post-seal envelope- and witness-bound grants > 6. Co-committed receipts > 7. Idempotent grant finalization > 8. Receipt-driven epistemic reconciliation --- ## 评测与实验结果 * **理论完备性保障 (Theoretical Guarantees) ：** 通过构建完整的注册轨迹足迹与单一的增长阶段，该架构成功在本地守卫与不兼容的外部资源预留之间建立了无环锁点顺序，严格满足了精确的可串行化条件与观测等价性边界。 * **实证性能表现 (Empirical Performance) ：** 团队构建了专门的证伪测试套件对系统实现规范进行了高强度检验。实验原型成功拦截了**全部注入的异常与并发冲突**，同时带来的平均提交额外开销仅有 **3.22 毫秒**，几乎可以忽略不计。 > ## Evaluation & Results > > * **Theoretical Guarantees:** Complete registered footprints and a single growing phase successfully induce an acyclic lock-point order over local guards and incompatible external reservations, satisfying precise serializability conditions and observational-equivalence boundaries. > * **Empirical Performance:** A dedicated falsification suite tested implementation obligations. The prototype successfully prevented **all injected anomalies** while incurring a negligible mean commit overhead of only **3.22 ms**. --- ## 资源与论文获取 * **全文链接 (Full-Text Links) ：** [查看 PDF](https://arxiv.org/pdf/2609.20261) | [实验性 HTML 网页](https://arxiv.org/html/2609.20261v1) | [TeX 源码](https://arxiv.org/src/2609.20261) * **授权协议 (License) ：** [知识共享署名 4.0 国际许可协议 (Creative Commons Attribution 4.0) ](http://creativecommons.org/licenses/by/4.0/) > ## Access & Resources > > * **Full-Text Links:** [View PDF](https://arxiv.org/pdf/2609.20261) | [Experimental HTML](https://arxiv.org/html/2609.20261v1) | [TeX Source](https://arxiv.org/src/2609.20261) > * **License:** [Creative Commons Attribution 4.0](http://creativecommons.org/licenses/by/4.0/)

面向 MIP* = RE 核心定理的长程自动化形式化证明

Sat, 19 Sep 2026 00:00:00 GMT

在理论计算机科学与量子物理的交叉领域，著名的 $\text{MIP}^* = \text{RE}$ 定理证明了多证明者量子纠缠交互式证明系统等价于所有递归可枚举语言，从根本上颠覆了人们对可计算性与量子纠缠的认知，其证明过程极其宏大繁琐，传统纯人工的形式化检验几乎需要顶尖专家团队耗费数年心血。为了攻克长程数学证明自动化中的“命题漂移”与“证明拼接”等关键瓶颈，本文研究团队创新性地提出了 **FormalFlow** 系统，巧妙引入现代软件工程开发思想，依托全局共享蓝图引导多个 AI 证明智能体 (AI proving agents) 在人类监督下开展嵌套规划、机器证明与严谨代码审查。借助该系统，团队历时 63 天成功在 Lean 4 中完成了 $\text{MIP}^* = \text{RE}$ 底层核心定理——“经典低单变量度数测试的量子可靠性”的完全形式化，产出超过 12.6 万行完全由 AI 智能体生成的代码，并在验证中精准排查与修复了原论文手写证明中存在的隐蔽缺陷与边界条件。这项突破不仅为深奥的量子复杂性理论构筑了经计算机验证的坚实基石，更生动展示出小规模科研团队借助自主 AI 智能体低成本、高质量验证重大科学猜想与顶尖数学证明的广阔前景。 --- # 面向 MIP* = RE 核心定理的长程自动化形式化证明 > # Long-Horizon Autoformalization of a Core Theorem Underlying MIP* = RE ## 核心概要 > ## Summary 里程碑式的数学形式化验证往往需要顶尖专家团队投入数年心血才能完成。本文提出了 **FormalFlow** 系统，在人类专家的统筹监督下协同多个 AI 证明智能体 (AI proving agents) ，共同攻克长程形式化验证 (long-horizon formalization) 中尤为棘手的命题漂移 (statement drift) 与证明拼接 (proof composition) 等核心挑战。FormalFlow 汲取了现代软件工程的研发思想，依托一份全局共享的蓝图 (shared blueprint) 来引导“规划—证明—评审”的嵌套闭环，并由智能体在全流程中持续强化严格的形式化验证。 > Landmark mathematical formalizations typically require specialist teams years of dedicated work to complete. This paper introduces **FormalFlow**, a system that coordinates AI proving agents under human supervision to overcome key challenges in long-horizon formalization, such as statement drift and proof composition. Inspired by software engineering principles, FormalFlow utilizes a shared blueprint to steer nested planning, proving, and review loops, with agents continuously reinforcing verification. 借助该系统，作者团队成功完成了经典低单变量度数测试 (classical low individual-degree test) 量子可靠性 (quantum soundness) 在 Lean 4 中的机器检验形式化证明——该定理正是著名的量子计算里程碑成果 $\text{MIP}^* = \text{RE}$ 的底层基石。整个形式化开发历时 63 天完成 (若引入更大规模的并行机制，这一周期还可进一步大幅压缩) ，最终产出的代码库包含 126,367 行完全由智能体自主生成的 Lean 代码。在严苛的机器检验过程中，系统成功识别并修正了原手写论文中存在的附加约束条件漏洞与中间推演谬误，同时在修正后的假设前提下依然完好保持了已发表论文中的最终误差界。这项突破不仅为量子复杂性理论 (quantum complexity theory) 奠定了经计算机完全验证的坚实基石，更展示出了一条极具扩展潜力的全新路径，使小规模科研团队也有能力以极高的性价比对重大前沿研究证明开展严格的形式化验证。 > Using this system, the authors completed a machine-checked Lean 4 proof for the quantum soundness of the classical low individual-degree test—a foundational theorem underlying the celebrated $\text{MIP}^* = \text{RE}$ result. Developed over 63 days (a timeline that could be further compressed through parallelism), the resulting library contains 126,367 lines of entirely agent-generated Lean code. The formalization process successfully identified and corrected side conditions and intermediate errors while maintaining the published final error bound. This achievement establishes a verified foundation for quantum complexity theory and demonstrates a scalable path toward the affordable verification of major research proofs by small teams. --- ## 文档元数据 > ## Document Metadata * **arXiv 编号：** [arXiv:2609.19814](https://arxiv.org/abs/2609.19814) * **主学科分类：** 量子物理 (Quantum Physics, `quant-ph`) * **次学科分类：** 人工智能 (Artificial Intelligence, `cs.AI`) 、计算机科学中的逻辑 (Logic in Computer Science, `cs.LO`) * **MSC 数学分类：** 68V20, 68V15, 81P68, 68Q15 * **提交日期：** 2026 年 9 月 17 日 * **论文作者：** Sirui Lu, Ruixuan Deng, Yanqiao Zhu, Zhengfeng Ji > * **arXiv ID:** [arXiv:2609.19814](https://arxiv.org/abs/2609.19814) > * **Primary Subject:** Quantum Physics (`quant-ph`) > * **Secondary Subjects:** Artificial Intelligence (`cs.AI`), Logic in Computer Science (`cs.LO`) > * **MSC Classes:** 68V20, 68V15, 81P68, 68Q15 > * **Submission Date:** September 17, 2026 > * **Authors:** Sirui Lu, Ruixuan Deng, Yanqiao Zhu, Zhengfeng Ji --- ## 论文摘要 > ## Abstract 以往，完成里程碑式的数学形式化验证往往需要顶尖专家团队耗费数年时间。我们提出了 FormalFlow 系统，在人类专家的监督下协同多个 AI 证明智能体 (AI proving agents) ，有效解决长程形式化过程中面临的命题漂移与证明拼接两大核心难题。该系统借鉴现代软件工程的原理与实践，利用一份全局共享的蓝图指导规划、证明与评审的嵌套闭环；智能体在整个形式化生命周期中持续强化形式验证与严格审查。基于该系统，我们成功完成了经典低单变量度数测试量子可靠性的机器检验 Lean 4 形式化证明，该定理正是支撑著名成果 $\text{MIP}^* = \text{RE}$ 的核心基石之一。整个证明开发历时 63 天；若借助更高并发度的并行机制，研发周期有望进一步缩短。最终生成的证明代码库包含 126,367 行 Lean 代码，且全部由智能体自主生成。在严格的形式化过程中，系统成功修正了原论文中的附加约束条件与中间推演谬误，同时在修正后的假设下依然完好保持了此前已发表的最终误差界。本项工作不仅为量子复杂性理论建立了经机器全面验证的严谨基石，更展示出了一条切实可行的技术路径，使小规模研究团队也能以低成本实现重大前沿科研证明的形式化验证。 > Landmark mathematical formalizations have taken specialist teams years to complete. We present FormalFlow, a system that coordinates AI proving agents under human supervision to address statement drift and proof composition in long-horizon formalization. Drawing on software engineering principles and practices, it uses a shared blueprint to guide nested planning, proving and review loops. Agents strengthen verification and review throughout formalization. We completed a machine-checked Lean 4 proof of the quantum soundness of the classical low individual-degree test, a core theorem underlying $\text{MIP}^* = \text{RE}$. Developing the proof took 63 days; greater parallelism could further reduce this time. The final library contains 126,367 lines of Lean code, all generated by agents. The formalization corrects side conditions and intermediate errors while preserving the published final error bound under corrected assumptions. This work provides a verified foundation for quantum complexity and demonstrates a route to affordable verification of major research proofs by small teams. --- ## 补充信息与资源 > ## Supplementary Information & Resources * **论文篇幅：** 全文共 72 页 (正文共 13 页，包含 4 幅图和 1 张表；补充附录共 57 页，包含 9 幅图和 17 张表；以及参考文献) 。 * **Lean 4 代码仓库：** [GitHub - LionSR/MIPStarRE](https://github.com/LionSR/MIPStarRE) * **获取全文：** * [查看 PDF](https://arxiv.org/pdf/2609.19814) * [HTML 在线版本 (实验性) ](https://arxiv.org/html/2609.19814v1) * [TeX 源码](https://arxiv.org/src/2609.19814) > * **Paper Length:** 72 pages total (13-page main text featuring 4 figures and 1 table; 57-page supplementary appendices featuring 9 figures and 17 tables; references). > * **Lean 4 Code Repository:** [GitHub - LionSR/MIPStarRE](https://github.com/LionSR/MIPStarRE) > * **Access Full-Text:** > * [View PDF](https://arxiv.org/pdf/2609.19814) > * [HTML Version (Experimental)](https://arxiv.org/html/2609.19814v1) > * [TeX Source](https://arxiv.org/src/2609.19814)

AI 智能体真的懂计算机体系结构吗？

Sat, 19 Sep 2026 00:00:00 GMT

随着人工智能技术深入底层硬件领域，AI 智能体 (AI Agent) 越来越多地被委以设计和优化硬件加速器的重任，并展现出令人瞩目的性能提升，但学界一直存在一个根本性疑问：AI 究竟是在真正理解并推理计算机体系结构，还是仅仅在多维参数空间中进行高效的盲目黑盒搜索？为了拨开这一迷雾，研究团队推出了全新的 AutoTuring 基准测试 (Benchmark) 框架，让同一个 AI 智能体在面对完全相同的 15 维硬件设计空间时，分别在赋予明确物理语义的“知情架构师”和抹去所有物理含义的“盲盒搜索者”两种不同设定下执行优化任务。评测结果揭示，掌握体系结构物理知识能让知情智能体在 FP16 GEMM 算子优化中相比盲盒智能体平均取得 12.3% 的性能提升，并大幅节省 70.1% 的底层硬件仿真器调用开销；然而，引入结构化的审校反思回路 (Critic Loop) 却能帮助盲盒智能体抹平绝大部分差距，表明体系结构先验认知与结构化反思机制在优化效能上表现出奇妙的相互替代特性。这项工作首次在保持问题本质与解空间完全一致的前提下，量化了“理解物理意义”对 AI 硬件设计的真实价值，为未来芯片自动化设计以及专用大语言模型 (Large Language Model, LLM) 智能体的协同演化提供了兼具洞察力与方法论意义的重要参考。 --- ## 内容概要 > ## Summary 随着 AI 智能体越来越多地参与硬件加速器的设计与优化，并在实践中屡获佳绩，一个根本性的疑问始终悬而未决：**AI 智能体究竟是在真正理解并推理计算机体系结构，还是仅仅在庞大的参数空间中进行盲目的黑盒搜索？** > As AI agents are increasingly used—and praised—for designing and optimizing hardware accelerators, a fundamental question remains: **Are they truly reasoning about computer architecture, or are they just blindly searching over parameter spaces?** 为了探寻真相，研究人员打造了名为 **AutoTuring** 的评估框架，让同一个 AI 智能体在两种截然不同的表述设定下，探索完全相同的 15 维硬件加速器设计空间： > To find out, researchers introduced **AutoTuring**, an evaluation framework that tests AI agents using the exact same 15-dimensional accelerator space under two different framings: 1. **知情设定 (Informed Framing)**：向智能体清晰展示具有明确物理含义的具名硬件架构旋钮，并配备详尽的硬件仿真器性能计数器反馈。 2. **盲盒设定 (Blind Framing)**：抹去所有物理背景与硬件语义，仅将参数作为被约束在 $[0,1]$ 区间内的匿名抽象变量呈现给智能体。 > 1. **Informed Framing:** Presented as named architectural knobs equipped with simulator counters. > 2. **Blind Framing:** Presented as anonymous variables constrained to $[0,1]$. 通过将评估器、合法参数空间以及理论可达的最优解完全锁定为同一标准，实验中唯一的变量就是这些配置参数对智能体而言是否具备实际的“物理与业务含义”。研究结果揭示，计算机体系结构的专业知识确实能带来显著回报——知情智能体的优化结果平均超越了模拟的 H200 基线 5.4%，相比盲盒状态下的智能体更是大幅领先 12.3%，同时所需的底层仿真器调用次数锐减了 70.1%。然而，这种优势并非不可替代：为盲盒智能体引入一套结构化的审校反思回路，能够帮助其追平绝大部分的性能差距；但同样的审校回路却几乎无法为知情智能体带来额外提升。这一现象生动地表明，体系结构领域知识与结构化反思机制在硬件优化中更多地表现为相互替代的关系，而非相互叠加的互补关系。 > By keeping the evaluator, legal space, and reachable optima identical, the only variable is whether the problem "means" anything to the agent. The findings reveal that architectural knowledge pays off—the informed agent beats a modeled H200 baseline by 5.4% and the blind agent by 12.3% on average, while requiring 70.1% fewer simulator calls. However, this advantage isn't exclusive: a structured critic loop helps the blind agent recover most of that performance gap while offering little benefit to the informed agent, suggesting that architectural knowledge and structured critique function as substitutes rather than complements. --- ## 论文元数据 > ## Paper Metadata * **arXiv 标识符：** [arXiv:2609.19387](https://arxiv.org/abs/2609.19387) [cs.AI] * **学科领域：** 人工智能 (`cs.AI`) ；硬件架构 (`cs.AR`) * **ACM 分类：** C.1.3；I.2.8；B.8.2 * **提交时间：** 2026 年 9 月 16 日 * **论文作者：** Ambika Sharan, Grigory Chirkov, Soheil Abbasloo > * **arXiv Identifier:** [arXiv:2609.19387](https://arxiv.org/abs/2609.19387) [cs.AI] > * **Subjects:** Artificial Intelligence (`cs.AI`); Hardware Architecture (`cs.AR`) > * **ACM Classes:** C.1.3; I.2.8; B.8.2 > * **Submitted on:** September 16, 2026 > * **Authors:** Ambika Sharan, Grigory Chirkov, Soheil Abbasloo --- ## 论文摘要 > ## Abstract 如今，AI 智能体越来越多地被委以芯片硬件设计的重任，且不断有报告指出它们取得了优异的优化成果。然而，这些报告固然证明了硬件设计得到了实质性改进，却始终无法解释改进背后的深层原因。一个成功优化了加速器的智能体，究竟是在深入理解并推理底层硬件机器的运行机理，还是仅仅在完全不理解参数实际物理意义的前提下展现了出色的数值搜索能力？唯有前者所代表的深度认知能力，才能够真正迁移到未来的下一代全新硬件架构中。现有的基准测试往往只更换不同的智能体，却保持问题的呈现形式固定不变，因此根本无法将这两种能力区分开来。本研究采取了截然相反的评估范式：AutoTuring 将同一个 15 维加速器设计空间分两次交给同一个智能体——一次呈现为附带仿真器计数器的具名硬件架构旋钮，另一次则呈现为限制在 $[0,1]$ 区间内的纯匿名数学变量；在此过程中，性能评估器、合法搜索空间以及可达的最优点完全保持一致，唯一的差异就是这道设计难题对智能体而言是否具备真实的“物理含义”。两种设定下的性能差距，正是衡量其体系结构理解力的标尺。在包含 9 个核心算子的半精度通用矩阵乘法 (FP16 GEMM) 任务组评测中，理解物理意义带来了实打实的回报：知情架构师智能体相比模拟的 H200 基准提升了 5.4%，相比盲盒状态下的自己平均领先 12.3%，同时仿真器调用次数大幅减少了 70.1%。然而，这种回报并非不可替代：引入结构化审校回路能够帮助盲盒智能体追平绝大部分性能差距，却无法给知情架构师带来任何额外收益；这表明硬件体系结构知识与结构化反思回路在优化中更多充当了相互替代的角色，而非锦上添花的互补关系。我们在此汇报这些在单款模拟加速器上经每种条件 5 至 6 次运行所得出的初步发现，并强调本研究的核心贡献在于这一创新性的对比评估方法本身，而非具体的某款加速器硬件设计。 > Agents are increasingly asked to design hardware, and increasingly reported to succeed. Such reports establish that a design improved; they cannot establish why. An agent that improves an accelerator may be reasoning about the machine, or may be searching competently over knobs whose meaning it never recovers -- and only the first transfers to the next architecture. Existing evaluations cannot tell the two apart, because they vary the agent while holding the framing of the problem fixed. We do the opposite. AutoTuring hands the same agent the same 15-dimensional accelerator space twice: once as named architectural knobs with simulator counters, once as anonymous variables on [0,1], with the evaluator, the legal space and the reachable optima held identical, so that the only thing that varies is whether the problem means anything. The gap between the two is the measurement. On a nine-kernel FP16 GEMM basket, meaning pays: the architect beats a modeled H200 by 5.4% and its blind counterpart by 12.3% on average, with 70.1% fewer simulator calls. It does not pay uniquely: a critic loop recovers most of that gap for the blind agent and buys the architect nothing, so architectural knowledge and structured critique behave as substitutes rather than as complements. We report these as preliminary findings -- five to six runs per condition on a single modeled accelerator -- and take the comparison itself, not the accelerator, to be the contribution. --- ## 相关链接与资源 > ## Links & Resources * **全文访问：** [查看 PDF](https://arxiv.org/pdf/2609.19387) | [HTML 版本](https://arxiv.org/html/2609.19387v1) | [TeX 源码](https://arxiv.org/src/2609.19387) * **引用与指标：** [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.19387) | [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.19387) | [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.19387) > * **Full-Text Access:** [View PDF](https://arxiv.org/pdf/2609.19387) | [HTML Version](https://arxiv.org/html/2609.19387v1) | [TeX Source](https://arxiv.org/src/2609.19387) > * **Citations & Metrics:** [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.19387) | [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.19387) | [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.19387)

重构 Packfile：在对象存储上直接运行 Git 的架构实践

Fri, 18 Sep 2026 00:00:00 GMT

将 Git 存储服务直接构建在低成本、高弹性的云端对象存储 (例如 AWS S3 或 Tigris) 之上，是许多团队探索云原生存储架构与降低运维开销的理想方案。然而，标准 Git 的 Packfile 打包机制是为本地磁盘设计的，严重依赖微秒级的内核内存映射 `mmap` 与高并发随机寻道；一旦搬到存在数十毫秒网络往返延迟的对象存储上，性能便会迅速恶化至不可用的状态。为了彻底攻克这一难题，开发者借鉴了经典 CD-ROM 光盘中 `.bin`/`.cue` 音轨索引的分离设计，提出了一种专为对象存储定制的列式 Packfile 格式，并巧妙利用精准的 HTTP Range 范围请求与异步预取机制。这一架构突破将 S3 API 请求量削减了数个数量级，在生产级基准测试中使代码推送与克隆提速高达 14.6 倍，为现代分布式版本控制与云端底层系统架构开辟了全新路径。 --- ## 核心概要 > ## Summary 构建一个以对象存储 (Object Storage) (例如 Tigris) 为底座的 Git 服务器，听起来似乎很直观：只需搭建一层文件系统转译层，让 Git 能够操作对象存储即可。然而，标准的 Git Packfile 是为本地磁盘量身设计的，高度依赖操作系统内核的 `mmap` 内存映射以及低延迟的本地文件系统极速读取。一旦将这些读取操作映射为跨越网络的对象存储往返，面对生产环境级别的大型仓库时，这种方案的扩展性便会彻底崩溃。 > Building a Git server backed by object storage (like Tigris) sounds straightforward: use a filesystem translation layer so Git can speak object storage. However, standard Git packfiles are designed for local disks and rely heavily on the kernel's `mmap` and fast, low-latency filesystem reads. When translated over network round-trips to object storage, this approach fails to scale for production-sized repositories. 为了解决这个问题，作者从老式 CD-ROM 光盘的 `.bin`/`.cue` 索引单中汲取灵感，发明了一种专为对象存储原生打造的全新 Packfile 格式。通过构建一个带有二进制索引文件 (`objects.cue`) 的列式存储结构，索引中同时记录了精确的文件偏移量以及压缩与解压后的双重尺寸，客户端得以发起高效的 HTTP Range 范围请求。这一专属设计不仅大幅削减了访问 S3/对象存储的请求次数，更将代码推送 (push) 和克隆 (clone) 的性能最高提升了整整 14.6 倍。 > To solve this, the author invented an object-storage-native packfile format inspired by CD-ROM `.bin`/`.cue` sheets. By creating a columnar store with a binary index (`objects.cue`) containing both compressed and uncompressed sizes alongside precise file offsets, clients can execute efficient HTTP Range requests. This custom approach drastically reduced S3/object storage requests and accelerated push/clone performance by up to 14.6x. --- ## Git 到底是什么？不过是一堆可怜的零散对象！ > ## What is Git? A Miserable Little Pile of Objects! 当你提交一次代码时，Git 会把你的改动作为经过压缩的、基于内容寻址的对象 (Content-Addressed Objects) 保存在 `.git` 目录中 (本文称之为“dotgit”)。 > When you make a commit, Git stores your changes as compressed, content-addressed objects inside the `.git` directory (referred to here as "dotgit"). ```bash $ mkdir ~/tmp/gitexample $ git init && git branch -m main $ echo "Hello, blog!" >> hello.txt $ git add . $ git commit -sm "chore: initial commit" ``` 这一操作会在底层生成一片由裸对象和具名引用构成的“对象之海”： > This produces a sea of bare objects and named references:

图 01: 对象的海洋与指向它们的引用命名

  .git/objects/                    refs/heads/main                                     │  ├── 1c/7a26a901..ec7966  ─────▶  commit 1c7a26a  │                                  │  ├── 8e/67afbb2e..857bd3  ─────▶    tree 8e67afb  │                                  │  hello.txt  └── 9c/c9867337..09fe26  ─────▶    blob 9cc9867                                          "Hello, blog!"   the filename is the sha1 of the bytes in the file, so the same  content is always, everywhere, the very same object

读取其中的某个对象需要对其进行解压： > Reading an object requires decompressing it: ```bash $ file .git/objects/**/* | grep -v directory .git/objects/1c/7a26a901724b4ce766655ac387413fb9ec7966: zlib compressed data .git/objects/8e/67afbb2ee6bdcbb79061dfdfb93febce857bd3: zlib compressed data .git/objects/9c/c9867337c2ebae85ba2350f901e0bcc209fe26: zlib compressed data $ python3 -c "import sys, zlib; sys.stdout.buffer.write(zlib.decompress(sys.stdin.buffer.read()))" < .git/objects/9c/c9867337c2ebae85ba2350f901e0bcc209fe26 blob 13Hello, blog! ```

突破 1.58 比特极限：三值大语言模型的极致压缩与加速

Fri, 18 Sep 2026 00:00:00 GMT

近年来，三值大语言模型 (Ternary Large Language Models, LLMs) 因极高的计算效率备受关注，其每个权重仅取 $\{-1, 0, +1\}$ 三个离散值，理论极限存储成本约为 1.585 比特。但在传统工程实践中，为了便于硬件以 2 的幂次方寻址，工程师普遍假设三种符号等概率分布，并采用“5 个三值权重打包进 1 字节”的折中方案，使实际开销反弹至 1.625 比特。研究人员系统分析了 29 个真实三值大模型，敏锐地发现权重中数字“0”的实际占比最高可达 51.5%。基于这一非均匀分布特性，本文提出了自适应分布布局方案 BITCOS，将稠密存在位图与紧凑符号向量协同编码，在 26 个模型中全面超越了标准打包方案，最低仅需 1.485 比特即可存储一个权重，一举打破了 1.58 比特的传统认知壁垒。更为重要的是，BITCOS 深度适配了现代 CPU 与 GPU 的向量指令集（如 AVX-512、AVX2 与 Intel Xe2），不仅大幅节省了内存与带宽，更在端到端解码中带来了高达 1.18 倍至 1.27 倍的吞吐量提升，为超低比特大语言模型在边缘设备和数据中心的高效落地扫清了关键障碍。 --- # 突破 1.58 比特极限：三值大语言模型的极致压缩与加速 > # Breaking the 1.58-bit Barrier for Ternary LLMs ## 论文概要 > ## Summary * **作者 (Authors) ：** Evangelos Georganas, Alexander Heinecke, Pradeep Dubey * **提交时间 (Submitted) ：** 2026 年 9 月 14 日 * **主要领域 (Primary Subject) ：** 人工智能 (`cs.AI`) / 机器学习 (`cs.LG`) * **arXiv 编号 (arXiv ID) ：** [2609.16338](https://arxiv.org/abs/2609.16338) > * **Authors:** Evangelos Georganas, Alexander Heinecke, Pradeep Dubey > * **Submitted:** September 14, 2026 > * **Primary Subject:** Artificial Intelligence (`cs.AI`) / Machine Learning (`cs.LG`) > * **arXiv ID:** [2609.16338](https://arxiv.org/abs/2609.16338)

直接问工具，别瞎猜：智能体工具调用自带运行进度，推理系统理应主动读取

Fri, 18 Sep 2026 00:00:00 GMT

在基于大语言模型 (Large Language Model, LLM) 的 AI 智能体 (AI Agent) 工作流中，调用外部工具往往需要耗费大量物理等待时间，在此期间请求的键值缓存 (KV Cache) 会持续占用极其宝贵的 GPU 显存。当前的大模型推理服务系统大多依赖“盲目猜测”来管理这些缓存（例如仅凭工具名称、历史耗时或调用前预估来决定是否换出显存），但这类静态估算在多变的真实场景下根本无法准确预测工具的运行时间。针对这一瓶颈，清华大学与合作团队提出了一项全新设计理念：正在运行的工具本身其实掌握着精确的内部进度，只是这些信号此前被智能体技术栈层层屏蔽了。作者团队提出让工具调用在运行时主动向推理系统报告进度，并设计了一套轻量级捕获机制。实验表明，该方案在 KV Cache 置换决策点的准确度相比现有最优预测器提升了数倍乃至一个数量级，在生产级推理引擎中成功将工具调用后的 p90 首字延迟 (Time-to-First-Token, TTFT) 降低超过 20%，性能逼近理想全知调度器 (Oracle)。 --- ## 总结 > ## Summary AI 智能体请求在执行过程中，往往需要花费大量的物理时间等待外部工具完成计算。在此期间，该请求对应的键值缓存 (KV Cache) 只能无所事事地滞留在寸土寸金的 GPU 显存当中。目前主流的大语言模型推理系统主要靠“盲猜”来决定如何管理这部分缓存——例如根据工具名称、历史执行耗时、调用前声明的预估时长，或是推理引擎自身的负载情况来进行判断。然而，任何在工具开始运行前就固定下来的静态估算，从根本上都无法准确预测工具的实际耗时，甚至连不同工具调用的耗时先后顺序都排不准。 > Agentic requests spend substantial wall-clock time waiting for external tools to execute, during which their KV cache unnecessarily occupies valuable GPU memory. Current LLM serving systems rely on guesswork to manage this cache (such as evaluating tool names, historical execution times, pre-call duration declarations, or engine occupancy), but pre-call estimates fundamentally fail to accurately predict or even rank tool durations. 这篇论文揭示了一个被大家忽视的真相：正在运行的工具自身其实已经掌握了必要的进度数据，只是这些关键信息被智能体软件栈连同工具封装给无声地“静音”了。为此，作者团队提出了一种全新的协作机制，让工具调用在执行过程中能够主动、明确地对外汇报其实时运行进度。 > This paper reveals that running tools already possess the necessary progress data, but are silenced by the agent stack. The authors propose a mechanism where tool calls explicitly report their progress during execution.

元认知引导：学习科学判断的内在结构

Fri, 18 Sep 2026 00:00:00 GMT

真正的科学探索绝非单向线性的推导答题，而是一个需要根据涌现的新证据，在“广泛假设探索”、“严格规程执行”与“批判性反思审视”之间灵活切换的长期动态过程。然而，当前的主流大语言模型 (Large Language Models, LLMs) 大多只针对最终输出结果进行监督优化，在面对需要敏锐科学直觉与多变策略掌控的长周期科研任务时往往难以胜任。为此，本篇论文提出了 **元认知引导 (Metacognitive Steering) ** 技术。研究团队深入挖掘了顶尖科学家在万亿参数混合专家 (MoE) 模型 Kimi 2.6 上的交互轨迹，成功捕捉并定位了支配科学判断的低维控制空间，进而在不改变模型既有参数的前提下，仅在推理阶段即可动态对特定神经网络层施加干预与调控。这一方法在自主科研系统 **Columbus-1** 中得到了实战检验，不仅独立发现了 Linux 蓝牙协议栈 BlueZ 中的 8 个严重安全漏洞，更成功主导了一枚 10 英尺高固体反推着陆火箭的设计与制造。该研究证明：对过程层面的科学判断进行建模，能够赋予前沿大语言模型可解释、高度动态且收放自如的科研推理决策能力。 --- # 元认知引导：学习科学判断的内在结构 > # Metacognitive Steering: Learning the Structure of Scientific Judgment **arXiv 编号 (arXiv ID) ：** [2609.16245](https://arxiv.org/abs/2609.16245) [cs.AI] **提交时间 (Submitted) ：** 2026 年 9 月 14 日 **作者 (Authors) ：** Vincent Karpf, Joseph Reth, Eike Gerhardt, Audrey Wang, Anna Butz, Jiehao Xing, Jialing Song, Larry Callahan > **arXiv ID:** [2609.16245](https://arxiv.org/abs/2609.16245) [cs.AI] > **Submitted:** 14 September 2026 > **Authors:** Vincent Karpf, Joseph Reth, Eike Gerhardt, Audrey Wang, Anna Butz, Jiehao Xing, Jialing Song, Larry Callahan --- ## 论文概要 > ## Summary 当前的大语言模型 (Large Language Models, LLMs) 主要针对最终输出结果的生成进行优化，而普遍缺乏在真实科学推理过程中所必需的长周期动态策略切换能力——例如在自由探索假设、规范严谨执行以及批判性复盘评估之间从容自如地过渡。 > Current large language models are primarily optimized for final output generation rather than the dynamic, long-horizon shifts required during genuine scientific reasoning (e.g., transitioning between exploration, disciplined execution, and critical reassessment). 本文提出了 **元认知引导 (Metacognitive Steering) **，这是一种在推理阶段直接生效的控制方法。它能够在不修改模型任何参数的前提下，精准识别模型当前的认知工作状态，并在特定神经网络层上动态介入调控。通过分析科学家在万亿参数混合专家 (Mixture-of-Experts, MoE) 模型 *Kimi 2.6* 上的真实交互轨迹，作者团队成功识别出控制科学判断行为的低维控制结构。随后，他们将该方法应用于自主科研系统 **Columbus-1** 中，成功引导模型完成了多项极具挑战性的复杂任务——例如在 Linux 蓝牙协议栈 BlueZ 中发现未公开的安全漏洞，以及设计能够通过不可节流固体火箭发动机实现动力反推着陆的火箭。这项工作充分表明，对推理过程层面的科学判断力进行建模，能够赋予前沿模型高度可解释且动态可控的科研策略决策能力。 > This paper introduces **Metacognitive Steering**, an inference-time control method that reads a model's cognitive regime and dynamically intervenes across specific neural layers without modifying its parameters. By analyzing scientist interaction traces on a trillion-parameter mixture-of-experts model (*Kimi 2.6*), the authors identify a low-dimensional control structure governing scientific judgment. Operationalized within the autonomous research system **Columbus-1**, this approach successfully directed complex tasks—such as discovering vulnerabilities in BlueZ and designing a propulsively landing solid-motor rocket—demonstrating how process-level judgment can enable interpretable, dynamic strategy control in frontier models. --- ## 元数据与参考信息 > ## Metadata & Reference Information * **主要学科领域 (Primary Subject) ：** 人工智能 (`cs.AI`) * **DOI 标识符：** [10.48550/arXiv.2609.16245](https://doi.org/10.48550/arXiv.2609.16245) * **全文获取链接 (Full-Text Links) ：** * [查看 PDF (View PDF) ](https://arxiv.org/pdf/2609.16245) * [HTML 网页版（实验性预览）](https://arxiv.org/html/2609.16245v1) * [TeX 源代码 (TeX Source) ](https://arxiv.org/src/2609.16245) > * **Primary Subject:** Artificial Intelligence (`cs.AI`) > * **DOI:** [10.48550/arXiv.2609.16245](https://doi.org/10.48550/arXiv.2609.16245) > * **Full-Text Links:** > * [View PDF](https://arxiv.org/pdf/2609.16245) > * [HTML Version (Experimental)](https://arxiv.org/html/2609.16245v1) > * [TeX Source](https://arxiv.org/src/2609.16245) --- ## 论文摘要 > ## Abstract 面向长周期科研任务的科学发现智能体 (AI Agent) ，必须能够根据不断演化的证据，在发散探索、规范执行与批判性重新评估之间自如切换。然而，现有的语言模型大多只使用现成科学成果进行训练，并依赖结果级信号进行优化，这导致模型在学习科学判断过程中所必需的“过程级动态切换”时，难以获得充分的监督信号。 > Long-horizon scientific discovery agents must alternate between exploration, disciplined execution, and critical reassessment as evidence changes. Current language models are trained primarily on the products of science and optimized using outcome-level signals, providing limited supervision for these process-level shifts in scientific judgment. 我们深入探索了这种科学判断力是否能够从科学家的人机交互轨迹中提炼出来，并直接用于掌控冻结的前沿大模型内部计算。通过采集真实科学研究活动中的对比干预数据，我们在万亿参数混合专家模型 Kimi 2.6 中发现了一个高度协调的低维控制结构。残差分析、注意力权重子空间对齐以及跨层奇异值分解 (Singular Value Decomposition, SVD) 的实验证据共同表明，在模型的中间深度层存在一个跨越核心神经网络层的“控制界面”。 > We investigate whether such judgment can be recovered from scientist interaction traces and used to control the internal computation of a frozen frontier model. Using contrastive interventions collected during real scientific research, we identify a coordinated, low-dimensional control structure within Kimi 2.6, a trillion-parameter mixture-of-experts model. Residual analysis, attention-weight subspace alignment, and cross-layer singular value decomposition converge on a mid-depth control surface spanning key layers. 基于这一发现，我们提出了 **元认知引导 (Metacognitive Steering) **。这是一种在模型推理阶段起效的控制器，它能在不修改模型参数的前提下，敏锐识别模型的当前认知状态，并针对性地在特定神经网络层上动态组合干预信号，以实现假设探索、程序收敛或批判性反思。行为分析表明，这种干预引导能够促成更持久的探索、主动的思路剪枝，以及对证据高度敏锐的综合研判。 > We introduce **Metacognitive Steering**, an inference-time controller that reads the model's cognitive regime and dynamically composes layer-specific interventions for exploration, procedural convergence, or critical reassessment without modifying model parameters. Behavioral analyses show that this control produces more sustained exploration, explicit pruning, and evidence-responsive synthesis. 我们将该方法在自主科研系统 **Columbus-1** 中付诸实战检验。该系统不仅成功锁定了 Linux 蓝牙协议栈 BlueZ 中 8 个已被独立复现、攻击者可触及的高危漏洞，还全程主导了一枚 10 英尺高火箭的设计、仿真模拟与物理加工制造——该火箭旨在通过不可节流的固体发动机完成极高难度的动力反推软着陆。综合这些实验成果，研究证明了过程层面的科学判断能够为大模型的推理策略提供清晰可解释、高度动态的有效控制监督。 > We operationalize the method in **Columbus-1**, an autonomous research system that identified eight independently reproduced, attacker-reachable vulnerabilities in BlueZ and directed the design, simulation, and fabrication of a ten-foot rocket intended to land propulsively using non-throttleable solid motors. Together, these results show that process-level scientific judgment can provide supervision for interpretable, dynamic control over a model's reasoning strategy.

Z.ai 详解基于 10 万张国产芯片的 GLM-5.3-Flash 推理系统架构构建

Fri, 18 Sep 2026 00:00:00 GMT

2026 年 9 月 17 日，人工智能企业 Z.ai 正式发布技术报告，详细披露了其如何在由超过 10 万张国产 AI 加速芯片组成的超大规模集群上，从零搭建起面向核心大模型 GLM-5.3-Flash 的生产级推理系统架构。这是国产算力芯片首次在十万卡级别生产环境中承受住超大规模真实业务流量的严苛检验。尤为引人瞩目的是，该系统的底层算子与架构性能调优并非单纯依赖人类工程师手动完成，而是由以 GLM-5.3 为核心驱动的“基础架构智能体” (Infra Agent) 自主探索与优化完成的。该系统成功应对了稀疏与线性注意力混合架构、高达 100 万 Token 的超长上下文窗口以及复杂多模态请求，最终使集群端到端吞吐量提升了整整三倍，硬件利用效率与单 Token 推理成本比肩主流 NVIDIA 硬件，生动展现了大模型递归自我进化的巨大工程威力。 --- ## 总结 > ## Summary 2026 年 9 月 17 日，Z.ai 发布了一份详尽的技术报告，全面展示了其如何从零构建起支撑 **GLM-5.3-Flash** 模型的生产级推理基础设施。该集群由超过 **10 万张国产 AI 加速芯片**组成，这一里程碑代表了国产算力芯片在生产环境中前所未有的部署规模。尤为突出的是，整个基础设施优化的绝大部分工作并不是单纯依赖人类工程师手动完成，而是由基于 **GLM-5.3 驱动的基础架构智能体 (Infra Agent)** 协同完成的。最终构建的推理系统能够游刃有余地支撑模型的复杂混合架构、高达 100 万 Token 的长上下文窗口以及海量多模态请求，其效率指标已可比肩业界主流的 NVIDIA 硬件。 > On September 17, 2026, Z.ai released a comprehensive technical report detailing how it successfully built a production-grade inference infrastructure for its **GLM-5.3-Flash** model from scratch. Operating across a cluster of over **100,000 Chinese-made AI accelerators**, this milestone represents an unprecedented scale for domestic silicon in production. Notably, a significant portion of the infrastructure optimization was driven by an **Infra Agent powered by GLM-5.3** rather than engineers alone. The resulting system successfully handles the model's complex hybrid architecture, 1-million-token context window, and multimodal requests, achieving efficiency levels comparable to mainstream NVIDIA hardware. --- ## 国产芯片上的超大规模推理挑战 > ## Scaling Inference on Domestic Silicon 在这种前所未有的超大体量下驾驭国产芯片集群，团队面临着极为特殊的工程挑战。Z.ai 不仅要应对单芯片片上显存容量与通信带宽的物理制约，还要直面国产算力生态尚不成熟的现实困境——算子底层支持不全、权威的标准开发文档时常缺位等问题屡见不鲜。 > Operating a massive cluster of Chinese-made accelerators at this scale presented unique hurdles. Z.ai faced limitations regarding on-chip memory capacity and bandwidth, coupled with an immature ecosystem where kernel support was incomplete and standard documentation was frequently missing. GLM-5.3-Flash 模型本身于 2026 年 8 月 26 日正式发布，拥有 3,200 亿总参数量和 180 亿激活参数，采用了融合稀疏注意力与线性注意力的前沿混合架构。在此前以 *ox-alpha* 的化名在 OpenCode 和 OpenRouter 平台上进行匿名盲测期间，该模型迅速成为两家平台上调用量最高的模型，在上线最初的六天内就处理了超过 62 万亿 Token 的庞大流量。 > The GLM-5.3-Flash model itself—launched on August 26, 2026—features 320 billion total parameters and 18 billion active parameters utilizing a hybrid architecture of sparse and linear attention. Following its anonymous testing phase on OpenCode and OpenRouter under the alias *ox-alpha*, it rapidly became the most-used model on those platforms, processing over 62 trillion tokens within its first six days. ## 密集反馈方法：为智能体打造高效闭环 > ## The Dense Feedback Method 在对复杂的推理基础设施进行深度调优时，传统的端到端性能指标所能提供的帮助十分有限；它们虽然能报警提示性能“发生了”下滑，却无法准确定位问题出在“何处”以及“为什么”。为了打破这种黑盒困境，Z.ai 独创性地引入了**密集反馈方法 (Dense Feedback Method)**。 > Traditional end-to-end metrics provide limited utility when optimizing complex infrastructures; they signal *that* performance dropped, but fail to pinpoint *why*. To bridge this gap, Z.ai implemented a **dense feedback method**. 该方法将以下多维度的诊断手段有机交织在一起： > This approach weaves together: * 正确性测试 * 运行时日志与关键事件 * 执行追踪轨迹 (Execution Traces) * 微基准测试 (Microbenchmarks) 与端到端宏观指标 > * Correctness tests > * Runtime logs and events > * Execution traces > * Microbenchmarks and end-to-end metrics 通过将这些元素整合为可快速重复执行的自动化工作流，基础架构智能体 (Infra Agent) 每次做出微调后，都可以在局部环境下迅速验证推想，而不必经历漫长的全量集群重新部署与全网压力测试等待。 > By unifying these elements into repeatable workflows, the Infra Agent could locally validate hypotheses without waiting for full deployment and load testing after every tweak. Z.ai 为这一反馈闭环确立了三大核心构建准则： > Z.ai established three core criteria for this feedback loop: 1. **局部精准性 (Locality)**：必须直接绑定到具体的启动参数、代码修改、计算算子、输入边界条件、执行线程或调用路径。 2. **极高时效性 (Efficiency)**：获取成本极其低廉，且反馈结果必须足够快速及时。 3. **客观可验证性 (Objective Verification)**：由标准参考实现和对照实验提供强力背书，以实锤确定系统性能因果机制，而非停留在表象的相关性推测上。 > 1. **Locality:** Tied directly to launch parameters, code changes, kernels, input conditions, threads, or execution paths. > 2. **Efficiency:** Inexpensive and timely to acquire. > 3. **Objective Verification:** Supported by reference implementations and controlled experiments to confirm root causes rather than mere correlations. 在双方协同工作时，人类工程师主要负责界定顶层系统边界并把控生产核心风险（如数值计算精度规范与高并发行为），而智能体则承担了海量细节分析、假设推演与底层代码修改。最终打造出的整套极致优化技术栈，集成了节点内张量并行 (Tensor Parallelism)、ReplaySSM 机制、W8A8 权重激活量化、INT8/FP8/BF16 混合精度缓存量化、Layer Split 层分割技术，以及编码-预填充-解码解耦 (Encode-Prefill-Decode Disaggregated) 架构。 > Working in tandem, engineers defined high-level system boundaries and assessed critical production risks (such as numerical semantics and concurrency behavior), while the agent handled analysis, hypotheses, and code modifications. The optimized stack combined intra-node tensor parallelism, ReplaySSM, W8A8 quantization, mixed-precision INT8/FP8/BF16 cache quantization, Layer Split, and an Encode-Prefill-Decode disaggregated architecture. ## 三大关键工程突破 > ## Three Key Engineering Triumphs

VQ-bench：可组合式矢量量化框架与评测基准

Fri, 18 Sep 2026 00:00:00 GMT

在人工智能与现代信息检索领域，高维向量是向量数据库和各类大语言模型 (Large Language Model, LLM) 运转的核心数据载体，但全精度的海量向量存储与检索面临着极高的硬件成本。矢量量化 (Vector Quantization, VQ) 技术通过将高维连续向量压缩为紧凑的离散编码，成为大幅削减内存占用并加速相似度计算的关键支柱。然而，学术界与工业界虽然涌现出繁多的量化算法，却因评估指标各异、测试数据集不同以及底层硬件优化不一致，长期缺乏公平可比的标准评测。为此，Pinecone 正式推出了开源框架与基准评测套件 VQ-bench，创新性地将绝大多数复杂量化器解构为由少量核心原语模块组装而成的流水线。VQ-bench 不仅为经典与前沿量化算法提供了统一严谨的横向对比舞台，更让开发者能够像搭积木一样自由组合、快速验证新型量化策略，为向量检索生态注入了强大的模块化基础设施。 --- ## 核心内容摘要 > ## Summary 对于向量数据库和大语言模型 (Large Language Model, LLM) 而言，矢量量化 (Vector Quantization, VQ) 是一项不可或缺的关键技术，能够大幅压缩高维向量的存储空间。然而，学术界近年来发表了浩如烟海的量化器算法，由于评测指标各异、测试数据集不统一以及针对不同硬件的底层优化大相径庭，想要对它们进行公平公正的横向对比，一直是一大行业痛点。 > Vector quantization (VQ) is crucial for reducing the storage size of high-dimensional vectors in vector databases and large language models (LLMs). However, evaluating and comparing the sheer volume of published quantizers has historically been challenging due to inconsistent metrics, datasets, and hardware optimizations. 为了攻克这一难题，Pinecone 推出了开源框架与基准评测套件 **VQ-bench**。它的精妙之处在于，将绝大多数主流量化器抽象为由少量基础“原语”组合而成的模块化“流水线”。本文将深入探讨 VQ-bench 如何实现量化器评测的标准化体系，详细拆解备受关注的 E-RaBitQ 等经典架构，并重点展示在 ArXiv 和 Yahoo 等公开数据集上的核心性能对比结果。 > To solve this, Pinecone introduces **VQ-bench**, an open-source framework and benchmarking suite that models most published quantizers as modular "pipelines" built from a small set of primitive operations. This post outlines how VQ-bench standardizes quantizer evaluation, breaks down popular architectures like E-RaBitQ, and highlights key performance comparisons across datasets like ArXiv and Yahoo. --- ## 引言 > ## Introduction 对于任何向量数据库来说，在对向量进行相似度检索之前，必须先将它们妥善存储在系统中。然而，以全精度直接保存海量的高维向量，硬件开销极为昂贵。**矢量量化 (Vector Quantization, VQ) ** 技术能够显著减少存储单个向量所需的比特位 (Bit) 数，因此成为了构建与运维高效向量数据库的核心技术基石。 > Before a vector database can search vectors, it has to store them. But storing high-dimensional vectors at full precision is quite expensive. **Vector quantization** (VQ) reduces the number of bits needed to store a vector, making it a critical part of maintaining a vector database. 正因为 VQ 在向量数据库与大模型体系中扮演着举足轻重的角色，每年学术界都会涌现出海量的相关研究论文。早在打造首批系统原型时，Pinecone 就已经深入应用了量化技术。不过精益求精的追求永无止境，于是我们着手对近年来的前沿研究成果展开全面调研与基准测试。在这个过程中，市面上数量极其庞大的量化器让我们眼花缭乱；更棘手的是，几乎每篇论文的评测标准都不尽相同——各家使用的测试数据集互不一致，衡量的性能指标五花八门，底层更针对不同的硬件架构做了各自定制的加速优化。我们始终无法在业内找到任何能够对主流顶尖量化算法进行系统性、横向公平对比的研究工作。 > Because VQ is so important (to both vector databases and LLMs), many research papers are published on the topic every year. Pinecone has been using quantization since its first prototypes. But we can always do better, so we set out to survey and benchmark newer results. We were pretty overwhelmed by just how many quantizers are out there. To make matters worse, every paper seemed to evaluate performance differently, measuring different metrics on different datasets and optimizing for different hardware. We were unable to find any systematic attempt to evaluate the leading methods against one another. 显然，想要从零开始百分之百忠实复现几十种量化器，本身就是一项巨大的挑战。幸运的是，随着我们对前沿文献的逐步深挖，一条清晰的规律浮出了水面：许多新发表的量化器本质上只是对现有算法的微小变体。事实上，绝大部分量化器都是基于一组数量相当有限的基础运算单元构建起来的。这激发了我们的灵感：如果我们把这些核心基础单元打包成一个开源原语库，让构建量化器变得如同照着菜单配菜一样简单——只需选定使用哪些原语并排定先后执行顺序，那会怎样？如此一来，我们便能在一个完全统一、透明且可复现的基准线上公平评测所有算法；同时，这也为研究人员探索现有量化器的全新变体、甚至是彻底发明全新的算法，铺平了实验道路。 > Of course, faithfully implementing dozens of quantizers from scratch comes with its own challenges. Luckily, as we dug deeper into the literature, we began to notice a pattern. Many published quantizers are actually just slight variations of existing ones. In fact, most of them are built from a relatively small set of primitive operations. That gave us an idea: what if we published an open-source library of these core primitives, where building a quantizer was as easy as writing a recipe of which primitives to use and in what order? Then, we would be able to evaluate all of these quantizers in a fair and reproducible way. It would also make it easier to experiment with new variations of existing quantizers or invent new ones altogether. 这就是 VQ-bench 项目的缘起。伴随着这篇博文，我们非常自豪地正式向社区开源发布 VQ-bench，包含以下核心资源： > This was the start of the VQ-bench project. With this post, we're excited to share VQ-bench with the public, including: * 公开的 [官方网站 (website) ](https://vq-bench.com)：持续更新并展示主流热门量化器的最新基准评测数据； * 开源 [GitHub 仓库](https://github.com/pinecone-io/vq-bench)：欢迎社区贡献您自己的量化器实现与运算原语； * 专题学术 [论文 (paper) ](https://arxiv.org/abs/2608.11240)：发表于 [VecDB@VLDB 2026](https://vecdb-ws.github.io/vldb2026) 研讨会，并附带完整的 [演讲幻灯片 (talk slides) ](https://vq-bench.com/slides.pdf)。 > * A public [website](https://vq-bench.com) with a running benchmark of popular quantizers > * A [GitHub repo](https://github.com/pinecone-io/vq-bench) where you can contribute your own quantizers and primitives > * A [paper](https://arxiv.org/abs/2608.11240) on VQ-bench (presented at [VecDB@VLDB 2026](https://vecdb-ws.github.io/vldb2026)), along with the [talk slides](https://vq-bench.com/slides.pdf). *请注意，目前发布的仅是 VQ-bench 的首个迭代版本；我们非常期待来自社区的宝贵反馈、勘误建议与代码贡献，后续我们也会持续纳入更多前沿量化器。* > *Note that this is just the first iteration of VQ-bench; we encourage feedback, corrections, and contributions, and we will add more quantizers over time.* --- ## 量化器核心接口 > ## Quantizers 通俗来说，**量化器 (Quantizer) ** 本质上是任何能够接收一组向量、将其压缩编码并在后续按需还原出目标信息的计算实体。在 VQ-bench 体系中，一个标准的量化器必须实现以下四种核心方法： > A **quantizer** is anything that can take a set of vectors, compress them, and recover desired information later on. In VQ-bench, a quantizer must implement four methods: | 接口方法 | 功能说明 | | :--- | :--- | | `fit` | 输入一组向量样本（以及可选的查询向量样本），训练并拟合出量化模型 (*model*) | | `encode` | 根据训练好的模型与输入向量集，为每个向量生成对应的紧凑压缩编码 (*codes*) | | `reconstruct` | 根据模型以及向量 *x* 的压缩编码，还原重构 (*reconstruct*) 出该向量的近似表示 | | `score` | 根据模型、查询向量 *q* 以及向量 *x* 的压缩编码，直接预估两者之间的点积相似度得分 (*score*) ⟨q, x⟩ | > | Method | Function | > | :--- | :--- | > | `fit` | given a sample of vectors (and optionally queries), learn a *model* | > | `encode` | given the model and a set of vectors, return per-vector *codes* | > | `reconstruct` | given the model and the code for vector *x*, *reconstruct* it | > | `score` | given the model, a query vector *q*, and the code for *x*, estimate the dot-product *score* ⟨q, x⟩ | --- ## 基础运算原语 > ## Primitives 在学术界的研究中，量化器极少是从零孤立构建的，它们大都是由一组基础运算单元拼接组装而成的。在 VQ-bench 中，这套核心运算单元被形式化定义为**原语 (Primitives) **。一个原语除了实现上述普通量化器具备的四种方法外，还额外实现了两个核心方法，用于精确定义数据如何流转传递给下一个处理阶段： > Quantizers are rarely built from scratch. In the literature, they are assembled from a small set of basic operations, which VQ-bench formalizes as **primitives**. A primitive implements the same four methods as any other quantizer, plus two more that specify exactly how it hands data to the next stage: | 接口方法 | 功能说明 | | :--- | :--- | | `apply` | 根据当前模型对输入向量进行变换，生成下游阶段所接收的数据表示 | | `apply_queries` | 根据当前模型对查询向量进行变换，生成下游阶段所接收的查询数据表示 | > | Method | Function | > | :--- | :--- | > | `apply` | given the model, transform the vectors into what the next stage should see | > | `apply_queries` | given the model, transform the queries into what the next stage should see | 此外，原语的 `reconstruct` 与 `score` 方法分别将下游下一阶段输出的重构向量与得分估计值作为输入，进而完成逆向的数据折叠与聚合。 > A primitive's `reconstruct` and `score` methods also take as input the next stage's reconstruction and score estimate, respectively. 这样一来，每个原语总共实现了六个核心方法。额外增加的两个方法构成了一套级联契约 (Chaining Contract) ，正是它们的存在，才使得各个原语能够像积木一样顺畅组合串联，这正是下一章节要探讨的核心。 > That makes six methods in total. The extra two are the chaining contract: they are what let primitives be composed, which is the subject of the next section. VQ-bench 将所有原语划分为三大主要类别： > VQ-bench implements three groups of primitives. * **调节器 (Conditioners) **：负责对输入数据进行预处理变换，并将处理后的数据传递给下游模块，例如去均值中心化 (`Center`) 、归一化 (`Normalize`) 、主成分分析 (`PCA`) 、随机旋转 (`RandomRotate`) 等； * **舍入器 (Rounders) **：负责将连续向量投影映射到有限码本 (Codebook) 中，并将逼近后的残差 (*residual*) 继续向下游传递，例如无符号整数映射 (`CastUint`) 、角度映射 (`CastAngular`) 、正态分布映射 (`CastNormal`) 、K 均值聚类 (`KMeans`) 等； * **切分器 (Splitters) **：负责将高维向量拆分为多个子向量切片，并为每个切片分配各自独立的原语处理链，例如子空间分段 (`Segment`) 等。 > * **Conditioners** transform the data and pass it downstream (`Center`, `Normalize`, `PCA`, `RandomRotate`, ...). > * **Rounders** cast each vector to a finite codebook, passing the *residual* downstream (`CastUint`, `CastAngular`, `CastNormal`, `KMeans`, ...). > * **Splitters** split the vectors and quantize each part with its own chain of primitives (`Segment`). --- ## 模块化流水线架构 > ## Pipelines 所谓的**流水线 (Pipeline) **，是指将两个或多个原语以链表形式级联组合而成的特殊量化器。压缩向量的过程就像沿着流水线正向穿行，而重构还原向量（或计算检索得分）的过程则是反向逆流而上。 > A **pipeline** is a special type of quantizer given by composing two or more primitives in a chain. Compressing a vector walks it forward through the chain, and recovering a vector (or its score) walks it backward. * **正向传递 (Forward pass) **：`fit` 与 `encode` 遵循相同的正向流转逻辑。在每个处理阶段，它们完成当前阶段的既定任务（训练拟合模型或生成编码）。随后调用 `apply` 方法对向量进行变换并递归传递给下一阶段。流程结束时，`fit` 会将各阶段的模型参数拼接合并，`encode` 则将各阶段生成的编码有序串联。 * **反向传递 (Backward pass) **：`reconstruct` 从流水线的最后一个阶段开始逆向执行。处于上层的每个阶段依次将自身对应的变换“撤销并叠加”回去（例如重新加上均值中心向量、做逆向旋转等），最终第一阶段就能还原出对原始向量的高精度近似。 * **得分计算 (`score`)**：整体执行逻辑与反向传递类似，唯一区别在于每个阶段都需要看到与当前阶段视角完全一致的查询向量。因此，它首先通过 `apply_queries` 将查询向量单向正向推导一遍，随后再在计算得分时执行反向聚合。 > * The forward pass: `fit` and `encode` follow the same path. At each stage, they perform that stage's job (learning the model / computing the codes). Then, they call `apply` to transform the vectors to the next stage and recurse. At the end, `fit` concatenates each stage's model and `encode` concatenates each stage's codes. > * The backward pass: `reconstruct` starts at the last stage. Each stage above it folds its own contribution back in (e.g., adding back the mean, undoing a rotation, etc.) until the first stage has an approximation of the original vector. > * `score` works the same way, except every stage needs the query as *it* saw the data. So, it begins by walking just the query forward with `apply_queries`. Then, it performs the backward pass on the score. 当然，量化器并非必须采用流水线架构。只要一个算法完整实现了上述四个核心方法，就可以作为合法的量化器接入体系，我们的接口充分包容了各类非流水线结构的算法设计。不过，文献中绝大多数已发表的量化方案都可以极其优雅地表达为原语流水线，这也正是这种解构设计能够作为统一基石的巨大价值所在。 > A quantizer does not have to be a pipeline. Anything that implements the four methods qualifies, and the interface leaves room for methods that are built some other way. But most published quantizers can be expressed as pipelines of primitives, which is what makes the decomposition worth building on. 以近年来备受关注的前沿量化算法 E-RaBitQ 为例（在我们的系列基准实验中，它的表现极为出色）。E-RaBitQ 的核心处理管线仅仅由四个原语串联而成： > For example, E-RaBitQ is a popular quantizer (which we found to be quite performant in our experiments). The E-RaBitQ pipeline consists of four primitives: 1. **中心化 (Center) **：从每个向量中减去整个数据集的均值向量； 2. **归一化 (Normalize) **：将每个向量缩放调整为单位范数（模长为 1）； 3. **随机旋转 (Random Rotation) **：对每个向量施加一个随机正交旋转（或随机阿达马旋转，Random Hadamard Rotation）； 4. **角度映射 (Angular Cast) **：在超球面上根据角度将每个向量映射到离它最近的 $b$ 位 ($b$-bit) 整数网格点上。 > 1. **Center:** subtract the average dataset vector from each vector > 2. **Normalize:** scale each vector to unit norm > 3. **Random Rotation:** apply a random orthogonal (or random Hadamard) rotation to each vector > 4. **Angular Cast:** snap each vector to a -bit integer grid by rounding to the nearest grid point in angle. 下图直观展示了这一流水线的数据流向，下方表格则详细列出了各个原语在此流水线中所执行的具体函数逻辑： > A diagram of this pipeline and table for the primitive functions are given below.

The E-RaBitQ pipeline.

| 接口方法 | 中心化 (Center) | 归一化 (Normalize) | 随机旋转 (Random Rotation) | 角度映射 (Angular Cast) | | :--- | :--- | :--- | :--- | :--- | | `fit` | 拟合数据集均值向量 μ | 无 (none) | 生成旋转随机种子 | 无 (none) | | `encode` | 无 (none) | 提取向量范数 ‖x‖ | 无 (none) | 提取网格点 grid(x) 及夹角余弦 cos(x, grid(x)) —— 每维度分配 b 比特及一个标量 | | `apply` | x → x − μ | x → x / ‖x‖ | x → Rx | x → x − ĝ，其中 ĝ = grid(x) / ‖grid(x)‖ | | `apply_queries` | 恒等变换 (identity) | 恒等变换 (identity) | q → Rq | 恒等变换 (identity) | | `reconstruct` | y → y + μ | y → ‖x‖ · y | y → Rᵀy | y → y + ĝ | | `score` | s → s + ⟨q, μ⟩ | s → ‖x‖ · s | 保持 s 不变，因查询向量同步经过了相同旋转 | s → s + ⟨q, ĝ⟩ / cos(x, grid(x)) | > | | Center | Normalize | Random Rotation | Angular Cast | > | :--- | :--- | :--- | :--- | :--- | > | `fit` | mean dataset vector μ | none | rotation seed | none | > | `encode` | none | the norm ‖x‖ | none | grid(x) and cos(x, grid(x)) — b bits per dimension and one scalar | > | `apply` | x → x − μ | x → x / ‖x‖ | x → Rx | x → x − ĝ, where ĝ = grid(x) / ‖grid(x)‖ | > | `apply_queries` | identity | identity | q → Rq | identity | > | `reconstruct` | y → y + μ | y → ‖x‖ · y | y → Rᵀy | y → y + ĝ | > | `score` | s → s + ⟨q, μ⟩ | s → ‖x‖ · s | s → s, since the query was rotated too | s → s + ⟨q, ĝ⟩ / cos(x, grid(x)) | --- ## 实验评测与结果分析 > ## Experimental Results 我们在来自 [VIBE](https://vector-index-bench.github.io/) 基准的 5 个公开数据集上，全面评测了包含 14 种经典及前沿算法的量化器套件。每个数据集均包含用于量化编码的基础向量集以及用于相似度打分的检索查询集。在此，我们重点展示其中两个典型数据集的评测结果：**ArXiv**（包含 1,344,643 个 768 维向量）和 **Yahoo**（包含 677,305 个 384 维向量）。更详尽完整的评测数据与交互图表均可在 [官方网站 (website) ](https://vq-bench.com) 上公开查阅。 > We evaluated a suite of 14 quantizers on 5 datasets from [VIBE](https://vector-index-bench.github.io/). Each dataset consists of vectors to encode and queries to score. Below, we present some results for two of the datasets: **ArXiv** (1,344,643 vectors in 768 dimensions) and **Yahoo** (677,305 vectors in 384 dimensions). You can view the full results on the [website](https://vq-bench.com).

4-bit 旋转量化技术深度剖析：消除离群值的极致压缩艺术

Fri, 18 Sep 2026 00:00:00 GMT

在海量向量检索与大语言模型 (LLM) 部署中，高维嵌入向量的爆炸式增长给服务器内存与带宽带来了严峻挑战。传统的标量或二值量化往往受制于向量维度的“离群值” (Outliers) ，导致低比特压缩时精度急剧下滑。开源向量数据库 Weaviate 在 1.39 版本中重磅升级了旋转量化 (Rotational Quantization, RQ) 技术，正式推出 4-bit 旋转量化，并通过快速沃尔什-阿达马变换 (Fast Walsh-Hadamard Transforms, FWHT) 和 SIMD 指令级优化打通了硬件性能瓶颈。该方案在保持与原始精度近乎一致召回率的同时，将堆内存占用骤降 45%，并显著超越了 TurboQuant 等前沿方案，为亿级规模向量检索树立了极致压缩与极速查询的新标杆。 --- ## 核心内容摘要 > ## Summary 在 Weaviate 1.39 版本中，**旋转量化 (Rotational Quantization, RQ)** 正式拓展支持了 4-bit 量化，并同步带来了一整套全方位的底层性能升级。这些改进涵盖了旋转计算、距离算子内核、内存访问通道以及编码流水线——不仅让 8-bit RQ 的运行速度实现了质的飞跃，更全新引入了 4-bit RQ。在提供与以往相当的召回率的同时，4-bit RQ 成功将**堆内存 (Heap) 占用削减了 45%**。本文将深入剖析这些性能提升背后的底层技术细节，深度剖析 RQ 在亿级海量数据规模下的扩展表现，并将其与前沿的 TurboQuant 算法进行详尽的横向对比。 > In Weaviate 1.39, **Rotational Quantization (RQ)** is extended to support 4-bit quantization alongside a comprehensive suite of performance updates. These enhancements cover rotations, distance kernels, memory access paths, and encoding pipelines—making 8-bit RQ significantly faster while introducing 4-bit RQ, which delivers comparable recall with a **45% heap reduction**. This post explores the technical details behind these improvements, analyzes RQ's behavior at massive scale, and compares its performance against TurboQuant. --- ## 技术引言 > ## Introduction 去年，我们推出了支持 8-bit 与 1-bit 规格的 [旋转量化 (Rotational Quantization, RQ)](https://weaviate.io/blog/8-bit-rotational-quantization) 。这些量化技术能够在大幅削减内存占用的同时实现极速向量检索，并且相比标量量化 (Scalar Quantization) 和二值量化 (Binary Quantization) 等同类替代方案，拥有显著更出色的召回率表现。 > Last year we introduced [Rotational Quantization](https://weaviate.io/blog/8-bit-rotational-quantization) (or RQ) with 8-bit and 1-bit sizes. These quantization techniques allow for fast vector search, while reducing memory usage, and at better recall than comparable alternatives such as scalar and binary quantization. 在全新的 Weaviate 1.39 中，我们进一步为 RQ 赋予了 4-bit 量化能力，并带来了一整套通用的量化性能优化。旋转计算、距离算子、编码逻辑以及内存访问路径均得到了全链路重构，最终达成了显著成效：1.39 中的 8-bit RQ 运行速度迎来了*大幅飞跃*，而全新的 4-bit RQ 则在保持相近召回率的前提下，实现了 **45%** 的堆内存降幅。 > Weaviate 1.39 extends RQ with 4-bit support, alongside a stack of quantization improvements in general. Rotations, distance kernels, encoding and the memory path have all been improved with net effect: 8-bit RQ is now *significantly faster* in 1.39 and 4-bit RQ provides similar recall with a **45%** heap reduction. 本文将完整记录这一工程探索背后的技术历程，并顺带解答开发者们最常关心的两个核心问题：随着数据集规模呈几何级数膨胀，RQ 的表现依然坚挺吗？以及，RQ 相比业界大热的 TurboQuant 究竟孰优孰劣？ > This post documents the story of that work, and along the way answers two questions people often ask: How does RQ hold up as datasets scale? And how does RQ compare to TurboQuant? --- ## 核心优化与架构升级 > ## Improvements 旋转量化基于 [Extended-RaBitQ](https://arxiv.org/abs/2409.09913) 算法演进而来，通过引入结构化快速旋转机制以及简化的逐向量区间拟合，大幅提升了编码速度。极速的编码性能（即将原始高维向量转换为量化压缩表示的过程）是衡量优秀量化算法的核心指标之一，因为它直接决定了向量数据导入与构建索引时的吞吐效率。 > Rotational quantization is based on [Extended-RaBitQ](https://arxiv.org/abs/2409.09913) with a structured fast rotation and simplified per-vector interval fitting to speed up encoding. Fast encoding performance (converting the original vector into its quantized representation) is an important part of a good quantization algorithm as it can have significant impacts on import performance. 这类方法的第一步，就是将原始向量与一个随机旋转矩阵相乘。这听起来或许有些反直觉，但随机旋转矩阵能赋予向量 [更优异的数学特性](https://weaviate.io/blog/8-bit-rotational-quantization#the-universal-power-of-random-rotations-making-every-vector-well-suited-for-scalar-quantization) ——尤其是抹平各维度上的尖锐离群值，使数值更加均匀地散布在整个量化区间的全长之上。 > The first step in these approaches is to multiply the original vector by a random rotation matrix. It may seem counter-intuitive but a random rotation matrix gives [better properties](https://weaviate.io/blog/8-bit-rotational-quantization#the-universal-power-of-random-rotations-making-every-vector-well-suited-for-scalar-quantization) to the vector in particular distributing the dimension values over the entire length of the quantization interval. 为了让随机旋转快如闪电，我们采用快速沃尔什-阿达马变换 (Fast Walsh-Hadamard Transforms, FWHT) 来对原始向量进行旋转。在 1.39 版本中，我们为 FWHT 增加了 SIMD 硬件指令集级优化支持。在输出结果与原有 Go 语言基线实现完全保持比特级一致 (Bit-identical) 的前提下，取得了如下显著的加速成绩： > To speed up the random rotation we use Fast Walsh-Hadamard Transforms (FWHT) to rotate the original vector. In 1.39, we added SIMD support for FWHT which led to the below improvements while being bit-identical to the Go reference: | 变换类型 (Transform) | 处理器架构 (CPU) | 1.38 版本 (Go 原生) | 1.39 版本 (SIMD 加速) | 加速比 (Speedup) | | :--- | :--- | ---: | ---: | ---: | | FWHT64 | Intel Xeon 8581C (amd64/AVX) | 81.3 ns | 26.5 ns | **3.1×** | | FWHT256 | Intel Xeon 8581C (amd64/AVX) | 515 ns | 84.5 ns | **6.1×** | | FWHT64 | Apple M1 (arm64/NEON) | 67.4 ns | 21.6 ns | **3.1×** | | FWHT256 | Apple M1 (arm64/NEON) | 428 ns | 96.2 ns | **4.5×** | > | Transform | CPU | 1.38 (Go) | 1.39 (SIMD) | Speedup | > | :--- | :--- | ---: | ---: | ---: | > | FWHT64 | Intel Xeon 8581C (amd64/AVX) | 81.3 ns | 26.5 ns | **3.1×** | > | FWHT256 | Intel Xeon 8581C (amd64/AVX) | 515 ns | 84.5 ns | **6.1×** | > | FWHT64 | Apple M1 (arm64/NEON) | 67.4 ns | 21.6 ns | **3.1×** | > | FWHT256 | Apple M1 (arm64/NEON) | 428 ns | 96.2 ns | **4.5×** | 配合对 SIMD 编码算子内核的一系列深度优化，整套 RQ 量化家族在向量编码性能上迎来了全方位的净提升： > Together with some other enhancements to SIMD encode kernels, this led to the following net increases in encoding performance across the whole RQ family: | 量化器类型 (Quantizer) | 处理器架构 (CPU) | 1.38 版本 | 1.39 版本 | 加速比 (Speedup) | | :--- | :--- | ---: | ---: | ---: | | RQ8 | Intel Xeon 8581C (amd64/AVX) | 27.3 µs | 7.11 µs | **3.8×** | | RQ1 | Intel Xeon 8581C (amd64/AVX) | 15.2 µs | 6.84 µs | **2.2×** | | RQ4 (非中心化 / uncentered) | Intel Xeon 8581C (amd64/AVX) | — | 6.36 µs | **1.39 新增** | | RQ4 (中心化 / centered) | Intel Xeon 8581C (amd64/AVX) | — | 8.08 µs | **1.39 新增** | | RQ8 | Apple M1 (arm64/NEON) | 14.7 µs | 6.18 µs | **2.4×** | | RQ1 | Apple M1 (arm64/NEON) | 13.0 µs | 6.18 µs | **2.1×** | | RQ4 (非中心化 / uncentered) | Apple M1 (arm64/NEON) | — | 5.65 µs | **1.39 新增** | | RQ4 (中心化 / centered) | Apple M1 (arm64/NEON) | — | 7.00 µs | **1.39 新增** | > | Quantizer | CPU | 1.38 | 1.39 | Speedup | > | :--- | :--- | ---: | ---: | ---: | > | RQ8 | Intel Xeon 8581C (amd64/AVX) | 27.3 µs | 7.11 µs | **3.8×** | > | RQ1 | Intel Xeon 8581C (amd64/AVX) | 15.2 µs | 6.84 µs | **2.2×** | > | RQ4 (uncentered) | Intel Xeon 8581C (amd64/AVX) | — | 6.36 µs | **new in 1.39** | > | RQ4 (centered) | Intel Xeon 8581C (amd64/AVX) | — | 8.08 µs | **new in 1.39** | > | RQ8 | Apple M1 (arm64/NEON) | 14.7 µs | 6.18 µs | **2.4×** | > | RQ1 | Apple M1 (arm64/NEON) | 13.0 µs | 6.18 µs | **2.1×** | > | RQ4 (uncentered) | Apple M1 (arm64/NEON) | — | 5.65 µs | **new in 1.39** | > | RQ4 (centered) | Apple M1 (arm64/NEON) | — | 7.00 µs | **new in 1.39** |

谁来教授哪个 Token？面向科学推理的验证器门控多专家策略内蒸馏

Thu, 17 Sep 2026 00:00:00 GMT

在利用大语言模型 (Large Language Model, LLM) 攻坚科学推理难题时，如何融合多个领域专家的智慧并注入到单一模型中是极具前景的方向。然而，传统的多教师策略内蒸馏 (On-Policy Distillation, OPD) 方法往往采取粗放的“序列级别”策略，误以为专家在整条推理链条的每一个位置都能提供有益指导，反而引入了大量噪音与负迁移。针对这一痛点，本文提出了“验证器门控多专家策略内蒸馏” (Verifier-Gated Multi-Expert On-Policy Distillation, VG-OPD) 架构，通过验证器实时评估专家的真实贡献并定位认知分歧，精准解答了“究竟由哪位专家来传授哪一个 Token”的核心问题。实验表明，该方案在 4B 与 8B 参数规模的学生模型上均取得了顶尖表现，不仅稳居 7 项科学推理基准测试的综合榜首，更为复杂推理场景下的精细化知识蒸馏提供了全新范式。 --- ## 执行摘要 > ## Executive Summary 多教师策略内蒸馏 (Multi-Teacher On-Policy Distillation, OPD) 是近年来崭露头角的一种前沿方法，它旨在将各垂直领域的专家能力融会贯通到统一模型中：先利用强化学习 (Reinforcement Learning, RL) 训练出各具专长的专家模型，再以学生模型自身采样生成的推理轨迹 (Rollouts) 为基础，将专家能力蒸馏给学生模型。 > Multi-teacher on-policy distillation (OPD) is an emerging approach for integrating specialist AI capabilities into a unified model by training experts using Reinforcement Learning (RL) and distilling them into a student model based on its own rollouts. 然而，传统方法通常在序列级别 (Sequence-level) 进行粗放的均匀监督分配——即把某个提示词 (Prompt) 整体分配给单一领域的专家教师，且整段回答中的每一个 Token 都被赋予相同的权重。这种做法错误地假设了专家老师在整段长回答的每一个字词上都同样有益、都能带来正向启发。 > Traditional methodologies assign supervision uniformly at the sequence level (allocating each prompt to a single domain teacher where every token receives identical weighting). This approach erroneously assumes that a teacher is uniformly helpful across an entire response. 本文的研究表明，在漫长的推理轨迹上，真正有价值的教师指导信号其实是高度稀疏且参差不齐的。为了攻克这一挑战，作者团队提出了**验证器门控多专家策略内蒸馏 (Verifier-Gated Multi-Expert On-Policy Distillation, VG-OPD)**，通过精巧的验证机制，系统解答了*“究竟谁该来教授哪一个 Token？”*的核心问题： > This paper demonstrates that useful teacher signals are actually sparse and heterogeneous along a reasoning trajectory. To address this, the authors introduce **Verifier-Gated Multi-Expert On-Policy Distillation (VG-OPD)**, which answers the question of *"who should teach which token?"* through a verification mechanism: * **反事实收益 (Counterfactual Gain)**：基于具体的答案判定准则评估专家的真实贡献，以此决定是否授予该专家教学权限； * **分歧精确定位 (Disagreement Localization)**：细致度量专家与学生之间的认知差异，精准圈定需要施加监督的关键位置； * **准则重要性设定 (Criterion Importance)**：合理设定各评价维度的权重参数； * **加性优势项注入 (Additive Advantage)**：将经过门控过滤的 KL 散度 (Kullback-Leibler Divergence) 作为 Token 级别的加性优势，无缝整合进分组相对策略优化 (Group Relative Policy Optimization, GRPO) 算法中。 > * **Counterfactual Gain:** Evaluates an expert's contribution based on specific answer criteria to authorize teaching rights. > * **Disagreement Localization:** Measures divergence between the expert and student to pinpoint supervision. > * **Criterion Importance:** Sets the appropriate weighting parameters. > * **Additive Advantage:** Incorporates the gated KL divergence into GRPO (Group Relative Policy Optimization) as a token-level additive advantage. --- ## 核心发现与卓越性能 > ## Key Findings & Performance * **全面领跑权威基准测试 (Benchmark Dominance)**：在面向科学推理任务、利用强化学习训练出的能力专家进行评估时，VG-OPD 在 **4B 和 8B 两种参数规模的学生模型**上均取得了 **7 个基准测试**的综合最优成绩，并在两种规模下均夺得了 5 项基准测试的第一名。尤其在知识密集型的科学推理挑战中，该方法带来的性能飞跃最为显著。 * **收益来源剖析 (Source of Gains)**：深入分析表明，性能提升的核心动力在于对监督信号进行了经过验证的精准局部定位，而非简单地堆叠更多教师模型或微调蒸馏损失函数。错配监督预算被证实是对模型破坏性最强的做法，盲目全盘蒸馏往往会把强化学习的性能反向拖拽至基线以下——而验证器门控蒸馏机制成功遏制了这一弊端。 > * **Benchmark Dominance:** When instantiated for scientific reasoning using RL-trained capability experts, VG-OPD achieved the best overall performance across **seven benchmarks** for both **4B and 8B student models**, ranking first on five benchmarks at both scales. The most significant performance gains occurred in knowledge-intensive scientific reasoning tasks. > * **Source of Gains:** Analysis reveals that improvements stem from localizing verified supervision rather than simply adding more teachers or altering the distillation loss. Misplacing the supervision budget proved to be the most damaging modification, and indiscriminate distillation often dragged RL performance below its baseline—a drawback successfully mitigated by gated distillation.

深入探秘 NVIDIA cuDNN Graph API：基于 Frontend 的算子融合、自动调优与计划复用

Thu, 17 Sep 2026 00:00:00 GMT

随着深度学习模型规模的爆发式增长，现代 GPU 硬件面临着严峻的显存带宽瓶颈，传统框架层逐个算子调用的模式会产生海量中间张量的读写开销。NVIDIA 推出的 cuDNN Graph API 及其高层封装 cuDNN Frontend 彻底打破了以往僵硬的库函数黑盒限制，允许开发者以有向计算图的形式自由编排算子。借助该接口，工程师能够将卷积或矩阵乘法与其后续的偏置加法、激活函数、AMAX 规约等操作深度融合成单个高效内核，并精细挑选最优的底层执行引擎。本文通过详实的单卡实测代码，全面拆解了算子融合、引擎自动调优、执行计划序列化持久化、动态维度内核缓存以及 CUDA Graph 捕获等核心技术，为构建极致吞吐与低延迟的生产级推理服务提供了深度实践参考。 --- ## 执行摘要 > ## Executive Summary 本教程带你深入传统深度学习框架底层，全面探索 NVIDIA **cuDNN Frontend** 计算图 API (Graph API) 的强大功能。它摒弃了过往僵化固定的库函数逐层调用逻辑，转而将所有前向或反向计算建模为由各个算子构成的有向计算图。这种贴近硬件底层的构建方式，赋予了开发者极高的控制权：不仅能够手动挑选最契合当前硬件的底层执行引擎 (Execution Engine)，还能构建高度优化的融合内核 (Fused Kernel) ——彻底消除偏置相加 (Bias Addition)、激活函数 (Activation) 以及 AMAX 极值规约等尾部运算产生的显存往返开销；此外，它还原生支持全引擎自动调优 (Autotuning)、执行计划序列化与反序列化 (Plan Serialization)、动态张量形状 (Dynamic Shapes) 处理，并能无缝接入 CUDA Graph 进行图捕获，从而消除内核启动开销。 > This tutorial explores NVIDIA’s **cuDNN Frontend** graph API from beneath traditional deep learning frameworks. Rather than relying on rigid library calls, computations are modeled as directed graphs of operations. This low-level approach allows developers to manually select execution engines, construct highly optimized fused kernels (eliminating unnecessary memory traffic for operations like bias additions, activations, and AMAX reductions), leverage autotuning, utilize plan serialization, handle dynamic shapes, and integrate with CUDA graph capture. 文中给出的所有实操代码均在单张 GPU 上完成了严格测试，并与原生 PyTorch 的参考基准进行了逐一比对，在确保数值计算绝对正确的前提下量化测量了算子融合与图优化带来的实际性能飞跃。 > All code examples are tested against PyTorch references on a single GPU to validate correctness and measure performance gains. --- ## 1. 环境配置与系统初始化 > ## 1. Environment Setup & Initialization 我们首先安装 `nvidia-cudnn-frontend` 依赖包，并配置动态链接器以确保系统能够精准定位并加载 `libcudnn.so` 动态库。接着，根据当前 GPU 的硬件计算能力 (Compute Capability) 自动匹配并初始化计算精度（在较新的架构上优先选择 `bfloat16`，否则采用 `float16`），同时定义了一套可复用的通用辅助函数，用于张量描述、计算图编译构建、显存工作区 (Workspace) 分配以及基准性能测试。 > We begin by installing the `nvidia-cudnn-frontend` package and configuring the dynamic loader to ensure `libcudnn.so` is correctly detected. We initialize the working precision (`bfloat16` or `float16` depending on compute capability) and define reusable helper functions for tensor descriptions, graph compilation, workspace allocation, and benchmarking. ```python import os import sys import glob import math import time import ctypes import traceback import subprocess RESULTS = {} def banner(title): print("\n" + "=" * 78) print(title) print("=" * 78) def section(name): def wrap(fn): def run(*a, **kw): banner(name) try: out = fn(*a, **kw) RESULTS[name] = out if isinstance(out, str) else "ok" return out except Exception as e: RESULTS[name] = f"SKIPPED / FAILED -> {type(e).__name__}: {e}" print(f"\n[!] {name} did not complete: {type(e).__name__}: {e}") traceback.print_exc(limit=3) return None return run return wrap banner("0. Install nvidia-cudnn-frontend and locate libcudnn") subprocess.run( [sys.executable, "-m", "pip", "install", "-q", "nvidia-cudnn-frontend"], check=True, ) import torch assert torch.cuda.is_available(), "No GPU. Runtime -> Change runtime type -> GPU." torch.backends.cudnn.enabled = True _ = torch.nn.functional.conv2d( torch.randn(1, 1, 8, 8, device="cuda"), torch.randn(1, 1, 3, 3, device="cuda") ) torch.cuda.synchronize() try: import nvidia.cudnn _libdir = os.path.join(os.path.dirname(nvidia.cudnn.__file__), "lib") os.environ["CUDNN_PATH"] = os.path.dirname(nvidia.cudnn.__file__) os.environ["LD_LIBRARY_PATH"] = _libdir + ":" + os.environ.get("LD_LIBRARY_PATH", "") for _so in sorted(glob.glob(os.path.join(_libdir, "libcudnn*.so*"))): try: ctypes.CDLL(_so, mode=ctypes.RTLD_GLOBAL) except OSError: pass except Exception as _e: print(f" (no pip cuDNN package found, relying on system cuDNN: {_e})") import cudnn print(" cuDNN frontend imported successfully.") banner("1. Environment") DEV = torch.device("cuda") MAJOR, MINOR = torch.cuda.get_device_capability() SM = MAJOR * 10 + MINOR CUDNN_VER = cudnn.backend_version() print(f" GPU : {torch.cuda.get_device_name(0)}") print(f" Compute capability : sm_{SM}") print(f" Torch / CUDA : {torch.__version__} / {torch.version.cuda}") print(f" cuDNN backend : {CUDNN_VER}") try: print(f" cuDNN version str : {cudnn.backend_version_string()}") except Exception: pass DTYPE = torch.bfloat16 if SM >= 80 else torch.float16 HAS_SDPA = SM >= 80 print(f" Working dtype : {DTYPE}") print(f" Fused SDPA usable : {HAS_SDPA}") HANDLE = cudnn.create_handle() TORCH2CUDNN = { torch.float16: cudnn.data_type.HALF, torch.bfloat16: cudnn.data_type.BFLOAT16, torch.float32: cudnn.data_type.FLOAT, torch.int32: cudnn.data_type.INT32, torch.int64: cudnn.data_type.INT64, torch.int8: cudnn.data_type.INT8, torch.uint8: cudnn.data_type.UINT8, } def tensor_of(graph, t, name): return graph.tensor( name=name, dim=list(t.size()), stride=list(t.stride()), data_type=TORCH2CUDNN[t.dtype], ) def scalar_of(graph, name): return graph.tensor( name=name, dim=[1, 1, 1], stride=[1, 1, 1], data_type=cudnn.data_type.FLOAT, is_pass_by_value=True, ) def build(graph, heur=None, policy=None): heur = heur or [cudnn.heur_mode.A, cudnn.heur_mode.FALLBACK] graph.validate() graph.build_operation_graph() graph.create_execution_plans(heur) graph.check_support() if policy is None: graph.build_plans() else: graph.build_plans(policy) return graph def workspace_for(graph): n = graph.get_workspace_size() return torch.empty(max(n, 1), device=DEV, dtype=torch.uint8) def bench(fn, warmup=10, iters=50): for _ in range(warmup): fn() torch.cuda.synchronize() s, e = torch.cuda.Event(True), torch.cuda.Event(True) s.record() for _ in range(iters): fn() e.record() torch.cuda.synchronize() return s.elapsed_time(e) / iters def tflops(flops, ms): return flops / (ms * 1e-3) / 1e12 def report(tag, ms, flops=None): extra = f" ({tflops(flops, ms):7.2f} TFLOP/s)" if flops else "" print(f" {tag:<34s} {ms:8.3f} ms{extra}") ``` --- ## 2. 算子融合实战：卷积 $\rightarrow$ 偏置加法 $\rightarrow$ ReLU > ## 2. Fused Convolution $\rightarrow$ Bias $\rightarrow$ ReLU 接下来我们着手构建第一个完整的计算图：将二维卷积 (2D Convolution)、偏置加法 (Bias Addition) 以及非线性激活函数 ReLU 紧密串联，全流程融合成单个 GPU 内核 (Single Kernel)。在张量内存布局上，所有输入输出均严格维持通道后置的 `channels_last` (NHWC) 内存格式，以便充分发挥 NVIDIA Tensor Core 硬件核心的向量化加速吞吐。 > We construct our first computation graph: a 2D convolution followed by a bias addition and a ReLU activation, fused entirely into a single kernel. Tensors are kept in `channels_last` (NHWC) format to match Tensor Core hardware requirements. ```python N, C, H, W = 32, 128, 56, 56 K, R, S = 256, 3, 3 PAD, STR, DIL = 1, 1, 1 P = (H + 2 * PAD - DIL * (R - 1) - 1) // STR + 1 Q = (W + 2 * PAD - DIL * (S - 1) - 1) // STR + 1 CONV_FLOPS = 2 * N * K * P * Q * C * R * S CONV_STATE = {} @section("2. Fused Conv -> Bias -> ReLU") def conv_fusion(): x = torch.randn(N, C, H, W, device=DEV, dtype=DTYPE).to(memory_format=torch.channels_last) w = torch.randn(K, C, R, S, device=DEV, dtype=DTYPE).to(memory_format=torch.channels_last) b = torch.randn(1, K, 1, 1, device=DEV, dtype=DTYPE) y = torch.empty(N, K, P, Q, device=DEV, dtype=DTYPE).to(memory_format=torch.channels_last) g = cudnn.pygraph( handle=HANDLE, name="conv_bias_relu", io_data_type=TORCH2CUDNN[DTYPE], intermediate_data_type=cudnn.data_type.FLOAT, compute_data_type=cudnn.data_type.FLOAT, ) X = tensor_of(g, x, "X") Wt = tensor_of(g, w, "W") Bt = tensor_of(g, b, "bias") conv = g.conv_fprop( image=X, weight=Wt, padding=[PAD, PAD], stride=[STR, STR], dilation=[DIL, DIL], compute_data_type=cudnn.data_type.FLOAT, ) biased = g.bias(input=conv, bias=Bt) Y = g.relu(input=biased) Y.set_output(True).set_data_type(TORCH2CUDNN[DTYPE]) Y.set_dim(list(y.size())).set_stride(list(y.stride())) t0 = time.perf_counter() build(g) build_ms = (time.perf_counter() - t0) * 1e3 ws = workspace_for(g) pack = {X: x, Wt: w, Bt: b, Y: y} g.execute(pack, ws) torch.cuda.synchronize() ref = torch.relu(torch.nn.functional.conv2d(x, w, bias=b.flatten(), padding=PAD)) err = (y.float() - ref.float()).abs().max().item() scale = ref.float().abs().max().item() print(f" problem : N{N} C{C} {H}x{W} -> K{K} {R}x{S} ({DTYPE})") print(f" build : {build_ms:.1f} ms workspace: {ws.numel()/1024:.1f} KiB") print(f" max |err|: {err:.4f} (ref max {scale:.2f}, rel {err/max(scale,1e-9):.2e})") assert err / max(scale, 1e-9) < 5e-2, "numerical mismatch vs PyTorch" ms_cudnn = bench(lambda: g.execute(pack, ws)) ms_torch = bench(lambda: torch.relu( torch.nn.functional.conv2d(x, w, bias=b.flatten(), padding=PAD))) print() report("cuDNN FE (single fused kernel)", ms_cudnn, CONV_FLOPS) report("PyTorch (conv+bias, then relu)", ms_torch, CONV_FLOPS) print(f" speedup: {ms_torch/ms_cudnn:.2f}x") CONV_STATE.update(graph=g, pack=pack, ws=ws, x=x, w=w, b=b, y=y) return f"{ms_cudnn:.3f} ms, {tflops(CONV_FLOPS, ms_cudnn):.1f} TFLOP/s" conv_fusion() ``` --- ## 3. 自动调优：全引擎候选配置基准评测 > ## 3. Autotuning: Benchmarking All Engine Configurations 与其完全盲目依赖库提供的默认启发式搜索，我们主动向系统查询了多种启发式决策模式（包含模式 `A`、模式 `B` 以及保底降级模式 `FALLBACK`），并通过 `cudnn.build_plan_policy.ALL` 策略构建出所有通过支持检查的候选执行计划 (Execution Plans)，从而横向测评各个可用引擎在当前硬件工况下的实际性能差异与算力发挥。 > Instead of relying purely on default heuristics, we query multiple heuristic modes (`A`, `B`, and `FALLBACK`) and build all candidate plans using `cudnn.build_plan_policy.ALL` to evaluate performance variations across available engines. ```python @section("3. Autotuning: build ALL plans, time each engine config") def autotune(): x, w, b, y = CONV_STATE["x"], CONV_STATE["w"], CONV_STATE["b"], CONV_STATE["y"] g = cudnn.pygraph( handle=HANDLE, name="conv_autotune", io_data_type=TORCH2CUDNN[DTYPE], intermediate_data_type=cudnn.data_type.FLOAT, compute_data_type=cudnn.data_type.FLOAT, ) X = tensor_of(g, x, "X") Wt = tensor_of(g, w, "W") Bt = tensor_of(g, b, "bias") Y = g.relu(input=g.bias( input=g.conv_fprop(image=X, weight=Wt, padding=[PAD, PAD], stride=[STR, STR], dilation=[DIL, DIL], compute_data_type=cudnn.data_type.FLOAT), bias=Bt)) Y.set_output(True).set_data_type(TORCH2CUDNN[DTYPE]) Y.set_dim(list(y.size())).set_stride(list(y.stride())) g.validate() g.build_operation_graph() g.create_execution_plans([cudnn.heur_mode.A, cudnn.heur_mode.B, cudnn.heur_mode.FALLBACK]) g.check_support() g.build_plans(cudnn.build_plan_policy.ALL) n_plans = g.get_execution_plan_count() print(f" {n_plans} candidate engine configs survived support checks\n") pack = {X: x, Wt: w, Bt: b, Y: y} timings = [] for i in range(n_plans): try: g.build_plan_at_index(i) ws_sz = max(g.get_workspace_size_plan_at_index(i), 1) ws = torch.empty(ws_sz, device=DEV, dtype=torch.uint8) ms = bench(lambda: g.execute_plan_at_index(pack, ws, i), warmup=3, iters=15) timings.append((ms, i, ws_sz)) print(f" plan {i:>3d}: {ms:8.3f} ms " f"{tflops(CONV_FLOPS, ms):7.2f} TFLOP/s ws={ws_sz/1024:8.1f} KiB") except Exception as e: print(f" plan {i:>3d}: unusable ({type(e).__name__})") assert timings, "no plan executed" timings.sort() best_ms, best_i, best_ws = timings[0] worst_ms = timings[-1][0] print(f"\n fastest = plan {best_i} @ {best_ms:.3f} ms") print(f" slowest = {worst_ms:.3f} ms -> {worst_ms/best_ms:.1f}x spread across engines") print(" Takeaway: heuristics are good, but for a hot shape you ship the") print(" autotuned index (or the serialized plan from section 6).") return f"best plan {best_i} @ {best_ms:.3f} ms ({worst_ms/best_ms:.1f}x spread)" autotune() ``` --- ## 4. 矩阵乘法尾声流水线：缩放、偏置、激活与 AMAX 规约 > ## 4. Matrix Multiplication Epilogues (Scaling, Bias, Activation, and AMAX) 现在我们将算子融合的范围扩展至批处理矩阵乘法 (Batched Matmul)，并在其尾声 (Epilogue) 阶段嫁接一条复杂的流水线：包含一个主机端标量 alpha 缩放因子相乘、偏置加法、非线性激活，以及提取绝对值最大值的 `AMAX` 极值规约（这一运算在最新的 FP8 低精度训练与量化工作流中至关重要）。 > We expand to batched matrix multiplication with a complex epilogue chain: a host scalar alpha scaling factor, bias addition, activation, and an `AMAX` reduction (useful for FP8 training quantization workflows). ```python @section("4. Matmul -> scale -> bias -> activation -> AMAX") def matmul_epilogue(): Bsz, M, Kd, Nd = 16, 512, 1024, 512 MM_FLOPS = 2 * Bsz * M * Nd * Kd a = torch.randn(Bsz, M, Kd, device=DEV, dtype=DTYPE) bm = torch.randn(Bsz, Kd, Nd, device=DEV, dtype=DTYPE) bias = torch.randn(1, 1, Nd, device=DEV, dtype=DTYPE) out = torch.empty(Bsz, M, Nd, device=DEV, dtype=DTYPE) amax = torch.empty(1, 1, 1, device=DEV, dtype=torch.float32) alpha_val = 0.125 alpha = torch.full((1, 1, 1), alpha_val, dtype=torch.float32) g = cudnn.pygraph( handle=HANDLE, name="matmul_epilogue", io_data_type=TORCH2CUDNN[DTYPE], intermediate_data_type=cudnn.data_type.FLOAT, compute_data_type=cudnn.data_type.FLOAT, ) A = tensor_of(g, a, "A") Bt = tensor_of(g, bm, "B") BIAS = tensor_of(g, bias, "bias") ALPHA = scalar_of(g, "alpha") acc = g.matmul(A=A, B=Bt, compute_data_type=cudnn.data_type.FLOAT) scaled = g.mul(a=acc, b=ALPHA) biased = g.bias(input=scaled, bias=BIAS) act_name = "relu" if hasattr(g, "gelu"): try: act = g.gelu(input=biased) act_name = "gelu" except Exception: act = g.relu(input=biased) else: act = g.relu(input=biased) print(f" activation used: {act_name}") OUT = act OUT.set_output(True).set_data_type(TORCH2CUDNN[DTYPE]) have_amax = True try: AMAX = g.reduction(input=act, mode=cudnn.reduction_mode.AMAX, compute_data_type=cudnn.data_type.FLOAT) AMAX.set_output(True).set_data_type(cudnn.data_type.FLOAT) AMAX.set_dim([1, 1, 1]).set_stride([1, 1, 1]) except Exception as e: have_amax = False print(f" (AMAX reduction unavailable here: {e})") build(g) ws = workspace_for(g) pack = {A: a, Bt: bm, BIAS: bias, ALPHA: alpha, OUT: out} if have_amax: pack[AMAX] = amax g.execute(pack, ws) torch.cuda.synchronize() ref = torch.matmul(a.float(), bm.float()) * alpha_val + bias.float() ref = torch.nn.functional.gelu(ref) if act_name == "gelu" else torch.relu(ref) rel = ((out.float() - ref).abs().max() / ref.abs().max()).item() print(f" shape : ({Bsz},{M},{Kd}) x ({Bsz},{Kd},{Nd})") print(f" rel err : {rel:.2e}") if have_amax: print(f" fused AMAX {amax.item():.4f} vs torch {ref.abs().max().item():.4f}") ms = bench(lambda: g.execute(pack, ws)) def torch_ref(): r = torch.baddbmm(bias.expand(Bsz, M, Nd), a, bm, beta=1.0, alpha=alpha_val) r = torch.nn.functional.gelu(r) if act_name == "gelu" else torch.relu(r) return r.abs().amax() ms_t = bench(torch_ref) print() report("cuDNN FE (one fused kernel)", ms, MM_FLOPS) report("PyTorch (bmm + act + amax)", ms_t, MM_FLOPS) print(f" speedup: {ms_t/ms:.2f}x -- the win is the epilogue traffic, not the GEMM") return f"{ms:.3f} ms, {tflops(MM_FLOPS, ms):.1f} TFLOP/s, {ms_t/ms:.2f}x vs torch" matmul_epilogue() ``` --- ## 5. 缩放点积注意力机制 (FlashAttention) 与执行计划序列化 > ## 5. SDPA (FlashAttention) & Plan Serialization 我们基于 cuDNN Graph API 实现了带有因果掩码 (Causal Masking) 的缩放点积注意力机制 (Scaled Dot-Product Attention, SDPA，即工业级 FlashAttention 实现，该特性需要 NVIDIA Ampere 架构及以上的 `SM80+` 硬件支持)；同时，我们展示了执行计划的序列化技术 (Plan Serialization) ——将编译完成的计算图序列化为二进制数据保存至磁盘，后续在生产部署时直接反序列化重载，并通过整数形式的唯一标识符 (UID) 绑定张量即可直接运行，从而彻底消除服务初次冷启动时的实时 JIT 编译等待。 > We implement Scaled Dot-Product Attention (SDPA) with causal masking (requiring Ampere `SM80+` architectures) and demonstrate plan serialization, allowing compiled graphs to be saved to disk, reloaded, and executed via integer UIDs to eliminate startup compilation delays. ```python @section("5. SDPA (Flash Attention) with causal masking") def sdpa_demo(): if not HAS_SDPA: raise RuntimeError(f"fused SDPA needs SM80+ (Ampere), this GPU is sm_{SM}") b, h, s, d = 4, 16, 1024, 64 scale = 1.0 / math.sqrt(d) SDPA_FLOPS = 4 * b * h * s * s * d * 0.5 q = torch.randn(b, h, s, d, device=DEV, dtype=DTYPE) k = torch.randn(b, h, s, d, device=DEV, dtype=DTYPE) v = torch.randn(b, h, s, d, device=DEV, dtype=DTYPE) o = torch.empty(b, h, s, d, device=DEV, dtype=DTYPE) g = cudnn.pygraph( handle=HANDLE, name="sdpa", io_data_type=TORCH2CUDNN[DTYPE], intermediate_data_type=cudnn.data_type.FLOAT, compute_data_type=cudnn.data_type.FLOAT, ) Q, Kt, V = tensor_of(g, q, "Q"), tensor_of(g, k, "K"), tensor_of(g, v, "V") causal = True try: O, _stats = g.sdpa(name="sdpa", q=Q, k=Kt, v=V, is_inference=True, attn_scale=scale, use_causal_mask=True) except TypeError: try: O, _stats = g.sdpa(name="sdpa", q=Q, k=Kt, v=V, is_inference=True, attn_scale=scale, diagonal_alignment=cudnn.diagonal_alignment.TOP_LEFT, right_bound=0) except Exception: causal = False O, _stats = g.sdpa(name="sdpa", q=Q, k=Kt, v=V, is_inference=True, attn_scale=scale) print(f" causal masking: {causal}") O.set_output(True).set_data_type(TORCH2CUDNN[DTYPE]) O.set_dim(list(o.size())).set_stride(list(o.stride())) build(g) ws = workspace_for(g) pack = {Q: q, Kt: k, V: v, O: o} g.execute(pack, ws) torch.cuda.synchronize() ref = torch.nn.functional.scaled_dot_product_attention(q, k, v, is_causal=causal, scale=scale) rel = ((o.float() - ref.float()).abs().max() / ref.float().abs().max()).item() print(f" shape : b{b} h{h} s{s} d{d} workspace {ws.numel()/1024:.1f} KiB") print(f" rel err : {rel:.2e}") ms = bench(lambda: g.execute(pack, ws)) ms_t = bench(lambda: torch.nn.functional.scaled_dot_product_attention( q, k, v, is_causal=causal, scale=scale)) print() report("cuDNN FE SDPA", ms, SDPA_FLOPS) report("torch SDPA (backend's choice)", ms_t, SDPA_FLOPS) print(" Note: torch may already be dispatching to cuDNN or FlashAttention,") print(" so parity here is the expected, healthy outcome.") return f"{ms:.3f} ms, {tflops(SDPA_FLOPS, ms):.1f} TFLOP/s" sdpa_demo() @section("6. Serialize a built graph, reload it, execute by UID") def serialization(): Bsz, M, Kd, Nd = 8, 256, 512, 256 a = torch.randn(Bsz, M, Kd, device=DEV, dtype=DTYPE) bm = torch.randn(Bsz, Kd, Nd, device=DEV, dtype=DTYPE) out = torch.empty(Bsz, M, Nd, device=DEV, dtype=DTYPE) UID_A, UID_B, UID_C = 1, 2, 3 g = cudnn.pygraph( handle=HANDLE, name="serializable_mm", io_data_type=TORCH2CUDNN[DTYPE], intermediate_data_type=cudnn.data_type.FLOAT, compute_data_type=cudnn.data_type.FLOAT, ) A = tensor_of(g, a, "A").set_uid(UID_A) Bt = tensor_of(g, bm, "B").set_uid(UID_B) C = g.matmul(A=A, B=Bt, compute_data_type=cudnn.data_type.FLOAT) C.set_output(True).set_data_type(TORCH2CUDNN[DTYPE]).set_uid(UID_C) t0 = time.perf_counter() build(g) cold_ms = (time.perf_counter() - t0) * 1e3 blob = g.serialize() print(f" cold build : {cold_ms:.1f} ms") print(f" serialized plan : {len(blob)} bytes (cache this to disk / ship it)") t0 = time.perf_counter() g2 = cudnn.pygraph() try: g2.deserialize(HANDLE, blob) except TypeError: g2.deserialize(blob) warm_ms = (time.perf_counter() - t0) * 1e3 print(f" deserialize : {warm_ms:.1f} ms -> {cold_ms/max(warm_ms,1e-6):.1f}x faster startup") ws = torch.empty(max(g2.get_workspace_size(), 1), device=DEV, dtype=torch.uint8) g2.execute({UID_A: a, UID_B: bm, UID_C: out}, ws, handle=HANDLE) torch.cuda.synchronize() ref = torch.bmm(a.float(), bm.float()) rel = ((out.float() - ref).abs().max() / ref.abs().max()).item() print(f" rel err after reload: {rel:.2e}") return f"{len(blob)} B blob, reload {cold_ms/max(warm_ms,1e-6):.1f}x faster than rebuild" serialization() ``` --- ## 6. 动态维度处理与 CUDA Graph 捕获加速 > ## 6. Dynamic Shapes and CUDA Graph Capture 在实际推理场景中，为了高效应对多变的批处理大小 (Batch Size) 或变长序列长度 (Sequence Length)，同时避免每次维度微调都触发昂贵的实时 JIT 重新编译，我们跨计算图共享了底层内核缓存 (Kernel Cache)。更进一步，我们还将编译好的执行计划封装进 CUDA Graph 的图捕获机制中，彻底抹平了每次迭代在 CPU 与 GPU 之间的内核启动延迟 (Kernel Launch Latency)。 > To manage dynamic batch sizes or sequence lengths efficiently without incurring continuous JIT compilation overhead, we share a kernel cache across graph iterations. Furthermore, we wrap execution plans inside CUDA graphs to remove per-iteration kernel launch latencies. ```python @section("7. Dynamic shapes with a shared kernel cache") def dynamic_shapes(): kc = cudnn.create_kernel_cache() def make(n): x = torch.randn(n, 64, 32, 32, device=DEV, dtype=DTYPE).to(memory_format=torch.channels_last) w = torch.randn(64, 64, 3, 3, device=DEV, dtype=DTYPE).to(memory_format=torch.channels_last) y = torch.empty(n, 64, 32, 32, device=DEV, dtype=DTYPE).to(memory_format=torch.channels_last) g = cudnn.pygraph( handle=HANDLE, name=f"dyn_{n}", io_data_type=TORCH2CUDNN[DTYPE], intermediate_data_type=cudnn.data_type.FLOAT, compute_data_type=cudnn.data_type.FLOAT, kernel_cache=kc, is_dynamic_shape_enabled=True, ) X, Wt = tensor_of(g, x, "X"), tensor_of(g, w, "W") Y = g.conv_fprop(image=X, weight=Wt, padding=[1, 1], stride=[1, 1], dilation=[1, 1], compute_data_type=cudnn.data_type.FLOAT) Y.set_output(True).set_data_type(TORCH2CUDNN[DTYPE]) Y.set_dim(list(y.size())).set_stride(list(y.stride())) t0 = time.perf_counter() build(g) ms = (time.perf_counter() - t0) * 1e3 ws = workspace_for(g) g.execute({X: x, Wt: w, Y: y}, ws) torch.cuda.synchronize() return ms times = [(n, make(n)) for n in (8, 16, 24, 32)] for n, ms in times: print(f" batch {n:>3d}: build {ms:7.1f} ms") first, rest = times[0][1], [m for _, m in times[1:]] print(f"\n first shape {first:.1f} ms, later shapes avg {sum(rest)/len(rest):.1f} ms") print(" The cache lets shape-variant graphs reuse an already-JIT'd kernel,") print(" which is what keeps variable batch/seqlen serving out of rebuild hell.") return f"first {first:.0f} ms vs subsequent {sum(rest)/len(rest):.0f} ms" dynamic_shapes() @section("8. CUDA Graph capture around a cuDNN execution plan") def cuda_graph_capture(): if not CONV_STATE: raise RuntimeError("section 2 did not run, nothing to capture") g, pack, ws = CONV_STATE["graph"], CONV_STATE["pack"], CONV_STATE["ws"] eager_ms = bench(lambda: g.execute(pack, ws)) side = torch.cuda.Stream() side.wait_stream(torch.cuda.current_stream()) with torch.cuda.stream(side): cudnn.set_stream(handle=HANDLE, stream=side.cuda_stream) for _ in range(3): g.execute(pack, ws, handle=HANDLE) torch.cuda.current_stream().wait_stream(side) torch.cuda.synchronize() cg = torch.cuda.CUDAGraph() with torch.cuda.graph(cg): cudnn.set_stream(handle=HANDLE, stream=torch.cuda.current_stream().cuda_stream) g.execute(pack, ws, handle=HANDLE) cudnn.set_stream(handle=HANDLE, stream=torch.cuda.current_stream().cuda_stream) replay_ms = bench(lambda: cg.replay()) report("plain execute()", eager_ms) report("cuda graph replay()", replay_ms) print(f" launch overhead removed: {(eager_ms-replay_ms)*1e3:.1f} us/iter") print(" Pointers are frozen at capture time -- reuse the same buffers and") print(" copy new data into them, or re-capture.") return f"{eager_ms:.3f} -> {replay_ms:.3f} ms via replay" cuda_graph_capture() banner("SUMMARY") for name, res in RESULTS.items(): print(f" {name:<58s} {res}") print(""" Where to go next - samples/python in the repo: FP8/MXFP8 attention, paged KV cache, MoE grouped GEMM - python/cudnn/: the open-sourced CuTe DSL kernels (SDPA, grouped GEMM + SwiGLU, block-sparse and native sparse attention) you can read and modify - debugging: CUDNN_FRONTEND_LOG_INFO=1 and CUDNN_FRONTEND_LOG_FILE=stdout (use level 10 during CUDA graph capture -- level 1 dumps tensors and is not capture-safe) """) ``` --- ## 总结 > ## Conclusion 将深度学习网络层重构为显式的有向计算图，使开发者得以对内核融合、执行引擎筛选、编译生命周期以及内核启动开销拥有极致而精准的控制力。cuDNN Frontend API 在许多高频严苛场景中表现尤为卓越，例如：定制化的算子融合（彻底免除激活与规约尾声中间变量的显存写回）、对频繁调用的热点形状 (Hot Shapes) 进行严密的自动调优，以及结合 CUDA Graph 与执行计划序列化技术打造高性能、极低延迟的生产级在线推理服务闭环。 > By structuring deep learning layers as explicit computation graphs, developers unlock precise control over kernel fusion, engine selection, compilation lifecycles, and launch overheads. The cuDNN Frontend API excels particularly in scenarios involving custom fusions (such as avoiding intermediate writes for activation and reduction epilogues), hot shapes requiring rigorous autotuning, and performance-critical low-latency serving loops enhanced via CUDA graphs and plan serialization. --- 欢迎查阅 **[完整源码链接](https://github.com/MARKTECHPOST-AI-MEDIA-INC/AI-Agents-Projects-Tutorials/blob/main/Deep%20Learning/cudnn_frontend_nvidia_tutorial_Marktechpost.ipynb)**。本项目所有成果归原作者研究团队所有。欢迎在 **[Twitter](https://x.com/intent/follow?screen_name=marktechpost)** 上关注我们，加入拥有 **[15 万+ 成员的机器学习 SubReddit 社区](https://www.reddit.com/r/machinelearningnews/)**，订阅 **[官方技术周刊](https://magic.beehiiv.com/v1/f5e63dd4-5653-4f09-83e2-321a8b1ba526?email={{email}})**，以及关注我们的 **[Telegram 频道](https://t.me/machinelearningresearchnews)**。 > Check out the **[FULL CODES here](https://github.com/MARKTECHPOST-AI-MEDIA-INC/AI-Agents-Projects-Tutorials/blob/main/Deep%20Learning/cudnn_frontend_nvidia_tutorial_Marktechpost.ipynb)**. All credit goes to the researcher of this project. Feel free to follow us on **[Twitter](https://x.com/intent/follow?screen_name=marktechpost)** and join our **[150k+ ML SubReddit](https://www.reddit.com/r/machinelearningnews/)**, subscribe to **[our Newsletter](https://magic.beehiiv.com/v1/f5e63dd4-5653-4f09-83e2-321a8b1ba526?email={{email}})**, and join our **[Telegram channel](https://t.me/machinelearningresearchnews)**. 如需洽谈 GitHub 开源项目推广、Hugging Face 专题页面合作、新品发布或线上技术研讨会等商务合作，欢迎 **[点击此处与我们联系](https://forms.gle/wbash1wF6efRj8G58)**。 > For partnership inquiries regarding GitHub repo promotions, Hugging Face pages, product releases, or webinars, **[connect with us here](https://forms.gle/wbash1wF6efRj8G58)**. 本文首发于 [MarkTechPost](https://www.marktechpost.com)，原文标题为 [Inside NVIDIA’s cuDNN Graph API: Fusion, Autotuning, and Plan Reuse with cuDNN Frontend](https://www.marktechpost.com/2026/09/15/inside-nvidias-cudnn-graph-api-fusion-autotuning-and-plan-reuse-with-cudnn-frontend/)。 > The post [Inside NVIDIA’s cuDNN Graph API: Fusion, Autotuning, and Plan Reuse with cuDNN Frontend](https://www.marktechpost.com/2026/09/15/inside-nvidias-cudnn-graph-api-fusion-autotuning-and-plan-reuse-with-cudnn-frontend/) appeared first on [MarkTechPost](https://www.marktechpost.com).

Stellar Colosseum：面向数学与理论计算机科学长程研究的多智能体竞技框架

Thu, 17 Sep 2026 00:00:00 GMT

尽管现代大语言模型在生成简短数学证明方面表现出色，但在面对需要长程决策与精密推演的前沿科学研究时，往往容易因链条脆弱和不确定性累积而遭遇瓶颈。为了攻克这一难题，来自学术界与 Google 等机构的研究团队联合推出了 **Stellar Colosseum** —— 一个与模型无关的多智能体协同与推理竞技框架，专为数学和理论计算机科学 (Theoretical Computer Science, TCS) 等高难度长程科研任务量身打造。该框架创新性地引入了备选策略探索、成熟度门控、结构化证明图、针对性证伪反馈闭环以及重叠随机采样树聚合机制，让多智能体在协同博弈中不断锤炼严密的学术论证。在涵盖 FOCS、STOC 等顶级学术会议的 TCS-Bench 定理证明评测中，Stellar Colosseum 斩获了高达 71.0% 的准确率，并在复杂算法挑战赛中攻克了绝大部分难题，为 AI 辅助前沿科学探索迈出了坚实一步。 --- # Stellar Colosseum：面向数学与理论计算机科学长程研究的多智能体竞技框架 > # Stellar Colosseum: A Many-Agent Harness for Long-Horizon Research in Mathematics and Theoretical Computer Science **作者：** Honghao Lin, David P. Woodruff, Yuan Deng, Jieming Mao, Song Zuo, Vahab Mirrokni **arXiv 预印本：** [arXiv:2609.15983 [cs.AI]](https://arxiv.org/abs/2609.15983) (提交于 2026 年 9 月 14 日，最后修订于 2026 年 9 月 15 日) **主分类：** 人工智能 (`cs.AI`) **其他分类：** 计算与语言 (`cs.CL`)、机器学习 (`cs.LG`) > **Authors:** Honghao Lin, David P. Woodruff, Yuan Deng, Jieming Mao, Song Zuo, Vahab Mirrokni > **arXiv:** [arXiv:2609.15983 [cs.AI]]() (Submitted on 14 Sep 2026, last revised 15 Sep 2026) > **Primary Subject:** Artificial Intelligence (`cs.AI`) > **Additional Subjects:** Computation and Language (`cs.CL`), Machine Learning (`cs.LG`) --- ## 摘要 > ## Summary 虽然现代语言模型在给出看似严密自洽的简短证明方面表现抢眼，但在面对长程科研难题时却屡屡碰壁。这类长程探索的成败，往往完全取决于一连串环环相扣、充满不确定性且高度脆弱的连锁决策——稍有不慎，推演便前功尽弃。 > While modern language models excel at producing plausible short proofs, they frequently stumble on long-horizon research problems where success hinges on a fragile sequence of uncertain and interdependent decisions. 为了突破这一瓶颈，研究团队推出了 **Stellar Colosseum** —— 一个专为数学与理论计算机科学 (Theoretical Computer Science, TCS) 复杂长程研究打造的模型无关推理框架。 > To bridge this gap, the authors introduce **Stellar Colosseum**, a model-agnostic inference framework designed for complex research in mathematics and theoretical computer science (TCS).

BlueLM-GUI 技术报告：面向自进化移动端 GUI 智能体的真机飞轮系统

Thu, 17 Sep 2026 00:00:00 GMT

当前，能够像人类一样自主操作手机应用的移动端图形用户界面智能体 (GUI Agent) 正成为前沿热点，但现有模型在模拟沙盒中表现出色，一到真实真机环境下往往频频碰壁。本论文推出了 BlueLM-GUI——一个拥有 350 亿参数 (35B-A3B) 的移动端 GUI 智能体，首次确立了以“真实物理设备”为核心的数据与演进飞轮系统。该系统贯彻“珍惜每个样本、交互基于真机、评测动态演进”三大原则，不仅将真机操作中的报错转化为宝贵的高质量训练数据，还在数百台真实物理手机上展开强化学习。在 MobileGUI-VBench 与 AndroidWorld 等权威基准测试中，BlueLM-GUI 均斩获顶尖成绩，大幅超越顶尖闭源与开源模型，为移动端智能体的工业规模化落地开辟了全新路径。 --- ## 执行摘要 > ## Executive Summary 在移动端，能够帮我们操作手机应用的图形用户界面 (GUI) 智能体 (AI Agent) 正在经历深刻演化——正从以往拼凑式的模块化框架，迅速转向浑然一体的原生端到端训练模型。然而，要想把这种智能体真正推向实际工业化落地，长期以来一直受制于三大棘手瓶颈： > Mobile Graphical User Interface (GUI) agents are rapidly transitioning from modular frameworks to native, end-to-end trained models. However, industrial deployment has historically been impeded by three persistent bottlenecks: 1. **分布偏差 (Distribution Mismatch)**：传统的沙盒模拟训练环境很难真实还原真实生产环境中的复杂状况； 2. **失败经验未充分利用 (Underutilized Failures)**：智能体在真机上执行任务遭遇的昂贵报错，往往被随手丢弃，未能转化为反哺训练的养料； 3. **基准测试易饱和 (Benchmark Saturation)**：固定的静态基准测试很快就会被模型“刷爆”，逐渐失去指引模型持续迭代的能力。 > 1. **Distribution Mismatch:** Sandbox training environments fail to accurately simulate real-world production conditions. > 2. **Underutilized Failures:** Expensive errors occurring on physical devices are typically discarded rather than leveraged for training. > 3. **Benchmark Saturation:** Static benchmarks quickly saturate, reducing their efficacy in guiding model iteration. 为了攻克这些难题，研究团队推出了 **BlueLM-GUI**——一个基于“以真机为中心”的自我演化飞轮打造的 350 亿参数 (35B-A3B) 移动端 GUI 智能体。BlueLM-GUI 严格秉持三大核心理念——**“珍惜每个样本” (Every Sample Matters)**、**“交互基于真机” (Every Rollout Is Real)** 以及 **“评测动态演进” (Every Query Evolves)**，成功跨越了虚拟模拟与真实应用之间的鸿沟，在多项主流权威评测中均斩获了顶尖水准的性能表现。 > To address these challenges, the authors introduce **BlueLM-GUI**, a 35B-A3B mobile GUI agent built around a real-device-centric self-improvement flywheel. By adhering to three core principles—**Every Sample Matters**, **Every Rollout Is Real**, and **Every Query Evolves**—BlueLM-GUI bridges the gap between simulated training and real-world deployment, achieving state-of-the-art performance across major evaluation suites. --- ## BlueLM-GUI 飞轮的核心设计原则 > ## Core Principles of the BlueLM-GUI Flywheel

标准库遮蔽：主流编程语言的模块加载陷阱与 AI 智能体安全危机

Wed, 16 Sep 2026 00:00:00 GMT

在动态编程语言的发展历史中，许多运行时环境默认会将当前工作目录或脚本所在路径置于模块搜索路径中，这为“模块遮蔽” (Module Shadowing) 攻击埋下了重大隐患——攻击者只需在目录下放置同名文件，就能偷梁换柱劫持原生标准库。尽管 Ruby、Perl、Node.js、Deno 和 Julia 等现代运行时已逐步从设计层面剔除或防范了这种隐式路径查找，但 Python 和 PHP 至今仍默认将工作目录置于搜索路径首位，在解压不可信压缩包或访问外部目录时极易受到供应链攻击。尤其在当今 AI 编程智能体 (AI Agent) 广泛自主解压缩外部文件并即时运行辅助脚本的时代，这一看似古老的语言设计特性正演变为不可忽视的高危远程代码执行隐患。 --- ## 概要 > ## Summary 在动态编程语言的发展历史中，许多语言在默认情况下都会将当前工作目录或脚本所在路径纳入模块搜索路径。这意味着，本地目录下的任意文件都有可能悄悄覆盖内置的核心库，这种安全隐患被称为“模块遮蔽” (Module Shadowing)。本文将系统梳理各大主流编程语言运行时如何应对这一安全挑战。像 Ruby、Perl、Node.js、Deno 和 Julia 等现代语言，早已从根本上彻底消除或严格限制了这种从上下文环境中自动查找路径的做法，以杜绝恶意脚本被意外执行；然而，像 Python 和 PHP 这样的老牌语言，至今仍默认将当前工作目录排在搜索路径的首位。这就带来了一个严重的安全漏洞：一旦程序解压了不可信的压缩包，或是进入了不受信任的文件目录，攻击者就能轻而易举地通过同名文件掉包标准库，达成恶意代码执行。 > Dynamic languages have historically included the current directory or script location in their module search paths by default, allowing local files to override built-in libraries (a vulnerability known as module shadowing). This write-up explores how various runtimes handle this security challenge. While languages like Ruby, Perl, Node.js, Deno, and Julia have eliminated or mitigated ambient path lookups to prevent malicious file execution, languages like Python and PHP still place the working directory first on the path, leaving them vulnerable to attacks where unpacked archives or untrusted folders compromise standard library resolution. --- ## 已彻底移除：将当前工作目录踢出默认搜索路径 > ## Removed 早在 2010 年 8 月发布的 [1.9.2](https://github.com/ruby/ruby/blob/master/doc/NEWS/NEWS-1.9.2) 版本中，Ruby 就果断从 `$LOAD_PATH` 中删除了代表当前目录的 `.`。当时的更新公告极其简短，只有一句话：“`$:` 不再包含当前目录，请改用 `require_relative`”。引入的 [`require_relative`](https://docs.ruby-lang.org/en/master/Kernel.html#method-i-require_relative) 方法能够直接基于当前调用文件所在的磁盘位置进行相对寻址，与进程当前的工作目录完全解耦。这样一来，脚本既能安全地加载与自身同目录的兄弟文件，又能确保整个进程中的其他 `require` 调用免受外部不可信工作目录的污染。由于当时 Ruby 从 1.8 升级至 1.9 本身就是一次重大破坏性升级，因此这次加载路径的调整直接顺理成章地一同落地，既没有专门申请通用漏洞披露 (Common Vulnerabilities and Exposures, CVE) 编号，也没有提供任何用于回退兼容的“逃生舱”环境变量。 > Ruby dropped `.` from `$LOAD_PATH` in [1.9.2](https://github.com/ruby/ruby/blob/master/doc/NEWS/NEWS-1.9.2), released August 2010. The NEWS entry is one line, “$: no longer includes the current directory, use require_relative”. [`require_relative`](https://docs.ruby-lang.org/en/master/Kernel.html#method-i-require_relative) resolves against the calling file’s location, independent of the process working directory, so a script can load its own siblings while keeping the working directory off the search path for every other `require` in the process. There was no CVE and no escape-hatch environment variable; 1.9 was already a compatibility break from 1.8 and the load-path change went in alongside everything else. Perl 则在 2017 年 5 月发布的 [5.26.0](https://perldoc.perl.org/perl5260delta#Removal-of-the-current-directory-(%22.%22)-from-@INC) 版本中，将 `.` 从模块加载数组 `@INC` 的末尾彻底移除。与 Ruby 不同，Perl 的这次改动直接关联了一起严重的安全漏洞 [CVE-2016-1238](https://www.nntp.perl.org/group/perl.perl5.porters/2016/07/msg238271.html)，该漏洞最初由 cPanel 团队提交：当一个脚本切换工作目录到公共临时目录 `/tmp` 并尝试加载可选模块时，就会毫无防备地直接执行本地恶意用户提前在 `/tmp/Module.pm` 中留下的任意代码。Perl 在传统上一向默认在 `@INC` 中携带 `.`，不过污点模式 (Taint Mode, `perl -T`) 一直会自动剔除它，这说明官方早已知晓其潜在风险。为了平稳过渡，Perl 5.26 新增了环境变量 [`PERL_USE_UNSAFE_INC=1`](https://perldoc.perl.org/perl5260delta#PERL_USE_UNSAFE_INC) 供老项目临时回退，下游各大 Linux 发行版也为此[维护了长达数年的兼容补丁](https://wiki.gentoo.org/wiki/Project:Perl/Dot-In-INC-Removal)，直至 CPAN 社区中所有依赖 `.` 路径的历史模块被逐一修复完毕。 > Perl removed `.` from the end of `@INC` in [5.26.0](https://perldoc.perl.org/perl5260delta#Removal-of-the-current-directory-(%22.%22)-from-@INC), released May 2017. That one did have a CVE, [CVE-2016-1238](https://www.nntp.perl.org/group/perl.perl5.porters/2016/07/msg238271.html), reported by cPanel: a script that changes directory to `/tmp` and then loads an optional module runs whatever a local user has left at `/tmp/Module.pm`. Perl had traditionally shipped with `.` on `@INC`, and taint mode (`perl -T`) had always stripped it, so the risk was documented long before the fix. 5.26 added [`PERL_USE_UNSAFE_INC=1`](https://perldoc.perl.org/perl5260delta#PERL_USE_UNSAFE_INC) to restore the old behaviour for the transition, and downstream distributions [carried patches for years](https://wiki.gentoo.org/wiki/Project:Perl/Dot-In-INC-Removal) while CPAN modules that depended on `.` being present were fixed one at a time. --- ## 架构原生防范：优先内置模块与显式路径设计 > ## Designed Out 在 Node.js 中，`require` 机制[会优先检查内置核心模块名称](https://nodejs.org/api/modules.html#core-modules)，其优先级高于磁盘上的 `node_modules` 或任何本地文件。因此，即便调用脚本的同一目录下存在一个名为 `http.js` 的文件，执行 `require('http')` 也只会返回官方原生模块。在 Node.js 中若要加载同目录下的本地文件，历来都必须显式使用相对路径，例如 `require('./http')`。从 Node.js 14.18.0 开始，官方还引入了 [`node:` 协议前缀](https://nodejs.org/api/modules.html#core-modules)，使得 `require('node:http')` 可以进一步绕过 require 缓存，并且类似 `node:test` 这样的全新内置模块甚至只能通过该前缀访问。不过需要指出的是，这种防护仅覆盖了核心标准库：当引入第三方库如 `require('express')` 时，Node.js 仍会[从调用方所在目录开始向上逐级检索 `node_modules`](https://nodejs.org/api/modules.html#loading-from-node_modules-folders)。这意味着，解压后的不可信压缩包里若暗藏 `node_modules/express`，仍会优先于全局或系统安装的版本被加载。类似地，Java 在运行时通过类路径 (Classpath) 解析类，虽然[在未设置 `-cp` 或 `CLASSPATH` 环境变量时默认类路径就是 `.` 当前目录](https://docs.oracle.com/en/java/javase/21/docs/specs/man/java.html)，但其启动类加载器 (Bootstrap Class Loader) 会在应用程序类加载器扫描类路径之前先一步锁定 `java.*` 等基础核心类，因此同目录下的 `./java/lang/String.class` 根本不可能实现偷梁换柱。而 Deno 则更为纯粹，它完全抛弃了上下文环境隐式搜索路径的设计理念：所有模块导入必须是 [URL 或明确的相对路径](https://docs.deno.com/runtime/fundamentals/modules/)，任何未在导入映射表 (Import Map) 中声明的裸说明符都会直接抛出异常。 > Node’s `require` [checks core module names first](https://nodejs.org/api/modules.html#core-modules), before `node_modules` or anything else on disk, so `require('http')` returns the built-in even with an `http.js` in the caller’s directory. Loading a neighbouring file has always taken an explicit relative path, `require('./http')`. Node 14.18.0 added the [`node:` prefix](https://nodejs.org/api/modules.html#core-modules) so `require('node:http')` bypasses the require cache as well, and some newer built-ins such as `node:test` are only reachable that way. That protection covers core modules only: `require('express')` [searches upward from the caller’s directory through each `node_modules`](https://nodejs.org/api/modules.html#loading-from-node_modules-folders), so a `node_modules/express` inside an extracted archive is resolved ahead of any installed copy. Java resolves classes at run time from a classpath, and the [default classpath when neither `-cp` nor `CLASSPATH` is set is `.`](https://docs.oracle.com/en/java/javase/21/docs/specs/man/java.html), but the bootstrap class loader finds `java.*` before the application loader searches the classpath, so a `./java/lang/String.class` is unreachable for the same reason a local `http.js` is in Node. Deno dropped the ambient search path entirely: every import is [a URL or a relative path](https://docs.deno.com/runtime/fundamentals/modules/), and a bare specifier outside the import map is an error. Julia 默认的 [`LOAD_PATH`](https://docs.julialang.org/en/v1/base/constants/#Base.LOAD_PATH) 变量被设定为 `["@", "@v#.#", "@stdlib"]`。这三个符号路径分别对应当前激活的项目环境、带有版本号的用户全局默认环境以及标准库。无论进程当前切换到哪个目录下，`@` 符号都会严格锚定当前激活的 `Project.toml` 清单文件，因此即使在不可信目录下执行脚本，模块导入也只会严格依据配置文件清单进行解析。PowerShell 的 `Import-Module` 命令则仅检索 [`$env:PSModulePath`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_psmodulepath) 环境变量，默认只涵盖单用户目录、全体用户共享目录以及 `$PSHOME` 核心目录；此外，PowerShell 在可执行文件寻址上也采用了同样的严密设计：在当前工作目录下运行脚本[必须显式添加 `.\` 前缀](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_scripts)，直接键入裸名称只会搜索系统 `$env:PATH`，绝不会误执行当前目录下的同名文件。 > Julia’s default [`LOAD_PATH`](https://docs.julialang.org/en/v1/base/constants/#Base.LOAD_PATH) is `["@", "@v#.#", "@stdlib"]`, three symbolic entries that expand to the active project environment, the user’s versioned default environment, and the standard library. `@` resolves to whichever `Project.toml` is active, regardless of which directory the process is in, so a script run from an untrusted directory looks up imports in a manifest file. PowerShell’s `Import-Module` searches [`$env:PSModulePath`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_psmodulepath), which defaults to the per-user, all-users and `$PSHOME` module directories only, and the shell applies the equivalent rule to executable lookup: running a script in the working directory [requires an explicit `.\` prefix](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_scripts), so a bare name searches only `$env:PATH`. --- ## 隐患仍存：工作目录优先与被遮蔽的标准库 > ## Still There 与上述语言形成鲜明对比的是，PHP 默认的 [`include_path`](https://www.php.net/manual/en/ini.core.php#ini.include-path) 依然将 `.` 放在第一位。这意味着，执行 `include 'config.php'` 时，程序会毫不犹豫地优先加载当前工作目录下的 `config.php`，而不是系统 PEAR 路径中的对应库文件。Lua 默认的 [`package.path`](https://github.com/lua/lua/blob/v5.4.0/luaconf.h#L208-L219) 则以 `./?.lua;./?/init.lua` 结尾；当执行 `require "json"` 且所有已安装路径均未匹配时，搜索机制就会顺延兜底到当前目录下的 `./json.lua`。好在 Lua 自身的官方标准库 (如 `string`、`table`、`math` 等) 在 [`require` 遍历任何磁盘路径之前就已经注册到了 `package.loaded` 表中](https://github.com/lua/lua/blob/v5.4.0/linit.c)，因此本地的 `./string.lua` 无法覆盖原生标准库，只有第三方未预载的名称会暴露给同名劫持风险。在这两种语言中，如果开发者想要将工作目录从搜索路径中剥离，通常必须通过 `php -d include_path=...` 命令行参数或配置 `LUA_PATH` 环境变量来全量重写整个搜索路径字符串。 > PHP’s `.` is the first entry in the default [`include_path`](https://www.php.net/manual/en/ini.core.php#ini.include-path), so `include 'config.php'` picks up a `config.php` in the working directory ahead of one under the PEAR path. Lua’s default [`package.path`](https://github.com/lua/lua/blob/v5.4.0/luaconf.h#L208-L219) ends with `./?.lua;./?/init.lua`, so `require "json"` falls through to `./json.lua` after the installed-path entries have all missed. Lua’s own standard libraries, `string`, `table`, `math` and the rest, are [registered in `package.loaded` before `require` searches any path](https://github.com/lua/lua/blob/v5.4.0/linit.c), so a `./string.lua` is unreachable and only third-party names are exposed. In both PHP and Lua, dropping the working-directory entry means overriding the full path string, via `php -d include_path=...` or the `LUA_PATH` environment variable. Python 的机制则更为直接：当直接运行 Python 脚本文件时，解释器会将脚本所在目录置入 `sys.path[0]`；使用 `-m` 运行模块时，置入的是当前工作目录；而在使用 `-c` 或进入交互式终端 (REPL) 时，置入的则是一个空字符串，其实际含义同样等同于当前工作目录。早在 Python 3.4 中，官方就[新增了隔离模式 (Isolated Mode, `-I`) ](https://docs.python.org/3/using/cmdline.html#cmdoption-I)，该模式会在禁用该前置路径的同时，一并屏蔽用户级 site-packages 以及所有的 `PYTHON*` 环境变量。为了提供更精细的控制，Python 3.11 引入了一个针对性更强的安全开关：[`-P` 参数与 `PYTHONSAFEPATH` 环境变量](https://docs.python.org/3/using/cmdline.html#cmdoption-P)，它仅单独剔除 `sys.path[0]` 中的本地目录，而保留其他环境变量完好无损。该功能背后的问题跟踪单 [bpo-13475](https://github.com/python/cpython/issues/57684) 早在 2011 年 11 月就已被提出。合入 `-P` 功能的核心开发者 Victor Stinner 还在其个人仓库中起草了一份 [PEP 提案草案](https://github.com/vstinner/misc/blob/main/cpython/pep_path0.rst)，主张以 Perl 5.26 为先例将 safe-path 设为解释器的全局默认行为。然而，Ruby 当年能够从容剔除 `.`，是因为其同步推出了 `require_relative` 为同级依赖提供了替代出路；而 Python 的[显式相对导入语法](https://docs.python.org/3/reference/import.html#package-relative-imports)仅在正式包结构内部有效。这意味着，一个单独的独立脚本 `script.py` 若要直接 `import helper` 加载同级辅助模块，其底层完全依赖 `sys.path[0]` 指向当前目录；一旦在 `-P` 模式下运行，就会立即因找不到模块而抛出 `ModuleNotFoundError` 异常。 > Python prepends the script’s directory as `sys.path[0]` when running a file, the working directory when running `-m`, and an empty string, meaning the working directory, when running `-c` or the REPL. Isolated mode, `-I`, [added in 3.4](https://docs.python.org/3/using/cmdline.html#cmdoption-I), suppresses that entry along with user site-packages and all `PYTHON*` environment variables. 3.11 added a narrower switch, [`-P` and `PYTHONSAFEPATH`](https://docs.python.org/3/using/cmdline.html#cmdoption-P), which drops only the `sys.path[0]` entry and leaves the rest of the environment alone. The tracker issue behind `-P`, [bpo-13475](https://github.com/python/cpython/issues/57684), was opened in November 2011. Victor Stinner, who landed `-P`, has a [draft PEP](https://github.com/vstinner/misc/blob/main/cpython/pep_path0.rst) in his personal repo proposing to make safe-path the default and citing Perl 5.26 as precedent. Ruby could drop `.` because `require_relative` shipped in the same release and gave scripts another way to load their siblings. Python’s [explicit relative imports](https://docs.python.org/3/reference/import.html#package-relative-imports) only work inside a package, so a standalone `script.py` importing `helper.py` from the same directory depends on `sys.path[0]` being that directory, and running it under `-P` fails with `ModuleNotFoundError`. 在历史设计上，Perl、Ruby 和 Lua 都倾向于将 `.` 放在搜索路径的最末端——排在标准库和第三方库目录之后。因此，工作目录中的文件只有在全系统都找不到该名称时，才会作为“兜底”被加载；这也是为什么 CVE-2016-1238 漏洞需要依赖脚本尝试加载某个“可选模块”才能触发。然而，Python 和 PHP 却反其道而行之，直接将外部环境目录推到了搜索路径的最前端。由于官方标准库在路径列表里排在后面，一旦工作目录下存在一个名为 `struct.py` 的文件，它就会直接被加载并偷换掉真正的 `struct` 模块，所有从磁盘读取的标准库模块都无一幸免。虽然直接编译进解释器内核的内置模块以及[自 Python 3.11 起](https://docs.python.org/3/whatsnew/3.11.html#faster-startup)被冻结在启动内存中的核心集合 (例如 `os`、`abc`、`io`、`codecs` 等) 会在 `PathFinder` 扫描磁盘之前由 `sys.meta_path` 上的 `BuiltinImporter` 和 `FrozenImporter` 优先处理，但像 `struct` 这样纯粹基于磁盘 `.py` 包装 C 扩展的标准库模块，仍会交由 `PathFinder` 检索，从而不可避免地面临被恶意掉包的命运。 > Perl, Ruby and Lua all put `.` last, after the standard-library and site directories, so a file in the working directory could only supply a name that was missing everywhere else, and CVE-2016-1238 accordingly needed a target script that loaded an optional module. Python and PHP put the ambient directory first. The standard library comes after it on the path, `struct.py` in the working directory is loaded in place of the real `struct`, and any standard-library module read from disk is exposed the same way. Modules compiled into the interpreter and, [since 3.11](https://docs.python.org/3/whatsnew/3.11.html#faster-startup), the frozen startup set (`os`, `abc`, `io`, `codecs` among them) are served by `BuiltinImporter` and `FrozenImporter` on `sys.meta_path` before `PathFinder` touches disk; `struct` is a plain `.py` wrapper around a C extension and goes to `PathFinder`. | 语言/运行时 | 默认路径条目 | 搜索优先级 | 标准库可被遮蔽？ | 移除版本 | 安全模式开关 | |---|---|---|---|---|---| | Ruby | `$:` 中的 `.` | 末位 (last) | 否 | 1.9.2 (2010) | | | Perl | `@INC` 中的 `.` | 末位 (last) | 否 | 5.26 (2017) | 可通过 `PERL_USE_UNSAFE_INC` 恢复 | | Node.js | 无 (none) | | 否 | | | | Java | 未设置 `-cp` 时的 `.` | | 否 | | 任意 `-cp` 均可替换该默认值 | | Deno | 无 (none) | | 否 | | | | Julia | 无 (依据项目清单) | | 否 | | | | Lua | `package.path` 中的 `./?.lua` | 末位 (last) | 否 | | | | PHP | `include_path` 中的 `.` | 首位 (first) | 是 | | | | Python | `sys.path` 中的脚本目录 / 工作目录 | 首位 (first) | 是 | | `-I` (3.4) , `-P` / `PYTHONSAFEPATH` (3.11) | > | | Entry on default path | Position | Stdlib shadowable | Removed in | Safe-mode switch | > |---|---|---|---|---|---| > | Ruby | `.` in `$:` | last | no | 1.9.2 (2010) | | > | Perl | `.` in `@INC` | last | no | 5.26 (2017) | (`PERL_USE_UNSAFE_INC` restores) | > | Node.js | none | | no | | | > | Java | `.` when `-cp` unset | | no | | any `-cp` replaces it | > | Deno | none | | no | | | > | Julia | none (project manifest) | | no | | | > | Lua | `./?.lua` in `package.path` | last | no | | | > | PHP | `.` in `include_path` | first | yes | | | > | Python | script dir / cwd in `sys.path` | first | yes | | `-I` (3.4), `-P` / `PYTHONSAFEPATH` (3.11) | --- ## 威胁模型：AI 编程智能体时代的新危机 > ## Threat Model Perl 5.26 的发行说明中曾将这一风险清晰定义为：脚本在“当前目录不受信任 (例如 `/tmp`) ”时加载可选模块。在官方修复之前的二十年里，人们采取的应对方案纯粹是防御性的操作规程：系统运维脚本在加载任何可选模块前必须先 `chdir` 切换到一个安全可信的目录，并且用户只应当在自己完全受控的目录中运行工具。显然，这些经验法则全都是为“人类开发者”或“固定工作目录的系统守护进程”量身定制的，因为人类在终端敲下回车之前，清楚地知晓解释器将在哪个目录下启动。 > The Perl 5.26 release notes describe the risk as a script loading an optional module “when its current directory is untrusted (such as `/tmp`)”, and the mitigation for the twenty years before that was procedural: system scripts `chdir` somewhere safe before loading anything optional, and users run tools only from directories they control, advice written for a person or a fixed-cwd daemon choosing where the interpreter runs. 然而，当当今的 AI 编程智能体自主下载一个压缩包、将其解压、在解压内容旁随手编写一个辅助脚本并立即执行时，这实质上就是换了个目录名称的 `/tmp` 经典提权攻击场景。不可信的压缩包直接决定了目录中包含哪些文件，而智能体自行生成的脚本正好成为了触发恶意模块导入的扳机。更关键的是，Python 是各类 AI 智能体生成辅助代码时的绝对首选：无论是 OpenAI 的[代码解释器 (Code Interpreter) ](https://developers.openai.com/api/docs/guides/tools-code-interpreter)，还是 Anthropic 的[代码执行工具 (Code Execution Tool) ](https://platform.claude.com/docs/en/agents-and-tools/tool-use/code-execution-tool)，亦或是本文直接针对的 Claude Code 智能体命令行工具，在运行临时脚本时无一例外都依赖 Python 解释器。在安全研究员 Rehberger 构造的概念验证攻击中，被掉包的遮蔽模块甚至还贴心地将调用转发给了真正的标准库，从而使数据解码器依然能返回完全正确的结果，神不知鬼不觉地完成了恶意载荷注入。这与之前讨论过的[子进程可通过环境变量注入的覆盖标志位 (Override Flags) ](https://nesbitt.io/2026/08/25/hardening-the-override-flag.html)如出一辙：在由人类开发者审慎指定工作目录或命令行标志的年代，这种设计或许还能勉强容忍；但在 AI 智能体自主解压压缩包并随之将解压目录隐式作为工作目录的新时代，这一历史遗留特性无疑为远程代码执行大开方便之门。 > A coding agent that downloads an archive, extracts it, writes a helper next to the contents, and runs the helper is the `/tmp` case with a different label on the directory. The archive determined the directory contents, and the agent’s own script is what triggers the import. Python is also the language agents default to for generated helpers: the sandboxed runtime for both OpenAI’s [Code Interpreter](https://developers.openai.com/api/docs/guides/tools-code-interpreter) and Anthropic’s [code execution tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/code-execution-tool), and the interpreter Claude Code, the target in the write-up, invokes for ad-hoc scripts. In Rehberger’s version the shadowed module forwards to the real one, so the decoder returns a correct result. It is the same class of default as the [override flags a subprocess can set through its environment](https://nesbitt.io/2026/08/25/hardening-the-override-flag.html): tolerable while a person supplied the working directory or the flag, and an agent processing an archive supplies the working directory as a side effect of extracting it.

一个向量能容纳多少思维？叠加推理的表征容量

Wed, 16 Sep 2026 00:00:00 GMT

在大语言模型 (LLM) 的多步复杂推理中，传统方法通常依赖“思维链” (Chain-of-Thought, CoT) 将思考步骤逐一解码为离散的 Token，但这往往会带来巨大的计算开销与显存瓶颈。为了突破这一限制，近期的连续思维与循环架构尝试将推理过程直接置于固定维度的隐状态向量中，利用向量的“叠加”能力同时表征多种备选思考。然而，一个根本性的理论难题在于：随着推理逐步推进，连续思维向量究竟该保留哪些历史信息？本文针对这一核心问题开展了严谨的理论与容量分析，得出了一项反直觉的关键发现：相比于只保留最新的即时推理前沿，将全部历史推导累积叠加进向量中，在面对相同的下游计算时反而能够显著降低所需的表征维度。其深层原因在于，包含有效信息的历史成分在向量空间中能够相干互强，而无关的备选项只会产生相互抵消的随机噪声。此外，研究还证明了在未来信息效用未知时，“均匀累积加权”具有极小化极大 (Minimax) 最优性。这项工作成功将隐空间中的叠加态从一种经验观察升华为构建高效、稳健神经推理系统的严谨架构设计原则。 --- # 一个向量能容纳多少思维？叠加推理的表征容量 > # How Many Thoughts Can a Vector Hold? The Capacity of Reasoning by Superposition * **arXiv 编号：** [2609.13747](https://arxiv.org/abs/2609.13747) [cs.AI] * **作者：** Hongyu Gu, Chang Liu, Jingwen Fu * **提交时间：** 2026年9月12日 * **主要学科：** 人工智能 (`cs.AI`) * **备注说明：** 20 页，4 张图表 > * **arXiv ID:** [2609.13747](https://arxiv.org/abs/2609.13747) [cs.AI] > * **Authors:** Hongyu Gu, Chang Liu, Jingwen Fu > * **Submitted:** September 12, 2026 > * **Primary Subject:** Artificial Intelligence (`cs.AI`) > * **Comments:** 20 pages, 4 figures --- ## 摘要概要 > ## Abstract Summary 大语言模型 (Large Language Model, LLM) 在解决复杂难题时，通常采用多步推理机制，将中间的思考推导过程显式编码为一个接一个的离散 Token，这就是我们熟知的思维链 (Chain-of-Thought, CoT)。然而，近期涌现的连续与循环计算方法，尝试将部分推理计算转移到固定维度的隐状态中，使得单个向量能够同时叠加多种可能的备选思维。 > Large language models typically solve complex problems through multi-step reasoning by encoding intermediate computations as discrete tokens (Chain-of-Thought). However, recent continuous and recurrent methods shift partial computations into fixed-dimensional latent states where a single vector can superpose multiple alternative thoughts simultaneously. 针对这一新兴范式，本文深入探究了一个底层的核心设计问题：**随着推理的步步推进，连续思维向量究竟应当保留哪些信息？** > This paper investigates a fundamental design question: **What should continuous thoughts preserve as reasoning proceeds?** * **反直觉的核心发现：** 直觉通常认为，与仅保留当前的即时推理前沿相比，将全部推理历史都保留在向量中 (即累积叠加) 会稀释向量状态并浪费表征容量；但作者的研究表明，在完成相同下游计算任务时，**累积叠加实际上反而只需要更低的表征维度**。 * **背后的作用机制：** 携带有价值信息的历史思维成分在向量空间中能够彼此相干增强，而无关的备选项则只会带来相互抵消的随机干扰。 * **架构设计准则：** 论文深入探讨了在未来信息效用未知的情况下，模型该如何为隐状态中累积的记忆赋予权重。虽然优先考虑近期或显著的信息看似合理，但容易造成“弱表征”瓶颈；而**均匀累积加权**能够彻底避免这一缺陷，并被证明在保障未来稳健推理方面具有**极小化极大最优性 (Minimax-Optimal)**。 > * **The Counter-Intuitive Finding:** While intuition suggests that retaining full reasoning history (cumulative superposition) would dilute states and waste representational capacity compared to keeping only the immediate reasoning frontier, the authors demonstrate that **cumulative superposition can actually require lower representational dimensions** under identical downstream computations. > * **The Mechanism:** Informative historical components coherently reinforce one another, whereas unrelated alternatives introduce random interference. > * **The Design Principle:** The paper addresses how models should weight memories accumulated inside latent states when future utility is unknown. While prioritizing recent or salient items creates weak-representation bottlenecks, **uniform cumulative weighting** avoids this flaw and is proven to be **minimax-optimal** for robust future reasoning. 总的来说，这项研究将隐空间中的“叠加态”现象，从一种单纯被观察到的神经网络经验现象，升华为了指导高效、高可靠神经计算的严谨架构设计原则。 > Ultimately, this research turns superposition from a mere observed latent-space phenomenon into a rigorous design principle for efficient, reliable neural computation. --- ## 文档元数据 > ## Document Metadata | 元数据字段 | 详情 | | :--- | :--- | | **引用方式** | `arXiv:2609.13747 [cs.AI]` | | **DOI 链接** | [10.48550/arXiv.2609.13747](https://doi.org/10.48550/arXiv.2609.13747) | | **许可协议** | [知识共享署名-非商业性使用-相同方式共享 4.0 国际 (CC BY-NC-SA 4.0)](http://creativecommons.org/licenses/by-nc-sa/4.0/) | > | Metadata Field | Details | > | :--- | :--- | > | **Cite As** | `arXiv:2609.13747 [cs.AI]` | > | **DOI** | [10.48550/arXiv.2609.13747](https://doi.org/10.48550/arXiv.2609.13747) | > | **License** | [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International](http://creativecommons.org/licenses/by-nc-sa/4.0/) | --- ## 全文与参考文献链接 > ## Full-Text & References Links * **论文获取：** * [查看 PDF](https://arxiv.org/pdf/2609.13747) * [HTML 版本 (实验性)](https://arxiv.org/html/2609.13747v1) * [TeX 源码](https://arxiv.org/src/2609.13747) * **外部文献工具与学术引用：** * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.13747) * [Google 学术 (Google Scholar)](https://scholar.google.com/scholar_lookup?arxiv_id=2609.13747) * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.13747) > * **Access Paper:** > * [View PDF](https://arxiv.org/pdf/2609.13747) > * [HTML Version (Experimental)](https://arxiv.org/html/2609.13747v1) > * [TeX Source](https://arxiv.org/src/2609.13747) > * **External Bibliographic Tools & Citations:** > * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.13747) > * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.13747) > * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.13747) --- *注：本文档包含衍生自仓库构建流水线的许可证图标 (例如：`` )。* > *Note: This document incorporates license graphics derived from the repository pipeline (e.g., ``).*

mKernel：面向多 GPU 与多节点的高性能融合算子库

Wed, 16 Sep 2026 00:00:00 GMT

随着大语言模型与前沿 AI 架构的参数规模不断攀升，单张显卡早已不堪重负，分布式训练与多卡推理已成为现代深度学习系统的基石。然而，跨 GPU 以及跨服务器节点之间的数据通信，往往是制约整体运行速度的严重瓶颈。传统方案通常在算子级别通过独立流来实现计算与通信重叠，带来的性能增益十分有限；而此前更高效的融合算子又普遍局限于单台机器内部的 NVLink 互联范围。为此，研究团队推出了 **mKernel** 算子库，首次在分块 (Tile) 粒度上实现了计算、节点内 NVLink 通信与跨节点 RDMA 网络传输的无缝流水线重叠。在 16 卡 H200 集群上的测试表明，mKernel 在 GEMM+AllReduce 和 Ring Attention 等核心算子上分别取得了高达 1.72 倍和 1.88 倍的显著加速，为大规模分布式 AI 系统的性能调优开辟了全新路径。 --- ## 📌 概述 > ## 📌 Summary 在训练和部署庞大的机器学习模型时，多卡之间的数据通信开销往往是拖慢整体运行速度的头号瓶颈。传统的系统级优化方法通常是在粗粒度的算子级别尝试重叠通信与计算 (例如分配不同的 CUDA 流) ，但这种方式能够挤出的性能提升十分有限。相比之下，融合算子 (Fused Kernel) 的表现更为亮眼——它能在 GPU 刚算完一小块数据 (Tile) 时就立刻将其发走，从而极大掩盖通信延迟；然而，此前的融合算子技术基本都被局限在单台服务器内部的 NVLink 高速互联范围内，无法直接扩展到多机集群的广阔天地。 > Communication bottlenecks significantly hinder the distributed training and inference of large machine learning models. Traditional approaches that overlap communication with computation at the kernel granularity (using separate streams) yield only limited performance improvements. While fused kernels perform better by transmitting output tiles immediately upon production, their adoption has largely been restricted to a single NVLink domain. 为了打破这一限制，本文推出了全新的多 GPU、多节点融合算子库 **mKernel**。它专为大规模分布式集群环境打造，首次在细粒度的“数据分块 (Tile) ”层面上，将核心计算任务、单机节点内的 NVLink 互联通信以及节点间的跨机 RDMA 网络传输无缝重叠在一起。 > This paper introduces **mKernel**, a novel library of multi-GPU, multi-node fused kernels designed to seamlessly overlap computation, intra-node NVLink communication, and inter-node RDMA at tile granularity.

OpWeave：面向异构大模型推理服务的灵活算子级解耦框架

Wed, 16 Sep 2026 00:00:00 GMT

在当今大语言模型 (Large Language Model, LLM) 的云端推理部署中，传统架构习惯将注意力机制 (Attention) 与前馈神经网络 (FFN/MoE) 等不同算子捆绑在同规格芯片上同机运行。然而，这两类核心算子在计算特征上存在天然分化：注意力机制重度受限于显存带宽，而前馈网络则极度消耗计算算力，导致高昂的 GPU 资源往往无法被充分利用。针对这一工业落地瓶颈，研究团队推出了端到端算子级解耦推理服务框架 OpWeave，首次实现了跨异构算力环境的灵活解耦调度。OpWeave 创新性地结合了严谨的理论成本模型、可高效应对混合注意力架构的规律感知规划器，以及基于 vLLM 的高性能执行运行时，打破了以往系统僵化的算子切分边界。在严格满足端到端延迟服务等级目标 (Service Level Objective, SLO) 的前提下，OpWeave 在同构与异构 GPU 集群上分别实现了高达 1.78 倍与 1.89 倍的服务成本削减，为大模型时代构建高性价比的基础设施开辟了全新路径。 --- ## 核心概述 > ## Summary **OpWeave** 是一套专为大语言模型 (Large Language Model, LLM) 基础设施打造的端到端框架，专门用于实现异构环境下的**算子级解耦服务 (Operator-Level Disaggregated Serving, ODS)**。 > **OpWeave** is an end-to-end framework designed for heterogeneous **Operator-Level Disaggregated Serving (ODS)** in Large Language Model (LLM) infrastructures. 传统的 LLM 推理服务系统习惯将所有计算算子集中部署在同一类芯片上同机运行；而前沿的系统优化方案正逐步将推理流水线拆解为更细粒度的阶段——例如在 Token 生成的解码 (Decode) 阶段，将注意力机制 (Attention) 与前馈网络 (Feed-Forward Network, FFN) 或混合专家网络 (Mixture of Experts, MoE) 的计算彻底分离开来，以此更好地契合不同硬件的性能特长，并支持各算子模块的独立弹性扩缩容。然而，现有的工业实现方案大多受制于固化僵硬的算子切分边界，且缺乏一套统一的分析框架来量化评估“何时进行解耦才真正划算与最具性价比”。 > While traditional LLM serving systems colocate all operations, modern optimizations increasingly disaggregate inference into finer-grained stages (such as separating attention from FFN or MoE execution during the decode phase) to improve hardware matching and enable independent scaling. However, existing implementations suffer from rigid operator boundaries and lack unified frameworks to determine when disaggregation is cost-effective. 针对上述核心挑战，OpWeave 依托三大核心组件给出了系统性解决方案： > OpWeave addresses these challenges through three main components: 1. **理论分析成本模型 (Analytical Cost Model)**：从数学原理上严格推导并界定了同构与异构 ODS 相比传统同机部署服务的效率收益理论边界。 2. **规律感知规划器 (Regularity-Aware Planner)**：联合协同优化算子切分策略与底层集群部署配置，即便面对架构复杂的混合注意力 (Hybrid-Attention) 模型，也能确保庞大的解空间保持高效可解。 3. **基于 vLLM 的执行运行时 (vLLM-Based Runtime)**：能够在各类异构设备资源池中平滑无缝地调度并执行带有动态算子阶段的自动化编排计划。 > 1. **Analytical Cost Model:** Mathematically bounds the efficiency gains of both homogeneous and heterogeneous ODS over traditional colocated serving. > 2. **Regularity-Aware Planner:** Jointly optimizes operator partitioning and deployment configuration, ensuring the search space remains tractable even for complex hybrid-attention architectures. > 3. **vLLM-Based Runtime:** Executes synthesized plans seamlessly across diverse device groups with dynamic operator stages.

Meta 推出 ZGateway：统一 ZippyDB 流量、每秒处理超 10 亿次请求的无状态代理层

Wed, 16 Sep 2026 00:00:00 GMT

在超大规模分布式架构中，客户端直连底层数据库集群极易导致连接数呈爆炸式膨胀。在 Meta 内部，支撑全系产品元数据、计数器与配置系统的分布式键值存储 ZippyDB，曾面临着上百万台客户端主机直连带来的严重连接蔓延 (Connection Sprawl) 与重连风暴危机。为了从根本上化解这一工程瓶颈，Meta 研发并上线了无状态代理层 ZGateway。它不仅解耦了客户端与底层存储节点，将单机连接数大幅缩减 97% 以上，更逐步演进为一个集动态请求合流、自适应负载分流、内存读缓存与跨地域容灾于一体的综合流量底座。如今，ZGateway 以仅约 6% 的极低计算开销，稳定承载着 ZippyDB 全网约 40% 的数据流量，每秒处理操作数突破 10 亿大关，为超大规模基础设施的代理层设计树立了全新标杆。 --- ## 📌 执行摘要 > ## 📌 Executive Summary Meta 正式发布了 **ZGateway**——这是一个架设在客户端应用程序与 [ZippyDB](https://engineering.fb.com/2021/08/06/core-infra/zippydb/) 之间的强大无状态代理层 (Stateless Proxy Tier)。ZippyDB 作为 Meta 内部广泛使用的分布式键值存储 (Key-Value Store)，承载着全系产品的核心元数据、计数器和配置系统。ZGateway 最初的诞生，是为了彻底解决上百万台客户端主机直连数据库所带来的严重连接蔓延 (Connection Sprawl) 难题；而如今，它已进化为一个功能完备的综合性流量平台，全面接管了请求批处理 (Batching)、准入控制 (Admission Control)、内存缓存 (Caching) 以及故障转移 (Failover) 等核心能力。目前，ZGateway 每秒处理的操作数已突破 **10 亿次**，在仅产生约 6% 极低计算开销的前提下，承载了整个 ZippyDB 全网约 40% 的流量。 > Meta has introduced **ZGateway**, a powerful stateless proxy tier positioned between client applications and [ZippyDB](https://engineering.fb.com/2021/08/06/core-infra/zippydb/), Meta's widely used key-value store powering product metadata, counters, and configurations. Originally designed to resolve severe connection sprawl across more than a million client hosts, ZGateway has evolved into a comprehensive platform handling batching, admission control, caching, and failover. Today, it processes over **1 billion operations per second** and carries roughly 40% of all ZippyDB traffic at a minimal computational overhead of about 6%. --- ## 为什么 ZippyDB 急需引入代理层 > ## Why ZippyDB Needed a Proxy 在传统的客户端直连架构 (Direct-Access Architecture) 下，每一个 ZippyDB 客户端都必须与它需要访问的所有数据库主机直接建立连接。由于单个客户端在业务周期内可能需要触达分散在数十万台主机上的数万个分片 (Shards)，导致客户端与数据库节点双方都背负了极其沉重的包袱——单机往往需要维系成千上万条持久的 TLS 连接。 > Under a direct-access architecture, every ZippyDB client directly connected to every database host it needed to reach. Because a single client could touch tens of thousands of shards across hundreds of thousands of hosts, both clients and database hosts carried massive burdens of tens of thousands of TLS connections. * **资源严重枯竭 (Resource Exhaustion)：** 哪怕只是空闲的静默连接，在链路两端也会持续霸占宝贵的内存、CPU 周期以及文件描述符 (File Descriptors)。 * **重连风暴频发 (Reconnection Storms)：** 随着客户端实例集群规模的膨胀，数据库端的入站连接数呈指数级蹿升，多次因文件描述符耗尽和内存溢出 (Out-Of-Memory, OOM) 导致服务进程直接崩溃。 * **灾难性的无限重启循环 (The Reboot Loop Incident)：** 在一次著名的系统事故中，一个路由 Bug 导致全网所有客户端对每一个分片都强行建立了一条独立连接，瞬时爆发的连接洪峰直接将整个数据库集群拖入了灾难性的死循环重启。 * **客户端治理寸步难行 (Client-Side Hurdles)：** 企图在客户端侧解决此问题完全不可行，因为全公司有数百个相互独立的业务团队各自维护着不同技术栈的客户端代码，推动统一改造犹如天方夜谭。 > * **Resource Exhaustion:** Each idle connection consumed valuable memory, CPU, and file descriptors on both ends. > * **Reconnection Storms:** Inbound connection counts grew with every client cohort, leading to crashes from file descriptor exhaustion and out-of-memory (OOM) errors. > * **The Reboot Loop Incident:** During one notable incident, a routing bug caused every client to open a connection per shard, throwing the entire database fleet into a catastrophic reboot loop. > * **Client-Side Hurdles:** Fixing this on the client side was completely impractical because hundreds of independent teams owned various parts of the client fleet. --- ## 什么是 ZGateway？ > ## What is ZGateway? [ZGateway](https://engineering.fb.com/2026/09/03/core-infra/zgateway-proxy-zippydb-meta/) 是部署在 ZippyDB 客户端与底层 ZServer 数据库集群之间的一个无状态代理层。它作为区域服务层运行，通过 Meta 的自研服务网格 (Service Mesh) [ServiceRouter](https://atscaleconference.com/servicerouter-hyperscale-service-mesh-at-meta/) 实现服务发现与路由，主要提供两种工作形态：**纯透明代理 (Pure Proxy)** 与**直读缓存 (Read-Through Cache)**。 > [ZGateway](https://engineering.fb.com/2026/09/03/core-infra/zgateway-proxy-zippydb-meta/) is a stateless proxy tier deployed between ZippyDB clients and the ZServer database fleet. Running as regional tiers discovered via Meta’s service mesh, [ServiceRouter](https://atscaleconference.com/servicerouter-hyperscale-service-mesh-at-meta/), it operates in two primary flavors: **a pure proxy** and **a read-through cache**. 在底层实现上，ZGateway 直接复用了 Meta 高度优化的厚客户端 (Thick Client) C++ ZippyDB 库，本质上相当于将一个功能强大的 ZippyDB 客户端封装成了一项云原生托管服务。 > Under the hood, ZGateway utilizes Meta's thick C++ ZippyDB client, effectively operating as a ZippyDB client run as a managed service.

基于 LoRA 与 HF Jobs 的异步 GRPO 训练：对象存储、代理与零 NCCL 实践

Tue, 15 Sep 2026 00:00:00 GMT

在大语言模型推理能力对齐与强化学习 (Reinforcement Learning, RL) 探索中，群体相对策略优化 (Group Relative Policy Optimization, GRPO) 已成为提升模型复杂推理能力的关键方法，但传统的分布式多节点训练往往严重依赖昂贵且配置繁琐的 NCCL 高速互联网络与跨节点共享文件系统。针对这一工程痛点，本文介绍了一种轻量优雅的分布式解耦训练方案：结合 TRL 库最新支持的 AsyncGRPOTrainer 与低秩适应 (Low-Rank Adaptation, LoRA) 技术，在相互网络隔离的 Hugging Face Jobs 容器环境中开展异步强化学习。利用仅数兆字节大小的 Rank-1 权重只需通过挂载的云端对象存储桶 (Storage Bucket) 进行同步，配合定制的反向代理服务器实现基于 KV 缓存前缀的智能路由与状态广播，彻底摆脱了复杂的跨节点 NCCL 通信依赖。通过进一步优化微批次打包、禁用梯度检查点并增大并发在飞请求上限，训练耗时从 3 小时 27 分钟大幅缩短至 53 分钟，获得了高达 3.9 倍的速度提升。 --- # 基于 LoRA 与 HF Jobs 的异步 GRPO 训练：对象存储、代理与零 NCCL 实践 > # Async GRPO with LoRA across HF Jobs: A Bucket, a Proxy, and No NCCL ## 核心概要 > ## Summary 本文探讨了如何借助 **AsyncGRPOTrainer** 与 **LoRA** (Low-Rank Adaptation) 低秩适应技术，在分布式 **Hugging Face Jobs** 之间运行异步强化学习 (Reinforcement Learning, RL) 训练，且全程无需依赖跨节点的共享网络文件系统或复杂的 NCCL 集群通信。 > This article explores how to run asynchronous Reinforcement Learning (RL) training via **AsyncGRPOTrainer** using **LoRA** (Low-Rank Adaptation) across distributed **Hugging Face Jobs** without relying on a shared network file system or NCCL. 本文的核心亮点包括： - **轻量级权重同步**：由于 Rank-1 的 LoRA 适配器仅有区区几兆字节大小，我们可以轻松通过各个独立 Job 间挂载的**对象存储桶 (Storage Bucket)** 进行文件同步，从而完全避开繁重的多节点网络直接通信。 - **代理路由与状态广播**：通过一个定制的代理服务器统一处理认证请求头，将生成采样任务 (Rollouts) 智能路由到匹配其 KV 缓存 (KV-cache) 前缀的副本节点，并将最新的适配器更新广播同步给所有副本。 - **显著的性能飞跃**：通过精细调优训练微批次 (Microbatches)、关闭梯度检查点 (Gradient Checkpointing) 并提高在飞请求 (In-flight) 上限，500 步的整体训练时间从 **3 小时 27 分钟锐减至 53 分钟**（取得了高达 3.9 倍的加速比）。 > Key highlights include: > - **Lightweight Synchronization:** Because rank-1 LoRA adapters are only a few megabytes, they can easily sync via a **Storage Bucket** mounted across separate jobs, avoiding heavy multi-node communication. > - **Proxy Routing & Broadcasting:** A custom proxy server handles authentication headers, routes rollouts to replicas matching their KV-cache prefix, and broadcasts adapter updates to all replicas. > - **Performance Gains:** By tweaking training microbatches, disabling gradient checkpointing, and raising in-flight limits, training time for 500 steps drops from **3 hours 27 minutes down to 53 minutes** (a 3.9× speedup). --- ## 背景引言 > ## Introduction 随着 [PR #7017](https://github.com/huggingface/trl/pull/7017) 的合并（并在 TRL v1.14 版本中正式发布），TRL 库的 [`AsyncGRPOTrainer`](https://huggingface.co/docs/trl/en/async_grpo_trainer) 迎来了对 LoRA 的原生支持。现在的异步训练器可以只针对轻量级 LoRA 适配器展开训练，无需更新完整模型，并且仅需将适配器参数实时同步到 vLLM 推理引擎中。本文将详细介绍一个基于该特性搭建的真实工程项目，在该项目中，模型的训练与推理彻底解耦，不再共享同一台物理机。 > LoRA support recently landed in TRL's [`AsyncGRPOTrainer`](https://huggingface.co/docs/trl/en/async_grpo_trainer) with [PR #7017](https://github.com/huggingface/trl/pull/7017) (shipped in TRL v1.14). The asynchronous trainer can now train a LoRA adapter instead of the full model, syncing only the adapter to vLLM. This post covers a real-world project built on top of it, where training and inference no longer share a machine. LoRA 训练在强化学习场景中表现出奇契合。正如 Thinking Machines 在其博文 [LoRA Without Regret](https://thinkingmachines.ai/blog/lora/) 中所揭示的，在基于策略梯度的强化学习中，哪怕仅使用 Rank 1 的 LoRA，其效果也能与全量微调 (Full Fine-Tuning) 旗鼓相当。背后的数学机理在于：每个回合中优势函数 (Advantage Function) 仅能提供大约 `~O(1)` 比特的信息增益，因此极低秩的 Rank-1 适配器便具备足够的容量去吸收并掌握这些信号。 > LoRA training is particularly suited for RL, as shown in Thinking Machines's blog [LoRA Without Regret](https://thinkingmachines.ai/blog/lora/). They demonstrate that LoRA can match full fine-tuning for policy-gradient RL, even with rank 1. This stems from the fact that the advantage function only gives `~O(1)` bits of information per episode, so a rank-1 adapter has enough capacity to absorb it. 从分布式系统工程的视角来看，一个 1.5B 参数量模型的 Rank-1 适配器文件仅有区区几兆字节，而完整模型权重则高达约 3 GB。我们无需在每次迭代后传输庞大的完整模型权重，仅需分发极小的适配器即可。此外，vLLM 支持同时在显存中保留多个 LoRA 适配器：旧的生成采样任务可以使用其启动时对应的旧策略从容执行完毕，而全新的采样任务则能无缝切换并使用最新的策略权重。 > From a systems perspective, a rank-1 adapter for a 1.5B model is a few megabytes, whereas the full model is ~3 GB. Instead of sending full weights after every update, we send just the adapter. Furthermore, vLLM can keep several adapters loaded at once: old rollouts finish with the policy they started with, while new rollouts use the latest one.

几乎零开销的状态-预测解耦架构

Tue, 15 Sep 2026 00:00:00 GMT

在传统自回归大语言模型 (Large Language Model, LLM) 中，隐藏状态往往要同时承担两项相互冲突的繁重任务：一方面需要高度压缩并总结历史上下文，另一方面又必须竭力预测下一个 Token。状态-预测解耦 (State–Prediction Separation, SPS) 技术虽然通过将单次前向传播拆分为“状态流”与“预测流”成功化解了这一矛盾，但以往方案却带来了约 1.9 倍的惊人预训练计算开销。本文提出了一套革新机制，借助巧妙的“免费暂停 Token” (Free Pause Token) 设计，使预测流在推理过程中不写入任何键值缓存 (KV Cache) 且不占用额外上下文位置，真正实现了推理阶段的近乎零开销。同时，配合双阶段拆分、共享门控 FFN 等多项训练期优化，在几乎不增加端到端耗时的情况下全面提升了模型预测精度，为大模型基础架构设计开辟了兼顾高收益与低成本的新路径。 --- ## 📌 内容概要 > ## Summary 状态-预测解耦 (State–Prediction Separation, SPS) 是一项旨在减轻语言模型双重竞争负担的架构技术——通过将模型的前向传播过程拆分为“状态流”与“预测流”，使模型无需同时兼顾“总结历史上下文”与“预测下一个 Token”这两个相互掣肘的目标。然而，传统的 SPS 方法计算代价高昂，通常需要消耗标准预训练浮点运算量 (FLOPs) 的约 $1.9\times$。 > State–prediction separation (SPS) is a technique that relieves language models of competing burdens—summarizing the context versus predicting the next token—by splitting the forward pass into a state stream and a prediction stream. However, traditional SPS is computationally expensive, typically costing roughly $1.9\times$ the standard pretraining FLOPs. 本文提出了一系列优化机制，不仅让状态-预测解耦在推理阶段变得**近乎零开销**，同时还大幅削减了训练阶段的额外负担： > This paper introduces mechanisms to make state–prediction separation **almost free** during inference while drastically reducing training overhead: * **免费暂停 Token (Free Pause Token) ：** 构建一种完全不向显存写入任何键值 (Keys / Values) 的预测流，使其直接复用序列中现有的位置，既不会膨胀上下文长度，也无需额外的 KV 缓存 (KV Cache) 或解码步数。 * **训练期加速优化 (Training Optimizations) ：** 引入两阶段拆分算法 (完整保留高效的 FlashAttention 内核算子) 、$w=0$ 预测窗口、共享门控前馈网络 (Shared Gated FFN) 以及仅在训练尾声分阶段引入解耦策略。 > * **Free Pause Token:** A prediction stream that writes no keys or values, allowing it to leverage existing sequence positions without increasing context length, requiring a KV cache, or adding decoding steps. > * **Training Optimizations:** Incorporates a two-pass split (preserving FlashAttention kernels), a $w=0$ prediction window, a shared gated FFN, and phasing the separation toward the tail of training. 这些优化方案协同发力，在等浮点算力 (Isoflop) 、等参数量 (Isoparameter) 与等训练 Token 量 (Isotoken) 的公平对比下，相比标准按次词预测训练的 Transformer 实现了显著性能飞跃，同时将实际物理耗时与硬件开销压至最低。 > Together, these adjustments achieve an *isoflop*, *isoparameter*, and *isotoken* improvement over standard next-token-trained transformers while minimizing wall-clock and hardware overhead. --- ## 🔍 论文摘要 > ## Abstract 状态-预测解耦 (SPS) 通过将前向传播划分为状态流与预测流，有效卸下了语言模型隐藏状态肩负的两大对立重担——即压缩上下文语义与预测下一个 Token。这种解耦固然收效显著，但代价也极其高昂：预测流本质上是对整个主干网络的二次前向遍历，使得预训练所需的 FLOPs 膨胀至约 $1.9\times$；而在采用灵活的注意力掩码时，实际训练物理耗时更是进一步恶化。 > State–prediction separation (SPS) relieves a language model's hidden state of two competing burdens—summarizing the context and predicting the next token—by splitting the forward pass into a state stream and a prediction stream. The separation works, but it is expensive: the prediction stream is a second pass over the whole backbone, costing $\sim$$1.9\times$ the pretraining FLOPs, and even more in terms of wall-clock time when using a flexible attention mask. 本篇论文致力于让状态-预测解耦变得近乎免费。我们将解耦设计推向极致，提出了“免费暂停 Token”机制：构建一种完全不写入任何键 (Keys) 或值 (Values) 的预测流，从而顺理成章地“搭便车”复用序列现有的位置坐标。在 10 亿 (1B) 参数规模的模型实测中，该机制为标准 Transformer 的下一个 Token 预测精度带来了 2 到 3 centinats 的切实提升；而且因为它没有引入任何额外的位置开销，因而在推理端完全没有代价——既不增加上下文长度，无需消耗 KV 缓存，也无需多余的解码步骤，延迟几乎毫无增加；至于推理阶段轻微增加的理论浮点计算量，由于当前推理吞吐的核心瓶颈并不在计算本身，因此基本可以忽略不计。 > This paper makes state–prediction separation almost free. We take the separation to its limit with a free pause token: a prediction stream that writes no keys or values at all and so rides the sequence's existing positions. It improves next-token prediction of a standard Transformer by 2-3 centinats in practice on a 1B parameter model, and because it adds no position it costs nothing at inference—no added context length, no KV cache, no decode steps, and essentially no latency, with the growth in inference flops typically irrelevant as it is not the active bottleneck on throughput. 如此一来，所有的计算额外开销便被完整转移并浓缩在训练阶段。为此，我们设计了四重核心机制来极限压缩训练成本： > The cost is therefore entirely in training where we use four mechanisms to drive it down: 1. **两阶段拆分机制 (Two-Pass Split) ：** 确保高效的 FlashAttention 内核算子始终可用； 2. **$w{=}0$ 预测窗口 ($w{=}0$ Prediction Window) ：** 极限裁剪预测流所关注的上下文窗口； 3. **共享门控 FFN (Shared Gated FFN) ：** 每个物理位置仅计算一次前馈网络，而非每个数据流各自重复计算； 4. **尾程分阶段解耦 (Phasing Separation) ：** 将解耦机制巧妙安排在预训练运行周期的尾声阶段切入。 > 1. A two-pass split that keeps FlashAttention kernels viable, > 2. The $w{=}0$ prediction window, > 3. A shared gated FFN that evaluates one FFN per position rather than one per stream, and > 4. Phasing the separation onto the tail of the run. 多项优化齐头并进，使得相对于高度优化的预训练基线管线，整体物理训练时间开销被大幅压低至 $1.33\times$，同时成功挽回了经典 SPS 约 94% 的性能增益；在更温和平滑的“质量/算力”权衡曲线下，训练耗时增幅甚至可以低至 $1.09\times$。此外，共享 FFN 优化还同步削减了推理所需的原始浮点运算量。最终，该研究达成了在等算力 (Isoflop) 、等参数 (Isoparameter) 以及等 Token (Isotoken) 条件下，相对于标准单次预测 Transformer 的全方位实质突破。 > Together these bring the overhead versus an optimized pretraining pipeline to $1.33\times$ wall-clock while recovering ~94% of the gain compared to SPS, and to as low as $1.09\times$ along a graceful quality/compute tradeoff. Furthermore, the FFN optimization reduces the raw flops required at inference time. The result is an isoflop, isoparameter, and isotoken improvement over standard next token trained transformers. --- ## 🔗 全文阅读与相关资源 > ## Links and Resources * **全文阅读链接：** * [阅读 PDF 论文](https://arxiv.org/pdf/2609.03807) * [网页版 HTML (实验性)](https://arxiv.org/html/2609.03807v4) * [TeX 源码包](https://arxiv.org/src/2609.03807) * **开源授权：** [知识共享署名 4.0 国际许可 (Creative Commons Attribution 4.0 International)](http://creativecommons.org/licenses/by/4.0/) > * **Full-Text Options:** > * [View PDF](https://arxiv.org/pdf/2609.03807) > * [HTML Version (Experimental)](https://arxiv.org/html/2609.03807v4) > * [TeX Source](https://arxiv.org/src/2609.03807) > * **License:** [Creative Commons Attribution 4.0 International](http://creativecommons.org/licenses/by/4.0/)

修复 NZXT Signal 4K30 采集卡 (第二部分)：攻克绿粉色画面显示异常

Tue, 15 Sep 2026 00:00:00 GMT

在成功修复了一块因电感虚焊而无法开机的二手 NZXT Signal 4K30 视频采集卡后，作者在接入特定 720p60 的 HDMI 信号源时遭遇了诡异的“绿粉色画面”异常。借助 AI (Claude) 的反编译辅助与硬件 UART 调试分析，作者顺藤摸瓜排查到设备芯片所采用的 ITE 官方驱动源码，精准锁定了因 DVI 模式信号识别错误导致误配 YUV 4:2:2 色彩格式的固件 Bug。通过逆向固件升级程序并仅修改 1 字节的汇编指令，作者成功修复了这一顽疾，让被官方弃用的老硬件重焕生机，并向社区公开了完整的免拆机刷机补丁。 --- # 修复 NZXT Signal 4K30 采集卡 (第二部分)：攻克绿粉色画面显示异常 > # Fixing an NZXT Signal 4K30, Part 2: The Green/Pink Video Bug ## 概要 > ## Summary 在通过重新焊接松脱的电感引脚、成功救活了一块在二手平台淘来的“故障” NZXT Signal 4K30 视频采集卡后，作者又发现了一个棘手的画面色彩 Bug：当接入某个特定的 720p60 HDMI 信号源时，采集到的视频画面居然严重偏色，呈现出失真的绿粉相间色调。在 AI (Claude) 的协助下，作者对该采集卡的固件以及板载 IT6805 HDMI 接收芯片的原厂驱动进行了深入逆向分析，最终将罪魁祸首锁定在 DVI 模式信号处理代码中的逻辑缺陷——该缺陷引发了 YUV 与 RGB 颜色模式的错误匹配。随后，作者通过逆向工程剖析了固件升级工具，仅通过修改底层机器码中的单个字节汇编指令便彻底解决了该问题，恢复了正常的视频色彩采集，并将这一补丁开源分享给了社区。 > After successfully repairing a dead, thrifted NZXT Signal 4K30 capture card by fixing a bad solder joint, the author discovered a stubborn color bug: a specific 720p60 HDMI source caused the captured video to appear in distorted green and pink hues. With the help of AI (Claude) to analyze the device's firmware and vendor drivers for the IT6805 HDMI receiver IC, the author traced the issue to a YUV vs. RGB mismatch caused by a bug in the handling of DVI-mode signals. By reverse-engineering the firmware updater and patching a single byte of assembly, the author successfully fixed the bug, restored proper color capture, and published the patch for the community. --- > --- ## 故障背景 > ## Background on the Issue 去年，我在 eBay 上廉价淘到了一台有故障的 NZXT Signal 4K30 USB 视频采集卡并[成功完成了硬件修复](https://www.downtowndougbrown.com/2025/01/easy-repair-of-a-defective-nzxt-signal-4k30-capture-card/)。在修好电路板上一颗接触不良的电感焊点、让设备顺利恢复供电后，我拿各种不同的 HDMI 设备对它进行了测试。正如我在上一篇文章中所提到的： > Last year, I [repaired an NZXT Signal 4K30 USB capture device](https://www.downtowndougbrown.com/2025/01/easy-repair-of-a-defective-nzxt-signal-4k30-capture-card/) bought cheaply on eBay. After fixing a bad solder joint on an inductor that restored power to the board, I tested it with various HDMI sources. As I noted in my initial post: > 💬 [原文引用 / Original Quote]: > 我确实遇到了一个这块卡“极不感冒”的 720p60 HDMI 信号源——采集出来的画面全变成了粉色和绿色。 > > I did find one 720p60 HDMI source that it doesn’t like — the captured video shows up as pink and green. 在网络论坛上，其他玩家在使用 PS5 和 Nintendo Switch 等游戏主机时也曾反馈过类似的画面偏色问题 (参见 Reddit 讨论帖 [1](https://www.reddit.com/r/NZXT/comments/17csgqa/picture_is_green_and_pink_from_signal_4k30/) 与 [2](https://www.reddit.com/r/obs/comments/16l8efg/pink_and_green_screen/))。下面就是从那个“刺头”信号源采集到的实际画面截图： > Similar issues had been reported by other users online regarding devices like the PS5 and Nintendo Switch (see Reddit threads [1](https://www.reddit.com/r/NZXT/comments/17csgqa/picture_is_green_and_pink_from_signal_4k30/) and [2](https://www.reddit.com/r/obs/comments/16l8efg/pink_and_green_screen/)). Here is what the captured video looked like from the problematic source:

画面色彩彻底错乱了——呈现出诡异的黄绿色和紫红色，这是极其典型的 RGB 与 YUV 视频色彩空间配置错位现象。鉴于 NZXT 似乎已经完全退出了视频采集卡市场 (各大主流零售渠道均已下架该设备，相关资料也被打入冷门支持归档)，指望联系官方售后出补丁显然是一条死胡同。 > The colors were completely wrong—green and purple—which is classic behavior for an RGB vs. YUV video mismatch. Because NZXT appears to have exited the capture card market (the device is no longer sold on major retailers and is relegated to support pages), contacting them for a fix was a dead end. --- > --- ## 借助 AI 展开逆向排查 > ## Investigating with AI 最近一段时间，我一直在尝试利用 Claude 协助进行深度的逆向工程与底层漏洞排查工作。例如之前为 Elgato Game Capture HD60 S 编写[通过逆向工程构建的 Linux 内核 V4L2 驱动](https://github.com/dougg3/hd60s-linux-driver) (当时我[借助 Ghidra 工具进行了反汇编分析](https://www.downtowndougbrown.com/2024/09/fixing-an-elgato-hd60-s-hdmi-capture-device-with-the-help-of-ghidra/))。于是我决定看看 Claude 能否帮我搞定这个困扰已久的固件幽灵。 > Recently, I've been using Claude for in-depth reverse engineering and bug investigations, such as writing a [reverse-engineered Linux kernel V4L2 driver](https://github.com/dougg3/hd60s-linux-driver) for the Elgato Game Capture HD60 S (which I [analyzed via Ghidra](https://www.downtowndougbrown.com/2024/09/fixing-an-elgato-hd60-s-hdmi-capture-device-with-the-help-of-ghidra/)). I decided to see if Claude could help solve this lingering firmware problem. 我给 Claude 提供了以下材料： * 在硬件维修过程中梳理出的设备元器件与芯片文档； * 来自 Reddit 社区关于该 Bug 的现象描述与截图； * [NZXT 于 2022 年发布的最终版固件更新包](https://support.nzxt.com/hc/en-us/articles/35642755429019-Signal-4K30-Downloads)； * Signal 4K30 采集卡内部采用的 [ITE IT6805 HDMI 接收芯片驱动代码仓库](https://github.com/Qiuzixing/IP5000-A302/tree/b7f61f612672301979b6f062129f8ffd0163c45d/modules/ast_modules/it6805)。 > I fed Claude: > * Documentation of the device's components gathered during the hardware repair. > * Descriptions and images of the bug from Reddit. > * [NZXT’s final 2022 firmware update](https://support.nzxt.com/hc/en-us/articles/35642755429019-Signal-4K30-Downloads). > * A GitHub repository containing [ITE’s driver for the IT6805 HDMI receiver IC](https://github.com/Qiuzixing/IP5000-A302/tree/b7f61f612672301979b6f062129f8ffd0163c45d/modules/ast_modules/it6805) used by the Signal 4K30. 不到 15 分钟，Claude 便赞同了我的猜想：这极大概率是 YUV 与 RGB 格式匹配错误引起的。为了进一步收窄排查范围，我使用便携式示波器找到了微控制器 (MCU) 未标明引脚定义的调试排针上的 TX (发送) 引脚，并在连入问题信号源时抓取了它的 UART 串口输出日志。结合我此前编写 Elgato HD60 S 驱动时积累的经验，我们赫然发现：**输入信号源设备当前居然是以 DVI 模式而非标准 HDMI 模式进行输出的**，这意味着传输的数据流中缺少了诸如 AVI InfoFrames 这类包含色彩元数据的附加数据包。 > Within 15 minutes, Claude agreed it was likely a YUV vs. RGB mismatch. To narrow it down, I used a portable oscilloscope to identify the TX pin on the microcontroller's unmarked debug header and captured its UART output while connected to the problematic source. Combined with details from my Elgato HD60 S driver, we discovered that **the source device was outputting in DVI mode instead of HDMI mode**, meaning it lacked extra data packets like AVI InfoFrames. --- > --- ## 在驱动源码中揪出根因 > ## Pinpointing the Bug in the Driver Code Claude 在 [ITE 原厂的 IT6805 芯片驱动源码](https://github.com/Qiuzixing/IP5000-A302/blob/b7f61f612672301979b6f062129f8ffd0163c45d/modules/ast_modules/it6805/iTE6805_SYS.c)中敏锐地发现了一段极度可疑的代码，而这段代码同样原封不动地存在于 NZXT 的固件中： > Claude highlighted a suspicious section in [ITE’s stock IT6805 driver](https://github.com/Qiuzixing/IP5000-A302/blob/b7f61f612672301979b6f062129f8ffd0163c45d/modules/ast_modules/it6805/iTE6805_SYS.c) that also existed in NZXT's firmware: ```c // REG6B[5:4]: Reg_ColMod_Set Input color mode set 00: RGB mode - 01: YUV422 mode, 10: YUV444 mode, 11: YUV420 mode chgbank(0); if (iTE6805_Check_HDMI_OR_DVI_Mode(iTE6805_DATA.CurrentPort) == MODE_HDMI) { HDMIRX_DEBUG_PRINT(("---- CSC HDMI mode ----\n")); ... hdmirxset(0x6B, 0x30, iTE6805_DATA.AVIInfoFrame_Input_ColorFormat << 4);// seting input format by info frame ??? do not need ??? ... } else { ... HDMIRX_DEBUG_PRINT(("---- CSC DVI mode ----\n")); hdmirxset(0x6B, 0x30, 0x10); // seting input format to RGB ... } ```

UltraQuant：面向重度上下文智能体的 4-bit KV Cache 压缩技术

Tue, 15 Sep 2026 00:00:00 GMT

在以 Claude Code 等为代表的多轮 AI 智能体 (AI Agent) 深度交互场景中，长系统提示与历史执行记录等长前缀被频繁复用，使得键值缓存 (KV Cache) 产生的显存占用呈爆炸式激增，成为限制 GPU 吞吐与高并发服务能力的头号瓶颈。针对这一极端内存受限的工业级部署难题，研究团队提出了创新的 4-bit KV 缓存压缩框架 **UltraQuant**。该方案创造性地结合了非对称 K/V 量化策略、Walsh-Hadamard 旋转变换以及针对 AMD CDNA4 架构量身定制的硬件加速解码内核，在削减一半 KV 显存开销的同时，保持了几乎无损的任务精度。真实生产环境重放测试表明，UltraQuant 在长上下文与高并发压力下实现了相比原生 BF16 最高 4.38 倍的有效请求吞吐量提升，为大规模落地智能体推理提供了高性价比的硬件级解法。 --- ## 📌 内容概要 > ## Summary **UltraQuant** 是一套专为在显存受限与高并发严苛条件下运行的“重度上下文” AI 智能体 (AI Agent) 所设计的全新 4 位键值缓存 (KV Cache) 压缩框架。通过将下游任务质量、缓存常驻留存率以及线上推理服务吞吐量进行联合端到端优化，UltraQuant 彻底破解了多轮智能体工作流中因超长文本前缀被反复调用而导致的巨大显存压力难题。 > **UltraQuant** is a novel 4-bit key-value (KV) cache compression framework designed specifically for context-heavy AI agents operating under memory-constrained, high-concurrency conditions. By jointly optimizing task quality, cache residency, and serving throughput, UltraQuant addresses the massive memory pressure introduced by multi-round agent workflows where long text prefixes are repeatedly reused. 其核心技术亮点包括： > Key technical highlights include: * **鲁棒的 4-bit 量化设计 (Robust 4-bit Design) ：** 采用针对 K 与 V 张量的非对称处理方案、Walsh-Hadamard 旋转变换、移除 QJL (Quick Johnson-Lindenstrauss) 以及分块缩放 (Block-Scale) 变体设计。 * **硬件加速服务优化 (Hardware-Accelerated Serving) ：** 引入了高度优化的解码注意力内核算子，并为基于 CDNA4 架构的 AMD GPU 量身打造了 FP4 近似计算路径 (支持 FP8 查询、FP4 KV 张量、UE8M0 分组缩放系数以及原生的 Scaled-MFMA 指令加速) 。 * **令人瞩目的性能飞跃 (Significant Performance Gains) ：** 在真实生产环境 Claude Code 的执行轨迹回放测试中，UltraQuant 相比原生 BF16 基线取得了高达 **2.71x** (在 MiniMax-M2.5 上) 和 **4.38x** (在 Qwen3-235B 上) 的达标请求吞吐量提升，在将 KV 显存开销直接砍半的同时，性能表现比肩甚至超越了硬件级 FP8 KV 缓存。 > * **Robust 4-bit Design:** Uses asymmetric K/V treatment, Walsh-Hadamard rotation, removal of QJL (Quick Johnson-Lindenstrauss), and block-scale variants. > * **Hardware-Accelerated Serving:** Introduces optimized decode-attention kernels and an FP4 approximation path tailored for AMD GPUs using CDNA4 architecture (featuring FP8 queries, FP4 KV tensors, UE8M0 group scales, and native scaled-MFMA support). > * **Significant Performance Gains:** Replays of production Claude Code traces show UltraQuant delivers **2.71x** (MiniMax-M2.5) and **4.38x** (Qwen3-235B) the qualified-request throughput of standard BF16 baselines, matching or exceeding hardware FP8 KV while cutting KV memory footprint in half. --- ## 📄 论文元数据 > ## Paper Metadata * **arXiv 标识符：** [arXiv:2606.20474](https://arxiv.org/abs/2606.20474) [cs.LG] * **论文作者：** Inesh Chakrabarti, David Limpus, Aditi Ghai Rana, Bowen Bao, Spandan Tiwari, Thiago Crepaldi, Ashish Sirasao * **主学科领域：** 机器学习 (`cs.LG`) * **次要学科领域：** 人工智能 (`cs.AI`)、系统性能 (`cs.PF`) * **收录会议/期刊：** EMNLP 2026 Industry Track (11 页，9 幅图表) * **提交历史版本：** * v1: 2026 年 6 月 18 日 * v3 (当前版本): 2026 年 9 月 11 日 > * **arXiv ID:** [arXiv:2606.20474](https://arxiv.org/abs/2606.20474) [cs.LG] > * **Authors:** Inesh Chakrabarti, David Limpus, Aditi Ghai Rana, Bowen Bao, Spandan Tiwari, Thiago Crepaldi, Ashish Sirasao > * **Primary Subject:** Machine Learning (`cs.LG`) > * **Secondary Subjects:** Artificial Intelligence (`cs.AI`), Performance (`cs.PF`) > * **Conference/Venue:** EMNLP 2026 Industry Track (11 pages, 9 figures) > * **Submission History:** > * v1: June 18, 2026 > * v3 (Current): September 11, 2026 --- ## 🔍 论文摘要 > ## Abstract 重度依赖上下文的智能体系统给键值缓存 (KV Cache) 带来了沉重的显存压力：长文本前缀在众多短轮次交互中被高频复用，而并发能力的高低直接决定了服务系统能否让 GPU 算力保持满载。针对这一核心应用场景，我们系统性地探索了 4-bit KV 缓存压缩技术，以 TurboQuant 风格的旋转变换与码本量化作为精度基准锚点，并以 vLLM 的 FP8 KV 缓存作为工业部署的工程锚点。本项研究主要包含三大核心贡献： > Context-heavy agents place substantial pressure on the key-value (KV) cache: long prefixes are reused across many short turns, while concurrency determines whether the serving system can keep GPUs utilized. We study 4-bit KV-cache compression for this setting, using TurboQuant-style rotation and codebook quantization as a quality anchor and vLLM FP8 KV caching as the deployment anchor. We report three contributions. 1. **工作负载建模 (Workload Framing) ：** 我们围绕多轮智能体工作负载重新构建了 4-bit KV 缓存的设计范式，在此范式下，任务完成质量、缓存驻留率与服务吞吐量必须作为一个整体进行联合评估。 2. **鲁棒实用的工程设计 (Robust Practical Design) ：** 我们阐述了使 4-bit 链路保持高鲁棒性所必需的工程设计抉择，包括针对 K/V 张量的非对称处理、Walsh-Hadamard 旋转变换、舍弃 QJL 方案以及引入分块缩放变体。 3. **硬件级服务推理优化 (Hardware Serving Optimizations) ：** 我们展示了针对 AMD GPU 的深度服务优化成果，包括高度优化的解码注意力内核，以及 UltraQuant 专属的 FP4 近似计算路径——该路径利用 FP8 查询、FP4 KV 张量、UE8M0 分组缩放因子，并在 CDNA4 硬件架构上提供了原生的 Scaled-MFMA 矩阵计算支持。 > 1. **Workload Framing:** We frame 4-bit KV caching around multi-round agent workloads where task quality, cache residency, and serving throughput must be measured jointly. > 2. **Robust Practical Design:** We describe the practical design choices needed to make the 4-bit path robust, including asymmetric K/V treatment, Walsh-Hadamard rotation, QJL removal, and block-scale variants. > 3. **Hardware Serving Optimizations:** We present serving optimizations on AMD GPUs, including optimized decode-attention kernels and UltraQuant, an FP4 approximation path that uses FP8 queries, FP4 KV tensors, UE8M0 group scales, and native scaled-MFMA support on CDNA4. 在对真实生产环境 Claude Code 追踪轨迹的自适应 SLO (Adaptive-SLO) 回放测试中，UltraQuant 相比 BF16 基线分别达成了 **2.71x** (搭载 MiniMax-M2.5) 与 **4.38x** (搭载 Qwen3-235B) 的达标请求吞吐量，在仅占用一半 KV 字节空间的情况下，性能比肩甚至超越了硬件级 FP8 KV 方案。UltraQuant 在长上下文、高并发且内存严重受限的服务场景中能够发挥出极其巨大的加速效益。 > On an adaptive-SLO replay of production Claude Code traces, UltraQuant sustains **2.71x** (MiniMax-M2.5) and **4.38x** (Qwen3-235B) the qualified-request throughput of the BF16 baseline, matching or exceeding hardware FP8 KV while using half the KV bytes. UltraQuant delivers its largest gains in long-context, high-concurrency, memory-constrained serving regimes. --- ## 🔗 访问与资源链接 > ## Access & Resources * **全文阅读链接：** * [阅读 PDF 论文](https://arxiv.org/pdf/2606.20474) * [网页版 HTML (实验性)](https://arxiv.org/html/2606.20474v3) * [TeX 源码包](https://arxiv.org/src/2606.20474) * **外部学术引用与检索工具：** * [Google 学术检索 (Google Scholar)](https://scholar.google.com/scholar_lookup?arxiv_id=2606.20474) * [Semantic Scholar 检索](https://api.semanticscholar.org/arXiv:2606.20474) * [NASA ADS 天体物理数据系统检索](https://ui.adsabs.harvard.edu/abs/arXiv:2606.20474) > * **Full-Text Options:** > * [View PDF](https://arxiv.org/pdf/2606.20474) > * [HTML Version (Experimental)](https://arxiv.org/html/2606.20474v3) > * [TeX Source](https://arxiv.org/src/2606.20474) > * **External Citations & Tools:** > * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2606.20474) > * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2606.20474) > * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2606.20474)

NVIDIA 开源 OSMO：一份 YAML 编排物理 AI 训练、仿真与真机测试

Tue, 15 Sep 2026 00:00:00 GMT

物理 AI (Physical AI) 与具身智能机器人的研发流程极其复杂，传统上往往割裂在数据中心训练集群、工作站物理仿真环境以及机器人机载边缘计算硬件这三大异构层级之间，迫使工程师维护大量繁重的胶水脚本和孤立调度器。为破解这一“三电脑”难题，NVIDIA 正式开源了 Kubernetes 原生的工作流编排工具——OSMO。开发团队只需编写一份简洁的 YAML 配置文件，即可在统一控制平面下无缝调度从分布式训练、物理仿真到硬件在环真机测试的端到端管线。该工具的开源大幅降低了异构算力编排的技术门槛，为物理 AI 与机器人系统的规模化工程落地注入了强大动力。 --- ## 概述 > ## Summary 在传统的机器人开发过程中，工程师们常常饱受“算力碎片化”的困扰：数据中心里的训练集群、工作站上的物理仿真环境，以及机器人机载的边缘计算硬件，往往各自为战，需要依赖五花八门的调度器和脆弱的“胶水脚本”强行串联。**NVIDIA OSMO** 正是为了化解这一痛点而生的开源云原生 (Kubernetes-native) 工作流编排调度工具。开发团队仅凭一份简洁的 YAML 配置文件，就能无缝协同跨越异构算力层级的物理 AI (Physical AI) 研发全链路——从模型训练、物理仿真再到硬件在环 (Hardware-in-the-Loop, HIL) 真机测试，彻底摆脱了编写繁琐底层基础设施代码的噩梦。 > Robot development traditionally suffers from a fragmented compute problem, requiring separate schedulers and glue scripts across data-center training clusters, workstation simulation environments, and edge hardware. **NVIDIA OSMO** solves this challenge as an open-source, Kubernetes-native workflow orchestrator. By leveraging a single YAML file, development teams can seamlessly orchestrate physical AI pipelines—spanning model training, physics simulation, and hardware-in-the-loop (HIL) robot testing—across heterogeneous compute tiers without writing complex infrastructure code. --- ## “三电脑”难题 > ## The Three Computer Problem NVIDIA 将物理 AI 的开发挑战形象地归纳为 **“三电脑”难题 (Three Computer Problem)**： > NVIDIA frames physical AI as a **three computer problem**: 1. **模型训练**：在数据中心级 GPU (如 GB200 或 H100 集群) 上大规模执行； 2. **物理仿真**：在工作站级 RTX 硬件上运行，用于物理动力学解算和传感器画面的高逼真渲染； 3. **部署与测试**：在 Jetson AGX Thor 等边缘端计算设备上运行，开展硬件在环验证。 > 1. **Training:** Executed on data-center GPUs (such as GB200 or H100 clusters). > 2. **Simulation:** Handled on workstation-class RTX hardware for physics and sensor rendering. > 3. **Deployment & Testing:** Run on edge devices like the Jetson AGX Thor for hardware-in-the-loop (HIL) validation. 在以往的工作流中，每个层级都高度依赖孤立的工具链，导致各阶段交接时摩擦不断，研发人员不得不编写大量脆弱的自定义编排脚本。**[NVIDIA OSMO](https://github.com/NVIDIA/OSMO)** 的破局思路在于：将这三大计算层级视为同一个统一控制平面下的不同后端资源。在工作流定义中，开发者只需指定特定的目标 *平台* (Platform) (例如 `gb200`、`rtx-pro-6000` 或 `jetson-agx-thor`)，便能彻底屏蔽底层繁杂的基础设施细节，由 OSMO 智能地将任务分发路由至最合适的计算资源池中。 > Traditionally, each tier relies on isolated tooling, resulting in friction-heavy handoffs and custom orchestration scripts. **[NVIDIA OSMO](https://github.com/NVIDIA/OSMO)** addresses this by treating all three tiers as backends under a unified control plane. Workflows abstract away underlying infrastructure by targeting a specific *platform* (e.g., `gb200`, `rtx-pro-6000`, or `jetson-agx-thor`), allowing OSMO to intelligently route tasks to appropriate compute pools. --- ## 工作流是如何运作的 > ## What a Workflow Looks Like 一个标准的 OSMO 管线可以在单份配置文件内无缝衔接不同阶段的串行任务与数据流： > A canonical pipeline links data across sequential tasks within a single configuration: * **`simulation`**：在 `rtx-pro-6000` 工作站硬件上运行 Isaac Sim 仿真容器； * **`train-policy`**：在由 8 块 GPU 组成的 `gb200` 集群上启动 PyTorch 训练容器，直接消费并读取上游仿真任务输出的数据； * **`evaluate-thor`**：将训练完成的策略部署到 `jetson-agx-thor` 边缘端设备上运行 ROS 应用程序，并将评测结果持久化记录到指定命名的数据集中。 > * **`simulation`**: Executes an Isaac Sim container on `rtx-pro-6000` hardware. > * **`train-policy`**: Runs a PyTorch container on an 8-GPU `gb200` cluster, consuming the simulation task's output. > * **`evaluate-thor`**: Deploys a ROS application on a `jetson-agx-thor` edge device using the trained policy, recording results to a named dataset. 在具体实现上，任务间的依赖关系通过 `inputs` 进行声明，数据持久化由 `outputs` 统筹，而硬件资源的精准分配则完全交由 `platform` 决定。根据 **[官方用户指南 (User Guide)](https://nvidia.github.io/OSMO/main/user_guide/)**，OSMO 还支持诸多高阶配置特性，包括串行与并行任务组、用于参数化管线的 Jinja 模板渲染、自动化失败重试策略，以及跨算力池的基于优先级的抢占式调度。 > Task dependencies are managed via `inputs`, persistence is handled through `outputs`, and resource allocation is determined by `platform`. The **[User Guide](https://nvidia.github.io/OSMO/main/user_guide/)** supports advanced configurations including serial and parallel task groups, Jinja templating for parameterized pipelines, automated retry policies, and priority-based preemption across compute pools. --- ## 核心能力 > ## Key Capabilities * **跨平台可移植性**：实现“一次编写，随处部署”。无论是在笔记本电脑上通过 Docker/KIND 进行本地调试，还是无缝横向扩展至 EKS、AKS、GKE 以及完全物理隔离 (Air-gapped) 的本地私有集群。在 6.3.0 版本中，OSMO 还引入了支持多云提供商的 `deploy-k8s.sh` 脚本以简化集群部署，并深度集成了 MinIO、Azure Blob 及 AWS S3 等主流存储方案。 * **交互式开发体验**：开发者可以直接将本地的 VS Code、Jupyter 或 SSH 会话挂载连接至远程 GPU 节点，使用 `exec` 交互式进入正在运行的容器任务，并通过 `osmo workflow rsync download` 命令实现文件的近实时双向同步。 * **先进的智能调度**：在 **[NVIDIA KAI Scheduler](https://github.com/NVIDIA/KAI-Scheduler)** 的强力驱动下，OSMO 针对多 GPU 工作负载支持 NVLink 拓扑感知的计算放置，并提供细粒度的组级超时控制，防止某项停滞卡死的任务阻塞其他关联工作流。 * **企业级安全与身份认证**：具备完备的企业级安全管控体系，内置基于角色的访问控制 (RBAC) 授权 Sidecar 容器、OAuth2 代理集成、Envoy 网关处的 TLS 终端终止，并原生支持公有云工作负载身份认证机制 (如 Azure Workload Identity 与 AWS IRSA)。 * **AI 智能体原生集成**：通过专门的 **[AGENTS.md](https://github.com/NVIDIA/OSMO/blob/main/AGENTS.md)** 指南和模型上下文协议 (Model Context Protocol, MCP) 部署路径提供开箱即用的原生支持，OSMO 能够与 **Claude Code、OpenAI Codex 以及 Cursor** 等主流编程智能体深度融合，实现工作流的自动化提交、运行监控与故障排查。 > * **Portability:** Write once and deploy locally via Docker/KIND on a laptop, or scale out to EKS, AKS, GKE, and air-gapped on-premise clusters. The 6.3.0 release introduces a multi-provider `deploy-k8s.sh` script to streamline provisioning alongside storage integrations for MinIO, Azure Blob, and AWS S3. > * **Interactive Development:** Developers can attach VS Code, Jupyter, or SSH sessions directly to remote GPU nodes, `exec` into active tasks, and utilize `osmo workflow rsync download` for real-time file synchronization. > * **Advanced Scheduling:** Powered by the **[NVIDIA KAI Scheduler](https://github.com/NVIDIA/KAI-Scheduler)**, OSMO supports NVLink topology-aware placement for multi-GPU workloads and configurable per-group timeouts to prevent stalled tasks from blocking sibling workflows. > * **Security and Identity:** Features robust enterprise controls including an RBAC authorization sidecar, OAuth2 proxy integration, TLS termination at the Envoy gateway, and cloud workload identity support (Azure Workload Identity, AWS IRSA). > * **Agent Integration:** Featuring native support via **[AGENTS.md](https://github.com/NVIDIA/OSMO/blob/main/AGENTS.md)** and Model Context Protocol (MCP) deployment paths, OSMO integrates directly with coding agents like **Claude Code, OpenAI Codex, and Cursor** to automate workflow submission, monitoring, and debugging. --- ## 核心要点 > ## Key Takeaways * **统一步调与协同编排**：只需一份统一的 YAML 配置，即可跨越不同的 Kubernetes 集群协调贯通模型训练、仿真渲染与边缘真机评测。 * **开箱即用，生产就绪**：基于 Apache-2.0 开源协议发布，可通过 NGC Helm Chart 一键部署，并提供基于 KIND 的本地快速上手工作流。 * **企业级特性全面加持**：依托 KAI Scheduler 实现具备 NVLink 拓扑感知的任务调度，配备细粒度超时控制与原生的 OAuth2/RBAC 安全认证体系。 * **生态成熟，久经实战**：已在 NVIDIA 内部的 GR00T、Isaac Lab、Isaac Sim 及 Isaac ROS 等前沿具身智能框架中历经高强度实战检验。 * **弃用与升级预警**：独立的 dataset 命令行工具与旧版 `/datasets` API 自 6.3 版本起已正式弃用，并将在 6.4 版本中彻底移除，全面转向由工作流原生接管的数据集输出机制。 > * **Unified Orchestration:** Coordinate training, simulation, and edge testing from a single YAML configuration across diverse Kubernetes clusters. > * **Production Ready:** Apache-2.0 licensed, available via NGC Helm charts, and features a local KIND quickstart workflow. > * **Enterprise Features:** Leverages KAI Scheduler with NVLink-aware placement, granular timeouts, and native OAuth2/RBAC security. > * **Ecosystem Proven:** Battle-tested across frameworks like GR00T, Isaac Lab, Isaac Sim, and Isaac ROS. > * **Deprecation Notice:** The standalone dataset CLI and legacy `/datasets` API are deprecated as of version 6.3 and removed in 6.4 in favor of workflow-managed dataset outputs. --- ## 相关资源与链接 > ## Resources & Links * **[GitHub 开源仓库](https://github.com/NVIDIA/OSMO)** * **[官方文档指南](https://nvidia.github.io/OSMO/main/user_guide/)** * **[版本发布说明](https://github.com/NVIDIA/OSMO/releases)** * **[Cookbook 实战范例](https://github.com/NVIDIA/OSMO/blob/main/cookbook)** * **[NVIDIA OSMO 产品主页](https://developer.nvidia.com/osmo)** > * **[GitHub Repository](https://github.com/NVIDIA/OSMO)** > * **[Official Documentation](https://nvidia.github.io/OSMO/main/user_guide/)** > * **[Release Notes](https://github.com/NVIDIA/OSMO/releases)** > * **[Cookbook Examples](https://github.com/NVIDIA/OSMO/blob/main/cookbook)** > * **[NVIDIA OSMO Product Page](https://developer.nvidia.com/osmo)**

AsyncFlow：面向高效大模型后训练的异步流式强化学习框架

Tue, 15 Sep 2026 00:00:00 GMT

在当今大语言模型 (Large Language Model, LLM) 的研发体系中，后训练阶段的强化学习 (Reinforcement Learning, RL) 已成为激发模型深度推理与泛化能力的关键核心技术。然而，现有的强化学习系统架构往往面临严峻的技术困境：同机部署架构难以应对大规模集群扩展，任务解耦架构则受阻于繁复的数据流交互与计算硬件的空转等待，且大多数主流框架均与特定的训练或推理底层引擎深度绑定。针对这一系列痛点，研究团队提出了专为高效后训练量身打造的异步流式强化学习框架 AsyncFlow。该框架凭借全流式分布式数据存储与调度传输模块、允许在参数陈旧度阈值内延迟更新的异步生产者-消费者工作流，以及架构层面上与底层引擎彻底解耦的面向服务接口设计，成功消除了硬件资源空转，取得了相比业内顶尖基准平均 1.59 倍的显著吞吐量提升，为下一代大规模强化学习训练系统的设计提供了重要的工程启示。 --- # AsyncFlow：面向高效大模型后训练的异步流式强化学习框架 > # AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Training ## 核心概述 > ## Summary 强化学习 (Reinforcement Learning, RL) 已成为大语言模型 (Large Language Model, LLM) 后训练阶段不可或缺的关键技术。然而，传统的强化学习框架面临着严峻的局限性：将训练与生成任务同机部署的系统往往受限于扩展性瓶颈；将任务物理分离部署的系统则在处理复杂数据流与硬件资源空转等待方面步履维艰；此外，绝大多数现有框架都与特定的训练或推理引擎深度绑定，缺乏通用性。 > Reinforcement learning (RL) has become critical in the post-training phase of Large Language Models (LLMs). However, traditional RL frameworks face severe limitations: task-collocated systems suffer from scalability bottlenecks, task-separated systems struggle with complex dataflows and resource idling, and most frameworks are tightly coupled to specific training or inference engines. 为了突破这些瓶颈，研究人员推出了 **AsyncFlow**——一个专为大模型后训练设计的异步流式强化学习框架，其核心特性包括： * **分布式数据存储与传输模块**：提供全局全景式数据管理与细粒度调度能力，天然支持全流式传输，实现自动化流水线重叠与动态负载均衡。 * **异步生产者-消费者工作流**：在预设的参数陈旧度 (Staleness) 容忍阈值内，策略性推迟参数同步更新，从而将计算设备的空闲等待时间压缩至最低。 * **计算架构完全解耦**：彻底摆脱与底层具体训练和推理引擎的强绑定，通过面向服务的 API 接口对外开放，提供高度模块化与灵活定制的使用体验。 * **卓越的系统性能**：相较于业界领先的基准系统，取得了平均 **1.59 倍**的端到端吞吐量提升。 > To overcome these challenges, researchers introduce **AsyncFlow**, an asynchronous streaming RL framework featuring: > * A **distributed data storage and transfer module** for panoramic data management, fine-grained scheduling, automated pipeline overlapping, and dynamic load balancing. > * An **asynchronous producer-consumer workflow** designed to minimize computational idleness by strategically deferring parameter updates within predefined staleness thresholds. > * **Architectural decoupling** from underlying training and inference engines, exposed via service-oriented user interfaces for a modular and customizable experience. > * **Superior performance**, demonstrating an average throughput increase of **1.59x** compared to state-of-the-art baselines. --- ## 论文元数据与发布信息 > ## Metadata & Publication Details | 字段 | 详情 | | :--- | :--- | | **arXiv ID** | [arXiv:2507.01663](https://arxiv.org/abs/2507.01663) [cs.LG] | | **主分类** | 机器学习 (`cs.LG`)、人工智能 (`cs.AI`) | | **提交历史** | • **v1:** 2025年7月2日
• **v2 (当前版本):** 2026年9月11日 | | **DOI** | [10.48550/arXiv.2507.01663](https://doi.org/10.48550/arXiv.2507.01663) | | **许可协议** | [Creative Commons Attribution-ShareAlike 4.0 International](http://creativecommons.org/licenses/by-sa/4.0/) ![license icon](./images/5283893486a4.png) | > | Field | Details | > | :--- | :--- | > | **arXiv ID** | [arXiv:2507.01663](https://arxiv.org/abs/2507.01663) [cs.LG] | > | **Primary Subject** | Machine Learning (`cs.LG`), Artificial Intelligence (`cs.AI`) | > | **Submission History** | • **v1:** Jul 2, 2025
• **v2 (Current):** Sep 11, 2026 | > | **DOI** | [10.48550/arXiv.2507.01663](https://doi.org/10.48550/arXiv.2507.01663) | > | **License** | [Creative Commons Attribution-ShareAlike 4.0 International](http://creativecommons.org/licenses/by-sa/4.0/) ![license icon](./images/5283893486a4.png) | --- ## 作者信息 > ## Authors * Zhenyu Han, Ansheng You, Haibo Wang, Kui Luo, Guang Yang, Wenqi Shi, Menglong Chen, Sicheng Zhang, Zeshun Lan, Chunshi Deng, Huazhong Ji, Wenjie Liu, Yu Huang, Yixiang Zhang, Chenyi Pan, Jing Wang, Xin Huang, Chunsheng Li, Jianping Wu > * Zhenyu Han, Ansheng You, Haibo Wang, Kui Luo, Guang Yang, Wenqi Shi, Menglong Chen, Sicheng Zhang, Zeshun Lan, Chunshi Deng, Huazhong Ji, Wenjie Liu, Yu Huang, Yixiang Zhang, Chenyi Pan, Jing Wang, Xin Huang, Chunsheng Li, Jianping Wu --- ## 论文摘要 > ## Abstract 强化学习 (Reinforcement Learning, RL) 已成为大语言模型 (Large Language Model, LLM) 后训练阶段的关键支柱技术。传统的同机共存强化学习框架面临着严峻的扩展性瓶颈，而任务解耦分离的强化学习框架在管理复杂数据流与解决计算资源闲置方面也遭遇重重挑战。此外，多数现存框架与底层的大模型训练或推理引擎深度绑定，难以灵活支持自主定制的计算引擎。为了应对这些挑战，我们提出了专为高效后训练量身打造的异步流式强化学习框架 AsyncFlow。具体而言，我们引入了一个分布式数据存储与传输模块，以全流式传输的方式提供全局全景式数据管理与细粒度调度能力。该架构天然支持强化学习各项任务之间的自动化流水线重叠与动态负载均衡。此外，我们提出了一种异步生产者-消费者工作流，通过在陈旧度容忍阈值内策略性推迟参数更新过程，从而最大限度减少计算资源的等待与闲置。最后，AsyncFlow 的核心功能在架构层面上与底层训练及推理引擎完全解耦，并封装为面向服务的用户接口，带来了高度模块化与可定制的用户体验。大量的实验结果表明，与当前最先进的基准系统相比，AsyncFlow 实现了平均 1.59 倍的吞吐量提升。本文所提出的系统架构为设计下一代强化学习训练系统提供了极具落地价值的实践见解。 > Reinforcement learning (RL) has become a pivotal technology in the post-training phase of large language models (LLMs). Traditional task-collocated RL frameworks suffer from significant scalability bottlenecks, while task-separated RL frameworks face challenges in managing complex dataflows and resolving resource idling. Furthermore, most existing frameworks are tightly coupled with LLM training or inference engines, making them difficult to support custom-designed engines. To address these challenges, we propose AsyncFlow, an asynchronous streaming RL framework tailored for efficient post-training. Specifically, we introduce a distributed data storage and transfer module that provides panoramic data management and fine-grained scheduling capabilities in a fully streamed manner. This architecture inherently enables automated pipeline overlapping among RL tasks and dynamic load-balancing. Moreover, we propose an asynchronous producer-consumer workflow, which is engineered to minimize computational idleness by strategically deferring the parameter update process within staleness thresholds. Finally, the core capabilities of AsyncFlow are architecturally decoupled from underlying training and inference engines and encapsulated by service-oriented user interfaces, offering a modular and customizable user experience. Extensive experiments demonstrate an average throughput of 1.59x compared to the state-of-the-art baseline. The architecture presented in this work provides actionable insights for designing next-generation RL training systems. --- ## 全文获取与相关资源 > ## Access Full-Text & Resources * **PDF 版本：** [查看 PDF](https://arxiv.org/pdf/2507.01663) * **HTML 在线版：** [arXiv HTML (实验性)](https://arxiv.org/html/2507.01663v2) * **TeX 源码：** [源代码 (.tar.gz)](https://arxiv.org/src/2507.01663) > * **PDF Version:** [View PDF](https://arxiv.org/pdf/2507.01663) > * **HTML Version:** [arXiv HTML (Experimental)](https://arxiv.org/html/2507.01663v2) > * **TeX Source:** [Source Code (.tar.gz)](https://arxiv.org/src/2507.01663)

2026 年各类 Unix 系统的平均负载机制深度剖析

Tue, 15 Sep 2026 00:00:00 GMT

在 Unix 与类 Unix 操作系统的运维监控中，“平均负载 (load average)”一直是最核心却也最容易被误读的性能指标。早在十年前，作者就曾对各类开源 Unix 系统的负载统计口径进行过深入梳理；如今时隔十年，作者结合最新的内核源码再度带来现代视角的权威解读。调查表明，几乎所有的现代类 Unix 阵营成员 (包括 NetBSD、OpenBSD、FreeBSD 和 Illumos) 都高度一致地将平均负载严格定义为“运行中与可运行进程”的数量；唯独 Linux 独树一帜，将处于不可中断等待状态 (TASK_UNINTERRUPTIBLE，如磁盘 I/O 或硬件驱动阻塞) 的任务一并纳入计算。本文不仅带你穿透各家内核源码的具体实现差异，更追溯了 Linux 这一特殊设计背后的历史渊源与技术权衡。 --- # 2026 年各类 Unix 系统的平均负载机制深度剖析 > # The Many Load Averages of Unix(es) in 2026 ## 概要 > ## Summary 在对各大开源 Unix 系统中“平均负载 (load average)”定义差异展开首次调查的整整十年后，本文结合最新的内核源码链接，对这一经典主题进行了全新的梳理。核心发现与十年前依然保持一致：几乎所有的 Unix 系统都严格将平均负载定义为“正在运行中与处于就绪态的可运行进程”的总数；唯独 Linux 独树一帜，将处于不可中断等待状态 (例如正在等待磁盘 I/O 或特定硬件驱动响应) 的进程同样计入了负载之中。 > A decade after investigating the divergent definitions of "load average" across various free Unixes, this article provides a modern refresher with current kernel source links. The core finding remains consistent: while virtually all Unix systems define the load average strictly by the count of running and runnable processes, Linux stands apart by also including processes in uninterruptible states (such as those waiting on disk I/O and specific hardware drivers). --- > --- ## 引言 > ## Introduction 十年前，我曾写过一篇题为[《Unix 家族各自为政的平均负载》](https://utcc.utoronto.ca/~cks/space/blog/unix/ManyLoadAveragesOfUnix)的文章，探讨不同开源 Unix 系统对“平均负载”的定义差异，并揭示了它们之间惊人的分歧。时光荏苒，许多系统的底层实现已然演进——再加上我也意识到十年前自己对某些 BSD 系统的机制理解可能存在偏差——是时候结合当前最新的内核源码链接来做一次全新的知识更新了。本次重点聚焦于 Linux 以及各大 `*BSDs` 系统。 > A decade ago, I wrote [The many load averages of Unix(es)](https://utcc.utoronto.ca/~cks/space/blog/unix/ManyLoadAveragesOfUnix) to explore how different free Unixes defined "load average," which revealed significant divergence. Since things have changed—and acknowledging potential misunderstandings of the BSDs a decade ago—it is time for a refresher complete with current kernel source links, focusing primarily on Linux and the `*BSDs`. 一句话概括核心结论：**除了 Linux 以外的所有现代类 Unix 系统，大体上都一致认同平均负载只统计可运行的进程；唯有 Linux 会额外将等待某些特定资源的进程 (尤其是等待磁盘 I/O 的任务) 一并计算在内。** > To summarize: **everyone except Linux agrees that the load average counts only runnable processes (more or less); only Linux also includes processes waiting on certain sorts of things, especially disk I/O.** *(关于 BSD 家族的这段演进历史，可参见[十年前的第一篇文章](https://utcc.utoronto.ca/~cks/space/blog/unix/ManyLoadAveragesOfUnix)。)* > *(For the historical path of the BSDs, see [the first entry](https://utcc.utoronto.ca/~cks/space/blog/unix/ManyLoadAveragesOfUnix).)* --- > --- ## NetBSD > ## NetBSD 在所有系统中，[NetBSD](https://netbsd.org/) 的实现最为一目了然，因为它的平均负载计算逻辑高度集中在单一源码文件中：[`sys/kern/kern_sync.c`](https://github.com/NetBSD/src/blob/trunk/sys/kern/kern_synch.c#L1192)。 > [NetBSD](https://netbsd.org/) offers the most accessible situation because its load average calculation is consolidated in a single place: [`sys/kern/kern_sync.c`](https://github.com/NetBSD/src/blob/trunk/sys/kern/kern_synch.c#L1192). NetBSD 的平均负载统计涵盖： * 所有正在运行或处于就绪态的可运行进程； * 正在通过 `fork` 系统调用创建中的子进程； * *在当前时钟滴答 (tick) 周期内*刚进入睡眠状态的进程 (这是由于 [`sched_lwp_stats()`](https://github.com/NetBSD/src/blob/trunk/sys/kern/kern_runq.c#L1035) 恰好在负载计算函数检查之前递增了 `l->l_slptime` 计数)。 > NetBSD's load average counts: > * All processes that are running or runnable > * Processes being created via a `fork` > * Processes that went to sleep *during the current tick* (because [`sched_lwp_stats()`](https://github.com/NetBSD/src/blob/trunk/sys/kern/kern_runq.c#L1035) increments `l->l_slptime` right before the load average calculation checks it). 尽管 NetBSD 的 [`getloadavg(3)`](https://man.netbsd.org/getloadavg.3) 手册页并未显式写明第三条这一技术细节，但这也合情合理：毕竟在当前滴答周期内刚休眠的进程，在被统计之前的大部分时间里很可能都在保持运行。 > While the NetBSD [`getloadavg(3)`](https://man.netbsd.org/getloadavg.3) manual page doesn't explicitly document this nuance, it is forgivable; processes that went to sleep this tick were presumably running earlier in the tick before being counted. --- > --- ## OpenBSD > ## OpenBSD 在 [OpenBSD](https://openbsd.org/) 中，平均负载是根据分配给各个 CPU 的可运行任务数量汇总计算而来的 ([源码参考 1](https://github.com/openbsd/src/blob/master/sys/kern/sched_bsd.c#L113))。每当进程进入或离开运行队列时，该计数都会被动态更新 ([源码参考 2](https://github.com/openbsd/src/blob/master/sys/kern/kern_sched.c#L268))。 > In [OpenBSD](https://openbsd.org/), the load average is computed from the number of runnable tasks assigned to each CPU ([cf](https://github.com/openbsd/src/blob/master/sys/kern/sched_bsd.c#L113)). This count is dynamically updated as processes enter and leave the run queue ([cf](https://github.com/openbsd/src/blob/master/sys/kern/kern_sched.c#L268)). OpenBSD 的 [`getloadavg(3)`](https://man.openbsd.org/man3/getloadavg.3) 手册页明确记录了这一定义。虽然早期 OpenBSD 曾在历史上把某些休眠线程也视作运行态，但[该机制已于 2017 年被彻底移除](https://github.com/openbsd/src/commit/92dafbefb3c1065c1d820ac18017916cb2ec3be9)。 > The OpenBSD [`getloadavg(3)`](https://man.openbsd.org/man3/getloadavg.3) explicitly documents this definition. Although OpenBSD historically considered some sleeping threads to be running, [this behavior was removed in 2017](https://github.com/openbsd/src/commit/92dafbefb3c1065c1d820ac18017916cb2ec3be9). *(与我十年前的认知相反，我现在确信 NetBSD 和 OpenBSD 中的 `slptime` 字段统计的是时钟滴答数，而非实际秒数。)* > *(Contrary to my belief a decade ago, I now believe the `slptime` fields in NetBSD and OpenBSD count ticks rather than seconds.)* --- > --- ## FreeBSD > ## FreeBSD 在 FreeBSD 中，平均负载依赖于一组封装略显晦涩的函数所维护的运行计数值 ([源码参考 1](https://cgit.freebsd.org/src/tree/sys/kern/kern_synch.c#n571)，[源码参考 2](https://cgit.freebsd.org/src/tree/sys/kern/sched_4bsd.c#n262))。据我所知，它主要是追踪进程 (线程) 进出运行队列的动态，这与 FreeBSD 的 [`getloadavg(3)`](https://man.freebsd.org/cgi/man.cgi?query=getloadavg&apropos=0&sektion=3&manpath=FreeBSD+15.1-RELEASE&format=html) 手册中给出的常规描述完全吻合。 > In FreeBSD, the load average relies on a running count managed via a somewhat opaque set of functions ([cf](https://cgit.freebsd.org/src/tree/sys/kern/kern_synch.c#n571), [also](https://cgit.freebsd.org/src/tree/sys/kern/sched_4bsd.c#n262)). As far as I can tell, this tracks processes (threads) as they enter and exit the run queues, aligning with the generic claims of the FreeBSD [`getloadavg(3)`](https://man.freebsd.org/cgi/man.cgi?query=getloadavg&apropos=0&sektion=3&manpath=FreeBSD+15.1-RELEASE&format=html) manual. --- > --- ## Illumos > ## Illumos 理解 Illumos 内核的具体实现需要花费不少精力，但从现有代码来看，它似乎同样完全依赖于正在运行以及就绪可运行的进程数量 ([源码参考](https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/os/clock.c#L1081))。这既契合了 Illumos 的 [`getloadavg(3)`](https://www.illumos.org/man/3C/getloadavg) 手册说明，也印证了[我十年前所描述的情况](https://utcc.utoronto.ca/~cks/space/blog/unix/ManyLoadAveragesOfUnix)。 > Understanding the Illumos kernel's implementation requires more effort than I am willing to spend here, but it appears to rely exclusively on running and runnable processes ([cf](https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/os/clock.c#L1081)). This matches the behavior claimed by the Illumos [`getloadavg(3)`](https://www.illumos.org/man/3C/getloadavg) and mirrors [the situation I described a decade ago](https://utcc.utoronto.ca/~cks/space/blog/unix/ManyLoadAveragesOfUnix). --- > --- ## Linux：独树一帜的特例 > ## Linux: The Odd One Out Linux 是整个家族中的绝对特例。尽管 Linux 的 [`getloadavg(3)`](https://www.man7.org/linux/man-pages/man3/getloadavg.3.html) 手册宣称平均负载仅统计就绪可运行的进程，但 [`proc_loadavg(5)`](https://www.man7.org/linux/man-pages/man5/proc_loadavg.5.html) 手册在技术层面上更为准确，而内核源码更是将这一点阐释得清清楚楚。 > Linux is the outlier. While the Linux [`getloadavg(3)`](https://www.man7.org/linux/man-pages/man3/getloadavg.3.html) claims that the load average only counts runnable processes, [`proc_loadavg(5)`](https://www.man7.org/linux/man-pages/man5/proc_loadavg.5.html) is technically correct, and the kernel source code makes it explicit. [`kernel/sched/loadavg.c`](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/sched/loadavg.c) 中的注释和代码均证实：Linux 的平均负载在计算可运行任务之外，还会将**不可中断睡眠 (Uninterruptible)** 的进程 (任务) 一并纳入。系统中有大量的子系统 (尤其是各种硬件驱动程序内部) 会将任务标记为 `TASK_UNINTERRUPTIBLE` 状态。因此，当这类任务在 [`kernel/sched/core.c`](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/sched/core.c#n2255) 中发生阻塞时，它们便会为系统负载做出“贡献”，且其负载增减在[任务阻塞时](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/sched/sched.h#n3071)和[任务重新激活时](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/sched/core.c#n3820)都会被精准追踪。 > Both the comments and code in [`kernel/sched/loadavg.c`](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/sched/loadavg.c) confirm that Linux's load average includes **uninterruptible** processes (tasks) alongside running or runnable ones. A wide variety of subsystems—often within hardware drivers—set tasks to `TASK_UNINTERRUPTIBLE`. Consequently, when such a task blocks in [`kernel/sched/core.c`](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/sched/core.c#n2255), it contributes to the system load, with adjustments tracked [on blocking](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/sched/sched.h#n3071) and [on reactivation](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/sched/core.c#n3820). 关于 Linux 具体实现的更多技术细节，我在[更早前的一篇文章](https://utcc.utoronto.ca/~cks/space/blog/linux/LoadAverageWhereFrom)中曾进行过详尽解析。 > I covered the specifics of the Linux implementation in detail in [an earlier entry](https://utcc.utoronto.ca/~cks/space/blog/linux/LoadAverageWhereFrom). --- > --- ## 历史溯源 > ## Historical Context 将等待磁盘 I/O 或处于不可中断等待状态的进程计入负载的做法，最早可以追溯到 [3BSD 的原始行为](https://utcc.utoronto.ca/~cks/space/blog/unix/LoadAverageOrigin)。后来，源于 4BSD 的各类商业 Unix 系统 (例如 SunOS) 也沿袭了这一惯例 (在 SunOS 中，等待 NFS 远程“I/O”的进程同样会推高平均负载，哪怕仅仅是因为远端 NFS 服务器卡死无响应)。 > Counting processes waiting for disk I/O or stuck in uninterruptible waits traces back to [original 3BSD behavior](https://utcc.utoronto.ca/~cks/space/blog/unix/LoadAverageOrigin). This practice was adopted by commercial Unixes derived from 4BSD, such as SunOS (where processes waiting on NFS "I/O" also contributed to the load average, even if delayed by an unresponsive NFS server). 正如 [Brendan Gregg 曾经撰文探讨过的](https://www.brendangregg.com/blog/2017-08-08/linux-load-averages.html)，极早期的 Linux 版本其实也只统计可运行进程。然而在 1993 年 10 月，Linux 引入修改，将不可中断状态的进程也囊括了进来。颇具讽刺意味的是，正当其他 Unix 变种纷纷着手废弃这一古老机制的时候，Linux 却通过这次改动在事实上倒向了 4BSD 的历史设计。 > As [Brendan Gregg has covered](https://www.brendangregg.com/blog/2017-08-08/linux-load-averages.html), very early Linux versions counted only runnable processes. However, in October 1993, this was extended to include uninterruptible processes. Ironically, this change aligned Linux with historical 4BSD behavior right around the time other Unix variants were phasing it out. --- > --- ## 后记：关于 Linux 进程状态的一点技术补充 > ## PostScript: A Technical Note on Linux States Linux 的 [`proc_loadavg(5)`](https://www.man7.org/linux/man-pages/man5/proc_loadavg.5.html) 在严谨的技术层面是准确的：内核向外汇报的所有处于 `TASK_UNINTERRUPTIBLE` 状态的进程，其状态标识均为 `'D'` ([源码参考 1](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/proc/array.c#n121)，[源码参考 2](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/linux/sched.h#n95))。 > Linux's [`proc_loadavg(5)`](https://www.man7.org/linux/man-pages/man5/proc_loadavg.5.html) is narrowly technically correct: all processes in `TASK_UNINTERRUPTIBLE` are reported by the kernel as state `'D'` ([cf](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/proc/array.c#n121), [also](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/linux/sched.h#n95)). 但需要注意的是，这**并不意味着**所有处于该状态的进程都在等待磁盘 I/O；系统中的各种组件——包括特定的 GPU 和以太网驱动——也会出于其他完全无关的等待场景而将进程置于该状态。 > However, this does *not* mean all such processes are waiting for disk I/O; various components—including certain GPU and Ethernet drivers—utilize this status for unrelated waiting states.

脚手架内部的上下文工程：战胜长程任务中上下文溢出与目标迷失的 4 大机制

Mon, 14 Sep 2026 00:00:00 GMT

在面对需要执行数十步甚至上百步的复杂长程任务时，仅依靠“大语言模型 + 简单调用循环”构建的 AI 智能体 (AI Agent) 往往会因为“上下文溢出”和“目标迷失”而功亏一篑。许多人寄希望于模型上下文窗口的无限扩张，但实测表明，随着输入长度增长，模型的注意力机制会发生严重的“上下文腐烂”，在长文本中迷失关键线索。真正的解法不在于模型本身，而在于包裹在模型外层的系统架构——“脚手架 (Harness)”。本文系统梳理了 LangChain Deep Agents、Claude Code、Manus、OpenAI Codex 以及 Amazon Bedrock AgentCore 等顶尖工业级框架的设计哲学，深入解析了它们在长程任务中赖以维持系统稳定、控制上下文预算与防范目标丢失的 4 大核心工程机制。 --- 在长程任务中，以简单循环运行的大语言模型 (Large Language Model, LLM) 智能体往往不可避免地会由于“上下文溢出”与“目标迷失”而走向失败。单纯扩大上下文窗口绝非万灵药，因为随着输入长度的不断攀升，模型的注意力表现会逐步退化。相反，真正的解决之道深植于“脚手架 (Harness)”——即围绕在模型周围的系统架构层之中。本文将深入探讨现代智能体框架 (如 LangChain Deep Agents、Claude Code、Manus、OpenAI Codex 以及 Amazon Bedrock AgentCore) 所采用的 4 大核心机制，解析它们如何在长时间跨度的长程操作中维护状态、规划上下文预算并保护智能体目标不被遗忘。 > **Summary:** On long-horizon tasks, LLM agents running in simple loops inevitably fail due to *context overflow* and *goal loss*. Expanding the context window is not a silver bullet because attention degrades as input length grows. Instead, the solution lies within the **harness**—the architectural layer surrounding the model. This article explores four core mechanisms used by modern agent frameworks (such as LangChain Deep Agents, Claude Code, Manus, OpenAI Codex, and Amazon Bedrock AgentCore) to maintain state, budget context, and protect agent goals over extended operations. --- ## 为什么更大的上下文窗口无法根治问题 > ## Why a Bigger Window Does Not Fix It 面对上下文长度的限制，人们最容易想到的直观解法就是直接把上下文窗口做大。然而，大量实证研究表明，这种方法的效果远不及预期。[Chroma 的上下文腐烂 (Context Rot) 报告](https://www.trychroma.com/research/context-rot)在评估了包括 GPT-4.1、Claude 4、Gemini 2.5 和 Qwen3 在内的 18 款大语言模型后发现，哪怕是在最简单的检索任务中，随着输入长度的增加，模型的表现也会变得越来越不可靠。 > The obvious fix for context limits is a larger context window. However, empirical evidence shows it helps less than expected. [Chroma’s Context Rot report](https://www.trychroma.com/research/context-rot) evaluated 18 LLMs (including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3) and found that performance grows increasingly unreliable as input length grows, even on simple retrieval tasks. Anthropic 发布的[上下文工程指南](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents)深入解释了这一底层机理： * 注意力机制对于 $n$ 个 Token 会产生 $n^2$ 对成对关系； * 每新增一个 Token，都会消耗模型有限的“注意力预算”； * 上下文是一种边际收益递减的稀缺资源，而不是一个深不见底的无限容积桶。 > Anthropic’s [context engineering guide](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents) explains the mechanism: > * Attention creates $n^2$ pairwise relationships for $n$ tokens. > * Every added token depletes a finite "attention budget." > * Context is a resource with diminishing returns, not an infinite bucket. 对于智能体的运行循环而言，实际情况比理论听起来还要严峻得多。Manus 的实测数据显示，一个典型的复杂任务通常需要大约 50 次工具调用，输入与输出的 Token 比例接近惊人的 100:1。每一次工具返回的环境观察结果都会直接涌入上下文并堆积在那里。随着对话拉长，最初的用户指令被不断推向窗口中央——而这恰恰是模型召回与注意力衰减最严重的“迷失在中间 (Lost-in-the-Middle)”盲区。因此，目标迷失绝不仅仅是模型偶尔犯错的小毛病，而是在长程任务中放任上下文野蛮生长所必然导致的工程必然。 > For an agent loop, this is worse than it sounds. Manus reports that a typical task needs around 50 tool calls, and the input-to-output token ratio runs near 100:1. Each observation lands in context and stays there. The original instruction drifts toward the middle of the window—the exact zone where recall degrades. Goal loss is not just a model bug; it is the expected outcome of an unmanaged context on long-running tasks. --- ## 机制 1：上下文预算控制与数据卸载 > ## Mechanism 1: Context Budgeting and Offloading 脚手架的首要职责，就是严格把关并裁决哪些内容**绝不能**进入上下文窗口。 > The first job of a harness is deciding what *never* enters the window at all. * **Deep Agents**：内置了两条极为严苛的数据卸载规则： 1. 当工具返回的内容超过 **20,000 个 Token** 时，系统会直接将其写入底层文件系统，并在上下文中替换为该文件路径以及前 10 行内容的简短预览。 2. 当会话上下文占用到模型窗口的 **85%** 时，较早前的写入与编辑工具调用记录 (由于其实际文件内容早已保存在磁盘上) 会被截断为轻量级的指针引用。只有当数据卸载手段用尽且空间依然不足时，系统才会动用会话摘要作为兜底方案。 * **Claude Code**：同样实践了极其类似的预算管控原则： * 自动记忆 (Auto-memory) 的体积被严格限制在前 200 行以内或 25KB 以下； * MCP 工具定义 (Tool Schema) 默认保持延迟加载状态，仅在需要时通过工具搜索按需加载完整 Schema； * 在完成上下文压缩后，任何重新读取且体积超过 5,000 Token 的文件，都只会以路径引用的形式返回，而非直接灌入原始内容。 > * **Deep Agents:** Ships with two strict offloading rules: > 1. When a tool response exceeds **20,000 tokens**, it is written to the filesystem and replaced with a file path plus a preview of the first 10 lines. > 2. When session context crosses **85%** of the model’s window, older write and edit tool calls (whose file contents already live on disk) are truncated to a pointer. Summarization is only used as a fallback when offloading runs out of room. > > * **Claude Code:** Applies similar budgeting principles: > * Auto-memory is capped at the first 200 lines or 25KB. > * MCP tool schemas remain deferred by default, loading full schemas on demand via tool search. > * After compaction, any re-read file over 5,000 tokens returns as a path reference rather than raw content.

面向高扇出智能体沙箱的内存压缩技术

Sun, 13 Sep 2026 00:00:00 GMT

随着高扇出 (High-Fanout) AI 智能体应用的大规模普及，单个复合任务往往需要同时并发生成数十上百个沙箱容器来进行代码测试与环境交互，这给宿主机系统带来了极其沉重的物理内存瓶颈。然而，这些沙箱并非完全独立，它们源自相同的初始模板并执行高度相似的代码轨迹，存在海量的跨沙箱与相对模板的内存冗余。传统操作系统的通用内存压缩机制无法识别这种智能体执行特征，面临压缩方式僵化、压缩范围保守以及触发时机被动的三大局限。为此，本文提出了专为 AI 智能体沙箱量身定制的内存压缩系统 **AgentZip**，通过挖掘模板相对差异与跨沙箱冗余、以恢复期预取替代压缩期保守筛选、并将重度压缩操作对齐到模型等待空闲期。在涵盖大语言模型训练与推理的多样化工作负载中，AgentZip 实现了最高达 8.7 倍的沙箱内存缩减，同时将激进压缩带来的执行延迟从 3.1 倍大幅降低至 1.40 倍，为大规模智能体并发执行提供了坚实高效的底层系统支撑。 --- ## 核心概要 > ## Summary 高扇出 AI 智能体 (AI Agent) 工作负载由于从共享模板中衍生出大量并发沙箱会话，从而造成了巨大的系统内存瓶颈。尽管传统的内存压缩机制由于压缩策略、作用范围和压缩时机的不匹配而难以应对这种场景，但本文提出了 **AgentZip**。AgentZip 专为 AI 智能体沙箱打造，深度利用了相对模板与跨沙箱的内存冗余，通过在恢复时进行页面预取来优化页面选择，并将耗时的压缩阶段与大语言模型 (Large Language Model, LLM) 的等待时间对齐。这不仅显著削减了内存占用，同时将任务执行减速控制在极低水平。 > High-fanout AI agent workloads create a massive memory bottleneck by spawning numerous concurrent sandbox sessions from shared templates. While traditional memory compression struggles with this due to mismatched compression strategies, scopes, and timings, this paper introduces **AgentZip**. Specifically designed for AI-agent sandboxes, AgentZip leverages template-relative and cross-sandbox redundancies, optimizes page selection via restore-time prefetching, and aligns compression phases with LLM waiting times. This dramatically reduces memory usage while minimizing execution slowdowns. --- ## 论文元数据 > ## Paper Metadata - **arXiv ID：** [arXiv:2609.11294](https://arxiv.org/abs/2609.11294) [cs.AI] - **学科领域：** 人工智能 (`cs.AI`)；操作系统 (`cs.OS`) - **提交日期：** 2026 年 9 月 10 日 - **作者列表：** - Mengming Li - Ceyu Xu - Qijun Zhang - Jiangnan Yu - Xiangfeng Sun - Haohui Mai - Zhiyao Xie - **全文获取：** - [查看 PDF](https://arxiv.org/pdf/2609.11294) - [网页版本](https://arxiv.org/html/2609.11294v1) - [TeX 源码](https://arxiv.org/src/2609.11294) > - **arXiv ID:** [arXiv:2609.11294](https://arxiv.org/abs/2609.11294) [cs.AI] > - **Subjects:** Artificial Intelligence (`cs.AI`); Operating Systems (`cs.OS`) > - **Submission Date:** 10 September 2026 > - **Authors:** > - Mengming Li > - Ceyu Xu > - Qijun Zhang > - Jiangnan Yu > - Xiangfeng Sun > - Haohui Mai > - Zhiyao Xie > - **Full-Text Access:** > - [View PDF](https://arxiv.org/pdf/2609.11294) > - [HTML Version](https://arxiv.org/html/2609.11294v1) > - [TeX Source](https://arxiv.org/src/2609.11294) --- ## 论文摘要 > ## Abstract 高扇出智能体工作负载正在引发日益严峻的内存瓶颈，因为单个复杂任务就可能催生出众多并发运行的沙箱会话。然而，这些沙箱绝非相互独立：它们不仅派生自同一个基础模板，而且执行着高度相关的运行轨迹，从而暴露出可观的相对模板冗余与跨沙箱内存冗余。 > High-fanout agent workloads create a growing memory bottleneck because a single task may spawn many concurrent sandbox sessions. Yet these sandboxes are far from independent: they originate from a shared template and execute related trajectories, exposing substantial template-relative and cross-sandbox memory redundancy. 传统的内存压缩技术在三个根本维度上与此类场景严重错配： 1. **如何压缩：** 无法有效挖掘非完全相同沙箱内存页之间的相似性。 2. **压缩什么：** 只能通过保守地筛选页面来被动控制缺页异常开销。 3. **何时压缩：** 压缩要么在遭遇内存严重压力时被动触发，要么在对智能体执行阶段毫无感知的情况下盲目运行。 > Conventional memory compression is poorly matched to this setting in three fundamental dimensions: > 1. **How to compress:** They fail to exploit similarity across non-identical sandbox pages. > 2. **What to compress:** They control page-fault overhead through conservative page selection. > 3. **When to compress:** Compression is either triggered reactively by memory pressure or performed without awareness of agent execution phases. 为了攻克上述难题，作者团队推出了 **AgentZip**——首个专为 AI 智能体沙箱量身打造的内存压缩系统： * **深度挖掘冗余：** 引入全新机制，充分利用相对模板冗余与跨沙箱数据冗余。 * **扩大范围与主动预取：** 将压缩作用域拓展到任何具有压缩增益的页面，将开销控制的重心从压缩时的保守挑选转移至恢复时的页面预取。 * **执行阶段感知调度：** 将计算密集型的压缩任务精准对齐到大模型的等待空闲期，避免对前台工具的高效执行产生干扰。 > To resolve these challenges, the authors present **AgentZip**, the first memory compression system designed specifically for AI-agent sandboxes: > * **Advanced Redundancy Exploitation:** Introduces mechanisms that leverage both template-relative and cross-sandbox redundancies. > * **Expanded Scope & Prefetching:** Broadens the compression scope to any page with a profitable representation, shifting overhead control from compression-time page selection to restore-time prefetching. > * **Phase-Aware Scheduling:** Aligns expensive compression tasks with LLM waiting periods to avoid interfering with foreground tool execution.

面向二进制对称信道的最优 $(n,4)$ 二进制码 Dong-Yang 分类定理的机器验证证明

Sun, 13 Sep 2026 00:00:00 GMT

在信息论与信道编码领域，寻找面向二进制对称信道 (Binary Symmetric Channels, BSCs) 的最优分组码是一项经典且严谨的数学课题。此前学者 Dong 与 Yang 提出了关于有限长度 $(n,4)$ 最优二进制码的完整分类定理，但传统的人工长篇手写推导往往容易潜藏细微的笔误与证明缺环。本文作者基于当代理论计算机科学推崇的交互式定理证明器 **Lean 4** ，借助现代 AI 工具辅助推导，首次实现了该数学定理的端到端机器形式化验证。在确保核心公理与主定理绝对正确的前提下，作者不仅纠正了 AI 生成代码中的瑕疵与冗余，还成功勘误了原始手写论文中的数处疏漏，充分展现了“大模型生成 + 形式化系统严谨闭环验证”在现代数学研究中的广阔前景。 --- # 面向二进制对称信道的最优 $(n,4)$ 二进制码 Dong-Yang 分类定理的机器验证证明 > # A Machine-Checked Proof of the Dong-Yang Classification of Optimal $(n,4)$ Binary Codes for BSCs [![license icon](./images/079cd8198ba3.png)](http://creativecommons.org/licenses/by-nc-sa/4.0/) --- ## 核心概述 > ## Summary 本文介绍了在 **Lean 4** 交互式证明器环境中，对 Dong 与 Yang 关于二进制对称信道 (Binary Symmetric Channels, BSCs) 上最优有限长 $(n,4)$ 二进制分组码分类定理所完成的端到端机器形式化验证。整个形式化过程深度借助了先进的 AI 工具，将原论文中的证明手稿输入大模型进行自动转换。为确保数学层面的绝对严格与无瑕疵，作者在 Lean 体系内对核心定理表述及底层公理进行了独立的人工形式化复核。本简报还详细记录了对 AI 形式化输出所做的必要修正与精简，并揭示了原论文手写推导中被机器形式化发现的具体疏漏。 > This paper presents a machine-checked formalization in **Lean 4** of Dong and Yang's classification theorem regarding optimal finite-length $(n,4)$ binary block codes for binary symmetric channels (BSCs). The formalization process relied heavily on feeding the proofs from the original paper into an AI tool. To guarantee absolute mathematical correctness, the authors independently verified the primary theorem statements and accepted axioms within Lean. The note also documents the necessary corrections and simplifications applied to the AI-generated formalization, alongside specific discrepancies discovered in the original text. --- ## 论文元数据 > ## Metadata * **arXiv 编号：** [arXiv:2609.10579](https://arxiv.org/abs/2609.10579) * **主分类：** 数学史与概览 (`math.HO`) * **次分类：** 人工智能 (`cs.AI`), 信息论 (`cs.IT`) * **提交日期：** 2026年9月5日 * **作者：** Shenghao Yang, Yanyan Dong > * **arXiv Identifier:** [arXiv:2609.10579](https://arxiv.org/abs/2609.10579) > * **Primary Subject:** History and Overview (`math.HO`) > * **Secondary Subjects:** Artificial Intelligence (`cs.AI`), Information Theory (`cs.IT`) > * **Submission Date:** September 5, 2026 > * **Authors:** Shenghao Yang, Yanyan Dong --- ## 资源链接与代码仓库 > ## Links & Resources * **全文获取：** * [查看 PDF](https://arxiv.org/pdf/2609.10579) * [HTML 在线版本（实验性）](https://arxiv.org/html/2609.10579v1) * [TeX 源码](https://arxiv.org/src/2609.10579) * **开源代码仓库：** [GitHub - shhyang/n4code_lean](https://github.com/shhyang/n4code_lean) * **开源许可协议：** [知识共享署名-非商业性使用-相同方式共享 4.0 国际许可 (CC BY-NC-SA 4.0)](http://creativecommons.org/licenses/by-nc-sa/4.0/) > * **Full-Text Access:** > * [View PDF](https://arxiv.org/pdf/2609.10579) > * [HTML Version (Experimental)](https://arxiv.org/html/2609.10579v1) > * [TeX Source](https://arxiv.org/src/2609.10579) > * **Source Code Repository:** [GitHub - shhyang/n4code_lean](https://github.com/shhyang/n4code_lean) > * **License:** [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International](http://creativecommons.org/licenses/by-nc-sa/4.0/)

递归语言模型训练中的脆弱性图谱

Sun, 13 Sep 2026 00:00:00 GMT

随着互联网上合成数据的爆发式增长，大语言模型 (Large Language Model, LLM) 的生成内容正不可避免地被回流用于新一轮模型的训练。这种“用自己生成的数据训练自己”的递归污染，往往会导致模型输出多样性严重衰退甚至发生“模型崩溃 (Model Collapse)”。本项研究深入剖析了 13 个开源模型在共享语料库上递归训练五代的表现，揭示出模型抗崩溃能力的“脆弱性图谱”——某些模型经历五代依然完好，另一些则迅速退化为无意义的重复词句。更关键的是，研究表明这种脆弱性由模型自身内在特性决定且与参数规模非单纯正相关，同时提出了仅需自我迭代两三代即可廉价预测该风险的诊断方法与有效的推理期缓解策略。 --- ## 📋 内容摘要 > ## 📋 Summary 当语言模型经历数代更迭、持续使用自身生成的文本进行递归训练时，其输出的多样性通常会急剧衰减并走向崩溃。然而，不同模型面对这一递归退化过程时，展现出的耐受力却有着天壤之别。 > When language models are recursively trained on their own generated text over multiple generations, output diversity typically collapses. However, different models exhibit drastically different resilience to this process. 本研究通过追踪 13 个公开发布的模型检查点 (Checkpoints) 在连续五代共享通用训练语料库中的表现，系统探究了这种**脆弱性图谱 (Fragility Spectrum)**。核心发现如下： > This paper investigates a **fragility spectrum** by observing 13 publicly released model checkpoints sharing a common training corpus over five generations. The findings reveal that: * **输出多样性差异悬殊**：历经五代递归训练后，不同模型生成的独立 4-gram (Unique 4-gram) 比例分布在 $0.187$ 至 $0.940$ 之间，差距高达约五倍；这意味着部分模型几乎未受污染影响，而另一些模型则已退化为高度机械重复的词句片段。 * **脆弱性是模型的固有属性**：模型的抗崩溃脆弱性主要取决于具体的模型检查点本身。无论是改变共享数据池的混合比例、混入人类撰写的真实文本，还是更换随机数种子，各模型的脆弱性排名相关性依然极高 (Spearman 相关系数维持在 $0.91\text{--}0.98$ 之间)。 * **参数规模并非唯一决定因素**：单纯的模型参数量并不能决定抗崩溃能力；即便在同一模型家族内部，不同参数规模的模型在抗崩溃性上也并不呈现单调变化。 * **快速预测与缓解策略**：仅需让目标模型在自身输出上独立迭代两到三代，即可低成本地推断出其在整个模型生态中的脆弱性。此外，在文本生成阶段收紧 `top-p` 采样过滤（截断低概率的长尾 Token），可在三代内几乎彻底遏制崩溃趋势，使脆弱性图谱上处于不同区间的各模型重新趋于稳定。 > * **Output Diversity Varies Widely:** After five generations, unique 4-gram outcomes range from $0.187$ to $0.940$ (a roughly five-fold spread), meaning some models remain virtually unaffected while others devolve into repetitive fragments. > * **Fragility is an Intrinsic Property:** Model vulnerability is a persistent characteristic of the specific checkpoint itself. It is largely unaffected by changes in data composition, human text mixing, or random seeds (Spearman correlations remain high at $0.91\text{--}0.98$). > * **Scale is Not the Sole Predictor:** Parameter size alone does not dictate fragility; model families do not scale monotonically in their resistance to collapse. > * **Fast Detection & Mitigation:** A model's ecosystem fragility can be cheaply predicted by letting it iterate on its own output for just two or three generations. Furthermore, interventions like tightening `top-p` generation-time filtering can nearly halt collapse entirely and stabilize diverse checkpoints across the spectrum. --- ## 📑 论文摘要 > ## 📑 Abstract AI 模型生成的合成文本正在源源不断地回流到训练语料库中，大量研究证据表明，反复在这类合成数据上进行训练会导致模型输出的多样性崩溃。先前的研究主要聚焦于现象本身，例如探讨哪些递归协议和数据配比会诱发崩溃。然而，在面对完全相同的递归训练过程时，不同的模型却表现出了截然不同的反应与耐受力。 > Model-generated text is finding its way back into training corpora, and there is plenty of evidence that training on such data over and over collapses output diversity. Prior work has studied the phenomenon itself: which protocols and which data mixtures cause collapse. But different models behave very differently under the same process. 我们固定了一套递归数据污染协议，让 13 个公开发布的模型检查点构成一个生态系统，使其在五代更迭中持续共享同一个训练语料池。历经五代训练后，各检查点的独立 4-gram 结果在 $0.187$ 到 $0.940$ 之间大幅波动，差距约达五倍：有些模型几乎完好无损，而另一些模型则退化成了机械重复的语言碎片。无论是改变共享数据池中的成分构成，还是混入人类真实文本，各模型抗崩溃能力的排序保持着极高的相关性 (Spearman 秩相关系数为 $0.91\text{--}0.97$ )，更改随机数种子后相关性同样维持在 $0.93\text{--}0.98$ 。 > We fix one recursive contamination protocol and let 13 publicly released checkpoints form an ecosystem that shares a common corpus for five generations. The unique 4-gram outcome after five generations ranges from $0.187$ to $0.940$ across checkpoints, a roughly five-fold spread: some models are barely touched, others degenerate into repetitive fragments. Changing the composition of the shared pool or mixing in human text keeps the Spearman correlation of the ordering at $0.91\text{--}0.97$, and changing the random seed keeps it at $0.93\text{--}0.98$. 因此，一个模型在递归训练下是否容易发生崩溃，实际上是该模型检查点本身固有的一项内在属性，而此前这一特性在很大程度上被学界所忽视。单纯的参数规模无法解释这种差异，因为在同一模型家族的三个不同尺寸梯队中，耐受力与尺寸并未呈现单调对应关系，且我们测试的所有静态评估指标也都无法对其进行有效预测。行之有效且计算开销极低的方法是：仅需让单个模型在自身生成的输出上独立迭代训练两到三代，便足以推断出其在更大规模生态环境中的脆弱性水平。此外，崩溃的恶化速度对干预手段也有明显的响应。在生成阶段收紧 `top-p` 采样以剔除低概率的长尾 Token ，几乎能在三代之内阻止崩溃的发生，并使横跨整个脆弱性图谱的六个检查点全部稳定下来；而单纯从数据端进行清洗过滤虽能减缓崩溃速度，却无法彻底阻止崩溃。 > Whether a model collapses easily under recursive training is, then, a property of the checkpoint itself, and one that has gone largely unexamined. Parameter scale alone does not explain it, since a three-size ladder within one family is not monotonic in size, and none of the static indicators we tested predicts it either. What does work is cheap: let a model iterate on its own output for two or three generations, and its fragility in the larger ecosystem can be inferred from that alone. Collapse speed also responds to intervention. Tightening top-p, which cuts the low-probability tail at generation time, nearly stops collapse within three generations and stabilizes six checkpoints spanning the whole spectrum together, while data-side filtering slows collapse without stopping it. --- ## 🔗 论文资源与相关链接 > ## 🔗 Additional Resources & Links * **阅读 PDF 论文**：[arXiv:2609.11149 PDF](https://arxiv.org/pdf/2609.11149) * **HTML 在线网页版**：[arXiv HTML (Experimental)](https://arxiv.org/html/2609.11149v1) * **TeX 源代码**：[arXiv Source Files](https://arxiv.org/src/2609.11149) * **引用与文献检索工具**： * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.11149) * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.11149) * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.11149) > * **View PDF:** [arXiv:2609.11149 PDF](https://arxiv.org/pdf/2609.11149) > * **HTML Version:** [arXiv HTML (Experimental)](https://arxiv.org/html/2609.11149v1) > * **TeX Source:** [arXiv Source Files](https://arxiv.org/src/2609.11149) > * **Citations & Tools:** > * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.11149) > * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.11149) > * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.11149)

连续扩散语言模型在规模化扩展上可媲美离散扩散

Sun, 13 Sep 2026 00:00:00 GMT

在图像生成领域大放异彩的扩散模型，近年来被广泛探索用于自然语言建模任务。然而，长期以来学界普遍认为连续扩散在文本领域难以规模化扩展，性能往往落后于离散扩散方案。本文重新审视了基于似然估计的连续扩散语言模型 (Diffusion Language Model, DLM) Plaid，通过将其网络架构与现代主流离散 DLM 对齐，构建了全新的 **RePlaid** 模型。该研究首次确立了能够直接媲美离散扩散模型的连续 DLM 扩展法则 (Scaling Law) ，将连续扩散与自回归模型之间的算力开销差距显著缩小至仅 $20\times$ ，并在 OpenWebText 数据集上取得了 22.1 的连续 DLM 最优困惑度 (Perplexity, PPL) 上界与卓越的生成质量。研究从理论上揭示了优化噪声调度以最小化 ELBO 方差能自然带来跨时间维度的均匀信息衰减，有力地证明了连续扩散在语言建模上兼具极高竞争力与扩展前景。 --- ## 论文元数据 > ## Metadata * **arXiv 标识符：** [arXiv:2605.18530](https://arxiv.org/abs/2605.18530) [cs.CL] * **学科分类：** 计算与语言 (`cs.CL`)；人工智能 (`cs.AI`)；机器学习 (`cs.LG`)；统计机器学习 (`stat.ML`) * **作者列表：** Zhihan Yang, Wei Guo, Shuibai Zhang, Subham Sekhar Sahoo, Yongxin Chen, Arash Vahdat, Morteza Mardani, John Thickstun * **提交历程：** 首次提交于 2026年5月18日；最新修订于 2026年9月9日 (v2) * **链接资源：** [查看 PDF](https://arxiv.org/pdf/2605.18530) | [HTML 版本](https://arxiv.org/html/2605.18530v2) | [DOI 索引](https://doi.org/10.48550/arXiv.2605.18530) > * **arXiv ID:** [arXiv:2605.18530](https://arxiv.org/abs/2605.18530) [cs.CL] > * **Subjects:** Computation and Language (`cs.CL`); Artificial Intelligence (`cs.AI`); Machine Learning (`cs.LG`); Machine Learning (`stat.ML`) > * **Authors:** Zhihan Yang, Wei Guo, Shuibai Zhang, Subham Sekhar Sahoo, Yongxin Chen, Arash Vahdat, Morteza Mardani, John Thickstun > * **Submission History:** Submitted on 18 May 2026; Last revised 9 Sep 2026 (v2). > * **Links:** [View PDF](https://arxiv.org/pdf/2605.18530) | [HTML Version](https://arxiv.org/html/2605.18530v2) | [DOI](https://doi.org/10.48550/arXiv.2605.18530) --- ## 核心执行概要 > ## Executive Summary 虽然扩散模型近期在自然语言处理社区引发了广泛关注，但在可扩展性 (Scalability) 方面，连续扩散长期以来普遍被认为明显逊色于离散方案。本研究打破了这一固有偏见，重新审视了基于似然的经典连续扩散语言模型 (DLM) **Plaid** ，并提出了将 Plaid 架构与现代离散 DLM 紧密对齐的全新变体 **RePlaid** 。 > While diffusion models have gained significant traction in language modeling, continuous diffusion has traditionally lagged behind discrete approaches in terms of scalability. This paper challenges that limitation by revisiting **Plaid**—a likelihood-based continuous diffusion language model (DLM)—and introducing **RePlaid**, which aligns Plaid's architecture with modern discrete DLMs. 该工作的核心发现与主要贡献包括： * **媲美离散的可扩展性：** RePlaid 建立了首个在规模化扩展上与离散 DLM 旗鼓相当的连续 DLM 扩展法则 (Scaling Law) ，其与自回归模型之间的算力消耗差距仅为 $20\times$ 左右。 * **更为优越的模型性能：** RePlaid 仅需更少的参数量即可击败 *Duo* ，并在过度训练 (Over-trained) 机制下全面超越 *MDLM* 。 * **顶尖基准表现：** 在 OpenWebText 基准评测中，RePlaid 在连续 DLM 中斩获了全新的最优困惑度 (PPL) 上界 **22.1** ，并展现出更出色的生成文本质量。 * **深刻的理论洞见：** 作者证明了通过优化噪声调度以最小化证据下界 (ELBO) 方差，会自然形成随时间呈线性衰减的交叉熵（信息损失），从而无需针对特定案例进行复杂的时间重参数化即可均匀分散去噪难度。此外，研究发现通过似然目标优化词嵌入能够构建出结构化的几何空间，成为带来显著似然增益的核心驱动力。 > Key findings and contributions of this work include: > * **Competitive Scaling:** RePlaid establishes the first scaling law for continuous DLMs that closely rivals discrete models, showing a compute gap of only $20\times$ compared to autoregressive models. > * **Superior Performance:** RePlaid outperforms *Duo* using fewer parameters and surpasses *MDLM* in the over-trained regime. > * **State-of-the-Art Results:** Evaluated on OpenWebText, RePlaid achieves a new state-of-the-art Perplexity (PPL) bound of **22.1** among continuous DLMs along with superior generation quality. > * **Theoretical Insights:** The authors demonstrate that optimizing the noise schedule to minimize ELBO variance naturally results in linear cross-entropy (information loss) over time, evenly distributing denoising difficulty without case-specific time reparameterizations. Furthermore, likelihood-based embedding optimization creates structured geometries that drive significant likelihood gains. --- ## 摘要 > ## Abstract 尽管扩散机制近期吸引了语言建模领域的极大关注，但连续扩散在规模扩展性上此前似乎一直落后于离散方法。为了挑战这一观点，我们重新探讨了基于似然驱动的连续扩散语言模型 (DLM) Plaid，并通过将 Plaid 的网络架构与现代离散 DLM 进行对齐，构建了 RePlaid。在这一统一评测体系下，我们确立了首个足以匹敌离散 DLM 的连续 DLM 扩展法则：RePlaid 与自回归模型相比仅存在 $20\times$ 的算力差距，在使用更少参数的情况下战胜了 Duo，并在过度训练机制下超越了 MDLM。我们在最新的连续 DLM 基准上对 RePlaid 进行了评估：在 OpenWebText 上，RePlaid 实现了 22.1 的全新最佳 PPL 上界，并展现出更为优越的生成质量。这些结果表明，当采用似然目标进行训练时，连续扩散是离散 DLM 极具竞争力且具备强扩展性的替代方案。此外，我们提供了理论视角来阐释似然训练的优势。我们证明，优化噪声调度以最小化 ELBO 的方差会自然带来跨时间的线性交叉熵（信息损失），这无需任何特定用例的时间重参数化即可平稳分布去噪难度。同时，我们发现通过似然优化词嵌入能形成富有规律的几何结构，并带来最为显著的似然收益。 > While diffusion has drawn considerable recent attention from the language modeling community, continuous diffusion has appeared less scalable than discrete approaches. To challenge this belief we revisit Plaid, a likelihood-based continuous diffusion language model (DLM), and construct RePlaid by aligning the architecture of Plaid with modern discrete DLMs. In this unified setting, we establish the first scaling law for continuous DLMs that rivals discrete DLMs: RePlaid exhibits a compute gap of only $20\times$ compared to autoregressive models, outperforms Duo while using fewer parameters, and outperforms MDLM in the over-trained regime. We benchmark RePlaid against recent continuous DLMs: on OpenWebText, RePlaid achieves a new state-of-the-art PPL bound of $22.1$ among continuous DLMs and superior generation quality. These results suggest that continuous diffusion, when trained via likelihood, is a highly competitive and scalable alternative to discrete DLMs. Moreover, we offer theoretical insights to understand the advantage of likelihood-based training. We show that optimizing the noise schedule to minimize the ELBO's variance naturally yields linear cross-entropy (information loss) over time. This evenly distributes denoising difficulty without any case-specific time reparameterization. In addition, we find that optimizing embeddings via likelihood creates structured geometries and drives the most significant likelihood gain. --- *(与本文相关的版权声明图标： ![license icon](./images/345c7ad61f1b.png) [查看许可协议](http://creativecommons.org/licenses/by/4.0/))* > *(License icon associated with this article: ![license icon](./images/345c7ad61f1b.png) [View License](http://creativecommons.org/licenses/by/4.0/))*

这不是一只烟斗：将 AI 系统视为语义抽象的形式化框架

Sun, 13 Sep 2026 00:00:00 GMT

正如著名超现实主义画作《这不是一只烟斗》所揭示的“画作并非烟斗本体”，AI 系统的输出同样不是客观世界事实本身，而是一种由工程化构建出来的符号表示与语义抽象。本文由抽象解释理论奠基人 Patrick Cousot 与形式化验证学者 Jade Alglave 共同撰写，旨在打破当前大模型仅凭“表面流畅性”制造事实幻觉的困局。作者通过严密的形式化语义框架，解耦了领域公认知识、外部参考源以及系统可访问信息三个核心维度，为 AI 的各类典型失效模式 (如过度外推、被推翻断言、数据源不匹配等) 奠定了数学诊断基础。这一框架为未来 AI 智能体在调用工具、引用事实和执行关键决策时，从单纯的“概率生成”迈向“权威可验证”提供了坚实的理论支柱。 --- ## 📌 内容概要 > # Summary **《这不是一只烟斗：将 AI 系统视为语义抽象》 (Ceci n'est pas une pipe: AI systems as semantic abstractions) ** 提出了一个严谨的形式化语义框架，用于深入分析 AI 系统输出的正确性。该论文由 Jade Alglave 与 Patrick Cousot 联合撰写，核心论点指出：AI 的输出结果绝不应被直接等同于客观事实或真实世界状态，而应被视作一种经过工程化构造的表征 (即一种语义抽象) 。 > **"Ceci n'est pas une pipe: AI systems as semantic abstractions"** proposes a formal semantic framework to analyze the correctness of AI system outputs. Authored by Jade Alglave and Patrick Cousot, the paper argues that an AI's output should be treated as an engineered representation (an abstraction) rather than an objective fact or direct description of the world. 通过严格区分**公认的领域知识**、**外部参考源信息**以及**当前系统可调用的信息**，作者建立了一套完备而严密的词汇体系，能够精准诊断 AI 常见的各类失效模式——例如主观过度外推、缺乏证据或已被证伪的断言、数据源与背景知识冲突、引用过时信息等。这有助于将 AI 的行为牢固锚定在明确的权威依据之上，而非仅仅停留在表面上的语言流畅度。 > By distinguishing between accepted domain knowledge, reference sources, and accessible system information, the authors establish a rigorous vocabulary to precisely diagnose common AI failure modes—such as extrapolation, refuted or unsupported assertions, source mismatches, and stale references—helping to ground AI actions in explicit authority rather than superficial fluency. --- ## 📑 论文元数据 > # Metadata * **arXiv 编号：** [arXiv:2607.09489](https://arxiv.org/abs/2607.09489) [cs.AI] * **学科领域：** 人工智能 (`cs.AI`)；程序设计语言 (`cs.PL`) * **论文作者：** Jade Alglave, Patrick Cousot * **首次提交：** 2026年7月10日 * **最新修订：** 2026年9月10日 (v2) * **DOI 链接：** [10.48550/arXiv.2607.09489](https://doi.org/10.48550/arXiv.2607.09489) > * **arXiv ID:** [arXiv:2607.09489](https://arxiv.org/abs/2607.09489) [cs.AI] > * **Subjects:** Artificial Intelligence (`cs.AI`); Programming Languages (`cs.PL`) > * **Authors:** Jade Alglave, Patrick Cousot > * **Submitted:** July 10, 2026 > * **Revised:** September 10, 2026 (v2) > * **DOI:** [10.48550/arXiv.2607.09489](https://doi.org/10.48550/arXiv.2607.09489) --- ## 📄 论文摘要 > # Abstract > AI 系统的输出并不是它看似正在描述的客观事实或现实世界状态，而是一种工程化构建的表征。我们提出了一个用于刻画 AI 系统的语义框架，以便能够系统检验此类表征的正确性。为此，我们严格区分了公认领域知识所证实的范畴、参考来源所陈述的内容，以及系统当前实际能够利用的信息。基于这套界定，我们能够为常见的 AI 失效模式给出精确定义：包括过度外推、被驳斥或缺乏支撑的论断、数据源与背景知识的不匹配、失效或过时的引用来源、擅自添加假设、未经授权的操作等等。我们期望该框架能够提供一套实用的规约与验证词汇，从而对 AI 系统的输出内容、引用依据、工具调用以及改变现实世界的行动进行严格审查，使其必须立足于可信断言与显式权威授权，而不再被表面上的顺畅表达所蒙蔽。 > > An AI system's output is not the fact or world state it appears to describe, but rather an engineered representation. We propose a semantic framework to describe AI systems, to be able to examine the correctness of such representations. To do so, we distinguish what is justified by accepted domain knowledge, what reference sources say, and what the system can currently use. This allows us to give precise definitions to common failures: extrapolation, refuted or unsupported assertion, sources versus knowledge mismatch, stale or refuted source, added hypotheses, unsupported use... We hope our framework gives a useful vocabulary for specifying and checking AI systems whose outputs, citations, tool calls, and world-changing actions must be justified by reliable claims and explicit authority rather than apparent fluency. --- ## 🔗 访问链接与资源 > # Links & Resources * **全文阅读：** * [查看 PDF](https://arxiv.org/pdf/2607.09489) * [HTML 网页版 (实验性)](https://arxiv.org/html/2607.09489v2) * [TeX 源码](https://arxiv.org/src/2607.09489) * **引用与学术指标：** * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2607.09489) * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2607.09489) * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2607.09489) > * **Full-Text Access:** > * [View PDF](https://arxiv.org/pdf/2607.09489) > * [HTML Version (Experimental)](https://arxiv.org/html/2607.09489v2) > * [TeX Source](https://arxiv.org/src/2607.09489) > * **Citations & Metrics:** > * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2607.09489) > * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2607.09489) > * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2607.09489)

超越提示词工程：基于对数几率空间融合的语音大模型高效鲁棒上下文偏置 (LOGIC)

Sun, 13 Sep 2026 00:00:00 GMT

语音大语言模型 (Speech Large Language Models, Speech LLMs) 在通用人机对话中展现出色，但由于训练数据固化，难以准确识别快速涌现的专有名词、联系人姓名或个性化歌单等领域实体。传统的提示词 (Prompting) 方案在实体列表增加时面临上下文窗口膨胀、推理延迟飙升与“迷失在中间”等问题；而生成式纠错 (Generative Error Correction, GEC) 则常导致过度纠正与虚假实体幻觉。为此，作者提出了 **LOGIC** 框架，直接在模型的解码层对数几率 (Logit) 空间中进行上下文信息融合，彻底将上下文注入与输入处理解耦，实现了相对提示词长度的常数级时间复杂度。在多模态模型 Phi-4-MM 跨越 11 种跨国语言环境的测试中，LOGIC 将实体词错误率 (WER) 相对降低了 9%，而虚警率增幅仅为微乎其微的 0.30%。 --- ## 📌 内容概要 > ## 📌 Summary 语音大语言模型 (Speech LLMs) 在日常通用对话任务中表现优异，但受限于静态的预训练知识库，往往难以有效识别迅速更迭的新词或特定领域实体 (例如个人联系人姓名、专业术语或个性化音乐播放列表) 。 > Speech Large Language Models (Speech LLMs) excel at general conversational tasks, but struggle to recognize rapidly emerging or domain-specific entities (such as contact names, technical jargon, or playlists) due to their static training knowledge. 传统的解决思路通常依赖**提示词工程** (Prompting) ，但其扩展能力极差——随着待识别实体列表的扩充，很容易遭遇上下文窗口饱和、推理延迟成倍上升以及“迷失在中间” (Lost-in-the-Middle) 现象。与此同时，另一种**生成式纠错** (GEC) 方案则经常面临“过度纠错”的问题，甚至会凭空凭造出音频中完全不存在的实体幻觉。 > Traditional solutions like **prompting** fail to scale efficiently, leading to context window limitations, high inference latency, and the "lost-in-the-middle" phenomenon. Meanwhile, **Generative Error Correction (GEC)** suffers from over-correction and entity hallucinations. 为破解上述难题，研究团队提出了 **LOGIC** (*Logit-Space Integration for Contextual Biasing*，即对数几率空间上下文偏置融合) 框架，该方法直接在神经网络的最终解码层实施干预。通过将上下文偏置的注入过程与输入端表征处理完全解耦，LOGIC 实现了相对于提示词长度的常数级时间复杂度 $\mathcal{O}(1)$。在搭载 `Phi-4-MM` 模型并覆盖 11 种多语言区域设置的实验中，LOGIC 取得了**实体词错误率 (Word Error Rate, WER) 平均相对下降 9%** 的显著提升，且**虚警率 (False Alarm Rate) 仅轻微上升 0.30%**。 > To solve this, the authors introduce **LOGIC** (*Logit-Space Integration for Contextual Biasing*), a framework operating directly in the decoding layer. By decoupling context injection from input processing, LOGIC achieves constant-time complexity relative to prompt length. Experiments using the `Phi-4-MM` model across 11 multilingual locales show an average **9% relative reduction in Entity Word Error Rate (WER)** with only a **0.30% increase in the False Alarm Rate**. --- ## 📄 论文元数据 > ## 📄 Paper Metadata * **arXiv 编号：** [2601.15397](https://arxiv.org/abs/2601.15397) * **主学科领域：** 人工智能 (`cs.AI`) * **次要学科领域：** 计算与语言 (`cs.CL`)，声音音频 (`cs.SD`) * **论文作者：** Peidong Wang, Jian Xue, Jinyu Li * **提交历史：** * `[v1]` 2026年1月21日 * `[v3]` 2026年9月10日 (最新修订版本) > * **arXiv ID:** [2601.15397](https://arxiv.org/abs/2601.15397) > * **Primary Subject:** Artificial Intelligence (`cs.AI`) > * **Secondary Subjects:** Computation and Language (`cs.CL`), Sound (`cs.SD`) > * **Authors:** Peidong Wang, Jian Xue, Jinyu Li > * **Submission History:** > * `[v1]` 21 Jan 2026 > * `[v3]` 10 Sep 2026 (latest revision) --- ## 🔍 论文摘要 > ## 🔍 Abstract > 受文化演变、网络热点变迁以及用户高度定制化数据驱动，新实体的快速涌现给现有的语音大语言模型 (Speech LLMs) 带来了严峻挑战。尽管这些模型在宽泛的通用对话任务中表现卓越，但其静态的训练期知识库极大地限制了其识别特定领域词汇的能力，例如通讯录人名、歌单名称或行业技术行话。现存主流方案主要依赖提示词，但其可扩展性极差：随着实体清单长度的增长，提示词方法面临着上下文窗口受限、推理延迟增加以及“迷失在中间”现象。另一类替代方法即生成式纠错 (GEC)，试图通过后处理机制重写转写文本，但又极易陷入“过度纠正”的陷阱，凭空臆造出说话人根本未提及的虚假实体。 > > 在本工作中，我们提出了 **LOGIC** (基于对数几率空间融合的上下文偏置机制) ，这是一个直接在解码层操作的高效且鲁棒的框架。与提示词机制截然不同，LOGIC 将上下文注入与输入处理深度解耦，从而保证了关于提示词长度的常数级时间复杂度。利用 Phi-4-MM 模型在横跨 11 种多语言本地化场景下开展的大规模实验表明，LOGIC 实现了实体词错误率 (Entity WER) 平均 9% 的相对下降，且虚警率增量仅为可以忽略不计的 0.30%。 > > The rapid emergence of new entities -- driven by cultural shifts, evolving trends, and personalized user data -- poses a significant challenge for existing Speech Large Language Models (Speech LLMs). While these models excel at general conversational tasks, their static training knowledge limits their ability to recognize domain-specific terms such as contact names, playlists, or technical jargon. Existing solutions primarily rely on prompting, which suffers from poor scalability: as the entity list grows, prompting encounters context window limitations, increased inference latency, and the "lost-in-the-middle" phenomenon. An alternative approach, Generative Error Correction (GEC), attempts to rewrite transcripts via post-processing but frequently suffers from "over-correction", introducing hallucinations of entities that were never spoken. > > > > In this work, we introduce **LOGIC** (Logit-Space Integration for Contextual Biasing), an efficient and robust framework that operates directly in the decoding layer. Unlike prompting, LOGIC decouples context injection from input processing, ensuring constant-time complexity relative to prompt length. Extensive experiments using the Phi-4-MM model across 11 multilingual locales demonstrate that LOGIC achieves an average 9% relative reduction in Entity WER with a negligible 0.30% increase in False Alarm Rate. --- ## 🔗 全文获取与参考文献 > ## 🔗 Full-Text & References * **论文研读：** [查看 PDF](https://arxiv.org/pdf/2601.15397) | [HTML 网页版](https://arxiv.org/html/2601.15397v3) * **源码与学术指标：** * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2601.15397) * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2601.15397) * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2601.15397) > * **Read the Paper:** [View PDF](https://arxiv.org/pdf/2601.15397) | [HTML Version](https://arxiv.org/html/2601.15397v3) > * **Source Code & Scholarly Citations:** > * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2601.15397) > * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2601.15397) > * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2601.15397)

语用信息论的数学理论：统一通信、控制与决策

Sun, 13 Sep 2026 00:00:00 GMT

1948 年 Claude Shannon 创立了经典信息论，奠定了现代数字通信的基石，但他明确将信息的语义与语用（即信息产生的作用和价值）排除在理论之外。如今随着具身智能 (Embodied AI)、网络化控制与自主系统的快速发展，智能系统不再仅仅追求比特的无损传输，更追求“信息能否有效指导正确的行动”。北京邮电大学 Kai Niu 与 Ping Zhang 教授发表了长达 152 页的重磅长文，提出了统一通信、控制与决策的“语用信息论”。该理论以“异途同归映射 (Isoteleia Mapping)”为数学核心，建立了语法-语义-语用三层信息体系，证明了推广 Shannon 经典理论的三大语用编码定理，确立了智能系统在资源约束下的语用效率极限与行为信道容量，为下一代任务导向通信与机器智能构筑了严谨坚实的数学根基。 --- ## 📌 核心概要 > ## Executive Summary 本文提出了一套完备且严谨的语用信息论 (Pragmatic Information Theory)，成功架起了沟通通信、控制理论与决策科学之间的统一桥梁。该理论的核心支柱是**异途同归映射 (Isoteleia Mapping)**，它在数学上形式化了“殊途同归 (Equifinality)”原则——即引导系统走向完全相同的最优行动的各种不同语义路径，在语用层面是完全等价的。通过彻底剔除与任务无关的冗余区分，该框架确立了“语法、语义、语用”的三层信息层次体系，通过证明全新的编码定理全面推广了经典香农信息论，为下一代智能系统、目标导向决策与具身智能 (Embodied AI) 提供了坚实可靠的数学根基。 > This paper introduces a comprehensive pragmatic information theory that bridges communication, control, and decision-making. At its heart is the **isoteleia mapping**, which formalizes equifinality—the principle that distinct semantic paths leading to the same optimal action are pragmatically equivalent. By discarding task-irrelevant distinctions, the framework establishes a three-tier hierarchy (syntactic, semantic, and pragmatic information), extends classical information theory through new coding theorems, and offers a rigorous foundation for next-generation intelligent systems, goal-directed action, and embodied AI. --- ## 📑 论文摘要 > ## Abstract 我们提出了一套统一通信、控制与决策过程的语用信息理论。其数学核心是**异途同归映射 (Isoteleia Mapping)**，形式化表征了“殊途同归”特性：能够达成相同最优行动的不同语义路径，在语用层面上皆属等价。这自然诱导出了一个由语法信息、语义信息与语用信息构成的三层抽象体系，每一层抽象都精准过滤掉与当前目标任务无关的冗余差异。 > We propose a pragmatic information theory unifying communication, control, and decision-making. Its core is the **isoteleia mapping**, formalizing equifinality: distinct semantic paths leading to the same optimal action are pragmatically equivalent. This induces a three-tier hierarchy of syntactic, semantic, and pragmatic information, each abstraction discarding task-irrelevant distinctions. 我们系统建立了语用熵、上/下互信息、信道容量以及率失真理论，并证明了推广 Shannon 经典理论成果的三大语用编码定理。我们将信息的语用价值 (Value of Information, VoI) 与信息成本 (Cost of Information, CoI) ，分别作为率失真理论与信道容量理论在决策论层面的对偶概念引入，并构建了用于跨层联合优化的 Lagrange 对偶框架。 > We develop pragmatic entropy, up/down mutual information, channel capacity, and rate-distortion, and prove three coding theorems generalizing Shannon's classical results. We introduce pragmatic value (VoI) and cost (CoI) of information as decision-theoretic duals to rate-distortion and capacity, respectively, and formulate a Lagrangian dual framework for cross-layer optimization. 语用效率上界： $$\mathcal{E}_p(\lambda) = \sup_R [\Phi_p(R) - \lambda\,\mathrm{CoI}_p(R)]$$ 量化了任何受限于资源约束的智能系统所能汲取的最大净效用，从而确立了一项根本性的“行为容量极限 (Behavioral Capacity Limit)”——将 Shannon 符号级的传输容量极限成功推广到了目标导向的具身行动层面。对于连续变量消息，该理论推广给出了闭式的高斯分布解析表达式；而在时序动态环境中，则通过结合序贯决策的 Bellman 方程予以拓展。该理论框架为任务导向通信、网络化控制、自主无人系统以及具身智能构筑了严密的数学基础，推动信息科学从追求“符号级的高保真度传输”迈向追求“信息指导行动的真实效能”，为下一代机器智能体系提供了统一的数学语言。 > The pragmatic efficiency bound > > $$\mathcal{E}_p(\lambda) = \sup_R [\Phi_p(R) - \lambda\,\mathrm{CoI}_p(R)]$$ > > quantifies the maximum net utility any resource-constrained intelligent system can extract, thereby establishing a fundamental behavioral capacity limit—generalizing Shannon's symbol-level capacity to goal-directed action. Extensions to continuous messages yield closed-form Gaussian expressions, while dynamic settings are addressed via a Bellman equation for sequential decision-making. This framework provides a rigorous foundation for task-oriented communication, networked control, autonomous systems, and embodied AI, shifting focus from symbol fidelity to the effectiveness of information in guiding actions, and offers a unified mathematical language for next-generation intelligent systems. --- ## 🧠 理论框架的核心支柱 > ## Key Framework Elements * **异途同归映射 (The Isoteleia Mapping)**：在数学上严格刻画了殊途同归现象，即多条不同的语义发展轨迹最终可收敛于唯一的语用最优行动输出。 * **三层信息层次架构**：涵盖语法、语义和语用三级渐进抽象，专为深度剔除与任务无关的冗余干扰而设计。 * **推广的广义信息度量**：正式确立了语用熵、上下互信息变体、语用信道容量以及语用率失真函数边界。 * **信息的语用价值与语用成本 (VoI & CoI)**：构建了与信道容量和率失真理论紧密对偶的决策论数学工具。 * **动态时序决策扩展**：引入动态规划与 Bellman 方程，实现从静态单步优化向长周期、时间依赖型序贯决策场景的无缝泛化。 > * **The Isoteleia Mapping:** Formalizes equifinality where multiple semantic trajectories converge on a singular optimal pragmatic outcome. > * **Three-Tier Information Hierarchy:** Syntactic, semantic, and pragmatic abstractions optimized to filter out task-irrelevant noise. > * **Generalized Information Measures:** Establishes pragmatic entropy, mutual information variations, channel capacity, and rate-distortion bounds. > * **Value and Cost of Information (VoI & CoI):** Decision-theoretic duals aligned with capacity and rate-distortion theories. > * **Dynamic Decision-Making:** Incorporates Bellman equations to seamlessly scale from static optimization to sequential, time-dependent scenarios.

语义提升算子与保持性下不可判定类的闭包性

Sun, 13 Sep 2026 00:00:00 GMT

在理论计算机科学中，著名的莱斯定理 (Rice's Theorem) 指出：关于程序静态语义性质的所有非平凡判断都是不可判定的。然而，在以自主智能体和自我修改系统 (Self-Modifying Systems) 为代表的动态环境下，人们不再仅仅关心静态问题“代码 $x$ 是否满足性质 $P$”，而是迫切需要验证“在系统被变换 $\Phi$ 动态重写后，性质 $P$ 是否仍能得以保持”。本文提出了形式化的“语义提升算子” ($\Lambda\Phi$)，深入探讨了该动态保持性问题的可判定性边界。研究证明，不可验证性质类在提升算子下依然保持封闭，且无限迭代该算子将直接攀升至算术阶层的 $\Pi_0^2$ 完全性，严格揭示了有限级监督验证器无法为自我修改系统提供无条件安全凭证的数学本质。 --- # 语义提升算子与保持性下不可判定类的闭包性 > # The Semantic Elevation Operator and the Closure of the Undecidable Class under Preservation **作者：** Jose Pascual Gumbau Mezquita **提交日期：** 2026年9月10日 **研究领域：** 计算机科学逻辑 (`cs.LO`)；人工智能 (`cs.AI`)；计算与语言 (`cs.CL`)；数理逻辑 (`math.LO`) **引用格式：** [arXiv:2609.11326 [cs.LO]](https://arxiv.org/abs/2609.11326) > **Authors:** Jose Pascual Gumbau Mezquita > **Submitted on:** 10 September 2026 > **Subjects:** Logic in Computer Science (`cs.LO`); Artificial Intelligence (`cs.AI`); Computation and Language (`cs.CL`); Logic (`math.LO`) > **Cite as:** [arXiv:2609.11326 [cs.LO]](https://arxiv.org/abs/2609.11326) --- ## 概要 > ## Summary 传统上，莱斯定理 (Rice's Theorem) 确立了程序静态语义性质的不可判定性。然而，对于能够自我修改的系统，我们必须评估当系统随着时间不断自我重写时，某项性质是否依然得以*保持*。这便将传统的静态查询“$x$ 是否满足性质 $P$？”转变为动态问题“在 $x$ 被变换 $\Phi$ 转换后，性质 $P$ 是否得以保持？”。 > Rice's theorem traditionally dictates the undecidability of static semantic properties for programs. However, self-modifying systems require evaluating whether a property remains *preserved* as the system rewrites itself over time, transforming the static query *"Does $x$ satisfy $P$?"* into the dynamic question *"Is $P$ preserved after $x$ is transformed by $\Phi$?"* 在本文中，Jose Pascual Gumbau Mezquita 使用**语义提升算子 ($\Lambda\Phi$)** 对这种转变进行了形式化建模。其核心研究发现包括： * **内涵性下的不可判定性：** 当 $\Phi$ 是内涵性的（即依赖于源代码本身而非仅仅取决于计算函数）时，提升后的性质仍然是不可判定的，该结论通过克林递归定理 (Kleene's Recursion Theorem) 巧妙规避了莱斯定理所要求的外延性前提。 * **闭包性质：** 不可验证性质构成的类 $\mathcal{U}$ 在语义提升算子下是封闭的。 * **算术阶层：** 对该算子进行无界迭代，其复杂度在算术阶层中不断攀升直至达到 $\Pi_0^2$ 完全性，从而在结构层面确立了其不可验证性特征。 * **监督倒退困境：** 论文证明了监督倒退 (Supervisory Regress) 无法终止；任何有限层级、能力不断增强的验证器塔都不可能给出无条件的安全性保证凭证。 * **未来方向：** 作者提出在有效拓扑斯 (Effective Topos) 中给出范畴论解释，将语义提升视为罗威不动点定理 (Lawvere's Fixed-Point Theorem) 的一个实例，作为未来的深入研究方向。 > In this paper, Jose Pascual Gumbau Mezquita formalizes this transition using a **semantic elevation operator ($\Lambda\Phi$)**. The key findings include: > * **Undecidability under Intensionality:** When $\Phi$ is intensional (depending on source code rather than just the computed function), the elevated property remains undecidable, bypassing the extensionality requirements of Rice's theorem via Kleene's recursion theorem instead. > * **Closure Properties:** The class $\mathcal{U}$ of non-verifiable properties is shown to be closed under the semantic elevation operator. > * **Arithmetical Hierarchy:** Unbounded iteration of the operator climbs the arithmetical hierarchy up to $\Pi_0^2$-completeness, solidifying non-verifiability as a structural characteristic. > * **Supervisory Regress:** The paper demonstrates that the supervisory regress does not terminate; no finite tower of increasingly capable verifiers can yield an unconditional certificate. > * **Future Directions:** A categorical interpretation within the effective topos, framing elevation as an instance of Lawvere's fixed-point theorem, is proposed for future work. --- ## 文章详情与链接 > ## Article Details & Links * **全文选项：** * [查看 PDF](https://arxiv.org/pdf/2609.11326) * [HTML 版本 (实验性)](https://arxiv.org/html/2609.11326v1) * [TeX 源码](https://arxiv.org/src/2609.11326) * **授权协议：** [知识共享署名-非商业性使用-禁止演绎 4.0 国际许可协议 (CC BY-NC-ND 4.0)](http://creativecommons.org/licenses/by-nc-nd/4.0/) > * **Full-Text Access:** > * [View PDF](https://arxiv.org/pdf/2609.11326) > * [HTML Version (Experimental)](https://arxiv.org/html/2609.11326v1) > * [TeX Source](https://arxiv.org/src/2609.11326) > * **License:** [Creative Commons Attribution-NonCommercial-NoDerivatives 4.0](http://creativecommons.org/licenses/by-nc-nd/4.0/) > > > --- ## 参考文献与指标 > ## References & Metrics * **唯一标识符：** [DOI: 10.48550/arXiv.2609.11326](https://doi.org/10.48550/arXiv.2609.11326) * **外部检索索引：** * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.11326) * [Google 学术](https://scholar.google.com/scholar_lookup?arxiv_id=2609.11326) * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.11326) > * **Identifiers:** [DOI: 10.48550/arXiv.2609.11326](https://doi.org/10.48550/arXiv.2609.11326) > * **External Indices:** > * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.11326) > * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.11326) > * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.11326)

解构 EGGROLL：大规模低秩进化策略的理论理解与改进

Sun, 13 Sep 2026 00:00:00 GMT

进化策略 (Evolution Strategies, ES) 作为一种无需梯度的黑盒优化方法，在不可微目标与强化学习中具有独特的分布式扩展优势，但其在参数动辄数十亿的大语言模型 (Large Language Model, LLM) 上常常因高昂的内存与通信开销而难以落地。为此，学术界提出了 EGGROLL 算法，通过采用低秩（如秩为一）的高斯扰动乘积来替代稠密高斯噪声，显著降低了计算与通信瓶颈。本论文深入揭示了有限秩低秩进化策略背后的数学机制，定量分析了其几何约束与方差效率，并提出了全新的留一法估计器——**LOO-ROLL**。在等价评估预算下，LOO-ROLL 将 Transformer 模块的均方误差削减了一半，并在 8B 级大模型的后训练测试中使 GSM8K 数学推理准确率从 65.9% 飙升至 80.0%。 --- ## 📌 内容摘要 > ## 📌 Summary **EGGROLL** 是一项旨在将进化策略 (Evolution Strategies, ES) 成功推向大语言模型 (LLM) 规模化应用的创新技术。它通过将传统密集的矩阵级高斯权重扰动替换为低秩高斯乘积（通常为秩为一的形式），从而大幅降低显存和通信开销。尽管该方法在计算上极其高效，但在几何层面上却引入了严苛的约束，因为每个低秩扰动仅存在于周围高维矩阵空间的一个测度为零的子集中。 > **EGGROLL** is a technique that makes evolution strategies (ES) practical for large language models (LLMs) by substituting dense Gaussian weight perturbations with low-rank Gaussian products (frequently rank-one). While computationally efficient, this introduces geometric constraints since each perturbation lives in a zero-volume subset of the ambient matrix space. 本篇论文系统探究了 EGGROLL 在有限秩设定下的理论力学原理，并在此基础上提出了全新的 **LOO-ROLL**（基于留一法 Leave-One-Out 的估计器）。LOO-ROLL 能够在完全保持种群优化性能的同时，将模型的评估计算成本直接削减一半。 > This paper investigates the theoretical mechanics of EGGROLL at finite rank and proposes **LOO-ROLL**, a leave-one-out estimator that retains population performance while cutting evaluation costs in half. --- ## 🔍 核心发现与研究贡献 > ## 🔍 Key Findings & Contributions * **理论机理剖析**：作者深入分析了在有限秩与非零扰动半径下 EGGROLL 的平均更新场，清晰揭示了一个显式预解算子 (Resolvent) 是如何作用于平滑后目标函数梯度的全过程。 * **稳定性与精确性**：尽管该预解算子可能会引入非保守分量并引起局部稳定性的偏移，但 EGGROLL 在任意秩与扰动半径下，对于所有二次目标函数依然能保持严格的精确性。 * **极高的方差效率**：在局部仿射模型假设下，相比于稠密高斯扰动进化策略，秩一扰动给梯度估计器带来的方差增加仅为 $\frac{2(m+n+1)}{mn+1}$ （例如在 $4096 \times 4096$ 的矩阵中，方差仅微幅增加了 $0.098\%$ ）。 * **LOO-ROLL 估计器**：提出了全新的留一法估计器，替代了 EGGROLL 原先每个扰动方向需要两次对偶评估的常规做法，仅需单次评估即可完成，在相同评估开销下使 Transformer 模块的估计器均方误差 (MSE) 降低了整整一半。 * **规模化实证验证**：在最高达 8B 参数的大语言模型的 10 种后训练场景中进行了严谨测试，LOO-ROLL 在其中 7 组配对测试中均取得了显著提升且无任何性能回退。值得注意的是，在 GSM8K 基准测试中，0.6B 模型的准确率从 $38.1\%$ 跃升至 $63.0\%$ ，而 8B 模型的准确率也从 $65.9\%$ 大幅攀升至 $80.0\%$ 。 > * **Theoretical Characterization:** The authors analyze the mean EGGROLL update field at finite rank and nonzero perturbation radii, showing how an explicit resolvent is applied to the smoothed objective's gradient. > * **Stability & Exactness:** While the resolvent can introduce nonconservative components and shift local stability, EGGROLL remains exact on all quadratic objectives across any rank and radius. > * **Variance Efficiency:** Under a local affine model, rank-one perturbations increase gradient estimator variance by only $\frac{2(m+n+1)}{mn+1}$ compared to dense Gaussian ES (e.g., just $0.098\%$ for a $4096 \times 4096$ matrix). > * **LOO-ROLL Estimator:** A new leave-one-out estimator that replaces EGGROLL's standard two antithetic evaluations per direction with a single evaluation, cutting estimator Mean Squared Error (MSE) in half for transformer blocks at equivalent evaluation costs. > * **Empirical Validation:** Tested across ten post-training settings on models up to 8B parameters, LOO-ROLL secured improvements in seven paired tests with no performance drop. Notably, GSM8K benchmark accuracy surged from $38.1\%$ to $63.0\%$ for 0.6B models, and from $65.9\%$ to $80.0\%$ for 8B models. --- ## 🔗 论文资源与相关链接 > ## 🔗 Links & Resources * [阅读 PDF 论文](https://arxiv.org/pdf/2609.10980) * [arXiv 在线 HTML 版](https://arxiv.org/html/2609.10980v1) * [TeX 源代码](https://arxiv.org/src/2609.10980) * [DOI 链接](https://doi.org/10.48550/arXiv.2609.10980) > * [View PDF](https://arxiv.org/pdf/2609.10980) > * [arXiv HTML Version](https://arxiv.org/html/2609.10980v1) > * [TeX Source](https://arxiv.org/src/2609.10980) > * [DOI](https://doi.org/10.48550/arXiv.2609.10980)

表征任务电力弹性：面向柔性电网的 AI 训练动态功率分配

Sun, 13 Sep 2026 00:00:00 GMT

在生成式 AI 技术迅猛发展的当下，大语言模型 (Large Language Model, LLM) 训练已经成为现代数据中心电力消耗激增的关键源头，电网供电上限正成为限制算力集群扩张的核心瓶颈。为了在电网负荷高峰期实现动态削峰填谷，系统调度必须精确把握：当调低 GPU 功率墙时，模型训练的 Token 产出速度究竟会发生怎样的变化？本文首次系统性地提出了“任务电力弹性 (Job Power Elasticity)”这一度量体系，并构建了标准化的电力弹性指数 (Power Flexibility Index, PFI)。通过在 NVIDIA H200 与 H100 集群上完成的 131 次大规模实测，团队证实 PFI 感知调度在整体供电减少 30% 的极限约束下，能够弥补 63% 的吞吐量损失，为打造电网友好型与绿色低碳的 AI 基础设施提供了坚实支撑。 --- # 表征任务电力弹性：面向柔性电网的 AI 训练动态功率分配 > # Characterizing Job Power Elasticity for Power-Flexible AI Training ## 核心执行摘要 > ## Executive Summary 大语言模型 (Large Language Model, LLM) 训练正迅速成为现代数据中心电力消耗增长的核心推手，供电容量的短缺已成为制约 AI 基础设施进一步扩张的瓶颈所在。本文首次对**任务电力弹性 (Job Power Elasticity)**开展了系统化表征——即研究在降低 GPU 功耗限制时，大模型训练吞吐量所产生的敏感度变化规律。为了量化这一特征，作者提出了**电力弹性指数 (Power Flexibility Index, PFI)**。这是一个用于评估降功耗性能代价的归一化指标，同时也可作为满足服务等级协议 (SLA) 的功率分配控制原语。通过在 H200 和 H100 GPU 集群上开展 131 次跨越不同模型规模的实测，该研究证明：基于 PFI 感知的智能功率分配机制能够在严苛的供电限制下最大化训练吞吐量，相比于传统的均匀功率削减方案，挽回了极其显著的算力性能损失。 > Large language model (LLM) training is rapidly becoming a primary driver of electricity demand in modern data centers, pushing power availability to a critical bottleneck. This paper presents the first systematic characterization of **job power elasticity**—the sensitivity of training throughput to GPU power reductions. To quantify this behavior, the authors introduce the **Power Flexibility Index (PFI)**, a normalized metric measuring performance cost that doubles as a control primitive for SLA-aware power allocation. Through empirical testing across 131 LLM training runs on H200 and H100 GPUs, the study demonstrates that intelligent, PFI-aware power allocation can maximize throughput under tight energy constraints, recovering a significant portion of performance compared to naive equal-weight allocations. --- ## 论文元数据 > ## Paper Metadata * **arXiv 编号：** [arXiv:2609.11542](https://arxiv.org/abs/2609.11542) [cs.AI] * **提交日期：** 2026年9月10日 * **所属领域：** 人工智能 (`cs.AI`) * **论文作者：** * Philip Colangelo * Charles Dawson * Shayan Sengupta * Ayse Coskun * Varun Sivaram > * **arXiv Identifier:** [arXiv:2609.11542](https://arxiv.org/abs/2609.11542) [cs.AI] > * **Submission Date:** September 10, 2026 > * **Subjects:** Artificial Intelligence (`cs.AI`) > * **Authors:** > * Philip Colangelo > * Charles Dawson > * Shayan Sengupta > * Ayse Coskun > * Varun Sivaram --- ## 论文摘要 > ## Abstract 大语言模型 (LLM) 训练是现代数据中心电力需求增长最迅猛的来源之一，供电容量已成为制约 AI 基础设施持续扩张的主要瓶颈。若能赋予此类计算工作负载灵活调节功耗的能力，不仅能为 AI 的算力增长释放额外的电力空间，还能平抑电价飙升并提高现有电网基础设施的利用率。然而，要实现这一电力灵活性，我们必须首先弄清：当降低 GPU 功耗时，训练任务的性能究竟会发生怎样的动态变化。 > Large language model (LLM) training is among the fastest-growing sources of electricity demand in modern data centers, and power availability is a primary bottleneck to continued AI infrastructure growth. Making the power consumption of these workloads flexible could unlock additional power for AI growth, limit increases in electricity prices, and improve the utilization of existing grid infrastructure. However, to realize this flexibility, we must first understand how the performance of training workloads changes when GPU power is reduced. 本文首次系统化地表征了 LLM 训练中的**任务电力弹性 (Job Power Elasticity)**（即吞吐量对功率降低的敏感度）。为了对这种弹性进行量化，我们提出了**电力弹性指数 (Power Flexibility Index, PFI)**，这是一个用于量化功率削减带来的性能代价的归一化指标，并为兼顾服务等级协议 (SLA) 的电力灵活性调度提供了控制原语。 > This paper presents the first systematic characterization of **job power elasticity** (the sensitivity of throughput to power reductions) in LLM training. To quantify elasticity, we introduce the **Power Flexibility Index (PFI)**, a normalized metric that quantifies the performance cost of power reductions and provides a control primitive for SLA-aware power flexibility. 我们从 131 次基于 H200 的 LLM 训练任务（外加 24 次 H200 验证任务以及 34 次匹配的 H100 任务）中收集了完整数据，涵盖稠密模型与混合专家 (Mixture-of-Experts, MoE) 架构、预训练与微调任务，集群规模最高达 32 块 GPU。我们发现，LLM 训练任务表现出显著但差异较大的电力弹性，并识别出了能够在运行时精准预测 PFI 的遥测信号指标。最终，我们证明了基于 PFI 感知的功率分配能够在电力受限条件下最大化集群总 Token 吞吐率 (Tokens/s)。在整体电力压减 30% 的情况下，基于 PFI 的功率分配方案为每个任务追回约 1.5k Tokens/s 的吞吐量，弥合了均匀分配基准与具备完美信息的理论最优解 (Oracle) 之间 63% 的性能差距。我们的研究成果确立了电力弹性作为训练任务一项可测物理特性的地位，并为构建感知电力、响应电网的现代化 AI 基础设施奠定了理论基础。 > We collect data from 131 LLM training runs on H200 (plus 24 H200 validation runs and 34 matched H100 runs), including both dense and mixture-of-experts models, pretraining and fine-tuning tasks, and up to 32 GPUs. We find that LLM training jobs exhibit substantial but variable power elasticity, and we identify telemetry signals that predict PFI at runtime. Finally, we demonstrate that PFI-aware power allocation maximizes total tokens/second throughput under power constraints. Under a 30% power reduction, PFI-aware power allocation recovers ~1.5k tokens/s per job, 63% of the performance gap between an equal-weight allocation and an oracle with perfect information. Our results establish power elasticity as a measurable property of training jobs and provide a foundation for power-aware, grid-responsive AI infrastructure. --- ## 核心贡献与关键发现 > ## Key Contributions & Findings 1. **首度系统性表征：** 全面测定了涵盖稠密模型、混合专家 (MoE) 架构、预训练及微调任务在内的 LLM 训练工作负载的任务电力弹性。 2. **电力弹性指数 (PFI)：** 提出了一个标准归一化指标来衡量削减功率输入所带来的性能代价，可作为高效的运行时控制原语。 3. **运行时遥测预测：** 发现了能够在大模型运行期间精准预判 PFI 指标的具体硬件与系统遥测信号。 4. **优化的功率分配策略：** 提出了 PFI 感知的电力管理方案，在严苛电力约束下成功最大化全局吞吐率 (Tokens/s)，在削减 30% 电力时单任务挽回约 1.5k Tokens/s 的性能。 > 1. **First Systematic Characterization:** Establishes job power elasticity for LLM training workloads across dense models, mixture-of-experts (MoE) architectures, pretraining, and fine-tuning tasks. > 2. **Power Flexibility Index (PFI):** Introduces a normalized metric to measure the performance cost associated with reduced power inputs, acting as an effective runtime control primitive. > 3. **Runtime Telemetry Prediction:** Identifies specific telemetry signals capable of accurately predicting PFI during active execution. > 4. **Optimized Power Allocation:** Proposes PFI-aware power management that successfully maximizes throughput (tokens/second) under strict power constraints, recovering ~1.5k tokens/s per job under a 30% power reduction. --- ## 论文获取与资源 > ## Access & Resources * [查看 PDF 全文](https://arxiv.org/pdf/2609.11542) * [TeX 源码获取](https://arxiv.org/src/2609.11542) * [DOI 官方链接](https://doi.org/10.48550/arXiv.2609.11542) > * [View PDF](https://arxiv.org/pdf/2609.11542) > * [TeX Source](https://arxiv.org/src/2609.11542) > * [DOI Link](https://doi.org/10.48550/arXiv.2609.11542)

真相从未泯灭：顺从上下文真实性探针中的完全混叠现象

Sun, 13 Sep 2026 00:00:00 GMT

在研究大语言模型 (Large Language Model, LLM) 是否存在刻意欺骗或隐瞒真相的行为时，机械可解释性领域广泛采用线性探针 (Probing) 技术探测模型内部激活状态中的“真实性”表征。然而，如果训练探针的数据集总是默认“诚实作答”与“完成任务要求”完全重叠，那么探针究竟是学到了“事实真相”，还是仅仅学会了“顺从指令”？本文深入剖析了这一根本性隐患，并提出了“完全混叠 (Perfect Aliasing)”理论：在顺从上下文中拟合的探针，在数学上根本无法区分真相与任务指令，导致模型在被诱导说谎的对抗场景下，传统探针的判别力全面崩溃（AUROC 跌至接近 0）。通过引入随机码本解耦语义与指令，并采用混合上下文拟合，作者在经过奖励训练的 Gemma-2-9B 模型上证实，即使模型表面上输出虚假谎言，其内部表征依然能够以 1.000 的完美 AUROC 恢复出客观事实，为大模型测谎与真实性表征探测敲响了警钟并提供了科学指引。 --- # 真相从未泯灭：顺从上下文真实性探针中的完全混叠现象 > # The Truth Was Never Gone: Perfect Aliasing in Compliant-Context Truth Probes > **arXiv:** [2609.10739](https://arxiv.org/abs/2609.10739) [cs.LG] > **DOI:** [10.48550/arXiv.2609.10739](https://doi.org/10.48550/arXiv.2609.10739) > **Author:** Dylan Jayabahu > **Submitted:** September 9, 2026 > **Subjects:** Machine Learning (`cs.LG`); Artificial Intelligence (`cs.AI`); Computation and Language (`cs.CL`) --- ## 📌 核心概述 > ## 📌 Summary 本文深入探讨了机器学习模型中“真实性探针 (Truth Probes)”的内在局限性，并提出了**完全混叠 (Perfect Aliasing)**的概念。这是一种语义识别层面的根本性失效：当“如实汇报”与“执行任务预设动作”恰好重合时，仅凭用于拟合的标签，探针在数学上根本无法区分这两个完全不同的目标。 > This paper investigates the limitations of "truth probes" in machine learning models, introducing the concept of **perfect aliasing**—a failure of semantic identification that occurs when truthful reporting and a task's prescribed action coincide, making it impossible for a probe to distinguish the two targets from its fitting labels alone. 本研究的核心发现与技术要点包括： * **数学恒等式：** 在受控的二元汇报博弈中，在顺从上下文 (Compliant Contexts) 下拟合的真实性探针与预设动作探针，求解的是同一个优化问题。在对抗上下文 (Rival Contexts) 下，它们的标签表现为严格互补，从而迫使两者的受试者工作特征曲线下面积 (AUROC) 得分之和恒等于 1 （这一恒等关系在跨越 751 个“单元-网络层”对的严密实测中达到了浮点数级别的极致精度）。 * **解耦与线性可恢复性：** 通过使用随机化码本将预设输出符号与底层语义动作解耦，并借助混合了顺从与对抗上下文的数据将真实性与预设动作彻底剥离，作者在经过奖励训练的 Gemma-2-9B 策略模型上进行了系统评估（该策略在所有被测试的对抗实验中均选择输出假话）。实验显示，传统探针的 AUROC 得分仅为 $0.006 \pm 0.005$ ，而在完全相同的测试保留激活值上，混合拟合探针的 AUROC 得分达到了完美的 $1.000$ 。 * **测量的内在局限：** 这些研究结论严格聚焦于探针“究竟测量了什么”，**并不等同于**证明了模型在功能机制上保留了主观信念，也不代表模型在因果链路上实际利用了所恢复的方向，更不能直接视作一套可工业化部署的测谎工具。 > Key takeaways and findings include: > * **Mathematical Identity:** In a controlled binary reporting game, truth and prescribed-action probes fitted on compliant contexts solve the same optimization problem. On rival contexts, their labels act as exact complements, forcing their AUROCs to sum to one (an identity holding across 751 cell-layer pairs to floating-point precision). > * **Decoupling and Linear Recoverability:** By separating prescribed output symbols from semantic action using randomized codebooks, and isolating truth from prescribed action via mixed compliant and rival contexts, the author tests a reward-trained Gemma-2-9B policy that answers falsely on all evaluated rival trials. While a conventional probe scores an AUROC of $0.006 \pm 0.005$, mixed-fit probes score $1.000$ on the exact same held-out activations. > * **Limitations of Measurement:** The findings strictly concern what a probe measures and **do not** establish preserved functional belief, causal use of the recovered direction, or a deployable deception detector. --- ## 🔗 快速链接与资源 > ## 🔗 Quick Links & Resources * **全文访问：** [查看 PDF](https://arxiv.org/pdf/2609.10739) | [HTML（实验性）](https://arxiv.org/html/2609.10739v1) | [TeX 源码](https://arxiv.org/src/2609.10739) * **代码与数据：** [GitHub 仓库](https://github.com/dylanjayabahu/perfect-aliasing) * **开源许可：** [知识共享署名 4.0 国际许可 (Creative Commons Attribution 4.0)](http://creativecommons.org/licenses/by/4.0/) ![license icon](./images/345c7ad61f1b.png) > * **Full-Text Access:** [View PDF](https://arxiv.org/pdf/2609.10739) | [HTML (Experimental)](https://arxiv.org/html/2609.10739v1) | [TeX Source](https://arxiv.org/src/2609.10739) > * **Code & Data:** [GitHub Repository](https://github.com/dylanjayabahu/perfect-aliasing) > * **License:** [Creative Commons Attribution 4.0](http://creativecommons.org/licenses/by/4.0/) ![license icon](./images/345c7ad61f1b.png) --- ## 📄 论文摘要 > ## 📄 Abstract 如果在一个“真实陈述”与“任务预设动作”完全重合的环境中拟合真实性探针，那么仅凭训练标签，探针根本无法区分这两个目标。我们将这种语义识别的失效称为完全混叠 (Perfect Aliasing)。在受控的二元汇报博弈中，针对顺从上下文拟合的真实性探针与预设动作探针解决的是完全相同的优化问题。然而在对抗上下文中，二者的标签互为补集，迫使两者的 AUROC 得分之和严格等于 1；这一恒等式在 751 个单元-网络层对上均以浮点数精度成立。我们通过随机化码本将预设输出符号与语义动作分离开，再通过混合顺从与对抗上下文拟合将真实性与预设动作剥离。针对一个在所有评估对抗实验中均给出虚假回答的奖励训练 Gemma-2-9B 策略模型，传统探针在三个训练随机种子下的平均 AUROC 仅为 $0.006 \pm 0.005$ ，而混合拟合探针在完全相同的测试保留激活值上取得了 $1.000$ 的 AUROC。由于混合拟合使用了更多训练样本且接触了带标签的对抗上下文，这一对比确立了表征的线性可恢复性，而非单纯归功于去相关的收益。我们还展示了两个在分布内表现均完美的顺从拟合探针，在相同的对抗激活值上得分分别为 $0.080$ 和 $0.986$ 。这些发现关乎探针实际测量到的对象：它们并不证明模型保留了功能性信念，不证明模型在因果链条上利用了所恢复的方向，也不代表构建出了可部署的欺骗检测器。论文附带了相关代码与汇总结果。 > > A truth probe fitted where truthful reporting and a task's prescribed action coincide cannot distinguish those targets from its fitting labels alone. We call this failure of semantic identification perfect aliasing. In a controlled binary reporting game, truth and prescribed-action probes fitted on compliant contexts solve the same optimization. On rival contexts their labels are complements, forcing their AUROCs to sum to one; this identity holds across 751 cell-layer pairs to floating-point precision. We separate prescribed output symbols from semantic action using randomized codebooks, then separate truth from prescribed action by fitting on mixed compliant and rival contexts. For a reward-trained Gemma-2-9B policy that answers falsely on all evaluated rival trials, the conventional probe scores $0.006 \pm 0.005$ AUROC across three training seeds, while mixed-fit probes score $1.000$ on the same held-out activations. Mixed fitting uses more training examples and access to labelled rival contexts, so this comparison establishes linear recoverability rather than isolating the benefit of decorrelation. We also show that two compliant-fit probes, both perfect in-distribution, score $0.080$ and $0.986$ on the same rival activations. The findings concern what a probe measures: they do not establish preserved functional belief, causal use of the recovered direction, or a deployable deception detector. Code and aggregate results accompany the paper. --- ## 📊 补充元数据 > ## 📊 Additional Metadata * **篇幅规格：** 正文共 36 页，包含 15 张图表。 * **论文引用：** > Jayabahu, D. (2026). The Truth Was Never Gone: Perfect Aliasing in Compliant-Context Truth Probes. arXiv preprint arXiv:2609.10739. > * **Length:** 36 pages, 15 figures. > * **Citation:** > > Jayabahu, D. (2026). The Truth Was Never Gone: Perfect Aliasing in Compliant-Context Truth Probes. arXiv preprint arXiv:2609.10739.

无盒漏洞分析：仅凭元数据检测 MCP 服务器的间接提示词注入漏洞

Sun, 13 Sep 2026 00:00:00 GMT

在传统网络安全体系中，分析软件漏洞通常离不开审查源代码（白盒测试）或进行实际交互探测（黑盒测试）。然而，面对闭源、远程部署或具有商业访问限制的现代 AI 智能体系统，安全人员往往既拿不到底层源码，也无法直接发起动态交互。为此，本研究提出了全新的“无盒漏洞分析 (No-box Vulnerability Analysis)”安全范式，仅凭工具注册时声明的功能元数据（输入、输出与副作用说明），即可推演潜在的安全缺陷。团队针对模型上下文协议 (Model Context Protocol, MCP) 构建了原型检测工具 `MCPSEC`，在无需运行目标工具的前提下成功识别了 98.9% 的真实间接提示词注入漏洞，为 AI 智能体生态的轻量化安全审计开辟了全新路径。 --- # 无盒漏洞分析：仅凭元数据检测 MCP 服务器的间接提示词注入漏洞 > # No-Box Vulnerability Analysis: Description-only Detection of Indirect Prompt Injection Vulnerabilities in MCP Servers > **arXiv:** [2609.10854 [cs.CR]](https://arxiv.org/abs/2609.10854) > **Subjects:** Cryptography and Security (`cs.CR`); Artificial Intelligence (`cs.AI`) > **Submitted:** September 9, 2026 > **Authors:** Zehua Zhang, Jie Hu, Pratham Hegde, Aditya Maheshbhai Gabani, Souradip Nath, Yibo Liu, Siyu Liu, Hongkai Chen, Hulin Wang, Zhuoer Lyu, Chang Zhu, Divij Handa, Yan Shoshitaishvili, Tiffany Bao, Ruoyu Wang, Adam Doupe --- ## 📋 核心概述 > ## 📋 Summary 传统的漏洞分析通常需要直接访问系统底层或在运行时进行动态交互。然而，当第三方安全分析人员需要对闭源、远程托管、商业付费壁垒限制或关键就地运行系统进行审计时，这些前置条件往往难以得到满足。 > Traditional vulnerability analysis generally requires direct system access or active runtime interaction. However, these prerequisites are often unavailable when third-party analysts must audit closed-source, remotely hosted, commercially gated, or critical *in situ* systems. 为了弥补这一关键技术空白，本文提出了一种名为“无盒漏洞分析 (No-box Vulnerability Analysis)”的全新安全评估范式。该范式完全不依赖系统代码访问权限或运行时交互，仅凭软件的功能元数据（输入参数、返回值以及副作用说明）来评估潜在风险。 > To bridge this gap, this paper introduces **"no-box vulnerability analysis"**—a novel security paradigm that evaluates software using only its functionality metadata (inputs, outputs, and side effects) without any access or runtime interaction. 为了验证该设想的可行性，研究团队推出了名为 `MCPSEC` 的原型系统。它专门用于审计模型上下文协议 (Model Context Protocol, MCP) 服务器中的间接提示词注入 (Indirect Prompt Injection) 漏洞，且全过程仅仅依赖工具在注册阶段公开的元数据。在针对 20 个广泛部署的 MCP 服务器（共包含 177 个工具）的测试中，`MCPSEC` 准确预测了 94 个经过人工核验的真实漏洞，召回率高达 98.9% ，远超标准大语言模型 (Large Language Model, LLM) 基线（其召回率为 84.2% ）。 > To demonstrate its feasibility, the authors present **`MCPSEC`**, a prototype designed to audit Model Context Protocol (MCP) servers for indirect prompt injection vulnerabilities solely using tool metadata provided at registration time. Evaluated across 20 widely deployed MCP servers (comprising 177 tools), `MCPSEC` accurately predicted 94 real verified vulnerabilities (achieving **98.9% recall**), outperforming a standard LLM baseline which achieved 84.2% recall. --- ## 📑 论文摘要 > ## 📑 Abstract 传统的漏洞分析主要依赖于系统底层权限或动态交互测试，但第三方安全分析人员在审计闭源、远程托管、在线运行的关键系统或商业受限软件时，这些条件通常都无法具备。因此，我们提出了一种全新的“无盒漏洞分析”范式。在该范式中，分析人员既没有系统访问权限，也无法进行运行时交互，仅能依靠功能元数据展开推演。这类元数据定义了系统的预期行为（包括输入、输出与副作用），同时也约束了与该行为相符的可能实现空间。我们主张在不接触、不运行目标系统的前提下，对基于给定元数据所派生的所有潜在实现中的漏洞进行科学假设；未来当具备更多访问权限时，分析人员便可快速验证这些假设。我们通过构建名为 MCPSEC 的原型工具验证了无盒漏洞分析的可行性。该工具仅使用服务器注册时公开的工具元数据，就能审计模型上下文协议 (MCP) 服务器中存在的间接提示词注入漏洞。我们在 20 个实际部署的 MCP 服务器（涵盖 177 个工具）上测试了 MCPSEC，其中人工核验确认存在漏洞的工具有 95 个。MCPSEC 将 143 个工具判定为存在安全隐患，并为每个有漏洞的工具生成了假设的攻击面与漏洞利用方法。仅凭元数据，MCPSEC 就成功预测出 94 个经过验证的真实漏洞（召回率达 98.9% ），而对比的大语言模型基准仅预测出 80 个（召回率 84.2% ）。总体而言，研究结果确立了无盒漏洞分析作为全新安全范式的有效性，并展现了其在真实工业系统中的实用价值。 > Conventional vulnerability analysis relies on either system access or dynamic interaction, all of which may be unavailable to third-party analysts auditing closed-source, remotely hosted, critical in situ systems, or commercially gated software. Therefore, we propose a new paradigm of no-box vulnerability analysis in which neither access nor runtime interaction is available, and only functionality metadata is available. Such metadata defines the intended behavior of the system, including its inputs, outputs, and side effects, while constraining the space of implementations consistent with that behavior. We propose hypothesizing about vulnerabilities that exist across all possible implementations of a given system metadata, without observing or interacting with the target system. An analyst can later validate these hypotheses when additional access is available. We showcase the feasibility of no-box vulnerability analysis through implementing a prototype called MCPSEC, which audits Model Context Protocol (MCP) servers for indirect prompt injection vulnerabilities using only the tool metadata exposed at server registration time. We evaluate MCPSEC on 20 widely deployed MCP servers comprising 177 tools, among which human evaluators confirm 95 vulnerable tools. MCPSEC identified 143 tools as vulnerable, and for each vulnerable tool, it produced a hypothesized vulnerability along with exploitation technique. Using metadata alone, MCPSEC predicted 94 (98.9% recall) real verified vulnerabilities, compared against an LLM baseline with 80 (84.2% recall). Overall, our results introduce no-box vulnerability analysis as a new analysis paradigm and demonstrate its practical feasibility in realistic systems. --- ## 🔗 快速链接与相关资源 > ## 🔗 Quick Links * **全文获取：** [查看 PDF](https://arxiv.org/pdf/2609.10854) | [实验性 HTML 页面](https://arxiv.org/html/2609.10854v1) | [TeX 源码](https://arxiv.org/src/2609.10854) * **DOI 标识：** [10.48550/arXiv.2609.10854](https://doi.org/10.48550/arXiv.2609.10854) * **外部学术索引工具：** * [Google 学术搜索 (Google Scholar)](https://scholar.google.com/scholar_lookup?arxiv_id=2609.10854) * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.10854) * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.10854) > * **Full-Text Access:** [View PDF](https://arxiv.org/pdf/2609.10854) | [HTML (Experimental)](https://arxiv.org/html/2609.10854v1) | [TeX Source](https://arxiv.org/src/2609.10854) > * **DOI:** [10.48550/arXiv.2609.10854](https://doi.org/10.48550/arXiv.2609.10854) > * **External Bibliographic Tools:** > * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.10854) > * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.10854) > * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.10854)

斩获 IMO 金牌的开源秘籍：面向奥林匹克数学的 Nemotron 训练实践

Sun, 13 Sep 2026 00:00:00 GMT

国际数学奥林匹克竞赛 (IMO) 一直被视为检验人工智能最高阶复杂逻辑推理能力的试金石，而以往取得顶尖成绩的方案往往依赖闭源专有模型或形式化定理证明器。本文探索了如何通过模型后训练 (Post-Training) 与测试时计算 (Test-Time Compute) 设计，仅凭自然语言生成严密的奥数证明。以 **Nemotron 3 Ultra** 为基座，研究团队通过监督微调 (Supervised Fine-Tuning, SFT) 与强化学习 (Reinforcement Learning, RL) 训练了两个专家模型，并构建出一套无需任何形式化证明器、外部代码工具或互联网连接的全开源推理流水线。该系统在 **IMO 2026 中斩获 30 分（满分 42 分），成功达到金牌门槛**。为了推动开源社区发展，团队完整开源了两个专家权重、全部训练与推理代码、解题答卷以及包含 200 道原创奥数题的全新评测基准 Nemotron-IMO-Bench。 --- ## 核心概要 > ## Summary 本文深入探讨了模型后训练 (Post-Training) 与测试时推理架构设计如何显著增强复杂奥林匹克数学竞赛中的自然语言证明生成能力。研究人员以 **Nemotron 3 Ultra** 为起点，利用监督微调 (Supervised Fine-Tuning, SFT) 和强化学习 (Reinforcement Learning, RL) 训练了两个专家模型检查点。 > This paper investigates how model post-training and test-time inference design enhance natural-language proof generation for complex Olympiad-level mathematics. Starting with **Nemotron 3 Ultra**, the researchers trained two specialist checkpoints utilizing supervised fine-tuning and reinforcement learning. 他们提出了一套完全基于开源模型的测试时计算流水线，该流程全程以自然语言运作，完全无需形式化证明器、外部代码工具或互联网访问。利用三个 Nemotron 3 Ultra 检查点（通用模型与两个后训练专家模型），该系统通过迭代搜索来生成、验证并提炼候选证明，随后进入独立的高算力筛选阶段敲定最终解答。 > They present a fully open-model test-time-compute pipeline operating entirely in natural language—requiring no formal provers, external tools, or internet access. Using three Nemotron 3 Ultra checkpoints (the general model and two post-trained specialists), the system executes an iterative search to generate, verify, and refine candidate proofs, followed by a separate high-compute selection stage. **核心里程碑：** 该系统在 **IMO 2026 中斩获 42 分中的 30 分**，成功突破了**金牌分数线**。 > **Key Achievement:** The system achieved a score of **30 out of 42 points at IMO 2026**, successfully hitting the **gold-medal threshold**. 作者团队已将以下成果全部公开发布： * 两个后训练专家检查点权重 * 训练数据集与训练代码 * 推理部署代码 * 参赛提交的完整解题答案 * **Nemotron-IMO-Bench**，一个包含 200 道全新奥数级题目的评测基准 > The authors have publicly released: > * The two post-trained checkpoints > * Training data and code > * Inference code > * Submitted solutions > * **Nemotron-IMO-Bench**, a novel benchmark consisting of 200 olympiad-level problems. --- ## 论文摘要 > ## Abstract > 💬 [原文引用 / Original Quote]: > We study how model post-training and test-time inference design affect natural-language proof generation for hard olympiad mathematics. Starting from Nemotron 3 Ultra, we train two specialist checkpoints using supervised fine-tuning and reinforcement learning, and evaluate checkpoint choice, verification, and refinement. Based on these findings, we present an open-model test-time-compute pipeline. The system operates entirely in natural language, with no formal prover, external tools, or internet access. Three Nemotron 3 Ultra checkpoints - the general-availability model and two post-trained specialists - power an iterative search that generates, verifies, and refines candidate proofs; a separate high-compute stage then selects each final submission. The system scored 30 out of 42 points at IMO 2026, reaching the gold-medal threshold. We release the two post-trained checkpoints as well as the training data, the training and inference code, the submitted solutions, and Nemotron-IMO-Bench, a new benchmark of 200 novel olympiad-level problems. 我们研究了模型后训练与测试时推理设计如何影响高难度奥林匹克数学中的自然语言证明生成。从 Nemotron 3 Ultra 出发，我们利用监督微调和强化学习训练了两个专用检查点，并系统评估了模型检查点选取、核验机制以及逐步提炼策略。基于这些发现，我们提出了一套开源模型的测试时计算流水线。该系统完全在自然语言环境中运作，既不需要形式化定理证明器，也不需要外部工具或互联网访问。三个 Nemotron 3 Ultra 检查点——通用公开模型与两个后训练专家模型——共同驱动迭代搜索，负责生成、核查并完善候选证明；随后通过独立的高算力评估阶段选定最终提交的解答。该系统在 IMO 2026 中获得 30 分（满分 42 分），成功跻身金牌门槛。我们公开发布了这两个后训练检查点权重，以及训练数据、训练与推理代码、提交的官方解答，并发布了包含 200 道原创奥数题目的全新基准测试 Nemotron-IMO-Bench。 --- ## 相关资源与导航 > ## Associated Resources & Navigation * **代码、数据与媒体资源：** 可通过 Hugging Face、DagsHub 以及 GitHub 集成平台获取。 * **文献与引用工具：** 可通过 NASA ADS、Google 学术、Semantic Scholar 和 BibTeX 访问。 > * **Code, Data & Media:** Available via Hugging Face, DagsHub, and GitHub integrations. > * **Bibliographic Tools:** Accessible via NASA ADS, Google Scholar, Semantic Scholar, and BibTeX.

扎根智能体记忆：面向企业级智能体的环境探测式记忆管理

Sun, 13 Sep 2026 00:00:00 GMT

随着持久化记忆系统逐步进入面向生产环境的 AI 智能体平台，长周期智能体得以在跨会话交互中持续积累经验。然而，传统的任务后记忆整理智能体 (Curator Agent) 往往仅局限于已完成的历史轨迹进行复盘，容易固化此前执行中的错误、过度泛化片面证据、或长期保留陈旧过时的无效知识。为此，本文提出了 **环境探测式管理 (Environment-Probing Curation)**——这是一种轻量且高度兼容生产部署的扩展方案，通过为异步运行的记忆管理智能体赋予最小权限的只读外部环境工具，使其能够主动校验、界定范围并实时刷新候选记忆，而完全无需重新训练底层模型、修改任务智能体架构或变更生产写权限。在基于 GitHub Copilot 真实 SDK 构建的测试环境中，该方法将数据库探索任务的通过率从 39% 提升至 73%，并在大幅削减工具调用量与推理开销的同时保持了出色的模式稳定性，为构建高可靠的企业级智能体记忆系统提供了开创性范式。 --- ## 📋 核心概要 > ## 📋 Summary 持久化记忆系统使长周期 AI 智能体 (AI Agent) 能够在多个会话之间持续积累实操经验，但标准的任务后整理机制往往容易引入错误推论、过度泛化局部证据，或是保留陈旧过时的无效知识。本文提出了**环境探测式管理 (Environment-Probing Curation)**，这是一种轻量且完美适配现有部署的扩展方案：它为异步运行的记忆整理智能体赋予遵循最小权限原则的只读外部工具，使其能够主动检验、界定范围并动态刷新候选记忆，而无需重新训练模型、无需改动任务智能体，也无需变更生产环境的写入权限。 > Persistent memory systems enable long-horizon AI agents to accumulate experience across sessions, but standard post-task curation often introduces errors, overgeneralizes evidence, or retains stale knowledge. This paper introduces **environment-probing curation**, a lightweight, deployment-compatible extension that empowers asynchronous curator agents with least-privilege, read-only world tools. This allows the system to actively check, scope, and refresh memories without requiring model retraining, task-agent modifications, or changes to production write authorities. 在基于 GitHub Copilot (GHCP) SDK 构建的类生产测试环境（包含 CLBench 数据库探索与 90 项改编自 APEX 的管理咨询任务）中的评估表明，环境探测机制显著提升了任务通过率，大幅降低了查询开销与计算成本，并在杜绝架构漂移 (Schema Drift) 的前提下显著增加了任务智能体的奖励收益。 > Evaluated on a production-like GitHub Copilot (GHCP) harness (using CLBench and 90 adapted APEX management-consulting tasks), environment probing significantly boosts pass rates, reduces query overhead and costs, and improves task-agent reward gains without schema drift. --- ## 📌 文档元数据 > ## 📌 Document Metadata | 字段 | 详细信息 | | :--- | :--- | | **arXiv 编号** | [arXiv:2609.11060](https://arxiv.org/abs/2609.11060) [cs.AI] | | **主要学科领域** | 人工智能 (`cs.AI`) | | **次要学科领域** | 软件工程 (`cs.SE`) | | **提交日期** | 2026 年 9 月 10 日 | | **作者** | Susheel Suresh, Hazel Mak, Sahil Bhatnagar, Chhaya Methani, Alejandro Gutierrez Munoz | | **论文链接** | [查看 PDF](https://arxiv.org/pdf/2609.11060) \| [网页版本](https://arxiv.org/html/2609.11060v1) \| [DOI](https://doi.org/10.48550/arXiv.2609.11060) | > | Field | Details | > | :--- | :--- | > | **arXiv Identifier** | [arXiv:2609.11060](https://arxiv.org/abs/2609.11060) [cs.AI] | > | **Primary Subject** | Artificial Intelligence (`cs.AI`) | > | **Secondary Subjects** | Software Engineering (`cs.SE`) | > | **Submission Date** | September 10, 2026 | > | **Authors** | Susheel Suresh, Hazel Mak, Sahil Bhatnagar, Chhaya Methani, Alejandro Gutierrez Munoz | > | **Links** | [View PDF](https://arxiv.org/pdf/2609.11060) \| [HTML Version](https://arxiv.org/html/2609.11060v1) \| [DOI](https://doi.org/10.48550/arXiv.2609.11060) | --- ## 📄 论文摘要 > ## 📄 Abstract > 💬 [原文引用 / Original Quote]: > Persistent memory is entering production-oriented agent platforms to help long-horizon agents accumulate experience across sessions. Yet a post-task curator agent restricted to completed trajectories can preserve errors, overgeneralize partial evidence, or retain stale knowledge. We introduce environment-probing curation, a deployment-compatible extension that gives an existing asynchronous curator agent least-privilege, read-only world tools to check, scope, and refresh candidate memories. It requires no model retraining and leaves the task agent, retriever, memory representation, and production write authority unchanged. > > In a production-like GitHub Copilot (GHCP) harness built on its SDK, we compare stateless execution, full in-context learning, GHCP + Mem, and GHCP + Mem (w/ Env Probing) on CLBench database exploration and 90 adapted APEX management-consulting tasks. > > * **On CLBench:** Probing raises pass rates from $39\%$ to $73\%$ and pass-discounted rewards from $8.60$ to $22.60$, while reducing queries from $8.8$ to $4.7$ per question and task-agent costs from $\$3.38$ to $\$1.68$. > * **Across APEX Worlds:** All 18 memory-versus-baseline mean reward comparisons are positive, task-agent tool calls fall by $16\text{--}75\%$, and probing delivers the best task-agent reward gain per dollar in five out of six worlds. > * **Model Stability:** Probing attains higher mean reward than standard GHCP + Mem on both Sonnet 4.6 and Opus 4.7 without schema drift. > > Environment probing successfully transforms existing agent-memory curation into an environment-informed, auditable process while preserving a compact task-time interface. 持久化记忆正逐步迈入面向生产的智能体平台，以帮助长周期智能体在跨会话交互中沉淀宝贵经验。然而，如果仅将任务后整理智能体限制在已完成的历史轨迹中，就极易固化推理错误、过度泛化局部证据，甚至保留陈旧过时的信息。我们提出了环境探测式管理：一种面向实际部署的扩展方案，它赋予现有异步整理智能体最小权限的只读真实世界工具，以主动核验、限定作用域并刷新候选记忆。该方案完全不需要重新训练模型，且保持了任务智能体、检索器、记忆表征格式以及生产写入权限的原封不动。在基于其实际 SDK 构建的类生产 GitHub Copilot (GHCP) 运行环境中，我们在 CLBench 数据库探索和 90 项改编的 APEX 管理咨询任务上，对比了无状态执行、完整上下文学习 (In-Context Learning)、GHCP + Mem 以及 GHCP + Mem（带环境探测）四种架构： * **在 CLBench 上：** 环境探测将通过率从 $39\%$ 大幅提升至 $73\%$，折扣奖励从 $8.60$ 跃升至 $22.60$，同时将每道题的查询量从 $8.8$ 次缩减至 $4.7$ 次，任务智能体开销从 $\$3.38$ 降至 $\$1.68$。 * **在各 APEX 环境中：** 所有 18 组记忆对比基线的平均奖励差异均为正值，任务智能体的工具调用次数减少了 $16\text{--}75\%$，且在六个测试世界中的五个世界里实现了最高的“单位美元任务奖励收益”。 * **模型稳定性表现：** 无论在 Sonnet 4.6 还是 Opus 4.7 模型上，带探测的环境管理均获得了高于标准 GHCP + Mem 的平均奖励，且没有发生任何架构漂移。环境探测成功将现有的智能体记忆整理升级为兼具环境感知与可审计性的稳健流程，同时在任务运行时保持了紧凑高效的接口契约。 --- ## 🔗 相关资源与工具 > ## 🔗 Additional Resources & Tools * **代码、数据与媒体：** 可通过 [Hugging Face](https://huggingface.co/huggingface)、[CatalyzeX 代码检索](https://www.catalyzex.com) 以及 [DagsHub](https://dagshub.com/) 查看获取情况。 * **交互式与文献检索工具：** 可通过 [Connected Papers](https://www.connectedpapers.com/)、[Litmaps](https://www.litmaps.co/)、[scite 智能引用](https://www.scite.ai/) 和 [alphaXiv](https://alphaxiv.org/) 进行深入探索。 > * **Code, Data & Media:** Check availability via [Hugging Face](https://huggingface.co/huggingface), [CatalyzeX Code Finder](https://www.catalyzex.com), and [DagsHub](https://dagshub.com/). > * **Interactive & Bibliographic Tools:** Explore via [Connected Papers](https://www.connectedpapers.com/), [Litmaps](https://www.litmaps.co/), [scite Smart Citations](https://www.scite.ai/), and [alphaXiv](https://alphaxiv.org/).

当合成数据带来负面效应：大模型智能体技能检索中的灾难性遗忘研究

Sun, 13 Sep 2026 00:00:00 GMT

随着大语言模型 (Large Language Model, LLM) 智能体逐步迈向复杂真实任务，智能体需要在运行时从庞大的工具库中精准检索出对应技能。在缺乏充足人工标注的背景下，工业界通常采用大模型生成的“合成数据”对检索器与重排器进行微调，但往往忽视了合成数据的分布局限性。本文依托一套可调度 34,396 个技能的工业级技能路由器，首次系统揭示了在合成数据上微调会诱发严重的“灾难性遗忘 (Catastrophic Forgetting)”，导致模型在真实业务数据和分布外 (OOD) 场景下的表现骤降。为此，研究团队引入了嵌入锚点正则化、无遗忘学习 (LwF) 与弹性权重整合 (EWC) 等持续学习机制，不仅有效稳固了真实场景的泛化表现，还将轻量级 Qwen 模型的分布内检索精度提升了 13.98% ，为构建稳定可靠的智能体生态提供了关键参考。 --- # 当合成数据带来负面效应：大模型智能体技能检索中的灾难性遗忘研究 > # When Synthetic Data Hurts: On Catastrophic Forgetting in Skill Retrieval for LLM Agents ## 核心概述 > ## Summary 随着大语言模型 (Large Language Model, LLM) 智能体越来越依赖在运行时动态检索外部技能，如何从海量技能库中挑选出最为契合的技能，已成为当前智能化落地的一大核心挑战。本文提出了一个能够支持高达 34,396 个技能的生产级技能路由器，并开展了一项大规模实证研究，深入评估了在有限真实监督数据与合成数据驱动下的技能检索表现。 > As Large Language Model (LLM) agents increasingly rely on external skills retrieved at runtime, selecting the right skills from massive repositories has become a critical challenge. This paper presents a production-grade skill router capable of handling 34,396 skills, alongside a large-scale study evaluating skill retrieval using limited real supervision and synthetic data. 作者团队在实验中发现：虽然在合成数据上微调可以有效提升同分布测试集上的检索表现，但这往往会诱发严重的**灾难性遗忘 (Catastrophic Forgetting)**，大幅削弱模型在真实世界及分布外 (Out-of-Distribution, OOD) 数据上的检索能力。为了有效化解这一矛盾，本研究评测了多种源自持续学习 (Continual Learning) 的缓解策略，包括嵌入锚点正则化 (Embedding-Anchor Regularization)、无遗忘学习 (Learning without Forgetting, LwF)、弹性权重整合 (Elastic Weight Consolidation, EWC) 以及 $L_2$ 初始化方法。实验结果表明，这些方法不仅成功稳固了分布外技能检索的性能底线，还将基于 0.6B Qwen 的检索器与重排器在合成数据分布内的技能检索准确率额外提升了 **13.98%** 。 > The authors discover that while fine-tuning on synthetic data improves in-distribution retrieval, it simultaneously triggers **catastrophic forgetting** on real and out-of-distribution (OOD) data. To counter this, the study evaluates several continual-learning-inspired mitigation strategies, including embedding-anchor regularization, Learning without Forgetting (LwF), Elastic Weight Consolidation (EWC), and $L_2$-initialization. Results demonstrate that these approaches not only preserve OOD skill retrieval performance but also boost synthetic in-distribution skill retrieval by **13.98%** for a 0.6B Qwen retriever and reranker. --- ## 论文元数据 > ## Document Metadata | 元数据字段 | 详细信息 | | :--- | :--- | | **arXiv 编号** | [`arXiv:2609.10750`](https://arxiv.org/abs/2609.10750) [cs.IR] | | **作者** | Syed Shariyar Murtaza, Yifan Nie, Utkarsh Soni, Eugene Wen, Arvid Frydenlund | | **主分类** | 信息检索 (`cs.IR`) | | **次分类** | 人工智能 (`cs.AI`), 机器学习 (`cs.LG`) | | **ACM 分类号** | H.3.3 | | **提交日期** | 2026年9月9日 | | **备注信息** | 正文 8 页，全文共 15 页；已被 EMNLP 2026 产业界赛道 (Industry Track) 录用 | | **DOI** | [10.48550/arXiv.2609.10750](https://doi.org/10.48550/arXiv.2609.10750) | | **开源许可协议** | [知识共享署名 4.0 国际许可 (Creative Commons Attribution 4.0 International)](http://creativecommons.org/licenses/by/4.0/) | > | Metadata Field | Details | > | :--- | :--- | > | **arXiv ID** | [`arXiv:2609.10750`](https://arxiv.org/abs/2609.10750) [cs.IR] | > | **Authors** | Syed Shariyar Murtaza, Yifan Nie, Utkarsh Soni, Eugene Wen, Arvid Frydenlund | > | **Primary Subject** | Information Retrieval (`cs.IR`) | > | **Secondary Subjects** | Artificial Intelligence (`cs.AI`), Machine Learning (`cs.LG`) | > | **ACM Classification** | H.3.3 | > | **Submission Date** | September 9, 2026 | > | **Comments** | 8 main pages, 15 pages total; accepted in EMNLP Industry Track 2026 | > | **DOI** | [10.48550/arXiv.2609.10750](https://doi.org/10.48550/arXiv.2609.10750) | > | **License** | [Creative Commons Attribution 4.0 International](http://creativecommons.org/licenses/by/4.0/) | --- ## 全文获取与访问链接 > ## Full-Text & Access Links * **PDF 版本：** [查看 PDF](https://arxiv.org/pdf/2609.10750) * **HTML 版本：** [实验性在线 HTML](https://arxiv.org/html/2609.10750v1) * **TeX 源码：** [arXiv 原生源码](https://arxiv.org/src/2609.10750) * **音频解读：** [收听音频概要](https://arxiv.org/audio/2609.10750) > * **PDF Version:** [View PDF](https://arxiv.org/pdf/2609.10750) > * **HTML Version:** [HTML (Experimental)](https://arxiv.org/html/2609.10750v1) > * **TeX Source:** [arXiv e-Print Source](https://arxiv.org/src/2609.10750) > * **Audio Summary:** [Listen to Audio Summary](https://arxiv.org/audio/2609.10750) --- ## 外部资源与关联工具 > ## External Resources & References * **学术引用与指标：** [Google 学术搜索 (Google Scholar)](https://scholar.google.com/scholar_lookup?arxiv_id=2609.10750) | [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.10750) | [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.10750) * **代码与关联工具：** [Hugging Face](https://huggingface.co/) | [CatalyzeX 论文代码检索](https://www.catalyzex.com) | [alphaXiv 论文讨论区](https://alphaxiv.org/) > * **Citations & Metrics:** [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.10750) | [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.10750) | [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.10750) > * **Code & Associated Tools:** [Hugging Face](https://huggingface.co/) | [CatalyzeX Code Finder](https://www.catalyzex.com) | [alphaXiv Discussion](https://alphaxiv.org/) --- *(注：根据规范保留许可协议图标)*

引入非基项子句学习拓展 SMT 求解能力

Sun, 13 Sep 2026 00:00:00 GMT

在软硬件形式化验证与程序分析中，可满足性模理论 (Satisfiability Modulo Theories, SMT) 求解器是至关重要的底层推理引擎。面对含有量词的复杂一阶逻辑命题，传统 SMT 求解器通常依靠“量词实例化”将其降解为无变量的具体基项 (Ground Instances) 并结合 CDCL(T) 算法进行冲突分析，但这种方法学到的冲突子句往往过于狭隘具体，错失了利用高阶非基项结构将证明长度实现指数级精简的良机。为此，本研究（LPAR 2026 扩展论文）提出了一种创新的统一演算体系，将基项实例化、CDCL(T) 规则与高阶非基项冲突分析有机融合。求解器在具体基项上高效推理，而在原始非基项子句上执行归结消解，从而提炼出更通用、无冗余的泛化子句，并成功引入时间次回溯机制，从数学上证明了该框架能够统一模拟 CDCL、SCL 及一阶逻辑消解法。 --- # 引入非基项子句学习拓展 SMT 求解能力 > # Extending SMT Solving with Non-Ground Clause Learning ## 核心概述 > ## Summary 量词实例化是求解含有变量的非基项可满足性模理论 (Satisfiability Modulo Theories, SMT) 问题的传统主流手段。在这一框架下，求解器首先生成具体的基项实例 (Ground Instances)，随后应用带有理论扩展的冲突驱动子句学习 (CDCL(T)) 风格的算法进行推理。然而，标准的冲突分析过程仅能学习到具体的基项子句，无法捕捉深层的非基项变量结构，从而错失了在理论上将证明长度缩减数个数量级（指数级精简）的绝佳机会。 > Quantifier instantiation is the traditional approach to non-ground Satisfiability Modulo Theories (SMT) solving, where solvers generate ground instances and apply CDCL(T)-style reasoning. However, standard conflict analysis only learns ground clauses, failing to capture the underlying non-ground structure from which exponential proof-length savings could otherwise be derived. 本文（作者 Yasmine Briefs 和 Christoph Weidenbach 在 LPAR 2026 录用论文的扩展版本）提出了一种统一的逻辑演算体系，将基项实例化、CDCL(T) 推理规则与非基项冲突分析深度结合。通过在原始非基项子句上直接执行归结消解 (Resolution) 步骤，求解器能够提炼出通用性更强、且不含冗余的学习子句，同时还成功融入了时间次回溯 (Chronological Backtracking) 机制。作者从形式化逻辑上严密证明了该框架能够完整模拟 CDCL、SCL(FOL)、SCL(T) 以及经典的一阶消解法。 > This paper—an extended version of the LPAR 2026 paper by Yasmine Briefs and Christoph Weidenbach—proposes a unified calculus combining ground instantiations, CDCL(T)-style rules, and non-ground conflict analysis. By performing resolution steps on original non-ground clauses, the solver yields significantly more general, non-redundant learned clauses, while also successfully integrating chronological backtracking. The authors formally prove that this framework simulates CDCL, SCL(FOL), SCL(T), and standard Resolution. --- ## 元数据与文档信息 > ## Metadata & Document Information | 字段 | 详情 | | :--- | :--- | | **arXiv 编号** | [arXiv:2609.11509](https://arxiv.org/abs/2609.11509) [cs.AI] | | **主要学科领域** | 人工智能 (`cs.AI`) | | **次要学科领域** | 计算机科学中的逻辑 (`cs.LO`) | | **作者** | Yasmine Briefs, Christoph Weidenbach | | **提交日期** | 2026年9月10日 | | **备注说明** | LPAR 2026 会议论文的扩展版 | | **DOI 链接** | [10.48550/arXiv.2609.11509](https://doi.org/10.48550/arXiv.2609.11509) | > | Field | Details | > | :--- | :--- | > | **arXiv Identifier** | [arXiv:2609.11509](https://arxiv.org/abs/2609.11509) [cs.AI] | > | **Primary Subject** | Artificial Intelligence (`cs.AI`) | > | **Secondary Subjects** | Logic in Computer Science (`cs.LO`) | > | **Authors** | Yasmine Briefs, Christoph Weidenbach | > | **Submission Date** | September 10, 2026 | > | **Comments** | Extended version of LPAR 2026 paper | > | **DOI** | [10.48550/arXiv.2609.11509](https://doi.org/10.48550/arXiv.2609.11509) | --- ## 论文摘要 > ## Abstract 量词实例化目前是非基项 SMT 求解的主流方法：求解器生成基项实例，并利用 CDCL(T) 风格的推理来求解派生出的基项 SMT 问题。当遭遇逻辑冲突时，现有的冲突分析仅学习基项子句，尽管该冲突实质上源自非基项子句的实例化。然而，非基项推理相比于纯基项推理，能够产生呈指数级简短的证明过程。我们提出了一种包含基项实例化、CDCL(T) 风格规则和非基项冲突分析的新型演算。求解器在基项实例上进行推理，但冲突分析的归结步骤直接作用于其原始的非基项子句。这使得学到的子句通常比基项冲突具备更高的通用性。在合理的策略下，学到的子句甚至完全不含冗余。我们还展示了如何将时间次回溯融入 SMT 求解过程中。我们的演算体系为 CDCL(T) 风格的 SMT 求解、一系列基于实例化的算法过程以及非基项子句学习提供了统一的理论框架，并证明了其能够模拟 CDCL、SCL(FOL)、SCL(T) 乃至经典消解法。 > Quantifier instantiation is currently the main approach to non-ground SMT solving: solvers generate ground instances and solve the resulting ground SMT problems with CDCL(T)-style reasoning. When a conflict is found, conflict analysis learns only a ground clause, even though the conflict comes from instances of non-ground clauses. Yet non-ground reasoning can give exponentially shorter proofs than purely ground reasoning. We propose a calculus that consists of ground instantiations, CDCL(T)-style rules, and non-ground conflict analysis. The solver reasons on ground instances, but the resolution steps of conflict analysis are performed on their original non-ground clauses. This produces learned clauses that are typically more general than the ground conflict. With a suitable strategy, the learned clauses are even non-redundant. We also show how chronological backtracking can be included in SMT solving. Our calculus gives a common setting for CDCL(T)-style SMT solving, a range of instantiation-based procedures, and non-ground clause learning, and we prove that it simulates CDCL, SCL(FOL), SCL(T), and even Resolution. --- ## 全文获取与外部资源 > ## Full-Text & External Resources * **PDF 版本：** [查看 PDF](https://arxiv.org/pdf/2609.11509) * **实验性 HTML 页面：** [arXiv 在线 HTML](https://arxiv.org/html/2609.11509v1) * **TeX 源码：** [下载源码](https://arxiv.org/src/2609.11509) * **分发许可协议：** [非独占分发许可协议](http://arxiv.org/licenses/nonexclusive-distrib/1.0/) > * **PDF Version:** [View PDF](https://arxiv.org/pdf/2609.11509) > * **Experimental HTML:** [arXiv HTML Version](https://arxiv.org/html/2609.11509v1) > * **TeX Source:** [Download Source](https://arxiv.org/src/2609.11509) > * **License:** [Non-exclusive distribution license](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

就绪与下发解耦：面向智能体大模型工作流的长尾感知调度

Sun, 13 Sep 2026 00:00:00 GMT

智能体大语言模型 (Agentic LLM) 工作流由多轮模型生成与频繁的外部工具交互交织而成，其端到端整体执行延迟不仅取决于单次模型推理的速度，更直接受制于各个已就绪轮次 (Turns) 何时被正式下发至执行引擎。当前主流运行时普遍采用“急切下发策略 (Eager Release Policy)”，即一旦轮次就绪便立即派发；但在系统高负载并发拥堵时，这种策略会导致大量已下发却未完成的工作在流水线中淤积，剥夺了工作流调度器动态重排序的控制权，进而引发极严重的长尾延迟恶化。为此，研究团队提出了 **长尾风险感知轮次下发调度方法**，创新性地将“轮次就绪”与“实际下发”彻底解耦，动态裁决何时派发以及控制管道内的未完成工作预算。该方法引入均值-条件风险价值 (CVaR) 优化目标，在高负载资源争用场景下将工作流的 P95 流转延迟大幅降低，带来最高达 3.50 倍的性能加速，为大规模多智能体系统的工业级调度提供了全新思路。 --- ## 📌 执行概要 > ## 📌 Executive Summary 智能体大语言模型 (Agentic Large Language Model, LLM) 工作流高度依赖于交替迭代的模型思考轮次与外部工具交互，这意味着整体任务的完成时间不仅受到纯模型推理速度的制约，更取决于已就绪的轮次在**何时**被正式下发到后端执行。 > Agentic Large Language Model (LLM) workflows rely heavily on iterative model turns interleaved with tool interactions, meaning that overall completion times are bottlenecked not just by inference speed, but by *when* ready turns are released for execution. 标准运行时系统通常采用**急切下发策略 (Eager Release Policy)**，即只要某一轮次就绪就立即派发执行。然而在严重的系统资源争用下，这种做法会在底层积压过多已下发但尚未完成的任务。由于这些轮次已经被提交进执行队列，工作流层级的调度器彻底失去了对它们重新排序调度的灵活性，导致长尾延迟 (Tail Latency) 严重恶化。 > Standard runtime environments utilize an **eager release policy**, dispatching every turn immediately upon readiness. However, under heavy system contention, this approach accumulates excessive uncompleted work. Because these turns are already submitted, workflow-level schedulers lose the ability to reorder them, causing severe tail latency degradation. 为了化解这一困境，作者团队提出了一种**长尾风险感知的轮次下发调度方法**。该方案通过动态裁决以下两个核心问题，将轮次的“就绪状态”与“下发动作”实现了解耦： 1. 下一个应当下发哪一个已就绪的轮次。 2. 流水线中应当维持多少处于已下发但未完成状态的工作量。 > To resolve this issue, the authors introduce a **tail-risk-aware turn release scheduling method**. This approach decouples turn readiness from release by dynamically deciding: > 1. Which ready turn should be released next. > 2. How much released yet unfinished work should be maintained in the pipeline.

少即是多：究竟是何种语音特征驱动了话轮结束检测？

Sun, 13 Sep 2026 00:00:00 GMT

在人机语音交互中，AI 能否准确判断说话人是否“已经说完”（即话轮结束检测，End-of-Turn Detection），是实现自然流畅对话的关键枢纽。过去行业普遍认为，结合文本语义与声学特征的多模态系统效果更佳，但这也带来了显著的计算延迟和语义依赖。本论文通过精细的模态消融实验，深入探究了声学、韵律与文本语义在流式话轮检测中的真实作用，惊人地发现：单纯结合“声学+韵律”特征不仅达到了 0.93 的 F1 分数与仅 400 毫秒的中位数延迟，而且加入文本语义反而会诱发更多抢话误判。这一发现颠覆了传统认知，证明人类日常对话的轮替更多是由语调起伏与停顿模式驱动，无需高昂的文本语义推理即可打造极速且稳健的对话系统。 --- ## 📌 内容摘要 > ## 📌 Summary 在对话式 AI (Conversational AI) 领域，准确检测用户何时结束发言，对于实现自然流畅的轮流交谈（话轮转换，Turn-Taking）至关重要。尽管近年来的系统频繁引入文本语义数据，但不同语音模态各自究竟贡献了多少价值，此前学术界与工业界一直未能厘清。 > In conversational AI, accurately detecting when a user has finished speaking is essential for natural, fluid turn-taking. While recent systems frequently incorporate semantic (text) data, the specific contributions of different speech modalities have remained unclear. 本论文基于一个轻量级三模态分类器，通过严谨的受控消融实验，深入探究了声学、韵律与语义信号在其中的具体作用。实验结果表明，**声学与韵律的特征组合**在准确率与延迟之间实现了最佳平衡——在 400 毫秒的中位数延迟下，取得了 0.93 的语句级 F1 分数，误报率仅为 7.8% 。出人意料的是，额外引入文本语义信号不仅未能改善整体性能，反而增加了过早切入（抢话）的误判几率。特征空间分析进一步证实，韵律特征具备极其出色的类别可分性，而文本表征则存在大面积重叠。这表明人类对话中的话轮转换主要由语调模式和停顿节奏驱动，而非句意在语义层面的完整性。 > This paper investigates the role of acoustic, prosodic, and semantic signals using a controlled ablation study with a lightweight trimodal classifier. The findings reveal that **acoustic-prosodic combinations** achieve the optimal balance of accuracy and latency—matching an utterance F1 score of 0.93 with a 7.8% false alarm rate at a median latency of 400ms. Surprisingly, adding text signals increases premature detections without improving overall performance. Feature space analysis demonstrates that prosodic features offer robust class separability, whereas text representations heavily overlap, suggesting that turn-taking is primarily driven by intonation and silence patterns rather than semantic completeness. --- ## 📋 论文摘要 > ## 📋 Abstract 在对话式 AI 中，准确判断说话者何时结束发言是实现自然话轮转换的关键所在。尽管近期的研究逐渐融合了语义信息，但各模态之间的相对贡献度依然不够明朗。我们在流式话轮结束检测任务中，使用轻量级三模态分类器，对声学、韵律与语义信号开展了严格受控的消融实验。在完全相同的训练条件下，“声学+韵律”的组合在准确率与低延迟之间达成了最优平衡：在仅 400 毫秒的中位数延迟下，取得了 0.93 的语句级 F1 值与 7.8% 的极低误报率。与之相反，引入文本语义非但没有提升整体表现，反而显著增加了过早触发的误报次数。特征空间分析印证了这一现象：韵律特征展现出了最强的类别区隔能力，而文本表征之间则存在大量重叠。这些研究结论表明，话轮转换的线索主要是通过语调起伏与停顿模式传递的，而非依赖语义层面的完整度；这也为构建摆脱昂贵文本推理开销、反应更灵敏且更可靠的实时语音交互系统指明了新路径。 > In conversational AI, detecting when a speaker has finished talking is crucial for natural turn taking. While recent work incorporates semantics, the relative contribution of different modalities remains unclear. We present a controlled ablation of acoustic, prosodic, and semantic signals for streaming end of turn detection using a lightweight trimodal classifier. Under identical training conditions, the acoustic prosodic combination achieves the best balance of accuracy and latency, achieving utterance F1 of 0.93 with 7.8% false alarms at 400ms median latency. Adding text increases premature detections without improving performance. Feature space analysis confirms that prosodic features have the strongest class separability, while text representations overlap substantially. These findings suggest that turn-taking is primarily conveyed through intonation and silence patterns rather than semantic completeness, enabling faster and more reliable systems without expensive text inference. --- ## 🔗 资源与相关链接 > ## 🔗 Links & Resources * **全文获取通道**： * [阅读 PDF 原文](https://arxiv.org/pdf/2609.11066) * [HTML 网页版 (实验性预览)](https://arxiv.org/html/2609.11066v1) * [TeX 源代码](https://arxiv.org/src/2609.11066) * **授权协议**：[Creative Commons Attribution 4.0](http://creativecommons.org/licenses/by/4.0/) view license * **文献引证与检索**： * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.11066) * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.11066) * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.11066) > * **Full-Text Access:** > * [View PDF](https://arxiv.org/pdf/2609.11066) > * [HTML Version (Experimental)](https://arxiv.org/html/2609.11066v1) > * [TeX Source](https://arxiv.org/src/2609.11066) > * **License:** [Creative Commons Attribution 4.0](http://creativecommons.org/licenses/by/4.0/) view license > * **External Citations:** > * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.11066) > * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.11066) > * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.11066)

大语言模型中的知识归因探测研究

Sun, 13 Sep 2026 00:00:00 GMT

大语言模型 (Large Language Model, LLM) 在实际落地中频繁出现“幻觉”现象。这些错误通常源于两大根源：其一是误读或篡改了给定的提示词上下文（违背忠实性），其二则是模型内部参数化记忆出现了事实偏差（违背真实性）。要彻底缓解幻觉，核心在于厘清模型生成特定答案时究竟主导调用了哪一部分知识——即实现精准的“贡献性知识归因” (Contributive Attribution)。本文指出，仅需在大模型中间隐藏层表征上训练轻量级的线性探测器 (Linear Probes) ，就能以极高置信度识别输出内容的知识来源。研究团队提出了 **AttriWiki** 自监督数据合成管线，摆脱了以往必须依赖人为制造知识冲突的缺陷，能够全自动生成高质量标签样本。基于 AttriWiki 训练的探测器在 Llama-3.1-8B、Mistral-7B 和 Qwen-7B 等模型上实现了高达 0.96 的 Macro-$F_1$ 得分，并能无缝零样本迁移至 SQuAD 与 WebQuestions 等经典问答基准，为大模型的可信部署与可解释性分析开辟了全新路径。 --- ## 概述 > ## Summary 大语言模型经常受到幻觉问题的困扰，这些幻觉通常源自两个截然不同的诱因：**忠实度违背 (Faithfulness Violations)**（即误用或曲解了提示词所提供的上下文）以及 **真实度违背 (Factuality Violations)**（即根植于模型自身内部参数化知识的事实性错误）。要有效化解这些问题，精准定位模型输出背后究竟主要由哪个知识源头主导——这一过程被称为*贡献性归因 (Contributive Attribution)*——显得至关重要。 > Large language models often suffer from hallucinations, which typically stem from two sources: **faithfulness violations** (misusing provided context) and **factuality violations** (errors originating from internal parametric knowledge). Mitigating these issues requires accurately identifying the dominant knowledge source behind an output—a process known as *contributive attribution*. 本研究表明，在模型隐藏层表征上训练的简单线性探测器 (Linear Probes) ，能够极其可靠地判断生成内容的知识归属源。作者团队推出了 **AttriWiki** 自监督管线，它无需人为构建知识冲突，便能自动化生成高质量的带标注训练数据。利用 AttriWiki 训练的探测器在 Llama-3.1-8B、Mistral-7B 和 Qwen-7B 等代表性模型上取得了高达 **0.96 的 Macro-$F_1$** 评分，并能无缝迁移至 SQuAD 与 WebQuestions 等基准测试（取得 0.94–0.99 的 Macro-$F_1$ 分数），同时展现出卓越的零样本泛化能力。此外，该研究重点指出，知识归因失配会急剧抬高模型的输出错误率，这更加印证了在大语言模型中建立先进知识归因分析框架的迫切性。 > This paper demonstrates that simple linear probes trained on hidden model representations can reliably identify the knowledge source of a generation. The authors introduce **AttriWiki**, a self-supervised pipeline that automatically generates high-quality labeled training data without relying on knowledge conflicts. Probes trained using AttriWiki achieve up to **0.96 Macro-$F_1$** across models like Llama-3.1-8B, Mistral-7B, and Qwen-7B, transfer seamlessly to benchmarks such as SQuAD and WebQuestions (0.94–0.99 Macro-$F_1$), and generalize zero-shot to prior benchmarks. Furthermore, the study highlights that attribution mismatches drastically elevate error rates, emphasizing the critical need for advanced knowledge attribution frameworks in LLMs. --- ## 元数据与参考信息 > ## Metadata & Reference Information * **arXiv 编号：** [arXiv:2602.22787](https://arxiv.org/abs/2602.22787) [cs.CL] * **学科领域：** 计算与语言 (`cs.CL`)；人工智能 (`cs.AI`) * **作者团队：** Ivo Brink, Alexander Boer, Dennis Ulmer * **提交时间轴：** * 首次提交：2026年2月26日 * 最新修订：2026年9月10日 (v3) * **许可协议：** [知识共享署名 4.0 国际许可 (CC BY 4.0)](http://creativecommons.org/licenses/by/4.0/) [![license icon](./images/345c7ad61f1b.png)](http://creativecommons.org/licenses/by/4.0/) > * **arXiv ID:** [arXiv:2602.22787](https://arxiv.org/abs/2602.22787) [cs.CL] > * **Subjects:** Computation and Language (`cs.CL`); Artificial Intelligence (`cs.AI`) > * **Authors:** Ivo Brink, Alexander Boer, Dennis Ulmer > * **Submission Timeline:** > * Submitted: 26 Feb 2026 > * Last Revised: 10 Sep 2026 (v3) > * **License:** [Creative Commons Attribution 4.0 International](http://creativecommons.org/licenses/by/4.0/) [![license icon](./images/345c7ad61f1b.png)](http://creativecommons.org/licenses/by/4.0/) --- ## 获取途径与资源 > ## Access & Resources * **全文格式：** * [阅读 PDF](https://arxiv.org/pdf/2602.22787) * [HTML 网页版 (实验性)](https://arxiv.org/html/2602.22787v3) * [TeX 源码](https://arxiv.org/src/2602.22787) * **文献检索与工具：** * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2602.22787) * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2602.22787) * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2602.22787) * [Hugging Face](https://huggingface.co/huggingface) > * **Full-Text Formats:** > * [View PDF](https://arxiv.org/pdf/2602.22787) > * [HTML (Experimental)](https://arxiv.org/html/2602.22787v3) > * [TeX Source](https://arxiv.org/src/2602.22787) > * **External Citations & Tools:** > * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2602.22787) > * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2602.22787) > * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2602.22787) > * [Hugging Face](https://huggingface.co/huggingface)