https://insight.aitobox.com/ AI Insight Pipeline & Tech Blog https://insight.aitobox.com/blog/posts/2026-10-05/Google-Research-将联邦学习引入机密计算-Gboard-现已支持外部可验证的差分隐私训练/ https://insight.aitobox.com/blog/posts/2026-10-05/Google-Research-将联邦学习引入机密计算-Gboard-现已支持外部可验证的差分隐私训练/ Mon, 05 Oct 2026 00:00:00 GMT 传统联邦学习 (Federated Learning, FL) 虽能在不收集用户原始数据的前提下协同训练模型，但用户和外界始终无法从密码学层面证实中央服务器是否私下记录了数据，也无法确证差分隐私 (Differential Privacy, DP) 噪声是否被严谨添加，存在着难以弥合的“信任鸿沟”。为了攻克这一关键技术挑战，Google Research 正式推出了基于可信执行环境 (Trusted Execution Environment, TEE) 构建的新一代机密联邦学习系统，将梯度计算从端侧转移至经过硬件远程证明的服务端机密飞地中执行。该系统结合了公共透明日志 (Public Transparency Logs) 与可重构构建技术，使得任何外部机构都能独立审计其服务端的完整执行逻辑，在业内首次实现了具有外部可验证性的中心差分隐私保障。目前该技术已全面投入生产环境，稳定支撑 Gboard 输入法的英语与日语下一词预测模型，不仅大幅提升了用户隐私防线，更将原本耗时数月的跨设备训练周期缩短了数倍。 --- # Google Research 将联邦学习引入机密计算：Gboard 现已支持外部可验证的差分隐私训练 > # Google Research Moves Federated Learning Into TEEs: Gboard Now Trains With Externally Verifiable Differential Privacy ## 内容总结 > ## Summary Google Research 正式公布了基于可信执行环境 (Trusted Execution Environment, TEE) 构建的下一代联邦学习 (Federated Learning, FL) 系统，在业界首次实现了外部可验证的中心差分隐私 (Differential Privacy, DP) 保障。通过将原本由手机端侧承担的梯度计算转移到经过硬件远程证明的服务端 TEE 中运行，并依托公共透明日志进行公开公示，该系统从根本上消除了传统联邦学习中长期存在的“信任鸿沟”。目前，这套全新架构已全面投入生产环境，成功支撑了 Gboard 输入法的英语与日语下一词预测模型，不仅大幅缩短了训练耗时，更为用户隐私筑起了一道坚实的密码学防线。 > Google Research has announced a next-generation Federated Learning (FL) system built on Trusted Execution Environments (TEEs), achieving externally verifiable central differential privacy (DP) guarantees for the first time. By shifting client gradient computation to attested server-side TEEs and utilizing public transparency logs, the system eliminates the traditional "trust gap" in federated learning. This architecture is already live in production, powering English and Japanese next-word prediction models on Gboard with significantly faster training times and enhanced privacy protections. --- ## 基于 TEE 的联邦学习解决了什么痛点？ > ## What Problem Does TEE-Based Federated Learning Solve? 早在 2017 年，Google 便率先推出了联邦学习 (FL) 技术，广泛应用于 Gboard 的 [下一词预测](https://arxiv.org/abs/1811.03604) 与智能撰写 (Smart Compose) 、Google Messages 的智能回复建议，以及 Android 系统的智能文本选择等日常功能中。 > Google introduced Federated Learning (FL) in 2017 to power features like [next-word prediction](https://arxiv.org/abs/1811.03604) and Smart Compose on Gboard, reply suggestions in Google Messages, and Smart Text Selection in Android. 然而，早期的联邦学习系统始终面临着难以逾越的信任鸿沟： > However, earlier systems suffered from a trust gap: * **缺乏可验证性 (Lack of Verification) ：** 用户设备在本地处理后上传数据用于实时聚合，但外部审计人员无法在密码学层面证实这些数据是否从未被服务器私下记录或窥探。 * **兼容性瓶颈 (Compatibility Issues) ：** 尽管 [Secure Aggregation](https://arxiv.org/abs/1611.04482) 安全聚合协议引入了密码学防护，但它无法兼容当前业界最前沿的中心差分隐私算法，例如 [矩阵分解 DP-FTRL](https://arxiv.org/abs/2202.08312) 。此外，用户仍必须无条件盲目相信 Google 会规范且正确地添加 DP 差分隐私噪声。 > * **Lack of Verification:** Devices uploaded data for immediate aggregation, but outsiders could not cryptographically verify that data was never logged or inspected. > * **Compatibility Issues:** While [Secure Aggregation](https://arxiv.org/abs/1611.04482) added cryptographic protection, it was incompatible with state-of-the-art central DP algorithms like [matrix factorization DP-FTRL](https://arxiv.org/abs/2202.08312). Furthermore, users had to blindly trust Google to apply DP noise correctly. 这项全新设计正是为了打破这一僵局：它将客户端的梯度计算迁移到了云端服务器上，并通过软硬件远程证明技术确保服务端运行逻辑绝对透明可查，从而使用户和外部机构彻底摆脱对云服务运营商的被动盲信。 > The new design addresses this by moving client gradient computation to the server and making the server logic attestable, removing the need to implicitly trust the operator. --- ## 这套系统是如何协同运转的？ > ## How Does the System Work? 在 Google 先前开展的 [机密联邦分析 (Confidential Federated Analytics) ](https://research.google/blog/confidential-federated-analytics/) 研究基础之上，新系统精密编排了四大核心组件： > Building upon Google’s earlier work in [confidential federated analytics](https://research.google/blog/confidential-federated-analytics/), the new system coordinates four core components: * **数据上传 (Data Upload) ：** 用户设备在本地对训练样本进行高强度加密，并预先授权访问策略。该策略精确界定了哪些 TEE 计算任务有权处理这些数据，且该策略必须公开发布到不可篡改的公开透明日志中。 * **KMS 与策略验证 (KMS and Policy Verification) ：** 基于运行 [RAFT 共识协议 (RAFT consensus protocol) ](https://raft.github.io/raft.pdf) 的 TEE 集群构建密钥管理系统 (Key Management System, KMS) ，仅当工作负载完全符合经预先授权的安全策略时，KMS 才会分发解密密钥。 * **工作负载执行 (Workload Execution) ：** 根节点 TEE (Root TEE) 负责执行 Python 训练循环，并利用基于 [TensorFlow Federated](https://www.tensorflow.org/federated) 衍生的 [Federated Language](https://github.com/google-parfait/federated-language) 将子任务安全分发给各个工作节点 TEE (Worker TEE) 。在整个计算过程中，系统最终仅对外输出符合 DP 差分隐私标准的模型权重。 * **容错与状态恢复 (Fault-Tolerant Recovery) ：** 每个训练轮次都会自动保存一份经 KMS 加密的系统恢复状态，确保无论根节点还是工作节点发生意外宕机故障，系统都能实现无缝恢复与断点续训。 > * **Data Upload:** Devices encrypt training examples locally and pre-authorize an access policy. This policy specifies which TEE computations may process the data and must be published in a public transparency log. > * **KMS and Policy Verification:** A Key Management System (KMS) built from TEEs running the [RAFT consensus protocol](https://raft.github.io/raft.pdf) releases decryption keys exclusively to workloads matching the authorized policy. > * **Workload Execution:** A root TEE runs a Python training loop and delegates subtasks to worker TEEs using [Federated Language](https://github.com/google-parfait/federated-language) (derived from [TensorFlow Federated](https://www.tensorflow.org/federated)). Only DP model weights are released. > * **Fault-Tolerant Recovery:** Each training round saves a KMS-encrypted recovery state to seamlessly handle root or worker node failures. --- ## 为什么说这里的隐私保障是真正“可验证”的？ > ## Why Is the Privacy Guarantee Verifiable? * **公开透明日志 (Public Transparency Logs) ：** 所有数据访问策略都会同步发布到 Sigstore 旗下的公开透明日志系统 [Rekor](https://docs.sigstore.dev/logging/overview/) 中。这使得任何外部独立审计员都可以全流程追踪用户设备可能喂给服务端的所有计算负载。 * **可重构构建 (Reproducible Builds) ：** KMS 密钥管理系统与核心数据处理的二进制程序均由开源代码编译而成，完全支持可重构构建 (Reproducible Builds) ，任何人都可以自行编译并核对二进制哈希值。 * **运行时安全机制 (Runtime Security) ：** 访问策略直接对 Python 训练程序的具体内容进行了严格约束。为了兼顾商业私有模型架构的机密性，TEE 允许在运行时以侧载形式加载序列化逻辑，前提是所有涉及数据隐私的关键代码必须在通过远程证明的程序中硬编码固化。此外，集群运维人员全局只能查看训练指标和经 DP 保护的模型权重，且上传的原始数据只在极其严苛的时间窗口内允许解密。 > * **Public Transparency Logs:** Access policies are published to [Rekor](https://docs.sigstore.dev/logging/overview/), Sigstore’s public transparency log, allowing external auditors to track every server workload a device could potentially feed. > * **Reproducible Builds:** The KMS and data processing binaries are reproducibly buildable from open-source code. > * **Runtime Security:** Policies directly describe the Python training program. To protect proprietary model architectures, TEEs support sideloading serialized logic at runtime, provided all privacy-relevant code remains hardcoded in the attested program. Workload operators only see metrics and DP model weights, and uploaded data can only be decrypted for a strictly limited window of time. --- ## Gboard 获得了哪些实打实的收益？ > ## What Did Gboard Gain? Gboard 输入法正是利用这套全新架构，顺利上线了英语和日语的下一词预测模型，并在实际工程中释放出两大核心架构优势： > Gboard utilized the system to roll out English and Japanese next-word prediction models, yielding two primary architectural advantages: 1. **异步解耦的灵活性 (Asynchronous Flexibility) ：** 系统会在服务端正式启动模型训练前先行收集并缓存所有加密上传的数据。如此一来，全球手机设备因昼夜作息规律带来的可用性剧烈波动不再成为训练进度的瓶颈，训练程序能够从容计算出全局最优的设备参与计划，并动态精准微调 DP 差分隐私参数。 2. **服务端高度并行化 (Server-Side Parallelization) ：** 随着计算瓶颈从低算力的移动设备转移到了高性能服务器，训练时间迎来了断崖式缩短。过去在跨设备联邦学习环境下往往需要耗费 1 至 2 个月才能完成训练的模型，如今可以在多台机器间高效并行展开，训练效率仅仅取决于可调配的 TEE 硬件算力规模。 > 1. **Asynchronous Flexibility:** All uploads are collected before server-side training begins. Diurnal swings in device availability no longer bottleneck training, allowing the program to compute an optimal participation schedule and dynamically tune DP parameters. > 2. **Server-Side Parallelization:** By moving the computational bottleneck to the server, training times dropped dramatically. Where previous FL models took 1 to 2 months to train, jobs now parallelize across machines, constrained only by available TEE hardware resources. --- ## 与其他主流联邦学习框架对比表现如何？ > ## How Does It Compare With Other FL Frameworks? | 特性 (Feature) | [Google TEE-based FL](https://research.google/blog/toward-provably-private-learning-from-federated-data/) | [NVIDIA FLARE](https://nvflare.readthedocs.io/en/main/welcome.html) | [Flower](https://github.com/flwrlabs/flower) | [Apple pfl-research](https://github.com/apple/pfl-research) | | :--- | :--- | :--- | :--- | :--- | | **主要用途 (Primary use) ** | 生产级跨设备训练 (已在 Gboard 实装上线) | 面向生产的 FL SDK，配备 Docker、Kubernetes 及云原生工具 | 构建联邦 AI 系统的通用框架 | 仅用于学术仿真模拟；不支持第三方部署 | | **客户端更新计算位置 (Where client updates are computed) ** | 服务端 TEE | 各参与计算节点本地 | 客户端本地 | 仅软件模拟 | | **硬件 TEE 支持 (Hardware TEE support) ** | 支持，基于 Project Oak 构建 | 支持：AMD SEV-SNP、Intel TDX、NVIDIA GPU 机密计算 | 并非核心框架自带组件 | 不支持 | | **差分隐私 (Differential privacy) ** | 中心 DP，支持外部独立验证 | DP 过滤器，通过 Opacus 实现 DP-SGD | 支持中心 DP 与本地 DP | 支持本地与中心 DP 机制 | | **其他隐私技术 (Other privacy tech) ** | KMS 门控解密机制；Willow 安全聚合容器 | 同态加密、隐私集合求交 (Private Set Intersection) | SecAgg 与 SecAgg+ | 不适用 (仿真环境) | | **服务端代码公开透明日志 (Public transparency log for server code) ** | 支持，采用 Sigstore Rekor | 未明确记录 | 未明确记录 | 不适用 | | **可重构 TEE 构建 (Reproducible TEE builds) ** | 支持 (包含 KMS 与数据处理二进制程序) | 未明确记录 | 不适用 | 不适用 | | **开源许可证 (License) ** | Apache 2.0 | Apache 2.0 | Apache 2.0 | Apache 2.0 | *数据来源：[Google Research 博客](https://research.google/blog/toward-provably-private-learning-from-federated-data/)、[Confidential Federated Compute 代码仓库](https://github.com/google-parfait/confidential-federated-compute)、[NVIDIA FLARE 官方文档](https://nvflare.readthedocs.io/en/main/welcome.html)、[FLARE 远程证明指南](https://nvflare.readthedocs.io/en/2.7.0/_sources/user_guide/confidential_computing/attestation.rst.txt)、[Flower 1.8 发布说明](https://flower.ai/blog/2024-04-03-announcing-flower-1.8-release)、[pfl-research 代码仓库](https://github.com/apple/pfl-research)。* > *Sources: [Google Research blog](https://research.google/blog/toward-provably-private-learning-from-federated-data/), [Confidential Federated Compute repo](https://github.com/google-parfait/confidential-federated-compute), [NVIDIA FLARE docs](https://nvflare.readthedocs.io/en/main/welcome.html), [FLARE attestation guide](https://nvflare.readthedocs.io/en/2.7.0/_sources/user_guide/confidential_computing/attestation.rst.txt), [Flower 1.8 release notes](https://flower.ai/blog/2024-04-03-announcing-flower-1.8-release), [pfl-research repo](https://github.com/apple/pfl-research).* --- ## 核心要点总结 > ## Key Takeaways * Google 成功将联邦学习的梯度计算从移动端设备迁移到了经过硬件证明的服务端 TEE 机密计算环境中。 * 依托 Rekor 公开透明日志与开源可重构构建机制，中心差分隐私保障在业界首次实现了外部独立可验证。 * Gboard 输入法已在生产环境中正式实装该升级架构，稳定支撑英语和日语的下一词预测模型。 * 模型训练周期大幅缩减，跨设备训练的瓶颈从漫长的单模型训练耗时彻底转变为取决于可用的 TEE 硬件算力规模。 * 核心 TEE 二进制代码与 Federated Language 组件均已在 Apache 2.0 协议下完全开源。 > * Google successfully transitioned FL gradient computation from mobile devices into attested server-side TEEs. > * Central DP guarantees are now externally verifiable via Rekor logs and reproducible builds. > * Gboard is actively shipping English and Japanese next-word models using this upgraded architecture. > * Training durations have been slashed, shifting the primary limitation from time-per-model to available TEE compute capacity. > * Core TEE binaries and Federated Language components have been open-sourced under the Apache 2.0 license. --- *** *欢迎查阅 [**学术论文**](https://arxiv.org/abs/2609.31494) 、 [**技术博客详情**](https://research.google/blog/toward-provably-private-learning-from-federated-data/) 以及 [**GitHub 开源仓库**](https://github.com/google-parfait/confidential-federated-compute) 。* > *Check out the [**Paper**](https://arxiv.org/abs/2609.31494), [**Technical details**](https://research.google/blog/toward-provably-private-learning-from-federated-data/), and [**GitHub Repo**](https://github.com/google-parfait/confidential-federated-compute).* https://insight.aitobox.com/blog/posts/2026-10-04/逆向实战-借助-AI-破解并修复-Altera-Quartus-Linux-jtagd-守护进程长达八年的隐蔽-Bug/ https://insight.aitobox.com/blog/posts/2026-10-04/逆向实战-借助-AI-破解并修复-Altera-Quartus-Linux-jtagd-守护进程长达八年的隐蔽-Bug/ Sun, 04 Oct 2026 00:00:00 GMT 在嵌入式开发与 FPGA 调试过程中，Altera (现已并入 Intel) 的 USB Blaster 下载线是开发者不可或缺的基础工具。然而在 Linux 平台下，其配套的 `jtagd` 守护进程自 Quartus 18.1 至 25.1 跨越八年多的时间里，一直潜伏着两项令人极其头疼的隐蔽故障：设备热插拔导致的 USB 通信数据损坏，以及首次通信时漫长的 30 秒卡顿超时。作者借助 Claude Opus 5.5 大语言模型 (Large Language Model, LLM) 进行辅助逆向工程，迅速定位到了底层 C++ 构造函数未初始化变量以及 FTDI FT245 芯片指令时序缺陷这两个深层根因。作者不仅详细剖析了 Bug 的复现与修复原理，还利用自动化电源循环完成了高强度压力测试，并为开源社区提供了开箱即用的二进制修补工具。 --- ## 内容概要 > ## Summary 在早前深入排查了廉价 Altera USB Blaster 克隆方案的各种硬件缺陷之后，作者近期重新审视了 Altera `jtagd` 守护进程 (daemon) 在 Linux 环境下两个长久存在且令人抓狂的陈年 Bug (其影响范围从 Quartus 18.1 一路贯穿至 25.1)。借助 Claude AI 辅助进行逆向工程，这两处故障的底层根因均被成功揪出并彻底修复：其一是在设备热插拔过程中，构造函数中遗留的未初始化变量导致旧版协议数据损坏；其二是存在缺陷的 FTDI 指令序列导致部分克隆设备出现长达 30 秒的响应超时。目前，作者已为开源社区发布了一款开箱即用的修补工具。 > After previously investigating issues with cheap Altera USB Blaster clones, the author revisited two persistent, frustrating Linux bugs in Altera's `jtagd` daemon (dating from Quartus 18.1 through 25.1). Using Claude AI to assist with reverse engineering, both root causes were successfully identified and patched: an uninitialized variable causing data corruption for older protocols during hot-plugging, and a problematic FTDI command sequence causing 30-second timeouts on certain clone devices. A ready-to-use patching tool has been published for the community. --- ## 引言 > ## Introduction 两年前，我在[排查 Altera USB Blaster 克隆设备的各种问题](https://www.downtowndougbrown.com/2024/07/fixing-more-cheap-altera-usb-blaster-clones-cpld-adventures/)上投入了相当大的精力。而在撰写那篇博文的过程中，我在 Linux 下遭遇了关于 `jtagd` 的几个极其恼人的偶发性问题，当时我始终无法理清其中的头绪： > I went kind of overboard [looking into issues with Altera USB Blaster clones a couple of years ago](https://www.downtowndougbrown.com/2024/07/fixing-more-cheap-altera-usb-blaster-clones-cpld-adventures/). While writing that post, I ran into some really frustrating intermittent problems in Linux with `jtagd` that I was never able to figure out: > 💬 [原文引用 / Original Quote]: > 当我在其中一台电脑上进行测试时，我还遇到了 Quartus Programmer 18.1 在 Linux 下一个非常奇怪且看似无关的毛病。从 Wireshark 抓包数据中，我能看到发出的 USB 通信流量直接出现了损坏——甚至在此刻 FT245 或 CPLD 都还没参与到数据处理中。这个问题波及了我手头所有的下载器，而不仅仅局限于基于 CPLD+FT245 的那些。 > > I also experienced a really weird unrelated problem in Quartus Programmer 18.1 in Linux while I was testing things on one of my computers. I could see corrupted outgoing USB traffic in my Wireshark capture — before the FT245 or CPLD were even involved in the equation. This problem affected all of my blasters, not just the CPLD+FT245 ones. > 💬 [原文引用 / Original Quote]: > 此外，在插入任何基于 CPLD 的 USB Blaster 之后，每当首次在 Linux 下尝试执行 JTAG 操作时，我有时仍然会注意到长达约 30 秒的严重延迟。这并非每次都会发生，但一旦出现，看起来就像是 `jtagd` 僵死在原地苦苦等待接收一个字节，而这个字节偏偏永远不会到来，导致其超时重试。在此之后，一切又恢复正常。 > > I still sometimes notice a long (~30 second) delay the first time I try to do a JTAG operation in Linux after plugging in any of my CPLD-based USB Blasters. It doesn’t happen every time, but when it does, it seems like `jtagd` is hung up waiting to receive a byte, which never happens, so it times out and tries again. It works fine after that. 由于当时我已经为了解决 USB Blaster 的其他杂七杂八问题搞得心力交瘁，对于继续深挖这几个如附骨之疽般烦人的偶发毛病彻底丧失了心力，索性就将其搁置一旁。这一搁置，就一直持续到了现在。 > Since I was already exhausted from dealing with all of the other USB Blaster problems at the time, I completely lost interest in digging further into these particular nagging issues, so I left them alone. Until now. 如今，AI 的飞速发展为我们解开这些技术小谜题开辟了一条新捷径，开发者不再需要耗费海量的沉没成本去盲目摸索。相信我，排错调试固然能带来令人兴奋的技术挑战，但面对那些时灵时不灵的偶发性故障，排查过程绝对毫无乐趣可言。 > AI has opened up the ability for some of these little mysteries to be closed out without having to spend a bunch of time investigating them. Believe me, debugging can be an exciting challenge, but intermittent issues are just not fun to diagnose at all. --- ## 借助 AI 辅助逆向工程 > ## Enter AI-Assisted Reverse Engineering 我把这两个难题统统抛给了 Claude Opus 5.5。它十分给力地直接切入对 `jtagd` 的逆向工程，并轻松锁定了这两处故障的病因。在调查并测试第一个 Bug 的某一时刻，它一度触碰到了网络安全防护规则 (cyber guardrails)。但我注意到，Opus 5.5 相比以往版本表现得更为聪明——在触发安全拦截时，它会主动尝试换一种更委婉的切入角度，甚至还会提醒我重新组织提示词 (Prompt)，而不是像以前那样不由分说直接把降级回退到 Opus 4.8 (而一旦 Opus 4.8 再次触碰防护规则，整个会话就可能直接报废)。这无疑是一项极具实用价值的体验改进。 > I fed both of these problems into Claude Opus 5.5. It happily jumped into reverse-engineering `jtagd` and easily identified both problems. At one point during its investigation and testing of the first bug, it started running into cyber guardrails, but I’ve noticed Opus 5.5 is a little better than past versions about trying a slightly different approach when a guardrail is hit, and even giving me a chance to reword my prompt instead of just immediately knocking me down to Opus 4.8 (and then potentially ending the session when Opus 4.8 hits a similar safeguard). That’s a very welcome improvement. 闲话休提，我想迅速与大家分享最终的排查发现，并附上一个现成的修补工具链接。如果你也长期被这几个 Bug 折磨得抓狂，可以直接拿它来修复你的 `jtagd`。首先，奉上补丁工具的项目地址： > Anyway, I thought I’d quickly share the final discoveries, along with a link to a tool you can use to patch your `jtagd` if these bugs have been driving you crazy. First, here’s the link to the patch tool: 👉 **[https://github.com/dougg3/altera-jtagd-linux-fixes](https://github.com/dougg3/altera-jtagd-linux-fixes)** --- ## Bug 1：USB 数据流损坏与热插拔死锁 > ## Bug 1: USB Traffic Corruption & Hot-Plug Hangs 导致 USB 通信数据损坏的第一个 Bug，最终被证实是 `JTAG_SERVER_USB_BLASTER` 类构造函数中遗留的一个未初始化变量所致。该构造函数本来把附近的一大堆相邻成员变量都置零了，但唯独这一个变量被开发者不小心遗漏了。 > The first bug that causes corruption in the USB traffic ended up being an uninitialized variable in the constructor for `JTAG_SERVER_USB_BLASTER`. The constructor sets a bunch of nearby variables to zero, but this one variable must have been accidentally forgotten. 如果这个未初始化变量在内存中的残留值恰好大于 0，本应专门发往 Altera 新型 USB-Blaster II 的额外数据包，就会被错误地塞进发出的 USB 数据流中，从而导致初代旧版 USB Blaster 陷入混乱。这彻底揭开了我在早先文章中提及的“数据损坏”之谜。与大语言模型掌握的海量背景知识相比，我脑海中当时并没有足够的知识储备去联想到这些所谓的“损坏乱码”，居然是发给二代新协议的合法数据！目前市面上我见过的所有克隆设备，采用的无一例外都是第一代 USB Blaster 通信协议，因此无一幸免都会中招。设备最终会僵死等待整整 30 秒，随后抛出 `Unable to read device chain - JTAG chain broken` (无法读取设备链 - JTAG 链路断开) 的报错。 > If its value happens to be greater than 0, some extra data intended only for Altera’s newer USB-Blaster II gets inserted into the outgoing USB data, which confuses older USB Blasters. This perfectly explains the data corruption I referred to in my earlier post. Unlike the LLM, I didn’t have enough background info in my head to make the connection that the “corrupt data” was for the newer USB Blaster protocol. All of the clone devices that I’ve seen speak the original USB Blaster protocol, so they’re all affected. They end up with a 30-second hang followed by an `Unable to read device chain - JTAG chain broken` error. 该问题似乎只有在保持 `jtagd` 持续后台运行的情况下拔出并重新插入下载器时才会触发，因为进程首次启动时的内存分配往往默认全为零。而每次进行热插拔，系统都会重新创建一个全新的 `JTAG_SERVER_USB_BLASTER` 对象，此时此前释放的堆内存碎片被重新复用，残留的脏数据便造成了未初始化变量非零。回想 2024 年我的很多开发调试都是在 Windows VM (虚拟机) 环境下进行的，因此在宿主机与虚拟机之间频繁热插拔设备是家常便饭。也许绝大多数普通用户很少会以这种方式频繁插拔，但这对于深度开发者来说，简直令人抓狂至极。 > It seems to only happen if you unplug and replug the blaster while leaving `jtagd` running, because the first allocation tends to always be zeroed out. Each hotplug results in a new `JTAG_SERVER_USB_BLASTER` object being created, and parts of the heap end up being reused. A lot of my original work back in 2024 involved a Windows VM, so it makes sense that I was probably hotplugging the device a lot. Maybe it’s not something a lot of people run into, but it was really freaking annoying. --- ## Bug 2：长达 30 秒的设备初始化卡顿 > ## Bug 2: The 30-Second Initialization Delay 导致 30 秒延迟的第二个 Bug 则要隐蔽微妙得多。它似乎取决于你的下载器具体采用了哪一种 FTDI FT245 系列芯片，甚至可能与设备连接到电脑的具体拓扑方式有关。问题的关键核心在于：当首次打开芯片通信时，`jtagd` 会严格按以下顺序向 FT245 发送一系列控制指令： > The second bug with the 30-second delay is a bit more subtle. It seems to depend on what type of FTDI FT245 series chip your blaster uses, and maybe even how it’s connected to your computer. The crux of the problem is that when it first opens the chip, `jtagd` sends the following commands to the FT245, in order: * 复位 (Reset) * 清空发送缓冲区 (Purge TX) * 清空接收缓冲区 (Purge RX) * 设置延迟定时器 (Set Latency Timer) > * Reset > * Purge TX > * Purge RX > * Set Latency Timer 这一连串特定的指令时序，似乎会导致某些克隆硬件在约 250 毫秒的时间窗口内对所有传入的数据充耳不闻。在通常情况下，短短 250 毫秒可能无足轻重，然而偏偏 `jtagd` 在下发完指令后会立即发起通信以扫描连接的 JTAG 扫描链，结果一头撞在铁板上，只能傻傻干等 30 秒直到超时。 > This particular sequence of commands appears to cause some clone devices to ignore incoming data for about 250 ms. This wouldn’t ordinarily be much of a problem, except `jtagd` immediately starts talking to it to figure out the connected JTAG chain, and then gets stuck waiting 30 seconds for a response. 我手头有若干款不同的下载器克隆，面对这套指令序列它们的反应各有千秋 (有的深受其害，有的却安然无恙)，但这些指令在本质上都属于标称的 FT245 控制命令。也许不同克隆版使用了不同的芯片批次或变种？但事实胜于雄辩：**只要调整这条指令序列，将 `Set Latency Timer` 提到最前面执行，该卡顿问题就烟消云散了。** > I have different blaster clones that each respond in their own different ways to this command sequence (some are affected, and some are not), but these are just FT245 commands. Maybe different clones have different chip revisions? All I know is: **changing this sequence so that `Set Latency Timer` comes first completely eliminates the problem.** --- ## 总结与压力测试验证 > ## Conclusion & Stress Testing 这两大缺陷至少早在 Quartus 18.1 版本中就已存在于 `jtagd` 之中，甚至直到最新的 Quartus 25.1 依然健在。Altera，如果你们看到了这篇文章，恳请务必考虑把至少第一个 Bug 修复掉！说老实话，对于第二个问题，我不确定它究竟算不算软件本身的“Bug”，还是仅仅因为某些克隆硬件上的芯片在电气特性上无法做到与原装正品完全严丝合缝，但无论如何，只要在 `Set Latency Timer` 指令后加入一个 300 毫秒的延时，大概率也能完美规避。 > Both of these issues have been in `jtagd` at least as far back as Quartus 18.1, and are still there as of Quartus 25.1. Altera, if you see this post, please consider fixing at least the first bug! I’m honestly not sure if the second one is actually a “bug” or if it’s just that certain clone devices have chips that don’t act exactly like the official device, but a 300 ms delay after `Set Latency Timer` would probably fix it too. 在给这两个问题打上补丁后，我手头能够找到的所有 USB Blaster 克隆版现在都能稳定如飞地工作了 (当然前提是配合我此前发布的固件及 CPLD 修复补丁)。为了进一步验证，我甚至翻出了一个兼容 [uhubctl](https://github.com/mvp/uhubctl) 的 USB 分线器，实现对我所有的克隆设备进行自动化的反复断电与通电。这让我能够建立一个闭环反馈环境，运行高强度的自动化压力测试，从而真真切切地证实补丁的有效性。 > With fixes for these two problems applied, all of the USB Blaster clones I have access to work great now (after my previous firmware/CPLD fixes are also applied, of course). I even used a USB hub compatible with [uhubctl](https://github.com/mvp/uhubctl) to automate cycling power to all of the clone devices I have. That enabled me to create a feedback loop to do a bunch of stress testing to really verify that the fix actually works. 事后回想，其实我完全有能力 (也本应该) 靠自己找出第一个 Bug。事实上，只要当时用一下内存分析工具 `valgrind`，它几乎就像用银盘呈递美食一样把答案双手奉上。当时自己居然没想过试一试，回想起来确实有些懊恼。彼时我正全身心沉浸在使用录制重放调试器 [rr](https://github.com/rr-debugger/rr) 试图精准复现故障，然而 `rr` 与我的主力开发机硬件并不兼容。当我千辛万苦把 Quartus 折腾安装到一台与 `rr` 兼容的老机器上时，故障却死活再也无法复现了。当时的我早就被折腾得心力交瘁，只得愤然作罢。不过公平地讲，这个 Bug 在 `jtagd` 中至少潜伏了整整八年之久，在此期间几乎没有引起任何人的警觉，除了 [Altera 官方论坛上的这篇简短讨论帖——当时有人察觉到了不对劲的蛛丝马迹](https://community.altera.com/discussions/quartus-prime/quartus-jtagd-fails-and-unable-to-program-max10-under-linux-18-1-0-625/320842/replies/320850)，可惜随后便石沉大海、无人问津。 > In reflection, I totally could (and should) have figured out the first bug on my own. It seems like `valgrind` would have just handed it to me on a silver platter. I’m kind of disappointed in myself for not thinking to try that. At the time, I was deep into an investigation with [rr](https://github.com/rr-debugger/rr) while trying to reproduce the bug, but it wasn’t compatible with my main development machine. When I got Quartus installed onto an older computer that *was* compatible with `rr`, I couldn’t replicate the issue anymore. I was sick of dealing with it all at that point, so I gave up. In my defense, this bug has been in `jtagd` for at least 8 years, and has mostly gone unnoticed, except for [this little post on Altera’s forums where someone realized something funky was going on](https://community.altera.com/discussions/quartus-prime/quartus-jtagd-fails-and-unable-to-program-max10-under-linux-18-1-0-625/320842/replies/320850) and was pretty much ignored. 我的技术博客绝不会沦为单纯鼓吹“快看 AI 为我做了什么”的炫技秀场。我真正希望达成的，是为开源社区提供行之有效的应急解决方案，并借此唤起 Altera 官方的重视，敦促他们早日修复这一历史遗留缺陷。此外，也许我确实有点偏执狂，但借助 AI 去修补闭源商业软件中令人抓狂的死穴 Bug，本身就是一件极具极客浪漫感与成就感的事。尤其是当你把过去以为根本不可能解开的陈年悬案亲手结案时，那种畅快感无与伦比。沿着这一思路，本文后续还会迎来一篇续作，我将在其中彻底揭开当年探索 USB Blaster 时遗留下的最后一个未解之谜。 > My blog isn’t turning entirely into a “hey, look at what AI did for me” showcase. What I’m trying to do is share a useful workaround with the community and hopefully raise awareness for Altera to fix their long-standing bug. Also, maybe I’m just crazy, but it’s kind of fun fixing annoying bugs in closed-source software with AI. Especially when it’s an old loose end from the past that I thought was impractical to tie up. With that in mind, there actually is going to be a follow-up to this post where I resolve one final unanswered question from my past USB Blaster investigations. https://insight.aitobox.com/blog/posts/2026-10-04/Prime-Intellect-正式推出-Prime-Inference-面向前沿开源模型的无服务器与专属算力推理平台/ https://insight.aitobox.com/blog/posts/2026-10-04/Prime-Intellect-正式推出-Prime-Inference-面向前沿开源模型的无服务器与专属算力推理平台/ Sun, 04 Oct 2026 00:00:00 GMT 随着开源前沿大语言模型 (LLM) 参数规模与应用复杂度的飞速跃升，如何在生产环境中以极低延迟、高吞吐和高可靠性提供稳定服务，已成为整个 AI 行业的核心工程瓶颈。知名去中心化 AI 计算与训练团队 Prime Intellect 正式推出了其全新推理平台 —— Prime Inference，提供无服务器端点与跨多数据中心的专属预留算力。该平台深度整合了包括 NVIDIA Blackwell 硬件与 NVIDIA Dynamo、vLLM、Mooncake、FlashInfer 在内的前沿开源技术栈，在公开发布前已在内部承受日均近万亿 Token 的极端压力考验。在 GLM-5.3 等前沿开源模型的基准测试中，该平台不仅实现了 100% 的高可用率，更以接近零失误的工具调用表现树立了开源推理服务的新标杆。Prime Inference 的正式推出，成功打通了从模型强化学习采样、自动化评估到生产部署的完整技术闭环。 --- ## 执行摘要 > ## Executive Summary Prime Intellect 正式推出了 **Prime Inference**，这是一个专为前沿开源模型打造的高性能推理服务平台。该平台跨多个数据中心同时提供灵活的无服务器 (Serverless) 端点与专属预留算力 (Reserved Capacity) 选项。在正式公开发布前，该平台已在团队内部历经高强度实战检验——日均吞吐近一万亿 Token，稳定支撑着强化学习采样 (RL Rollouts)、合成数据生成、模型自动化评估以及长时间运行的代码智能体 (Coding Agents) 任务。通过搭载以 NVIDIA Blackwell 为代表的先进硬件，并深度融合 vLLM、NVIDIA Dynamo、Mooncake 与 FlashInfer 等顶尖开源服务工具链，Prime Inference 实现了极高的吞吐性能、卓越的系统在线率以及接近于零的工具调用 (Tool Call) 失败率。 > Prime Intellect has launched **Prime Inference**, a high-performance serving platform designed specifically for frontier open-source models. Offering both serverless endpoints and reserved capacity across multiple datacenters, the platform processed nearly a trillion tokens per day internally before its public release to handle RL rollouts, synthetic data generation, evaluations, and long-running coding agents. By integrating advanced hardware like NVIDIA Blackwell and combining cutting-edge open-source serving tools (such as vLLM, NVIDIA Dynamo, Mooncake, and FlashInfer), Prime Inference delivers high throughput, exceptional uptime, and near-zero tool-call error rates. --- ## 什么是 Prime Inference？ > ## What is Prime Inference? Prime Inference 在 Prime Intellect 的开源训练技术体系中充当着基石般的服务层。由于该团队此前已研发了包括 `prime-rl`、验证器 (Verifiers) 以及安全沙盒 (Sandboxes) 等后训练 (Post-training) 关键工具，这一全新推理层的加入使得整体技术架构实现了闭环：线上部署的模型在生产环境中生成的真实运行轨迹 (Production Traces)，能够直接无缝反哺并注入后续的训练迭代循环中。 > Prime Inference serves as the foundational serving layer for Prime Intellect’s open training stack. Because the company already develops post-training tools like `prime-rl`, verifiers, and sandboxes, this new inference layer closes the loop: deployed models generate production traces that directly feed back into subsequent training cycles. 据 Prime 官方公布的数据，其托管的 [GLM-5.3](https://huggingface.co/zai-org/GLM-5.3) 端点在 OpenRouter 平台的性能排行中跻身最快梯队，自上线以来不仅保持着 100% 的极高可用率，其工具调用错误率更是几乎降为零。 > Prime reports that its [GLM-5.3](https://huggingface.co/zai-org/GLM-5.3) endpoint ranks among the fastest on OpenRouter, accompanied by a near-zero tool-call error rate and 100% uptime since launch. https://insight.aitobox.com/blog/posts/2026-10-03/冲突监督改变的是承诺而非能力-在惯例无关评分下归零的12.29个标准差数据排布效应/ https://insight.aitobox.com/blog/posts/2026-10-03/冲突监督改变的是承诺而非能力-在惯例无关评分下归零的12.29个标准差数据排布效应/ Sat, 03 Oct 2026 00:00:00 GMT 在大语言模型 (Large Language Model, LLM) 的监督微调 (Supervised Fine-Tuning, SFT) 与训练中，当训练数据包含两种互不兼容但各自正确的表达惯例时，数据的输入顺序往往会对最终评测产生显著干扰，常被认为深刻影响了模型能力。本文通过严格控制训练预算与数据集，深入剖析了学习率调度器与数据排布之间的相互作用机制，证明数据排序所引发的剧烈波动并未真正改变模型的底层“能力”，而仅仅是决定了模型在没有明确指令时优先“承诺”遵循哪一种特定惯例。令人惊叹的是，当引入“惯例无关”的评估准则 (只要模型在任一合规惯例下求解正确即可) 时，此前高达 $12.29\sigma$ 的巨大排布偏差直接归零，模型综合准确率在各种数据排布下保持高度守恒。进一步实验发现，只需在提示词 (Prompt) 中显式注明当前生效的惯例，即可轻松化解数据顺序带来的偏好锁定，并释放出高达 87.5% 的能力上限，为大模型数据工程与公平评测提供了极具穿透力的新视角。 --- ## 📌 核心要点 > ## 📌 Summary 面对同一批问题，如果采用两种互不相容却各自正确的表达规范对机器学习模型展开训练，模型究竟会学到什么？作者在严格控制训练算力预算、训练数据集以及优化路径的前提下，深入探究了数据的先后排列顺序与学习率调度策略如何联手塑造模型的最终表现。 > This paper investigates what happens when a machine learning model is trained on identical problems written under two incompatible yet equally correct conventions. By keeping the training budget, dataset, and path strictly controlled, the author examines how the arrangement of data and the learning rate schedule influence model outcomes. 该项研究的核心结论包括： > Key takeaways from the research include: * **学习率调度器充当加权平均算子 (The Learning Rate Schedule as an Averaging Operator) ：** 在优化算法中，数据排布效应与学习率调度是以互相独立的乘法因子形式发挥作用的。学习率衰减策略能够避免大步长与未收缩的余项在最终一步发生剧烈碰撞，从而有效平抑了数据顺序带来的影响；而恒定学习率策略则不具备这种平抑机制。 * **惯例承诺与底层能力的辨析 (Commitment vs. Capability) ：** 研究证明，“顺序至关重要”与“顺序无关紧要”实际上只是同一控制旋钮的两端。训练数据的排布顺序只是促使模型在无约束时“承诺”遵循某一种具体惯例，而并未真正改变其解决问题的底层“能力”。 * **惯例无关下的能力守恒 (Convention-Agnostic Conservation) ：** 在引入不区分惯例的综合评估指标后，原本测得的 $12.29\sigma$ 剧烈排布跳变效应**直接归零** (在十二个测试组中，$\text{acc}_A + \text{acc}_B$ 的总和波动始终保持在 9.7% 的极小范围内) 。 * **提示词干预消除偏差 (Prompt Intervention) ：** 只要在提示词中显式标注当前生效的规范惯例，即可绕过数据顺序带来的排布锁定，并成功恢复高达 87.5% 的联合能力上限。 > * **The Learning Rate Schedule as an Averaging Operator:** The arrangement effect and schedule enter the optimization as separate, multiplied factors. While decaying schedules prevent large step sizes and uncontracted remainders from colliding at the final step, constant schedules do not, which moderates ordering effects. > * **Commitment vs. Capability:** "Order matters" and "order does not matter" are demonstrated to be two ends of a single knob. Training order forces a model to *commit* to a specific convention rather than altering its underlying *capability*. > * **Convention-Agnostic Conservation:** The measured $12.29\sigma$ arrangement switch drops to **exactly zero** under a convention-agnostic metric ($\text{acc}_A + \text{acc}_B$ remains constant within 9.7% across twelve arms). > * **Prompt Intervention:** Marking the active convention explicitly in the prompt bypasses the ordering switch and recovers 87.5% of the union ceiling. --- ## 📄 论文摘要 > ## 📄 Abstract 如果用两种互不兼容、但各自完全正确的表达惯例来书写同一批问题，并以此训练模型，那么这些数据的输入先后顺序究竟会在模型参数中留下怎样的印记？学习率调度策略绝非这一问题的旁观背景，它本质上是一个加权平均算子，正是它决定了最终的答案。我们从理论上证明了一个上界：在该上界中，数据排布和学习率调度以完全解耦的乘法因子形式作用于数据排序效应——排布方式仅体现为分块周期 (Block Period) ，而调度策略则仅决定了最终模型状态能对训练历程中的某一个特定时刻施加多大的权重。衰减型学习率调度器无法在同一个时刻同时出现大步长与未收缩的余项；相反，恒定学习率调度器在训练的最后一步恰恰会遭遇这种冲突。尽管在预训练阶段已有研究指出衰减调度能够平抑排序效应，但揭示其内在机理、实现两者因素的数学分离，以及对这两部分进行严格受控的定量测量，则是本文的核心原创贡献。我们在同一语料库、相同算力预算下设计了十种数据排序，除优化路径外完全保持一致，并在仅有 `lr_scheduler_type` 不同的两类调度策略下各运行了一次：在恒定学习率下，中间分布在分配份额上跨越了 0.2221，对比底限达到 11.63，且随着排布分块程度的加深呈现单调变化。而在所有已发表实验所普遍采用的单次余弦 (Cosine) 衰减调度下，原本自身分辨率足以分辨出约十种状态的十个实验组，最终仅坍缩为两种可区分的状态。“顺序重要”与“顺序无关”，本质上不过是同一个旋钮的两端。优化路径真正在参数中刻下的，仅仅是模型到底承诺遵循哪一种惯例，而传统的精确匹配 (Exact-Match) 基准测试根本无法洞察这一真相。在全部十二个实验分支中，尽管惯例分配份额在 0.04 到 0.87 之间大幅波动，但两者的准确率之和 $\text{acc}_A + \text{acc}_B$ 的变动幅度始终保持在 9.7% 以内；因此，本文所测量到的高达 $12.29\sigma$ 的排布跳变效应，在不区分具体惯例的评估指标下其实恰好归零。该守恒定律在采用衰减调度的模型组中表现得尤为纯粹，而恒定学习率组因包含更多噪声而不易直接观测。只要在提示词中明确标注当前生效的惯例，这种由数据排序引起的跳变就会彻底瓦解，并能达到联合能力上限的 87.5%。 > 💬 [原文引用 / Original Quote]: > "Train a model on the same problems written under two incompatible conventions, both correct, and ask what the ordering of that data writes into the parameters. The learning-rate schedule is not a background condition for that question. It is the averaging operator, and it decides the answer. We prove a bound in which the arrangement and the schedule enter the ordering effect as separate multiplied factors: the arrangement only as a block period, the schedule only as how much weight the endpoint can place on any one moment of the run. A decaying schedule cannot put a large step size and an uncontracted remainder at the same moment; a constant one does exactly that at the last step. That decay moderates ordering effects has been reported in pretraining; the mechanism, the separation, and a controlled measurement of both halves are ours. Ten orderings of one corpus, one budget, everything but the path held fixed, run twice under families differing in `lr_scheduler_type` and nothing else: at a constant rate the interior spans 0.2221 in allocation, 11.63 contrast floors, monotone in how blocked the arrangement is. Under the single cosine every published arm uses, the same ten arms occupy two distinguishable states where their own resolution would allow about ten. 'Order matters' and 'order does not matter' are the two ends of one knob. What the path writes is which convention the model commits to, and no exact-match benchmark can see it. Across twelve arms acc_A+acc_B is constant to within 9.7% while the allocation share runs 0.04 to 0.87, so the 12.29-sigma arrangement switch this paper measures is exactly zero under a convention-agnostic metric. That conservation is quoted from the decayed family throughout, the constant-rate one being a noisier place to read it. Marking the convention in the prompt collapses the switch and reaches 87.5% of the union ceiling." --- ## 🛠️ 论文元数据与全文链接 > ## 🛠️ Metadata & Access Links * **论文备注 (Comments) ：** 62 页 (62 pages) * **授权协议 (License) ：** [Creative Commons Attribution 4.0 International](http://creativecommons.org/licenses/by/4.0/)  * **全文获取 (Full-Text Access) ：** * [查看 PDF (View PDF) ](https://arxiv.org/pdf/2610.00234) * [HTML 版本 (Experimental) ](https://arxiv.org/html/2610.00234v1) * [TeX 源码 (TeX Source) ](https://arxiv.org/src/2610.00234) * **引文检索与工具 (Citations & Tools) ：** * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2610.00234) * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2610.00234) * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2610.00234) https://insight.aitobox.com/blog/posts/2026-10-03/全局连贯性困境-当每个智能体都正确而团队依然出错-多智能体协同的局部到全局语义基石/ https://insight.aitobox.com/blog/posts/2026-10-03/全局连贯性困境-当每个智能体都正确而团队依然出错-多智能体协同的局部到全局语义基石/ Sat, 03 Oct 2026 00:00:00 GMT 在当今的大语言模型 (Large Language Model, LLM) 应用中，多智能体协同已成为应对复杂任务的核心范式，但系统在实际运行中常常遭遇一种难以捉摸的困境：每一个 AI 智能体 (AI Agent) 依据自身视角做出的决策都完全合理，但团队汇总出的最终结果却漏洞百出。针对这一长期困扰业界的难题，学者 Xin Heng 在本篇论文中深入剖析了这一“全局连贯性 (Global Coherence) ”问题，并给出了严格的数学证明——“观测混淆不可能定理 (Observation-Aliasing Impossibility Theorem) ”，指出在观测上下文缺失的情况下，单纯增加推理步数、引入更多角色或扩充通信轮次，在理论上根本无法消除错误。为此，论文基于拓扑学与层论提出了“局部到全局运行时语义框架” ($\mathcal{X}$)，确立了“模型仅负责提议行动、管控底座全权掌控共享状态与提交”的设计准则。该研究为多智能体系统从松散对话迈向高可靠工程落地奠定了严谨的理论基石。 --- ## 执行摘要 > ## Executive Summary 多智能体 AI 系统常常深陷一个致命的系统性缺陷：**每个独立的智能体都能做出局部完全合理的正确决策，但团队协同起来却得出一个全盘错误的荒诞结果。** > Multi-agent AI systems frequently suffer from a critical systemic flaw: **individual agents can make locally valid decisions while the team jointly produces an invalid result.** 在这篇论文中，Xin Heng 对这一现象——即 **全局连贯性问题 (global coherence problem) **——展开了深入探究。作者指出，这种故障的根源在于共享状态管理的底层失效，而非模型本身不够聪明。为了厘清系统能力的严格理论边界，作者提出了 **观测混淆不可能定理 (Observation-Aliasing Impossibility Theorem) **，从数学上证明了局部层面的小修小补、增加思考推理步数或在智能体之间发送更多消息，都无法弥补因观测上下文不全而带来的天然盲区。 > In this paper, Xin Heng investigates this phenomenon—termed the **global coherence problem**—arguing that it represents a fundamental failure of shared state management rather than a deficiency in model intelligence. To establish strict theoretical boundaries, the author introduces the **Observation-Aliasing Impossibility Theorem**, proving that localized improvements, increased reasoning steps, or additional messaging cannot overcome incomplete observational context. 为破解这一死局，论文提出了一套严谨的 **从局部到全局运行时语义框架 ($\mathcal{X}$) **。大量实证评估充分证明：采用具备全局管控能力的运行底座 (harness) 来统一协调状态，其可靠性远远优于单纯依赖去中心化智能体之间的相互通信。 > To resolve this, the paper presents a robust **local-to-global runtime semantics framework ($\mathcal{X}$)**, backed by empirical evaluations demonstrating that enforcing global state harnesses vastly outperforms reliance on decentralized agent communication. --- ## 核心理论贡献 > ## Key Theoretical Contributions https://insight.aitobox.com/blog/posts/2026-10-03/AWS-Strands-Labs-开源-Strands-Decider-2B-115毫秒极速响应的单次前向决策大模型/ https://insight.aitobox.com/blog/posts/2026-10-03/AWS-Strands-Labs-开源-Strands-Decider-2B-115毫秒极速响应的单次前向决策大模型/ Sat, 03 Oct 2026 00:00:00 GMT 在构建现代 AI 智能体 (AI Agent) 系统的过程中，传统的大语言模型 (Large Language Model, LLM) 往往扮演着“事无巨细皆需长篇大论”的角色，即便是面对二选一确认、意图路由或工具选择等简单的确定性判定，也必须逐字解码生成 Token，带来了不可忽视的延迟开销与格式幻觉隐患。为攻克这一工程痛点，AWS Strands Labs 正式开源了参数量为 19 亿的“快思考系统” (System One) 决策模型 Strands Decider 2B。该模型创新性地去除了传统大语言模型的解码生成头，改用极轻量的指针分类头，在单次前向传播中仅需约 115 毫秒即可直接返回离散选项、概率或评分，同时保持了极佳的统计校准水平。这一成果不仅能够在消费级显卡或 Apple Silicon 上本地流畅运行，更以零幻觉、低延迟和全开源的特性，为下一代端到端智能体系统的敏捷编排与实时安全护栏树立了全新的技术标杆。 --- ## 执行摘要 > ## Executive Summary AWS Strands Labs 正式推出了 **Strands Decider 2B**，这是一款包含 19 亿参数的开源“快思考系统” (System One) 决策模型。与传统的大语言模型 (Large Language Model, LLM) 不同，Strands Decider 并不生成自由文本；相反，它能够精确评估当前上下文状态与特定类型的问题，直接返回离散选项、是否概率 (yes/no) 或结构化的评分评估。该模型支持在本地 CPU、消费级 GPU 以及搭载 Apple Silicon 的 Mac 电脑上运行，中位推理延迟仅约 115 毫秒，专为工具选择、模型路由以及智能体安全护栏 (Guardrails) 等讲求高效率与校准精度的关键任务量身打造。目前，其完整模型权重已在 [Hugging Face](https://huggingface.co/StrandsAgents/strands-decider-2B-hobson-v19) 上完全开源，遵循宽松的 Apache-2.0 许可证。 > AWS Strands Labs has launched **Strands Decider 2B**, an open-source "System One" decision model containing 1.9 billion parameters. Unlike traditional Large Language Models (LLMs), Strands Decider does not generate text; instead, it evaluates a state and typed questions to return a discrete choice, a yes/no probability, or a scored evaluation. Running locally on CPUs, consumer GPUs, or Apple Silicon Macs with a median latency of roughly 115 ms, the model is designed for efficient, calibrated tasks like tool selection, model routing, and agent guardrails. Its weights are openly available on [Hugging Face](https://huggingface.co/StrandsAgents/strands-decider-2B-hobson-v19) under the Apache-2.0 license. --- ## 部署方式与易用性 > ## Deployment & Accessibility Strands Decider 2B 完全支持在本地与私有化自托管环境中独立部署： > Strands Decider 2B is fully deployable for local and self-hosted environments: * **便捷安装**：只需执行 `pip install strands-decider` 即可完成一键安装，同时提供命令行工具 (CLI) 与 HTTP 服务端； * **环境要求**：由于内置的服务器默认绑定至本地回环地址 `127.0.0.1` 且未集成原生身份验证，若在生产环境中向外部开放，必须配置外部鉴权代理层； * **推理服务**：目前阶段尚无托管型的第三方云推理服务商接入该模型。 > * **Installation:** Easily installed via `pip install strands-decider`, providing both a CLI and an HTTP server. > * **Requirements:** Because the bundled server binds to `127.0.0.1` without built-in authentication, production deployments require an external auth layer. > * **Inference:** Currently, no hosted third-party inference providers serve the model. --- ## 决策模型的核心机制 > ## What a Decision Model Does 作为自 TypeSafe AI 推出 Jev 以来不断壮大的“快思考系统” (System One) 模型门类的新成员，决策模型将输出范围严格限制在预定义选项列表或特定数值刻度之内，彻底避免了标准大语言模型常见的无序自由文本生成与输出漂移。 > As part of a growing category of "System One" models that emerged following TypeSafe AI’s launch of Jev, decision models restrict their outputs strictly to predefined options or numerical scales, avoiding the arbitrary text generation common in standard LLMs. Strands Decider 原生支持三类各具特色的结构化提问类型： > Strands Decider supports three distinct question types: * **`choice` (单项选择) **：从 $N$ 个预定义候选选项中挑选出 1 个最佳选项； * **`noul` (二元概率) **：在 0 到 1 之间精确评估针对某一命题的“是/否”判定置信概率； * **`score` (标准评分) **：依据预设的有序评判基准量表对输入内容进行打分评级。 > * **`choice`**: Selects 1 out of $N$ possible options. > * **`noul`**: Evaluates a yes/no probability between 0 and 1. > * **`score`**: Rates an input against an ordered rubric. 模型的每一次输出均严格受限于所允许的候选选项子集，并附带经过严格统计校准的置信度评分。需要注意的是，该模型**并不适合**用于处理复杂的长链推理、代码编写、多轮闲聊对话或长篇文本摘要等任务。 > Every output returns a strict subset of the allowed options alongside a calibrated confidence score. The model is **not** suited for complex reasoning, coding, conversational chat, or summarization tasks. --- ## 架构设计：拆掉“发言嘴”的大语言模型 > ## Architecture: An LLM with Its Mouth Removed 该模型构建于 [Qwen3.5-2B-Base](https://huggingface.co/Qwen/Qwen3.5-2B-Base) 基础架构之上，并针对分类决策的执行效率重构了核心网络层： > The model is built upon the [Qwen3.5-2B-Base](https://huggingface.co/Qwen/Qwen3.5-2B-Base) architecture, modifying the core structure for classification efficiency: * **轻量指针头 (The Pointer Head)**：彻底弃用了传统大语言模型庞大的因果语言建模输出头，转而换上了一个仅约 100 万参数 (约 1M) 的精简指针头； * **单步推理机制 (Inference Mechanism)**：该指针头直接将 `<answer>` 标记位置的隐藏状态 (Hidden State) 与每个候选选项末尾 Token 的隐藏状态进行点积比对。这使得模型无需经历缓慢的自回归**解码循环** (Decoding Loop)，仅凭单次前向传播 (Forward Pass) 即可瞬间给出决策结果； * **模型主干与精度 (Torso & Scale)**：模型主干网络采用了 rank-16 的 LoRA 微调，而指针分类头则以高精度的 `fp32` 格式执行。由于候选标签集是在每次 API 请求中动态给出的，因此可支持的选项数量在理论上几乎不受限制 (当前发布的模型检查点为 `v19`) ； * **极速多任务复用 (Efficiency)**：针对同一段文本输入同时进行多项问题评估时具有极致的性能优化——原始文本的状态仅需计算读取一次，后续附加的多个判定问题只会产生极小的额外 Token 计算开销。 > * **The Pointer Head:** The traditional language-modeling head is discarded and replaced with a lightweight pointer head (~1M parameters). > * **Inference Mechanism:** The head compares the hidden state at the `<answer>` position against the hidden state at each option’s last token. This allows the model to yield results in a single forward pass **without a decoding loop**. > * **Torso & Scale:** The torso utilizes a rank-16 LoRA, while the head executes in `fp32`. Because label sets are defined dynamically via requests, option counts are virtually uncapped (current released checkpoint: `v19`). > * **Efficiency:** Evaluating multiple questions for a single text input is highly optimized—the state is read just once, and subsequent questions incur minimal token overhead. --- ## 基准测试与推理延迟 > ## Benchmarks and Latency 在 [JevBench](https://benchmarkheaven.com/jev-models) 公开基准测试集上，`v19` 版本的模型检查点展现出了极具竞争力的性能水准： > Measured against public sets on [JevBench](https://benchmarkheaven.com/jev-models), the `v19` checkpoint delivers competitive performance: * **公开测试准确率**：0.723 (在 JevBench v1 的 231 项评测任务中成功解决 167 项) ； * **概率校准表现**：Brier 得分为 0.342，预期校准误差 (Expected Calibration Error, ECE) 仅为 0.052； * **分级难度表现**：简单任务达 1.000；标准任务为 0.875；困难任务为 0.505； * **硬件延迟表现 (RTX 3090)**：中位延迟仅 115 毫秒，p95 延迟为 299 毫秒； * **硬件延迟表现 (M3 Pro)**：预热后中位延迟为 153 毫秒 (输入文本在 300 个 Token 以内) 。 > * **Public Accuracy:** 0.723 (167 out of 231 tasks on JevBench v1). > * **Calibration:** Brier score of 0.342; Expected Calibration Error (ECE) of 0.052. > * **Tier Breakdown:** Easy tasks: 1.000; Standard tasks: 0.875; Hard tasks: 0.505. > * **Latency (RTX 3090):** 115 ms median, 299 ms p95. > * **Latency (M3 Pro):** 153 ms warm median (under 300 tokens). https://insight.aitobox.com/blog/posts/2026-10-03/AI-监管能否实现零知识-兼顾机密隐私与可验证性的密码学监督边界/ https://insight.aitobox.com/blog/posts/2026-10-03/AI-监管能否实现零知识-兼顾机密隐私与可验证性的密码学监督边界/ Sat, 03 Oct 2026 00:00:00 GMT 随着人工智能日益深入医疗诊断评估、前沿药物分子研发等关键领域，AI 系统频繁接触并处理大量核心机密数据，如何在不公开底层敏感数据的前提下证明其输出结果的正确性，成为保障 AI 安全可信的核心瓶颈。本文深入探讨了在引入外部物理实验、人类判断或网络查询的预言机辅助计算 (oracle-aided computation) 场景下，交互式论证能否达成零知识 (Zero Knowledge) 隐私保护；研究首先证明了一个奠基性的不可能结果：在标准的 random oracle model 下，即便证明方与验证方具备无限算力，通用的 zero knowledge 证明在数学上也是无法实现的，这一理论限制同样适用于当下的主流 AI 监督范式辩论机制 (Debate) 。针对这一看似不可逾越的理论鸿沟，作者给出了极具突破性的密码学解决方案：只要预言机为返回的每个答案附加密码学签名，那么仅需依赖标准的 collision-resistant hash functions ，就能在多项式时间内以极高效率实现完备的 zero knowledge 验证。该成果不仅从密码学理论上划定了兼顾隐私保密与计算可验证性的清晰边界，更为无需依赖可信对手或鲁棒性假设的新一代可扩展 AI 监督架构铺平了道路。 --- # AI 监管能否实现零知识 (Zero Knowledge) ？兼顾机密隐私与可验证性的密码学监督边界 > # Can AI Oversight Be Zero Knowledge? ## 📌 核心概要 > ## 📌 Summary 如今的 AI 系统正在处理越来越多的核心机密数据——比如在员工健康评估中调用的个人病历，或是创新药研发中涉及商业秘密的独家分子结构。在这种背景下，如何在完全不公开底层敏感数据的前提下，让人确信 AI 给出的结论是完全正确的？这成为了至关重要的现实难题。本文针对带有外部辅助输入的 oracle-aided computations (预言机辅助计算) ，深入探讨了其 interactive arguments (交互式论证) 能否达成 **zero knowledge** (确保 verifier 在验证过程中，除了确信输出正确之外，无法获取任何额外机密信息) 。 > As AI systems increasingly process confidential data—such as medical records for fitness-for-duty assessments or proprietary molecular structures for drug discovery—verifying the correctness of their outputs without exposing the underlying data becomes crucial. This paper investigates whether interactive arguments for oracle-aided computations can achieve **zero knowledge** (ensuring the verifier learns nothing beyond the output's correctness). 作者首先给出了一个奠基性的理论“不可能定理”：在经典的 random oracle model 下，通用的 zero-knowledge proofs 根本无法覆盖所有的 oracle-aided computations ；哪怕赋予 prover (证明者) 和 verifier (验证者) 无限的计算时间与算力，这一局限也无法打破。不过研究者并未止步于此，而是进一步提出了一套精巧的正向解决方案：只要 oracle (预言机) 对其返回的每一个答案进行密码学签名，那么仅仅依托基础的 collision-resistant hash functions ，就能让多项式时间内高效运行的 prover 与 verifier，对任意 oracle-aided computations 完成严谨的 zero knowledge 验证。 > The authors demonstrate a fundamental impossibility result: in the random oracle model, zero-knowledge proofs do not exist for all oracle-aided computations, even with unbounded provers and verifiers. However, they introduce a positive solution: if the oracle signs each answer cryptographically, every oracle-aided computation can be verified in zero knowledge with efficient provers and verifiers using only collision-resistant hash functions. --- ## 📋 论文元数据 > ## 📋 Metadata * **arXiv 编号：** [arXiv:2610.01995](https://arxiv.org/abs/2610.01995) [cs.AI] * **研究领域：** 人工智能 (`cs.AI`)；计算复杂度 (`cs.CC`)；密码学与安全 (`cs.CR`) * **论文作者：** Alessandro Chiesa, Ziyi Guan, Burcu Yildiz * **提交时间：** 2026 年 10 月 1 日 * **DOI：** [10.48550/arXiv.2610.01995](https://doi.org/10.48550/arXiv.2610.01995) > * **arXiv ID:** [arXiv:2610.01995](https://arxiv.org/abs/2610.01995) [cs.AI] > * **Subjects:** Artificial Intelligence (`cs.AI`); Computational Complexity (`cs.CC`); Cryptography and Security (`cs.CR`) > * **Authors:** Alessandro Chiesa, Ziyi Guan, Burcu Yildiz > * **Submitted:** October 1, 2026 > * **DOI:** [10.48550/arXiv.2610.01995](https://doi.org/10.48550/arXiv.2610.01995) --- ## 📄 论文摘要 > ## 📄 Abstract 现代 AI 系统正愈发频繁地基于机密数据生成关键结果——典型应用如根据私密医疗病历评估员工的健康适职状况，或是根据未公开的分子化学结构预测候选新药的理化性质。面对这类高敏感场景，如何在严守数据机密、杜绝底层泄密的前提下，令人信服地验证 AI 输出的正确性，已成为一个迫在眉睫的课题。近期的一系列学术探索尝试借助针对 oracle-aided computation (预言机辅助计算) 的 interactive proofs (交互式证明) 与 debate (辩论机制) 来校验 AI 输出；在这些体系中，计算的正确性往往依赖于外部 oracle (例如人类主观判断、物理化学实验或万维网查询) 。以往研究的核心诉求通常是效率，即追求让 verifier (验证者) 以远快于原本计算的极高速度完成验证。然而在理论上，通用的 oracle-aided computation 根本无法实现这种超高效校验，导致现有成果不得不依赖严苛的额外假设。本文则另辟蹊径，将研究焦点全面转向**隐私保护**：在允许 verifier 以计算规模的多项式时间正常运行的前提下，我们提出一个根本性问题——面向 oracle-aided computation 的 interactive arguments (交互式论证) 能否达成 **zero knowledge** ，从而确保 verifier 除了知晓“结论确实正确”之外，对底层的任何机密数据都毫无所悉？ > AI systems increasingly produce outputs from confidential data, such as a fitness-for-duty assessment from medical records or the predicted properties of a drug candidate from its secret structure. It is important to verify that such outputs are correct without revealing the underlying data. A recent line of work studies verification of AI outputs via interactive proofs and debate for oracle-aided computation, where correctness may depend on an oracle such as human judgment, a physical experiment, or the web. These works focus on verification by a verifier that runs much faster than the computation. However, such efficient verification is impossible for general oracle-aided computation, and these works therefore rely on additional assumptions. We focus instead on privacy: allowing the verifier to run in time polynomial in the computation, we ask whether interactive arguments for oracle-aided computation can be zero knowledge, so that the verifier learns nothing about the confidential data beyond the correctness of the output. 我们的数学证明给出了一个确凿的结论：**在一般情况下，这根本无法做到**。在密码学经典的 random oracle model 中，通用性的 zero-knowledge proofs 无法涵盖所有的 oracle-aided computations ；即便允许 prover 与 verifier 的运行耗时远远超过计算本身 (哪怕算力无限) ，这一不可行性依然牢固成立。更引人注目的是，这项理论“不可能禁令”还直接延伸到了当前被学术界视作可扩展监督 (scalable oversight) 经典范式的 debate 机制。 > We prove that, in general, they cannot. In the random oracle model, there are no zero-knowledge proofs for all oracle-aided computations, even if both the prover and the verifier are allowed to run much longer than the computation itself. The impossibility extends to debate, a canonical model for scalable oversight. 令人振奋的是，研究团队在揭示局限的同时也提出了一条极富建设性的破局路径：只要让外部 oracle 在返回每个应答时附带密码学签名 (cryptographic signature) ，那么在仅依赖基础的 collision-resistant hash functions 的最低密码学假设下，任意 oracle-aided computation 均能由多项式时间内高效运行的 prover 与 verifier 实现完备的 zero knowledge 验证。除了强有力的隐私保障外，这项设计还为 AI 的可扩展监督提供了一种全新的替代范式：它既不需要像 debate 模型那样假设博弈中必定存在一个“诚实的对手”，也无需像以往的单证明者协议 (single-prover protocols) 那样苛求计算过程本身具备鲁棒性，从而在机制设计上展现出了卓越的稳健性。 > On the positive side, we show that if the oracle attaches a cryptographic signature to each of its answers, then every oracle-aided computation can be verified in zero knowledge with an efficient prover and verifier, assuming only collision-resistant hash functions. Beyond privacy, this also gives an alternative approach to scalable oversight that relies neither on an honest opponent, as in debate, nor on the robustness of the computation, as in prior single-prover protocols. --- ## 🔗 全文与相关资源 > ## 🔗 Full-Text & Resources * **[查看 PDF (View PDF)](https://arxiv.org/pdf/2610.01995)** * **[HTML 版本 (Experimental)](https://arxiv.org/html/2610.01995v1)** * **[TeX 源码 (TeX Source)](https://arxiv.org/src/2610.01995)** * **授权许可：** [知识共享署名 4.0 国际许可协议 (Creative Commons Attribution 4.0 International)](http://creativecommons.org/licenses/by/4.0/)  > * **[View PDF](https://arxiv.org/pdf/2610.01995)** > * **[HTML Version (Experimental)](https://arxiv.org/html/2610.01995v1)** > * **[TeX Source](https://arxiv.org/src/2610.01995)** > * **License:** [Creative Commons Attribution 4.0 International](http://creativecommons.org/licenses/by/4.0/)  https://insight.aitobox.com/blog/posts/2026-10-02/软件遗产标识符-SWHID-构建通用开源代码档案的内在指纹与持久化生态/ https://insight.aitobox.com/blog/posts/2026-10-02/软件遗产标识符-SWHID-构建通用开源代码档案的内在指纹与持久化生态/ Fri, 02 Oct 2026 00:00:00 GMT 开源代码作为现代数字社会的基石，长期面临着代码仓库失效、平台关闭导致链接 404 以及依赖版本难以永久追溯等持久化痛点。由法国国家信息与自动化研究所 (Inria) 主导的 Software Heritage 项目通过去重的 Merkle DAG 结构，构建了囊括全球数百亿源文件的通用代码档案馆，并推出了基于内容内在哈希的国际标准 SWHID (SoftWare Hash IDentifier) 。本文深入探讨了开源元数据平台 ecosyste.ms 与 Software Heritage 的深度整合实践，作者开发了包括 `swh-git`、多语言 SWHID 生成库及 WebAssembly 客户端在内的完整开源工具链，将人类可读的软件包标识符与防篡改的内在代码指纹连接起来。这一生态不仅系统性量化评估并补充了核心开源软件的归档覆盖率，还为科研软件追踪与软件物料清单 (SBOM) 的长期可重现性奠定了坚实的基础设施。 --- ## 内容概要 > ## Summary 本文探讨了如何将全球开源代码归档库 **Software Heritage (SWH)** 与软件哈希标识符 **SoftWare Hash IDentifiers (SWHIDs)** 深度整合到 [ecosyste.ms](https://ecosyste.ms) 开源元数据平台中。受到巴黎 CodeCommons 全会上一场技术演讲的启发，作者详细阐述了打造一系列配套开源工具的全过程——其中包括专为 Git 定制的归档传输助手 `swh-git`、面向 Ruby 与 Go 的 SWHID 原生生成库、基于 WebAssembly 的浏览器端客户端，以及 OpenAPI 规范接口封装。这一系列工具旨在打通软件包身份标识与底层代码指纹，显著提升关键开源包的归档完整度，进而为软件生态的长久可重现性保驾护航。 > This article explores the integration of **Software Heritage (SWH)** archives and **Software Hash IDentifiers (SWHIDs)** into the [ecosyste.ms](https://ecosyste.ms) platform. Inspired by a presentation at the CodeCommons plenary in Paris, the author details the creation of various open-source tools—including `swh-git`, language-specific SWHID generators for Ruby and Go, a WebAssembly client, and an OpenAPI wrapper—to connect package identities, improve archiving coverage, and enable long-term software reproducibility. --- ## 目录 > ## Table of Contents - [Software Heritage 开源代码档案馆](#software-heritage-开源代码档案馆) - [SWHID 软件哈希标识符](#swhid-软件哈希标识符) - [swh-git 传输层助手](#swh-git-传输层助手) - [计算 SWHID 指纹](#计算-swhid-指纹) - [Web API 接口封装](#web-api-接口封装) - [关键软件包的归档覆盖率](#关键软件包的归档覆盖率) - [science.ecosyste.ms 科学计算软件追踪](#scienceecosystems-科学计算软件追踪) - [packages.ecosyste.ms 软件包版本全生命周期指纹](#packagesecosystems-软件包版本全生命周期指纹) - [扫描与深度分析代码档案库](#扫描与深度分析代码档案库) > - [Software Heritage](#software-heritage) > - [SWHIDs](#swhids) > - [swh-git](#swh-git) > - [Computing SWHIDs](#computing-swhids) > - [Web API](#web-api) > - [Critical Package Coverage](#critical-package-coverage) > - [science.ecosyste.ms](#scienceecosystems) > - [packages.ecosyste.ms](#packagesecosystems) > - [Scanning the Archive](#scanning-the-archive) --- ## Software Heritage 开源代码档案馆 > ## Software Heritage 自法国国家信息与自动化研究所 (Inria) 于 2016 年发起该项目以来，[Software Heritage](https://www.softwareheritage.org/) 一直致力于为人类全部公开源代码构建一座通用“数字代码档案馆”。它的爬虫常年穿梭于 GitHub、GitLab、Bitbucket、各大中小型代码托管平台、软件包中心以及各大 Linux 发行版源码库，将采集到的所有代码统一沉淀在一个全局去重的 Merkle DAG 结构之中。截至目前，这张庞大的代码知识图谱已汇聚了来自 4.45 亿个代码源头的约 300 亿个源码文件、230 亿个目录以及 60 亿次代码提交记录。 > [Software Heritage](https://www.softwareheritage.org/) has been building a universal archive of source code since Inria launched the project in 2016. It crawls GitHub, GitLab, Bitbucket, smaller forges, package registries, and distro sources, storing everything in a single deduplicated Merkle DAG. Currently, this graph holds roughly 30 billion source files, 23 billion directories, and 6 billion commits across 445 million origins. 即使同一个文件出现在成百上千个不同的地方 (例如某个文件既存在于 npm 的 tarball 压缩包中，又被打包进 Debian 源码包) ，Software Heritage 也只会在底层存储一次，直接通过其二进制字节哈希进行内容寻址，而图谱系统则负责记录该文件曾出现过的每一个代码源头 (Origin) 。其内部的 [Vault](https://docs.softwareheritage.org/devel/swh-web/uri-scheme-api-vault.html) 服务支持按需将任意历史快照、提交记录或目录重新组装为可直接下载的 git bundle 或 tarball 归档，从而确保代码检索完全立足于内容本身，彻底摆脱对原始代码托管服务商与 URL 链接的脆弱依赖。 > Files appearing in multiple places (such as an npm tarball and a Debian source package) are stored only once, addressed by their byte hash, while the graph tracks every origin. The [Vault](https://docs.softwareheritage.org/devel/swh-web/uri-scheme-api-vault.html) can reconstruct any snapshot, commit, or directory on demand as a downloadable git bundle or tarball, ensuring content-based retrieval rather than reliance on original hosting URLs. ## SWHID 软件哈希标识符 > ## SWHIDs 档案库中的每一个对象都被赋予了一个结构形如 `swh:1:<type>:<sha1>` 的 [SoftWare Hash IDentifier](https://www.swhid.org/) (SWHID) 。 > Every object in the archive is assigned a [SoftWare Hash IDentifier](https://www.swhid.org/) structured as `swh:1:<type>:<sha1>`. 其五大核心对象类型包括： > The five core types include: * `cnt` (文件内容字节) * `dir` (目录树结构) * `rev` (代码提交) * `rel` (版本标签) * `snp` (仓库快照) > * `cnt` (file bytes) > * `dir` (directory tree) > * `rev` (commit) > * `rel` (tag) > * `snp` (repository snapshot) 用户还可以为 SWHID 添加可选的限定修饰符 (Qualifiers) ，用以锚定来源 URL、爬虫访问记录 (Visits) 、目录内部的相对路径乃至具体的代码行号区间。该规范的 1.2 版本已被正式发布为国际标准 ISO/IEC 18670，标志着全球首个针对软件内在标识符的国际标准正式确立。 > Optional qualifiers can pin origin URLs, visits, paths inside directories, or line ranges. Version 1.2 of the specification was published as ISO/IEC 18670, establishing the first international standard for intrinsic software identifiers. SWHID 具备所谓“内在性 (Intrinsic)”——它完全由底层数据字节直接计算生成，不需要任何中心化的注册表或发证权威背书。由于这些哈希的计算规则与 Git 的对象哈希机制天然对齐，开发者本地任何一份 Git 检出仓库中，实际上都已经包含了完全合规的 SWHID。与之相对，形如 `pkg:pypi/astropy@7.1.0` 的软件包 URL (Package URL, purl) 虽然直观且人类可读，但它们的解析严重依赖于中心化注册中心的持续在线与可用。将这两大标识符体系无缝桥接起来，正是 ecosyste.ms 平台致力于实现的核心目标。 > SWHIDs are intrinsic—computed directly from bytes without a registry or issuing authority. Because these hashes align with Git's object hashes, any local Git checkout already contains valid SWHIDs. While package URLs (purls) like `pkg:pypi/astropy@7.1.0` are human-readable, resolving them depends on registries remaining active. Bridging these two identifier schemes forms a core focus of ecosyste.ms. ## swh-git 传输层助手 > ## swh-git 开源工具 [swh-git](https://github.com/andrew/swh-git) 作为 Git 远程助手 (Git Remote Helper) ，通过引入全新的 `swh::` 自定义传输层协议，允许开发者直接从归档库中检出并还原历史代码： > The [swh-git](https://github.com/andrew/swh-git) git remote helper adds an `swh::` transport layer to retrieve archived code: ```bash git clone swh::github.com/octocat/Hello-World git clone swh::swh:1:snp:f72e9d06dd0e58236a34328f094ca894a809f230 ``` 该助手会在后台向 Software Heritage 的 Vault 申请一份 git-bare bundle 归档包，将其缓存在本地磁盘，随后通过 Git 原生的 `git upload-pack` 协议无缝完成仓库克隆。对于那些在 GitHub 上早已显示 404、或者原始代码托管站点已经关门大吉的失效仓库而言，这提供了一道至关重要的代码灾难恢复生命线，不仅完整保留了提交历史与标签，还全面支持目录级的 SWHID 检索。 > The helper requests a git-bare bundle from the Vault, caches it locally, and serves clones via `git upload-pack`. It provides a vital recovery mechanism for 404ing repositories or defunct forges, supporting full commit histories, tags, and directory SWHIDs. ## 计算 SWHID 指纹 > ## Computing SWHIDs 为了在不同编程语言生态中原生计算 SWHID，作者开发了一套跨语言的工具链： > To compute SWHIDs natively across different languages, the author developed: * [swhid gem](https://github.com/andrew/swhid) (Ruby) ：全面支持五种核心对象类型的解析、限定修饰符处理与指纹生成。 * [swhid-go](https://github.com/andrew/swhid-go) (Go) ：提供便捷的命令行工具 (CLI) ，并内置了针对 SHA-1 哈希碰撞的防御性检测机制。 * [swhid-wasm](https://github.com/andrew/swhid-wasm)：将 `swhid-go` 编译为 WebAssembly 模块，支持直接在前端浏览器中对软件包注册表中的包或本地归档文件进行客户端哈希计算。 > * [swhid gem](https://github.com/andrew/swhid) (Ruby): Handles parsing, qualifiers, and generation for all five object types. > * [swhid-go](https://github.com/andrew/swhid-go) (Go): Includes a CLI and built-in SHA-1 collision detection. > * [swhid-wasm](https://github.com/andrew/swhid-wasm): A WebAssembly compilation of `swhid-go` enabling client-side hashing in the browser from package registries or local archives. Ruby 和 Go 的这套实现均已接入上游官方的 [一致性测试套件](https://github.com/swhid/test-suite) (conformance test suite) ，确保生成的标识符严格符合国际规范。 > Both the Ruby and Go implementations integrate with the upstream [conformance test suite](https://github.com/swhid/test-suite). ## Web API 接口封装 > ## Web API 为了高效调用 Software Heritage 的开放接口 (`archive.softwareheritage.org/api/1`) ，作者开发了 [swh-openapi](https://github.com/andrew/swh-openapi) 工具，能够从官方文档字符串 (docstring) 中自动提取并生成标准 OpenAPI 接口描述文件；同时还配套推出了 Go 语言客户端 SDK [swh-go](https://github.com/andrew/swh-go) ，大幅降低了第三方开发者接入归档生态的门槛。 > To interact with the Software Heritage API (`archive.softwareheritage.org/api/1`), the author created [swh-openapi](https://github.com/andrew/swh-openapi) to generate OpenAPI descriptions from docstring data, along with [swh-go](https://github.com/andrew/swh-go) as a companion client. ## 关键软件包的归档覆盖率 > ## Critical Package Coverage 借助 [swh-critical](https://github.com/andrew/swh-critical) 审计工具，作者对 [packages.ecosyste.ms 关键软件包数据集](https://packages.ecosyste.ms/critical) 中的约 9,800 个核心开源包进行了系统性摸底评估： > An audit using [swh-critical](https://github.com/andrew/swh-critical) evaluated ~9,800 packages in the [packages.ecosyste.ms critical set](https://packages.ecosyste.ms/critical): * **71.3%** 的源码仓库已将其当前默认分支的最新提交纳入归档； * **89.9%** 的匹配版本发布标签 (Release Tags) 均在档案库中完整就绪。 > * **71.3%** of source repositories had their current default-branch commit archived. > * **89.9%** of matched release tags were present. 通过将这些处于高依赖核心位置的软件包任务队列，主动推送给 Software Heritage 的代码即时保存服务 [Save Code Now](https://archive.softwareheritage.org/save/) ，ecosyste.ms 能够有效引导爬虫调度算法，将宝贵的采集与算力资源优先倾斜给那些全行业重度依赖的基础源码项目。 > By feeding high-dependency package queues into Software Heritage's [Save Code Now](https://archive.softwareheritage.org/save/) service, ecosyste.ms can help prioritize crawler resources toward heavily relied-upon source code. ## science.ecosyste.ms 科学计算软件追踪 > ## science.ecosyste.ms [science.ecosyste.ms](https://science.ecosyste.ms) 平台正将这一归档闭环率先应用于科研软件生态。系统会自动拉取学术影响力较高的高分科研项目，自动执行仓库克隆并在本地生成 SWHID 指纹，随后批量调用 Software Heritage 的已知对象接口 (known-objects API) 进行同步校验。一旦验证归档成功，对应的 SWHID 元数据便会与学术数字对象唯一标识符 (DOI) 和学术引用信息并肩展示在项目页面上，同时通过开放的 [统计数据 JSON API](https://science.ecosyste.ms/swhids) 供学界直接调用。 > The [science.ecosyste.ms](https://science.ecosyste.ms) platform applies this feedback loop to research software. High-score projects undergo automated cloning, SWHID generation, and batch-posting to Software Heritage's known-objects API. Confirmed contributions and SWHID metadata are displayed alongside DOIs and citations on project pages and via a [stats page JSON API](https://science.ecosyste.ms/swhids). ## packages.ecosyste.ms 软件包版本全生命周期指纹 > ## packages.ecosyste.ms 未来，[packages.ecosyste.ms](https://packages.ecosyste.ms) 平台的集成计划将进一步深入到具体的“软件包版本”层面——直接为官方发布的每个 tarball 压缩包计算对应的目录级 SWHID。这意味着在软件物料清单 (Software Bill of Materials, SBOM) 中声明的每一个依赖组件 (例如 `pkg:npm/lodash@4.17.21`) ，未来都能精确解析到一个可独立查验、防篡改的 SWHID 权威指纹，从而为各大主流软件包注册中心建立起一套持续滚动的归档覆盖率度量体系。 > Future integration on [packages.ecosyste.ms](https://packages.ecosyste.ms) will extend this coverage to package versions by computing directory SWHIDs for published tarballs. This allows SBOM components (e.g., `pkg:npm/lodash@4.17.21`) to resolve to independently verifiable SWHIDs, establishing a rolling measurement of registry coverage. ## 扫描与深度分析代码档案库 > ## Scanning the Archive 借助开源的 [git-pkgs](https://github.com/git-pkgs) 工具套件——涵盖用于清单提取的 [manifests](https://github.com/git-pkgs/manifests)、开源协议分析的 [licenses](https://github.com/git-pkgs/licenses) 以及信息速览工具 [brief](https://github.com/git-pkgs/brief)——开发者能够将整个 Software Heritage 档案库当作一座容量惊人的代码分析语料库。当它与 `swh-git` 强强联手时，无论跨越多少历史版本迭代，针对项目依赖链、合规许可证乃至构建工具链的深度分析，都能实现完全可重现与可验证。 > Using the [git-pkgs](https://github.com/git-pkgs) suite—including [manifests](https://github.com/git-pkgs/manifests), [licenses](https://github.com/git-pkgs/licenses), and [brief](https://github.com/git-pkgs/brief)—developers can treat the Software Heritage archive as a massive analysis corpus. Combined with `swh-git`, analyses of dependencies, licenses, and toolchains become fully reproducible across historical software versions. https://insight.aitobox.com/blog/posts/2026-10-02/自我进化的算法设计智能体-基于种群筛选策略优化打破上下文进化停滞/ https://insight.aitobox.com/blog/posts/2026-10-02/自我进化的算法设计智能体-基于种群筛选策略优化打破上下文进化停滞/ Fri, 02 Oct 2026 00:00:00 GMT 在大语言模型 (Large Language Model, LLM) 驱动的自主算法设计中，现有的纯上下文进化方案在面对芯片设计、底层算子等高门槛专业领域时极易遭遇“进化停滞”，而传统的参数微调方法又受困于高质量专业算法语料的极度匮乏。为此，南京大学与华为等机构的研究团队提出了全新的参数自进化框架——种群筛选策略优化 (Population-Curated Policy Optimization, PCPO) ，在理论上证明了通过渐进学习自生成算法能够打破停滞的“改进链 (Improvement Chain) ”命题，并借助全局种群与混合策略更新机制，让智能体直接在自生成的高质量多样化算法中持续迭代进化。在电子设计自动化 (Electronic Design Automation, EDA) 的芯片全局布局学习率调度任务中，基于 8B 参数开源底座的 PCPO 仅需 4 个芯片案例的轻量训练，就在全部 16 个测试案例上全面超越了 OpenEvolve 和 ShinkaEvolve 等前沿上下文进化基准，性能足以比肩顶尖闭源模型 GPT-5.5，并显著降低了推理阶段的 Token 开销。此外，PCPO 在 4 款 GPU 底层算子设计任务中相比 PyTorch Eager 基线实现了平均 8.27 倍的显著加速，为解决复杂工业与高性能计算领域的算法自动化探索开辟了全新路径。 --- ## 📋 内容概要 > ## 📋 Summary 如今，大语言模型 (LLM) 越来越多地以自主智能体的形态参与到复杂现实任务中，承担起算法设计与持续优化的重任。虽然基于**上下文进化框架** (In-Context Evolutionary Frameworks) 的探索方法颇受青睐，但在需要高度专业领域知识的复杂场景下，这类方法往往很快就会陷入能力瓶颈与进化停滞；而传统的**参数化适配** (Parametric Adaptation) 方法若想内化专业知识，又严重依赖海量的特定领域语料，这在高级算法设计这一高精尖场景中无疑是极其匮乏的。 > Large Language Models (LLMs) are increasingly utilized as autonomous agents to design and refine algorithms for complex real-world tasks. While **in-context evolutionary frameworks** are popular, they often plateau quickly in domains requiring specialized knowledge. Conversely, traditional **parametric adaptation** requires large datasets of domain-specific corpora, which are extremely scarce for advanced algorithm-design scenarios. 为了弥合这一鸿沟，本文提出了**种群筛选策略优化** (Population-Curated Policy Optimization, PCPO) ——这是一种高样本效率的参数自进化框架，允许智能体直接从自身探索生成的算法中进行反思与学习。 > To bridge this gap, this paper introduces **Population-Curated Policy Optimization (PCPO)**—a sample-efficient parametric self-evolution framework where agents explore and learn directly from self-generated algorithms. https://insight.aitobox.com/blog/posts/2026-10-02/突破算力天花板-Galahad-字节级精准内存机制让大模型文档读取变成一次性开销/ https://insight.aitobox.com/blog/posts/2026-10-02/突破算力天花板-Galahad-字节级精准内存机制让大模型文档读取变成一次性开销/ Fri, 02 Oct 2026 00:00:00 GMT 长期以来，大语言模型 (Large Language Model, LLM) 的在线推理服务普遍采用“无状态”架构，即针对同一份长文档的每一次用户提问，模型都必须从头重新计算完整的注意力状态，导致算力与能耗被极其严重地重复浪费。为攻克这一算力天花板瓶颈，研究者提出了名为 **Galahad** 的新型内存层，专为 vLLM、SGLang 和 llama.cpp 等主流推理框架量身设计。Galahad 由两大核心组件构成：**Taliesin** 能够对完全相同的字节级文本块实现键值缓存 (KV Cache) 状态的持久化存储与瞬时重载，真正将文档读取转变为“一次性开销”；而 **Blaise** 则负责直接维护文档并按需向模型精准投喂问题相关的特定片段。在包含 9.7 万 Token 的高难度语料测试中，配合 Gemma 4 31B 模型，Galahad 将问答准确率从基线的 10% 提升至 100% 满分，推理延迟降至 0.59–0.64 秒，且保存的注意力状态实现了 100% 比特级无损匹配。这项研究成果成功推动了大模型服务从低效的“无状态”计算迈向真正高效的“状态化推理” (Stateful Inference) 新范式。 --- ## 执行摘要 > ## Executive Summary 传统上，大语言模型 (Large Language Model, LLM) 的推理服务在各个请求之间是完全“无状态”的。这意味着每当收到针对同一份文档的新提问时，模型都必须从第一个 Token 开始重新计算整篇文档的注意力状态——这无疑在底层硬件上白白浪费了极其庞大的算力与电力。为了突破这一算力天花板的严苛限制，本文隆重推出了 **Galahad**——一套专为 vLLM、SGLang 以及 llama.cpp 等主流开源推理框架量身打造的高效内存层。 > Large Language Model (LLM) serving is traditionally stateless, meaning that a model recomputes a document's attention state from the first token upon every new request—wasting massive amounts of compute and energy. To address this compute ceiling constraint, this paper introduces **Galahad**, a memory layer designed for frameworks like vLLM, SGLang, and llama.cpp. Galahad 巧妙借助两大核心组件，成功将大模型的文档读取转变为“一次性开销”： > Galahad transforms LLM reading into a one-time cost by leveraging two main components: * **Taliesin：** 能够针对字节完全一致的文本块，自动持久化保存并极速重新加载模型的键值 (Key-Value, KV) 状态，彻底告别重复计算。 * **Blaise：** 直接在底层接管长文档管理，每次仅向模型精准传递回答当前问题所必需的特定章节片段。 > * **Taliesin:** Saves and reloads a model's key-value (KV) state for blocks of text matching identical bytes. > * **Blaise:** Manages documents directly, passing only the specific sections required by a question. https://insight.aitobox.com/blog/posts/2026-10-02/当推理陷入迷途-失控推理的注意力动态演化与实时干预/ https://insight.aitobox.com/blog/posts/2026-10-02/当推理陷入迷途-失控推理的注意力动态演化与实时干预/ Fri, 02 Oct 2026 00:00:00 GMT 随着大推理模型 (Large Reasoning Models, LRMs) 的兴起，依靠延长思维链进行深度思考已成为攻克高难度数学与逻辑难题的核心范式，但这种漫长的思维过程极易误入歧途，蜕变为无休止的冗余验证与无限生成死循环，导致推理开销激增与算力资源枯竭。针对现有多依赖生硬截断长文本或浅层重复检测等被动防御手段的局限，研究团队提出了创新的 RADAR (Reasoning-state Analysis via Dynamic Attention Responses) 分析框架。该框架首次将大模型的推理生成过程形式化解构为四大离散状态，不仅能在推理运行时实时判定思维状态，更从底层注意力机制上清晰刻画了良性反思如何一步步恶化为失控生成。在此基础上，研究进一步提出了无需二次微调训练的 Attention Realignment 实时干预机制，在表层文本出现显性重复之前便将异常注意力分布导向回正常基准，在几乎无损模型原生解题能力的前提下成功消除了死循环，为保障大推理模型的高效稳定运行开辟了全新路径。 --- ## 概述与核心摘要 > ## Overview & Summary 本文深入研究了 **Large Reasoning Models (LRMs)** 的失效模式。这类大推理模型往往借助更长推理步骤的“深度思考”在复杂任务中斩获优异表现，但这种漫长的思维延伸过程经常会退化为冗余的重复验证循环与无限生成状态。这些失控的推理行为不仅会成倍推高推理成本，带来计算资源枯竭的风险，还会严重拉低线上服务的响应质量。 > This paper investigates the failure modes of **Large Reasoning Models (LRMs)**, which often leverage extended reasoning to excel at complex tasks. However, this prolonged thinking process can frequently degenerate into redundant verification loops and persistent generation cycles. These uncontrolled behaviors drive up inference costs, risk resource exhaustion, and degrade service quality. 为了打破现有被动应对策略 (例如直接暴力截断输出或仅针对字面词句做浅层重复检测) 的局限，作者提出了 **RADAR** (**R**easoning-state **A**nalysis via **D**ynamic **A**ttention **R**esponses) 框架。这一创新框架带来了以下关键突破： * **四状态形式化解构**：将 LRM 的生成过程细分为**四种界限分明的思维状态**； * **实时状态判定**：在模型推理过程中**实时识别**当前所处的推理状态； * **机理解析与演化映射**：从注意力机理上剖析有益的自我反思是如何逐步蜕化为失控生成的； * **引入 Attention Realignment**：提出了一种精准的纠偏干预策略，将偏离常态的异常注意力分布拉回健康基准模式——在几乎不损耗模型任务求解能力的前提下，成功阻断陷入死循环。 > To address the limitations of existing reactive mitigations (such as output truncation or superficial repetition detection), the authors introduce **RADAR** (**R**easoning-state **A**nalysis via **D**ynamic **A**ttention **R**esponses). This novel framework: > * Operationalizes LRM generation into **four distinct states**. > * Identifies current reasoning states **in real-time**. > * Mechanistically maps how healthy reflection devolves into uncontrolled generation. > * Introduces **Attention Realignment**, a corrective intervention that shifts abnormal attention distributions back toward healthy patterns—successfully reducing looping while preserving task performance. --- ## 论文元数据与发表详情 > ## Metadata & Publication Details * **arXiv 编号：** [arXiv:2609.38817](https://arxiv.org/abs/2609.38817) [cs.AI] * **研究领域：** 人工智能 (`cs.AI`) ；计算与语言 (`cs.CL`) * **提交时间：** 2026 年 9 月 30 日 * **数字对象唯一标识符 (DOI) ：** [10.48550/arXiv.2609.38817](https://doi.org/10.48550/arXiv.2609.38817) * **知识共享许可：** [Creative Commons Attribution 4.0](http://creativecommons.org/licenses/by/4.0/)  > * **arXiv ID:** [arXiv:2609.38817](https://arxiv.org/abs/2609.38817) [cs.AI] > * **Subjects:** Artificial Intelligence (`cs.AI`); Computation and Language (`cs.CL`) > * **Submitted on:** September 30, 2026 > * **DOI:** [10.48550/arXiv.2609.38817](https://doi.org/10.48550/arXiv.2609.38817) > * **License:** [Creative Commons Attribution 4.0](http://creativecommons.org/licenses/by/4.0/)  https://insight.aitobox.com/blog/posts/2026-10-02/Perplexity-发布-pplx-embed-v2-context-9b-preview-兼顾直接答案与支撑证据的上下文向量检索模型/ https://insight.aitobox.com/blog/posts/2026-10-02/Perplexity-发布-pplx-embed-v2-context-9b-preview-兼顾直接答案与支撑证据的上下文向量检索模型/ Fri, 02 Oct 2026 00:00:00 GMT 在传统的检索增强生成 (Retrieval-Augmented Generation, RAG) 系统中，长文档通常被粗暴地切分成孤立的文本块，检索模型往往只能押注单一的“黄金段落”，极易丢失周围至关重要的上下文信息。针对这一行业痛点，Perplexity 携手 turbopuffer 正式发布了开源上下文向量模型 **`pplx-embed-v2-context-9b-preview`**。该模型采用创新的“查询感知上下文压缩教师”训练机制，打破了非黑即白的单段落标注限制，使模型在单次向量检索中就能同时捕获直接答案与支撑证据。在全新基准测试 `context-bench` 与 `ConTEB` 上，该模型表现刷新行业上限，不仅在证据召回率上显著超越同类顶尖商业模型，还原生支持 int8 量化与 Matryoshka 灵活降维，在大幅缩减向量存储成本的同时保持极致性能。 --- ## 核心概要 > ## Summary Perplexity 研究团队联合 turbopuffer 正式发布了 **`pplx-embed-v2-context-9b-preview`**，这是一款专为检索增强生成 (Retrieval-Augmented Generation, RAG) 流程量身定制的先进上下文向量模型。与传统模型依赖割裂孤立的文本块或死守单一“黄金段落 (Gold Passage) ”不同，该模型能够结合整篇文档的全局语境来精准评估每个文本块。通过在训练阶段引入创新的“查询感知上下文压缩教师模型”，它学会了在一次检索中同时找回直接答案以及不可或缺的支撑证据。目前该模型已通过 Hugging Face 以开源 MIT 许可证发布，不仅在多项上下文检索基准测试中斩获顶尖成绩，还原生支持 int8 量化与 Matryoshka 降维技术，在性能突破的同时大幅降低了向量存储开销。 > Perplexity Research, in collaboration with turbopuffer, has released **`pplx-embed-v2-context-9b-preview`**, an advanced contextual embedding model designed specifically for Retrieval-Augmented Generation (RAG) pipelines. Unlike traditional models that rely on isolated chunks or a single "gold passage," this model evaluates chunks in the full context of the document. Utilizing a novel query-aware context compression teacher during training, the model learns to retrieve both the direct answer and the necessary supporting evidence simultaneously. Released under an open-source MIT license via Hugging Face, it achieves state-of-the-art performance on context retrieval benchmarks while optimizing storage with native int8 quantization and Matryoshka dimensions. --- ## 部署方式与获取渠道 > ## Deployment and Availability * **获取方式：** 采用自托管预览形式，模型权重已按 MIT 开源许可证在 [Hugging Face](https://huggingface.co/perplexity-ai/pplx-embed-v2-context-9b-preview) 正式开放。 * **环境要求：** 加载运行需要安装 `transformers>=5.4.0`，并开启 `trust_remote_code=True`。 * **API 状态：** 暂未上线 Perplexity 官方 API。 * **特别说明：** 官方模型卡片提示，后续模型权重与接口可能会在不保证向后兼容的前提下进行更新调整。 > * **Access Type:** Self-hosted preview with open weights available on [Hugging Face](https://huggingface.co/perplexity-ai/pplx-embed-v2-context-9b-preview) under the MIT license. > * **Requirements:** Loading requires `transformers>=5.4.0` with `trust_remote_code=True`. > * **API Status:** Not yet available on the Perplexity API. > * **Note:** The model card warns that weights and interfaces may change without backward compatibility. --- ## 为什么传统的“黄金段落”方案难以满足需求 > ## Why the Traditional "Gold Passage" Approach Falls Short 常规的 RAG 系统在处理长文档时，习惯将长文粗暴地切分成互不相干的独立分块。然而在实际阅读中，单个分块往往严重依赖文档其他位置出现的外部上下文——例如某个人名实体、所属章节标题或关键术语定义。为了化解这一困境，上下文感知模型引入了**晚期分块 (late chunking) **技术：先将整篇完整文档进行单次编码，随后再针对每个文本块执行池化处理。 > Standard RAG systems split long documents into independent chunks. However, individual chunks frequently rely on external context—such as an entity, heading, or definition—stated elsewhere in the document. Contextual models combat this using **late chunking**, where the entire document is encoded in a single pass before being pooled per chunk. 然而，传统的训练范式通常存在以下几大硬伤： > Traditional training paradigms typically suffer from several limitations: 1. **非黑即白的二元局限：** 现有数据集针对每个查询通常仅标注唯一的“黄金文本块”，其他所有文本块 (即便包含至关重要的上下文背景句) 都会被武断地视为负样本。 2. **标注成本居高不下：** 依赖大语言模型 (Large Language Model, LLM) 进行数据标注的成本随数据集规模呈线性增长。 3. **分块策略僵化死板：** 标签往往与某种单一且预先固定的分块切分规则强绑定，缺乏泛化适应能力。 > 1. **Binary Limitations:** Datasets usually mark only one "gold chunk" per query, treating all other chunks (including vital contextual sentences) as negatives. > 2. **Annotation Costs:** LLM annotation costs scale linearly with dataset size. > 3. **Rigid Labels:** Labels are strictly tied to a single, predetermined chunking strategy. --- ## 训练机制深度剖析 > ## How the Training Works 该模型的训练架构摒弃了生硬的二元标签，转而采用一个**查询感知上下文压缩模型**充当教师模型 (Teacher) 。该教师模型会对用户的查询与整篇文档进行联合分析，并为其中的每一个 Token 进行打分： > The training architecture replaces hard binary labels with a **query-aware context compression model** acting as a teacher. The teacher analyzes the query and document jointly to score every token: * **文本块相关性：** 通过计算每个文本块内得分排名前 $n$ (top-$n$) 的 Token 平均分来确定相关度。 * **软目标概率分布：** 在正样本相关文档内的所有文本块上，应用带温度调节的 Softmax 概率分布 (同时对完全无关文档中的文本块直接赋零分) 。 * **蒸馏损失：** 通过计算教师模型与学生模型分布之间的前向 KL 散度 (Forward KL Divergence) 进行匹配拟合。 * **文档级损失：** 借鉴了 ColBERT 的 MaxSim 机制，引入 InfoNCE 损失函数，以文档中表现最佳的文本块得分作为整个文档的代表分。 > * **Chunk Relevance:** Determined by calculating the mean of the top-$n$ token scores inside each chunk. > * **Soft Target:** Uses a temperature-scaled softmax distribution over chunks within the positive document (while assigning zero to chunks in irrelevant documents). > * **Distillation Loss:** Calculated using forward KL divergence matching between the teacher and student distributions. > * **Document Loss:** Inspired by ColBERT’s MaxSim, utilizing an InfoNCE loss where a document's score is represented by its best-performing chunk. 在训练过程中，批次会动态随机采样多样化的分块切分策略。不同的文本块通过一个专门学习得到的 `<|chunk_sep|>` 特殊 Token 进行分隔，随后执行均值池化 (Mean-Pooling) 。由于教师模型仅在训练阶段发挥作用，因此在实际部署推理时完全不会带来额外的延迟或存储开销。 > During training, batches dynamically sample random chunking strategies. Chunks are separated by a learned `<|chunk_sep|>` token and mean-pooled. Because the teacher is only active during training, inference incurs zero added latency or storage overhead. 该模型的底层架构脱胎于团队自研的 9B 参数规模 ColBERT 检索模型，并通过线性投影层输出 2048 维向量表示；结合[套娃表征学习 (Matryoshka Training) ](https://arxiv.org/abs/2205.13147)，模型原生支持截断至 1024 维并搭配 int8 原生量化。最终发布的模型则是一份精心调配的模型融合套件 (Model Soup) ，集成了基于 50 多种语言、约 430 个数据集训练得出的多个检查点 (Checkpoints) 。 > The architecture originates from an in-house 9B ColBERT retrieval model using a linear projection to output 2048 dimensions, with [Matryoshka training](https://arxiv.org/abs/2205.13147) supporting 1024-dimensional truncation and native int8 quantization. The final release is a model soup combining multiple checkpoints trained across roughly 430 datasets in over 50 languages. --- ## 全新评估基准：`context-bench` > ## `context-bench`: A New Evaluation Benchmark 为了严防评测数据受到污染，[turbopuffer](https://turbopuffer.com/) 专门闭门打造并持有私有基准测试集 `context-bench`。该基准覆盖 21 个垂直领域，包含 38,894 篇长文档以及 2,099 个真实评测查询： > Developed and held privately by [turbopuffer](https://turbopuffer.com/) to prevent data contamination, `context-bench` evaluates models across 2,099 queries spanning 38,894 documents across 21 domains. * 按照句子级别分块后，共切分衍生出 **2,458,072 个独立文本块**。 * 目标文档的长度中位数约为 6,100 个 Token。 * 测试查询严谨考察了 12 种截然不同的上下文理解能力，从代词指代消歧一路涵盖到复杂的表格结构解析。 > * Sentence chunking generates **2,458,072 individual chunks**. > * The median target document length is roughly 6,100 tokens. > * Queries rigorously test 12 distinct contextual capabilities, ranging from pronoun resolution to complex table structures. 评测指标包含 **Document@K**、**Answer@K**、**Evidence Recall@K** 以及 **All-Evidence@K**，测试过程独立于特定的索引配置，直接针对全部文本块进行穷举评估。 > Metrics include **Document@K**, **Answer@K**, **Evidence Recall@K**, and **All-Evidence@K**, evaluated exhaustively against all chunks independently of index configurations. --- ## 评测战报与核心性能亮点 > ## Results & Performance Highlights * **`context-bench` 在 $K = 10$ 下的表现：** 答案召回率达到 45.5%，证据召回率达到 40.6%，全证据召回率达到 31.1%。 * **文档级召回率：** 在 $K = 1$ 时为 15.2%，在 $K = 10$ 时高达 61.6%。 * **竞品正面对决：** 在 $K = 10$ 的测试中，答案召回率超越 `voyage-context-4` 达 14.4 个百分点，证据召回率领先 5.0 个百分点。 * **[ConTEB](https://github.com/illuin-tech/contextual-embeddings) 基准表现：** 在所有参评模型中斩获最高的平均 nDCG@10 评分。 * **极致存储效率：** 1024 维 int8 配置 (单个向量仅占 1 KB 存储) 的性能表现甚至微弱超越了占用 2048 维 float32 (单向量 8 KB) 的 `voyage-context-4`。 * **对分块大小的鲁棒性：** 当切分文本块的大小从 64 个 Token 扩展到 512 个 Token 时，模型的平均得分仅从 81.0% 轻微波动至 79.9%，表现出极高的稳定性。 > * **`context-bench` at $K = 10$:** 45.5% answer recall, 40.6% evidence recall, and 31.1% all-evidence recall. > * **Document Recall:** 15.2% at $K = 1$ and 61.6% at $K = 10$. > * **Versus Competitors:** Outperforms `voyage-context-4` by 14.4 points on answer recall and 5.0 points on evidence recall at $K = 10$. > * **[ConTEB](https://github.com/illuin-tech/contextual-embeddings):** Achieves the highest average nDCG@10 among evaluated models. > * **Storage Efficiency:** A 1024-dim int8 configuration (1 KB per vector) slightly outperforms `voyage-context-4` at 2048-dim float32 (8 KB). > * **Chunk Size Robustness:** Average scores shift mildly from 81.0% to 79.9% as chunk sizes range from 64 to 512 tokens. --- ## 主流竞品横向技术对比 > ## Comparison with Closest Competitors | 特性对比 | `pplx-embed-v2-context-9b-preview` | [`voyage-context-4`](https://blog.voyageai.com/2026/06/29/voyage-context-4/) | [`pplx-embed-context-v1-4B`](https://huggingface.co/perplexity-ai/pplx-embed-context-v1-4b) | [`Nemotron-3-Embed-8B`](https://huggingface.co/nvidia/Nemotron-3-Embed-8B-BF16) | | :--- | :--- | :--- | :--- | :--- | | **分块向量表征** | 上下文感知 (Contextual) | 上下文感知 (Contextual) | 上下文感知 (Contextual) | 单独切块编码 (Independent) | | **获取方式** | 开源权重，遵循 MIT 许可证 | 托管 API (Voyage, MongoDB Atlas) | 开源权重，MIT 许可证；Perplexity API | 开源权重，遵循 OpenMDW-1.1 协议 | | **参数量** | 官方博文标注为 9B (Hugging Face 标注为 8B) | 未公开 (MoE 混合专家架构底座) | 4B | 约 8B | | **向量维度** | 2048, 1024 | 2048, 1024, 512, 256 | 2560 (支持 Matryoshka 降维) | 4096 (支持切片) | | **量化输出** | 原生 int8 | int8, uint8, binary, ubinary | int8, binary | Float (浮点) | | **上下文窗口** | 评测最高支持至 32,768 个 Token | 单次请求 32K；搭配自动切块可达 120K | 32K | 32,768 | | **自动分块 (Auto-chunking)** | 否 | 是 | 否 | 否 | | **价格资费** | 自托管 (免费开源) | 每百万 Token 收费 $0.12 (前 2 亿 Token 免费) | 自托管或通过 API 调用 | 自托管 (免费开源) | *(数据来源：[Voyage 官方文档](https://docs.voyageai.com/docs/contextualized-chunk-embeddings)，以及截至 2026 年 9 月 30 日上述各模型卡片信息)*。 > *(Sources: [Voyage docs](https://docs.voyageai.com/docs/contextualized-chunk-embeddings), linked model cards as of September 30, 2026).* --- ## 核心技术要点总结 > ## Key Takeaways * **Token 级教师机制的革新：** 成功摆脱了传统单一“黄金文本块”强标签的僵化约束。 * **答案与证据兼收并蓄：** 模型能够在统一的分块索引空间内，高效同步检索出直接答案与不可或缺的支撑证据。 * **基准测试性能卓越：** 在 `context-bench` 的 Answer@10 评测中达到 **45.5%** 的高召回率，相比同类竞品 `voyage-context-4` 大幅领先 14.4 个百分点。 * **开源可用与生态进展：** 基于 MIT 许可证的开源模型权重现已全量开放下载，Perplexity 官方 API 接口集成则将在后续推出。 > * A token-level teacher mechanism successfully replaces restrictive single gold-chunk labels. > * The model efficiently retrieves direct answers alongside necessary supporting evidence inside a unified chunk index. > * `context-bench` Answer@10 reaches **45.5%**, outperforming `voyage-context-4` by 14.4 points. > * Open MIT weights are available immediately, while official Perplexity API integration remains pending. --- *欢迎深入阅读官方[技术博客长文](https://www.perplexity.ai/hub/blog/contextual-embedding-beyond-the-gold-passage)探索完整技术实现细节，并前往获取[开源模型权重](https://huggingface.co/perplexity-ai/pplx-embed-v2-context-9b-preview)。* > *Explore the complete [technical details](https://www.perplexity.ai/hub/blog/contextual-embedding-beyond-the-gold-passage) and access the [model weights](https://huggingface.co/perplexity-ai/pplx-embed-v2-context-9b-preview).* https://insight.aitobox.com/blog/posts/2026-10-01/TLA+-到底能验证什么-又无法验证什么-从并发系统设计到-AI-编码的形式化边界/ https://insight.aitobox.com/blog/posts/2026-10-01/TLA+-到底能验证什么-又无法验证什么-从并发系统设计到-AI-编码的形式化边界/ Thu, 01 Oct 2026 00:00:00 GMT 近期，Claude Code 创造者 Boris Cherny 透露 Opus 成功运用形式化规范语言 TLA+ 发现了代码中的并发竞态条件，引发了技术社区对形式化验证与 AI 智能体编程结合的热烈探讨。著名 TLA+ 布道者与教育家 Hillel Wayne 撰文指出，尽管 TLA+ 在设计复杂并发系统、杜绝逻辑缺陷方面威力惊人，但它绝非能够一劳永逸解决 AI 软件开发难题的“银弹”。本文系统梳理了 TLA+ 原生支持验证的状态不变量与时序活性属性，深入剖析了它无法原生表达多步动作、可达性及超属性等关键盲区，并探讨了工程师采用的精巧变通方案与其局限。这一辨析清晰划定了形式化方法与自动化编码的能力边界，为理性看待 AI 辅助工程系统设计提供了极为清醒深刻的技术视角。 --- ## 核心概述 > ## Summary 最近，随着利用 AI 智能体 (AI Agent，例如 Claude Code 和 Opus) 借助 TLA+ 排查竞态条件 (Race Condition) 的演示引爆技术圈，形式化验证 (Formal Verification) 再次掀起了全网热议。诚然，TLA+ 是设计复杂并发系统 (Concurrent Systems) 、验证系统安全性 (Safety) 与活性 (Liveness) 属性时近乎无往不利的神兵利器，但它绝非万能的“银弹”。本文将深入探讨 TLA+ 原生能够表达与检验的逻辑边界、它**无法**直接表达的关键盲区 (例如可达性、超属性以及跨多步动作) ，并揭示工程师为了拓展其能力上限所采用的各种精妙 (却略显繁杂) 的变通技巧。 > Following recent excitement around using AI agents (like Claude Code and Opus) to find race conditions via TLA+, formal verification has sparked widespread internet discussion. While TLA+ is an exceptionally powerful tool for designing complex concurrent systems and verifying safety and liveness properties, it is not a silver bullet. This article explores what TLA+ can express and check natively, the notable limitations regarding what it *cannot* express (such as reachability, hyperproperties, and multi-step actions), and the clever (yet messy) workarounds engineers use to stretch its capabilities. --- ## 引言 > ## Introduction 上周，Claude Code 的创造者 Boris Cherny 透露，Opus 能够借助 TLA+ [^tla] 成功定位代码中的竞态条件。 > Last week Boris Cherny, the inventor of Claude Code, mentioned that Opus was able to use TLA+[^tla] to find race conditions in code. 一时间，整个互联网都在热烈探讨形式化验证。 > And now everybody on the internet is talking about formal verification. 作为一名长年深耕 TLA+ 教学 ( [1](https://link.springer.com/book/10.1007/978-1-4842-3829-5) [2](https://learntla.com/) ) 与积极 [倡导者](https://newsletter.pragmaticengineer.com/p/formal-methods-with-hillel-wayne) 的同行者，看到这种现象我确实感到振奋！TLA+ 在设计复杂并发系统并确保其免受逻辑 Bug 困扰方面表现极其出色。 [^antithesis] 但与此同时，作为一名始终主张理性思考的技术人，这种近乎盲目的狂热又令我深感担忧。我看到许多人断言：形式化方法 (Formal Methods) 将一劳永逸地彻底解决智能体驱动软件开发中的可靠性问题——而这完全是无稽之谈。 > As a long-time educator ([1](https://link.springer.com/book/10.1007/978-1-4842-3829-5) [2](https://learntla.com/)) and [advocate](https://newsletter.pragmaticengineer.com/p/formal-methods-with-hillel-wayne) of TLA+, this is really exciting! TLA+ is great at designing complex concurrent systems and making sure they're bug-free.[^antithesis] As a long-time advocate of level-headedness, this new euphoria worries me. I read a lot of people saying that formal methods will solve the problem of agentic software development once and for all, and that's nonsense. 关于 TLA+ 在实际保证能力上的局限性，业界已经有过汗牛充栋的讨论，例如 [正确的设计并不会自动转化为正确的代码实现](https://buttondown.com/hillelwayne/archive/what-if-the-spec-doesnt-match-the-code/) 。因此，在本次通讯中我想探讨一个截然不同的局限维度：要验证一个属性，前提是你必须能够把它描述出来！那么，到底有哪些属性是 TLA+ 根本连表达都无法表达的？ > Enough words have been spilled about the weaknesses of TLA+ in terms of what it can guarantee, like how [correct designs don't automatically translate into correct code](https://buttondown.com/hillelwayne/archive/what-if-the-spec-doesnt-match-the-code/). So I'd like to focus on a different limitation for this newsletter: to verify a property, we need to have a property to verify! So what are the properties that TLA+ can't even express? *(本文假设读者具备一些 TLA+ 的基础知识。如果你完全是初学者，建议先阅读上文提到的 [1] [2] 两处资料，或者 [阅读这篇入门文章](https://buttondown.com/hillelwayne/archive/tla-from-first-principles/) 。)* > *(This assumes some basic knowledge of TLA+. If you're a total beginner, check out those [1] [2] things above or [read here](https://buttondown.com/hillelwayne/archive/tla-from-first-principles/).)* --- ## TLA+ 能够验证什么 > ## What TLA+ Can Check TLA+ 将整个系统的运行过程划分为一组**行为** (Behavior) 。每个行为都是一条状态序列，好比“一号红绿灯是绿灯，接着变黄灯，最后变红灯”。在每个状态中，我们可以用常规的布尔表达式来描述现状，比如“四号灯为绿灯”或“所有灯都是红灯”。此外，我们还可以借助三个“时序”逻辑运算符来扩展表达式的表达能力： > TLA+ divides the system into a set of behaviors. Each behavior is a sequence of states, like "light one is green, then yellow, then red". In each state we can express a regular boolean expressions like "Light four is green" or "All lights are red." We can also modify expressions with three "temporal" logical operators: * `[]P` (即“始终为 P”) ：如果 P 在当前状态*以及后续每一个*未来状态中均成立，则该式为真。例如：`[](at_most_one_green)` 表示从当前开始的每一个状态中，绿灯的数量绝不会超过一盏。 * `P'` (即“P 的后继状态 / P prime”) ：如果 P 在紧邻的下一个状态中为真，则该式为真。例如：`light="green" && light'="red"` 表示信号灯从绿灯变为红灯。 * `<>P` (即“最终为 P / eventually P”) ：如果 P 在当前状态*或未来至少某一个*状态中为真，则该式为真。例如：`<>(light4 = "yellow")` 表示四号灯当前是黄灯，或者在未来的某个状态中必定会变为黄灯。 > * `[]P` ("always P") is true if P is true in the current state *and every* future state. Example: `[](at_most_one_green)` is true if every state going forward has no more than one green light. > * `P'` ("P prime") is true if P is true in the next state. Example: `light="green" && light'="red"` is true if the light changes from green to red. > * `<>P` ("eventually P") is true if P is true in the current state *or in at least one* future state. Example: `<>(light4 = "yellow")` is true if light4 is yellow or is yellow in a future state. 当我们说 P 是系统的一项属性时，意味着它在系统每条行为的初始状态下都为真。因此，如果我们检查属性 `[]P`，就代表 `[]P` 在每个初始状态中都成立；而根据“始终” (Always) 的逻辑定义，这意味着从该初始状态出发的每一个未来状态中 P 均成立，即 P 在所有行为的每一个状态中都为真。我们将此称为**不变量** (Invariant) ，它是我们在 TLA+ 中验证的最核心、最基础的属性之一。 > When we say that P is a property of the system, we mean it is true in the initial state of every behavior. So if we check the property `[]P`, that means that `[]P` is true in every initial state, and then by the definition of "always" means that P is true in every future state from that initial state, meaning it is true in every state of every behavior. We call this an **invariant**, and is one of the most foundational properties we check in TLA+. 我们还可以将 `[]` 与带撇的后继状态标记 `'` 结合，构建出**动作属性** (Action Properties) ，或者说状态跃迁的变式。例如 `[](x' >= x)` 表示 x 的新值始终大于或等于其旧值。另一个有趣的例子是 `[](P => P')`：一旦 P 为真，它就永远不可能再次变为假。实际的 TLA+ 规范由于引入了一种名为“停滞不变性” (Stutter-Invariance) 的机制而略微复杂一些，不过那属于更深层的实现细节。动作属性与不变量同属于**安全性** (Safety) 属性，用通俗的语言来说就是“坏事永远不会发生”。 [我曾专门写过一篇文章探讨安全性与活性](https://www.hillelwayne.com/post/safety-and-liveness/) 。 > We can also compose `[]` with primes to get **action properties**, or change variants. `[](x' >= x)` is true if the new value of x is always greater than equal to the old value of x. Another fun one is `[](P => P')`: once P is true, it can never become false again. The actual TLA+ is a little more complicated because of a thing called "stutter-invariance", but that's extra details. Action properties and invariants are both **safety** properties, which roughly means "something bad never happens". [I wrote an article on safety and liveness here](https://www.hillelwayne.com/post/safety-and-liveness/). 顺便提一下，**活性** (Liveness) 指的是“好事最终总会发生”。所有的活性属性都是基于 `<>` 构建的。单纯的 `<>P` 仅表示“在每条行为中，P 至少在某一个状态下成立”，对于系统规范而言，这种约束通常太弱了。但通过逻辑复合，我们可以构建出更丰富、更有价值的活性属性： > Liveness, btw, is "something good always happens". All liveness properties are based off of `<>`. By itself, `<>P` just means "P is true in at least one state of every behavior", which is usually too weak to be a good system property. But with composition, we can make more interesting liveness properties: * `[]<>P` ：如果从任意状态出发，P 在未来的至少一个状态中必定为真，则该式为真。这通常用来建模故障恢复机制，比如“只要节点发起新的主节点选举，它们最终总能达成共识选出主节点”。 * `<>[]P` ：如果在某个时间点之后，P 变为真且此后永远保持为真，则该式为真。这非常适合用来证明算法能够正确终止并收敛到正确结果。 * `[](P => <>Q)` ：如果在 P 成立的每一个状态之后，总存在一个使 Q 成立的未来状态，则该式为真。这可以证明因果关系，比如证明 P 最终必定触发 Q，或者“投递到队列中的所有消息最终都会出现在读者的读取历史中”。由于该公式的逻辑书写较为冗长，TLA+ 提供了语法糖 `P ~> Q` (即 P leads to Q，P 最终导致 Q) 。 > * `[]<>P` is true if, in every state, P is true in at least one future state. This can represent recovery mechanisms like "If the nodes have a new leader election, they will eventually agree on a leader." > * `<>[]P` is true if, at some point in time, P becomes true and remains true forever. This is great for showing that algorithms terminate with the correct results. > * `[](P => <>Q)` is true if, for every state where P is true, there is a future state where Q is true. This can do things like show that P eventually causes Q, or that "all messages put on the queue are eventually in a reader's history." Parsing the formula is a little confusing, so we have the sugar `P ~> Q` (P leads to Q). 虽然 TLA+ 还提供了像 `ENABLED` 和 `<<A>>_v` 这样能玩出更多花样的运算符，但我们在实际工程中检验的大多数属性依然是不变量、动作属性与活性属性。此外还有 [精炼](https://buttondown.com/hillelwayne/archive/refinement-without-specification/) (Refinement) ，精炼是安全性和活性的综合体，本身就是一个值得专门立题深聊的庞大概念。 > There's a couple of other operators, like `ENABLED` and `<<A>>_v`, which open up other tricks, but the majority of the stuff we check are invariants, action properties, and liveness. And [refinement](https://buttondown.com/hillelwayne/archive/refinement-without-specification/), which is a combination of safety and liveness and a topic unto itself. --- ## TLA+ 无法做到的事 > ## What TLA+ Can't Do 让我们从最显而易见的一点谈起：如果你根本不知道如何把系统的某项属性抽象表达为一个精确的逻辑公式，那么 TLA+ 对你没有任何帮助——实际上，任何形式化方法都无能为力。如果你无法用严格的数学或逻辑语言去形式化人类对“鸟”的认知概念，你就绝不可能证明你的应用程序一定能够识别鸟类。遗憾的是，现实工程中我们所关心的许多极其关键的系统特性，恰恰都属于这一类无法被简单数学化的概念。 > Let's start with the obvious one: if you don't know how to represent your property as a logical formula, then TLA+ can't help you. Nor can any formal method. If you can't formalize the human notion of a bird, you can't prove your app recognizes birds. And, unfortunately, a lot of important properties we care about fall into this category. 其次是一些过于具体的细节限制。TLA+ 的安全性属性要么作用于单个独立状态 (如不变量) ，要么作用于单步跃迁 (如动作属性) 。你无法原生定义一个跨越两步或更多步骤的属性，比如“按下 `delete` 键紧接着按下 `undo` 撤销键，系统将完全恢复到初始状态”，或者“按下 `power` 电源键后，计算机必须在十步之内成功开机”。此外，我们既无法直接对浮点数运算定义属性，也无法对 [物理真实时间进行定义，TLA+ 只能处理逻辑时间](https://buttondown.com/hillelwayne/archive/physical-vs-logical-time/) 。 > Next, the overly specific things. TLA+ safety properties work on the level of either individual states (invariants) or single step (action properties). You can't natively define a property over two or more steps, like "pressing `delete` and then `undo` gives you back the original state", or "once `power` is pressed, the computer turns on within ten steps". We also can't define properties on floating point operations or over [real time, just logical time](https://buttondown.com/hillelwayne/archive/physical-vs-logical-time/). 现在谈谈最令我着迷的一项根本性局限。TLA+ 的属性在语义上默认隐含了针对“所有行为”的全称量词量化。我刚才提到检查 `[]P` 意味着“P 在每一个状态下都为真”，但其实质含义是：“对于系统可能产生的所有行为，`[]P` 在该行为的初始状态上均成立”。换言之，TLA+ 能够检验的任何系统属性，都必须在**每一个单独的行为路径**上各自成立。 > Now for limit that most interests me. TLA+ properties are implicitly quantified over all behaviors. I said that checking `[]P` means "P is true in every state," but what it actually means is "for all behaviors, `[]P` is true of that behavior's initial state." Any property TLA+ can check of the system must be a property that is true for every individual behavior. 这意味着什么被排除在外了呢？答案是：被漏掉的内容远超你的想象！ > What does that leave out? A lot more than you'd expect! 其一，我们无法表达“**存在**至少一条行为使得 P 为真”。也就是说，我们无法直接断言某种状态 P 是可能发生的，哪怕我们在某次执行中并未真正触发它。一个典型的例子是证明“一款游戏是存在通关可能的”。我们把这类属性称为 [可达性属性](https://buttondown.com/hillelwayne/archive/proving-whats-possible) (Reachability Properties) 。更高级的可达性属性还包括“从任意初始状态出发 P 都是可达的”，或者“从任何满足 Q 的状态出发 P 都是可达的”。 > For one, we can't do "there *exists* a behavior where P is true". So we can't say that P is possible, even if we don't actually reach it. One example of this would be proving that a game is winnable. We call these [reachability properties](https://buttondown.com/hillelwayne/archive/proving-whats-possible). More advanced reachability properties would be things like "P is reachable from every initial state" or "P is reachable from any state where Q is true". 其二，我们无法跨越**一组行为集合**来定义属性。在形式化理论中，这被称为**超属性** (Hyperproperty) 。假设我们正在对手机硬件进行建模，希望验证“省电模式消耗的能量始终低于普通模式”。这一属性表述为：“在执行任意相同的动作序列时，省电模式下的功耗绝不高于普通模式”。如果要推翻这个结论，你必须同时给出两条行为路径进行对照：这两条路径除了一个始于省电模式、另一个始于普通模式外完全一致，且普通模式的功耗更低。仅凭单一的行为轨迹根本无法完成这种跨轨迹对比，因此这在 TLA+ 中是无法自然原生检验的。 > We also can't define properties over a *set* of behaviors. This is called a **hyperproperty**. Say we we're modeling phone hardware and want to verify that energy saving mode always uses less energy than normal mode. The property is "any sequence of actions uses no more power in energy saving mode than in regular mode." To refute this, you'd need to give me two behaviors that were identical *except* one starts in energy saving mode and the other does not, and the regular mode uses less power. One single behavior isn't enough to cut it, so this is impossible to naturally check in TLA+. 超属性听起来可能有些小众，但它实际上涵盖了 [海量的安全属性](https://en.wikipedia.org/wiki/Non-interference_(security)) (例如信息流的非干涉性) 以及所有统计学意义上的属性 (比如“95% 的响应时间必须在 5 毫秒以内”) 。 > Hyperproperties might seem niche but they cover a [ton of security properties](https://en.wikipedia.org/wiki/Non-interference_(security)) and all statistical properties ("the 95%ile response time is 5ms"). 最后一点稍偏学术化：我们无法将属性定义在全局的状态空间 (State Space) 之上。例如，我们无法断言“从状态 X 到状态 Y 之间存在且仅存在一条路径”。我不太确定这种约束在工程实践中具体有多大用处。大多数这类“元属性” (Metaproperties) 似乎都有潜在的理论价值，但我目前还无法举出十分具体的现实应用场景。 > Finally, and this one's a little more academic, we can't define properties over the state space as a whole. We can't say, for example, that there's only one path from state X to Y. I don't know how useful this would be in practice. Most of these kinds of "metaproperties" seem like they have potential to be meaningful, I just don't know what specifically. https://insight.aitobox.com/blog/posts/2026-10-01/Perplexity-揭秘自研-Rust-检索系统-Photon-将-p99-延迟从-800ms-降至-65ms/ https://insight.aitobox.com/blog/posts/2026-10-01/Perplexity-揭秘自研-Rust-检索系统-Photon-将-p99-延迟从-800ms-降至-65ms/ Thu, 01 Oct 2026 00:00:00 GMT 随着索引规模与知识库的飞速膨胀，传统开源检索引擎在面对严苛的高并发请求时，常常深陷长尾延迟高企与磁盘索引合并卡顿的泥潭。为了从根本上破除这一瓶颈，知名 AI 搜索产品 Perplexity 放弃了对原有开源分支修修补补的做法，完全采用 Rust 语言从零自研了全新的分布式检索与重排引擎 Photon。Photon 巧妙结合了自适应倒排列表、带算力预算的剪枝遍历，以及基于 Linux 顶尖内核特性 `io_uring` 的异步批量磁盘读取，成功将生产环境的 p99 检索延迟从约 800 ms 极限压缩至仅 65 ms，同时减少了 20% 的服务器基建资源需求。依托 Photon 打造的 Fast Search API 为 AI 智能体 (AI Agent) 工作流提供了兼具毫秒级极速响应与超高性价比的检索底座，为下一代 AI 搜索架构的性能演进树立了行业标杆。 --- ## 概要 > ## Summary AI 搜索领头羊 Perplexity 正式推出了自研的高性能检索与排序引擎 **Photon**，该系统完全采用 **Rust** 语言编写。在此之前，Perplexity 依赖的开源魔改引擎长期受困于尾部高延迟与磁盘索引合并卡顿；而全新登场的 Photon 不仅全面承接了 Perplexity 生产环境的所有线上流量，还驱动了全新推出的 **Fast Search** API 模式。通过巧妙运用自适应倒排列表 (Adaptive Posting Lists) 、带算力预算的遍历算法，以及基于 `io_uring` 的异步磁盘读取技术，Photon 将线上检索的 p99 延迟从约 800 ms 断崖式压降至约 65 ms，不仅节省了 20% 的服务器计算资源，更让 AI 智能体 (AI Agent) 工作流的运行成本直降 68%。 > Perplexity has introduced **Photon**, a high-performance, in-house retrieval and ranking engine written in **Rust**. Built to replace an open-source forked engine that struggled with tail latency and disk index fusion spikes, Photon powers all of Perplexity's production traffic as well as the new **Fast Search** API mode. By leveraging adaptive posting lists, budgeted traversal, and `io_uring` asynchronous reads, Photon dramatically reduces p99 retrieval latency from ~800 ms down to ~65 ms while saving 20% on serving infrastructure and enabling a 68% cost reduction for AI agent workflows. --- ## 为何 Perplexity 要替换旧引擎？ > ## Why Perplexity Replaced Its Old Engine 随着 Perplexity 索引数据量的不断膨胀，原先基于开源修改的旧版检索引擎遭遇了三大致命瓶颈： > As Perplexity's index grew, the legacy open-source retrieval engine hit three major bottlenecks: * **尾部延迟高企 (Tail Latency)**：生产环境的 p99 延迟一直徘徊在 800 ms 左右。由于索引数据总量远远超出了物理内存 (RAM) 的承载上限，无法使用 `mlock` 将数据全部常驻内存，导致冷读取时频繁触发严重缺页异常 (page faults) ，进而造成查询严重卡顿。 * **合并尖峰卡顿 (Merge Spikes)**：后台执行磁盘索引融合 (fusion) 操作时，会导致 p99 延迟猛增至约 1.2 秒，并且这种糟糕的卡顿每次会长达 10 到 15 分钟。 * **灾备恢复缓慢 (Slow Recovery)**：部署并同步备用集群需要耗费一周以上的时间，在此期间部分响应失败的比率也显著攀升。 > * **Tail Latency:** Production p99 latency hovered around 800 ms. Because the dataset exceeded available RAM, `mlock` was unviable, leading to major page faults and query stalls during cold reads. > * **Merge Spikes:** Disk index fusion operations caused p99 latency to spike to ~1.2 seconds for durations of 10 to 15 minutes. > * **Slow Recovery:** Deploying and syncing auxiliary clusters took over a week, simultaneously increasing the rate of partial responses. 面对这些难以逾越的技术障碍，Perplexity 团队清醒地意识到：与其在缝缝补补的旧开源分支上继续耗费精力，不如从零自研一套专属引擎。这不仅架构更纯粹、性能更强劲，综合维护成本也远比守着老系统更低廉。 > To overcome these hurdles, the Perplexity team determined that building a custom engine from scratch was cleaner, more performant, and cheaper than maintaining their fork. --- ## Photon 的技术实现原理 > ## How Photon Works Photon 采用了由负载均衡器、代理 Broker 节点以及专属分片组 (shard groups) 协同编排的分布式架构： > Photon utilizes a distributed architecture orchestrated by a load balancer, broker nodes, and dedicated shard groups: * **自适应倒排列表 (Adaptive Posting Lists)**：针对短列表，将其直接内联保存在单个内存页中；长列表则被切分为固定文档 ID 的数据块。其中，稀疏块采用有序偏移数组搭配跳跃搜索 (galloping search) ；稠密块则使用位图 (bitmaps) 实现极速的单比特成员资格判定。 * **带预算的剪枝遍历 (Budgeted Traversal)**：借鉴类 WAND 算法的思想，系统将倒排列表划分为主导列表 (driving lists) 与探测列表 (probe lists) ，在无需逐个读取精确词频的情况下，以极低的计算代价快速评估候选文档的分数上下界。 * **Docblob 紧凑记录 (Docblob Records)**：每个文档都存储为一份紧凑格式的记录，对词项采用 Elias-Fano 编码压缩，使得重排阶段每个文档仅需进行一次查询即可搞定。 * **异步批量读取 (Batched Async Reads)**：各记录的偏移量均提前计算就绪，从而能够通过 Linux 内核的 `io_uring` 特性实施批量的异步磁盘读取。在缓存设计上，系统放弃了存在锁竞争的全局共享 LRU 列表，转而采用无锁友好的 CLOCK 淘汰策略。 * **构建与服务解耦 (Separate Build and Serve)**：由独立的构建节点直接从 YTsaurus 表中生成带版本号的分片索引，这让控制中心能够平滑无缝地轮转线上服务节点，并利用历史查询日志提前完成缓存预热。 > * **Adaptive Posting Lists:** Short lists are stored inline within a single page, while longer lists are split into fixed-document ID blocks. Sparse blocks rely on sorted offset arrays with galloping search, and dense blocks use bitmaps for single-bit membership lookups. > * **Budgeted Traversal:** Utilizing a WAND-like algorithm, lists are separated into driving and probe lists to check candidate score bounds cheaply before reading exact term frequencies. > * **Docblob Records:** Each document contains a compact record utilizing Elias-Fano encoding for terms, requiring only a single lookup per document during ranking. > * **Batched Async Reads:** Record offsets are predetermined, enabling batch disk reads via `io_uring`. Caching bypasses locks by employing a CLOCK eviction policy instead of a shared LRU list. > * **Separate Build and Serve:** Dedicated nodes build versioned shard indexes from YTsaurus tables, allowing a controller to rotate serving groups smoothly while pre-warming caches with historical query logs. --- ## 生产落地效果 > ## Production Results 全量迁移至 Photon 为 Perplexity 带来了基础设施与系统性能的双重飞跃： > Migrating to Photon delivered massive infrastructure and performance gains: * **p99 延迟断崖式下降**：检索与排序阶段的延迟从约 800 ms 直降至约 65 ms (仅涵盖 Photon 负责的处理阶段) 。 * **资源利用率大幅提升**：与先前的旧节点相比，Photon 运行所需的**在线服务机器数量减少了 20%**。 * **单文档数据密度更高**：单个文档所能存储的数据量提升了约 **2.5 倍**，显著增强了全局排序的准确性与质量。 * **极致内存优化**：如果采用传统方案通过 `mlock` 将同等规模的数据集强行锁在内存中，所需的常驻内存容量将达到 Photon 实际消耗的约 4.6 倍。 * **集群发布稳定可靠**：索引版本更新与切换时，再也不会出现因合并而引发的延迟飙升。 > * **p99 Latency Reduction:** Retrieval and ranking latency dropped from ~800 ms to ~65 ms (covering Photon's stages only). > * **Resource Efficiency:** Photon operates on **20% fewer serving machines** than the legacy nodes. > * **Higher Data Density:** It stores roughly **2.5x more data per document**, improving overall ranking quality. > * **Memory Optimization:** Pinning the equivalent dataset via `mlock` would have required ~4.6x the resident memory Photon consumes. > * **Stable Deployments:** Index version switches no longer cause latency spikes. --- ## Fast Search：为智能体量身定制的速度与成本优势 > ## Fast Search: Speed and Cost for Agents 借助 Photon 的强劲能力，Perplexity 搜索 API 推出了一种全新的 **Fast Search** (快速搜索) 模式 (请求参数为 `search_type: "fast"`，定价仅为每 1,000 次请求 1 美元) ，专为 AI 智能体的高频自动化调用而设计。 > Photon enables a new **Fast Search** mode via the Perplexity Search API (`search_type: "fast"` at `$1 per 1,000 requests`), tailored specifically for agentic workflows. https://insight.aitobox.com/blog/posts/2026-10-01/CounterSteer-利用激活导向抑制大模型智能体中的间接提示词注入攻击/ https://insight.aitobox.com/blog/posts/2026-10-01/CounterSteer-利用激活导向抑制大模型智能体中的间接提示词注入攻击/ Thu, 01 Oct 2026 00:00:00 GMT 随着大语言模型 (Large Language Model, LLM) 深度融入各类 AI 智能体 (AI Agent) 场景，自动化处理外部网页、邮件和工具输出已成为常态，但外部不受信任内容中潜藏的“间接提示词注入攻击”却成为了巨大的安全隐患。Mark Russinovich 提出的 CounterSteer 带来了一种全新且高效的推理阶段防御方案：无需对模型进行二次微调，也不依赖额外的辅助防护模型，而是直接在预填阶段从模型残差流中识别并剔除与恶意指令相关的激活方向。这一常开型机制让攻击者无法察觉并绕过检测，在五款主流开源架构模型上将攻击成功率压制至接近于零，为智能体生态系统的安全落地提供了一套开箱即用且近乎无损的全新防御范式。 --- ## 核心执行摘要 > ## Executive Summary **CounterSteer** 是一种创新的推理阶段防御机制，专门用于化解大语言模型 (LLM) 智能体中的间接提示词注入漏洞。在模型处理输入的预填 (Prefill) 阶段，CounterSteer 能够精准锁定与“遵从恶意指令”相关的残差流 (Residual Stream) 激活方向并直接将其减去；这种做法既不需要重新微调模型、也无需部署额外的辅助审查模型或引入额外的 Token ，便能干净利落地瓦解恶意指令对智能体执行流程的劫持。 > **CounterSteer** is an innovative inference-time defense mechanism designed to mitigate indirect prompt injection vulnerabilities in Large Language Model (LLM) agents. By identifying and subtracting residual-stream directions associated with malicious instruction-following during the prefill phase, CounterSteer effectively neutralizes instructional takeovers without requiring model fine-tuning, auxiliary models, or added tokens. --- ## 论文元数据 > ## Paper Metadata * **论文标题：** CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering * **作者：** Mark Russinovich * **主要学科领域：** 密码学与安全 (`cs.CR`) * **次要学科领域：** 人工智能 (`cs.AI`) * **arXiv 编号：** [arXiv:2609.36570 [cs.CR]](https://arxiv.org/abs/2609.36570) * **提交时间：** 2026 年 9 月 29 日 > * **Title:** CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering > * **Author:** Mark Russinovich > * **Primary Subject:** Cryptography and Security (`cs.CR`) > * **Secondary Subject:** Artificial Intelligence (`cs.AI`) > * **arXiv Identifier:** [arXiv:2609.36570 [cs.CR]](https://arxiv.org/abs/2609.36570) > * **Submitted:** September 29, 2026 --- ## 论文摘要 > ## Abstract 间接提示词注入就像是给 AI 智能体设下的“特洛伊木马”陷阱：当智能体检索外部网页、文档或数据时，潜藏在其中的不受信任文本会诱导智能体将其误判为正当指令。为此，CounterSteer 提出了一套简洁明了的五步配方流程：通过构建成对的对照测试场景（两组场景唯一的差异仅在于是否听从了嵌入的指令），为每个大语言模型精准拟合出一个对应的残差流向量方向。只有当该方向成功通过预先设立的因果验证与通用能力门控检验时，才会被保留并投入使用。 > Indirect prompt injection tricks LLM agents into treating untrusted retrieved text as legitimate instructions. CounterSteer presents a streamlined, five-step recipe to fit a residual-stream direction per model using paired episodes (differing only in whether an embedded instruction is followed). This direction is retained only if it successfully passes pre-specified causal and capability gates. 在实际部署运行中，只要智能体调用工具并接收返回结果，系统就会在预填计算阶段从每一个工具结果 Token 中直接减去这一学习到的方向。由于这种神经层面的干预始终处于常开生效状态，系统中完全不存在任何可供攻击者探测或绕过的“防火墙规则”与检测开关。 > During deployment, the learned direction is subtracted from every tool-result token during the prefill phase. Because the edit is always active, there is no detection mechanism for attackers to evade. --- ## 核心发现与性能表现 > ## Key Findings & Performance * **攻击成功率大幅降低：** 在涵盖五大厂商血统、参数量跨越 80 亿 (8B) 到 1060 亿 (106B) 的五款开源权重模型上，面对预留的测试集攻击，未防御时的攻击成功率高达 **0.21–1.00** （最高可达 100%），而开启 CounterSteer 防护后断崖式骤降至 **0.00–0.17** 。 * **AgentDojo 沦陷率大幅压制：** 在权威智能体基准测试 AgentDojo 中，智能体的沦陷率从原先的 **0.10–0.49** 锐减至 **0.006–0.079** （最低降至千分之六）。 * **出色保留正常任务效用：** 在排版归一化的良性任务效用测试中保持了 **93–100%** 的原有性能；尽管在针对经过导向干预的内容进行复杂推理时会产生一定程度的任务相关性能开销，但日常通用能力几乎毫无损伤。 * **极其坚固的防御鲁棒性：** * **白盒梯度攻击抵御：** 即便攻击者完全获知已部署的干预向量并通过白盒梯度发起攻击，在 52 个测试场景中至多也仅能攻破 2 个； * **人类红队实战攻击免疫：** 在回放的 2,052 次人类安全红队实战攻击样本中，攻击成功率彻底归零 (0/2052) ； * **黑盒语义重构绕过困难：** 黑盒语义重构搜索 (Framing Searches) 在 18 个开发集测试样本中仅能侥幸绕过 3 个。 > * **Attack Success Reduction:** Across five open-weights models (ranging from 8B to 106B parameters across five vendor lineages), held-out attack success rates plummeted from an undefended **0.21–1.00** down to **0.00–0.17** when defended. > * **AgentDojo Compromise Rate:** Decreased from **0.10–0.49** down to **0.006–0.079**. > * **Utility Retention:** Retains **93–100%** typography-normalized benign utility, though some task-dependent costs arise when reasoning over steered content. > * **Robustness:** > * White-box gradient attacks through the deployed vector successfully compromised at most 2 out of 52 episodes. > * Zero out of 2,052 replayed human red-team attacks succeeded. > * Black-box framing searches successfully bypassed the defense in only 3 out of 18 development samples. --- ## 局限性与未来展望 > ## Limitations & Future Work 虽然 CounterSteer 几乎彻底瓦解了针对执行流程的恶意指令劫持，但在**参数操纵** (Parameter Manipulation) ——即攻击者在原本合法的工具调用中夹带自选恶意参数（例如正常调用转账功能，却篡改了转账接收人账号）——方面仅能实现部分抵御（在 18 个测试样本中成功防御了 13 个）。究其内在原因，这类决策往往是在模型真正生成并输出参数的那一瞬间才呈现出线性可读的内部表征，而非在生成前的预填检查位点即可捕获；因此，未来还需要结合额外的**参数来源溯源控制** (Argument-Provenance Controls) 机制，才能将这一安全通路彻底封堵。 > While CounterSteer largely neutralizes instructional takeovers, **parameter manipulation** (attacker-chosen arguments embedded in otherwise legitimate calls) is only partially resisted (13 out of 18). Because the decision becomes linearly readable at argument emission rather than at pre-generation inspection sites, additional **argument-provenance controls** are required to fully secure these pathways. https://insight.aitobox.com/blog/posts/2026-10-01/ARC-KV-摊薄锚点搜索开销的重构式-KV-Cache-极致压缩技术/ https://insight.aitobox.com/blog/posts/2026-10-01/ARC-KV-摊薄锚点搜索开销的重构式-KV-Cache-极致压缩技术/ Thu, 01 Oct 2026 00:00:00 GMT 在长上下文大语言模型 (LLM) 推理场景中，键值缓存 (KV Cache) 的显存占用随输入长度线性暴涨，在长系统提示词或长文档频繁复用的任务中构成了严峻的内存瓶颈。现有的重构式压缩方案 (如基于正交匹配追踪 OMP 的 Attention Matching) 虽然压缩性能出众，但依赖耗时的逐次迭代搜索锚点，导致前期压缩计算开销极其沉重。为此，研究团队提出了基于“选择性摊薄 (Selective Amortization) ”理念的 **ARC-KV** 技术：通过训练轻量级的价值感知索引器单次打分筛选真实 Key 锚点，将昂贵的搜索成本跨查询彻底均摊。实验表明，在 Llama-3.1-8B-Instruct 模型及 QuALITY、RULER 和 LongBench 等权威长文本基准上，ARC-KV 即使仅保留 10% 的 KV Cache 也能超越基线精度，同时将压缩耗时从 959.8 秒大幅缩减至 37.3 秒 (加速达 25.73 倍) ，实现了速度与精度的双重突破。 --- # ARC-KV：摊薄锚点搜索开销的重构式 KV Cache 极致压缩技术 > # ARC-KV: Amortizing Anchor Search for Reconstruction-Based KV Cache Compaction ## 核心内容综述 > ## Summary 在长上下文大语言模型 (Large Language Model, LLM) 的推理过程中，显存开销居高不下一直是业界痛点。这主要是因为键-值缓存 (Key-Value Cache, KV Cache) 的体积会随着序列长度呈线性暴涨——尤其在面对需要多次响应不同查询的超长可复用前缀 (Context Prefixes) 时，这一负担更显沉重。虽然像 Attention Matching 这样基于数学重构的压缩方法能够凭借紧凑精炼的 KV Cache 取得优异的任务性能，但它们严重依赖通过正交匹配追踪 (Orthogonal Matching Pursuit, OMP) 进行反复迭代的锚点搜索，这直接造成了巨大的计算性能瓶颈。 > Long-context large language model (LLM) inference often struggles with high memory overhead because Key-Value (KV) caches grow linearly with sequence length—a problem that is especially burdensome for long, reusable context prefixes. While reconstruction-based methods like Attention Matching offer strong performance using compact KV caches, their reliance on iterative anchor search via OMP (Orthogonal Matching Pursuit) creates a massive computational bottleneck. 为了解决这一难题，作者团队推出了 **ARC-KV**——一种基于“选择性摊薄 (Selective Amortization) ”理念的全新 KV Cache 压缩方法。ARC-KV 能够在不同上下文之间学习一套通用的可复用锚点选择策略，同时保留针对各个具体上下文的专属重构，从而彻底摆脱了反复繁琐搜索的算力泥潭。 > To solve this, the authors introduce **ARC-KV**, a novel KV cache compaction method based on the principle of *selective amortization*. ARC-KV learns a reusable anchor-selection policy across contexts while retaining context-specific reconstruction. https://insight.aitobox.com/blog/posts/2026-10-01/AI-充当编译器-无需-Triton-编译器即可将-Triton-算子直译为-PTX-代码/ https://insight.aitobox.com/blog/posts/2026-10-01/AI-充当编译器-无需-Triton-编译器即可将-Triton-算子直译为-PTX-代码/ Thu, 01 Oct 2026 00:00:00 GMT 随着深度学习模型结构与新型 GPU 硬件加速器的快速演进，为新硬件手工构建与维护传统的编译器后端正变得极其昂贵且耗时。本研究提出了创新的 AI 降级直译 (AI lowering) 范式，直接借助大语言模型 (Large Language Model, LLM) 智能体取代繁琐的传统编译器优化与降级流水线，将高级的 Triton 算子端到端直译为底层的 NVIDIA PTX (Parallel Thread Execution) 汇编代码。在跨越 Ada、Hopper 以及最新的 Blackwell 架构 GPU 上对 22 个核心算子的全面实测中，AI 直译算子的性能达到了官方自动调优 (Auto-tuned) Triton 编译器的 0.83 倍至 3.34 倍。这一突破表明，AI 编译器能够有效加速新型计算芯片的基础软件栈落地，并在硬件微架构的极致优化中发挥超越传统编译器的关键作用。 --- ## 执行摘要 > ## Executive Summary 随着编程模型、计算工作负载以及底层硬件加速器的快速迭代演进，为每一种新芯片开发并维护传统的编译器后端已经变得代价高昂且难以维系。本篇研究论文深入探讨了 **AI 降级直译 (AI lowering)** ——一种彻底颠覆传统的技术范式：由大语言模型完全取代传统编译器中层层递进的代码优化与底层代码降级流水线。 > As programming models, workloads, and hardware accelerators rapidly evolve, building and maintaining conventional compiler backends becomes prohibitively expensive. This research paper investigates **"AI lowering"**—a novel paradigm where large language models (LLMs) entirely replace the traditional optimizing and lowering pipeline. 具体而言，研究团队构建了一套专用的智能体框架 (Agentic Harness) ，驱动大语言模型智能体将高级的 Triton 算子直接直译为 NVIDIA 底层的 PTX (Parallel Thread Execution) 指令代码，完全绕开了官方的 Triton 编译器。在涵盖 12 个通用核心算子 (在 Ada、Hopper 以及 Blackwell GPU 上完成测试) 以及来自近期前沿机器学习文献的 10 个算子的全面基准评测中，经由 AI 直译生成的算子取得了官方自动调优 (Auto-tuned) Triton 性能的 **0.83 倍至 3.34 倍**。 > Specifically, the authors build an agentic harness utilizing an LLM agent to translate Triton kernels directly into NVIDIA PTX (Parallel Thread Execution) code, bypassing the Triton compiler altogether. Evaluated across twelve common kernels (tested on Ada, Hopper, and Blackwell GPUs) and ten kernels from recent machine learning literature, the AI-lowered kernels achieved **0.83x to 3.34x the performance** of autotuned Triton. --- ## 核心发现与性能亮点 > ## Key Findings & Performance Highlights AI 降级直译之所以能实现显著的性能加速，核心在于它能够自主发掘并应用一系列传统编译器流水线往往会遗漏的高级深度优化策略： > The performance gains achieved through AI lowering stem from advanced optimizations that standard compiler pipelines typically miss: * **BitDelta (实现 3.34 倍加速) ：** 直接将紧密打包的二进制权重在线解码为 Tensor Core 的计算操作数； * **FlashAttention (实现 1.37 倍加速) ：** 在 Tensor 内存中为每个线程分配一整行完整的 Softmax 数据； * **卷积窗口 (Convolution Windows，实现高达 2.23 倍加速) ：** 高效复用相互重叠的卷积窗口，大幅削减冗余访存开销。 > * **BitDelta (3.34x speedup):** Decoding packed binary weights directly into Tensor Core operands. > * **FlashAttention (1.37x speedup):** Assigning each thread a complete softmax row in tensor memory. > * **Convolution Windows (up to 2.23x speedup):** Reusing overlapping convolution windows efficiently. --- ## 正确性验证与新一代 GPU 架构支持 > ## Verification & Modern GPU Architecture Support 为了从根本上确保生成代码的正确性，研究团队构建了一套稳健的评估测试套件，集成了端到端的全方位验证能力。他们扩展了现有的 PTX 静态验证工具 **Volta** (一款已有的 PTX 验证器) 以支持现代 GPU 架构，并专门针对 **Blackwell** 架构中的 `tcgen05` Tensor Core 硬件接口引入了针对性建模支持。 > To ensure correctness, the authors built a robust evaluation harness featuring comprehensive verification support. They expanded **Volta** (an existing PTX verifier) to support modern GPU architectures, introducing specific capabilities for the **Blackwell** `tcgen05` Tensor Core interface. 这需要在验证工具中精确模拟并建模三项极其复杂的现代硬件微架构特性： > This required modeling three intricate architectural features: 1. **统一托管的 Tensor 内存 (Managed Tensor Memory)**； 2. **基于描述符的操作数内存布局 (Descriptor-based Operand Layouts)**； 3. **由提交 (Commits)、等待 (Waits)、内存屏障 (Memory Barriers) 以及代理栅栏 (Proxy Fences) 严密协同的硬件级异步执行流**。 > 1. Managed tensor memory > 2. Descriptor-based operand layouts > 3. Asynchronous execution coordinated through commits, waits, memory barriers, and proxy fences --- ## 总结与未来展望 > ## Conclusion & Future Outlook 上述研究成果清晰地预示着一个全新技术范式的到来：AI 编译器正在逐步取代人工专门编写的中间表示 (Intermediate Representation, IR) 以及静态检查器。通过这种端到端直译的方式，开发者能够大幅减少为新型通用 GPU 以及定制专用硬件加速器构建全套底层软件栈所需的时间与工程精力。 > These results point toward an emerging paradigm where AI compilers replace custom-written intermediate representations (IRs) and static checkers. By doing so, they drastically reduce the time and engineering effort required to bring up software stacks for new general-purpose and custom hardware accelerators. --- *[View PDF](https://arxiv.org/pdf/2609.36800) | [TeX Source](https://arxiv.org/src/2609.36800)* <img alt="license icon" role="presentation" src="./images/345c7ad61f1b.png"> https://insight.aitobox.com/blog/posts/2026-09-30/阿里通义发布-Qwen-Audio-3.1-Realtime-具备自主思考-工具调用与话轮决策的全双工语音大模型/ https://insight.aitobox.com/blog/posts/2026-09-30/阿里通义发布-Qwen-Audio-3.1-Realtime-具备自主思考-工具调用与话轮决策的全双工语音大模型/ Wed, 30 Sep 2026 00:00:00 GMT 随着人机语音交互从传统的“语音识别+大语言模型+语音合成”拼接链路向端到端全双工实时交互演进，如何让 AI 在倾听的同时自主思考、灵活调用外部工具并精准掌控说话时机，成为打造下一代拟人化语音智能体 (Voice Agent) 的核心瓶颈。Alibaba 旗下的 Qwen 团队正式推出了全新升级的音频模型技术栈 Qwen-Audio-3.1，其中领衔的旗舰模型 Qwen-Audio-3.1-Realtime 专为支持复杂工具调用的全双工语音交互而深度设计。该模型创新构建了“思考、行动、表达”三层训练架构，通过策略内蒸馏与强化学习赋能，不仅支持高达 262K 的超长上下文处理，而且大幅降低了背景杂音误触与口头垫字概率。与此同时，QwenCloud 全面大幅下调了音频套件的 API 定价 (降幅最高达 95%)，为开发者构建高响应、高智能且极具性价比的实时语音应用铺平了道路。 --- ## 执行摘要 > ## Executive Summary Alibaba 旗下的 Qwen 团队近日重磅推出了 **Qwen-Audio-3.1**，这是一套涵盖自动语音识别 (Automated Speech Recognition, ASR)、语音合成 (Text-to-Speech, TTS) 以及实时互动的全套五模型音频技术栈。本次发布的领衔旗舰是 **Qwen-Audio-3.1-Realtime**——一款专门面向具备工具调用与执行能力的语音智能体 (Voice Agent) 所打造的全双工语音模型 (Full-Duplex Speech Model)。伴随此次发布，Alibaba 将其全线音频套件的 API 调用价格最高下调了 95%。目前，该模型仅作为托管 API 通过 QwenCloud 以 WebSockets 协议独家提供服务 (`qwen-audio-3.1-realtime-plus`)，暂未公布开源权重的计划。 > Alibaba’s Qwen team has introduced **Qwen-Audio-3.1**, a comprehensive 5-model audio stack spanning ASR, TTS, and real-time interaction. Headlining the release is **Qwen-Audio-3.1-Realtime**, a full-duplex speech model engineered specifically for voice agents capable of tool execution. Alongside the launch, Alibaba has slashed API pricing by up to 95% across its audio suite. Currently, the model is deployable exclusively as a managed API via QwenCloud over WebSockets (`qwen-audio-3.1-realtime-plus`), with no open weights announced. --- ## QwenCloud 上线内容 > ## What Ships on QwenCloud 作为本次发布的旗舰模型，Qwen-Audio-3.1-Realtime 同时支持文本与音频作为输入与输出，并拥有高达 **262K Token 的超长上下文窗口** (最大支持 245K 输入与 16K 输出)。系统的默认调用速率限制设定为每分钟 60 次请求及每分钟 10 万 Token。 > The flagship model accepts both text and audio as inputs and outputs, boasting a **262K token context window** (245K max input, 16K max output). Default limits are set at 60 requests and 100K tokens per minute. * **定价策略：** * 音频输入：每 100 万 Token 计费 6.40 美元 * 文本输入：每 100 万 Token 计费 0.80 美元 * 文本与音频输出：每 100 万 Token 计费 24.00 美元 (输出文本免费) * **核心特性：** 支持函数调用 (Function Calling)、内置网络搜索、结构化输出、上下文缓存 (Context Caching) 以及模型微调。 > * **Pricing:** > * Audio Input: $6.40 per 1M tokens > * Text Input: $0.80 per 1M tokens > * Text & Audio Output: $24.00 per 1M tokens (output text is free of charge) > * **Key Features:** Function calling, built-in web search, structured outputs, context caching, and fine-tuning. 此外，Qwen 还同步推出了 **Qwen-Audio-3.1-ASR-Flash-Filetrans**，这是一款专门针对离线长音频转录场景量身优化的配套模型。该模型全面支持热词定制、说话人分离、标点预测，并兼具多语言文本识别与汉语方言解析能力，其定价为每 100 万 Token 输入 0.15 美元、输出 0.47 美元。 > Additionally, Qwen released **Qwen-Audio-3.1-ASR-Flash-Filetrans**, a companion model optimized for offline long-audio transcription. It supports hot words, speaker separation, punctuation, and recognition of multilingual text alongside Chinese dialects, priced at $0.15 for input and $0.47 for output per 1M tokens. --- ## 架构设计：一个声音背后的双模型协同 > ## Architecture: 2 Models Behind 1 Voice 该系统在统一共享的音频编码器 (Audio Encoder) 与大语言模型 (Large Language Model, LLM) 设计之上，通过两个彼此协同的独立模型实现运行： > The system operates using two distinct models built upon a shared Audio Encoder and LLM design: 1. **全双工决策模型：** 实时预测模型当前是应当保持倾听、开口说话、立即停止还是恢复发言； 2. **语音转文本模型：** 负责将回答内容生成为文本形式； 3. **上下文感知语音渲染器：** 结合对话历史、音色提示与声学环境上下文，将生成的文本实时转换为流式语音。 > 1. **A Full-Duplex Decision Model:** Predicts whether to continue listening, speak, stop, or resume. > 2. **A Speech-to-Text Model:** Generates the response content as text. > 3. **A Context-Aware Voice Renderer:** Converts the generated text into streaming speech, conditioned on conversation history, voice cues, and acoustic context. https://insight.aitobox.com/blog/posts/2026-09-30/谷歌开源-RRSI-框架-让-AI-智能体自主优化-Harness-架构且避免过拟合/ https://insight.aitobox.com/blog/posts/2026-09-30/谷歌开源-RRSI-框架-让-AI-智能体自主优化-Harness-架构且避免过拟合/ Wed, 30 Sep 2026 00:00:00 GMT 在追求更高性能的 AI 智能体 (AI Agent) 研发过程中，工程师通常需要不断优化系统外围的脚手架——即所谓的“Harness 架构” (涵盖提示词、工具链、记忆机制与控制调度等) 。然而，传统的自主进化算法往往会导致智能体“死记硬背”评测集中的特定题目，陷入严重的过拟合陷阱，导致在面对全新的未知任务时表现大幅滑坡。针对这一痛点，Google Cloud AI Research 携手多所顶尖高校开源了 RRSI (Regularized Recursive Self-Improvement，正则化递归自我改进) 框架。该框架创新性地在底层模型权重完全冻结的前提下，通过引入类似于经典机器学习惩罚项的“正则化”约束机制，让智能体不仅能安全自主地修改重写自身的外围 Harness 架构，还能有效遏制代码膨胀与噪声干扰。在 8 大主流基准测试中，经由 RRSI 进化的智能体在未见过的测试集上实现了全面稳步提升，同时推理 Token 消耗降低多达 36%，为构建真正具备通用泛化与自主进化能力的 AI 智能体系统指明了全新方向。 --- ## 执行摘要 > ## Executive Summary **Google Cloud AI Research** 携手 UNC-Chapel Hill、Stanford 以及 Washington University in St. Louis 联合推出了 **RRSI (Regularized Recursive Self-Improvement)** 开源框架。这项突破性框架允许大语言模型 (Large Language Model, LLM) 智能体在底层模型权重完全冻结的前提下，自主重构并进化自身的 Harness 架构——涵盖提示词、工具集、记忆机制、控制流以及子智能体。通过对自我改进闭环引入严格的正则化机制，RRSI 确保智能体获得的性能提升不仅局限于训练任务，还能稳健地迁移泛化到从未针对优化过的全新基准测试中。 > **Google Cloud AI Research**, in collaboration with UNC-Chapel Hill, Stanford, and Washington University in St. Louis, has released **RRSI (Regularized Recursive Self-Improvement)**. This innovative framework allows LLM agents to rewrite their own harnesses—including prompts, tools, memory, control flow, and sub-agents—while keeping model weights entirely frozen. By regularizing the improvement loop, RRSI ensures that performance gains translate robustly to benchmarks the agent has never optimized against. 该框架基于 **Apache 2.0 许可证** 开源发布，运行环境要求 Python 3.10+，并全面支持任意 LiteLLM 模型标识符 (默认配置为 Vertex AI 上的 Claude Opus 4.8) 。 > The framework is available under the **Apache 2.0 license**, requires Python 3.10+, and supports any LiteLLM model string (with defaults configured for Claude Opus 4.8 on Vertex AI). --- ## 为什么自主优化的 Harness 架构容易发生过拟合？ > ## Why Self-Improving Harnesses Overfit 传统的 Harness 架构进化闭环通常采用“提出修改建议、在固定任务集上打分评测、保留胜出方案”的模式。然而，由于系统在每一轮迭代中都反复面对同一套测试题目，这种闭环往往不可避免地走向了“死记硬背”。[RRSI 的研究](https://arxiv.org/abs/2609.24972) 指出，这一流程会引发三大典型的失效模式： > Traditional harness evolution loops propose edits, score them against a fixed set of tasks, and retain the winners. Because the same tasks are repeatedly evaluated every round, the loop tends to memorize them. According to the [RRSI research](https://arxiv.org/abs/2609.24972), this process introduces three primary failure modes: 1. **特定基准拟合 (Benchmark-specific fitting)**：智能体学会了针对特定题库走捷径，甚至记住评测集的特有逻辑； 2. **追逐随机噪声 (Noise chasing)**：由于评估本身的波动性，把随机产生的微小分值波动误当成实质性进步； 3. **复杂度盲目堆积 (Complexity accumulation)**：代码和提示词不断变长、系统越来越臃肿，却缺乏真正提升通用能力的实质改动。 > 1. **Benchmark-specific fitting** > 2. **Noise chasing** > 3. **Complexity accumulation** 这三种失效模式共同拉大了“进化集上的漂亮高分”与“实际分布外 (Out-of-Distribution, OOD) 泛化性能”之间的鸿沟。 > Each failure mode widens the gap between high evolve-set scores and actual out-of-distribution transfer performance. --- ## RRSI 的工作原理 > ## How RRSI Works RRSI 允许 Harness 架构中的每个组件都保持可编辑与可塑状态，但为整个搜索过程的演进步伐引入了极其严苛的正则化控制。 > RRSI allows every component of the harness to remain editable, but introduces rigorous regularization to how the search process moves. https://insight.aitobox.com/blog/posts/2026-09-30/任务的算力天花板-Transformer-究竟何时能脱离思维链取得成功/ https://insight.aitobox.com/blog/posts/2026-09-30/任务的算力天花板-Transformer-究竟何时能脱离思维链取得成功/ Wed, 30 Sep 2026 00:00:00 GMT 近年来，以 OpenAI o1 为代表的前沿推理模型在给出最终答案前，总会生成冗长而详尽的“思维链” (Chain of Thought, CoT) 。然而学术界与工业界一直存在激烈争论：这些看似深思熟虑的中间步骤，究竟是在切实执行复杂的深层计算，还是仅仅为了迎合人类思考习惯的表面修饰？来自宾夕法尼亚大学等机构的研究团队 (Jiashu He, Jinxuan Fan, Xiao Xiao, Radu Marculescu, Alejandro Ribeiro) 从电路复杂度的视角切入，将 Transformer 抽象为浅层电路，并提出了“任务天花板” (Ceiling of a Task) 这一全新概念。该研究通过严谨的数学证明与实证测试表明，对于具有强串行依赖特性的复杂任务，一旦剔除或破坏思维链，模型准确率就会不可逆地跌落至任务的理论天花板乃至纯粹的随机猜测水平。这项工作不仅为理解思维链的必要性提供了坚实的理论基石，也清晰划定了 Transformer 在不借助思维链时所能触及的能力边界。 --- ## 📌 执行摘要 > ## 📌 Executive Summary 本文深入探讨了推理模型在输出前生成的长长“思维链” (Chain of Thought, CoT) ，究竟是在切实分步执行真正的计算工作，还是仅仅流于形式的表面装饰。通过将 Transformer 抽象为理论计算机科学中的“浅层电路”，作者团队提出了 **任务天花板** (task's ceiling) 这一核心概念——即浅层计算电路在固定次数的前向传播中，所能达到的理论最高准确率极限。 > This paper investigates whether the "chains of thought" generated by reasoning models perform genuine computational work or merely serve as decorative outputs. By modeling transformers as shallow circuits, the authors introduce the concept of a **task's ceiling**—the upper limit of accuracy achievable by a shallow circuit in a constant number of passes. 通过严谨的理论证明，并结合在有限群字问题 (finite group word problems) 以及 MATH-500、AIME 等前沿数学基准上的实证评测，研究团队证实：一旦人为抹除或破坏思维链，模型的解题准确率便会如理论预言一般迅速跌落，其性能极限被牢牢锁死在任务自身的结构天花板之下 (而在前后步骤具有严格依赖关系的最大串行任务中，表现更是会彻底崩塌至随机瞎猜水平) 。归根结底，“任务天花板”这把理论标尺，清晰而有力地界定了 Transformer 何时能够脱离思维链自如应对，何时又必须高度依赖思维链才能取得突破。 > Through theoretical proofs and empirical evaluations on finite group word problems, MATH-500, and AIME datasets, the study demonstrates that removing or corrupting the chain of thought predictably degrades model performance down to the task's structural ceiling (or to random chance for maximally serial tasks). Ultimately, the "ceiling of a task" successfully delineates when a transformer can—and cannot—succeed without its chain of thought. --- ## 📑 论文摘要 > ## 📑 Abstract 推理模型在给出最终答案之前往往会生成长长的思维链，但这些中间内容究竟是在进行货真价实的计算，还是很大程度上只是流于形式的表面装饰，学术界对此一直存有争议。我们通过将 Transformer 视为计算理论中的“浅层电路” (shallow circuit) ，对这一核心问题展开了系统研究。 > Reasoning models generate long chains of thought before they answer, yet it is debated whether the content of these chains does real computational work or is largely decorative. We study this question by viewing a transformer as a shallow circuit. 在固定网络层数的 Transformer 中，单次前向传播的计算深度是恒定的常数，因此只要运行模型的次数是固定的常数，整个计算过程在本质上就是一个浅层电路。我们将浅层电路在某一任务上所能达到的极限准确率称为该任务的 **天花板** (ceiling) ；如果一个任务的理论天花板小于 1 (即无法被浅层电路完美解决) ，那么该任务便被定义为 **串行任务** (serial task) 。 > One forward pass through a fixed number of layers has constant depth, so any procedure that runs the model a constant number of times is a shallow circuit. We call the best accuracy that a shallow circuit can reach on a task the **ceiling** of the task, and a task is **serial** if its ceiling lies below one. 针对串行任务，无论模型采用了何种训练方式，我们严格证明了对所有 Transformer 均普适成立的三大核心结论： 1. **必要性 (Necessity)**：将思维链替换为任何不依赖其具体内容的形式 (例如无意义的填充 Token 或对题目的单纯复述) ，都会导致模型准确率直线下跌至任务天花板；而在最大串行任务中，准确率更是会跌至随机猜测水平。 2. **深度门槛 (Depth)**：对于准确率成功突破天花板的模型，绝不可能存在任何浅层计算能够生成其思维链，哪怕仅仅是生成大致的近似文本也无法实现。 3. **局部性 (Locality)**：一旦思维链生成完毕，模型只需经过一次浅层前向计算便能直接得出最终答案，这证明所有复杂的串行推理实质上全都发生在思维链的推导过程之中。 > We prove three results on serial tasks that hold for every transformer, no matter how it was trained: > 1. **Necessity:** Replacing the chain by anything that does not depend on its content, such as filler tokens or a restatement of the question, drives the accuracy down to the ceiling, and on a maximally serial task down to chance. > 2. **Depth:** No shallow computation can write the chain of a model whose accuracy exceeds the ceiling, not even approximately. > 3. **Locality:** The answer is one shallow pass away from the finished chain, so all of the serial reasoning happens in the chain. 在理论天花板已知的有限群字问题 (word problems of finite groups) 上，无论是否借助强化学习 (reinforcement learning, RL) ，从零训练的小型 Transformer 都精准复现了理论预测的数值：经过思维链训练的模型能够完美应对任意长度的输入，但一旦将思维链擦除，准确率立即滑落至随机瞎猜水平；未经过思维链训练的模型，其准确率随着输入长度的增加不可阻挡地跌向任务天花板；而主流开源权重推理模型在面对以自然语言表述的相同问题时，一旦失去思维链，其性能同样跌回基准线。 > On word problems of finite groups, whose ceilings are known, small transformers trained from scratch, with or without reinforcement learning, attain the predicted numbers: chain-trained models solve every input length and fall to chance when the chain is erased, chainless models collapse to the ceiling as the input length grows, and open-weight reasoning models given the same problem in words return to the baseline without their chain. 在颇具挑战性的 MATH-500 和 AIME 高难度数学基准上，直接抹去思维链会导致开源推理模型的准确率剧烈暴跌 0.52 至 0.82；实验还揭示了一个关键细节：将思维链中的句子次序打乱 (sentence shuffle) 对准确率几乎毫无影响，但一旦进行细粒度的 Token 级随机打乱 (token shuffle) ，其破坏性便与彻底抹除思维链完全一样。对于使用 GRPO 算法 (无论配合真实奖励还是随机奖励) 训练出的模型检查点，上述规律同样完全成立。由此可见，“任务天花板”这一理论工具，确切解答了 Transformer 何时能够脱离思维链取得成功。 > On MATH-500 and AIME, erasing the chain costs open reasoning models 0.52 to 0.82 accuracy, a sentence shuffle is harmless, and a token shuffle is as harmful as erasing; the same holds for checkpoints trained by GRPO with a correct or a random reward. The ceiling of a task therefore answers when a transformer can succeed without its chain of thought. --- ## 📋 论文元数据与更多信息 > ## 📋 Metadata & Additional Information | 字段 | 详情 | | :--- | :--- | | **学科领域** | 人工智能 (`cs.AI`) | | **引用格式** | `arXiv:2609.33134 [cs.AI]` | | **DOI** | [10.48550/arXiv.2609.33134](https://doi.org/10.48550/arXiv.2609.33134) | | **开源许可** | [Creative Commons Attribution 4.0 International](http://creativecommons.org/licenses/by/4.0/) | | **全文链接** | [查看 PDF](https://arxiv.org/pdf/2609.33134) \| [HTML 在线版本](https://arxiv.org/html/2609.33134v1) \| [TeX 源码](https://arxiv.org/src/2609.33134) | > | Field | Details | > | :--- | :--- | > | **Subjects** | Artificial Intelligence (`cs.AI`) | > | **Cite as** | `arXiv:2609.33134 [cs.AI]` | > | **DOI** | [10.48550/arXiv.2609.33134](https://doi.org/10.48550/arXiv.2609.33134) | > | **License** | [Creative Commons Attribution 4.0 International](http://creativecommons.org/licenses/by/4.0/) | > | **Full-Text Links** | [View PDF](https://arxiv.org/pdf/2609.33134) \| [HTML Version](https://arxiv.org/html/2609.33134v1) \| [TeX Source](https://arxiv.org/src/2609.33134) | https://insight.aitobox.com/blog/posts/2026-09-30/ProofLoom-基于证明义务驱动的随机优化理论形式化构建系统/ https://insight.aitobox.com/blog/posts/2026-09-30/ProofLoom-基于证明义务驱动的随机优化理论形式化构建系统/ Wed, 30 Sep 2026 00:00:00 GMT 在现代机器学习与优化理论中，形式化定理证明 (Formal Theorem Proving) 被视为保障数学论证绝对严谨的终极防线，但将其应用于复杂的随机优化 (Stochastic Optimization) 前沿研究始终面临严峻的工程与理论壁垒。研究人员不仅需要在 Lean 4 交互式证明器中构建极其严密的算法模型，还必须建立庞大的领域知识体系以衔接底层基础数学库与算法收敛性分析，在此过程中极易因对模型进行微小修改而无意篡改或削弱原有的数学结论。为此，研究团队推出了 **ProofLoom** —— 一个首创的、基于证明义务驱动 (Proof-Obligation-Driven) 的全自动大语言模型 (Large Language Model, LLM) AI 智能体 (AI Agent) 系统，它能够自主推演构建 Lean 4 算法模型及其支撑理论体系，并通过签名契约、独立裁判与形式化审计等多层安全护栏彻底杜绝虚假假设与结论降级。在横跨 15 项经典教材与顶会论文的评测中，ProofLoom 产出了逾 49 万行完全不含 `sorry` 占位符的形式化代码并大幅超越现有所有基准，更惊人的是，它在 22 篇已公开发表的顶会论文与经典著作中成功排查出 28 处推导漏洞、公式错误与算法建模不匹配，展现了形式化验证在科学理论发现与纠错中的巨大威力。 --- # ProofLoom：基于证明义务驱动的随机优化理论形式化构建系统 > # ProofLoom: Proof-Obligation-Driven Theory Construction for Autoformalizing Research-Level Stochastic Optimization **作者：** Feiming Wang, Daibo Li, Kun Yuan **主要学科领域：** 人工智能 (`cs.AI`) **次要学科领域：** 优化与控制 (`math.OC`) **arXiv 编号：** [`arXiv:2609.34960`](https://arxiv.org/abs/2609.34960) **提交日期：** 2026 年 9 月 28 日 **相关资源：** [PDF 原文](https://arxiv.org/pdf/2609.34960) | [GitHub 仓库与代码](https://github.com/Trace231/ProofLoom) > **Authors:** Feiming Wang, Daibo Li, Kun Yuan > **Primary Subject:** Artificial Intelligence (`cs.AI`) > **Secondary Subjects:** Optimization and Control (`math.OC`) > **arXiv ID:** [`arXiv:2609.34960`](https://arxiv.org/abs/2609.34960) > **Submission Date:** September 28, 2026 > **Resources:** [PDF View](https://arxiv.org/pdf/2609.34960) | [GitHub Repository & Code](https://github.com/Trace231/ProofLoom) --- ## 摘要概览 > ## Abstract Summary 要在 Lean 定理证明器中对前沿研究级的随机优化 (Stochastic Optimization) 理论进行形式化证明，绝非易事。这不仅需要建立高度严谨的算法模型，还需要搭建庞大的领域理论框架，将底层的基础数学库与前沿算法的收敛性证明紧密联结起来。在这一过程中，研究人员如果为了迁就机器证明的可行性而对模型进行微调，极易在无意之间改动底层原本要表达的核心数学命题。 > Formalizing research-level stochastic optimization within the Lean theorem prover requires both a rigorous algorithm model and an extensive domain theory that connects foundational math libraries to convergence proofs. Modifying a model to ensure provability can inadvertently alter the underlying mathematical claims. 为了攻克这一瓶颈，本文作者推出了 **ProofLoom** —— 一个全自动的大语言模型 (Large Language Model, LLM) AI 智能体 (AI Agent) 系统，专门用于实现**基于证明义务驱动的理论构建 (Proof-Obligation-Driven Theory Construction) **。 > To overcome this, the authors introduce **ProofLoom**, a fully automated Large Language Model (LLM) agent system designed for **Proof-Obligation-Driven Theory Construction**. https://insight.aitobox.com/blog/posts/2026-09-30/CoWindow-注意力-基于协同覆盖机制的高效长上下文注意力架构/ https://insight.aitobox.com/blog/posts/2026-09-30/CoWindow-注意力-基于协同覆盖机制的高效长上下文注意力架构/ Wed, 30 Sep 2026 00:00:00 GMT 在大语言模型 (Large Language Model, LLM) 处理超长文本的场景下，传统全注意力机制 (FullAttn) 会让每一个注意力头都重复扫描完整的因果历史 Token ，带来了庞大的冗余计算和显存带宽瓶颈。为攻克这一难题，本文研究团队创新性地提出了 **CoWindow Attention (CoWA)** 架构，将全因果历史的访问权限巧妙分散到不同的键值头 (KV-head) 中，变“单头重复全扫描”为“多头协同全覆盖”。在 CoWA 架构下，所有注意力头共同共享近邻对角窗口与前缀注意力汇聚点 (Prefix-Sink) ，而将剩余的长程历史切分为互补的滑动窗口；每个单独的注意力头虽只进行稀疏采样，但所有头聚合后的并集却能完整覆盖 100% 的因果上下文。这种基于确定性位置规则的设计无需复杂的动态路由模块，能天然契合张量并行加速；实测表明，在 128K 超长序列下，CoWA 训练反向传播延迟降低为原来的 $\frac{1}{8.6}$ ，解码显存开销降低为原来的 $\frac{1}{7.6}$ ，并在 14B 与 32B 规模下保持了完全媲美 FullAttn 的卓越性能。 --- # CoWindow 注意力：全因果覆盖是一种群体协同特性 > # CoWindow Attention: Full Causal Coverage Is a Collective Property **作者：** Jingze Shi, Zhangyang Peng, Xianduo Li, Yanlin Qi, Xiaotian Lin, Haoxian Chen, Liangdong Wang, Guang Liu, Yuyu Luo **arXiv ID：** [arXiv:2609.32704](https://arxiv.org/abs/2609.32704) [cs.AI] **提交时间：** 2026年9月26日 **许可协议：** [Creative Commons Attribution 4.0](http://creativecommons.org/licenses/by/4.0/) *(查看 [协议图示](./images/345c7ad61f1b.png))* > **Authors:** Jingze Shi, Zhangyang Peng, Xianduo Li, Yanlin Qi, Xiaotian Lin, Haoxian Chen, Liangdong Wang, Guang Liu, Yuyu Luo > **ArXiv ID:** [arXiv:2609.32704](https://arxiv.org/abs/2609.32704) [cs.AI] > **Submitted:** 26 September 2026 > **License:** [Creative Commons Attribution 4.0](http://creativecommons.org/licenses/by/4.0/) *(View [License Image](./images/345c7ad61f1b.png))* --- ## 📋 内容概要 > ## 📋 Summary 传统的注意力机制 (如 **FullAttn** ) 会让每一个注意力头都重复扫描完整的因果历史 Token ，这在底层硬件上引发了极其庞大的冗余计算和显存带宽开销。 > Traditional attention mechanisms (such as **FullAttn**) repeatedly expose the complete causal history to every single attention head, creating substantial redundant computation and memory traffic. 为了解决这个问题，作者团队推出了 **CoWindow Attention (CoWA)** 架构。这是一种结构化的注意力设计，它将因果历史的访问权限合理分摊到各个键值头 (KV-head) 上，而不是在每个头中进行机械复制： > To solve this, the authors introduce **CoWindow Attention (CoWA)**, a structured attention architecture that distributes access to the causal history across Key-Value (KV) heads rather than duplicating it: * **共享窗口 (Shared Windows) ：** 所有注意力头共同共享主对角线附近的近邻窗口以及前缀注意力汇 (prefix-sink) 窗口。 * **分片长程窗口 (Partitioned Long-Range Windows) ：** 剩余的历史上下文则被互补的长程窗口切分与分配。 * **协同全覆盖 (Collective Coverage) ：** 尽管每个独立的注意力头仅稀疏地关注远端 Token ，但它们组合并集后即可实现 100% 的全因果覆盖。 > * **Shared Windows:** All heads share near-diagonal and prefix-sink windows. > * **Partitioned Long-Range Windows:** Complementary long-range windows partition the remaining history. > * **Collective Coverage:** While each individual head attends sparsely to distant tokens, their union provides full causal coverage. 这种由位置规则直接决定的固定稀疏模式，既不需要额外训练路由器或动态索引器，在模型预训练与推理阶段也始终保持严格一致，同时能够天然无缝地契合 KV-head 的张量并行计算。 > This position-defined pattern requires no learned router or indexer, remains consistent across training and inference, and naturally aligns with KV-head tensor parallelism. --- ## 📊 核心发现与性能亮点 > ## 📊 Key Findings & Performance Highlights * **模型精度 (Accuracy) ：** 在 8K 上下文 Token 的同窗口消融实验中，具备 100% 协同覆盖特性的 CoWA 取得了 **89.73% 的准确率** ，与 FullAttn 的 **89.97%** 几乎完全持平，并且大幅超越了采用重复长程窗口的基线方案。 * **计算效率与运行提速 (Efficiency & Speed) ：** 在结合张量并行的 128K 超长序列注意力算子基准测试中，CoWA 相比 FullAttn 实现了显著的性能跃升： * 训练阶段前向传播延迟降低为原来的 **$\frac{1}{7.4}$** (加速 7.4 倍) ； * 训练阶段反向传播延迟降低为原来的 **$\frac{1}{8.6}$** (加速 8.6 倍) ； * 推理阶段解码延迟降低为原来的 **$\frac{1}{3.0}$** (加速 3.0 倍) 。 * **显存深度优化 (Memory Optimization) ：** 在训练期间，各计算卡 (Rank) 的峰值算子显存占用与 FullAttn 基本一致；而在推理自回归解码期间，峰值显存消耗大幅降低至原来的 **$\frac{1}{7.6}$** 。 * **缩放定律 (Scaling Laws) ：** 在覆盖 0.6B 到 14B 参数规模的缩放定律训练实验中，CoWA 在显著降低全流程训练计算量 (FLOPs) 的同时，其困惑度表现与 FullAttn 紧密契合。基于此训练出的 14B 与 32B 最终模型，在知识储备、逻辑推理以及长上下文检索等核心基准测试中的得分均与 FullAttn 模型不相上下。 > * **Accuracy:** In a window-matched ablation at 8K context tokens, CoWA achieves **89.73% accuracy** (with 100% collective coverage), compared to **89.97%** for FullAttn, significantly outperforming duplicated long-range window baselines. > * **Efficiency & Speed:** In a 128K-token attention-operator benchmark with tensor parallelism, CoWA improves performance over FullAttn by: > * **7.4× reduction** in training forward latency. > * **8.6× reduction** in training backward latency. > * **3.0× reduction** in inference decoding latency. > * **Memory Optimization:** Per-rank peak operator memory matches FullAttn during training and is **7.6× lower** during decoding. > * **Scaling Laws:** Across scaling-law training runs from 0.6B to 14B parameters, CoWA closely tracks FullAttn in perplexity while reducing total training FLOPs. Resulting 14B and 32B models achieve comparable knowledge, reasoning, and long-context retrieval scores to FullAttn models. --- ## 🔗 相关链接与学术资源 > ## 🔗 Links & Resources * [查看论文 PDF (View PDF)](https://arxiv.org/pdf/2609.32704) * [TeX 源码 (TeX Source)](https://arxiv.org/src/2609.32704) * [实验性 HTML 版本 (HTML Version)](https://arxiv.org/html/2609.32704v1) * [DOI 官方链接 (DOI Link)](https://doi.org/10.48550/arXiv.2609.32704) > * [View PDF](https://arxiv.org/pdf/2609.32704) > * [TeX Source](https://arxiv.org/src/2609.32704) > * [HTML Version (Experimental)](https://arxiv.org/html/2609.32704v1) > * [DOI Link](https://doi.org/10.48550/arXiv.2609.32704) https://insight.aitobox.com/blog/posts/2026-09-25/条件协同消融-CoAx-揭示-Transformer-电路中的自愈备份组件/ https://insight.aitobox.com/blog/posts/2026-09-25/条件协同消融-CoAx-揭示-Transformer-电路中的自愈备份组件/ Fri, 25 Sep 2026 00:00:00 GMT 在机械可解释性研究中，研究人员通常通过定位内部“电路”——即在因果层面支撑特定行为的关键组件集合——来探究 Transformer 大模型的运作机制。然而，这类模型普遍存在神奇的“自愈”能力：一旦切除或消融掉主要的执行组件，处于休眠状态的备用组件便会悄然激活并接管任务，导致原本看似完整的解释性电路在干预验证下出现遗漏和失真。针对这一干预盲区，研究团队提出了“条件电路补全”问题，并开发了全新的条件协同消融 (Conditional Co-Ablation, CoAx) 评估方法，通过衡量主要组件移除后候选组件消融效应的增长量，精准揪出潜藏的备份组件。实验表明，CoAx 不仅在 GPT-2 的间接宾语识别任务中以 0.941 的高 ROC-AUC 精准找回了全部已知备用头，还在涵盖 6 种主流架构家族的 8 款模型上展现了极佳的普适性，为构建更可靠、更完备的 Transformer 可解释性回路奠定了坚实基础。 --- # 条件协同消融 CoAx：揭示 Transformer 电路中的自愈备份组件 > # Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuits ## 核心概要 > ## Summary 在机械可解释性 (Mechanistic Interpretability) 研究中，研究人员通常通过定位内部“电路”——即在因果层面支撑特定行为的一组模型内部组件——来解释 Transformer 的运行机理。然而，Transformer 展现出了一种显著的 **自愈机制 (Self-Repair)**：当主要的核心组件被消融时，平时处于休眠状态的备用组件便会被主动激活，并迅速接管其功能。这在研究中造成了一个难以察觉的盲区：一个在完整模型中足以解释当前行为的电路，在用于检验它的干预测试下，却往往暴露出不完备性。 > In mechanistic interpretability, researchers often explain transformer behavior by identifying "circuits"—sets of internal model components that causally support a specific behavior. However, transformers exhibit **self-repair**: when primary components are ablated, dormant backup components can activate to take over their function. This creates a blind spot where a circuit explaining behavior in an intact model becomes incomplete under the interventions used to test it. 为了解决这一问题，研究团队形式化提出了 **条件电路补全 (Conditional Circuit Completion)** 任务——旨在专门寻找那些仅在主要核心组件集合被移除后才显现出因果重要性的隐藏组件。为此，他们推出了 **条件协同消融 (Conditional Co-Ablation, CoAx)** 方法。该方法的核心思路是：通过测量主要组件集合移除后候选组件消融效应的增长量，对所有潜在备用组件进行因果重要性排序。 > To address this, the authors formulate **conditional circuit completion**—the task of identifying components that become causally important only after a primary set is removed. They introduce **Conditional Co-Ablation (CoAx)**, a method that ranks candidate components by the growth in their ablation effect after primary-set removal. https://insight.aitobox.com/blog/posts/2026-09-25/有品味的智能体-长程任务中决策品味-Taste-的度量与提升/ https://insight.aitobox.com/blog/posts/2026-09-25/有品味的智能体-长程任务中决策品味-Taste-的度量与提升/ Fri, 25 Sep 2026 00:00:00 GMT 大语言模型 (Large Language Model, LLM) 智能体 (AI Agent) 正在被寄予厚望去攻克复杂的长程任务，但在漫长的探索过程中，面对岔路口选择哪条技术路线，往往直接决定了整项任务的成败。现有的评估基准普遍只看最终结果是否成功，忽视了智能体在关键转折点的中间判断力——作者将这种在长程任务中做出明智方向决策的能力定义为智能体的“决策品味 (Taste) ”。为填补这一评估空白，研究团队无需任何人工标注，直接从工程与科研任务轨迹中自动挖掘分叉点，构建了全新的自动化基准测试 (Benchmark) Taste-Bench。评测发现即便是当前最强大的前沿模型也难以应对这一挑战 (最高准确率仅为 59.7%) ，且简单增加推理算力无法改善品味；但令人振奋的是，通过蒸馏具有后验视角的教师模型判断力，可以有效培养智能体的品味，并在具挑战性的 SWE-bench Pro 上显著提升了端到端的任务成功率。 --- # 有品味的智能体：长程任务中决策品味 Taste 的度量与提升 > # The Tasteful Agent: Measuring and Improving Taste in Long-Horizon Tasks **arXiv 编号:** [2609.25804](https://arxiv.org/abs/2609.25804) [cs.AI] **论文作者:** Wenbo Pan, Zhichao Liu, Shujie Liu, Jingying Zeng, Chin-Yew Lin, Xianfeng Tang, Yan Lu, Qi He, Xiaohua Jia **提交时间:** 2026年9月22日 (修订于：2026年9月23日) > **arXiv:** [2609.25804](https://arxiv.org/abs/2609.25804) [cs.AI] > **Authors:** Wenbo Pan, Zhichao Liu, Shujie Liu, Jingying Zeng, Chin-Yew Lin, Xianfeng Tang, Yan Lu, Qi He, Xiaohua Jia > **Submitted:** September 22, 2026 (Revised: September 23, 2026) --- ## 📌 概要 > ## 📌 Summary 随着大语言模型 (Large Language Model, LLM) 智能体 (AI Agent) 面对日益复杂的多步骤长程 ("long-horizon") 任务，它们在探索推进过程中所做出的各项中间决策——例如决定优先检验哪一个科研假设，抑或选择哪种架构方案进行落地构建——在根本上主导了最终的成败。在本文中，作者将智能体在长程探索中审时度势、做出卓越决策的核心能力，生动地定义为智能体的**“品味 (Taste) ”**。 > As Large Language Model (LLM) agents tackle increasingly complex, multi-step ("long-horizon") tasks, their intermediate decisions—such as choosing which hypothesis to test or architectural implementation to build—ultimately dictate their success. The authors define the capability to make superior long-horizon decisions as an agent's **"taste."** 由于现有的评估基准测试 (Benchmark) 普遍只关注智能体最终是否跑通了端到端任务，而忽视了对其行进过程中的中间判断力进行考量，作者团队提出了 **Taste-Bench**——一个无需任何人工标注、完全从工程与科研真实任务轨迹中自动化挖掘构建的全新基准测试。评测结果揭示了一个耐人寻味的现象：即便是当前最先进的前沿大模型在面对这项指标时也显得力不从心 (最高准确率仅为 59.7%) ，而且单纯为模型分配更多的推理计算预算也无法真正提升其决策品味。不过令人振奋的是，研究团队证实品味*确实*能够借助知识蒸馏被有效习得，这不仅显著优化了智能体面对陌生未知任务时的决策水平，更直接带动了在高难度软件工程评测 SWE-bench Pro 上的端到端任务成功率。 > Because current benchmarks only evaluate end-to-end task completion rather than intermediate judgment, the authors introduce **Taste-Bench**, a novel automated benchmark mined from real agent trajectories in engineering and research tasks without human annotation. Their evaluations reveal that even frontier models struggle with this metric (topping out at 59.7% accuracy), and that simply allocating a larger reasoning budget does not improve taste. Finally, they demonstrate that taste *can* be successfully trained via distillation, improving both decision-making on unseen tasks and end-to-end success on SWE-bench Pro. --- ## 📑 概览与元数据 > ## 📑 Overview & Metadata * **主要研究领域：** 人工智能 (`cs.AI`) * **论文篇幅：** 33 页，8 幅图 * **相关资源：** * [代码仓库 (GitHub) ](https://github.com/wbopan/tastebench) * [数据集 (Hugging Face) ](https://huggingface.co/datasets/wenbopan/taste-bench) * [DOI 链接](https://doi.org/10.48550/arXiv.2609.25804) > * **Primary Subject:** Artificial Intelligence (`cs.AI`) > * **Length:** 33 pages, 8 figures > * **Resources:** > * [Code Repository (GitHub)](https://github.com/wbopan/tastebench) > * [Dataset (Hugging Face)](https://huggingface.co/datasets/wenbopan/taste-bench) > * [DOI Link](https://doi.org/10.48550/arXiv.2609.25804) --- ## 📝 摘要 > ## 📝 Abstract 大语言模型智能体在长程复杂任务中的应用正变得日益广泛，而它们在探索路径上所迈出的每一步中间抉择——比如优先验证哪个潜在假设，抑或选择在哪个技术方案的基础之上继续推进——直接决定了整个运行流程的最终结局。能够在这些关键节点上做出精准而富有远见的决策，正在成为工程型与科研型智能体不可或缺的核心竞争力。在本文中，我们将这种在长程任务中明辨方向、做出优异决策的高阶判断力，统称为智能体的“品味 (Taste) ”。 > LLM agents increasingly work on long-horizon tasks, and the decisions they make along the way, such as which hypothesis to test or which implementation to build on, determine the outcome of the whole run. Making these decisions well is becoming a key capability for both engineering and research agents. We refer to the ability to make good long-horizon decisions as the taste of an agent. 尽管现有的各类基准测试能够衡量智能体在长程任务中的端到端成功概率，但没有一项基准能够单独度量智能体的品味。针对这一空白，我们打造了 **Taste-Bench**——这是一个完全由智能体在真实工程与科研任务中生成的运行轨迹自动构建而成的品味评测基准。测试中的每一个问题都对应着一个关键的“决策分叉点 (Decision Fork) ”，即在任务推进的某一特定时刻，前方同时存在数条可行路径，且其中一条已被验证能导向更优的结果；被测模型必须在完全无法窥探分叉后续演变的前提下，仅凭当前上下文在多个方向中做出决断。 > While existing benchmarks measure the end-to-end success of agents on long-horizon tasks, none of them measures the taste of an agent. To address this problem, we build **Taste-Bench**, a benchmark of taste questions constructed automatically from trajectories that agents produced in engineering and research tasks. Each question presents a decision fork, a point in a trajectory where multiple directions are available and one of them leads to a better outcome, and the evaluated model chooses among these directions without seeing what happens after the fork. 我们在无需任何人工标注介入的情况下，通过比对同一任务的多条并行尝试轨迹，并挖掘单次探索中陷入弯路后修正折返的迂回片段，全自动化提取出了这些宝贵的分叉点。我们在 Taste-Bench 上系统评测了多款业界前沿模型，结果发现即便表现最佳的模型，其答题正确率也仅为 **59.7%**。深入分析表明，若决定某条路径优劣的关键证据出现在轨迹中较靠后的阶段，所有模型的判断难度都会急剧飙升；更关键的是，单纯增加思考时间或扩大推理计算预算，并不能实质性改善模型的判断准确率。最后，我们证实了品味并非不可捉摸，而是完全可以被后天训练的：通过将拥有完整执行后验视角的教师模型判断力蒸馏给学生模型，学生模型在未曾见过的全新任务中展现出了更优越的决策能力，并在严格留出的 SWE-bench Pro 复杂任务评测中大幅刷新了端到端解决成功率。 > We mine these forks automatically from parallel attempts at the same task and from detours inside a single trajectory, without needing human annotation. We evaluate frontier models on Taste-Bench and find that the best model answers only **59.7%** of the questions correctly. We further find that forks whose deciding evidence appears later in the trajectory are much harder for every model, and that a larger reasoning budget does not improve the accuracy. Finally, we show that taste can be trained. We distill the judgment of a teacher that has seen the outcome into a student model, and the student makes better decisions on unseen tasks and improves end-to-end success on held-out SWE-bench Pro tasks. https://insight.aitobox.com/blog/posts/2026-09-25/包管理器沙盒化演进-操作系统级隔离-代码移除与-AI-智能体供应链安全/ https://insight.aitobox.com/blog/posts/2026-09-25/包管理器沙盒化演进-操作系统级隔离-代码移除与-AI-智能体供应链安全/ Fri, 25 Sep 2026 00:00:00 GMT 长期以来，开发者在安装开源依赖包时默认赋予了其极高的系统执行权限，这使得安装脚本成为供应链投毒与恶意蠕虫攻击的重灾区。随着 AI 编码智能体 (AI Coding Agent) 的广泛普及，智能体自主拉取、安装未经人工审计的第三方包已成常态，进一步放大了这一安全风险。为此，以 Homebrew 为代表的主流包管理器正在发起一场深远的架构重塑：全面转向 Linux 内核级 Landlock 与 macOS Seatbelt 操作系统级沙盒，严格切分下载与安装阶段，甚至激进地彻底废弃任意可执行安装脚本，改由声明式签名数据驱动。这一技术演进标志着开发者生态正告别对“安装期任意代码”的盲目信任，构筑起抵御现代智能体时代软件供应链威胁的底层防御新标准。 --- ## 概述 > ## Summary 当前，主流包管理器正在经历一场重大的安全范式变革，其核心方向是转向 **操作系统级沙盒 (OS-level sandboxing)** 与 **彻底移除可执行代码 (code removal)**。在日益猖獗的供应链攻击 (如 Shai-Hulud 蠕虫病毒以及 nx 被黑事件) 威胁下，加之未经人工审查的大语言模型 (Large Language Model, LLM) 编码智能体日益普及并频繁自主安装外部依赖，开源项目维护者们正下定决心，彻底终结对软件包在安装期运行任意代码的盲目信任。 > Package managers are undergoing a significant security shift toward **os-level sandboxing** and **code removal**. Driven by escalating supply chain attacks (such as the Shai-Hulud worm and the nx compromise) and the parallel rise of unreviewed LLM coding agents, maintainers are moving away from trusting arbitrary install-time code. 诸如 Homebrew 7.0、opam 以及现代 JavaScript 包管理器等工具，纷纷开始推行极其严格的“下载与安装阶段物理隔离”，依托操作系统内核级防护屏障 (例如 Linux 上的 **Landlock** 与 macOS 上的 **Seatbelt**)，并针对生命周期钩子脚本设立严格的白名单机制。与此同时，另一条技术路线也在快速兴起——用结构化的声明式签名数据替代随意执行的脚本，从而在源头上将攻击面彻底归零。 > Tools like Homebrew 7.0, opam, and modern JavaScript package managers are implementing strict fetch/install phase splits, OS-level kernel boundaries (like **Landlock** on Linux and **Seatbelt** on macOS), and allowlists for lifecycle scripts. Meanwhile, a secondary strategy—replacing arbitrary executable scripts with declarative, signed data—is gaining traction to eliminate the attack surface entirely. --- ## 已发布的变革与架构演进 > ## Shipped Changes & Evolution Homebrew 7.0.0 推出了“更强沙盒机制” (stronger sandboxing)，将软件包安装过程严格拆分为两个阶段：支持网络访问的“下载抓取阶段” (fetch phase) 和完全离线、只读环境下的“编译安装阶段” (install phase)。在沙盒构建模式下，默认严禁读取用户的家目录；针对此前利用 LaunchServices 逃逸 cask 沙盒的漏洞，新版本通过显式禁止启动应用程序、拦截 Mach 服务调用以及限制 Unix Socket 连接进行了彻底修复。此外，以往在 Ruby formula 中允许随意编写的 `post_install` 代码块以及 cask 预运行脚本已被正式弃用；取而代之的是经过数字签名的声明式 `*_steps` 数据指令 (相关脚本执行能力的最终移除期限设定为 2027 年 12 月)。 > Homebrew 7.0.0 shipped with "stronger sandboxing," splitting package installation into a network-enabled fetch phase and an offline, read-only install phase. Sandboxed builds are now blocked from reading the user's home directory by default, and a cask sandbox escape via LaunchServices was patched by denying app launching, Mach services, and Unix socket connections. Furthermore, arbitrary Ruby formula `post_install` blocks and cask flight blocks are deprecated in favor of declarative `*_steps` delivered as signed data (with a removal deadline of December 2027). 尽管 Homebrew 早在 2015 年就开始利用 `sandbox-exec` 来隔离 macOS 下的 formula 构建过程，但直到最近数月，这一措施才被正式提升为系统性的安全防御工程： > Homebrew has run macOS formula builds under `sandbox-exec` since 2015, but recent months turned this into a formal security program: * **6 月**：Homebrew 6.0.0 引入了基于 Bubblewrap 的 Linux 构建沙盒，并要求用户对第三方软件源 (taps) 进行显式信任授权； * **7 月**：底层实现由 Bubblewrap 切换为内核原生内置的 **Landlock**，这使得沙盒能够在无特权的 Docker 容器环境中直接运转； * **8 月**：彻底移除 Bubblewrap 代码，并明确规定任何沙盒规则的调整都必须通过严格的正规安全审查。 > * **June:** Homebrew 6.0.0 introduced a Linux build sandbox using Bubblewrap and required explicit trust for third-party taps. > * **July:** Bubblewrap was replaced by **Landlock**, which is built into the kernel and operates inside unprivileged Docker. > * **August:** Bubblewrap was entirely removed, and sandbox rule changes now require formal security review. *注：对第三方源 (taps) 的显式信任依然是防范恶意 cask 的首道防线；沙盒机制主要用于遏制意外破坏，并为安装流程增添安全护栏。* > *Note: Tap trust remains the primary protection against malicious third-party casks; sandboxing merely limits accidental damage and adds installation safeguards.* --- ## AI 智能体与通用安全技术栈 > ## Agents and the Shared Security Stack Homebrew 探索出的这套沙盒架构，与过去一年中各大主流大语言模型 (LLM) 编程智能体厂商所采纳的防护体系不谋而合。诸如 Claude Code、Codex CLI、Cursor 和 Gemini CLI 等工具，普遍依赖 macOS 的 **Seatbelt** 以及 Linux 的 **Landlock/seccomp** 来收容与限制未经人工审查的代码。之所以必须下沉到操作系统内核级隔离，是因为频繁弹窗请求用户授权已被证明形同虚设 (Anthropic 的公开数据显示，多达 93% 的授权弹窗都会被用户不假思索地直接点击批准)。 > Homebrew's sandboxing mechanisms mirror those adopted by LLM coding agent vendors over the past year. Tools like Claude Code, Codex CLI, Cursor, and Gemini CLI rely on **Seatbelt** (macOS) and **Landlock/seccomp** (Linux) to contain unreviewed code because asking users for permission prompts every time proved ineffective (Anthropic reports users approve roughly 93% of prompts). AI 智能体面临的安全困境与包管理器的威胁模型存在着深刻的重叠，原因显而易见：**智能体会自主执行依赖包的安装**。近年来多起震动业界的攻击事件已经清晰地展示了恶意依赖包是如何轻而易举攻破开发者环境的： > The agent problem and package manager problem deeply overlap because **agents install packages**. High-profile attacks have demonstrated how malicious packages can compromise developer environments: * **Shai-Hulud 蠕虫病毒**：通过 npm 的 `postinstall` 安装后钩子脚本大肆扩散，并通过向 Claude Code 的本地配置文件注入恶意钩子实现长期隐蔽潜伏； * **nx 供应链入侵事件**：攻击者在依赖包中夹带恶意载荷，试图调用本地 AI 工具刺探并窃取开发者的凭据与密钥； * **编程助手会话劫持**：攻击者通过污染会话上下文，操纵 AI 编码助手向企业内部代码仓库主动推荐含有后门的恶意依赖包。 > * The **Shai-Hulud worm** spread via npm postinstall scripts and persisted by injecting hooks into Claude Code's settings file. > * The **nx compromise** shipped payloads attempting to leverage local AI tools to harvest credentials. > * Attackers have hijacked coding assistant sessions to propagate poisoned package recommendations across internal repositories. 这种安全技术的相互借鉴与跨界融合在多个开源项目中随处可见，例如 Homebrew (支持 `brew sandbox-exec` 以及嵌套沙盒的处理机制)、pnpm (社区中一份由 AI 智能体撰写的 Issue 提议使用 Bubblewrap、Landlock 和 Seatbelt 对脚本进行安全封装) 以及 Deno 等运行时环境。 > This cross-pollination is visible in projects like Homebrew (`brew sandbox-exec` and nested-sandbox handling), pnpm (proposing script wrapping in bubblewrap/Landlock/Seatbelt via an agent-authored issue), and Deno. --- ## 生态现状全景盘点 > ## State of the Ecosystem https://insight.aitobox.com/blog/posts/2026-09-25/QuantWM-面向世界模型与视频生成的时序一致性-2-Bit-KV-Cache-量化技术/ https://insight.aitobox.com/blog/posts/2026-09-25/QuantWM-面向世界模型与视频生成的时序一致性-2-Bit-KV-Cache-量化技术/ Fri, 25 Sep 2026 00:00:00 GMT 在视频生成与物理世界模型中，用于保存历史上下文的键值缓存 (KV Cache) 显存开销随着生成时长的增加而急剧膨胀，成为模型在实际部署中的关键瓶颈。尽管现有的 2-bit KV Cache 极限量化方案在静态基准测试 (如 VBench) 上表现优异，但在实际视频生成中仍频繁出现严重的画面闪烁与时序不一致问题。针对这一难题，新加坡管理大学等机构的研究团队深入探究了注意力机制的误差分布特性，提出了严格因果且免训练的 2-bit KV Cache 量化框架 —— QuantWM。该方案创新性地结合量化敏感度感知聚类 (QSAC) 与主子空间注意力补偿 (PSAC) 算法，精准稳定了注意力输出并锁定时空 Token 选取。实验表明，QuantWM 在实现高达 6.20 倍显存压缩的同时显著提升了生成画面的时序连贯性与视觉品质，为长视频实时生成与具身智能世界模型的高效落地提供了强大的技术支撑。 --- # QuantWM：面向世界模型与视频生成的时序一致性 2-Bit KV Cache 量化技术 > # QuantWM: Temporally Consistent 2-Bit KV Cache Quantization for World Models and Video Generation **作者：** Jiaqi Zhao, Xiaobin Hu, Bo Yin, Junpeng Jiang, Miao Zhang, Shuicheng Yan **研究领域：** 计算机视觉与模式识别 (`cs.CV`)；人工智能 (`cs.AI`) **arXiv：** [2609.26425 [cs.CV]](https://arxiv.org/abs/2609.26425) | **DOI：** [10.48550/arXiv.2609.26425](https://doi.org/10.48550/arXiv.2609.26425) **提交历史：** 于 2026 年 9 月 22 日提交；最后修订于 2026 年 9 月 23 日 (v2)。 > **Authors:** Jiaqi Zhao, Xiaobin Hu, Bo Yin, Junpeng Jiang, Miao Zhang, Shuicheng Yan > **Subjects:** Computer Vision and Pattern Recognition (`cs.CV`); Artificial Intelligence (`cs.AI`) > **arXiv:** [2609.26425 [cs.CV]](https://arxiv.org/abs/2609.26425) | **DOI:** [10.48550/arXiv.2609.26425](https://doi.org/10.48550/arXiv.2609.26425) > **Submission History:** Submitted on 22 Sep 2026; last revised 23 Sep 2026 (v2). --- ## 📌 内容概要 > ## 📌 Summary 对于现代视频生成大模型与世界模型而言，键值缓存 (Key-Value Cache, KV Cache) 极其庞大的显存开销已成为落地部署的核心瓶颈。尽管现有的 2-bit KV Cache 量化技术在诸如 VBench 等标准基准测试 (Benchmark) 中看似取得了“近乎无损”的分数，但在实际生成连续视频时，画面依然存在严重的**时序闪烁**与**画质劣化**。 > Key-Value (KV) cache memory is a significant deployment bottleneck for modern video generation models and world models. While existing 2-bit KV cache quantization methods achieve near-lossless performance on standard benchmarks (such as VBench), they still suffer from severe **temporal flickering** and **visual degradation**. **核心发现与洞察：** > **Key Findings & Insights:** * **Key 与 Value 扰动对比：** 深入分析表明，尽管 Key 向量量化带来的数值重构误差实际上小于 Value 向量，但令人意外的是，它所引发的最终生成质量劣化反而要严重得多。 * **注意力机制的关键影响：** Key 向量中即便极其微小的量化扰动，也会直接改变注意力分数的计算结果 ($QK^\top$)，进而导致 Query 在检索历史信息时所选取的时空 Token 发生漂移，最终彻底打破了视频帧之间的时序一致性。 > * **Key vs. Value Perturbations:** Deeper investigations reveal that Key quantization produces smaller reconstruction errors than Value quantization, yet surprisingly leads to significantly larger output degradation. > * **The Role of Attention:** Small Key perturbations alter attention logits ($QK^\top$), shifting the temporal-spatial tokens selected by Queries and breaking temporal consistency. **解决方案 (`QuantWM`)：** 为了在无需重新训练模型的前提下，显式保护注意力分数与 Token 选取的准确性，作者团队推出了名为 **QuantWM** 的免训练、严格因果 2-bit KV Cache 量化框架。QuantWM 核心依赖两项相辅相成的关键技术： > **Proposed Solution (`QuantWM`):** > To explicitly preserve attention logits and token selection without requiring model training, the authors introduce **QuantWM**, a training-free, strictly causal 2-bit KV cache quantization framework. QuantWM relies on two complementary techniques: 1. **量化敏感度感知聚类 (Quantization-Sensitivity-Aware Clustering, QSAC)：** 综合考量历史 Query 的敏感度分布与残差动态范围，精心筛选适合 INT2 表示的 Key 聚类质心，最大限度减少对注意力计算起决定性作用的敏感通道上的量化误差。 2. **主子空间注意力补偿 (Principal-Subspace Attention Compensation, PSAC)：** 通过低秩投影技术，沿着起主导作用的 Query 主子空间精确补偿 Key 的剩余量化误差，以极高的计算效率直接稳定注意力分数的输出。 > 1. **Quantization-Sensitivity-Aware Clustering (QSAC):** Jointly considers historical Query sensitivity and residual ranges to select INT2-friendly Key centroids, minimizing quantization errors in attention-critical channels. > 2. **Principal-Subspace Attention Compensation (PSAC):** Restores remaining Key errors along the dominant Query subspace via low-rank projections, directly stabilizing attention logits efficiently. --- ## 🔬 实验结果 > ## 🔬 Experimental Results 在包括 **Causal-Forcing**、**LingBot-World-v2**、**HY-World 1.5**、**Matrix-Game-2** 以及 **Longcat-Video** 在内的多种业界主流架构上开展的广泛评估表明，`QuantWM` 能够： > Extensive evaluations across multiple prominent architectures—including **Causal-Forcing**, **LingBot-World-v2**, **HY-World 1.5**, **Matrix-Game-2**, and **Longcat-Video**—demonstrate that `QuantWM`: * 大幅改善生成视频的视觉品质与时序一致性； * 在图像级与视频级质量评测指标上均全面超越现有方法； * 在引入极小额外计算开销的前提下，实现高达 **6.20 倍的 KV Cache 显存压缩**。 > * Substantially improves visual quality and temporal consistency. > * Outperforms existing methods across both image and video quality metrics. > * Achieves up to **6.20× KV cache memory compression** with minimal additional computational overhead. --- ## 🔗 论文链接与相关资源 > ## 🔗 Links & Resources * **全文访问：** [查看 PDF](https://arxiv.org/pdf/2609.26425) | [在线 HTML (实验性)](https://arxiv.org/html/2609.26425v2) | [TeX 源码](https://arxiv.org/src/2609.26425) * **授权协议：** [知识共享署名 4.0 国际许可协议 (Creative Commons Attribution 4.0)](http://creativecommons.org/licenses/by/4.0/) *(根据说明保留下方协议图标)*： <img alt="license icon" role="presentation" src="./images/345c7ad61f1b.png"> > * **Full-Text Access:** [View PDF](https://arxiv.org/pdf/2609.26425) | [HTML (Experimental)](https://arxiv.org/html/2609.26425v2) | [TeX Source](https://arxiv.org/src/2609.26425) > * **License:** [Creative Commons Attribution 4.0](http://creativecommons.org/licenses/by/4.0/) *(License icon preserved below per instructions)*: https://insight.aitobox.com/blog/posts/2026-09-25/Linux-程序对-proc-目录的隐秘依赖远超想象-systemd-沙盒限制与审计实战/ https://insight.aitobox.com/blog/posts/2026-09-25/Linux-程序对-proc-目录的隐秘依赖远超想象-systemd-沙盒限制与审计实战/ Fri, 25 Sep 2026 00:00:00 GMT 在现代 Linux 系统安全加固中，systemd 提供了诸如 `ProcSubset=pid` 等沙盒化配置，试图对受限服务屏蔽除自身进程以外的 `/proc` 虚拟文件信息。然而，这种隔离策略极易引发意料之外的程序故障。通过借助 Linux 审计子系统 (Linux Audit Subsystem) 追踪系统调用，我们发现诸如 Go、Python、Rust 等语言运行时和常见系统工具在冷启动时对 `/proc` 和 `/sys` 存在着极其深入且隐蔽的访问依赖，用于并发配额检测或内存分配调优。本文深入剖析了这些依赖背后的原因，并演示了如何使用审计规则精准监控此类行为，为服务器安全沙盒构建与服务排错提供实践指引。 --- ## 概要 > ## Summary 当 systemd 启用类似 `ProcSubset=pid` 的沙盒选项限制服务视图时 (该选项会隐藏除任务和进程本身之外的所有内容) ，往往会导致诸多意料之外的程序异常。通过使用 Linux 审计子系统 (Linux Audit Subsystem) ，我们可以直观地洞察到常见的 Linux 应用程序、语言运行时以及基础工具在启动之初对 `/proc` 和 `/sys` 的依赖程度究竟有多深——它们通常需要借此完成并发调优、内存管理或底层环境探测。 > When systemd restricts a service's view of `/proc` using options like `ProcSubset=pid` (which hides everything except task- and process-related information), it can cause unexpected issues. Using the Linux audit subsystem, we can discover just how heavily common Linux programs, language runtimes, and standard utilities rely on reading `/proc` and `/sys` on startup—often for concurrency tuning, memory management, or environment detection. --- ## 对 /proc 的隐秘依赖 > ## The Hidden Reliance on `/proc` 在探讨 Ubuntu 26.04 中 systemd 的 `apache2.service` 是如何 [限制 Apache](https://utcc.utoronto.ca/~cks/space/blog/linux/Ubuntu2604ApacheRestrictions) 时，一个引人注目的配置项是 [`ProcSubset=pid`](https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html#ProcSubset=)。该配置会将服务在 `/proc` 目录下所能看到的内容大幅精简，除了与当前任务和进程相关的条目之外，其余信息全部对服务隐匿 (具体行为可参考 [Linux 内核文档](https://docs.kernel.org/filesystems/proc.html#mount-options) 说明) 。 > Yesterday's exploration into how Ubuntu 26.04's systemd `apache2.service` [restricted Apache](https://utcc.utoronto.ca/~cks/space/blog/linux/Ubuntu2604ApacheRestrictions) highlighted the use of [`ProcSubset=pid`](https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html#ProcSubset=). This setting removes everything from a service's view of `/proc` apart from task- and process-related entries (as outlined in [the kernel documentation](https://docs.kernel.org/filesystems/proc.html#mount-options)). systemd 官方文档明确警告过，这项配置可能会对正在运行的程序产生影响。出于好奇，想弄清楚普通程序究竟有多频繁地查询 `/proc`，我借助 Linux 审计子系统进行了监测，而得到的答案令人印象深刻：**其频率相当可观。** > Systemd's documentation explicitly warns that this may affect running programs. Curious to see how often regular programs query `/proc`, I used the Linux audit subsystem to find the answer: **a fair bit.** 虽然 [proc(5)](https://www.man7.org/man7/linux/man-pages/man5/proc.5.html) 以及相关手册页已经详尽阐述了系统的各项信息结构，但程序对这些文件的直接读取规模仍然十分惊人： > While system information is broadly covered by [proc(5)](https://www.man7.org/man7/linux/man-pages/man5/proc.5.html) and related manual pages, the scale of direct file reads is surprising: * **网络与系统负载 (Network & Uptime) ：** `ifconfig` 会检查 `/proc/net`；`ip` 命令会读取 `/proc/filesystems`；而 `uptime` 则会读取 `/proc/uptime` 和 `/proc/loadavg` (尽管 [`getloadavg(3)`](https://www.man7.org/linux/man-pages/man3/getloadavg.3.html) 似乎是直接调用了 [`sysinfo(2)`](https://man7.org/linux/man-pages/man2/sysinfo.2.html)) 。 > * **Network & Uptime:** `ifconfig` checks `/proc/net`, the `ip` command reads `/proc/filesystems`, and `uptime` reads `/proc/uptime` and `/proc/loadavg` (though [`getloadavg(3)`](https://www.man7.org/linux/man-pages/man3/getloadavg.3.html) appears to call [`sysinfo(2)`](https://man7.org/linux/man-pages/man2/sysinfo.2.html) directly). https://insight.aitobox.com/blog/posts/2026-09-25/Cursor-智能体工程实践-长程任务-Token-效率与系统-Harness-深度优化指南/ https://insight.aitobox.com/blog/posts/2026-09-25/Cursor-智能体工程实践-长程任务-Token-效率与系统-Harness-深度优化指南/ Fri, 25 Sep 2026 00:00:00 GMT 随着 AI 智能体在真实开发场景中承担起越来越复杂、跨度更长的编程任务，如何高效组装并管理庞大的上下文，已成为决定大模型推理效率与成本的核心瓶颈。在过去数月里，Cursor 团队针对其智能体底层系统架构 (Agent Harness) 展开了全方位深度重构与多层优化，在丝毫不牺牲任务完成质量的前提下，成功将整体 Token 开销降低了 7%。这项工程涵盖了精简系统提示词、动态按需加载工具、结合 GPT-5.6 显式断点优化 Prompt 缓存、压缩文件读取行号开销，以及制定更智能的子智能体分流策略等多项突破。本文详细复盘了 Cursor 在超大规模真实流量下的上下文工程实战经验，为构建低成本、高可靠的长程 AI 智能体系统提供了极具价值的工程参考。 --- ## 概述 > # Summary 随着 AI 智能体 (AI Agent) 开始接管更具挑战性、耗时更长的复杂任务，如何高效组织和管理上下文 (Context)，已成为决定大模型推理效率的核心生命线。在过去的几个月里，Cursor 团队针对其智能体底层运行架构 (Agent Harness) 展开了多层面的深度系统优化——在丝毫不降低任务输出质量的前提下，成功将全局 Token 成本削减了 7%。其中的关键优化举措包括： > As AI agents tackle more ambitious, longer-running tasks, the way context is assembled and managed becomes critical to inference efficiency. Over the past few months, Cursor optimized its agent harness across multiple layers—reducing token costs by 7% overall without degrading quality. Key optimizations include: * **精简系统提示词 (Trimming System Prompts)**：随着基础模型自身能力的飞速提升，移除了冗余的指令约束与防护栏，将系统提示词 (System Prompt) 大幅缩减了约 66%； * **动态工具加载 (Dynamic Tool Loading)**：将低频使用的内置工具以及 MCP 工具从静态上下文中剥离，仅在需要调用时动态按需加载； * **深度优化 Prompt 缓存 (Optimized Prompt Caching)**：充分利用 GPT-5.6 引入的显式缓存断点 (Cache Breakpoints)，并通过稳定请求前缀，将冷缓存未命中率降低了 20%； * **压缩代码文件读取开销 (Compressed File Reads)**：在读取文件内容时，从过去的逐行编号改为每十行标注一次行号，在保证模型精准定位的前提下，将缓存读取 Token 减少了 1.6%； * **智能化子智能体协作策略 (Strategic Subagent Usage)**：重新梳理了子智能体 (Subagent) 的派发时机、协作方式与模型选型逻辑，有效避免了不必要的跨智能体协同开销。 > * **Trimming System Prompts:** Removed redundant instructions and guardrails as models became natively more capable, cutting system prompts by roughly 66%. > * **Dynamic Tool Loading:** Offloaded low-frequency built-in tools (and MCP tools) from static context, loading them only when needed. > * **Optimized Prompt Caching:** Leveraged explicit cache breakpoints (introduced in GPT-5.6) and stabilized the request prefix to reduce cold cache misses by 20%. > * **Compressed File Reads:** Switched from numbering every single line of code in file reads to numbering every tenth line, reducing cache-read tokens by 1.6%. > * **Strategic Subagent Usage:** Streamlined when and how subagents are deployed and which models they use, avoiding unnecessary coordination overhead. --- ## 精简系统提示词 > ## Trimming the system prompt 在智能体与大模型的每一次交互轮次中，Cursor 都会在模型正式执行任务前注入预设的上下文。这其中包括系统提示词以及智能体可调用工具的详细定义。由于这部分固定上下文会贯穿整个会话始终，它逐渐成为了我们完全可控的成本支出中最庞大的一项。 > Every agent turn includes context supplied by Cursor before the model begins working. This includes the system prompt and definitions for the tools the agent can use. Because this context is included throughout a conversation, it had become one of the largest sources of spend that we fully control. 早先在大模型能力尚显稚嫩的时期，我们不得不巨细靡遗地在提示词中详尽交代各项规则：如何调用工具、如何规划任务步骤，以及如何执行代码修改的工作流程。不仅如此，我们还得设置层层严密的防护栏，严防各种偶发的异常行为——比如模型突然倾倒出极其冗长的哈希码、吐出不可读的二进制数据，或是滥用 Emoji 表情符号等。 > When models were less capable, we had to spell out instructions for tool usage, task management, and code-change workflows. We also had to guard against strange behaviors like extremely long hash dumps, binary output, and emojis. 而随着大模型底座能力的快速进化，许多昔日的保姆式指导已不再必要。与其用冗长清单罗列一堆诸如“严禁执行某操作”、“你必须”、“极度重要”等警告，我们如今只需清晰定义工具的具体行为特征，各类模型便能心领神会、规范遵循。这种原生理解能力的跃升在各个主流模型家族中均得到了印证，使我们得以大刀阔斧地削减了约 66% 的系统提示词篇幅。 > As models improved, much of that direction became unnecessary. Instead of long lists of "DO NOT do this," "You must," or "Important" instructions, we could simply define how a tool behaves and models would generally comply. This was true across model families, allowing us to trim roughly 66% of our system prompt. 随着新模型的迭代引入，我们始终动态增删各项指令，引导新模型发挥最佳表现，而这些实战经验后续也会反馈到下一代模型的训练流程之中。依托海量真实用户群体进行严谨的 A/B 测试，是针对真实线上流量高效优化系统 Harness 的关键法宝。离线评测基准 (Evals) 固然是一种快捷实用的参考代理，但它们往往侧重于刻意设计的“高难度”极限场景，无法准确映射现实世界中用户请求的真实分布情况。 > Over time, we continue to add and remove instructions as new models require new guidance, which then flows into the training of future models. Leveraging A/B tests on a large user base is crucial to effectively optimizing the harness for real traffic. While evals can be a fast and useful proxy, they often represent "hard" problems and don't properly reflect the true distribution of user requests. --- ## 动态按需加载工具 > ## Loading tools only when needed 系统提示词仅仅是 Cursor 在每一轮交互中向模型传递的上下文的一部分。另一个关键开销来源是工具定义。过去一年间，随着我们为 Cursor 智能体注入了越来越多强劲的新特性 (例如后台终端进程监控、云端子智能体调度，以及更稳健的网页内容抓取能力)，工具定义的体积出现了急剧膨胀。尽管这些工具在特定场景下都极为关键，但实际上绝大多数工具在不到 20% 的会话中才会被真正调用。 > The system prompt is only one part of the context Cursor supplies on every turn. Another is tool definitions, which had grown dramatically over the course of the year as we added more powerful capabilities to the Cursor agent, including background shell monitoring, cloud subagents, and more reliable access to web content. Most of these tools are important, but each is needed in fewer than 20% of conversations. 这就为我们带来了一个绝佳的效率优化契机：既能确保所有工具随时待命可用，又无需在每次向模型发送请求时都塞入冗长的完整工具定义。今年早些时候，我们通过将 MCP 工具移入动态上下文，仅在实际触发时才按需加载，成功解决了类似的技术痛点。那一项优化为调用了 MCP 工具的会话总体降低了高达 46.9% 的 Token 消耗。 > That created an opportunity to improve efficiency by keeping tools available without including their full definitions in every request. We'd solved a similar problem earlier this year when we moved MCP tools into dynamic context, loading them only when needed. This reduced total tokens by 46.9% across sessions that called an MCP tool. 如今，我们将这一成熟的设计思路全面推广到了 Cursor 自身的原生内置工具上。 > We have now applied the same technique to our own built-in tools. 为了精准权衡哪些工具应该保留在静态上下文中，我们基于工具的调用频率以及模型是否必须从一开始就看到工具定义，设计了多组配置方案展开广泛的 A/B 测试。在测试过程中，我们密切追踪了 Token 使用量、实际调用成本、响应延迟、工具调用报错率以及智能体的整体活跃度指标，严格确保在大幅压缩成本的同时，开发体验和任务完成质量没有发生丝毫滑坡。 > To decide which tools to keep in static context, we A/B tested several configurations based on how often each tool was used and whether models needed to see it from the start. We tracked token usage, cost, latency, tool-call errors, and overall agent usage to make sure the savings did not degrade quality. 最终评测的结果是：我们将代码读取、全局检索、文本编辑以及 Shell 终端执行等高频核心工具常驻在静态上下文中。此外，我们也保留了 `ask_question` (因为某些模型在看不到该工具时容易产生臆想调用的幻觉问题)，以及对特定产品交互流至关重要的工具 (例如规划模式 Plan Mode 下的 `create_plan`)。除此以外的所有剩余工具，如今全部改为在智能体明确需要时才动态加载。 > Ultimately, we kept the high-frequency tools for reading, searching, editing, and using the shell in static context. We also retained `ask_question`, which some models tended to hallucinate calls for, and tools that are crucial to specific product flows, such as `create_plan` in Plan Mode. The remaining tools now load when the agent needs them. --- ## 提升 Prompt 缓存复用率 > ## Improving cache reuse 在成功精简了单次请求中承载的静态上下文体积之后，我们的下一步重心转向了如何最大化提升跨轮次重复上下文的缓存利用效率。 > After reducing the amount of static context in each request, we improved how effectively repeated context could be cached across turns. 在智能体的每个交互回合中，客户端都会向后端发送一个体量可观的完整请求，其中打包了工具列表、系统指令、运行环境配置以及迄今为止的全部会话历史。显而易见，请求前部的大块内容在多轮对话中几乎是一成不变的，只有尾部的对话记录会随着任务推进而不断延展拉长。 > Every agent turn resends a long request containing tools, system instructions, setup, and the conversation so far. Much of the beginning stays the same from one turn to the next, while the conversation at the end continues to grow. Prompt 缓存 (Prompt Caching) 技术允许模型服务商直接复用未发生改变的请求前缀，从而大幅节省预填充计算。然而，各大服务商在缓存的可配置粒度上差异显著。在 GPT-5.6 问世之前，缓存边界完全由服务端根据最新请求的内容自动推导截取。尽管工具和系统提示词本身极少变动，但此前并无法被显式打上独立复用的标记。 > Prompt caching allows the model provider to reuse that unchanged prefix. However, caching configurability can vary by provider. Before GPT-5.6, the cache boundary was determined automatically based on the latest request. Even though tools and system instructions rarely changed, they were not cleanly marked as reusable on their own. 自从 GPT-5.6 发布以来，OpenAI API 在其默认的隐式缓存机制之外，正式开放了让客户端自定义标记显式缓存断点 (Cache Breakpoints) 的能力。现在，我们在请求中极其稳定的前缀层之后、不断变长的会话内容之前，精准插入了缓存断点。这一改动让后续的交互轮次能够更稳、更多地命中并复用未经篡改的前置静态缓存。 > Since GPT-5.6, the OpenAI API allows clients to mark explicit cache breakpoints alongside its default implicit caching. We now place breakpoints after stable layers of the request and before the growing conversation, allowing later turns to reuse more of the unchanged prefix. <div class="media-border-container overflow-hidden"> <img alt="Diagram showing explicit cache breakpoints separating stable request context from the growing conversation" class="block dark:hidden" src="./images/a2a4fedd0b5c.png" width="100%"/> <img alt="Diagram showing explicit cache breakpoints separating stable request context from the growing conversation" class="hidden dark:block" src="./images/c39d5971789c.png" width="100%"/> </div> 需要注意的是，只有当前缀本身保持高度稳定时，缓存断点才能真正发挥功效。为此，我们对每一次请求前部放置的内容进行了更为严格的规范与收紧：系统提示词和工具定义区域严格仅存放极少变更的基准内容；而对于经常浮动变化的配置信息，则被统统推移到缓存断点之后，集中收纳至我们专门设计的“虚拟用户消息 (Phantom User Message)”中。该消息专门承载与具体用户或单次请求强相关的动态上下文，例如自定义技能 (Skills)、子智能体配置以及即时环境变量等。 > Breakpoints only help if the prefix itself stays stable, so we also tightened what sits at the front of each request. We did this by reserving tools and system instructions for content that rarely changes, and by moving more variable setup past the cache boundaries into our "phantom user message." This holds user- and request-specific context like skills, subagents, and environment info. 这一套组合拳打下来，成功将冷缓存未命中率降低了 20%。 > These changes reduced the rate of cold cache misses by 20%. --- ## 压缩文件读取开销 > ## Compressing file reads 智能体在自主运行过程中动态追加的工作上下文，是另一处极其庞大的 Token 消耗重灾区，而其中绝大部分开销都源于对代码文件的频繁读取。 > Another large source of token spend is the context an agent adds as it works, much of which comes from reading files. Cursor 智能体通过专门的 `Read` 工具读取文件内容。在传统实现中，该工具会为代码的每一行都打上行号。之所以这样做，是因为大语言模型自身并不擅长精准数行数，而后续向用户解释说明或执行代码替换时，又必须精确引用具体的代码行区间。 > Cursor's agent reads files through a `Read` tool, which traditionally numbered every line because models are not good at counting lines on their own and need to cite specific sections for the user. 单纯标注一行行号仅仅耗费大约 3 到 5 个 Token，看似微不足道；但当智能体在一个复杂的工程会话中一口气通读数万行代码时，每一行都加上行号所累加起来的上下文体积就相当惊人了。 > A single line number uses only around three to five tokens, but when an agent reads tens of thousands of lines during a session, numbering every one adds a meaningful amount of context. 为了减轻这一不必要的负担，我们对行号渲染进行了稀疏化改造：改为仅在每十行代码处标注一次行号。实测表明，这一采样频率已完全足够模型准确推导并引用目标代码行，同时在不产生任何质量劣化的情况下，直接将读取文件产生的缓存读取 Token 减少了 1.6%。 > We reduced that overhead by including line numbers only on every tenth line. This is still frequent enough for models to cite code properly, and the change reduced cache-read tokens by 1.6% with no reduction in quality. --- ## 科学调度子智能体 > ## Using subagents strategically 随着智能体执行长周期任务的时间不断增加，将子任务委托给子智能体 (Subagent) 协作的机会也随之增多。这本身能够显著削减 Token 开销，因为每个子智能体在被派生时，通常都会启动一个崭新而干净的上下文窗口，而无需背负父智能体漫长繁杂的历史对话包袱；当子智能体完成任务并汇报结论后，父智能体只需接收核心结果，即可继续向下执行，避免了将子智能体繁琐的中间工作轨迹全盘塞回主会话。 > Longer agent runs create more opportunities to delegate work to subagents. This can reduce token spend because each subagent typically starts with a fresh context window rather than carrying the parent agent's full conversation. Once it reports its results, the parent can continue without carrying the subagent's full working context. 然而，父子智能体之间的这种上下文隔离并非毫无代价——它不可避免地会带来“协同税 (Coordination Tax)”。由于彼此无法实时共享完整的上下文，子智能体很可能会重复执行某些已被验证过的工作，或是朝着主任务中早已废弃的方向徒劳探索。 > This kind of context isolation between agents and subagents does carry a coordination tax, though, because agents that do not share context can duplicate work or pursue tasks that are no longer necessary. 为了在充分享受子智能体带来的效率红利的同时避免过高的协同损耗，我们做出了两项关键调整。首先，我们果断删除了原本提示词中强烈鼓励智能体在探索代码库时派生子智能体的强制指令。随着子智能体范式在预训练数据中愈发普及，以及研究人员在后训练 (Post-Training) 阶段对其深度对齐，当今的模型已经原生掌握了何时应该分工。移除这些多余的催促提示词后，智能体在子智能体的调用决策上反而变得更加克制而均衡。 > We made two changes to capture the efficiency benefits without adding unnecessary coordination. First, we removed instructions that strongly encouraged agents to use subagents for codebase exploration. As subagents became more prevalent in training data and researchers incorporated them into post-training, models learned this pattern natively. Removing the extra prompting produced more balanced subagent usage. 其次，我们进一步收紧了子智能体的模型选择机制。Cursor 本身支持选用后台任意可用的大模型来孵化子智能体，这使得不同模型之间互补短板成为可能——例如将高昂的顶尖规划模型与轻量便宜的模型搭配，分别负责顶层构思与底层代码落地。我们重新规范了工具调用参数：智能体只有在收到用户明确指示或系统 Harness 统一调度时，才会切换选用不同的模型，杜绝了模型随意自我分流带来的失控与溢价。 > We also tightened how subagents select models. Cursor can spawn subagents using any of our available models, which makes it possible to shore up blind spots across models or pair an expensive planning model with a cheaper one for implementation. We updated the tool arguments so agents choose a different model only when directed by the user or the harness. --- ## 持续深耕系统 Harness 的极致能效 > ## Continuing to improve harness efficiency 未来，我们将继续密切度量上下文在超长程运行中的堆积规律，并持续探索如何在绝不折损智能体产出质量的前提下，通过系统 Harness 进一步剔除多余的重复计算。随着技术的演进，我们期待 Token 的消耗增速将远低于智能体实际完成的工作量增速。与此同时，我们已将这些宝贵的工程实战经验复用到 Grok Bot 等项目中，致力于为其量身定制最适配的 Harness 优化策略，让用户能够以最低的成本开销交付最高价值的工程成果。 > We'll continue measuring how context accumulates across longer runs and testing where the harness can reduce repeated processing without affecting agent quality. Over time, we expect this will allow token use to grow far more slowly than the amount of work agents can complete. We've also taken these learnings to Grok Bot, where we're working to optimize its unique harness so that users can accomplish the most work at the lowest cost. https://insight.aitobox.com/blog/posts/2026-09-25/Contrastive-LM-开源-CLM-8B-动作评分提速-9-倍的开源-System-One-决策模型/ https://insight.aitobox.com/blog/posts/2026-09-25/Contrastive-LM-开源-CLM-8B-动作评分提速-9-倍的开源-System-One-决策模型/ Fri, 25 Sep 2026 00:00:00 GMT 在 AI 智能体 (AI Agent) 的决策循环中，传统大语言模型大多采用自回归机制逐字生成文本，这种“深思熟虑”的 System Two 慢思考架构在需要频繁评估候选动作时往往带来高昂的时间延迟。为此，Contrastive-LM 正式开源了首个对比语言模型 (Contrastive Language Model, CLM) —— **CLM-8B**，开创了专用于动作评估的开源 System One“快思考”决策模型新范式。该模型由冻结的 Qwen3-8B 骨干网络与仅 20M 参数的轻量投影头构成，彻底摆脱了冗长的文本生成，直接利用状态与动作嵌入向量的点积极速输出动作的概率分布。在零样本基准测试中，CLM-8B 的动作评分速度比闭源模型 Jev 提升高达 9 倍，候选动作扩展至 1,000 个时更可提速 13 倍，并在 DeepSWE 和 Terminal-Bench 2.1 编程基准上刷新了代码验证器的业界最前沿 (SOTA) 成绩。作为基于 Apache-2.0 许可证完全开源且仅需单张消费级 GPU 即可流畅部署的模型，CLM-8B 为高实时性智能体系统和代码验证流水线注入了兼具极致性能与开源可控的全新动力。 --- ## 核心概述 > ## Summary Contrastive-LM 正式发布了 **CLM-8B**，这是全新一类**对比语言模型 (Contrastive Language Model, CLM) **中的首个开源力作。与传统模型逐字生成文本不同，CLM 会对照当前环境状态对所有候选动作进行快速评估并直接输出概率，为 TypeSafe AI 旗下的 Jev 等闭源商业模型提供了一个高性能的开源替代方案。该模型基于 Apache-2.0 许可证开源发布，仅重 75 MB 的轻量级头部借助 vLLM 驱动的 Qwen3-8B 编码器，可在单张 NVIDIA GPU 上极其高效地运行。得益于深度优化的向量缓存机制与扎实的三阶段训练策略，CLM-8B 在零样本任务中的推理延迟比 Jev 降低了最高达 9 倍，并在 DeepSWE 和 Terminal-Bench 2.1 等代码基准测试中树立了验证器 (Verifier) 性能的业界全新标杆。 > Contrastive-LM has introduced **CLM-8B**, the first open-source model in a new class of **Contrastive Language Models (CLMs)**. Instead of generating text, CLM evaluates candidate actions against the current state and outputs probabilities, providing a high-performance open alternative to proprietary models like TypeSafe AI's Jev. Distributed under the Apache-2.0 license, the 75 MB head runs efficiently on a single NVIDIA GPU using vLLM for the Qwen3-8B encoder. Featuring optimized caching mechanisms and a robust three-stage training recipe, CLM-8B achieves up to 9× faster latency than Jev in zero-shot tasks and establishes new state-of-the-art verifier results on coding benchmarks like DeepSWE and Terminal-Bench 2.1. --- ## System One 模型的核心功用 > ## What a System One Model Does TypeSafe AI 打造的 Jev 并不会返回长篇文字，而是直接返回带有明确概率分布的类型化数值。CLM 完全对齐了这一交互接口，并在 [CLM GitHub 仓库](https://github.com/Contrastive-LM/CLM) 中提供了一套完全兼容的 API，主要支持三大核心查询类型： > TypeSafe AI's Jev returns typed values with probabilities instead of text. CLM targets this exact interface, offering a compatible API through the [CLM GitHub repository](https://github.com/Contrastive-LM/CLM) that exposes three core question types: * **Noul：** 计算并返回某个陈述或命题为真时的置信概率。 * **Choice (单选) ：** 从预先声明的候选集合中挑选一个最优项，并附带各自的概率分布。 * **Score (评分) ：** 在有序的量表或评分准则上，计算并返回期望的得分等级。 > * **Noul:** Returns the probability that a statement is true. > * **Choice:** Picks one option from a declared set, accompanied by probabilities. > * **Score:** Returns an expected level on an ordered rubric. 针对 TypeSafe API 所设计的结构化请求，可以直接使用 CLM 的 Python 客户端进行无缝重放和执行，无需对调用逻辑做任何更改。 > Requests structured for TypeSafe’s API can be seamlessly replayed using CLM’s Python client. --- ## CLM 的工作原理 > ## How CLM Works CLM 通过双向 InfoNCE 损失函数 (InfoNCE Loss) 同时训练状态编码器 (State Encoder) 与动作编码器 (Action Encoder) 。两个编码器均采用冻结的 [Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) 骨干网络，仅外挂一个 20M 参数的可训练投影头 (Projection Head) 。在训练阶段，模型会拉近每个当前状态与实际所采取动作之间的表征距离，同时将该状态与所有未被采纳的备选动作推远，形成鲜明的对比表征。 > CLM trains a state encoder and an action encoder utilizing a bidirectional InfoNCE loss. Each encoder consists of a frozen [Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) backbone coupled with a 20M-parameter trainable projection head. During training, the model pulls each state toward the action actually taken while pushing it away from alternatives. 在推理阶段，系统通过计算状态向量与各个动作嵌入向量的点积 (Dot Product) 来为候选动作打分，再经过 Softmax 函数将其转化为标准答案的概率分布。在典型的 AI 智能体运行循环中，环境状态在每一步都会改变，但可选动作集合通常基本保持固定。针对这一特点，`clm-serve` 引入了高效的向量缓存机制，从而规避了大量冗余运算。在配备 3 个可选动作的单张 RTX 4090 显卡上，复访状态的推理延迟从 1.7 ms 骤降至 0.6 ms；而当候选动作扩展到约 1,000 个时，其运行速度最高可达 Jev 的 13 倍。 > At inference time, candidate actions are scored via the dot product of the state and action embeddings, converted into an answer distribution via softmax. Because agent states change every step while action sets remain mostly fixed, `clm-serve` caches vectors to avoid redundant computations. On a single RTX 4090 with 3 actions, revisited state latency drops from 1.7 ms to 0.6 ms, enabling speeds up to 13× faster than Jev with ~1,000 candidates. --- ## 三阶段训练秘方 > ## A 3-Stage Training Recipe 1. **预训练 (Pre-training) ：** 基于约 6000 万对 [Nemotron DQA 问答对数据](https://huggingface.co/datasets/Contrastive-LM/CLM-v0.1-Pretrain-Nemotron) 展开基础预训练。 2. **中期训练 (Mid-training) ：** 引入由 Gemini 2.5 Flash-Lite 合成的约 3000 万条高质量难负例 (Hard Negatives) 数据进行强化训练。 3. **后训练 (Post-training) ：** 融合源自 Agent Data Protocol、Endless-Terminals 和 LiteCoder-Terminal-SFT 的约 100 万条真实智能体运行轨迹 (Agent Trajectories) 数据。 > 1. **Pre-training:** Performed on ~60M [Nemotron DQA question-answer pairs](https://huggingface.co/datasets/Contrastive-LM/CLM-v0.1-Pretrain-Nemotron). > 2. **Mid-training:** Utilizes ~30M synthetic hard negatives generated by Gemini 2.5 Flash-Lite. > 3. **Post-training:** Incorporates ~1M agent trajectories sourced from Agent Data Protocol, Endless-Terminals, and LiteCoder-Terminal-SFT. 在约 10 万道预留的测试问答上，仅进行预训练即可达到 52.1% 的 Top-1 准确率；而在经历中期训练后，该指标进一步跃升至 69.2% (相比之下，若从一开始就完全只在难负例数据上训练，准确率在过拟合之前便会过早触顶停留在 62.4%) 。 > On ~100K held-out questions, pre-training alone achieves 52.1% top-1 accuracy, which rises to 69.2% after mid-training (whereas training exclusively on hard negatives from the start peaks prematurely at 62.4% before overfitting). --- ## 与 Jev 的零样本性能对比 > ## Zero-Shot Results Against Jev | 评估任务 | CLM-8B 延迟 | Jev 延迟 | CLM-8B 成功率 | Jev 成功率 | | :--- | :--- | :--- | :--- | :--- | | T-Rex 游戏 | 16.5 ms | 149.8 ms | 5/5 | 5/5 | | 工具调用 (BFCL v4) | 76.8 ms | 125.5 ms | 95.2% | 99.2% | | WikiRacing (维基竞速) | 79.8 ms | 225 ms | 26/30 | 30/30 | | Super Mario (超级马里奥) | 33.5 ms | 132.6 ms | 5/5 | 5/5 | > | Task | CLM-8B Latency | Jev Latency | CLM-8B Success | Jev Success | > | :--- | :--- | :--- | :--- | :--- | > | T-Rex game | 16.5 ms | 149.8 ms | 5/5 | 5/5 | > | Tool calling (BFCL v4) | 76.8 ms | 125.5 ms | 95.2% | 99.2% | > | WikiRacing | 79.8 ms | 225 ms | 26/30 | 30/30 | > | Super Mario | 33.5 ms | 132.6 ms | 5/5 | 5/5 | 最为惊艳的 9 倍加速成果出现在 [T-Rex 游戏](https://github.com/Contrastive-LM/CLM/blob/main/examples/t_rex/README.md) 中，因为在该任务中相同的动作会在不同状态间频繁重复出现。在任务表现方面，CLM 在 T-Rex 游戏与 Super Mario 任务上与 Jev 打成平手，在工具调用和 WikiRacing 上略有微小差距；但在推理响应速度上，CLM 在所有测试基准中均展现出压倒性优势，全面大幅超越 Jev。 > The headline 9× speedup is observed on the [T-Rex game](https://github.com/Contrastive-LM/CLM/blob/main/examples/t_rex/README.md), where actions frequently repeat across states. While CLM matches Jev on the T-Rex and Super Mario tasks and slightly trails on tool calling and WikiRacing, it consistently outperforms Jev in inference speed across the board. --- ## CLM 充当编程智能体的结果验证器 > ## CLM as a Verifier for Coding Agents 在典型的智能体验证流水线中，通常由生成器 (Generator) 采样出多个候选解决方案，再由验证器 (Verifier) 选出其中的最优解。研究团队在单张 H100 GPU 上针对预留测试子集进行了全面评估 (其中 DeepSWE 的候选方案由 Opus 5 生成，Terminal-Bench 2.1 的候选方案由 Fable 5 生成) ： > In agent verification pipelines, a generator samples multiple candidate solutions and a verifier selects the best one. Evaluated on held-out subsets (using Opus 5 for DeepSWE and Fable 5 for Terminal-Bench 2.1 candidates) on an H100: | 基准测试 | Pass@1 基础值 | CLM (微调版) | Jev | CLM 延迟 | Jev 延迟 | | :--- | :--- | :--- | :--- | :--- | :--- | | DeepSWE | 73.7% | 81.6% | 71.1% | 79 ms | 449 ms | | Terminal-Bench 2.1 | 84.0% | 87.6% | 83.1% | 32 ms | 131 ms | > | Benchmark | Pass@1 | CLM (Fine-Tuned) | Jev | CLM Latency | Jev Latency | > | :--- | :--- | :--- | :--- | :--- | :--- | > | DeepSWE | 73.7% | 81.6% | 71.1% | 79 ms | 449 ms | > | Terminal-Bench 2.1 | 84.0% | 87.6% | 83.1% | 32 ms | 131 ms | 这些评测结果在验证器基准测试上创下了全新的业界最前沿 (SOTA) 纪录。值得注意的是，Jev 评测得分甚至低于未经挑选的基线 Pass@1 指标，这意味着使用 Jev 进行筛选其表现反而不如随机抽样；相比之下，CLM 借助轻量级的 [微调预测头](https://github.com/Contrastive-LM/CLM/blob/main/docs/FINETUNING.md)，不仅显著提升了解题准确率，还同时实现了 4.1 倍至 5.7 倍的推理加速。 > These results establish new SOTA verifier benchmarks. Notably, Jev scores below the baseline Pass@1 metric, meaning selection with Jev degrades performance compared to random sampling, whereas CLM delivers accuracy improvements alongside a 4.1× to 5.7× speedup utilizing lightweight [fine-tuned heads](https://github.com/Contrastive-LM/CLM/blob/main/docs/FINETUNING.md). --- ## 核心要点总结 > ## Key Takeaways * **颠覆传统架构：** CLM-8B 直接对候选动作进行打分与概率评估，彻底告别了自回归逐字生成文本的传统模式。 * **极致推理能效：** 在零样本任务评测中，推理延迟相比 Jev 降低最高达 9 倍。 * **强悍的验证器能力：** 经过微调的预测头在 DeepSWE 上取得了 81.6% 的准确率，在 Terminal-Bench 2.1 子集上斩获 87.6% 的优异成绩。 * **专为智能体循环优化：** 深度缓存状态与动作向量，大幅削减智能体多步执行中的单步等待延迟。 * **完全开源自由：** 预测头基于 Apache-2.0 许可证发布，专为在单张 NVIDIA GPU 上私有化部署量身打造。 > * **Alternative Architecture:** CLM-8B evaluates and scores candidate actions rather than generating text autoregressively. > * **High Efficiency:** Delivers up to 9× lower latency than Jev during zero-shot evaluations. > * **Strong Verifier Performance:** Fine-tuned model heads achieve 81.6% accuracy on DeepSWE and 87.6% on Terminal-Bench 2.1 subsets. > * **Optimized Agent Loops:** Caches state and action vectors to drastically lower step-by-step latency. > * **Open Source:** Apache-2.0 licensed head designed for self-hosting on a single NVIDIA GPU. --- ## 相关资源与链接 > ## Resources & Links * [Hugging Face 上的 CLM-8B 模型主页](https://huggingface.co/Contrastive-LM/CLM-v0.1-8B) * [官方 GitHub 代码仓库](https://github.com/Contrastive-LM/CLM) * [项目博客与 Notion 主页](https://contrastive-lm.notion.site/) * [完整数据集与模型合集](https://huggingface.co/Contrastive-LM) > * [CLM-8B Model on Hugging Face](https://huggingface.co/Contrastive-LM/CLM-v0.1-8B) > * [Official GitHub Repository](https://github.com/Contrastive-LM/CLM) > * [Project Blog & Notion](https://contrastive-lm.notion.site/) > * [Data & Models Collection](https://huggingface.co/Contrastive-LM) https://insight.aitobox.com/blog/posts/2026-09-24/超越自然语言-专为自主科学研究打造的智能体原生形式化语言-Lara/ https://insight.aitobox.com/blog/posts/2026-09-24/超越自然语言-专为自主科学研究打造的智能体原生形式化语言-Lara/ Thu, 24 Sep 2026 00:00:00 GMT 随着自主 AI 智能体 (AI Agent) 逐步渗透到科学研究的各个环节，海量科研成果的涌现速度早已远超人类同行审阅与复核的精力极限。然而，时至今日，学术界的思想交流依然严重依赖自然语言——这种非形式化的表达媒介充斥着语义模糊、隐性假设与难以追踪的局限性，导致机器根本无法对其逻辑进行可靠的自动化审计。为破解这一困境，研究团队推出了 **Lara**：一种专为自主科学打造的机器可检查语言与协议，它将传统的学术论证转化为可直接执行的计算工件，为人机协同构建起一个高效的“认知内核” (Epistemic Kernel) 。借助声明式桥接机制与交互式定理证明器 Lean 4 约 11.7 万行代码的严格形式化验证，Lara 能够将跨文献的论证交织成全局统一、可审计的知识网络，并在毫秒级时间内对任何科学论断的有效状态完成复核。 --- # 超越自然语言：专为自主科学研究打造的智能体原生形式化语言 Lara > # Beyond Natural Language: An Agent-Native Language for Autonomous Science **arXiv 编号:** [2609.25421](https://arxiv.org/abs/2609.25421) [cs.PL] **DOI 标识:** [10.48550/arXiv.2609.25421](https://doi.org/10.48550/arXiv.2609.25421) **论文作者:** Yifeng He, Jiachen Liu **提交时间:** 2026年9月21日 **研究领域:** 编程语言 (cs.PL) ；人工智能 (cs.AI) > **arXiv:** [2609.25421](https://arxiv.org/abs/2609.25421) [cs.PL] > **DOI:** [10.48550/arXiv.2609.25421](https://doi.org/10.48550/arXiv.2609.25421) > **Authors:** Yifeng He, Jiachen Liu > **Submitted:** 21 September 2026 > **Subjects:** Programming Languages (`cs.PL`); Artificial Intelligence (`cs.AI`) --- ## 📋 概要 > ## 📋 Summary 当自主 AI 智能体 (AI Agent) 逐步接管科学探索的各个环节时，人类有限的审阅能力正被呈爆发式增长的科研产出迅速超越。然而，时至今日，学术界的思想交流依然严重依赖自然语言散文——这种非形式化的表达媒介充满了语义模糊、隐性假设以及难以追踪的局限性，导致机器根本无法对其进行可靠的自动化逻辑审查。 > As autonomous AI agents take over every phase of scientific inquiry, human review capacities are vastly outpaced. However, scientific communication still depends on natural-language prose—an informal medium fraught with ambiguity, hidden assumptions, and untracked limitations that machines cannot reliably audit. 为了弥合这一鸿沟，作者团队推出了 **Lara**：一种专为验证与修订科研论断而设计的机器可检查语言与协议。通过将传统学术论证转化为可直接执行的计算工件，Lara 为自主科学量身打造了一个“认知内核” (Epistemic Kernel) 。它不仅能为科研智能体构建全自动化的验证流水线，还能通过显式声明的逻辑桥接，将跨论文的论点交织成一个全局统一、随时可审计的知识网络，从而让科研人员与机器系统都能在毫秒之内快速复核任何已编码论断的有效状态。 > To bridge this gap, the authors introduce **Lara**: a machine-checkable language and protocol designed to verify and revise research claims. By converting traditional research arguments into executable artifacts, Lara offers an epistemic kernel tailored for autonomous science. It facilitates automated validation pipelines for research agents, connects arguments across papers into a unified auditable network via declared bridges, and allows both humans and machines to recheck the standing of any encoded claim in milliseconds. --- ## 🔑 Lara 的核心特性 > ## 🔑 Key Features of Lara * **显式声明 (Explicit Declarations) ：** 论文作者需要显式定义其核心论断、底层的支撑证据与假设前提，以及所有已知的反驳观点或局限性。 * **确定性检查 (Deterministic Checking) ：** 由一个轻量级的确定性检查器来处理这些相互作用，并为每个论断赋予一个完全可复现的明确状态： * `"justified"` (已充分论证) * `"defeated"` (已被驳倒) * `"contested"` (尚存争议) * `"gap"` (识别不完整的支撑证据，并精准定位尚未解答的具体问题) * **严谨性实证验证 (Rigorous Validation) ：** 论文中的多项案例研究展示了 Lara 的强大实用性，其涵盖了经验性文献综述、无实测数据的纯哲学思辨，以及在假定公理被撤回时分析论断支撑丧失情况的级联场景。 * **形式化验证 (Formal Verification) ：** 作者团队建立了论断检查与跨上下文论点迁移的元理论，并在定理证明器 **Lean 4** 中对语义保证进行了严密的机械化形式化证明 (代码量高达约 117,000 行，仅留有 3 处论证在纸质手稿上推导) 。公开的元理论做到了零 `"sorry"` 占位符 (即证明过程完整闭环、毫无遗漏) ，完全仅依赖于 Lean 的 3 个标准公理。 > * **Explicit Declarations:** Authors explicitly define their core claims, underlying supporting evidence and assumptions, and any known objections or limitations. > * **Deterministic Checking:** A lightweight, deterministic checker processes these interactions, assigning each claim a reproducible status: > * `"justified"` > * `"defeated"` > * `"contested"` > * `"gap"` (identifies incomplete support and pinpoints the exact unanswered question) > * **Rigorous Validation:** Case studies demonstrate Lara's utility across empirical reviews, purely philosophical debates without measurements, and scenarios analyzing the loss of support when an assumed axiom is withdrawn. > * **Formal Verification:** The authors establish the metatheory of claim checking and cross-context argument transport, mechanizing the semantic guarantees in **Lean 4** (spanning approximately 117,000 lines, with only three arguments left on paper). The public metatheory is `"sorry"`-free and relies solely on Lean's three standard axioms. --- ## 🔗 获取论文与相关资源 > ## 🔗 Access & Resources * **全文阅读选项 (Full-Text Options) ：** * [阅读 PDF 原文](https://arxiv.org/pdf/2609.25421) * [HTML 网页版 (实验性功能) ](https://arxiv.org/html/2609.25421v1) * [TeX 源码](https://arxiv.org/src/2609.25421) * **授权许可 (License) ：** [知识共享署名 4.0 国际许可协议 (Creative Commons Attribution 4.0) ](http://creativecommons.org/licenses/by/4.0/)  * **文献检索与引用工具 (Bibliographic Tools) ：** * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.25421) * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.25421) * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.25421) > * **Full-Text Options:** > * [View PDF](https://arxiv.org/pdf/2609.25421) > * [HTML Version (Experimental)](https://arxiv.org/html/2609.25421v1) > * [TeX Source](https://arxiv.org/src/2609.25421) > * **License:** [Creative Commons Attribution 4.0](http://creativecommons.org/licenses/by/4.0/)  > * **Bibliographic Tools:** > * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.25421) > * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.25421) > * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.25421) https://insight.aitobox.com/blog/posts/2026-09-24/诺基亚开源-AnyJev-零训练让任何开源大模型转变为精准校准的结构化决策模型/ https://insight.aitobox.com/blog/posts/2026-09-24/诺基亚开源-AnyJev-零训练让任何开源大模型转变为精准校准的结构化决策模型/ Thu, 24 Sep 2026 00:00:00 GMT 在工业界大语言模型 (Large Language Model, LLM) 的实际落地中，绝大多数业务本质上属于从固定选项中选择答案的结构化决策，例如客服路由、意图分类与合规审查，而无需生成冗长的自由文本。然而，直接读取模型的下一个 Token 概率不仅极易受选项排列顺序的干扰，其置信度也严重失真、缺乏统计校准。为此，Nokia 应用研究团队开源了轻量级 Python 库 AnyJev。该工具无需任何参数微调或模型训练，仅通过优雅的循环排列与运行期动态先验校正，便能让主流开源大模型即插即用地转变为高精度、概率可靠的结构化决策模型。基准测试表明，AnyJev 显著消除了位置偏置与先验偏置，在极其严苛的 5% 误差控制下，将模型的可自动决策比例从 7.7% 大幅提升至 52.0%，为企业构建高可靠自动化业务流提供了强有力的生产级保障。 --- ## 核心摘要 > ## Summary Nokia 应用研究团队正式开源了 Python 库 **AnyJev**。该工具无需任何二次微调与训练，便能将各类开源大语言模型 (Large Language Model, LLM) 改造为经过精准校准的结构化决策模型。AnyJev 目前已发布至 PyPI 并遵循 Apache-2.0 开源协议，专为工业生产中“从固定选项集中挑选最佳答案”而非“发散生成自由文本”的分类与决策场景量身定制。通过创新的循环排列 (Cyclic Shifts) 与批次动态校准技术，AnyJev 有效攻克了困扰大模型分类的先验偏置 (Prior Bias) 与位置偏置 (Position Bias) 顽疾，在显著提升决策准确率与稳定性的同时，大幅降低了预期校准误差 (Expected Calibration Error, ECE) 。 > Nokia’s applied research team has released **AnyJev**, an open-source Python library designed to transform any open-source Large Language Model (LLM) into a calibrated decision model without requiring any training. Available on PyPI under the Apache-2.0 license, AnyJev is optimized for production tasks that require selecting an answer from a fixed set of options rather than generating free-form text. By addressing common pitfalls like prior bias and position bias through advanced cyclic shifts and batch calibration, AnyJev significantly improves model accuracy, stability, and calibration error (ECE). --- ## 什么是 AnyJev？ > ## What is AnyJev? AnyJev 借鉴并沿用了 TypeSafe AI 旗下的 System One 决策模型 **Jev** 的接口设计规范。开发者只需传入一个强类型的结构化问题，系统便能直接输出结构化决策结果，并附带可信度极高的置信概率。这一概率值直接从大模型输出层的下一个 Token 概率分布中提取，整个过程完全摆脱了冗长的文本生成、繁琐的正则解析或额外的参数训练。 > AnyJev borrows its interface from **Jev**, the System One decision model launched by TypeSafe AI. It allows developers to input a typed question and receive a structured decision paired with a dependable probability threshold. This probability is extracted directly from the model's next-token distribution without generating text, parsing outputs, or performing training. 该库原生支持三类核心决策问题： * **`choice`**：从预先设定的 $K$ 个候选选项中挑选出单一最佳答案。 * **`noul`**：二元“是/否”判定查询。 * **`score`**：将预测答案归类到若干有序的数值或类别区间 (分箱) 中。 > The library natively supports three primary question types: > * **`choice`**: Selects a single option from a predefined set of $K$ alternatives. > * **`noul`**: A binary yes/no query. > * **`score`**: Categorizes the answer into one of several ordered numerical or categorical bins. --- ## 直接读取 Logits 的致命缺陷 > ## The Problem With Reading Logits Directly 在传统工程实践中，许多人习惯通过限制大模型仅输出特定的选项标签，进而直接读取并比较对应 Token 的原始得分 (Logits) 。然而，Nokia 研究团队指出，这种看似省事的“走捷径”做法存在两大严重缺陷： 1. **顺序极度敏感 (Order Sensitivity)**：仅仅改变选项在提示词中的排列先后顺序，模型的最终选择就可能发生彻底反转； 2. **概率失真严重 (Poor Calibration)**：直接计算出的置信概率往往过度自信或偏差过大，缺乏统计学上的真实可信度。 > Many existing workflows restrict the model's next-token generation to explicit option labels to read their raw scores. However, the Nokia research team highlights two major flaws with this shortcut: > 1. **Order Sensitivity:** The model's selected answer can completely change simply by reordering the options. > 2. **Poor Calibration:** The resulting probabilities are unreliable and uncalibrated. 官方技术文档指出，这些根本问题源于大模型固有的两大偏置机制： * **先验偏置 (Prior Bias)**：无论上下文语境为何，模型内在更偏好特定的文本标签（例如天然更倾向于输出“Yes”而非“No”）； * **位置偏置 (Position Bias)**：模型往往对选项列表中的特定排位（例如首项或末项等视觉位置）产生天然的选择偏好。 > According to the official documentation, these issues stem from two primary biases: > * **Prior Bias:** The model inherently favors specific labels (such as "Yes" over "No") regardless of the context. > * **Position Bias:** The model favors specific visual slots or positions within the option list. --- ## AnyJev 的工作机制：L0 与 L1 双层校准 > ## How AnyJev Works: L0 and L1 该库生成的每一次决策结果都包含一个 `level` 字段，用于指定所采用的校准深度： > Every decision generated by the library includes a `level` field that dictates the calibration depth: https://insight.aitobox.com/blog/posts/2026-09-24/深度解析-Ubuntu-26.04-对-Apache-systemd-服务的安全限制收紧与避坑指南/ https://insight.aitobox.com/blog/posts/2026-09-24/深度解析-Ubuntu-26.04-对-Apache-systemd-服务的安全限制收紧与避坑指南/ Thu, 24 Sep 2026 00:00:00 GMT 随着 Ubuntu 26.04 长期支持版本的推进，Canonical 官方进一步收紧了核心服务的基础安全策略，在 `apache2.service` 的 systemd 单元中默认开启了严苛的沙盒限制机制。这些加固策略虽然在源头上封堵了大量提权与越权漏洞，但对于大量使用 CGI 脚本、动态 PHP 扩展或 JIT 运行时的现实业务而言，极易产生静默崩溃与权限异常。本文结合具体参数，深度剖析了各项配置的底层原理、排障难点及应对方案，并提炼出兼顾安全性与兼容性的实操配置建议。 --- ## 核心概述 > ## Summary 在 Ubuntu 26.04 中，Canonical 为 `apache2.service` 引入了极其严苛的 systemd 服务单元配置文件。对于那些只在 `/var/www` 下提供静态网页文件的极简 Web 服务器而言，这些限制大概率相安无事；但对于架构稍显复杂的业务环境——比如通过 `suexec` 为不同用户执行独立 CGI 脚本的部门服务器、各类 PHP 应用程序，或是挂载了多种 Apache 扩展模块的系统——这些限制将引发严重的兼容性故障。本文将深入剖析该版本中那些令人瞩目的 systemd 限制项，指出潜在的隐患暗坑，并帮助大家甄别出哪些安全加固配置依然保持合理且实用。 > In Ubuntu 26.04, Canonical has introduced a heavily restricted `apache2.service` systemd unit file. While these restrictions are likely harmless for basic web servers serving static files from `/var/www`, they can cause significant compatibility issues for more complex setups—such as departmental servers running user CGIs via `suexec`, PHP applications, or various Apache modules. This document breaks down the eyebrow-raising systemd restrictions introduced in this release, highlights potential pitfalls, and distinguishes them from the security settings that remain sensible and practical. --- ## 1. 潜藏隐患的高风险安全限制 > ## 1. Potentially Problematic Restrictions Apache 服务配置中新增的以下 systemd 设置，很可能会干扰非标准环境配置、动态代码执行或老旧的历史遗留脚本： > The following systemd settings added to the Apache service file may interfere with non-standard configurations, dynamic execution, or legacy scripts: https://insight.aitobox.com/blog/posts/2026-09-24/Lean-Pool-由-AI-智能体自主维护与持续扩充的形式化数学知识库/ https://insight.aitobox.com/blog/posts/2026-09-24/Lean-Pool-由-AI-智能体自主维护与持续扩充的形式化数学知识库/ Thu, 24 Sep 2026 00:00:00 GMT 形式化数学通过交互式定理证明器将深奥严密的数学证明转化为计算机可严格验证的代码，但传统形式化数学库的扩充极度依赖顶尖数学家的手工编写，人力维护成本高昂且发展进程相对缓慢。为了突破这一制约形式化数学规模化演进的长期瓶颈，研究者 Vasily Ilin 提出了 **Lean Pool** 这一开创性架构，旨在构建一个完全由 AI 智能体 (AI Agent) 自主扩充、日常维护与深度优化的形式化数学资源池。该项目依托现代定理证明框架 Lean 4，巧妙架起了前沿人工智能与形式化符号推理之间的桥梁，探索了智能体自动生成证明、跨项目整合知识与持续重构代码的全新范式。这一突破不仅为自动化数学发现与大规模可靠知识库的协同构建指明了方向，也标志着形式化数学正式迈向由 AI 自主驱动的智能化演进新阶段。 --- # Lean Pool：由 AI 智能体自主维护与持续扩充的形式化数学知识库 > # Lean Pool: An AI-Maintained Archive of Formalized Mathematics ## 内容概要 > ## Summary **Lean Pool** 是一个动态演进的形式化数学知识库，完全由 AI 智能体 (AI Agent) 自主进行扩充、日常维护与深度优化。该项目由 Vasily Ilin 提出，在 Lean 形式化框架内搭建起连接人工智能与自动定理证明的桥梁。 > **Lean Pool** is a dynamic repository of formalized mathematics that is autonomously grown, maintained, and optimized by AI agents. Proposed by Vasily Ilin, the project bridges artificial intelligence and automated theorem proving within the Lean framework. --- ## 论文元数据 > ## Paper Metadata | 属性 | 详情 | | :--- | :--- | | **标题** | Lean Pool: An AI-Maintained Archive of Formalized Mathematics | | **作者** | Vasily Ilin | | **主要学科领域** | 人工智能 (`cs.AI`) | | **提交日期** | 2026 年 9 月 21 日 | | **arXiv 编号** | [arXiv:2609.25199](https://arxiv.org/abs/2609.25199) | | **数字对象标识符 (DOI) ** | [10.48550/arXiv.2609.25199](https://doi.org/10.48550/arXiv.2609.25199) | | **备注信息** | 共 52 页，含 6 幅图。包含已导入项目的目录列表。 | > | Attribute | Details | > | :--- | :--- | > | **Title** | Lean Pool: An AI-Maintained Archive of Formalized Mathematics | > | **Authors** | Vasily Ilin | > | **Primary Subject** | Artificial Intelligence (`cs.AI`) | > | **Submitted on** | September 21, 2026 | > | **arXiv Identifier** | [arXiv:2609.25199](https://arxiv.org/abs/2609.25199) | > | **DOI** | [10.48550/arXiv.2609.25199](https://doi.org/10.48550/arXiv.2609.25199) | > | **Comments** | 52 pages, 6 figures. Includes a catalogue of imported projects. | --- ## 论文摘要 > ## Abstract Lean Pool 是一个形式化数学知识库，完全由 AI 智能体负责自主扩充、日常维护与持续优化。 > Lean Pool is a repository of formalized mathematics. It is grown, maintained and optimized by AI agents. --- ## 访问链接与相关资源 > ## Access Links & Resources * **全文阅读：** * [查看 PDF](https://arxiv.org/pdf/2609.25199) * [HTML 网页版本 (实验性) ](https://arxiv.org/html/2609.25199v1) * [TeX 源码](https://arxiv.org/src/2609.25199) * **授权协议：** [知识共享署名 4.0 国际许可协议 (Creative Commons Attribution 4.0) ](http://creativecommons.org/licenses/by/4.0/)  * **引用与检索：** * [Google 学术搜索 (Google Scholar) ](https://scholar.google.com/scholar_lookup?arxiv_id=2609.25199) * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.25199) * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.25199) > * **Full-Text:** > * [View PDF](https://arxiv.org/pdf/2609.25199) > * [HTML Version (Experimental)](https://arxiv.org/html/2609.25199v1) > * [TeX Source](https://arxiv.org/src/2609.25199) > * **License:** [Creative Commons Attribution 4.0](http://creativecommons.org/licenses/by/4.0/)  > * **Citations & References:** > * [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.25199) > * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.25199) > * [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.25199) https://insight.aitobox.com/blog/posts/2026-09-24/Kyutai-开源-Voice-of-Reason-首个通过强化学习原生解算数学题的端到端语音大模型/ https://insight.aitobox.com/blog/posts/2026-09-24/Kyutai-开源-Voice-of-Reason-首个通过强化学习原生解算数学题的端到端语音大模型/ Thu, 24 Sep 2026 00:00:00 GMT 长期以来，传统语音交互主要依赖“语音转文字 $\rightarrow$ 文本大模型 $\rightarrow$ 文字转语音”的级联架构，不仅通信时延高，还会损失语调与情绪等副语言特征；而新一代原生端到端语音模型虽然能实现无缝实时对话，却因必须按节律持续输出音频流，在面对复杂数学多步推理任务时举步维艰。为攻克这一核心技术难题，开源人工智能前沿机构 Kyutai 正式开源了 **Voice of Reason** 系列模型，这是业界首个通过强化学习 (Reinforcement Learning, RL) 原生解算口语数学题的端到端语音大模型。该方案基于开源的 `GLM-4-Voice-9B`，巧妙融合高质量监督微调 (Supervised Fine-Tuning, SFT) 与类 GRPO 的强化学习策略，彻底摆脱了中间文本转录与外部文本大语言模型 (Large Language Model, LLM) 的依赖。实验评测显示，该模型在口语 GSM8K 数学基准测试上的解题准确率从基线的 27.3% 飙升至 77.1%，同时完美保持了语音的高自然度与流畅度。目前两款 9B 参数的 BF16 权重已全面开源，单张 H100 GPU 即可实现低成本私有化部署，为打造真正具备深度逻辑思考能力的原生实时语音交互智能体铺平了道路。 --- ## 核心速览与内容概要 > ## TL;DR / Summary 人工智能研究机构 Kyutai 正式发布了开源权重模型 **Voice of Reason**。该项目基于开源模型 `GLM-4-Voice-9B` 打造，包含两款能够直接在语音交互中“边说边想”解算数学难题的原生端到端语音模型。通过将监督微调 (Supervised Fine-Tuning, SFT) 与强化学习 (Reinforcement Learning, RL) 有机结合——全程无需任何中间文本转录或外部文本大语言模型——新模型将口语版 GSM8K 数学评测基准的准确率从基准的 **27.3% 跨越式提升至 77.1%**。目前，两款 BF16 精度的模型权重文件均已在 Hugging Face 开放下载，单张 H100 GPU 即可完成本地私有化部署。 > Kyutai has released **Voice of Reason**, a pair of open-weight speech-to-speech models built on top of `GLM-4-Voice-9B` that solve math problems natively out loud. By combining Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL)—without any intermediate transcription steps or text LLMs—the models dramatically boost spoken GSM8K accuracy from a baseline of **27.3% up to 77.1%**. Both BF16 checkpoints are available on Hugging Face and can be self-hosted on a single H100 GPU. --- ## 部署要求与开源资源 > ## Deployment & Availability * **硬件要求**：两款 BF16 精度权重均可完整装入单张 H100 GPU。运行需依赖 [GLM-4-Voice 代码仓库](https://github.com/THUDM/GLM-4-Voice) 提供的语音编解码器 (Speech Tokenizer & Decoder) 。 * **开源协议**：沿用 GLM-4-Voice 的开源许可证。 * **模型权重文件 (Checkpoints) **： * [`glm-4-voice-of-reason-9b`](https://hf.co/kyutai/glm-4-voice-of-reason-9b) * [`glm-4-voice-of-reason-stitch-9b`](https://hf.co/kyutai/glm-4-voice-of-reason-stitch-9b) > * **Hardware Requirements:** Both BF16 checkpoints fit on a single H100 GPU. Requires the [GLM-4-Voice repo](https://github.com/THUDM/GLM-4-Voice) for its speech tokenizer and decoder. > * **License:** Inherits the GLM-4-Voice license. > * **Checkpoints:** > * [`glm-4-voice-of-reason-9b`](https://hf.co/kyutai/glm-4-voice-of-reason-9b) > * [`glm-4-voice-of-reason-stitch-9b`](https://hf.co/kyutai/glm-4-voice-of-reason-stitch-9b) --- ## 为什么语音模型在数学推理上普遍表现不佳？ > ## Why Speech Models Lag on Math 在复杂推理任务中，传统的级联流水线架构 (即语音转文字 $\rightarrow$ 文本大语言模型 $\rightarrow$ 文字转语音) 依然占据主导地位。然而，各个独立阶段不仅会不断累加响应延迟，还会丢失诸如语调、重音等至关重要的副语言信息 (Paralinguistic Cues) 。 > Traditional cascaded pipelines (speech-to-text $\rightarrow$ text LLM $\rightarrow$ text-to-speech) still dominate in reasoning tasks. However, each stage introduces latency and drops crucial paralinguistic cues like tone. 相比之下，原生语音模型必须以恒定的时间间隔持续吐出音频，以维持实时的人机对话互动感，这极大地限制了模型在幕后用于深度思考的“隐藏推理 Token”数量。在基准表现上，原始版本的 `GLM-4-Voice` 在 [GSM8K](https://arxiv.org/abs/2110.14168) 上的得分仅为 27.3%；尽管此前的 [STITCH](https://arxiv.org/abs/2507.15375) 方法通过引入推理块机制将得分提升到了 58.7%，但 Voice of Reason 真正开创了先河——它标志着强化学习首次在原生语音模型的数学逻辑推理中获得成功应用。 > Speech-native models must emit audio at regular intervals to maintain interactivity, strictly limiting the hidden reasoning tokens they can afford. While the base `GLM-4-Voice` scores 27.3% on [GSM8K](https://arxiv.org/abs/2110.14168), and the earlier [STITCH](https://arxiv.org/abs/2507.15375) method raised that to 58.7% using reasoning chunks, Voice of Reason marks the first successful application of Reinforcement Learning to math reasoning in speech-native models. --- ## 模型训练机制详解 > ## How the Training Works 在底层输出结构上，`GLM-4-Voice` 采用了序列交替机制：按固定节律**生成 13 个文本 Token，紧接着生成 26 个音频 Token**，循环往复。 > `GLM-4-Voice` interleaves its outputs sequentially: **13 text tokens followed by 26 audio tokens**, repeating. 1. **第一阶段：监督微调 (Stage-1 SFT) **：模型在来自 [Orca-Math](https://arxiv.org/abs/2402.14830) 的 150,616 道题目上进行微调。所有题目均使用 Qwen3-235B 进行了口语化重写，并借助 Kyutai 自研的 [DSM TTS](https://arxiv.org/abs/2509.08753) 系统合成为多种说话风格的口语音频。仅靠监督微调阶段，模型的数学解题准确率就直接从 27.3% 攀升至 61.7%。 2. **第二阶段：强化学习 (Stage-2 RL) **：对于每一个口语数学问题，模型在采样温度 0.9 下采样生成 4 个回答候选。裁判模型 (`Qwen3-235B-A22B-2507`) 在盲审机制下 (即不依赖标准参考答案) 对解码后的文本给出 0 或 1 的二元奖励 (Binary Reward) 。算法采用与 [GRPO](https://arxiv.org/abs/2402.03300) 相关的 REINFORCE 目标函数计算群组相对奖励 (Group-Relative Rewards) ，并移除了 PPO 中的截断裁剪操作与 KL 散度正则化约束。强化学习训练共调用了 16 张 H100 GPU，历经 1,500 次 RL 梯度更新完成训练。 > 1. **Stage-1 SFT:** Trained on 150,616 problems from [Orca-Math](https://arxiv.org/abs/2402.14830). Problems were rewritten for speech using Qwen3-235B and voiced in various styles using Kyutai's [DSM TTS](https://arxiv.org/abs/2509.08753). SFT alone lifts accuracy from 27.3% to 61.7%. > 2. **Stage-2 RL:** For each spoken question, the model samples 4 replies at temperature 0.9. A judge (`Qwen3-235B-A22B-2507`) scores the decoded text via a binary reward (blind to the reference answer). Rewards are group-relative using a REINFORCE objective related to [GRPO](https://arxiv.org/abs/2402.03300), while dropping PPO clipping and KL regularization. Training used 16 H100 GPUs for 1,500 RL updates. https://insight.aitobox.com/blog/posts/2026-09-24/HBQ-兼顾硬件效率与精度的分层块量化架构-加速大模型推理/ https://insight.aitobox.com/blog/posts/2026-09-24/HBQ-兼顾硬件效率与精度的分层块量化架构-加速大模型推理/ Thu, 24 Sep 2026 00:00:00 GMT 随着大语言模型 (Large Language Model, LLM) 规模的不断激增，海量参数给端侧与数据中心推理带来了巨大的内存带宽与能耗负担，量化压缩技术因而成为突破这一算力瓶颈的关键钥匙。然而，传统块量化 (Block Quantization, BQ) 架构在硬件执行效率与模型输出精度之间存在着显著的矛盾：采用大分块能通过充分摊薄反量化和累加成本来大幅提升芯片吞吐，却因数值动态范围受限而导致精度严重劣化；小分块虽能保留精度，却带来了沉重的硬件开销。针对这一两难困局，入选微体系结构顶级学术会议 MICRO 2026 的学术研究提出了 **HBQ (Hierarchical Block Quantization)** 架构。该方法将大尺寸分块与超低开销的有效数 (Significand, SIG) 缩放相结合，在第二层分级中精准补偿了大分块带来的量化误差，同时完美适配了权重与激活值迥异的统计分布特性。在 28nm ASIC 专用加速芯片上的实测表明，HBQ 实现了对权重、激活值与 KV Cache 的端到端量化支持；在保持甚至超越更高位宽精度的前提下，相较于当前最先进的仅权重量化技术实现了 **2.3 倍的面积效率与 4.6 倍的能效提升**，相比经典块量化方案系统能耗降低最高达 3.3 倍，为大语言模型的高效硬件落地提供了突破性的新路径。 --- ## 📌 内容总结 > ## 📌 Summary **HBQ (Hierarchical Block Quantization)** 是一套兼顾硬件执行效率感知、专为大语言模型 (Large Language Model, LLM) 高精度推理打造的创新量化框架。通过深入探索 Block Quantization (BQ) 中硬件效率与计算精度之间的权衡关系，研究人员发现：采用较大的数据分块虽然能够通过摊薄反量化与数据累加的硬件开销来提升芯片效率，但往往会造成严重的模型精度下降。为破解这一两难困境，HBQ 提出了创新的分层量化策略——以大分块作为第一层基础，并引入硬件开销极低的有效数 (Significand, SIG) 缩放作为第二层量化。这种分层设计既能契合权重与激活值截然不同的数值分布特性，又能高精度地弥补大分块带来的量化误差。在 28nm ASIC 专用加速器上的实测表明，HBQ 在模型精度与硬件效率两个核心维度上，均大幅超越了当前业内最先进的 Block Quantization 方案以及仅权重量化方法。 > **HBQ (Hierarchical Block Quantization)** is a hardware-efficiency-aware quantization framework designed for accurate Large Language Model (LLM) inference. By exploring the trade-offs in Block Quantization (BQ), the authors identify that larger blocks improve hardware efficiency (by amortizing dequantization and accumulation costs) but hurt accuracy. HBQ resolves this dilemma by pairing large blocks with low-overhead significand (SIG) scaling for second-level quantization, effectively compensating for errors while accommodating distinct weight and activation distributions. Implemented on a 28nm ASIC accelerator, HBQ significantly outperforms state-of-the-art BQ and weight-only quantization methods in both accuracy and hardware efficiency. --- ## 📋 论文元数据 > ## 📋 Metadata * **arXiv 编号：** [arXiv:2609.00450](https://arxiv.org/abs/2609.00450) [cs.LG] *(v2)* * **研究领域：** 机器学习 (`cs.LG`)；人工智能 (`cs.AI`)；硬件架构 (`cs.AR`) * **录用会议：** 第 59 届 IEEE/ACM 国际微体系结构研讨会 (**MICRO 2026**) * **提交日期：** 2026 年 8 月 31 日 (最后修订：2026 年 9 月 21 日) * **论文作者：** * Chun-Ting Chen * Dongmin Han * Hangyeol Mun * Jake Hyun * Arnab Raha * Amit Agarwal * Mark Anders * Mohamed Abdelfattah * Jae-sun Seo * **代码仓库：** [GitHub - SeoLabCornell/HBQ](https://github.com/SeoLabCornell/HBQ.git) > * **arXiv ID:** [arXiv:2609.00450](https://arxiv.org/abs/2609.00450) [cs.LG] *(v2)* > * **Subjects:** Machine Learning (`cs.LG`); Artificial Intelligence (`cs.AI`); Hardware Architecture (`cs.AR`) > * **Conference Acceptance:** Accepted to the 59th IEEE/ACM International Symposium on Microarchitecture (**MICRO 2026**) > * **Submission Date:** August 31, 2026 (Last revised: September 21, 2026) > * **Authors:** > * Chun-Ting Chen > * Dongmin Han > * Hangyeol Mun > * Jake Hyun > * Arnab Raha > * Amit Agarwal > * Mark Anders > * Mohamed Abdelfattah > * Jae-sun Seo > * **Code Repository:** [GitHub - SeoLabCornell/HBQ](https://github.com/SeoLabCornell/HBQ.git) --- ## 🔍 论文摘要 > ## 🔍 Abstract Block Quantization (BQ) 技术通过同时对神经网络的权重与激活值进行低比特压缩，能够显著降低大模型的计算与存储开销，是加速大模型推理的强力手段。然而，该技术底层的架构设计空间至今仍未被充分挖掘。通过在硬件能耗与模型精度之间进行深入的系统性探索，我们发现分块尺寸 (Block Size) 是决定整体性能最为核心的权衡要素：分块越大，反量化计算与数据累加所耗费的硬件成本被摊薄得越彻底，芯片运行效率也就越高；然而，过大的分块会导致量化误差急剧积累，从而显著损害模型输出的准确率。 > Block Quantization (BQ) enables efficient LLM inference by quantizing both weights and activations, but its design space remains underexplored. Through hardware-accuracy design space exploration, we identify block size as a key trade-off: larger blocks improve hardware efficiency by amortizing dequantization and accumulation costs, but degrade accuracy. 正是基于这一深刻洞察，我们提出了分层块量化架构 **HBQ (Hierarchical Block Quantization)**。该架构创造性地将粗粒度的大尺寸分块与开销极低的有效数 (Significand, SIG) 缩放机制深度融合，构建起精巧的第二级量化体系。通过 SIG 缩放微调，不仅能够精确补偿大分块粗放量化所带来的精度损失，还能自适应地契合权重与激活值各自独特的统计分布模式，实现精度与算力成本的双赢。 > Motivated by this insight, we propose **Hierarchical Block Quantization (HBQ)**, which combines large blocks with low-overhead significand (SIG) scaling for second-level quantization. SIG scaling effectively compensates for large-block quantization errors while accounting for distinct weight and activation distributions. * **HBQ-A** 仅采用超紧凑的 W4A5 配置，便成功达到了 W4A16 级别的模型精度，且硬件占用面积比 NVFP4 更加精简。 * **HBQ-E** 在精度上全面超越现有 Block Quantization 方法的同时，将硬件成本进一步大幅削减了 17%。 > * **HBQ-A** achieves W4A16-level accuracy with W4A5 and lower area than NVFP4. > * **HBQ-E** further reduces hardware cost by 17% while outperforming existing BQ methods in accuracy. 我们在 28nm ASIC 专用加速器上，全面实现了针对权重、激活值以及 KV Cache 的端到端 HBQ 硬件支持，并创新性地引入了部分和 (Partial-Sum) BQ 技术以大幅削减累加过程中的能耗开销。实测数据表明：在保持相当精度的前提下，HBQ 相比当前顶尖的仅权重量化方案，实现了 **2.3 倍的面积效率与 4.6 倍的能效提升**；相比先前的各路 Block Quantization 方案，更是实现了 **1.6 至 3.3 倍的系统能耗削减与 1.5 至 3 倍的推理提速**。 > We implement HBQ for weights, activations, and KV cache in a 28nm ASIC accelerator and introduce partial-sum BQ to reduce EMA energy. At comparable accuracy, HBQ achieves **2.3×/4.6× higher area/energy efficiency** than state-of-the-art weight-only quantization and **1.6–3.3× lower system energy with 1.5–3× speedup** over prior BQ methods. --- ## 🔗 链接与资源 (Links & Resources) > ## 🔗 Links & Resources * **论文全文：** [查看 PDF](https://arxiv.org/pdf/2609.00450) | [HTML 版本](https://arxiv.org/html/2609.00450v2) * **开源代码：** [GitHub 仓库](https://github.com/SeoLabCornell/HBQ.git) * **许可协议：** [Creative Commons Attribution 4.0](http://creativecommons.org/licenses/by/4.0/)  * **引用与索引：** [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.00450) | [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.00450) | [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.00450) > * **Full-Text Access:** [View PDF](https://arxiv.org/pdf/2609.00450) | [HTML Version](https://arxiv.org/html/2609.00450v2) > * **Source Code:** [GitHub Repository](https://github.com/SeoLabCornell/HBQ.git) > * **License:** [Creative Commons Attribution 4.0](http://creativecommons.org/licenses/by/4.0/)  > * **Citations & Metrics:** [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.00450) | [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.00450) | [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.00450) https://insight.aitobox.com/blog/posts/2026-09-23/Transformers-原生支持运行-llama.cpp-GGUF-量化模型：Apple-Silicon-性能逼近原生/ https://insight.aitobox.com/blog/posts/2026-09-23/Transformers-原生支持运行-llama.cpp-GGUF-量化模型：Apple-Silicon-性能逼近原生/ Wed, 23 Sep 2026 00:00:00 GMT 长期以来，以 llama.cpp 为代表的轻量化 C/C++ 运行时凭借紧凑的 GGUF 量化格式和极高的端侧执行效率，成为了开发者在个人电脑 (尤其是 Apple Silicon Mac) 上运行大语言模型 (Large Language Model, LLM) 的首选方案；而 Hugging Face 的 Transformers 虽是学术研究、微调实验与模型定义的事实标准，但在本地量化推理方面却因 Python 运行开销和缺乏专属低比特算子支持而略显吃力。为了打破两大主流生态之间的壁垒，Transformers 官方宣布通过接入高性能算子库原生支持 GGUF 格式。该方案直接复用了针对 Apple Silicon Metal 优化的 ggml 底层计算内核，并对 Python 生成循环中的同步瓶颈进行了深度重构与优化。基准测试表明，Transformers 在运行 Qwen3.5 等前沿模型时吞吐量已极其逼近原生 llama.cpp。这一突破让开发者既能保留 PyTorch 和 Transformers 灵活易用的开发生态 (如直接挂载钩子、动态调试与全流程评估) ，又能零性能损耗地畅享轻量级 GGUF 带来的端侧推理红利。 --- **核心摘要**：Hugging Face 旗下的 `transformers` 库现已原生支持在本地高效运行 `llama.cpp` (GGUF) 量化模型。通过复用底层的 `ggml` 算子内核 (首发通过 Metal 针对 Apple Silicon 进行深度优化) ，并大幅削减 Python 生成循环中的调度开销，开发者现在无需牺牲本地推理性能，即可直接借助熟悉的 `transformers` API 无缝加载、评估、微调以及对外提供 GGUF 权重检查点服务。 > **Summary:** Hugging Face `transformers` now natively supports running efficient `llama.cpp` (GGUF) quantized models locally. By reusing underlying `ggml` kernels (initially optimized for Apple Silicon via Metal) and reducing overhead in Python's generation loops, developers can now seamlessly load, evaluate, fine-tune, and serve GGUF checkpoints using familiar `transformers` APIs without sacrificing local inference performance. --- ## 目录 > ## Table of Contents - [项目概述](#overview) - [什么是 GGUF 文件格式？](#what-is-the-gguf-file-format) - [在 Transformers 中加载 GGUF 模型](#load-gguf-with-transformers) - [通过常用接口部署 GGUF 服务](#serve-gguf-with-your-preferred-interface) - [对比 llama.cpp 的性能基准测试](#benchmarking-against-llamacpp) - [Transformers 与 llama.cpp：优势互补的两大生态](#transformers-and-llamacpp-complementary-roles) - [超越 GGUF：为更多架构注入 ggml 算子加速](#beyond-gguf-ggml-kernels-for-more-models) - [基于 Python 与 PyTorch 的本地高速推理实现](#fast-local-inference-with-python-and-pytorch) - [复用 ggml 的 Metal 硬件内核](#reusing-ggmls-metal-kernels) - [消除瓶颈：让 CPU 与 GPU 高效协同](#keeping-the-cpu-and-gpu-working-together) - [当前局限与未来规划](#current-limitations-and-next-steps) - [致谢](#acknowledgments) > - [Overview](#overview) > - [What is the GGUF File Format?](#what-is-the-gguf-file-format) > - [Load GGUF with Transformers](#load-gguf-with-transformers) > - [Serve GGUF with Your Preferred Interface](#serve-gguf-with-your-preferred-interface) > - [Benchmarking Against llama.cpp](#benchmarking-against-llamacpp) > - [Transformers and llama.cpp: Complementary Roles](#transformers-and-llamacpp-complementary-roles) > - [Beyond GGUF: ggml Kernels for More Models](#beyond-gguf-ggml-kernels-for-more-models) > - [Fast Local Inference with Python and PyTorch](#fast-local-inference-with-python-and-pytorch) > - [Reusing ggml's Metal Kernels](#reusing-ggmls-metal-kernels) > - [Keeping the CPU and GPU Working Together](#keeping-the-cpu-and-gpu-working-together) > - [Current Limitations and Next Steps](#current-limitations-and-next-steps) > - [Acknowledgments](#acknowledgments) --- ## 项目概述 > ## Overview 我们在 **transformers** 中正式加入了对 GGUF 模型的高效运行支持。现在，开发者只需借助早已烂熟于心的 transformers 常用 API，即可直接加载和运行适配个人笔记本电脑显存容量的模型检查点 (Checkpoint) 。从 Hugging Face Hub 上挑选心仪的 GGUF 模型，一行 `from_pretrained` 即可载入内存，在本地机器上立即开启模型推理与内容生成。 > We're adding support for running GGUF models efficiently in **transformers**, so you can use checkpoints sized for your laptop's memory through the familiar transformers APIs. Pick a GGUF from the Hub, load it with `from_pretrained`, and start generating on your own machine. 如今在个人笔记本电脑上流畅运行 AI 模型已经不再是奢望，而 [llama.cpp](https://github.com/ggml-org/llama.cpp) 在这场本地化浪潮中功不可没。其轻量高效的推理引擎正是诸如 Ollama、LM Studio 和 Jan 等主流桌面端 AI 客户端背后的核心驱动力。与苹果生态的 [MLX](https://github.com/ml-explore/mlx) 等项目齐头并进，它让本地离线推理真正走入了大众的日常工作流，成为了极具实用价值的生产力工具。 > Running AI models on your laptop has become much easier, and [llama.cpp](https://github.com/ggml-org/llama.cpp) has been a big part of that. Its inference engine powers local AI tools such as Ollama, LM Studio, and Jan. Alongside projects like [MLX](https://github.com/ml-explore/mlx), it has helped make local inference a practical option for everyday use. 由 llama.cpp 团队倾力打造的 **GGUF** 格式，早已成为本地端侧推理领域事实上的通用标准。该团队不仅在 Hugging Face Hub 的 [ggml-org 主页](https://huggingface.co/ggml-org) 下维护发布了海量量化模型检查点，同时诸如 [Unsloth](https://huggingface.co/unsloth)、[LM Studio Community](https://huggingface.co/lmstudio-community) 以及开源量化先锋 [bartowski](https://huggingface.co/bartowski) 等著名发布者，也提供了覆盖各种精度规格、开箱即用的 GGUF 检查点，方便用户根据自身设备的硬件配置按需取用。迄今为止，各类 GGUF 模型在社区中的累计下载量已达数百万次之多。 > **GGUF**, developed by the llama.cpp team, is a widely used format for local inference. The team also shares quantized checkpoints under [ggml-org on the Hub](https://huggingface.co/ggml-org). Publishers such as [Unsloth](https://huggingface.co/unsloth), [LM Studio Community](https://huggingface.co/lmstudio-community), and [bartowski](https://huggingface.co/bartowski) also provide ready-to-use GGUF checkpoints in a range of quantizations, so users can pick the version that fits their machine. GGUF models have been downloaded millions of times. 为了将 Transformers 的运行速度提升至足以与 `llama.cpp` 原生性能平起平坐的水平，我们通过 [`kernels`](https://huggingface.co/docs/kernels/index) 算子库直接复用了 `ggml` 底层的核心计算算子，同时深度精简了 `generate` 生成循环中的调度与同步开销。在推进的第一阶段，我们重点攻关基于 Apple Silicon 平台的本地推理加速，并首批全面适配了 Qwen3.5 架构。 > To bring performance close to `llama.cpp`, we're reusing its underlying `ggml` kernels through the [`kernels`](https://huggingface.co/docs/kernels/index) library and reducing overhead in `generate`. Our initial focus is local inference on Apple Silicon, starting with the Qwen3.5 architecture. --- ## 什么是 GGUF 文件格式？ > ## What is the GGUF File Format? [GGUF](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md) 是一种将模型权重与其全部元数据 (包括分词器 Tokenizer 信息以及可选的对话模板 Chat Template) 打包在单个独立文件中的轻量封装格式。它原生支持多种不同粒度的量化级别，允许开发者以极微小的精度折损为代价，大幅降低模型的显存占用。以 `Q4_K_M` 这一经典变体为例，它巧妙采用了混合张量精度的策略：核心权重主要压缩为 4-bit 表示，而在关键敏感层张量上则维持更高的计算精度。 > [GGUF](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md) packages model weights and metadata, including tokenizer information and an optional chat template, into a single file. It supports different quantization levels, letting you trade some precision for a smaller memory footprint. Variants such as `Q4_K_M` mix tensor precisions, using mostly 4-bit weights while keeping sensitive tensors at higher precision. 以下是以 [Unsloth 维护的 Qwen3.5-4B](https://huggingface.co/unsloth/Qwen3.5-4B-GGUF/tree/main) 为例，不同量化规格对模型体积的具体影响对比： > Here's how quantization changes the file size of [Unsloth's Qwen3.5-4B](https://huggingface.co/unsloth/Qwen3.5-4B-GGUF/tree/main): | GGUF 变体规格 | 文件大小 | 特性权衡 (Tradeoff) | | :--- | ---: | :--- | | `BF16` | 8.42 GB | 未量化的基准参考版本 | | `Q6_K` | 3.53 GB | 相比更小变体保留了更高的精度 | | `Q5_K_M` | 3.14 GB | 体积与精度兼得的折中优选方案 | | `Q4_K_M` | 2.74 GB | 本地推理最实用的黄金起点 | 我们建议大多数用户可以先从 `Q4_K_M` 开始上手体验；若设备的可用内存或显存较为充裕，则可以进一步尝试 `Q5_K_M` 或 `Q6_K`。如果想深入探究不同量化类型背后的技术差异，可以参阅 [Hugging Face Hub 的 GGUF 官方文档](https://huggingface.co/docs/hub/gguf#quantization-types)。 > We suggest starting with `Q4_K_M`, then trying `Q5_K_M` or `Q6_K` if you have more memory available. The [Hub's GGUF documentation](https://huggingface.co/docs/hub/gguf#quantization-types) describes the available quantization types in detail. --- ## 在 Transformers 中加载 GGUF 模型 > ## Load GGUF with Transformers 开始上手前，你的环境需要满足以下基本要求： * **一台搭载 Apple Silicon 芯片的 Mac 电脑**； * **与官方发布的 [ggml-quantization 算子编译版本](https://huggingface.co/kernels/ggml-org/ggml-quantization) 相兼容的 PyTorch 版本** (通常涵盖最新的两个 PyTorch 正式版本) ； * **最新版本的 transformers 库** (当前推荐使用 main 分支代码) 以及相匹配的 `kernels` 算子库。 > To get started, you need: > * **An Apple Silicon Mac**. > * **A PyTorch version supported by the published [ggml-quantization kernel builds](https://huggingface.co/kernels/ggml-org/ggml-quantization)** (usually the two latest PyTorch releases). > * **The latest version of transformers** (main branch for now) and a compatible version of `kernels`. ```bash pip install -U "git+https://github.com/huggingface/transformers.git" kernels ``` 加载 GGUF 模型非常简单，只需在调用 `from_pretrained` 时，将模型在 Hub 上的 `model_id` 以及目标文件名传入 `gguf_file` 参数即可： > To load a GGUF model, pass its Hub `model_id` and filename as `gguf_file` to `from_pretrained`: ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "unsloth/Qwen3.5-4B-GGUF" filename = "Qwen3.5-4B-Q4_K_M.gguf" tokenizer = AutoTokenizer.from_pretrained(model_id, gguf_file=filename) model = AutoModelForCausalLM.from_pretrained( model_id, gguf_file=filename ) ``` 模型加载完成后，后续的所有推理与调用代码均完全沿用标准的 transformers API 接口： > Everything after loading uses the standard transformers API: ```python messages = [{"role": "user", "content": "Explain why the sky is blue in a few sentences."}] inputs = tokenizer.apply_chat_template( messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt", ).to(model.device) with torch.inference_mode(): outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) ``` > 💬 [原文引用 / Original Quote]: > **警告**：如果没有安装兼容的量化算子内核，加载器将回退到对模型进行反量化 (Dequantization) 处理，这将显著增加内存消耗。 > > **Warning:** Without a compatible quantization kernel, the loader falls back to dequantizing the model and uses significantly more memory. --- ## 通过常用接口部署 GGUF 服务 > ## Serve GGUF with Your Preferred Interface 你可以使用 [`transformers serve`](https://huggingface.co/docs/transformers/main/en/serve-cli/serving) 命令行工具，快速对外暴露一个与 OpenAI 规范完全兼容的 API 接口服务： > You can expose an OpenAI-compatible API using [`transformers serve`](https://huggingface.co/docs/transformers/main/en/serve-cli/serving): ```bash pip install -U "transformers[serving] @ git+https://github.com/huggingface/transformers.git" kernels transformers serve "unsloth/Qwen3.5-4B-GGUF:Qwen3.5-4B-Q4_K_M.gguf" ``` 随后，你可以在诸如 [Jan](https://www.jan.ai/docs/desktop/remote-models/custom-endpoint) 或 [Pi](https://pi.dev) 等第三方客户端中配置如下自定义连接参数，轻松接入并使用该本地模型： > You can connect clients like [Jan](https://www.jan.ai/docs/desktop/remote-models/custom-endpoint) or [Pi](https://pi.dev) via these custom settings: | 配置项 (Setting) | 配置值 (Value) | | :--- | :--- | | Base URL | `http://localhost:8000/v1` | | Model ID | `unsloth/Qwen3.5-4B-GGUF:Qwen3.5-4B-Q4_K_M.gguf` | --- ## 对比 llama.cpp 的性能基准测试 > ## Benchmarking against llama.cpp 在本地端侧推理性能方面，我们的核心参照基准是 `llama.cpp`。在配备 **32 GB 统一内存的 MacBook Pro M2 Max** 设备上进行的实测结果显示，Transformers 所实现的生成吞吐量已极其逼近原生 `llama.cpp` 的表现。 > Our reference for local inference performance is `llama.cpp`. Measured on a **MacBook Pro M2 Max, 32 GB unified memory**, transformers achieves throughput remarkably close to native `llama.cpp`. [](./images/cb3f00fa2c44.svg) --- ## Transformers 与 llama.cpp：优势互补的两大生态 > ## Transformers and llama.cpp 在 [GGML 与 llama.cpp 正式加入 Hugging Face 大家庭](https://huggingface.co/blog/ggml-joins-hf) 之际，我们曾阐述过二者之间优势互补的分工定位：`llama.cpp` 为本地高效推理筑牢了坚实基石，而 `transformers` 则为大模型的结构定义与生态拓展提供了核心底座。 > When [GGML and llama.cpp joined Hugging Face](https://huggingface.co/blog/ggml-joins-hf), we described their complementary roles: `llama.cpp` provides a foundation for local inference, while `transformers` provides a foundation for model definition. 在 PyTorch 中原生支持 GGUF，彻底解锁了许多极具生产力的高阶工作流： * **畅享 Python 灵活实验环境**：利用熟悉的 PyTorch 工具链，通过挂载钩子 (Hooks) 检查中间层激活值、自由定制修改前向传播 (Forward Pass) 逻辑，或是快速构建自定义网络层的原型； * **便捷评估 GGUF 模型质量**：直接复用现有的评估测试流水线，精准衡量量化检查点的输出质量； * **基于 GGUF 权重进行微调**：只需传入 `GgufConfig(dequantize=True)` 将量化权重反量化为浮点，即可无缝衔接标准的模型训练与微调 (Fine-Tuning) 工作流： > GGUF support unlocks powerful workflows in PyTorch: > * **Experiment in Python:** Inspect intermediate activations with hooks, modify forward passes, or prototype custom layers using familiar PyTorch tools. > * **Evaluate GGUF Models:** Use your existing evaluation pipelines to measure quantized checkpoint quality. > * **Fine-tune from GGUF:** Dequantize weights and continue with standard training workflows using `GgufConfig(dequantize=True)`: ```python import torch from transformers import AutoModelForCausalLM, GgufConfig model = AutoModelForCausalLM.from_pretrained( "unsloth/Qwen3.5-4B-GGUF", gguf_file="Qwen3.5-4B-Q4_K_M.gguf", quantization_config=GgufConfig(dequantize=True), dtype=torch.bfloat16, ) ``` --- ## 超越 GGUF：为更多架构注入 ggml 算子加速 > ## Beyond GGUF: ggml Kernels for More Models 更为诱人的技术前景，在于将 `ggml` 的极致性能推广到目前 `llama.cpp` 尚未官方支持的更广阔模型天地中。通过在 PyTorch 中直接调用 `ggml` 算子内核与量化方案，我们可以直接为计算机视觉、音频处理以及多模态 (Multimodal) 流水线中的各类前沿研究模型、全新网络架构及自定义变体提供显著的计算加速，再也无需苦苦等待专用独立运行时的繁琐适配。 > The bigger opportunity is bringing `ggml`'s performance to models that `llama.cpp` does not support. By utilizing `ggml` kernels and quantization schemes directly in PyTorch, we can accelerate operations for research models, new architectures, and custom variants across computer vision, audio, and multimodal pipelines without waiting for dedicated runtime implementations. --- ## 基于 Python 与 PyTorch 的本地高速推理实现 > ## Fast Local Inference with Python and PyTorch https://insight.aitobox.com/blog/posts/2026-09-22/阿里开源-Qwen-Image-2.1-统一文生图与图像编辑的-7B-全能模型/ https://insight.aitobox.com/blog/posts/2026-09-22/阿里开源-Qwen-Image-2.1-统一文生图与图像编辑的-7B-全能模型/ Tue, 22 Sep 2026 00:00:00 GMT 在生成式 AI (Generative AI) 领域，文生图与图像编辑以往往往由相互独立的模型或检查点分别处理，既增加了部署复杂度，也限制了多任务协同的效率。阿里巴巴 Qwen 团队正式开源发布了视觉生成全能模型 **Qwen-Image-2.1**，成功将高品质文本生图与高级多图参考编辑融合至单个 7B 架构之中。该模型基于 32 层的单流扩散 Transformer (Diffusion Transformer, DiT) 设计，并结合 8B 的 Qwen3-VL 文本编码器，原生支持多达 10 张图像的多图参考编辑、涂鸦圈选等局部控制、原生透明通道 (RGBA) 输出以及超高清 2K 图像生成。凭借新颖的混合粒度注意力机制与前缀键值缓存 (Prefix KV Cache) 复用技术，Qwen-Image-2.1 在显著提升推理速度与部署经济性的同时，还在自研评测基准 Qwen-Image-Bench 上斩获 60.28 的高分，强力领跑当前开源权重图像模型，为视觉生成领域树立了兼具卓越画质与高能效的全新标杆。 --- ## 📋 内容总结 > ## 📋 Summary 阿里巴巴 Qwen 团队正式发布了 **Qwen-Image-2.1** —— 这是一款统一了文本生成图像 (Text-to-Image) 与高阶图像编辑能力的 7B 开源权重模型。该模型采用 32 层的扩散 Transformer (Diffusion Transformer, DiT) ，并搭配 8B 的 Qwen3-VL 文本编码器，仅凭单一检查点 (Checkpoint) 即可全能支持多图参考编辑、局部精细修改、原生透明通道 (RGBA) 输出以及 2K 分辨率的高清图像生成。得益于混合粒度注意力机制 (Mixed-Granularity Attention) 与前缀键值缓存 (Prefix KV Cache) 技术，该模型在交付开源权重领域顶尖 SOTA 表现的同时，还在 Diffusers、ComfyUI、vLLM-Omni 以及 SGLang 等主流平台上大幅优化了推理速度与部署效率。 > Alibaba’s Qwen team has released **Qwen-Image-2.1**, a unified 7B open-weight model for text-to-image generation and advanced image editing. Featuring a 32-layer Diffusion Transformer (DiT) combined with an 8B Qwen3-VL text encoder, this single checkpoint supports multi-reference editing, local edits, native transparency (RGBA), and 2K resolution generation. Thanks to mixed-granularity attention and prefix KV caching, the model delivers state-of-the-art open-weight performance while optimizing speed and deployment efficiency across platforms like Diffusers, ComfyUI, vLLM-Omni, and SGLang. --- ## 🚀 核心概述 > ## 🚀 Overview 阿里巴巴 Qwen 团队推出了 [Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1)，这是一款兼具文本生图与图像编辑功能的统一视觉生成模型。其视觉生成核心组件在 32 个单流 DiT 层中容纳了 7B 参数，仅需一个模型权重检查点，便能全面涵盖文生图、多参考图编辑、局部针对性修改以及带透明通道的 RGBA 输出等多种生成任务。 > Alibaba’s Qwen team has introduced [Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1), a unified text-to-image generation and image editing model. Its visual generation component packs 7B parameters across 32 single-stream DiT layers, with a single checkpoint covering text-to-image, multi-reference editing, local edits, and transparent RGBA output. **该模型目前可以直接部署吗？** 对于学术研究与效果评估而言，答案是肯定的。模型发布首日 (Day 0) 便已全面获得 Diffusers、ComfyUI、vLLM-Omni、SGLang 和 LightX2V 等生态工具与框架的开箱即用支持；若计划将其投入商业化生产环境部署，则需单独获取 Qwen 官方的商业许可授权。 > **Is it deployable?** Yes, for research and evaluation. Day 0 support covers Diffusers, ComfyUI, vLLM-Omni, SGLang, and LightX2V. Commercial deployment requires a separate license from Qwen. --- ## 📉 轻量进化：从 20B 到 7B > ## 📉 From 20B to 7B 早先于 2025 年 8 月以 Apache 2.0 许可证开源的初代 [Qwen-Image](https://huggingface.co/Qwen/Qwen-Image) 是一个参数量达 20B 的大模型，且其图像编辑功能独立存放在另一个专门的检查点中。而新一代 Qwen-Image-2.1 将这两项重任合二为一，模型体积仅为初代的约三分之一，使其成为当前 Qwen-Image 家族中综合表现最为均衡、性价比最突出的代表之作。 > The original [Qwen-Image](https://huggingface.co/Qwen/Qwen-Image) shipped in August 2025 as a 20B model under Apache 2.0, where image editing lived in a separate checkpoint. Qwen-Image-2.1 folds both jobs into one model at about a third of the size, making it the most balanced and cost-effective model in the Qwen-Image series. *关于显存与算力规划的说明：* 7B 这一参数规模仅特指负责去噪生成的扩散 Transformer 主体。在实际运行流水线 (Pipeline) 时，系统还会同时加载一个 8B 参数的 Qwen3-VL 文本编码器。 > *Note on capacity planning:* The 7B figure covers the diffusion transformer only. The pipeline also loads an 8B Qwen3-VL encoder. --- ## ⚙️ 核心架构设计 > ## ⚙️ Architecture 官方 [GitHub 代码仓库](https://github.com/QwenLM/Qwen-Image-2.1#architecture) 详细列出了模型的 4 大核心构成组件： > The [GitHub Repo](https://github.com/QwenLM/Qwen-Image-2.1#architecture) lists 4 core components: * **Transformer：** 32 层、7B 参数，采用带块因果注意力 (Block-Causal Attention) 的单流设计。 * **文本编码器 (Text Encoder)：** Qwen3-VL 8B，负责将文本提示词指令与作为条件的参考图像编码并融合为统一的多模态表征。 * **变分自编码器 (VAE)：** 64 通道 RGBA 自编码器，具备 16 倍空间压缩率并原生支持透明通道。 * **调度器 (Scheduler)：** 流匹配 (Flow Matching) 框架，结合欧拉离散调度 (Euler Discrete Scheduling) 与动态时间步平移 (Dynamic Shifting) 策略。 > * **Transformer:** 32 layers, 7B parameters, single-stream design with block-causal attention. > * **Text encoder:** Qwen3-VL 8B, which encodes text instructions and condition images into a unified representation. > * **VAE:** 64-channel RGBA autoencoder with 16x spatial compression and native transparency. > * **Scheduler:** Flow Matching with Euler discrete scheduling and dynamic shifting. https://insight.aitobox.com/blog/posts/2026-09-22/长生命周期-AI-智能体的权限撤销机制-根域静默协议与异步执行安全/ https://insight.aitobox.com/blog/posts/2026-09-22/长生命周期-AI-智能体的权限撤销机制-根域静默协议与异步执行安全/ Tue, 22 Sep 2026 00:00:00 GMT 随着自主执行技术的飞速发展，长生命周期 AI 智能体 (AI Agent) 已经超越了传统的同步交互模式，广泛依赖异步任务队列、第三方回调、委托凭证与云端服务展开复杂协作。然而，当系统管理员或用户尝试撤销其授权时，传统的进程退出、操作取消或单纯的凭证撤销机制往往存在严重盲区，极易导致脱离管控的后台“僵尸操作”继续执行，或误伤其他获得独立合法授权的共享任务。针对这一核心安全痛点，Genliang Zhu 与 Chu Wang 提出了全新的 Root-Scoped Quiescence 协议，通过精确划定授权根边界、引入 Antichains 与信道 Token 守恒对账，从数学上确保授权撤销后所有前置操作均被精准核算、后置越权行为被彻底切断。该研究不仅给出了完备的形式化安全证明，还在真实的系统测试套件中充分验证了其抵御系统崩溃、重放攻击与异步晚期效应的稳健能力，为构建高可靠、可组合的分布式智能体安全生态提供了关键理论与工程基石。 --- # 长生命周期 AI 智能体的权限撤销机制：根域静默协议与异步执行安全 > # Authorization Revocation for Long-Running AI Agents: Root-Scoped Quiescence under Delegation and Asynchronous Execution **Authors:** Genliang Zhu, Chu Wang **Subjects:** Programming Languages (`cs.PL`); Artificial Intelligence (`cs.AI`); Cryptography and Security (`cs.CR`) **arXiv ID:** [arXiv:2609.21284 [cs.PL]](https://arxiv.org/abs/2609.21284) **Submitted:** 18 September 2026 --- ## 📌 执行摘要 > ## 📌 Executive Summary 在现代分布式自主计算中，长生命周期 AI 智能体的实际存活时间往往远超最初启动它们的原始进程——它们手握各种授权凭据，在后台分发委托任务、排队等待回调、预留外部资源，并持续触发服务商侧的云端操作。此时，传统的常规应对手段（例如任务取消、进程退出或直接撤销凭据）便暴露出严重不足：它们既无法彻底关闭在权限切割点之前就已经派发出去的每一个载体通道，也无法精准区分那些同时持有其他合法授权的共享协作工作。 > Long-running AI agents frequently outlive their initiating processes by leveraging credentials, delegated tasks, queues, callbacks, reservations, and provider-side operations. Standard methods like cancellation, process exit, and credential revocation fall short because they fail to close every pre-cut carrier or distinguish independently authorized shared work. 为了从根本上解决这一安全困境，本文提出了 **Root-Scoped Quiescence** 授权静默协议。该协议是一套全新设计的安全控制机制，旨在确保一旦某个授权根被正式废弃注销，所有与该切割点相关的既有前置操作都会被完整追踪与清算对账；与此同时，切割时刻之后的所有未经授权操作均被严格且彻底地杜绝。通过严密的数学证明与广泛的实证测试，作者展示了如何为复杂的异步 AI 系统构建起坚固、具备模块可组合性且在系统崩溃面前稳健如初的授权安全边界。 > This paper introduces **root-scoped authorization quiescence**, a novel protocol designed to ensure that once an authorization root is retired, all associated pre-cut actions are accounted for while unauthorized post-cut actions are strictly prevented. Through rigorous mathematical proofs and extensive empirical testing, the authors demonstrate how to maintain secure, compositional, and crash-resilient authorization boundaries for asynchronous AI systems. --- ## 🔍 摘要与核心概念 > ## 🔍 Abstract & Core Concepts 长生命周期 AI 智能体借助凭据、委托任务、消息队列、异步回调、资源预留及服务商侧操作，能够脱离发起进程而在后台持续运转。然而，传统的操作取消、进程退出和凭据撤销方法，既无法彻底封堵切割时刻之前派发的每一个有效载体，也无法精确区分由第三方独立充分授权的共享工作。 > Long-running AI agents outlive initiating processes through credentials, delegated tasks, queues, callbacks, reservations, and provider-side operations. Cancellation, process exit, and credential revocation neither close every pre-cut carrier nor distinguish independently authorized shared work. 为了化解上述顽疾，作者对 **Root-Scoped Quiescence** 授权静默进行了形式化定义： * 对于每一个已声明的目标终点 (Sink) ，系统都会出具一份专门凭证，完整核算在本地隔离屏障 (Fence) 建立前、基于已注销旧授权根纪元原子所接纳的一切与切割相关的合法操作； * 凭证严格排除了在隔离屏障生效之后基于该旧原子的任何受保护接纳行为，同时无缝支持任务精确重绑定 (Rebind) 至当前有效且独立充分的新授权支撑之上。 > To resolve this, the authors define **root-scoped authorization quiescence**: > * For each manifested sink, a certificate accounts for every cut-relevant acceptance under the retired root-epoch atom that precedes its local fence. > * It excludes protected acceptance under that atom after the fence, while permitting exact rebind to a current, independently sufficient support. https://insight.aitobox.com/blog/posts/2026-09-22/让-AI-智能体编写超越现有SOTA性能的-Rust-极限代码-迭代压测实战指南/ https://insight.aitobox.com/blog/posts/2026-09-22/让-AI-智能体编写超越现有SOTA性能的-Rust-极限代码-迭代压测实战指南/ Tue, 22 Sep 2026 00:00:00 GMT 本文探讨了如何利用现代前沿的大语言模型 (Large Language Model, LLM) 智能体，通过严密的约束条件、精细的提示词工程以及持续迭代的基准测试压测 (Benchmaxxing) ，编写出性能大幅超越现有顶尖开源库 (SOTA) 的 Rust 高性能代码。作者以 UMAP 降维算法、决策树、模板引擎以及 ASCII/字符云渲染等项目为例，展示了在杜绝 unsafe 危险代码的前提下，如何通过制定严格的防作弊基准规则、鼓励底层创新算法以及派生轻量级子智能体，实现 2 倍至 32 倍的惊人性能飞跃。这一探索不仅打破了 AI 生成代码往往是低质“代码垃圾 (Slop) ”的刻板印象，更为未来利用 Rust 重构并加速 Python 生态的高性能工程化落地指明了切实可行的路径。 --- ## 核心内容摘要 > ## Summary 本文深入探讨了如何借助具备自主行动能力的大语言模型 (Large Language Model, LLM) 智能体，在精准约束条件、精细提示词工程以及严密基准测试的引导下，编写出性能显著超越当前顶尖开源库 (State-of-the-Art, SOTA) 的 Rust 代码 (性能提升达 2 倍至 32 倍) 。通过将 Rust 原生具备的高运行速度与内存安全特性，与 Python 生态集成方案 (通过 PyO3) 以及一套持续迭代的“极限压测 (Benchmaxxing) ”工作流紧密结合，作者生动展示了如何在杜绝智能体基准测试作弊的同时，在机器学习库、Web 服务器及多媒体处理工具等多元领域中，激发出颠覆性的底层算法优化潜力。 > This article explores how modern agentic Large Language Models (LLMs)—guided by precise constraints, prompt engineering, and rigorous benchmarking—can write Rust code that significantly outperforms current state-of-the-art libraries (achieving 2x to 32x speedups). By combining Rust's native speed and memory safety with Python integration (via PyO3) and an iterative "benchmaxxing" pipeline, the author demonstrates how to prevent agents from cheating on benchmarks while unlocking radical, low-level algorithmic optimizations across machine learning libraries, web servers, and multimedia tools. --- ## 引言 > ## Introduction 早在 2025 年 1 月，我曾做过一项测试：如果仅仅给大语言模型提出“写出更优秀的代码”这一要求，它们究竟能否做到？虽然当时的 Claude Sonnet 3.5 倾向于堆砌臃肿的功能特性，而非产出纯粹极致的优化逻辑，但最终生成的代码在运行速度上确实变快了。这促使我设想了这样一种未来可能：大语言模型可以通过自动生成底层高性能的 **Rust** 代码，并借助 **PyO3** 工具库与 Python 实现桥接无缝绑定，从而让开发者写出极其飞速的 Python 代码。 > In January 2025, I tested whether LLMs could write better code simply if asked to “write better code.” While Claude Sonnet 3.5 added bloated features rather than purely optimized logic, the resulting code was indeed faster. This led to a hypothetical future where LLMs could write superfast Python code by generating underlying **Rust** code and bridging them with **PyO3**. 在 Opus 4.5 及后续一系列前沿模型发布后的数月测试与实验中，我终于能够确信：*只要设定了恰到好处的防护栏与约束规则*，当下的自主智能体大语言模型完全能够编写出速度大幅超越当前顶尖开源方案的 Rust 代码。在这篇文章中，我将系统拆解实现这些数倍乃至数十倍性能飙升所运用的提示词、基准测试规范与核心技术技巧。 > After months of testing and experimenting since the release of Opus 4.5 and subsequent frontier models, I can confirm that modern agentic LLMs write Rust code significantly faster than current state-of-the-art approaches *when given appropriate guardrails and constraints*. This post outlines the prompts, benchmarks, and techniques used to achieve these massive speedups. --- ## 持续迭代的“极限压测”策略 > ## Iterative “Benchmaxxing” 为了让软件跑得飞快，我将优化目标锁定在 Rust 上——看中的正是它首屈一指的执行速度、卓越的内存安全保障以及直接编译为 WebAssembly (WASM) 的能力。而为了确保代码库在实际工程中安全可靠，我强行施加了一条不可逾越的硬性约束：**绝不允许编写 unsafe 不安全代码**。 > To make software faster, I targeted Rust for its speed, memory safety, and ability to compile to WebAssembly (WASM). To keep the codebase practical, I enforced one strict constraint: **no unsafe code**. 我的首个测试课题是从零手写重新实现经典机器学习算法——具体而言，是在尽量减少 Rust 外部依赖的前提下，重新实现用于高维数据降维的 **UMAP** (Uniform Manifold Approximation and Projection，统一流形逼近与投影) 算法。 > My first test case involved reimplementing machine learning algorithms from scratch—specifically **UMAP** (Uniform Manifold Approximation and Projection) for dimensionality reduction—with minimal Rust dependencies. 借助 Rust 生态中功能强大且严密的 **`criterion`** 基准测试工具库，智能体能够在每一轮优化迭代中精确跟踪并量化代码的性能变化。 > Using Rust’s comprehensive **`criterion`** benchmarking crate, agents could track performance across iterations.  https://insight.aitobox.com/blog/posts/2026-09-22/大模型生成的-GPU-算子到底能触达多少真实工作负载-KernelBench-局限与-DLRM-突破/ https://insight.aitobox.com/blog/posts/2026-09-22/大模型生成的-GPU-算子到底能触达多少真实工作负载-KernelBench-局限与-DLRM-突破/ Tue, 22 Sep 2026 00:00:00 GMT 随着大语言模型 (LLM) 代码生成能力的突飞猛进，利用 AI 自动编写高性能 GPU 算子已成为性能工程领域的前沿热点，不少测试基准也显示大模型生成的算子速度超越了原生 PyTorch。然而，这项研究提出了一个关键却长期被忽视的问题：在真实深度学习模型的端到端整机耗时中，这些大模型生成的算子究竟能触达多大比例？实验表明，在主流的 Transformer 架构中，由于 80% 至 86% 的耗时都被工业界经过极致优化的基础算子 (如 `cuBLAS GEMM` 和 `FlashAttention`) 牢牢占据，大模型优化带来的端到端整体加速被锁死在 1% 左右；而在包含大量长尾算子的推荐系统 (DLRM) 中，可优化的耗时占比高达 58.2%，大模型算子成功实现了 8.63% 的端到端加速。此外，研究还敏锐地发现了权威基准 KernelBench 的严重评测漏洞——由于数值容差过宽，全零张量竟然也能通过正确性测试，为此作者团队提出了更稳健的无量纲尺度不变量评估方法。 --- # 大模型生成的 GPU 算子到底能触达多少真实工作负载？KernelBench 局限与 DLRM 突破 > # How Much of a Real Workload Can LLM-Generated GPU Kernels Actually Reach? **作者：** Gaurav Agarwal, Ashish Garg, Isha Singhal **发布时间：** 2026 年 9 月 17 日 **arXiv：** [2609.21058 [cs.DC]](https://arxiv.org/abs/2609.21058) | **DOI：** [10.48550/arXiv.2609.21058](https://doi.org/10.48550/arXiv.2609.21058) **相关资源：** [GitHub 仓库 (代码、数据与评测)](https://github.com/gauravapiscean/kernel-headroom) > **Authors:** Gaurav Agarwal, Ashish Garg, Isha Singhal > **Published:** 17 September 2026 > **arXiv:** [2609.21058 [cs.DC]](https://arxiv.org/abs/2609.21058) | **DOI:** [10.48550/arXiv.2609.21058](https://doi.org/10.48550/arXiv.2609.21058) > **Resources:** [GitHub Repository (Code, Data, and Evaluations)](https://github.com/gauravapiscean/kernel-headroom) --- ## 概要 > ## Summary 随着大语言模型 (Large Language Model, LLM) 技术的飞速演进，AI 编写定制 GPU 算子 (GPU Kernels) 的能力日益增强，甚至在不少任务上超越了标准 PyTorch 代码的执行效率。然而，这篇论文深入探讨了一个极其关键却长期被行业忽视的核心问题：**在真实深度学习模型的实际运行总耗时 (Wall-Clock Runtime) 中，这些由大模型自动生成的 GPU 算子到底能管辖、覆盖多大的比例？** > Recent advancements allow language models to write custom GPU kernels that can outperform standard PyTorch code. However, this paper investigates a crucial, overlooked question: **What fraction of a real model's wall-clock runtime do such LLM-generated kernels actually govern?** 作者团队在知名算子基准测试 KernelBench 上对前沿大模型进行了系统评估。结果表明，尽管前沿大模型编写正确算子的成功率高达 91.1% (相比之下开源权重模型仅为 30.4%)，但它们在真实业务负载中带来的端到端性能增益却非常有限。原因在于，对于主流的 Transformer 架构模型而言，高达 80% 至 86% 的运行时间都被工业界千锤百炼的基础算子 (如 `cuBLAS GEMM` 和 `FlashAttention`) 牢牢占据，这使得大模型能优化的长尾空间极其微小，最终折算到整机的现实端到端加速幅度被死死限制在 1% 左右。相反，推荐系统展现出了显著更高的可优化耗时比例 (58.2%)，为此作者构建并推出了全新的基准测试套件 **DLRM-Bench**，在该套件中大模型生成的算子取得了切实可衡量的加速突破。最后，该研究还曝光了 KernelBench 正确性验证机制中的一个严重漏洞——由于绝对容差设定过宽，全零张量竟然也能蒙混过关判定为正确；针对这一问题，作者提出了无量纲的尺度不变量验证指标作为更稳健的替代方案。 > Evaluating frontier models on benchmarks like KernelBench, the authors discover that while frontier models generate correct kernels for 91.1% of problems (compared to just 30.4% for open-weights alternatives), end-to-end performance gains on real workloads are severely limited. For transformer models, 80–86% of runtime is spent inside optimized primitives like `cuBLAS GEMM` and `FlashAttention`, capping realistic end-to-end improvements at roughly 1%. Conversely, recommender systems show higher addressable fractions (58.2%), leading the authors to introduce **DLRM-Bench**, a new benchmark suite where LLM-generated kernels achieve measurable speedups. Finally, the study exposes a flaw in KernelBench's correctness validation—where a tensor of zeros can pass tests due to loose tolerances—and proposes scale-invariant replacements. --- ## 核心发现 > ## Key Findings https://insight.aitobox.com/blog/posts/2026-09-22/Hugging-Face-发布-Tokenizers-v1-重构-SIMD-位运算与零内存分配带来3-30倍加速/ https://insight.aitobox.com/blog/posts/2026-09-22/Hugging-Face-发布-Tokenizers-v1-重构-SIMD-位运算与零内存分配带来3-30倍加速/ Tue, 22 Sep 2026 00:00:00 GMT 在大语言模型 (Large Language Model, LLM) 的推理与训练过程中，分词 (Tokenization) 环节以往常被视作微不足道的开销。然而随着模型吞吐量的飞速攀升，原本不起眼的分词流程已逐渐演变成让昂贵 GPU 陷入空闲等待的关键数据瓶颈。为了彻底攻克这一难题，Hugging Face 对其基石级开源库 Tokenizers 进行了底层的全方位架构重构，正式推出即将发布的 Tokenizers v1。该版本在严格保证输出 Token ID 与原有模型完全兼容的前提下，通过引入基于 SIMD 的 Bitcannon 文本切分技术、无堆内存分配的合并循环以及线程级词缓存等前沿工程手段，实现了相比 v0.23 版本 3 到 30 倍的惊人性能飞跃，为高并发服务与海量数据预处理扫清了吞吐障碍。 --- ## 概要 > ## Summary 随着机器学习模型运行越来越快、处理的工作负载越来越大，系统的数据瓶颈也在悄然发生转移。分词 (Tokenization) ——这一传统上被视作轻量级的前置步骤，在海量数据集训练、应对高并发请求或处理超长上下文输入时，很容易迅速沦为整条流水线的性能瓶颈，最终导致昂贵的 GPU 算力白白处于闲置等待状态。 > As machine learning models grow faster and handle larger workloads, data bottlenecks shift. Tokenization—traditionally a lightweight step—can quickly become a bottleneck when training on massive datasets, serving concurrent requests, or processing lengthy inputs, ultimately leaving expensive GPUs sitting idle. 为了解决这一痛点，Hugging Face 团队针对即将发布的 **Tokenizers v1** 展开了极致的性能攻坚。新版本在完全保留输出兼容性、API 接口、词表以及合并优先权重 (Merge Ranks) 的同时，实现了大幅性能跃升 (比 v0.23 版本提速高达 3 至 30 倍) 。版本 1 带来了一场彻底的底层架构重构： > To solve this, the Hugging Face team has heavily focused on performance for the upcoming **Tokenizers v1**. Built to be significantly faster (up to 3 to 30x faster than v0.23) while preserving output compatibility, APIs, vocabularies, and merge ranks, version 1 introduces a complete architectural refactor: * **工作空间拆分 (Workspace Split) ：** 将核心运行时 (`tk-encode`) 与模型训练、序列化以及转换工具解耦模块化。 * **Bitcannon：** 用基于单指令多数据流 (SIMD) 驱动的位流运算取代了通用正则表达式引擎来进行文本切分。 * **零内存分配模型与合并循环重写 (No-Alloc Model & Merge-Loop Rewrite) ：** 通过由调用方持有的临时暂存缓冲区 (Scratch Buffers) 与侵入式链表 (Intrusive Linked Lists) ，彻底消除了分词过程中的堆内存分配开销。 * **词缓存 (The Word Cache) ：** 缓存先前已处理过的预分词片段 (Pre-tokens) ，以此加速重复文本段的处理。 * **原生并发 (Native Parallelism) ：** 支持跨多线程共享且线程安全的高并发编码，消除了互斥锁竞争。 > * **Workspace Split:** Modularizes the core runtime (`tk-encode`) from training, serialization, and conversion tools. > * **Bitcannon:** Replaces general-purpose regex engines with SIMD-powered bitstream operations for text splitting. > * **No-Alloc Model & Merge-Loop Rewrite:** Eliminates heap allocations during tokenization using caller-owned scratch buffers and intrusive linked lists. > * **The Word Cache:** Caches previously processed pre-tokens to speed up repeated text segments. > * **Native Parallelism:** Enables shared, thread-safe encoding without lock contention. --- ## V1 版本究竟是什么 > ## What V1 Is 版本 1 在生成与 v0.23 完全相同的 Token ID 的同时，全面榨干了所有受支持分词器家族 (包括 BPE、WordPiece 和 Unigram) 的性能极限。 > Version 1 produces the exact same token IDs as v0.23 while maximizing performance across all supported tokenizer families (including BPE, WordPiece, and Unigram). 一条标准的分词处理流水线通常由四个阶段组成： > A standard tokenization pipeline runs in four stages: 1. **规范化 (Normalization) ：** 对原始文本执行小写转换或应用 Unicode 规范化处理。 2. **预分词 (Pre-tokenization) ：** 将连续文本初步拆分为更小的切片 (“预分词片段/pre-tokens”) 。 3. **核心模型 (Model) ：** 将预分词片段转化为具体的 Token 并将其映射到词表中的 ID (这正是 v1 重点攻克的核心性能瓶颈) 。 4. **后处理 (Post-processing) ：** 添加特定模型所需的特殊 Token (如起始、结束标记等) 。 > 1. **Normalization:** Lowercasing or applying Unicode normalization to raw text. > 2. **Pre-tokenization:** Splitting text into smaller chunks ("pre-tokens"). > 3. **Model:** Converting pre-tokens into tokens and mapping them to vocabulary IDs (the primary performance bottleneck addressed in v1). > 4. **Post-processing:** Adding model-specific special tokens. https://insight.aitobox.com/blog/posts/2026-09-22/CodeMidas-直接从源代码构建大规模代码智能体强化学习环境/ https://insight.aitobox.com/blog/posts/2026-09-22/CodeMidas-直接从源代码构建大规模代码智能体强化学习环境/ Tue, 22 Sep 2026 00:00:00 GMT 当前，利用强化学习 (Reinforcement Learning, RL) 训练具备复杂软件工程能力的代码智能体 (Coding Agent)，核心瓶颈在于缺乏海量且自带可靠验证器的真实任务环境。传统做法高度依赖 GitHub 的 Issue 讨论与提交历史 (Commit)，不仅数据获取难度大，且任务类型极为受限。为此，研究团队提出了创新框架 CodeMidas，首次实现仅以开源源代码本身为任务输入，端到端自动生成具备可执行验证能力的大规模强化学习环境。实验证明，在该环境下通过组相对策略优化 (Group Relative Policy Optimization, GRPO) 训练的模型，在问题修复、完整程序构建及终端操作等多个基准测试中均实现大幅跃升，并展现出更深入的代码探索与自主验证行为。该成果为低成本、大规模构建智能体强化学习数据飞轮提供了全新的范式。 --- ## 内容概要 > ## Summary 要通过强化学习 (RL) 培养出能力稳健的代码智能体，通常需要大量兼具多样性且配备可靠验证器的真实任务。然而，传统构建方法往往严重依赖软件开发过程中留下的痕迹——例如 GitHub 上的 Issue 问题单和提交历史 (Commit)。这种模式极大地限制了能够被提取出来的任务数量与任务形态的多样性。 > Training robust coding agents via reinforcement learning (RL) typically requires diverse tasks paired with reliable verifiers. Traditionally, methods rely heavily on development artifacts like GitHub issues and commits, which restricts the overall volume and variety of extractable tasks. 为了打破这一瓶颈，**CodeMidas** 推出了一套极具创新性的智能体流水线。它无需任何辅助的开发日志，直接将已有代码库中已经实现的功能转化为可交互、可运行的强化学习环境，把源代码本身作为唯一针对特定任务的输入源。 > To overcome this bottleneck, **CodeMidas** introduces an innovative agentic pipeline that transforms the implemented functionality of existing codebases directly into executable RL environments using source code as the sole task-specific input. 该方法的核心亮点包括： > Key highlights of the approach include: * **全流程智能体算力分配 (Agentic Compute Allocation)**：让智能体深度参与环境构建的每一个阶段——由智能体自主探索代码库功能以拟定软件行为规范，基于原始代码构建具备真实执行反馈的测试用例，并通过实际运行检查和多次解法推演 (Rollout) 对候选任务进行严格过滤。 * **空前的数据集规模 (Massive Scale)**：构建出的数据集包含了从 **3,185 个开源代码库**中提炼出的 **5,545 个训练任务**，广泛覆盖 **23 种编程语言**与 **15 个技术领域**。 * **显著的性能跃升 (Performance Gains)**：在这些任务上使用组相对策略优化 (GRPO) 对 MiMo-V2.5 模型进行训练，在五项不同的软件工程基准测试中全面提升了模型表现： * **问题修复 (Issue Repair)**：DeepSWE (+11.7%) * **完整程序构建 (Whole-Program Construction)**：ProgramBench (+17%) * **终端操作工作 (Terminal Work)**：Terminal-Bench v2.1 (+8.5%) * **智能体行为的深度进化 (Behavioral Improvements)**：消融实验与轨迹分析表明，扩大高质量训练任务的规模能够促使智能体进行更深入的代码库探索，并自发展现出更加丰富多样的自我验证行为。 > * **Agentic Compute Allocation:** Agents explore codebase functionality to formulate behavioral specifications, build execution-grounded tests based on original code, and rigorously filter tasks via execution checks and repeated solution rollouts. > * **Massive Scale:** The resulting dataset encompasses **5,545 training tasks** extracted from **3,185 open-source codebases**, spanning **23 programming languages** and **15 technical domains**. > * **Performance Gains:** Training the MiMo-V2.5 model using Group Relative Policy Optimization (GRPO) on these tasks significantly boosts performance across five diverse software benchmarks: > * **Issue Repair:** DeepSWE (+11.7%) > * **Whole-Program Construction:** ProgramBench (+17%) > * **Terminal Work:** Terminal-Bench v2.1 (+8.5%) > * **Behavioral Improvements:** Ablations and trajectory analysis demonstrate that scaling high-quality training tasks encourages agents to perform deeper codebase exploration and exhibit more diverse self-verification behaviors. --- ## 论文元数据 > ## Paper Metadata * **arXiv 编号 (arXiv ID)**：[arXiv:2609.22068](https://arxiv.org/abs/2609.22068) [cs.AI] * **主要学科领域 (Primary Subject)**：计算机科学 > 人工智能 (`cs.AI`) * **提交日期 (Submission Date)**：2026 年 9 月 18 日 * **论文作者 (Authors)**：Bowen Ye, Lei Li, Shicheng Li, Zihao Yue, Linghao Zhang, Hanglong Lv, Yuanxin Liu, Wenhan Ma, Hao Tian, Rang Li, Jinhao Dong, Yikai Zhao, Xiangwei Deng, Hailin Zhang, Liang Zhao, Qi Liu, Lingpeng Kong, Tong Yang, Fuli Luo > * **arXiv ID:** [arXiv:2609.22068](https://arxiv.org/abs/2609.22068) [cs.AI] > * **Primary Subject:** Computer Science > Artificial Intelligence (`cs.AI`) > * **Submission Date:** September 18, 2026 > * **Authors:** Bowen Ye, Lei Li, Shicheng Li, Zihao Yue, Linghao Zhang, Hanglong Lv, Yuanxin Liu, Wenhan Ma, Hao Tian, Rang Li, Jinhao Dong, Yikai Zhao, Xiangwei Deng, Hailin Zhang, Liang Zhao, Qi Liu, Lingpeng Kong, Tong Yang, Fuli Luo --- ## 论文摘要 > ## Abstract 通过强化学习 (RL) 训练高水平的代码智能体，离不开配有可靠验证器的多样化任务。虽然开源代码库蕴藏着海量此类任务的宝藏，但现有方法通常依赖于代码开发过程中留下的衍生记录（如 Issue 问题单和提交历史 Commit），这极大地限制了可提取任务的范围与丰富度。为了更好地扩大强化学习环境的规模，我们提出了 **CodeMidas**——这是一个基于智能体的新型流水线，仅将源代码作为针对特定任务的唯一输入，就能将现有代码库中已实现的功能直接转化为可交互执行的强化学习环境。 > Training capable coding agents via reinforcement learning (RL) requires diverse tasks with reliable verifiers. Open-source codebases offer a rich source of such tasks, while existing methods typically rely on development artifacts such as issues and commits, limiting the range of tasks that can be extracted. To better scale RL environments, we present **CodeMidas**, an agentic pipeline that turns implemented functionality in existing codebases into executable RL environments using source code as its only task-specific input. CodeMidas 在环境构建的每个阶段都充分投入了智能体算力：由智能体深入探索已实现的代码功能以制定软件行为规范；基于对原始代码的真实运行构建验证测试；并通过代码执行检查与多次解法推演 (Rollout) 来检验并筛选候选任务。最终构建生成的数据集包含了来自 3,185 个开源代码库的 5,545 个训练任务，涵盖 23 种编程语言以及 15 个技术领域。 > CodeMidas allocates agentic compute to every stage of environment construction: agents explore implemented functionality to formulate behavioral specifications, construct tests grounded in execution of the original code, and validate and filter candidate tasks through execution checks and repeated solution rollouts. The resulting dataset has 5,545 training tasks from 3,185 open-source codebases spanning 23 programming languages and 15 technical domains. 在这些任务上使用 GRPO 算法对 MiMo-V2.5 模型进行训练，显著提升了其在五项各具特色的基准测试上的表现，涵盖问题修复 (DeepSWE 提升 11.7%)、完整程序构建 (ProgramBench 提升 17%) 以及终端操作工作 (Terminal-Bench v2.1 提升 8.5%)。消融实验表明，增加高质量训练任务的数量能够稳步推动性能提升。轨迹分析进一步显示，经强化学习训练后的智能体展现出了更优秀的自主行为，例如显著增加了对代码库的探索深度，并展现出更为丰富多样的自我验证行为。这些研究结果证明，仅凭源代码本身即可作为可扩展的基础底座，用于构建高质量强化学习环境并全面提升代码智能体解决各类复杂软件任务的能力。 > Training MiMo-V2.5 on these tasks with GRPO improves performance on all five diverse benchmarks, covering issue repair (DeepSWE + 11.7%), whole-program construction (ProgramBench +17%), and terminal work (Terminal-Bench v2.1 +8.5%). Ablations show that increasing the number of high-quality training tasks improves performance. Trajectory analysis shows the RL-trained agent demonstrates better behaviors like increasing codebase exploration and more diverse self-verification. These results establish source code as a scalable foundation for constructing RL environments that improve coding agents across diverse software tasks. --- ## 资源获取与论文链接 > ## Access & Resources * **论文全文链接 (Full-Text Links)**：[阅读 PDF](https://arxiv.org/pdf/2609.22068) | [网页版 HTML (实验版)](https://arxiv.org/html/2609.22068v1) | [TeX 源码](https://arxiv.org/src/2609.22068) * **数字对象标识符 (DOI)**：[10.48550/arXiv.2609.22068](https://doi.org/10.48550/arXiv.2609.22068) > * **Full-Text Links:** [View PDF](https://arxiv.org/pdf/2609.22068) | [HTML (Experimental)](https://arxiv.org/html/2609.22068v1) | [TeX Source](https://arxiv.org/src/2609.22068) > * **Digital Object Identifier (DOI):** [10.48550/arXiv.2609.22068](https://doi.org/10.48550/arXiv.2609.22068) https://insight.aitobox.com/blog/posts/2026-09-21/如何精准捕获代码缺陷-基于-Rust-正则引擎的高效模糊测试实战/ https://insight.aitobox.com/blog/posts/2026-09-21/如何精准捕获代码缺陷-基于-Rust-正则引擎的高效模糊测试实战/ Mon, 21 Sep 2026 00:00:00 GMT 在复杂系统与底层基础库的开发中，传统的单元测试往往只能覆盖开发者预想到的常规路径，极易漏掉隐蔽的边界状态与特征交互缺陷。本文源自知名技术讨论社区 Lobste.rs 上关于“测试有效性”的探讨，作者以 Rust 生态核心的 `regex` 正则引擎历史真实 Bug 为例，展示了如何脱离臃肿庞杂的外部测试框架，仅凭一个轻量级的伪随机数生成器 (PRNG) 与群集测试 (Swarm Testing) 策略，高效生成极短却极具杀伤力的边缘用例。通过将待测引擎与标准实现 (Oracle) 交叉比对，这种轻巧敏捷的方法不仅执行速度极快，还能在极短时间内精准揪出传统测试难以触及的深层逻辑漏洞。这种实战经验表明，高质效的测试并不依赖海量算力与庞大测试数据，而是取决于精巧的测试架构设计与特征组合策略。 --- * **发布日期：** 2026 年 9 月 19 日 * **原文出处：** [Lobste.rs 讨论区](https://lobste.rs/s/mkv2pl/unit_tests_mark_territory_more_than#c_5nimhl) > * **Published on:** Sep 19, 2026 > * **Source:** [Lobste.rs Discussion](https://lobste.rs/s/mkv2pl/unit_tests_mark_territory_more_than#c_5nimhl) --- ## 📌 核心概述 > ## 📌 Summary 本文通过为 Rust 语言的核心正则库 `regex` 量身打造一个轻量级模糊测试器 (Fuzzer) ，深入探讨了生成式随机测试 (Generative Testing) 相比于传统单元测试的强大优势。作者并未依赖笨重庞大的测试框架，而是展示了如何仅凭一个极简的伪随机数生成器 (PRNG) ，搭配权威比对基准 (Oracle) ——直接将 `regex` 与轻量版 `regex_lite` 的运行结果进行交叉比对——就能精准揪出那些隐藏极深的代码缺陷。 > This article explores the effectiveness of generative (randomized) testing compared to traditional unit tests by building a custom fuzzer for the Rust `regex` crate. Rather than relying on complex frameworks, the author demonstrates how a lightweight, pseudo-random number generator (PRNG) coupled with an oracle (comparing `regex` against `regex_lite`) can successfully uncover hidden bugs. 文中所阐述的核心原则包括：采用 **群集测试 (Swarm Testing)** 策略 (随机分配特性分布与字符集) 、专注于挖掘 **短小而刁钻的边界极端用例** 而非盲目堆砌海量数据负载，以及将漏掉的 Bug 视作测试框架本身的设计不足并不断反哺完善。 > Key principles covered include **swarm testing** (randomizing feature distributions and alphabets), targeting **small and tricky edge cases** rather than massive payloads, and treating missed bugs as failures in the testing harness itself. --- ## 缺陷溯源：一个真实的正则表达式 Bug > ## The Bug 对于正则表达式 `".abb|b"` 和输入字符串 `"zabb"`，旧版本的 `regex` crate 出现了一个离奇的错误：它返回的第一个匹配项居然是 `"b"`，而不是理应匹配的完整字符串 `"zabb"`： > For the `".abb|b"` regex and `"zabb"` input, an older version of the `regex` crate incorrectly returned `b` as the first match instead of the entire `zabb` string: ```rust use regex; fn main() { let r = regex::Regex::new(".abb|b").unwrap(); let m = r.find("zabb").unwrap(); // Fails with regex-automata=0.4.15: assert_eq!(m.as_str(), "zabb") } ``` 我们该如何主动发现这类诡异的缺陷呢？正则表达式引擎属于纯粹的算法实现，这使它们成为了应用生成式测试的绝佳温床。而在算法测试领域，最强大、最直接的手段莫过于将待测系统的输出与已知正确的权威答案 (即测试预言机 Oracle) 进行比对。虽然原始评论中提到其测试工具当时缺乏可用的 Oracle，但在构建稳健系统时，我们在架构设计之初就应当把 Oracle 纳入考量。幸运的是，在针对 `regex` 库的案例中，我们完全可以直接拉来 `regex_lite` 库作为权威参考，将两者的输出进行全自动交叉验证。 > How do we find bugs like this? Regular expression engines are pure algorithms, making them prime candidates for generative testing. The most powerful technique for testing algorithms is to compare them against a known correct answer (an oracle). While the original comment noted their fuzzer lacked an oracle, robust systems should be designed with oracles in mind. Fortunately, for our `regex` case study, we can simply cross-check outputs against the `regex_lite` crate. --- ## 生成测试字符串 > ## Generating a String 我们可以借助一个极简的伪随机数生成器 (PRNG) ，搭建一个纯粹的随机字符串生成工具： > We can build a simple random string generator using a pseudo-random number generator: ```rust use fastrand::Rng; ``` 在进行随机化测试时，人们往往有一种下意识的直觉：疯狂生成海量的数据载荷 (例如动辄 5 GiB 的超大输入) 。然而在现实中，软件缺陷通常孕育自各种特性之间微妙而精细的小规模交互，而非单纯因为数据量巨大。 > When performing randomized testing, the instinct is often to generate massive payloads (e.g., 5 GiBs of input). However, bugs usually stem from small, intricate interactions between features. 我们的生成策略遵循两步走方案： > Our strategy follows a two-step approach: 1. **确立基准字符集**：从现有的单元测试中提取并固定基础字符表； 2. **动态子集抽样**：在每一次迭代中，随机挑选该字符表的一个 **子集** (即群集测试 Swarm Testing 的精髓) ，随后仅使用这些被挑中的字符生成随机长度的测试字符串。 > 1. Fix a base alphabet derived from unit tests. > 2. For each iteration, pick a random *subset* of that alphabet (**swarm testing**), then generate a string of random length using only those characters. 为了把运行性能压榨到极致，我们在各轮迭代之间通过预分配策略循环复用内存缓冲区，彻底避免频繁的动态内存申请： > To maximize performance, we reuse memory across iterations via static allocation: ```rust use fastrand::Rng; fn main() { let mut rng = Rng::new(); // Re-use the same memory for all tests. let mut text_alphabet: Vec<u8> = vec![]; let mut text: Vec<u8> = vec![]; for _ in 0..1_000_000 { // It's unlikely that a counter example with // 7 different letters exists, while there // isn't one with just 6. alphabet_swarm(&mut rng, b"abcdef", &mut text_alphabet); let text = gen_string(&mut rng, &text_alphabet, &mut text); } } fn alphabet_swarm<'a>( rng: &mut Rng, all: &[u8], pick: &'a mut Vec<u8>, ) { pick.clear(); pick.extend(all); rng.shuffle(pick); let count = rng.usize(1..=pick.len()); pick.truncate(count); } fn gen_string<'a>( rng: &mut Rng, alphabet: &[u8], result: &'a mut Vec<u8>, ) -> &'a str { result.clear(); // Again, this is a short string. // Longer failures are not likely. let count = rng.usize(0..8); for _ in 0..count { result.push(alphabet[rng.usize(0..alphabet.len())]); } str::from_utf8(result).unwrap() } ``` --- ## 设定正则语法特性的分布权重 > ## Generating a Regex Distribution 在生成正则表达式时，我们同样贯彻这种群集测试思路： > We apply the same swarm-testing strategy to generate regular expressions: * 随机激活正则语法语义特性的某一个子集； * 随机决定生成表达式的长度规模； * 全程复用内存缓冲区。 > * Pick a subset of active regex features. > * Pick sizes at random. > * Re-use memory buffers. 首先，我们为不同的正则表达式语法特性定义离散权重，而不是非开即关的布尔开关： > First, we define weights for different regex features rather than using binary toggles: ```rust #[derive(Default, Debug)] struct ReOptions { alt: u16, // | rep: u16, // * any: u16, // . lit: u16, // 'a' sum: u16, alphabet: Vec<u8>, } ``` 接下来，我们实现群集抽样逻辑，在运行期间动态随机配置各项语法特性的权重与字符集： > Next, we implement the swarm logic to dynamically configure weights and alphabets: ```rust impl ReOptions { fn swarm(&mut self, rng: &mut Rng, alphabet_full: &[u8]) { // We _still_ want to enable a few features at a time. self.alt = if rng.bool() { 0 } else { rng.u16(0..100) }; self.rep = if rng.bool() { 0 } else { rng.u16(0..100) }; self.any = if rng.bool() { 0 } else { rng.u16(0..100) }; self.lit = rng.u16(1..100); self.sum = self.alt + self.rep + self.any + self.lit; assert!(self.sum > 0); alphabet_swarm(rng, alphabet_full, &mut self.alphabet); } } ``` --- ## 递归生成正则表达式 > ## Generating a Regex 我们采用递归方式构造复杂的正则表达式语法树，通过向下传递输出缓冲区以及一个控制表达式规模的 `size` 参数来精确约束生成深度： > We construct regular expressions recursively, passing down output buffers and a `size` parameter to control expression length: ```rust fn gen_re( rng: &mut Rng, options: &ReOptions, result: &mut Vec<u8>, ) { result.clear(); let size = rng.u8(0..8); gen_re_rec(rng, options, result, size); } fn gen_re_rec( rng: &mut Rng, options: &ReOptions, result: &mut Vec<u8>, size: u8, ) { if size == 0 { return; // Base case, empty regex. } // Pick one of the features, according to weights. let mut p = rng.u16(0..options.sum); if p < options.alt { // Alternation distributes the size // among the two children. let size_left = rng.u8(0..=size - 1); let size_right = size - size_left - 1; assert!(size == size_left + 1 + size_right); result.push(b'('); gen_re_rec(rng, options, result, size_left); result.extend(b")|("); gen_re_rec(rng, options, result, size_right); result.push(b')'); return; } p -= options.alt; if p < options.rep { result.push(b'('); gen_re_rec(rng, options, result, size - 1); result.extend(b")*"); return; } p -= options.rep; if p < options.any { gen_re_rec(rng, options, result, size - 1); result.push(b'.'); return; } p -= options.any; if p < options.lit { gen_re_rec(rng, options, result, size - 1); let index = rng.usize(0..options.alphabet.len()); let lit = options.alphabet[index]; result.push(lit); return; } unreachable!(); } ``` --- ## 缺陷搜索主循环 > ## Search Loop 考虑到编译正则表达式本身涉及状态机构建，属于计算密集型操作，我们可以将每次编译出的一对正则对象保留下来，复用于数千个随机输入字符串的匹配验证： > Because compiling regular expressions is computationally expensive, we can test multiple input strings against a single compiled regex pair: ```rust fn main() { let mut rng = Rng::new(); let mut options = ReOptions::default(); let mut text_alphabet: Vec<u8> = vec![]; let mut text: Vec<u8> = vec![]; let mut re: Vec<u8> = vec![]; let mut test_count: u32 = 0; for _ in 0..1_000_000 { options.swarm(&mut rng, b"abcdef"); alphabet_swarm(&mut rng, b"abcdefx", &mut text_alphabet); gen_re(&mut rng, &options, &mut re); let re = str::from_utf8(&re).unwrap(); let r1 = regex::Regex::new(re).unwrap(); let r2 = regex_lite::Regex::new(re).unwrap(); for _ in 0..1000 { test_count += 1; let text = gen_string(&mut rng, &text_alphabet, &mut text); let m1 = r1.find(text) .map_or("not found", |it| it.as_str()); let m2 = r2.find(text) .map_or("not found", |it| it.as_str()); if m1 != m2 { eprintln!("err re={re} text={text} m1={m1} m2={m2}"); return; } if test_count % 500_000 == 0 { eprintln!("ok re={re} text={text}"); } } } } ``` https://insight.aitobox.com/blog/posts/2026-09-20/PrismML-发布三值大模型-Ternary-Bonsai-2-27B-仅-5.9GB-保留-98.2-性能/ https://insight.aitobox.com/blog/posts/2026-09-20/PrismML-发布三值大模型-Ternary-Bonsai-2-27B-仅-5.9GB-保留-98.2-性能/ Sun, 20 Sep 2026 00:00:00 GMT 近年来，如何在保持前沿大语言模型 (Large Language Model, LLM) 强大能力的同时，将其体积极致压缩以便在消费级设备上流畅运行，始终是 AI 领域的核心技术挑战之一。PrismML 团队近期正式开源了基于 Qwen3.8 27B 深度优化的三值大模型——**Ternary Bonsai 2 27B**。该模型采用宽松友好的 Apache 2.0 协议，创新性地将权重限制为仅包含 -1、0、+1 的三值离散表示，成功把原本高达 53.80 GB 的庞大模型压缩至惊人的 5.93 GB (平均每个权重仅占 1.72 比特) ，并在 20 项涵盖推理、数学与代码的基准测试中平均保留了原模型 98.2% 的顶尖性能。凭借对文本与图像的多模态理解能力以及 262K 的超长上下文窗口支持，该模型不仅可以在普通 16 GB 笔记本或单张 24 GB 消费级显卡上高吞吐流畅运行，更标志着百亿级端侧大模型向极致能效与平民化部署迈出了里程碑式的一步。 --- ## 核心概述 > ## Summary [PrismML](https://prismml.com/) 正式推出了 [Ternary Bonsai 2 27B](https://prismml.com/news/bonsai-2-27b)，这是基于 [Qwen3.8 27B](https://huggingface.co/Qwen/Qwen3.8-27B) 打造的高能效三值权重衍生模型。该模型采用 Apache 2.0 开源协议，将原本高达 53.80 GB 的半精度 (FP16) 原始模型彻底瘦身至仅 5.93 GB，并在 20 项基准测试中令人惊叹地保留了父模型 98.2% 的平均性能。Ternary Bonsai 2 27B 不仅支持文本与图像理解，还具备 262K Token 的超长上下文窗口；借助 PrismML 提供的定制化运行时，它能够在普通的 16 GB 笔记本电脑或单张 24 GB 显存显卡上轻松完成本地部署。 > [PrismML](https://prismml.com/) has introduced [Ternary Bonsai 2 27B](https://prismml.com/news/bonsai-2-27b), a highly efficient ternary-weight derivative of [Qwen3.8 27B](https://huggingface.co/Qwen/Qwen3.8-27B). Compressing the original 53.80 GB (FP16) model down to a mere 5.93 GB, this Apache 2.0-licensed model preserves an impressive 98.2% of its parent model's average performance across 20 benchmarks. Supporting text, images, and a 262K-token context window, Ternary Bonsai 2 27B can be deployed on a standard 16 GB laptop or a single 24 GB GPU using PrismML’s custom runtimes. --- ## 什么是 Ternary Bonsai 2 27B？ > ## What is Ternary Bonsai 2 27B? 该模型完整继承了 Qwen3.8 27B 的基础架构，总参数量达 273.6 亿 (27.36B) ，具体分布如下： > The model preserves the foundational architecture of the Qwen3.8 27B with its 27.36 billion parameters, distributed as follows: * **语言主干网络 (Language Backbone) ：** 243.5 亿 (24.35B) 参数 (采用混合注意力机制：约 75% 线性注意力层和 25% 全注意力层) ； * **嵌入层与语言模型头 (Embeddings & LM Head) ：** 25.4 亿 (2.54B) 参数； * **视觉塔 (Vision Tower) ：** 4.7 亿 (0.47B) 参数 (以独立的 0.63 GB GGUF 文件分发，仅在处理图像时载入) 。 > * **Language Backbone:** 24.35B parameters (utilizing hybrid attention: ~75% linear-attention and 25% full-attention layers). > * **Embeddings & LM Head:** 2.54B parameters. > * **Vision Tower:** 0.47B parameters (shipped as a separate 0.63 GB GGUF file, loaded only when processing images). 三值权重广泛应用于嵌入层、注意力投影矩阵、MLP 投影层以及语言模型头 (LM Head) 。全模型仅有 2620 万 (26.2M) 参数 (占比仅 0.0976%) 依然保留在较高精度，这些高精度参数主要集中在循环状态路径与归一化权重上。 > Ternary weights are applied across embeddings, attention projections, MLP projections, and the LM head. Only 26.2M parameters (0.0976%)—specifically the recurrent state path and normalization weights—remain in higher precision. ## 三值量化格式是如何工作的？ > ## How Does the Ternary Format Work? 该模型的权重被严格限制在三个离散值之间：**-1、0 或 +1**。 > The model utilizes weights restricted to three values: **-1, 0, or +1**. * **比特分配 (Bit Allocation) ：** 每 128 个权重组成一组，共享一个 FP16 缩放因子 (Scale) 。单个三进制数值携带 $\log_2(3)$ (约 1.585) 比特的信息量。结合缩放因子开销与保留的高精度张量，模型的实际运行开销为**每个权重 1.72 比特 (1.72 bits per weight)**。 * **打包方案 (Packings) ：** 为了在实际算子内核上获得极致性能，[技术白皮书](https://github.com/PrismML-Eng/Bonsai-demo/blob/main/bonsai-2-27b-whitepaper.pdf) 详述了两种 GGUF 打包格式： * `PTQ1_0`：以每个权重 1.76 比特的密度紧凑打包三进制位 (Trit) (模型总大小 5.93 GB) ； * `PQ2_0`：将每个三进制位存入一个 2 比特的槽位中 (模型总大小 7.25 GB) ，能够实现更迅速的解包速度。 * **基底旋转 (Rotated Basis) ：** 遵循 [SpinQuant](https://arxiv.org/abs/2405.16406) 的技术方案，PrismML 在三值分配前引入了分块阿达马旋转 (Hadamard Rotation，分块大小为 1,024) ，并在运行时对激活值应用相应的数学变换。 > * **Bit Allocation:** Every group of 128 weights shares a single FP16 scale. A ternary value carries $\log_2(3)$ (approx. 1.585) bits. Combined with the scale overhead and high-precision tensors, the model operates at **1.72 bits per weight**. > * **Packings:** To optimize performance on real kernels, the [whitepaper](https://github.com/PrismML-Eng/Bonsai-demo/blob/main/bonsai-2-27b-whitepaper.pdf) details two GGUF packings: > * `PTQ1_0`: Packs trits densely at 1.76 bits per weight (5.93 GB total size). > * `PQ2_0`: Stores each trit in a 2-bit slot (7.25 GB total size), offering faster unpacking. > * **Rotated Basis:** Following [SpinQuant](https://arxiv.org/abs/2405.16406) methodology, PrismML applies a blockwise Hadamard rotation (block size 1,024) before ternary assignment, with corresponding transformations applied to activations during runtime. ## 与 Qwen3.8 27B 的基准测试对比如何？ > ## How Does It Score Against Qwen3.8 27B? 评估测试由 PrismML 在思考模式下开展，测试环境基于 NVIDIA H100 GPU，并结合了 EvalScope 与 vLLM 推理框架： > Evaluations were conducted by PrismML in thinking mode using EvalScope and vLLM on H100 GPUs: | 能力维度 | Qwen3.6 27B | Qwen3.8 27B | Ternary Bonsai 2 27B | 性能保持率 | | :--- | :---: | :---: | :---: | :---: | | 知识与推理能力 | 84.71 | 86.66 | 83.95 | 96.9% | | 数学计算能力 | 94.64 | 97.06 | 96.57 | 99.5% | | 代码编写能力 | 82.57 | 82.17 | 81.58 | 99.3% | | 智能体与工具调用能力 | 80.05 | 79.74 | 77.57 | 97.3% | | 指令遵循能力 | 74.53 | 81.25 | 82.66 | 101.7% | | 视觉多模态能力 | 79.82 | 81.64 | 78.59 | 96.3% | | **综合评分 (20 项基准测试) ** | **83.6** | **85.4** | **83.9** | **98.2%** | > | Capability | Qwen3.6 27B | Qwen3.8 27B | Ternary Bonsai 2 27B | Retention | > | :--- | :---: | :---: | :---: | :---: | > | Knowledge and reasoning | 84.71 | 86.66 | 83.95 | 96.9% | > | Math | 94.64 | 97.06 | 96.57 | 99.5% | > | Coding | 82.57 | 82.17 | 81.58 | 99.3% | > | Agentic and tool calling | 80.05 | 79.74 | 77.57 | 97.3% | > | Instruction following | 74.53 | 81.25 | 82.66 | 101.7% | > | Vision | 79.82 | 81.64 | 78.59 | 96.3% | > | **Overall (20 benchmarks)** | **83.6** | **85.4** | **83.9** | **98.2%** | 相较于体积为 7.3 GB 的传统量化构建版本 (例如平均得分仅 75.2 的 `IQ2_XXS`) ，Bonsai 2 展现出了飞跃式的提升——在 AIME26 数学竞赛测试中取得 95.83 分 (对比后者的 78.6 分) ，在 LiveCodeBench v6 编程基准测试中取得 90.07 分 (对比后者的 70.05 分) 。 > Compared to conventional quantization builds like `IQ2_XXS` (averaging 75.2 at 7.3 GB), Bonsai 2 demonstrates dramatic improvements—scoring 95.83 on AIME26 (vs. 78.6) and 90.07 on LiveCodeBench v6 (vs. 70.05). ## 哪些能力依然存在质量损耗？ > ## Where Does It Still Lose Quality? 尽管 98.2% 的综合平均保留率非常优异，但在特定的长流程复杂任务中，模型性能的下滑依然相对明显： > While the 98.2% overall average is robust, quality drops are more pronounced in specific long-horizon workflows: * **智能体工作负载 (Agentic Workloads) ：** 在 [Terminal-Bench 2.1](https://www.tbench.ai/news/terminal-bench-2-1) 终端基准测试中，Bonsai 2 得分为 52.8 分 (Qwen3.8 原型为 69.7 分) ；在 [SWE-bench Verified](https://openai.com/index/introducing-swe-bench-verified/) 软件工程智能体基准测试中，其得分为 60.8 分 (对比全精度的 80.6 分，性能保持率约为 75%) 。 * **推理算力投入 (Reasoning Effort) ：** 中等思考预算 (Medium-effort) 下的平均得分为 79.3 分 (对比 FP16 基准的 82.6 分) ，且当前版本暂不支持低思考预算 (Low-effort) 执行。*(注：所有评估数据均源自 PrismML 官方公布，尚待第三方独立复现与验证。)* > * **Agentic Workloads:** On [Terminal-Bench 2.1](https://www.tbench.ai/news/terminal-bench-2-1), Bonsai 2 scores 52.8 compared to Qwen3.8's 69.7. On [SWE-bench Verified](https://openai.com/index/introducing-swe-bench-verified/), it scores 60.8 against 80.6 (~75% retention). > * **Reasoning Effort:** Medium-effort execution averages 79.3 (compared to 82.6 for the FP16 baseline), and low-effort execution is unsupported. *(Note: All evaluation data stems from PrismML and awaits independent verification.)* ## 在真实硬件上的运行速度如何？ > ## How Fast Is It on Real Hardware? 基于 PrismML 自研算子内核在批大小为 1 (Batch Size 1) 解码时的实测性能 (2026 年 9 月测试数据) ： > Benchmarked at batch size 1 decode on custom PrismML kernels (September 2026): * **硬件实测速度：** * **RTX 5090：** 142.5 Token/秒 (每生成一个 Token 仅耗电 0.582 毫瓦时/mWh) ； * **RTX 4090：** 96.7 Token/秒 (采用 `PTQ1_0` 格式) ； * **NVIDIA L4 (72W) ：** 32.1 Token/秒； * **Apple M5 Max：** 46.8 Token/秒； * **Apple M5 Pro：** 27.7 Token/秒。 * **打包格式效率对比：** `PTQ1_0` 在 Ada 架构显卡与 L4 上表现最佳；而在 Blackwell、Hopper、Ampere 以及 Apple 芯片架构上，`PQ2_0` 的运行速度更快。PrismML 同时表示，相较于全精度 8B 模型，该模型的整体能效比提升了 40%。 > * **Hardware Speeds:** > * **RTX 5090:** 142.5 tokens/sec (0.582 mWh per token). > * **RTX 4090:** 96.7 tokens/sec (using `PTQ1_0`). > * **NVIDIA L4 (72W):** 32.1 tokens/sec. > * **Apple M5 Max:** 46.8 tokens/sec. > * **Apple M5 Pro:** 27.7 tokens/sec. > * **Packing Efficiency:** `PTQ1_0` excels on Ada-generation cards and the L4, whereas `PQ2_0` is faster on Blackwell, Hopper, Ampere, and Apple silicon architectures. PrismML also claims a 40% improvement in energy efficiency over full-precision 8B models. ## 如何在本地运行？ > ## How Do You Run It? * **GGUF 格式运行：** 需要使用 PrismML 专门优化的 [llama.cpp 分支仓库](https://github.com/PrismML-Eng/llama.cpp) (原生 llama.cpp 目前无法识别 `PTQ1_0` 与 `PQ2_0` 格式) 。参照 [Bonsai-demo 代码仓库](https://github.com/PrismML-Eng/Bonsai-demo/) 的使用说明，运行 `./setup.sh` 脚本，随后执行 `./scripts/start_llama_server.sh`，即可在本地 `localhost:8080` 启动推理服务。 * **Apple Silicon 平台：** 可直接选用针对 Mac 优化的 [MLX 整合包](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-mlx-2bit) 及其附带的模型加载器。 * **浏览器端体验：** 可以访问 [WebGPU 在线演示](https://huggingface.co/spaces/webml-community/ternary-bonsai-2-webgpu-kernels) ，在现代网络浏览器中直接体验端侧免安装运行。 > * **GGUF Format:** Requires PrismML’s [llama.cpp fork](https://github.com/PrismML-Eng/llama.cpp) (stock llama.cpp rejects `PTQ1_0` and `PQ2_0` formats). Follow instructions in the [Bonsai-demo repo](https://github.com/PrismML-Eng/Bonsai-demo/) by running `./setup.sh` followed by `./scripts/start_llama_server.sh` to launch a local server at `localhost:8080`. > * **Apple Silicon:** Use the [MLX pack](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-mlx-2bit) with its bundled loader. > * **Browser:** Explore the [WebGPU demo](https://huggingface.co/spaces/webml-community/ternary-bonsai-2-webgpu-kernels) to run the model directly inside a web browser. ## 核心要点速览 > ## Key Takeaways * **极致压缩：** 模型最终体积仅为 5.93 GB，比 53.80 GB 的 FP16 全精度基准缩小了约 9.1 倍； * **极高保留率：** 在 20 项基准测试中平均得分达到 83.9 分，保留了原版 Qwen3.8 27B 模型 98.2% 的顶尖性能； * **极速推理：** 在 RTX 5090 显卡上能够达到每秒 142.5 Token，在 Apple M5 Max 芯片上可达到每秒 46.8 Token； * **长流程智能体局限：** 在 SWE-bench 等长跨度复杂任务中，性能保持率下降至全精度的约 75%； * **需专属运行时：** 当前运行必须依赖 PrismML 定制适配的 llama.cpp 分支或 MLX 运行时。 > * **Massive Reduction:** 5.93 GB model size, operating ~9.1x smaller than the 53.80 GB FP16 baseline. > * **High Retention:** Achieves an 83.9 average across 20 benchmarks, retaining 98.2% of the parent Qwen3.8 27B model's performance. > * **Blazing Fast:** Delivers 142.5 tokens per second on an RTX 5090 and 46.8 tokens per second on an M5 Max. > * **Agentic Limitations:** Long-horizon tasks like SWE-bench drop to roughly 75% retention compared to full precision. > * **Custom Runtimes Required:** Requires PrismML's specialized llama.cpp fork or MLX runtime to execute. --- https://insight.aitobox.com/blog/posts/2026-09-20/Linkup-发布-SPARSEUP-仅-149M-参数的开源最强稀疏嵌入模型/ https://insight.aitobox.com/blog/posts/2026-09-20/Linkup-发布-SPARSEUP-仅-149M-参数的开源最强稀疏嵌入模型/ Sun, 20 Sep 2026 00:00:00 GMT 在信息检索与检索增强生成 (RAG) 领域，稠密向量模型虽然应用广泛，但在生僻专有名词匹配和可解释性上存在天然局限，而传统的稀疏模型又往往面临词表膨胀和高计算开销的挑战。为此，Linkup Research 正式发布了开源稀疏嵌入模型 (Sparse Embedding Model) —— **SPARSEUP**。该模型基于 149M 参数的 ModernBERT 架构打造，采用 Apache 2.0 许可证完全开源，在 BEIR-13 基准测试中斩获 56.4 的平均 nDCG@10 得分，成为目前 150M 参数以下公开可用的最强词表稀疏编码器。通过 Logit 平移、单位置 Top-k 截断和大小写折叠等三项精巧架构优化，SPARSEUP 巧妙破解了传统 SPLADE 模型停用词泛滥的难题，并结合 Seismic 倒排索引实现了约 380 微秒的单线程极速检索。这一成果不仅补齐了现代信息检索中轻量级稀疏范式的技术拼图，更为开发者构建高效率、强可解释性的混合检索系统提供了全新利器。 --- ## 核心概述 > ## Executive Summary Linkup Research 正式开源了基于机器学习的稀疏嵌入模型 (Sparse Embedding Model) —— **SPARSEUP**。该模型基于 149M 参数的 ModernBERT 骨干网络构建，并以宽松的 Apache 2.0 许可证公开发布。SPARSEUP 在知名的 BEIR-13 基准测试 (Benchmark) 中取得了 56.4 的平均 nDCG@10 得分，被 Linkup 定位为当前 150M 参数量级以下公开可用的最强基于词表的稀疏编码器。开发者可以通过 Hugging Face，借助 Transformers 或 Sentence Transformers 库轻松部署 SPARSEUP。该模型恰好补齐了此前 LightOn 模型套件的技术拼图，在保持极高检索效率与人类直观可读权重的特性的同时，为稠密检索 (Dense Retrieval) 和晚期交互检索 (Late-Interaction Retrieval) 提供了极具竞争力的稀疏检索替代方案。 > Linkup Research has released **SPARSEUP**, an open-source learned sparse embedding model built on a 149M-parameter ModernBERT backbone and distributed under the Apache 2.0 license. Achieving a 56.4 average nDCG@10 on BEIR-13, Linkup positions SPARSEUP as the strongest public vocabulary-based sparse encoder under 150M parameters. Deployable via Hugging Face using Transformers or Sentence Transformers, SPARSEUP bridges the gap in LightOn's recent model suite, offering a viable sparse alternative to dense and late-interaction retrieval styles while maintaining high efficiency and human-readable weights. --- ## 为什么需要稀疏模型？为什么是现在？ > ## Why a Sparse Model, and Why Now? 目前绝大多数开源检索模型都依赖于稠密表征 (Dense Representations)，即为每段文本生成一个固定维度的稠密向量。与此不同的是，稀疏模型直接在预定义的整个词表上输出权重分布，其中的每一个维度都精确对应一个真实的 Token。这种设计的精妙之处在于，生成的稀疏向量能够无缝对接到传统的倒排索引 (Inverted Indexes) 搜索引擎中；同时，由于每个激活维度都对应具体词汇，权重具备天然的人类可读性，并且在命中冷门、罕见的生僻词与专有名词时表现尤为强悍。 > Most open retrieval models rely on dense representations (one vector per text). In contrast, sparse models output weights over a vocabulary, where each dimension maps to a real token. This design allows vectors to integrate smoothly into inverted indexes while remaining human-readable and highly effective at matching rare words. SPARSEUP 的研发直接受到了 LightOn 近期发布 [DenseOn and LateOn](https://huggingface.co/papers/2607.27178) 的启发。LightOn 当时开源了全套训练数据、训练秘方 (Recipe)、一个稠密模型以及一个晚期交互模型。而 SPARSEUP 采用相同的骨干网络家族与微调 (Fine-Tuning) 数据，恰好补全了缺失的“稀疏模型”版块，从而让开发者能够站在完全平等的基准线上，横向对比这三种主流的信息检索范式。 > The development of SPARSEUP was motivated by LightOn’s release of [DenseOn and LateOn](https://huggingface.co/papers/2607.27178), which provided open data, a training recipe, a dense model, and a late-interaction model. SPARSEUP fills the missing sparse slot by utilizing the same backbone family and fine-tuning data, allowing developers to compare all three retrieval paradigms side by side. --- ## SPARSEUP 是如何构建的 > ## How SPARSEUP is Built SPARSEUP 的训练起点是 [LateOn-unsupervised](https://huggingface.co/lightonai/LateOn-unsupervised) 检查点，但该检查点并未包含掩码语言模型 (Masked Language Modeling, MLM) 头。Linkup 团队将 ModernBERT 原生的 MLM 预测头重新“嫁接”回模型中，并纯粹采用对比学习 (Contrastive Learning) 方法，基于 [LightOn 的微调混合数据集 (LightOn’s fine-tuning mixture)](https://huggingface.co/datasets/lightonai/embeddings-fine-tuning) 展开微调。 > Training begins from the [LateOn-unsupervised](https://huggingface.co/lightonai/LateOn-unsupervised) checkpoint, which lacked an MLM head. The Linkup team grafted ModernBERT’s original MLM head back onto the model and performed fine-tuning using [LightOn’s fine-tuning mixture](https://huggingface.co/datasets/lightonai/embeddings-fine-tuning) with contrastive learning exclusively. * **训练策略：** 每个查询 (Query) 都从 50 个候选池中采样配对 7 个难负例 (Hard Negatives)，同时结合批次内负例 (In-batch Negatives) 共同训练。整个训练流程摒弃了复杂的交叉编码器蒸馏 (Cross-Encoder Distillation)，单张 H100 GPU 即可高效完成训练。 * **突破原生 SPLADE 的局限：** 若在该骨干网络上直接应用标准的 SPLADE 方案，会导致输出表征过于稠密，充斥着大量无意义的停用词 (Stopwords)。Linkup 通过三项核心架构调整完美化解了这一难题： 1. **Logit 平移 (Logit shifting)：** 编码器通过计算 `log(1 + ReLU(x - 15))`，巧妙抵消了 ModernBERT MLM 原始 Logit 在初始化阶段过大从而导致对数函数饱和的问题。 2. **逐位置 Top-k 截断 (Per-position top-k)：** 每个输入的 Token 在进入最大池化 (Max Pooling) 之前，仅保留权重最高的前 12 个词表维度。这种设计限制了每个 Token 的词汇扩展幅度，而非死板地限制最终向量的总长度。 3. **大小写折叠 (Case folding)：** 将字节级 BPE (Byte-level BPE) 中形如 `heat`、`Heat`、`Ġheat` 和 `ĠHeat` 等同义变体统一折叠归并到单一 ID 下，并保留其中的最大权重，从而将词表输出维度从约 50k 显著压缩至约 34k。 > * **Training Strategy:** Each query is paired with 7 hard negatives sampled from a pool of 50, alongside in-batch negatives. The process excludes cross-encoder distillation and fits efficiently on a single H100 GPU. > * **Overcoming Vanilla SPLADE Limitations:** A standard SPLADE implementation on this backbone resulted in overly dense token bags saturated with stopwords. Linkup resolved this with three core architectural adjustments: > 1. **Logit shifting:** The encoder computes `log(1 + ReLU(x - 15))` to counteract ModernBERT MLM logits that otherwise saturate the log function at initialization. > 2. **Per-position top-k:** Each input token retains only its 12 strongest vocabulary dimensions prior to max pooling, capping expansion per token rather than total vector size. > 3. **Case folding:** Byte-level BPE variants like `heat`, `Heat`, `Ġheat`, and `ĠHeat` are folded onto a single ID while preserving the largest weight, reducing output dimensions from ~50k to ~34k. 在输入格式上，查询和文档分别添加 `[Q]` 与 `[D]` 前缀作为标识，检索得分通过点积 (Dot Product) 计算。在评估基准中，查询的最大长度设为 128 个 Token，文档的最大长度设为 512 个 Token。 > Queries and documents use `[Q]` and `[D]` prefixes respectively, scored via dot product. Evaluation max lengths are set to 128 tokens for queries and 512 for documents. --- ## 基准测试结果 > ## Benchmark Results 根据官方 [模型卡片 (Model Card)](https://huggingface.co/Linkup-Platform/linkup-sparseup-embed-v1) 披露的数据，在 BEIR-13 基准测试 (采用 nDCG@10 指标，且不包含 MS MARCO) 中，SPARSEUP 与其他主流稀疏编码器的对比表现如下： > Evaluated against other sparse encoders on BEIR-13 (nDCG@10, excluding MS MARCO) per the [model card](https://huggingface.co/Linkup-Platform/linkup-sparseup-embed-v1): | 模型 | BEIR-13 平均得分 | | :--- | :--- | | **SPARSEUP** | **56.4** | | [opensearch-neural-sparse-encoding-doc-v3-gte](https://huggingface.co/opensearch-project/opensearch-neural-sparse-encoding-doc-v3-gte) | 54.6 | | opensearch-neural-sparse-encoding-v1 | 52.44 | | ModernBERT-VT | 52.4 | | splade-v3 | 51.7 | | [granite-embedding-30m-sparse](https://huggingface.co/ibm-granite/granite-embedding-30m-sparse) | 50.6 | | LACONIC-1B *(10 亿参数，属于不同规模量级)* | 58.7 | > | Model | BEIR-13 avg | > | :--- | :--- | > | **SPARSEUP** | **56.4** | > | [opensearch-neural-sparse-encoding-doc-v3-gte](https://huggingface.co/opensearch-project/opensearch-neural-sparse-encoding-doc-v3-gte) | 54.6 | > | opensearch-neural-sparse-encoding-v1 | 52.44 | > | ModernBERT-VT | 52.4 | > | splade-v3 | 51.7 | > | [granite-embedding-30m-sparse](https://huggingface.co/ibm-granite/granite-embedding-30m-sparse) | 50.6 | > | LACONIC-1B *(1B parameters, different size class)* | 58.7 | https://insight.aitobox.com/blog/posts/2026-09-19/我是如何借助-AI-与-Lean-证明康威50年猜想的-Dan-Abramov-的一月通关实录/ https://insight.aitobox.com/blog/posts/2026-09-19/我是如何借助-AI-与-Lean-证明康威50年猜想的-Dan-Abramov-的一月通关实录/ Sat, 19 Sep 2026 00:00:00 GMT 本文记录了著名技术开发者 Dan Abramov 作为一名自称的“数学小白”，如何仅凭前沿大语言模型 (Claude 与 ChatGPT) 以及交互式定理证明工具 Lean 4，历时一个月攻克传奇数学家 John Conway 悬置整整 50 年的“全数整数细分猜想” (Omnific Integer Refinement Conjecture) 的奇幻实录。在经历初期 AI 严重胡言乱语、“黑话”满天飞与伪证明破产的泥潭后，作者开创性地搭建了包含项目经理、红队攻防、文献逆向与形式化编译的多智能体协同实验室架构。通过严格的代码隔离、定理反演与编译器内核硬校验，最终成功编译出了一份经 Lean 4 内核无额外公理检验通过的完整证明证书。本文兼具幽默坦诚的个人叙事与前沿形式化验证的深度思考，展现了生成式 AI 赋能前沿跨界探索的巨大潜能。 --- ## 核心概要 > # Summary 一名自称“数学小白”的业余爱好者，耗时整整一个月，完全凭借前沿大语言模型 (Large Language Model, LLM) —— Claude 与 ChatGPT，以及形式化定理证明工具 Lean 4，向数学界的公开未解难题发起了冲击：试图证明 John Conway 在 50 年前提出、关于超现实数 (Surreal Numbers) 的**全数整数细分猜想 (Omnific Integer Refinement Conjecture)**。尽管起初遭遇了严重的 AI 幻觉（充斥着大量虚构行话的“AI 垃圾内容”），作者随后精心打造了一套多智能体协同实验室工作流，结合严格的 Lean 4 形式化验证、独立代码文件沙箱隔离与层层递进的红队审计机制。最终，整套系统成功编译出了一份经 Lean 4 编译器内核完备校验的康威猜想证明证书，并向全球数学界同行公开邀请形式化证伪。 > An amateur mathematician ("math noob") spends a month using AI frontier models (Claude and ChatGPT) and the Lean theorem prover to attempt an open mathematical problem: proving John Conway’s 50-year-old **omnific integer refinement conjecture** regarding surreal numbers. Despite initial struggles with AI hallucinations ("slop" and made-up terminology), the author builds a multi-agent laboratory workflow combined with rigorous Lean formalization, independent file isolation, and step-by-step audits. Ultimately, the system successfully compiles a kernel-checked proof certificate of Conway's conjecture in Lean, inviting formal peer refutation. --- ## 我是如何“凭直觉与感觉”搞定康威猜想证明的 > ## How I Vibed a Proof of Conway’s Conjecture **日期：** 2026年9月18日 **作者：** Dan Abramov ([Ko-fi](https://ko-fi.com/gaearon) | [GitHub 源码仓库](https://github.com/gaearon/conway-refinement)) > **Date:** September 18, 2026 > **Author:** Dan Abramov ([Ko-fi](https://ko-fi.com/gaearon) | [GitHub Source](https://github.com/gaearon/conway-refinement)) 几个月前，AI 在数学领域取得突破的新闻开始频频登上各大媒体头条。[“搞个大突破” (Do a breakthrough)](https://www.theargumentmag.com/p/computer-do-a-breakthrough-no-mistakes) 甚至成了 Twitter 上的热门技术热梗。自然而然地，我也按捺不住好奇心：像我这样一个连高等微积分题解都要到处找答案的[数学小白](https://github.com/gaearon/analysis-solutions)，是不是也能随便找一个悬而未决的数学开放难题，然后让前沿模型帮我把它解出来？ > A few months ago, AI math results started making headlines. [“Do a breakthrough”](https://www.theargumentmag.com/p/computer-do-a-breakthrough-no-mistakes) became a Twitter meme. Naturally, I became curious whether I, too, a [math noob](https://github.com/gaearon/analysis-solutions), could find some open mathematical problem and then have a frontier model solve it. 这花光了我整整一个月的全部业余时间，并烧掉了[天文数字般的 Token](#how-many-tokens)；但我想，我已经成功拿到了由 Lean 4 严格检验通过的[数学证明](https://github.com/gaearon/conway-refinement)，攻克了 John Conway 在 50 年前留下的这一著名猜想： > It took me an entire month of my free time and a [boatload](#how-many-tokens) of tokens, but I believe I’ve obtained a Lean [proof](https://github.com/gaearon/conway-refinement) of this conjecture posed by John Conway 50 years ago:  Conway 的细分猜想断言：全数整数具备优良的因数细分性质——如果四个全数整数满足 $ab = cd$，那么必然存在另外四个整数 $e, f, g, h$，使得 $a = ef$、$b = gh$、$c = eg$ 且 $d = fh$。 > Conway’s refinement conjecture claims that omnific integers have a refinement property: if *ab = cd*, there are integers *e*, *f*, *g*, *h* with *a = ef*, *b = gh*, *c = eg*, *d = fh*. 需要坦诚说明的是，我的证明目前**尚未**经过人类专业数学家的独立同行评审。不过，我掌握着[相当扎实的依据](https://github.com/gaearon/conway-refinement#why-i-think-its-correct)相信这一证明是坚实成立的，并且我非常真诚地欢迎任何人前来挑错和证伪。 > My proof has *not* been independently verified by mathematicians. However, I have [decent reasons](https://github.com/gaearon/conway-refinement#why-i-think-its-correct) to believe the proof is correct, and I genuinely invite a refutation. 目前，该证明已经完全通过了[来自 Palomar 注册表](https://palomar-registry.org/entry?id=PALOMAR-2026-09-03-000002&version=1)的严格机械化检查，几位精通 Lean 4 和该领域的专家也确认[形式化命题本身的陈述](https://github.com/gaearon/conway-refinement/blob/264445c93b78554c408e99e4e7f663693b4e91ab/ConwayRefinement/Standalone/Mathlib/InlineConwayRefinement.lean#L246-L257)完全准确。因此，只要我的证明代码没有意外触发 Lean 4 底层内核的致命 Bug，那么它大概率也是货真价实成立的。 > The proof has passed the mechanical checks [from the Palomar registry](https://palomar-registry.org/entry?id=PALOMAR-2026-09-03-000002&version=1), and a few people familiar with both Lean and the field said that [the statement](https://github.com/gaearon/conway-refinement/blob/264445c93b78554c408e99e4e7f663693b4e91ab/ConwayRefinement/Standalone/Mathlib/InlineConwayRefinement.lean#L246-L257) seems correct. So, assuming my proof doesn’t rely on a Lean kernel bug, it’s likely to be legit too. 在这篇长文中，我将详细记录我的探索路径，以及一路上所踩过的坑和收获的认知。 > In this post, I’ll describe my approach, and some things I learned along the way. --- ## 第一天 > ## First Day 我一直觉得，“在完全不懂数学本质的情况下把它解出来”这个念头既荒谬又离奇，但恰恰是这种荒诞感，让这件事显得格外诱人。 > I thought the idea of “solving” a math problem without understanding its substance is rather absurd, which of course made it all the more appealing. 不过，我可不想随随便便找个平庸的题做；我想要的是那种能够真正牵引我、让我心潮澎湃的问题。 > However, I didn’t just want *any* result; I wanted something that pulls me. https://insight.aitobox.com/blog/posts/2026-09-19/当智能体执行提交-认知可串行化保障复杂系统一致性/ https://insight.aitobox.com/blog/posts/2026-09-19/当智能体执行提交-认知可串行化保障复杂系统一致性/ Sat, 19 Sep 2026 00:00:00 GMT 随着自主 AI 智能体 (AI Agent) 逐步接管复杂的生产与业务流程，它们不再仅仅是回答问题的交互助手，而是开始主动对系统状态执行写操作并调用各类外部服务。然而，智能体在漫长推理过程中所依赖的前提依据——包括数据库状态、检索知识、系统策略和委派权限——往往具有随机性且可能在后台被并发修改，使得传统数据库的事务隔离机制难以保证决策与提交动作之间的一致性。为了解决这一制约智能体安全落地的关键瓶颈，本文提出了“认知可串行化 (Cognitive Serializability) ”理论与 TCT 框架。该方案在数学上形式化了认知推导的一致性边界，并在工程实测中以仅 3.22 毫秒的极低额外开销拦截了全部并发异常，为构建高可靠、强一致的智能体自治系统奠定了坚实的理论与工程基石。 --- # 当智能体执行提交：认知可串行化保障复杂系统一致性 > # When AI Agents Commit: Cognitive Serializability Across Data, Evidence, Policy, and Authority **作者：** Jun He, Deying Yu **提交时间：** 2026年7月28日 **主要领域：** 人工智能 (`cs.AI`) **次要领域：** 分布式、并行与集群计算 (`cs.DC`) **arXiv 标识符：** [arXiv:2609.20261 [cs.AI]](https://arxiv.org/abs/2609.20261) **DOI：** [10.48550/arXiv.2609.20261](https://doi.org/10.48550/arXiv.2609.20261) > **Authors:** Jun He, Deying Yu > **Submitted:** 28 July 2026 > **Primary Subject:** Artificial Intelligence (`cs.AI`) > **Secondary Subjects:** Distributed, Parallel, and Cluster Computing (`cs.DC`) > **arXiv Identifier:** [arXiv:2609.20261 [cs.AI]](https://arxiv.org/abs/2609.20261) > **DOI:** [10.48550/arXiv.2609.20261](https://doi.org/10.48550/arXiv.2609.20261) --- ## 核心概述 在现代分布式系统中，自主 AI 智能体往往需要结合各类输入来动态推导出对系统状态的修改动作——这些输入涵盖数据库读取、检索召回的背景证据、业务治理策略、模型自身的置信度以及外部委派的执行权限。然而，由于这些输入具有概率随机性，并且极易在智能体长时推理的“思考”期间在后台发生变动，传统的数据库隔离级别 (仅仅对最终提交的事务进行物理排序) 以及常规的智能体事务处理机制往往难以应对。除非系统契约能巨细靡遗地把所有推导前提都显式转化为谓词条件，否则它们根本无法保证智能体最终提交的修改动作与其推导依据在逻辑上达成统一且有效的严格一致。 为了彻底解决这一痛点，本文正式提出了**认知可串行化 (Cognitive Serializability) **理论与一整套系统框架 (TCT) 。该框架在丝毫不牺牲系统吞吐与响应性能的前提下，能够为复杂智能体系统提供零误差的正确性保障以及确定的向前推进能力。 > ## Summary > > Autonomous AI agents dynamically derive state mutations from a wide range of inputs—including database reads, retrieved evidence, governance policies, internal beliefs, and delegated authority. Because these inputs are stochastic and often change while reasoning is actively in progress, traditional database isolation (which orders submitted transactions) and standard agentic transaction processing fall short. They fail to establish a unified valid point of agreement between a mutation and its underlying derivation inputs unless the contract explicitly represents all relevant predicates. > > This paper introduces **Cognitive Serializability** and a comprehensive framework (**TCT**) to guarantee zero-error soundness and positive progress in agentic systems without sacrificing performance. --- ## 核心概念与运行机制 * **类型化依赖 Token (Typed Dependency Tokens) ：** 清晰区分数据的“内容完整性”与规则的“通用适用性”。 * **可信中介机制 (Trusted Mediation) ：** 精准捕获并记录在智能体推理过程中所使用的全部确切数值。 * **严格认知可串行化 (Strict Cognitive Serializability) ：** 确保所有已提交的执行效果都遵循严格的串行顺序，并在逻辑上等价于所有暴露给推导过程的前提数值全程未发生任何改变。系统防护栅栏 (Fences) 将持续生效，直至运行时事件最终完成封装持久化域的封口确认。 * **效果兼容型认知准入 (Effect-Compatible Cognitive Admission，弱化版) ：** 依据并发持有的最新依赖向量与当前最新策略，对修改效果重新进行合规校验，从而巧妙避开了对原始随机推导过程进行强行串行化的巨大开销。 * **TCT 框架 (The TCT Framework) ：** 深度融合以下八大核心组件以强制执行系统一致性： 1. 不可变的版本化可执行定义 2. 注册表派生的授权计划 3. 密封信封机制 4. 守卫先行提交事务 5. 密封后与信封及见证绑定的授权授予 6. 协同提交凭据回执 7. 幂等的授权终结确认 8. 回执驱动的认知状态对齐 > ## Key Concepts & Mechanisms > > * **Typed Dependency Tokens:** Distinguish between content integrity and general applicability. > * **Trusted Mediation:** Captures the exact values exposed to an agent's reasoning process. > * **Cognitive Serializability (Strict):** Ensures that committed effects follow a strict serial order alongside a logical event where every value exposed to derivation remains entirely unchanged. Fences persist until the runtime event finalizes the sealed durability domain. > * **Effect-Compatible Cognitive Admission (Weaker):** Recertifies an effect against a concurrently held current dependency vector and current policy, bypassing the need to serialize the original stochastic derivation. > * **The TCT Framework:** Combines several core components to enforce consistency: > 1. Immutable versioned executable definitions > 2. Registry-derived authority plans > 3. Sealed envelopes > 4. Guard-first commit transactions > 5. Post-seal envelope- and witness-bound grants > 6. Co-committed receipts > 7. Idempotent grant finalization > 8. Receipt-driven epistemic reconciliation --- ## 评测与实验结果 * **理论完备性保障 (Theoretical Guarantees) ：** 通过构建完整的注册轨迹足迹与单一的增长阶段，该架构成功在本地守卫与不兼容的外部资源预留之间建立了无环锁点顺序，严格满足了精确的可串行化条件与观测等价性边界。 * **实证性能表现 (Empirical Performance) ：** 团队构建了专门的证伪测试套件对系统实现规范进行了高强度检验。实验原型成功拦截了**全部注入的异常与并发冲突**，同时带来的平均提交额外开销仅有 **3.22 毫秒**，几乎可以忽略不计。 > ## Evaluation & Results > > * **Theoretical Guarantees:** Complete registered footprints and a single growing phase successfully induce an acyclic lock-point order over local guards and incompatible external reservations, satisfying precise serializability conditions and observational-equivalence boundaries. > * **Empirical Performance:** A dedicated falsification suite tested implementation obligations. The prototype successfully prevented **all injected anomalies** while incurring a negligible mean commit overhead of only **3.22 ms**. --- ## 资源与论文获取 * **全文链接 (Full-Text Links) ：** [查看 PDF](https://arxiv.org/pdf/2609.20261) | [实验性 HTML 网页](https://arxiv.org/html/2609.20261v1) | [TeX 源码](https://arxiv.org/src/2609.20261) * **授权协议 (License) ：** [知识共享署名 4.0 国际许可协议 (Creative Commons Attribution 4.0) ](http://creativecommons.org/licenses/by/4.0/) > ## Access & Resources > > * **Full-Text Links:** [View PDF](https://arxiv.org/pdf/2609.20261) | [Experimental HTML](https://arxiv.org/html/2609.20261v1) | [TeX Source](https://arxiv.org/src/2609.20261) > * **License:** [Creative Commons Attribution 4.0](http://creativecommons.org/licenses/by/4.0/) <img alt="license icon" role="presentation" src="./images/345c7ad61f1b.png"/> https://insight.aitobox.com/blog/posts/2026-09-19/FormalFlow-面向-MIP-RE-核心定理的长程自动化形式化证明/ https://insight.aitobox.com/blog/posts/2026-09-19/FormalFlow-面向-MIP-RE-核心定理的长程自动化形式化证明/ Sat, 19 Sep 2026 00:00:00 GMT 在理论计算机科学与量子物理的交叉领域，著名的 $\text{MIP}^* = \text{RE}$ 定理证明了多证明者量子纠缠交互式证明系统等价于所有递归可枚举语言，从根本上颠覆了人们对可计算性与量子纠缠的认知，其证明过程极其宏大繁琐，传统纯人工的形式化检验几乎需要顶尖专家团队耗费数年心血。为了攻克长程数学证明自动化中的“命题漂移”与“证明拼接”等关键瓶颈，本文研究团队创新性地提出了 **FormalFlow** 系统，巧妙引入现代软件工程开发思想，依托全局共享蓝图引导多个 AI 证明智能体 (AI proving agents) 在人类监督下开展嵌套规划、机器证明与严谨代码审查。借助该系统，团队历时 63 天成功在 Lean 4 中完成了 $\text{MIP}^* = \text{RE}$ 底层核心定理——“经典低单变量度数测试的量子可靠性”的完全形式化，产出超过 12.6 万行完全由 AI 智能体生成的代码，并在验证中精准排查与修复了原论文手写证明中存在的隐蔽缺陷与边界条件。这项突破不仅为深奥的量子复杂性理论构筑了经计算机验证的坚实基石，更生动展示出小规模科研团队借助自主 AI 智能体低成本、高质量验证重大科学猜想与顶尖数学证明的广阔前景。 --- # 面向 MIP* = RE 核心定理的长程自动化形式化证明 > # Long-Horizon Autoformalization of a Core Theorem Underlying MIP* = RE ## 核心概要 > ## Summary 里程碑式的数学形式化验证往往需要顶尖专家团队投入数年心血才能完成。本文提出了 **FormalFlow** 系统，在人类专家的统筹监督下协同多个 AI 证明智能体 (AI proving agents) ，共同攻克长程形式化验证 (long-horizon formalization) 中尤为棘手的命题漂移 (statement drift) 与证明拼接 (proof composition) 等核心挑战。FormalFlow 汲取了现代软件工程的研发思想，依托一份全局共享的蓝图 (shared blueprint) 来引导“规划—证明—评审”的嵌套闭环，并由智能体在全流程中持续强化严格的形式化验证。 > Landmark mathematical formalizations typically require specialist teams years of dedicated work to complete. This paper introduces **FormalFlow**, a system that coordinates AI proving agents under human supervision to overcome key challenges in long-horizon formalization, such as statement drift and proof composition. Inspired by software engineering principles, FormalFlow utilizes a shared blueprint to steer nested planning, proving, and review loops, with agents continuously reinforcing verification. 借助该系统，作者团队成功完成了经典低单变量度数测试 (classical low individual-degree test) 量子可靠性 (quantum soundness) 在 Lean 4 中的机器检验形式化证明——该定理正是著名的量子计算里程碑成果 $\text{MIP}^* = \text{RE}$ 的底层基石。整个形式化开发历时 63 天完成 (若引入更大规模的并行机制，这一周期还可进一步大幅压缩) ，最终产出的代码库包含 126,367 行完全由智能体自主生成的 Lean 代码。在严苛的机器检验过程中，系统成功识别并修正了原手写论文中存在的附加约束条件漏洞与中间推演谬误，同时在修正后的假设前提下依然完好保持了已发表论文中的最终误差界。这项突破不仅为量子复杂性理论 (quantum complexity theory) 奠定了经计算机完全验证的坚实基石，更展示出了一条极具扩展潜力的全新路径，使小规模科研团队也有能力以极高的性价比对重大前沿研究证明开展严格的形式化验证。 > Using this system, the authors completed a machine-checked Lean 4 proof for the quantum soundness of the classical low individual-degree test—a foundational theorem underlying the celebrated $\text{MIP}^* = \text{RE}$ result. Developed over 63 days (a timeline that could be further compressed through parallelism), the resulting library contains 126,367 lines of entirely agent-generated Lean code. The formalization process successfully identified and corrected side conditions and intermediate errors while maintaining the published final error bound. This achievement establishes a verified foundation for quantum complexity theory and demonstrates a scalable path toward the affordable verification of major research proofs by small teams. --- ## 文档元数据 > ## Document Metadata * **arXiv 编号：** [arXiv:2609.19814](https://arxiv.org/abs/2609.19814) * **主学科分类：** 量子物理 (Quantum Physics, `quant-ph`) * **次学科分类：** 人工智能 (Artificial Intelligence, `cs.AI`) 、计算机科学中的逻辑 (Logic in Computer Science, `cs.LO`) * **MSC 数学分类：** 68V20, 68V15, 81P68, 68Q15 * **提交日期：** 2026 年 9 月 17 日 * **论文作者：** Sirui Lu, Ruixuan Deng, Yanqiao Zhu, Zhengfeng Ji > * **arXiv ID:** [arXiv:2609.19814](https://arxiv.org/abs/2609.19814) > * **Primary Subject:** Quantum Physics (`quant-ph`) > * **Secondary Subjects:** Artificial Intelligence (`cs.AI`), Logic in Computer Science (`cs.LO`) > * **MSC Classes:** 68V20, 68V15, 81P68, 68Q15 > * **Submission Date:** September 17, 2026 > * **Authors:** Sirui Lu, Ruixuan Deng, Yanqiao Zhu, Zhengfeng Ji --- ## 论文摘要 > ## Abstract 以往，完成里程碑式的数学形式化验证往往需要顶尖专家团队耗费数年时间。我们提出了 FormalFlow 系统，在人类专家的监督下协同多个 AI 证明智能体 (AI proving agents) ，有效解决长程形式化过程中面临的命题漂移与证明拼接两大核心难题。该系统借鉴现代软件工程的原理与实践，利用一份全局共享的蓝图指导规划、证明与评审的嵌套闭环；智能体在整个形式化生命周期中持续强化形式验证与严格审查。基于该系统，我们成功完成了经典低单变量度数测试量子可靠性的机器检验 Lean 4 形式化证明，该定理正是支撑著名成果 $\text{MIP}^* = \text{RE}$ 的核心基石之一。整个证明开发历时 63 天；若借助更高并发度的并行机制，研发周期有望进一步缩短。最终生成的证明代码库包含 126,367 行 Lean 代码，且全部由智能体自主生成。在严格的形式化过程中，系统成功修正了原论文中的附加约束条件与中间推演谬误，同时在修正后的假设下依然完好保持了此前已发表的最终误差界。本项工作不仅为量子复杂性理论建立了经机器全面验证的严谨基石，更展示出了一条切实可行的技术路径，使小规模研究团队也能以低成本实现重大前沿科研证明的形式化验证。 > Landmark mathematical formalizations have taken specialist teams years to complete. We present FormalFlow, a system that coordinates AI proving agents under human supervision to address statement drift and proof composition in long-horizon formalization. Drawing on software engineering principles and practices, it uses a shared blueprint to guide nested planning, proving and review loops. Agents strengthen verification and review throughout formalization. We completed a machine-checked Lean 4 proof of the quantum soundness of the classical low individual-degree test, a core theorem underlying $\text{MIP}^* = \text{RE}$. Developing the proof took 63 days; greater parallelism could further reduce this time. The final library contains 126,367 lines of Lean code, all generated by agents. The formalization corrects side conditions and intermediate errors while preserving the published final error bound under corrected assumptions. This work provides a verified foundation for quantum complexity and demonstrates a route to affordable verification of major research proofs by small teams. --- ## 补充信息与资源 > ## Supplementary Information & Resources * **论文篇幅：** 全文共 72 页 (正文共 13 页，包含 4 幅图和 1 张表；补充附录共 57 页，包含 9 幅图和 17 张表；以及参考文献) 。 * **Lean 4 代码仓库：** [GitHub - LionSR/MIPStarRE](https://github.com/LionSR/MIPStarRE) * **获取全文：** * [查看 PDF](https://arxiv.org/pdf/2609.19814) * [HTML 在线版本 (实验性) ](https://arxiv.org/html/2609.19814v1) * [TeX 源码](https://arxiv.org/src/2609.19814) > * **Paper Length:** 72 pages total (13-page main text featuring 4 figures and 1 table; 57-page supplementary appendices featuring 9 figures and 17 tables; references). > * **Lean 4 Code Repository:** [GitHub - LionSR/MIPStarRE](https://github.com/LionSR/MIPStarRE) > * **Access Full-Text:** > * [View PDF](https://arxiv.org/pdf/2609.19814) > * [HTML Version (Experimental)](https://arxiv.org/html/2609.19814v1) > * [TeX Source](https://arxiv.org/src/2609.19814) https://insight.aitobox.com/blog/posts/2026-09-19/AI-智能体真的懂计算机体系结构吗-AutoTuring-基准测试揭示底层思考本质/ https://insight.aitobox.com/blog/posts/2026-09-19/AI-智能体真的懂计算机体系结构吗-AutoTuring-基准测试揭示底层思考本质/ Sat, 19 Sep 2026 00:00:00 GMT 随着人工智能技术深入底层硬件领域，AI 智能体 (AI Agent) 越来越多地被委以设计和优化硬件加速器的重任，并展现出令人瞩目的性能提升，但学界一直存在一个根本性疑问：AI 究竟是在真正理解并推理计算机体系结构，还是仅仅在多维参数空间中进行高效的盲目黑盒搜索？为了拨开这一迷雾，研究团队推出了全新的 AutoTuring 基准测试 (Benchmark) 框架，让同一个 AI 智能体在面对完全相同的 15 维硬件设计空间时，分别在赋予明确物理语义的“知情架构师”和抹去所有物理含义的“盲盒搜索者”两种不同设定下执行优化任务。评测结果揭示，掌握体系结构物理知识能让知情智能体在 FP16 GEMM 算子优化中相比盲盒智能体平均取得 12.3% 的性能提升，并大幅节省 70.1% 的底层硬件仿真器调用开销；然而，引入结构化的审校反思回路 (Critic Loop) 却能帮助盲盒智能体抹平绝大部分差距，表明体系结构先验认知与结构化反思机制在优化效能上表现出奇妙的相互替代特性。这项工作首次在保持问题本质与解空间完全一致的前提下，量化了“理解物理意义”对 AI 硬件设计的真实价值，为未来芯片自动化设计以及专用大语言模型 (Large Language Model, LLM) 智能体的协同演化提供了兼具洞察力与方法论意义的重要参考。 --- ## 内容概要 > ## Summary 随着 AI 智能体越来越多地参与硬件加速器的设计与优化，并在实践中屡获佳绩，一个根本性的疑问始终悬而未决：**AI 智能体究竟是在真正理解并推理计算机体系结构，还是仅仅在庞大的参数空间中进行盲目的黑盒搜索？** > As AI agents are increasingly used—and praised—for designing and optimizing hardware accelerators, a fundamental question remains: **Are they truly reasoning about computer architecture, or are they just blindly searching over parameter spaces?** 为了探寻真相，研究人员打造了名为 **AutoTuring** 的评估框架，让同一个 AI 智能体在两种截然不同的表述设定下，探索完全相同的 15 维硬件加速器设计空间： > To find out, researchers introduced **AutoTuring**, an evaluation framework that tests AI agents using the exact same 15-dimensional accelerator space under two different framings: 1. **知情设定 (Informed Framing)**：向智能体清晰展示具有明确物理含义的具名硬件架构旋钮，并配备详尽的硬件仿真器性能计数器反馈。 2. **盲盒设定 (Blind Framing)**：抹去所有物理背景与硬件语义，仅将参数作为被约束在 $[0,1]$ 区间内的匿名抽象变量呈现给智能体。 > 1. **Informed Framing:** Presented as named architectural knobs equipped with simulator counters. > 2. **Blind Framing:** Presented as anonymous variables constrained to $[0,1]$. 通过将评估器、合法参数空间以及理论可达的最优解完全锁定为同一标准，实验中唯一的变量就是这些配置参数对智能体而言是否具备实际的“物理与业务含义”。研究结果揭示，计算机体系结构的专业知识确实能带来显著回报——知情智能体的优化结果平均超越了模拟的 H200 基线 5.4%，相比盲盒状态下的智能体更是大幅领先 12.3%，同时所需的底层仿真器调用次数锐减了 70.1%。然而，这种优势并非不可替代：为盲盒智能体引入一套结构化的审校反思回路，能够帮助其追平绝大部分的性能差距；但同样的审校回路却几乎无法为知情智能体带来额外提升。这一现象生动地表明，体系结构领域知识与结构化反思机制在硬件优化中更多地表现为相互替代的关系，而非相互叠加的互补关系。 > By keeping the evaluator, legal space, and reachable optima identical, the only variable is whether the problem "means" anything to the agent. The findings reveal that architectural knowledge pays off—the informed agent beats a modeled H200 baseline by 5.4% and the blind agent by 12.3% on average, while requiring 70.1% fewer simulator calls. However, this advantage isn't exclusive: a structured critic loop helps the blind agent recover most of that performance gap while offering little benefit to the informed agent, suggesting that architectural knowledge and structured critique function as substitutes rather than complements. --- ## 论文元数据 > ## Paper Metadata * **arXiv 标识符：** [arXiv:2609.19387](https://arxiv.org/abs/2609.19387) [cs.AI] * **学科领域：** 人工智能 (`cs.AI`) ；硬件架构 (`cs.AR`) * **ACM 分类：** C.1.3；I.2.8；B.8.2 * **提交时间：** 2026 年 9 月 16 日 * **论文作者：** Ambika Sharan, Grigory Chirkov, Soheil Abbasloo > * **arXiv Identifier:** [arXiv:2609.19387](https://arxiv.org/abs/2609.19387) [cs.AI] > * **Subjects:** Artificial Intelligence (`cs.AI`); Hardware Architecture (`cs.AR`) > * **ACM Classes:** C.1.3; I.2.8; B.8.2 > * **Submitted on:** September 16, 2026 > * **Authors:** Ambika Sharan, Grigory Chirkov, Soheil Abbasloo --- ## 论文摘要 > ## Abstract 如今，AI 智能体越来越多地被委以芯片硬件设计的重任，且不断有报告指出它们取得了优异的优化成果。然而，这些报告固然证明了硬件设计得到了实质性改进，却始终无法解释改进背后的深层原因。一个成功优化了加速器的智能体，究竟是在深入理解并推理底层硬件机器的运行机理，还是仅仅在完全不理解参数实际物理意义的前提下展现了出色的数值搜索能力？唯有前者所代表的深度认知能力，才能够真正迁移到未来的下一代全新硬件架构中。现有的基准测试往往只更换不同的智能体，却保持问题的呈现形式固定不变，因此根本无法将这两种能力区分开来。本研究采取了截然相反的评估范式：AutoTuring 将同一个 15 维加速器设计空间分两次交给同一个智能体——一次呈现为附带仿真器计数器的具名硬件架构旋钮，另一次则呈现为限制在 $[0,1]$ 区间内的纯匿名数学变量；在此过程中，性能评估器、合法搜索空间以及可达的最优点完全保持一致，唯一的差异就是这道设计难题对智能体而言是否具备真实的“物理含义”。两种设定下的性能差距，正是衡量其体系结构理解力的标尺。在包含 9 个核心算子的半精度通用矩阵乘法 (FP16 GEMM) 任务组评测中，理解物理意义带来了实打实的回报：知情架构师智能体相比模拟的 H200 基准提升了 5.4%，相比盲盒状态下的自己平均领先 12.3%，同时仿真器调用次数大幅减少了 70.1%。然而，这种回报并非不可替代：引入结构化审校回路能够帮助盲盒智能体追平绝大部分性能差距，却无法给知情架构师带来任何额外收益；这表明硬件体系结构知识与结构化反思回路在优化中更多充当了相互替代的角色，而非锦上添花的互补关系。我们在此汇报这些在单款模拟加速器上经每种条件 5 至 6 次运行所得出的初步发现，并强调本研究的核心贡献在于这一创新性的对比评估方法本身，而非具体的某款加速器硬件设计。 > Agents are increasingly asked to design hardware, and increasingly reported to succeed. Such reports establish that a design improved; they cannot establish why. An agent that improves an accelerator may be reasoning about the machine, or may be searching competently over knobs whose meaning it never recovers -- and only the first transfers to the next architecture. Existing evaluations cannot tell the two apart, because they vary the agent while holding the framing of the problem fixed. We do the opposite. AutoTuring hands the same agent the same 15-dimensional accelerator space twice: once as named architectural knobs with simulator counters, once as anonymous variables on [0,1], with the evaluator, the legal space and the reachable optima held identical, so that the only thing that varies is whether the problem means anything. The gap between the two is the measurement. On a nine-kernel FP16 GEMM basket, meaning pays: the architect beats a modeled H200 by 5.4% and its blind counterpart by 12.3% on average, with 70.1% fewer simulator calls. It does not pay uniquely: a critic loop recovers most of that gap for the blind agent and buys the architect nothing, so architectural knowledge and structured critique behave as substitutes rather than as complements. We report these as preliminary findings -- five to six runs per condition on a single modeled accelerator -- and take the comparison itself, not the accelerator, to be the contribution. --- ## 相关链接与资源 > ## Links & Resources * **全文访问：** [查看 PDF](https://arxiv.org/pdf/2609.19387) | [HTML 版本](https://arxiv.org/html/2609.19387v1) | [TeX 源码](https://arxiv.org/src/2609.19387) * **引用与指标：** [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.19387) | [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.19387) | [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.19387) > * **Full-Text Access:** [View PDF](https://arxiv.org/pdf/2609.19387) | [HTML Version](https://arxiv.org/html/2609.19387v1) | [TeX Source](https://arxiv.org/src/2609.19387) > * **Citations & Metrics:** [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.19387) | [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.19387) | [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.19387) https://insight.aitobox.com/blog/posts/2026-09-18/重构-Packfile-在对象存储上直接运行-Git-的架构实践/ https://insight.aitobox.com/blog/posts/2026-09-18/重构-Packfile-在对象存储上直接运行-Git-的架构实践/ Fri, 18 Sep 2026 00:00:00 GMT 将 Git 存储服务直接构建在低成本、高弹性的云端对象存储 (例如 AWS S3 或 Tigris) 之上，是许多团队探索云原生存储架构与降低运维开销的理想方案。然而，标准 Git 的 Packfile 打包机制是为本地磁盘设计的，严重依赖微秒级的内核内存映射 `mmap` 与高并发随机寻道；一旦搬到存在数十毫秒网络往返延迟的对象存储上，性能便会迅速恶化至不可用的状态。为了彻底攻克这一难题，开发者借鉴了经典 CD-ROM 光盘中 `.bin`/`.cue` 音轨索引的分离设计，提出了一种专为对象存储定制的列式 Packfile 格式，并巧妙利用精准的 HTTP Range 范围请求与异步预取机制。这一架构突破将 S3 API 请求量削减了数个数量级，在生产级基准测试中使代码推送与克隆提速高达 14.6 倍，为现代分布式版本控制与云端底层系统架构开辟了全新路径。 --- ## 核心概要 > ## Summary 构建一个以对象存储 (Object Storage) (例如 Tigris) 为底座的 Git 服务器，听起来似乎很直观：只需搭建一层文件系统转译层，让 Git 能够操作对象存储即可。然而，标准的 Git Packfile 是为本地磁盘量身设计的，高度依赖操作系统内核的 `mmap` 内存映射以及低延迟的本地文件系统极速读取。一旦将这些读取操作映射为跨越网络的对象存储往返，面对生产环境级别的大型仓库时，这种方案的扩展性便会彻底崩溃。 > Building a Git server backed by object storage (like Tigris) sounds straightforward: use a filesystem translation layer so Git can speak object storage. However, standard Git packfiles are designed for local disks and rely heavily on the kernel's `mmap` and fast, low-latency filesystem reads. When translated over network round-trips to object storage, this approach fails to scale for production-sized repositories. 为了解决这个问题，作者从老式 CD-ROM 光盘的 `.bin`/`.cue` 索引单中汲取灵感，发明了一种专为对象存储原生打造的全新 Packfile 格式。通过构建一个带有二进制索引文件 (`objects.cue`) 的列式存储结构，索引中同时记录了精确的文件偏移量以及压缩与解压后的双重尺寸，客户端得以发起高效的 HTTP Range 范围请求。这一专属设计不仅大幅削减了访问 S3/对象存储的请求次数，更将代码推送 (push) 和克隆 (clone) 的性能最高提升了整整 14.6 倍。 > To solve this, the author invented an object-storage-native packfile format inspired by CD-ROM `.bin`/`.cue` sheets. By creating a columnar store with a binary index (`objects.cue`) containing both compressed and uncompressed sizes alongside precise file offsets, clients can execute efficient HTTP Range requests. This custom approach drastically reduced S3/object storage requests and accelerated push/clone performance by up to 14.6x. --- ## Git 到底是什么？不过是一堆可怜的零散对象！ > ## What is Git? A Miserable Little Pile of Objects! 当你提交一次代码时，Git 会把你的改动作为经过压缩的、基于内容寻址的对象 (Content-Addressed Objects) 保存在 `.git` 目录中 (本文称之为“dotgit”)。 > When you make a commit, Git stores your changes as compressed, content-addressed objects inside the `.git` directory (referred to here as "dotgit"). ```bash $ mkdir ~/tmp/gitexample $ git init && git branch -m main $ echo "Hello, blog!" >> hello.txt $ git add . $ git commit -sm "chore: initial commit" ``` 这一操作会在底层生成一片由裸对象和具名引用构成的“对象之海”： > This produces a sea of bare objects and named references: <figure><figcaption>图 01: 对象的海洋与指向它们的引用命名</figcaption><pre> .git/objects/ refs/heads/main │ ├── 1c/7a26a901..ec7966 ─────▶ commit 1c7a26a │ │ ├── 8e/67afbb2e..857bd3 ─────▶ tree 8e67afb │ │ hello.txt └── 9c/c9867337..09fe26 ─────▶ blob 9cc9867 "Hello, blog!" the filename is the sha1 of the bytes in the file, so the same content is always, everywhere, the very same object</pre></figure> 读取其中的某个对象需要对其进行解压： > Reading an object requires decompressing it: ```bash $ file .git/objects/**/* | grep -v directory .git/objects/1c/7a26a901724b4ce766655ac387413fb9ec7966: zlib compressed data .git/objects/8e/67afbb2ee6bdcbb79061dfdfb93febce857bd3: zlib compressed data .git/objects/9c/c9867337c2ebae85ba2350f901e0bcc209fe26: zlib compressed data $ python3 -c "import sys, zlib; sys.stdout.buffer.write(zlib.decompress(sys.stdin.buffer.read()))" < .git/objects/9c/c9867337c2ebae85ba2350f901e0bcc209fe26 blob 13Hello, blog! ```