跳转至

Z.ai 详解基于 10 万张国产芯片的 GLM-5.3-Flash 推理系统架构构建

Z.ai Details GLM-5.3-Flash Inference Build on 100,000 Chinese Chips

文章背景与核心概要

2026 年 9 月 17 日，人工智能企业 Z.ai 正式发布技术报告，详细披露了其如何在由超过 10 万张国产 AI 加速芯片组成的超大规模集群上，从零搭建起面向核心大模型 GLM-5.3-Flash 的生产级推理系统架构。这是国产算力芯片首次在十万卡级别生产环境中承受住超大规模真实业务流量的严苛检验。

尤为引人瞩目的是，该系统的底层算子与架构性能调优并非单纯依赖人类工程师手动完成，而是由以 GLM-5.3 为核心驱动的“基础架构智能体” (Infra Agent) 自主探索与优化完成的。该系统成功应对了稀疏与线性注意力混合架构、高达 100 万 Token 的超长上下文窗口以及复杂多模态请求，最终使集群端到端吞吐量提升了整整三倍，硬件利用效率与单 Token 推理成本比肩主流 NVIDIA 硬件，生动展现了大模型递归自我进化的巨大工程威力。

总结

Summary

2026 年 9 月 17 日，Z.ai 发布了一份详尽的技术报告，全面展示了其如何从零构建起支撑 GLM-5.3-Flash 模型的生产级推理基础设施。该集群由超过 10 万张国产 AI 加速芯片组成，这一里程碑代表了国产算力芯片在生产环境中前所未有的部署规模。尤为突出的是，整个基础设施优化的绝大部分工作并不是单纯依赖人类工程师手动完成，而是由基于 GLM-5.3 驱动的基础架构智能体 (Infra Agent) 协同完成的。最终构建的推理系统能够游刃有余地支撑模型的复杂混合架构、高达 100 万 Token 的长上下文窗口以及海量多模态请求，其效率指标已可比肩业界主流的 NVIDIA 硬件。

On September 17, 2026, Z.ai released a comprehensive technical report detailing how it successfully built a production-grade inference infrastructure for its GLM-5.3-Flash model from scratch. Operating across a cluster of over 100,000 Chinese-made AI accelerators, this milestone represents an unprecedented scale for domestic silicon in production. Notably, a significant portion of the infrastructure optimization was driven by an Infra Agent powered by GLM-5.3 rather than engineers alone. The resulting system successfully handles the model's complex hybrid architecture, 1-million-token context window, and multimodal requests, achieving efficiency levels comparable to mainstream NVIDIA hardware.

国产芯片上的超大规模推理挑战

Scaling Inference on Domestic Silicon

在这种前所未有的超大体量下驾驭国产芯片集群，团队面临着极为特殊的工程挑战。Z.ai 不仅要应对单芯片片上显存容量与通信带宽的物理制约，还要直面国产算力生态尚不成熟的现实困境——算子底层支持不全、权威的标准开发文档时常缺位等问题屡见不鲜。

Operating a massive cluster of Chinese-made accelerators at this scale presented unique hurdles. Z.ai faced limitations regarding on-chip memory capacity and bandwidth, coupled with an immature ecosystem where kernel support was incomplete and standard documentation was frequently missing.

GLM-5.3-Flash 模型本身于 2026 年 8 月 26 日正式发布，拥有 3,200 亿总参数量和 180 亿激活参数，采用了融合稀疏注意力与线性注意力的前沿混合架构。在此前以 ox-alpha 的化名在 OpenCode 和 OpenRouter 平台上进行匿名盲测期间，该模型迅速成为两家平台上调用量最高的模型，在上线最初的六天内就处理了超过 62 万亿 Token 的庞大流量。

The GLM-5.3-Flash model itself—launched on August 26, 2026—features 320 billion total parameters and 18 billion active parameters utilizing a hybrid architecture of sparse and linear attention. Following its anonymous testing phase on OpenCode and OpenRouter under the alias ox-alpha, it rapidly became the most-used model on those platforms, processing over 62 trillion tokens within its first six days.

密集反馈方法：为智能体打造高效闭环

The Dense Feedback Method

在对复杂的推理基础设施进行深度调优时，传统的端到端性能指标所能提供的帮助十分有限；它们虽然能报警提示性能“发生了”下滑，却无法准确定位问题出在“何处”以及“为什么”。为了打破这种黑盒困境，Z.ai 独创性地引入了密集反馈方法 (Dense Feedback Method)。

Traditional end-to-end metrics provide limited utility when optimizing complex infrastructures; they signal that performance dropped, but fail to pinpoint why. To bridge this gap, Z.ai implemented a dense feedback method.

该方法将以下多维度的诊断手段有机交织在一起：

This approach weaves together:

  • 正确性测试
  • 运行时日志与关键事件
  • 执行追踪轨迹 (Execution Traces)
  • 微基准测试 (Microbenchmarks) 与端到端宏观指标
  • Correctness tests
  • Runtime logs and events
  • Execution traces
  • Microbenchmarks and end-to-end metrics

通过将这些元素整合为可快速重复执行的自动化工作流，基础架构智能体 (Infra Agent) 每次做出微调后，都可以在局部环境下迅速验证推想，而不必经历漫长的全量集群重新部署与全网压力测试等待。

By unifying these elements into repeatable workflows, the Infra Agent could locally validate hypotheses without waiting for full deployment and load testing after every tweak.

Z.ai 为这一反馈闭环确立了三大核心构建准则：

Z.ai established three core criteria for this feedback loop:

  1. 局部精准性 (Locality)：必须直接绑定到具体的启动参数、代码修改、计算算子、输入边界条件、执行线程或调用路径。
  2. 极高时效性 (Efficiency)：获取成本极其低廉，且反馈结果必须足够快速及时。
  3. 客观可验证性 (Objective Verification)：由标准参考实现和对照实验提供强力背书，以实锤确定系统性能因果机制，而非停留在表象的相关性推测上。
  1. Locality: Tied directly to launch parameters, code changes, kernels, input conditions, threads, or execution paths.
  2. Efficiency: Inexpensive and timely to acquire.
  3. Objective Verification: Supported by reference implementations and controlled experiments to confirm root causes rather than mere correlations.

在双方协同工作时，人类工程师主要负责界定顶层系统边界并把控生产核心风险（如数值计算精度规范与高并发行为），而智能体则承担了海量细节分析、假设推演与底层代码修改。最终打造出的整套极致优化技术栈，集成了节点内张量并行 (Tensor Parallelism)、ReplaySSM 机制、W8A8 权重激活量化、INT8/FP8/BF16 混合精度缓存量化、Layer Split 层分割技术，以及编码-预填充-解码解耦 (Encode-Prefill-Decode Disaggregated) 架构。

Working in tandem, engineers defined high-level system boundaries and assessed critical production risks (such as numerical semantics and concurrency behavior), while the agent handled analysis, hypotheses, and code modifications. The optimized stack combined intra-node tensor parallelism, ReplaySSM, W8A8 quantization, mixed-precision INT8/FP8/BF16 cache quantization, Layer Split, and an Encode-Prefill-Decode disaggregated architecture.

三大关键工程突破

Three Key Engineering Triumphs

1. 确保数值计算的绝对正确

1. Ensuring Numerical Correctness

在对切分与非切分算子执行路径进行严格交叉验证时，系统捕获到了 KDA 算子在上下文并行 (Context Parallelism) 路径下的一个累积误差隐患。具体而言，底层的 tl.dot 运算在接收到 FP32 精度输入时，默认回退到了 TF32 模式进行计算，导致数值误差在超长上下文处理过程中被持续放大。

Validating partitioned versus unpartitioned kernel paths uncovered an accumulation error within the Context Parallelism path of the KDA kernel. Specifically, the tl.dot operation defaulted to TF32 computation despite receiving FP32 inputs, causing errors to compound over long contexts.

  • 解决方案：将输入计算精度显式指定为 tf32x3（通过调用三次 TF32 Tensor Core 运算来模拟更高精度），成功彻底恢复了高精度的计算输出。
  • 上游生态回馈：该补丁已被正式合并至开源的 Flash Linear Attention 核心库中，作为可选的高精度执行路径，并针对缺乏原生 TF32 硬件支持的平台提供了回退保护方案。
  • The Fix: Explicitly setting input precision to tf32x3 (utilizing three TF32 Tensor Core operations) restored high-precision results.
  • Upstream Integration: The patch was merged into Flash Linear Attention as an opt-in accuracy path, with fallback protocols for platforms lacking native TF32 support.

2. 突破 KV 传输中的并发锁瓶颈

2. Resolving KV Transfer Concurrency Bottlenecks

工程师们曾设立过一个极为严苛的性能目标：预填充 (Prefill) 阶段叠加键值传输 (KV Transfer) 的总耗时，相比于单纯进行 Prefill 的基准耗时，性能劣势必须控制在 5% 以内。然而初期基准测试显示性能开销差距竟超过了 20%。通过深入链路追踪发现，问题根源在于 DeepEP v1.2.1 中，节点内的调度与合并 (dispatch and combine) 底层调用未能在执行期间释放 Python 全局解释器锁 (Global Interpreter Lock, GIL)，导致处理通信的 Mooncake Transfer Python 线程长期无法及时争抢到锁。

Engineers set a performance target: Prefill plus KV Transfer must run within 5% of the Prefill-only baseline. Initial benchmarks exposed gaps exceeding 20%, traced back to DeepEP v1.2.1 where intranode dispatch and combine calls failed to release the Python GIL (Global Interpreter Lock). This blocked the Mooncake Transfer Python thread from acquiring the lock in time.

  • 解决方案：在关键的 C++ 底层计算密集执行区间内，主动释放 Python 的 GIL 锁。
  • 最终成效：在完全相同的严苛测试条件下，两者的性能差距骤降到了 1% 以下，几乎消除了传输瓶颈。
  • The Fix: Releasing the GIL during critical C++ execution intervals.
  • The Result: The performance gap dropped below 1% under identical test conditions.

3. 底层算子性能极致调优

3. Kernel Performance Optimization

Z.ai 让智能体主动提炼 SGLang、Flash Linear Attention 和 DeepGEMM 等开源业界顶尖项目中的手写算子优化技巧，并将其转化为可复用的调优骨架。

Z.ai tasked the agent with distilling handwritten kernel techniques from projects like SGLang, Flash Linear Attention, and DeepGEMM into reusable optimization skeletons.

  • KDA 解码算子优化：通过对除法运算实施底层指令级优化，使解码算子的单次执行时间缩短了 9.6%。
  • V 维度分块融合 (Tile Merging)：在精准识别出计算是核心瓶颈后，智能体将重复的 FP32 归一化和门控计算全部合并到了同一个线程块 (Thread Block) 内完成，并将中间结果直接保留在片上寄存器中，相比旧版算子实现了高达 1.71 倍的加速比。
  • KDA Decode Optimization: Division optimizations cut execution time by 9.6%.
  • V-Dimension Tile Merging: After identifying compute as the primary bottleneck, the agent merged repeated FP32 normalization and gating computations into a single thread block with register-resident intermediate results, achieving a 1.71× speedup over the previous version.

成果验证与递归自我进化的工程范式

Results and Recursive Self-Improvement

从模型最初的适配移植到达到全面的生产就绪状态，整个项目耗时不到两周。最终，集群的端到端吞吐量相比初始基准翻了整整三倍，硬件利用率与单 Token 推理成本全面追平了主流 NVIDIA GPU 的水平。

Moving from initial model adaptation to full production readiness took less than two weeks. Ultimately, end-to-end throughput tripled relative to the initial baseline, with hardware utilization efficiency and per-token costs reaching parity with mainstream NVIDIA GPUs.

Z.ai 将这一具有里程碑意义的工程实践，定位于一次极具说服力的早期递归自我进化 (Recursive Self-Improvement) 尝试——模型本身正在积极主动地优化其赖以运行的推理系统。与此同时，公司也特别强调：明确工程目标、划定架构边界以及对生产风险进行终审把控，依然是不可或缺的人类核心职责。

Z.ai frames this milestone as a compelling early iteration of recursive self-improvement, with the model actively optimizing the very inference system it runs on. However, the company emphasizes that defining objectives, setting architectural boundaries, and assessing risk remain vital human responsibilities.