跳转至

HEAT：通过近似与权重协同适配实现更快速的全同态推理

文章背景与核心概要

全同态加密（FHE）允许服务器直接在加密的用户提示词上运行大语言模型，而无需对其进行解密，从而在云计算环境中提供了极高的隐私保护。然而，现有的FHE推理方法由于计算开销巨大而进展缓慢。由于密文原生仅支持加法、乘法和旋转，且在需要昂贵的“自举”（bootstrapping）操作之前仅允许有限的乘法深度，因此所有模型的非线性部分都必须使用基于迭代乘法的方法进行近似。

传统方法在整个模型中采用统一的迭代次数，而不是适应局部误差容限，这导致更高的迭代次数虽然能带来更好的精度，但会更快耗尽可用的计算深度。为了克服这一痛点，本文作者引入了同态加密感知训练（HEAT）微调框架，将每个非线性层的迭代次数视为可学习参数，从而实现迭代次数与模型权重在训练过程中的协同适配。

通过直接针对任务目标优化这些迭代次数，HEAT使模型能够在推理过程中原生吸收近似误差，而无需进行结构更改或从头重新训练。在加密的GPT-2解码任务上进行测试时，HEAT实现了显著的性能提升：迭代次数减少了3.1倍，自举次数减少了1.6倍，端到端延迟降低了1.4倍，并且与校准基线相比提高了解码一致性。

arXiv ID: 2609.01730 [cs.CR]
Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
Authors: Alessandro Zirilli, Davide Marincione, Evgenios M. Kornaropoulos, Giuseppe Ateniese, Emanuele Rodolà
Submitted: 1 September 2026
Links: View PDF | HTML Version | TeX Source

Abstract Summary

arXiv ID: 2609.01730 [cs.CR]
Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
Authors: Alessandro Zirilli, Davide Marincione, Evgenios M. Kornaropoulos, Giuseppe Ateniese, Emanuele Rodolà
Submitted: 1 September 2026
Links: View PDF | HTML Version | TeX Source

全同态加密（FHE）使服务器能够在加密的用户提示词上直接执行大语言模型，而无需对其进行解密。然而，当前FHE推理方法的速度仍然慢得令人望而却步。由于密文原生仅支持加法、乘法和旋转，且在需要进行高开销的“自举”（bootstrapping）操作之前仅允许有限的乘法深度，因此每个模型的非线性部分都必须使用基于迭代乘法的方法进行近似。

Fully homomorphic encryption (FHE) enables servers to execute large language models directly on encrypted user prompts without decrypting them. However, current FHE inference methods remain prohibitively slow. Because ciphertexts natively support only addition, multiplication, and rotation—and allow only a bounded multiplication depth before requiring costly "bootstrapping" operations—every model nonlinearity must be approximated using iterative multiplication-based methods.

传统上，较高的迭代次数会带来更好的精度，但会更快耗尽可用的计算深度，从而触发高延迟的自举。现有方法依赖于整个模型中的统一迭代次数，而不是适应局部的误差容限。

Traditionally, higher iteration counts yield better precision but exhaust the available computation depth faster, triggering latency-heavy bootstraps. Existing approaches rely on uniform iteration counts across the entire model rather than adapting to local error tolerances.

为了克服这一问题，作者推出了同态加密感知训练（HEAT），这是一个将每个非线性层的迭代次数视为可学习参数的微调框架。这使得迭代次数和模型权重能够在训练过程中协同适配。通过直接针对任务目标优化这些迭代，HEAT使模型能够在推理过程中原生吸收近似误差，而无需进行结构更改或从头重新训练。

To overcome this, the authors introduce Homomorphic Encryption-Aware Training (HEAT), a fine-tuning framework that treats per-nonlinearity iteration counts as learnable parameters. This allows the iteration counts and model weights to co-adapt during training. By optimizing these iterations directly against the task objective, HEAT enables models to natively absorb approximation errors during inference without requiring structural changes or retraining from scratch.

在加密的GPT-2解码任务上进行测试时，HEAT实现了显著的性能提升： * \(3.1\times\) 更少的迭代次数 * \(1.6\times\) 更少的自举次数 * \(1.4\times\) 更低的端到端延迟 * 与校准基线相比，提高了解码一致性。

Tested on encrypted GPT-2 decoding, HEAT achieves significant performance improvements: * \(3.1\times\) fewer iterations * \(1.6\times\) fewer bootstraps * \(1.4\times\) lower end-to-end latency * Improved decode agreement compared to the calibrated baseline.

Metadata & Reference Information

Metadata & Reference Information

license icon