跳转至

位置至关重要：带有Token裁剪与重排的ViT分布式推理中的特征反演攻击

文章背景与核心概要

随着视觉Transformer（ViT）在分布式推理（Split Inference）架构中的广泛应用，边缘设备常将中间层的Token表示卸载至远程云端。为了优化资源利用率并防止隐私泄露，业界通常会采用Token裁剪（减少计算与通信开销）和Token重排（打乱空间布局以阻碍隐私泄露）等技术。然而，本文的研究表明，Token重排实际上只提供了一种“虚假的隐私保护”，传输的Token嵌入中依然保留了大量的空间位置信息。

为了证实这一漏洞，作者提出了空间对齐重构攻击（SARA）流水线。该方法能够预测Token位置、恢复其空间布局、利用特征空间掩码自编码器重构丢失的嵌入，并最终恢复原始输入图像。虽然Token裁剪能提供更强的保护，但当保留的Token仍包含足够语义和位置信息时，严重的隐私泄露问题依然存在。为此，作者提出了一种轻量级的边缘端防御机制：剥离位置嵌入，并通过知识 distillation（知识蒸馏）逐步适配边缘端Transformer块，在不改变云端模型且不损害任务准确率的前提下成功抵御了SARA攻击。

📌 Summary

Vision Transformers (ViTs) are frequently utilized in split-inference frameworks where edge devices offload intermediate token representations to a remote cloud. To optimize resource utilization, techniques like token reduction (minimizing computation/communication costs) and token shuffling (disrupting spatial layout to obstruct privacy leakage) are commonly applied.

However, this paper demonstrates that token shuffling provides only an illusion of privacy. Despite the disrupted spatial structure, transmitted token embeddings still harbor substantial positional information. To prove this, the authors introduce the Spatially Aligned Reconstruction Attack (SARA), a pipeline capable of predicting token positions, restoring their layout, reconstructing missing embeddings, and recovering the original input image. While token reduction offers more resilient protection, severe privacy leakage persists when retained tokens preserve sufficient semantic and positional details.

To counter these vulnerabilities, the authors propose a lightweight edge-side defense that strips positional embeddings and progressively adapts edge-side transformer blocks via knowledge distillation—successfully defending against SARA without altering the cloud-side model or harming task accuracy.

视觉Transformer（ViTs）常被用于分布式推理框架中，边缘设备在此架构下将中间Token表示卸载至远程云端。为了优化资源利用，通常会应用Token裁剪（最小化计算/通信成本）和Token重排（打乱空间布局以阻碍隐私泄露）等技术。

然而，本文证明了Token重排仅提供了一种隐私的幻觉。尽管空间结构被打破，但传输的Token嵌入仍然包含大量的空间位置信息。为了证明这一点，作者引入了空间对齐重构攻击（SARA），这是一个能够预测Token位置、恢复其布局、重构丢失的嵌入并恢复原始输入图像的流水线。虽然Token裁剪提供了更具韧性的保护，但当保留的Token保留了足够的语义和位置细节时，严重的隐私泄露依然存在。

为了应对这些漏洞，作者提出了一种轻量级的边缘端防御方案，该方案剥离了位置嵌入，并通过知识蒸馏逐步适应边缘端的Transformer块——在不改变云端模型或损害任务准确性的情况下，成功防御了SARA攻击。

📑 Abstract

Vision Transformers (ViTs) are increasingly used in split-inference systems, where edge devices transmit intermediate token representations to a remote cloud. In this setting, token reduction lowers computation and communication costs, while token shuffling disrupts the spatial organization of the transmitted tokens, potentially limiting information leakage. However, their privacy benefits remain unclear against feature inversion attacks, which attempt to reconstruct the input from the transmitted embeddings. In this work, we show that, despite disrupting the spatial structure required by conventional reconstruction attacks, transmitted token embeddings retain substantial positional information. Based on this observation, we introduce the Spatially Aligned Reconstruction Attack (SARA), a unified pipeline that predicts token positions, restores their spatial layout, reconstructs missing embeddings using a feature-space masked autoencoder, and recovers the input image. Our results demonstrate that token shuffling provides only apparent privacy, as SARA largely reconstructs the original token organization. Token reduction offers stronger protection, but significant leakage persists when the retained tokens preserve sufficient semantic and positional information. Finally, we introduce a lightweight edge-side defense that removes positional embeddings and progressively adapts the edge-side transformer blocks through knowledge distillation. It substantially reduces attack performance against SARA, while preserving downstream task accuracy and requiring no changes to the cloud-side model.

视觉Transformer（ViTs）正日益被广泛应用于分布式推理系统中，边缘设备在其中将中间的Token表示传输到远程云端。在此设定下，Token裁剪降低了计算和通信成本，而Token重排则打乱了传输Token的空间组织，从而可能限制信息泄露。然而，针对试图从传输嵌入中重构输入的特征演反演攻击，它们的隐私保护效益尚不明确。在这项工作中，我们表明，尽管破坏了常规重构攻击所需的空间结构，但传输的Token嵌入仍然保留了大量的空间位置信息。基于这一观察，我们引入了空间对齐重构攻击（SARA），这是一个统一的流水线，能够预测Token位置、恢复其空间布局、使用特征空间掩码自编码器重构丢失的嵌入，并恢复输入图像。我们的结果表明，Token重排仅提供表面上的隐私，因为SARA在很大程度上重构了原始的Token组织。Token裁剪提供了更强的保护，但当保留的Token保留了足够的语义和位置信息时，显著的泄露依然存在。最后，我们引入了一种轻量级的边缘端防御机制，它去除了位置嵌入，并通过知识蒸馏逐步调整边缘端的Transformer块。它大幅降低了SARA的攻击性能，同时保持了下游任务的准确性，且无需对云端模型做任何修改。

访问论文： 查看 PDF | HTML (实验性) | TeX 源码 许可协议： 知识共享署名 4.0 license icon 外部书目工具： * Google Scholar * Semantic Scholar * NASA ADS