跳转至

图像分类神经网络中，权重高的神经元一定重要吗？

文章背景与核心概要

本文探讨了图像分类神经网络中长期被广泛默认的一个假设：高权重神经元等同于模型的重要性。通过在 CIFAR-10 和 Mini-ImageNet 数据集上采用重叠分析、扰动测试以及消融-再训练实验等严格的评估方法，作者证明了神经元的重要性实际上呈现非线性关系。

核心研究发现表明，权重最高的前 10% 神经元与对准确率真正起决定作用的神经元之间仅有极少量的重叠；虽然移除这些高权重神经元会严重损害网络性能，但低权重神经元同样做出了显著贡献。这一发现挑战了传统的“权重-重要性”等同假设，并为神经网络剪枝、模型加密以及后门防御等下游任务提供了更为精细的理论见解。

执行摘要 (Executive Summary)

This paper investigates the widely assumed equivalence between high-weight neurons and model importance in image classification neural networks. Through a rigorous assessment methodology comprising overlap analysis, perturbation tests, and ablation-retraining experiments on the CIFAR-10 and Mini-ImageNet datasets, the authors demonstrate that neuron importance is actually nonlinear.

Key findings reveal that top high-weight neurons show limited overlap with accuracy-impacting neurons, and while their removal severely harms network performance, low-weight neurons also contribute significantly. This challenges traditional weight-importance assumptions and provides refined insights for downstream tasks such as neural network pruning, encryption, and backdoor defense.

本文探讨了图像分类神经网络中长期被广泛假定的一个结论：高权重神经元等同于模型重要性。通过在 CIFAR-10 和 Mini-ImageNet 数据集上开展涵盖重叠分析、扰动测试和消融-再训练实验的严格评估方法，作者证明了神经元的重要性实际上是非线性的。

关键研究结果表明，权重最高的前列神经元与影响准确率的神经元之间重叠度有限；虽然移除它们会严重损害网络性能，但低权重神经元也做出了重要贡献。这一发现挑战了传统的权重-重要性假设，并为神经网络剪枝、加密和后门防御等下游任务提供了更为精细的见解。

论文元数据 (Paper Metadata)

  • arXiv Identifier: arXiv:2607.25529 [cs.AI]
  • Primary Subject: Computer Science > Artificial Intelligence (cs.AI)
  • Authors: Qitao Chen, Dongfu Yin, Xirui Yang, Zhaoye Li, Liang Xiao
  • Submission Date: July 28, 2026 (Last revised August 27, 2026)
  • Digital Object Identifier (DOI): 10.48550/arXiv.2607.25529
  • arXiv 标识符： arXiv:2607.25529 [cs.AI]
  • 主要学科： 计算机科学 > 人工智能 (cs.AI)
  • 作者： Qitao Chen, Dongfu Yin, Xirui Yang, Zhaoye Li, Liang Xiao
  • 提交日期： 2026年7月28日（最近修订于2026年8月27日）
  • 数字对象唯一标识符 (DOI)： 10.48550/arXiv.2607.25529

摘要 (Abstract)

As neural network models for image classification advance, neurons play critical roles in pruning, backdoor defense, and interpretability. Yet existing work lacks clarity on the weight-importance relationship. We address this with a neuron importance assessment method using three experiments:

  1. Quantifying overlap between high-weight and accuracy-impacting neurons.
  2. Analyzing high-weight neuron perturbation effects.
  3. Testing post-retraining accuracy after high-weight neuron ablation.

随着图像分类神经网络模型的不断发展，神经元在模型剪枝、后门防御和可解释性方面发挥着关键作用。然而，现有研究对于“权重与重要性”之间的关系仍缺乏清晰的认识。为此，我们提出了一种神经元重要性评估方法，并通过三项实验进行研究：

  1. 量化高权重神经元与影响准确率的神经元之间的重叠度。
  2. 分析高权重神经元的扰动效应。
  3. 测试高权重神经元消融后的再训练准确率。

Experiments on CIFAR-10 and Mini-ImageNet reveal key patterns: * Overlap Analysis: The top 10% high-weight neurons overlap with important ones by only about 25% at maximum, dropping further in subsequent intervals. * Perturbation Tests: Top 10% high-weight neurons cause 45–80% accuracy degradation under certain operations compared to 3–7% for random perturbations, but a third of them show minimal impact. * Ablation-Retraining Results: Removing the top 10% high-weight neurons leaves accuracy 10–20% below baseline with no recovery, while ablating the top 0.1% allows near-full recovery.

在 CIFAR-10 和 Mini-ImageNet 上的实验揭示了以下关键规律： * 重叠分析： 权重最高的前 10% 神经元与重要神经元的重叠率最高仅约 25%，在后续区间中这一比例还会进一步下降。 * 扰动测试： 在某些操作下，权重最高的前 10% 神经元会导致 45–80% 的准确率下降（相比之下随机扰动仅导致 3–7%），但其中有三分之一的神经元表现出极小的影响。 * 消融-再训练结果： 移除权重最高的前 10% 神经元会导致准确率比基线低 10–20% 且无法恢复，而消融前 0.1% 的神经元则可以实现几乎完全的恢复。

Notably, some low-weight intervals show 10–17% degradation when perturbed, comparable to mid-range high-weight neurons. These results confirm not all high-weight neurons are important: their importance is nonlinear, and low-weight neurons also contribute significantly. This challenges weight-importance equivalence, offering refined neuron role insights that support applications like encryption (prioritizing critical high-weight neurons) and pruning (removing non-critical ones).

值得注意的是，某些低权重区间在受到扰动时会表现出 10–17% 的性能下降，这与中等高权重神经元的影响相当。这些结果证实了并非所有高权重神经元都是重要的：它们的重要性呈非线性分布，且低权重神经元也做出了重要贡献。这一发现挑战了“权重与重要性等同”的观点，为理解神经元作用提供了更精细的见解，进而支持了模型加密（优先处理关键的高权重神经元）和模型剪枝（移除不关键的神经元）等应用。

访问全文与资源 (Access Full-Text & Resources)

External Tools & References

外部工具与参考资料