跳转至

文章背景与核心概要

传统的端到端反向传播在深度学习中取得了巨大成功，但其生物学合理性一直受到质疑。局部学习方法作为一种可行的替代方案，通过局部目标函数避免了全局反向传播，但在深度网络中往往因缺乏结构化的局部目标而导致特征学习能力较弱。本文旨在研究调整隐藏状态的优化顺序是否能够缓解这一局限性。

为此，作者提出了一种“边界优先（boundary-first）”的推理调度方案，将神经网络模型划分为不同的区块（chunks）。该方法首先协调区块边界处的隐藏状态，然后再精细化调整各个区块内部的表征。在 CIFAR-10 数据集上将该方法应用于预测编码网络（PCNs）时，边界优先方案带来了显著的性能提升：标准参数化下准确率提升了 9.77%，而 \(\mu\)-参数化下准确率提升了 5.51%。诊断分析进一步表明，这种优化策略能够带来更显著的早期层更新、更低的初始到最终中心核对齐（CKA）以及更高的逐层梯度多样性，这些都是特征学习能力增强的表现。这些发现确立了边界优先、基于区块的推理作为预测编码架构稳健设计原则的地位。

A Study of Hidden-State Optimization Order in Predictive Coding Networks

Authors: Xueyuan Li, Danilo Vasconcellos Vargas
Submitted: September 1, 2026
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv: 2609.00686 [cs.LG] | DOI: 10.48550/arXiv.2609.00686

📌 Summary

Local learning methods provide a viable alternative to traditional end-to-end backpropagation, but often suffer from weak feature learning in deep networks due to unstructured local objectives. This study investigates whether the order in which hidden states are optimized can mitigate this limitation.

The authors propose a boundary-first inference schedule that partitions neural models into distinct chunks. By first coordinating hidden states at the chunk boundaries before refining internal representations, this method improves local learning stability and effectiveness. When instantiated within predictive coding networks (PCNs) on the CIFAR-10 dataset, the boundary-first approach yields significant performance gains: * +9.77% accuracy improvement under a standard parametrization. * +5.51% accuracy improvement under a \(\mu\)-parametrization.

Diagnostic analyses reveal that this optimization strategy leads to more non-trivial early-layer updates, lower initial-to-final Centered Kernel Alignment (CKA), and greater layerwise gradient diversity—all indicators of enhanced feature learning. These findings establish boundary-first, chunk-based inference as a robust design principle for predictive-coding architectures.

📄 Abstract

局部学习方法为端到端反向传播提供了一种替代方案，但它们无结构的局部目标在深度网络中容易导致特征学习较弱。我们研究了隐藏状态优化顺序是否能够解决这一局限性。我们提出了一种边界优先的推理调度方法，将模型划分为多个区块，首先协调区块边界处的隐藏状态，然后精细化每个区块内的表征。我们将此调度应用于预测编码网络（PCNs）中——这是一种在推理过程中显式暴露隐藏活动和预测误差的局部学习框架。在 CIFAR-10 数据集上，所产生的边界优先预测编码实例化方案在标准参数化下将准确率提升了 \(9.77\%\)，在 \(\mu\)-参数化下提升了 \(5.51\%\)。诊断分析进一步表明，该方案带来了更多非平凡的早期层更新、更低的初始到最终 CKA 以及更多样化的逐层梯度，这与更强的特征学习相一致。这些结果支持将边界优先、基于区块的推理作为预测编码训练的实用设计原则，并推动了其在更广泛的局部学习系统中的研究。

Local learning methods offer an alternative to end-to-end backpropagation, but their unstructured local objectives can produce weak feature learning in deep networks. We study whether the order of hidden-state optimization can address this limitation. We propose a boundary-first inference schedule that partitions a model into chunks, first coordinates hidden states at chunk boundaries, and then refines representations within each chunk. We instantiate this schedule in predictive coding networks (PCNs), a local-learning framework in which hidden activities and prediction errors are explicitly exposed during inference. On CIFAR-10, the resulting boundary-first predictive-coding instantiation improves accuracy over standard predictive coding by \(9.77\%\) under a standard parametrization and by \(5.51\%\) under a \(\mu\)-parametrization. Diagnostic analyses further show more non-trivial early-layer updates, lower initial-to-final CKA, and more diverse layerwise gradients, consistent with stronger feature learning. These results support boundary-first, chunk-based inference as a practical design principle for predictive-coding training and motivate its study in broader local-learning systems.

📜 Submission History

提交历史：

  • [v1] Tue, 1 Sep 2026 04:07:04 UTC (223 KB)

license icon