跳转至

面向忆阻器的哈amard水库计算：规模化、结构化且无乘法器的递归结构

文章背景与核心概要

水库计算（Reservoir Computing, RC）作为一种围绕固定（即未训练）循环层构建循环神经网络的方法，是神经形态硬件的理想候选方案。然而，传统的面向忆阻器的水库计算虽然借鉴了忆阻器动力学，但仍依赖于物理实现成本高昂的密集循环权重矩阵。本文提出了一种面向神经形态硬件（特别是忆阻器设备）的全新结构化且无乘法器的水库计算方法。

通过使用由符号对角线、置换以及快速沃尔什-哈达玛变换（FWAT）构建的正交算子来替代密集循环权重矩阵，该方法在不需要矩阵实体化的前提下，能够高效地扩展到大规模（\(N = 8192\)）。作者证明，这种结构化模型每步仅需 \(O(N)\) 个参数和 \(O(N \log N)\) 次运算，与密集矩阵实现相比，可实现高达 50 倍的速度提升和 \(10^4\) 倍的内存缩减，同时在众多基准测试中保持了优秀的性能。

元数据与参考信息 (Metadata & Reference Information)

摘要 (Abstract)

水库计算（RC）围绕一个固定（即未训练）的循环层来设计循环神经网络，是神经形态硬件的天然候选方案。面向忆阻器的水库从忆阻器器件动力学中推导神经元动力学，但仍然依赖于密集循环矩阵，这在物理实现上代价高昂。

Reservoir Computing (RC) designs Recurrent Neural Networks around a fixed, i.e., untrained, recurrent layer, and is a natural candidate for neuromorphic hardware. Memristive-friendly reservoirs derive the neuron dynamics from memristive-device kinetics, but still rely on dense recurrent matrices, which are expensive to realize physically.

在本文中，我们用一个结构化的正交算子代替了密集矩阵，该算子由符号对角线、置换和快速沃尔什-哈达玛变换构成。该算子无乘法器，每步需要 \(O(N)\) 个参数和 \(O(N\log N)\) 次运算，并且从不作为矩阵实体化。我们将其实例化于标准以及面向忆阻器的回声状态网络中，每个单元具有一个二值输入连接。

In this paper, we replace the dense matrix with a structured orthogonal operator, built from sign diagonals, a permutation, and a fast Walsh-Hadamard transform. The operator is multiplier-free, requires \(O(N)\) parameters and \(O(N\log N)\) operations per step, and is never materialized as a matrix. We instantiate it in a standard and in a memristive-friendly Echo State Network, with one binary input connection per unit.

我们的数学分析表明，精确正交性产生了一个在循环缩放中紧致的回声状态条件，以及在设计时可预测的噪声响应。此外，该算子在单次应用中即可对整个状态进行混合。

Our mathematical analysis shows that exact orthogonality yields an echo state condition that is tight in the recurrent scaling, and a noise response that is predictable at design time. Moreover, the operator mixes the whole state in a single application.

在 20 个分类和 7 个回归基准测试（水库规模高达 \(N = 8192\)）上的实验表明，结构化模型与密集正交水库相匹配，并且在平均性能上以随着规模扩大的优势超越了循环水库。此外，我们在三个硬件平台上对循环步骤进行了计时，结果表明其比密集矩阵乘积快高达 50 倍，内存占用减小了 \(10^4\) 倍。最后，我们对算子进行了消融研究，并测量了对噪声、量化、器件失配和离散故障的响应。

Experiments on twenty classification and seven regression benchmarks, at reservoir sizes up to \(N = 8192\), show that the structured models match dense orthogonal reservoirs, and achieve better mean performance than the cycle reservoir by a margin that widens with size. Furthermore, we time the recurrent step on three hardware platforms, where it is up to \(50\times\) faster than a dense product and \(10^4\times\) smaller in memory. Finally, we ablate the operator and measure the response to noise, quantization, device mismatch, and discrete faults.