文章背景与核心概要

神经网络究竟能否从视觉输入中真正学习到代数规则，还是仅仅死记硬背了观察到的模式？为了探究这一问题，本文以 MNIST/EMNIST 数字作为状态、模运算作为动作，在联合嵌入预测架构（JEPA）世界模型中展开了深入研究。

传统的监督学习和基线 JEPA 模型在面对未见过的操作时往往无法进行外推。为此，作者提出了 BRo-JEPA（块旋转 JEPA）。该模型通过将算术操作表示为块旋转（block-rotations），在潜空间中捕捉了模运算的循环结构。实验表明，仅通过原始操作（\(\pm 1\)）进行训练，该模型便实现了严格的零样本泛化——使用 ResNet-18 编码器在 MNIST 上达到了 99.44% 的准确率，在 EMNIST 上达到了 94.35%。

BRo-JEPA: Learning Modular Transformations in Latent Space

arXiv: 2606.01372 [cs.LG]
Authors: Divyansh Jha, Yuanfang Xie, Brennen Yu, Varan Mehra
Submitted: 31 May 2026 (v1), last revised 18 Aug 2026 (v2)
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

📌 Summary

Can neural networks genuinely learn algebraic rules from visual inputs, or do they merely memorize observed patterns? This paper investigates this question using MNIST/EMNIST digits as states and modular arithmetic operations as actions within a Joint Embedding Predictive Architecture (JEPA) world model.

While standard supervised and baseline JEPA models struggle to extrapolate to unseen operations, the authors introduce BRo-JEPA (Block-Rotation JEPA). By representing arithmetic operations as block-rotations, BRo-JEPA captures the cyclic structure of modular arithmetic in latent space. Trained exclusively on primitive operations (\(\pm 1\)), the model achieves strict zero-shot generalization—reaching 99.44% accuracy on MNIST and 94.35% on EMNIST using a ResNet-18 encoder.

📝 Abstract

神经网络能否从视觉输入中学习代数规则，还是仅仅拟合了观察到的模式？我们以 JEPA 风格的世界模型为研究平台，将 MNIST（或 EMNIST 字母）作为状态、模运算作为动作来研究这一问题。标准的监督学习和带有操作嵌入的 JEPA 基线模型在已见操作上能获得高准确率，但无法可靠地外推到未见操作。我们提出了 BRo-JEPA，这是一个带有块旋转预测器的世界模型，它将算术操作表示为旋转，从而在潜空间中呈现出模运算的循环结构。通过将动作应用为旋转，BRo-JEPA 预测器学习旋转角度以将潜表示与底层模结构对齐，从而实现了严格的零样本操作泛化。尽管最佳的块旋转监督基线在 MNIST 上的零样本准确率仅为 54.54%，在 EMNIST 上仅为 25.13%，但采用 ResNet-18 编码器的 BRo-JEPA 分别达到了 99.44% 和 94.35%，且它仅在原始操作 \(\pm 1\) 上进行过训练。我们的结果表明，当潜变换编码了底层的模结构时，世界模型完全能够学习代数规则。

Can neural networks learn algebraic rules from visual inputs, or do they merely fit observed patterns? We study this question using MNIST (or EMNIST letters) as states and modular arithmetic operations as actions in a JEPA-style world model. Standard supervised and JEPA baselines with operation embeddings achieve high accuracy on seen operations but fail to extrapolate reliably to unseen operations. We propose BRo-JEPA, a world model with a block-rotation predictor that represents arithmetic operations as rotations, resulting in the cyclic structure of modular arithmetic in latent space. By applying actions as rotations, the BRo-JEPA predictor learns the rotation angles to align the latent representations with the underlying modular structure which enables strict zero-shot operation generalization. While our best block-rotation supervised baseline reaches only 54.54% zero-shot accuracy on MNIST and 25.13% on EMNIST, BRo-JEPA with a ResNet-18 encoder achieves 99.44% and 94.35% respectively, despite being trained only on the primitive operations \(\pm\)1. Our results suggest that world models can learn algebraic rules when the latent transformations encode the underlying modular structure.

🔗 Links & Resources

🗂️ Submission History

提交历史：

[v1] Sun, 31 May 2026, 18:01:43 UTC
[v2] Tue, 18 Aug 2026, 16:15:35 UTC (Current version)

[v1] Sun, 31 May 2026, 18:01:43 UTC

[v2] Tue, 18 Aug 2026, 16:15:35 UTC (Current version)