跳转至

自动驾驶课程学习的规模化扩展

文章背景与核心概要

近年来，面向自动驾驶的批处理仿真器通过在几天内对数千个交通场景和数十亿次交互进行智能体训练，彻底改变了强化学习（RL）的研究方式。然而，由于标准的域随机化方法采取均匀采样场景的策略，将大量计算资源浪费在对学习贡献甚微的案例上，样本效率依然是制约性能的瓶颈。

本文推出了 CL4AD，这是首个将课程学习（Curriculum Learning）融入批处理自动驾驶仿真器的工作，它将场景选择构建为一个无监督环境设计问题。通过利用成功率、行为真实性以及后悔估计效用函数，CL4AD 显著加速了训练并减少了计算开销。该研究在大规模实验中展示了突破性的性能提升，为高效的自动驾驶强化学习训练开辟了新途径。

📌 总结 (Summary)

Batched simulators for autonomous driving have revolutionized reinforcement learning (RL) by training agents across thousands of traffic scenarios and billions of interactions within days. However, sample efficiency remains a bottleneck because standard domain randomization uniformly samples scenarios, wasting compute on cases that contribute little to learning.

面向自动驾驶的批处理仿真器通过在几天内对数千个交通场景和数十亿次交互进行智能体训练，彻底改变了强化学习（RL）。然而，样本效率仍然是一个瓶颈，因为标准的域随机化会均匀采样场景，从而将计算资源浪费在对学习贡献甚微的案例上。

This paper introduces CL4AD, the first integration of curriculum learning into batched autonomous driving simulators by framing scenario selection as an unsupervised environment design problem. By leveraging success rates, behavioral realism, and regret-estimation utility functions, CL4AD significantly accelerates training and reduces computational overhead.

本文推出了 CL4AD，这是首个将课程学习整合到批处理自动驾驶仿真器中的方法，它将场景选择构想为一个无监督环境设计问题。通过利用成功率、行为真实性和后悔估计效用函数，CL4AD 显著加速了训练并减少了计算开销。

📋 元数据 (Metadata)

  • arXiv ID: arXiv:2608.22549 [cs.AI]
  • 学科分类 (Subjects): 人工智能 (cs.AI)；机器学习 (cs.LG)
  • 提交日期 (Submission Date): 2026年8月23日
  • 状态 (Status): 正在 NeurIPS 2026 审稿中（31页，18幅图）
  • 作者 (Authors): Cevahir Koprulu, David Paz, Feng Tao, Yuliang Guo, Xinyu Huang, Ufuk Topcu, Liu Ren

🔍 摘要 (Abstract)

Batched simulators for autonomous driving have recently enabled training reinforcement learning (RL) agents at scale, encompassing thousands of traffic scenarios and billions of interactions within a matter of days. Although such high-throughput feeds RL algorithms faster than ever, their sample-efficiency has not kept pace: As the standard training scheme, domain randomization uniformly samples scenarios, thereby consuming a vast number of interactions on cases that contribute little to learning. Curriculum learning offers a remedy by adaptively prioritizing scenarios that matter most to policy improvement. We present CL4AD, the first integration of curriculum learning into batched autonomous driving simulators by framing scenario selection as an unsupervised environment design problem. We introduce utility functions that shape curricula based on success rates and the realism of the agent's behavior, in addition to existing regret-estimation functions. Large-scale experiments in GPUDRIVE demonstrate that curriculum learning achieves a 99% success rate a billion steps earlier than domain randomization, reducing wall-clock time by 77%, and outperforms heuristic curricula with static and dynamic attributes, with only one exception at the largest scale. An ablation under limited compute shows that curriculum learning improves sample efficiency by 67%. We also investigate how utility functions behave at scale, and how prioritized scenarios evolve during training. We release an implementation of CLForAD in GPUDRIVE.

近期，面向自动驾驶的批处理仿真器使得大规模训练强化学习（RL）智能体成为可能，能够在几天内涵盖数千个交通场景和数十亿次交互。尽管这种高吞吐量以空前的速度为强化学习算法提供数据，但其样本效率却没有跟上步伐：作为标准的训练方案，域随机化会均匀地采样场景，从而在对学习贡献甚微的案例上消耗了大量的交互次数。课程学习通过自适应地优先处理对策略改进最为关键的场景，为这一问题提供了解决方案。我们提出了 CL4AD，这是首个将课程学习集成到批处理自动驾驶仿真器中的方法，它将场景选择构想为一个无监督环境设计问题。除了现有的后悔估计函数外，我们还引入了基于成功率和智能体行为真实性来塑造课程的效用函数。在 GPUDRIVE 中进行的大规模实验表明，课程学习比域随机化早十亿步达到 99% 的成功率，将挂钟时间减少了 77%，并且在最大规模下的唯一特例之外，优于具有静态和动态属性的启发式课程。在有限计算资源下的消融实验表明，课程学习将样本效率提高了 67%。我们还研究了效用函数在大规模下的表现，以及优先场景在训练过程中的演变。我们在 GPUDRIVE 中开源了 CLForAD 的实现。

🔗 全文与资源 (Full-Text & Resources)

📚 参考文献与学术工具 (References & Academic Tools)