跳转至

文章背景与核心概要

大语言模型（LLM）在自动化代码生成领域取得了显著进展，其中可执行反馈比单纯的文本模仿提供了更可靠的学习信号。硬件验证是该领域的一个关键应用，消耗了现代芯片设计工作的大部分精力，而高覆盖率测试平台（Testbench）激励生成则是其中的主要瓶颈。

本文介绍了 CHORUS，一个超越传统“监督微调（SFT）到强化学习（RL）”流水线的新型后训练框架。通过利用在分阶段 SFT 期间生成的行为多样化检查点，并借助密集奖励强化学习将其转化为专门的任务级专家，CHORUS 通过无需训练的模型合并或进一步的后训练，有效地汇聚了互补优势。最终，CHORUS 将这些功能整合到一个紧凑的 4B 模型中，在 CVDP-ECov 上实现了令人瞩目的 88.0% Pass@1，比体量庞大得多的 DeepSeek-R1（671B）高出 13.5 个百分点。

CHORUS: Complementary Experts for High-Coverage Testbench Stimulus Generation

Executive Summary

Large language models (LLMs) have significantly advanced automated code generation, leveraging executable feedback as a reliable learning signal. Hardware verification represents a critical domain in this space, consuming a major portion of modern chip design efforts, with high-coverage testbench stimulus generation standing out as a primary bottleneck.

This paper introduces CHORUS, a novel post-training framework that surpasses standard Supervised Fine-Tuning (SFT) to Reinforcement Learning (RL) pipelines. By harnessing behaviorally diverse checkpoints generated during staged SFT and transforming them into specialized task-level experts via dense-reward RL, CHORUS effectively pools complementary strengths through training-free model merging or further post-training. Ultimately, CHORUS consolidates these capabilities into a single compact 4B model, achieving an impressive 88.0% Pass@1 on CVDP-ECov—outperforming the much larger DeepSeek-R1 (671B) by 13.5 percentage points.

Paper Metadata

  • arXiv Identifier: arXiv:2608.10090 [cs.AI]
  • Primary Subject: Artificial Intelligence (cs.AI)
  • Secondary Subject: Machine Learning (cs.LG)
  • Submission Date: August 10, 2026
  • Authors:
  • Hejia Zhang
  • Sheng Lu
  • Zhongming Yu
  • Chia-Tung Ho
  • Brucek Khailany
  • Jishen Zhao

Abstract

Large language models (LLMs) have advanced code generation, where executable feedback provides a more reliable learning signal than textual imitation alone. Hardware verification is an important application of code generation and accounts for a substantial fraction of modern chip design effort, with high-coverage testbench stimulus generation as a key task. We present CHORUS, a post-training framework that pushes performance beyond what a conventional supervised fine-tuning (SFT)-to-reinforcement learning (RL) pipeline achieves. CHORUS builds on two observations. First, staged SFT produces behaviorally diverse checkpoints, and dense-reward RL turns them into strong experts with comparable aggregate performance but distinct task-level strengths. Second, these complementary strengths can be exploited through either training-free model merging or further post-training to outperform the best individual expert. By consolidating the resulting specialists into a single 4B model, CHORUS achieves 88.0% Pass@1 on CVDP-ECov, outperforming DeepSeek-R1 (671B) by 13.5 percentage points.

Key Contributions & Core Insights

  1. Exploiting Staged SFT & Dense-Reward RL:

    Demonstrates that staged Supervised Fine-Tuning yields behaviorally diverse checkpoints, which can subsequently be scaled via dense-reward Reinforcement Learning into distinct experts possessing unique task-level strengths.

  2. Complementary Expert Integration:

    Proves that these complementary strengths can be effectively combined using either training-free model merging or advanced post-training methods to exceed the performance metrics of any single individual expert.

  3. State-of-the-Art Efficiency:

    Successfully consolidates specialist models into a streamlined 4B parameter model, achieving 88.0% Pass@1 on CVDP-ECov and outperforming the massive 671B DeepSeek-R1 model by a substantial margin of 13.5 percentage points.

license icon view license