跳转至

自我进化的算法设计智能体：基于种群筛选策略优化打破上下文进化停滞

Self-Evolving Algorithm-Design Agents: Escaping In-Context Evolutionary Stagnation via Population-Curated Policy Optimization

Authors: Chen Lu, Ke Xue, Siyuan Xu, Mingxuan Yuan, Chao Qian
Subjects: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
arXiv: 2609.38757 [cs.AI] | Submitted: 30 September 2026

文章背景与核心概要

在大语言模型 (Large Language Model, LLM) 驱动的自主算法设计中，现有的纯上下文进化方案在面对芯片设计、底层算子等高门槛专业领域时极易遭遇“进化停滞”，而传统的参数微调方法又受困于高质量专业算法语料的极度匮乏。为此，南京大学与华为等机构的研究团队提出了全新的参数自进化框架——种群筛选策略优化 (Population-Curated Policy Optimization, PCPO) ，在理论上证明了通过渐进学习自生成算法能够打破停滞的“改进链 (Improvement Chain) ”命题，并借助全局种群与混合策略更新机制，让智能体直接在自生成的高质量多样化算法中持续迭代进化。在电子设计自动化 (Electronic Design Automation, EDA) 的芯片全局布局学习率调度任务中，基于 8B 参数开源底座的 PCPO 仅需 4 个芯片案例的轻量训练，就在全部 16 个测试案例上全面超越了 OpenEvolve 和 ShinkaEvolve 等前沿上下文进化基准，性能足以比肩顶尖闭源模型 GPT-5.5，并显著降低了推理阶段的 Token 开销。此外，PCPO 在 4 款 GPU 底层算子设计任务中相比 PyTorch Eager 基线实现了平均 8.27 倍的显著加速，为解决复杂工业与高性能计算领域的算法自动化探索开辟了全新路径。

📋 内容概要

📋 Summary

如今，大语言模型 (LLM) 越来越多地以自主智能体的形态参与到复杂现实任务中，承担起算法设计与持续优化的重任。虽然基于上下文进化框架 (In-Context Evolutionary Frameworks) 的探索方法颇受青睐，但在需要高度专业领域知识的复杂场景下，这类方法往往很快就会陷入能力瓶颈与进化停滞；而传统的参数化适配 (Parametric Adaptation) 方法若想内化专业知识，又严重依赖海量的特定领域语料，这在高级算法设计这一高精尖场景中无疑是极其匮乏的。

Large Language Models (LLMs) are increasingly utilized as autonomous agents to design and refine algorithms for complex real-world tasks. While in-context evolutionary frameworks are popular, they often plateau quickly in domains requiring specialized knowledge. Conversely, traditional parametric adaptation requires large datasets of domain-specific corpora, which are extremely scarce for advanced algorithm-design scenarios.

为了弥合这一鸿沟，本文提出了种群筛选策略优化 (Population-Curated Policy Optimization, PCPO) ——这是一种高样本效率的参数自进化框架，允许智能体直接从自身探索生成的算法中进行反思与学习。

To bridge this gap, this paper introduces Population-Curated Policy Optimization (PCPO)—a sample-efficient parametric self-evolution framework where agents explore and learn directly from self-generated algorithms.

核心贡献与关键洞察

Key Contributions & Insights

  1. “改进链”理论命题 (The Improvement Chain Proposition) ： 从解析角度严密刻画了上下文进化停滞的内在机理，从理论上证明了通过渐进学习自身连续生成的算法，能够在局部空间内显著提升其邻近潜在优质算法的生成概率。
  2. 种群筛选策略优化 (Population-Curated Policy Optimization, PCPO) ： 提出了一种全新框架，巧妙利用全局种群与混合策略更新机制，有效留存、复用高质量且多样化的自生成算法，并引导策略持续向更强算法方向迁移。
  3. 电子设计自动化 (Electronic Design Automation, EDA) 落地： 在芯片全局布局学习率调度的算法设计任务中，PCPO 仅在 4 个芯片案例上进行微调训练，就在全部 16 个跨测试芯片案例上全面超越了 OpenEvolve 和 ShinkaEvolve 等业内最先进的上下文进化方法。即便仅采用 8B 参数规模的开源底座模型，PCPO 依然取得了比肩 GPT-5.5 等顶尖闭源前沿模型的优异表现；同时，得益于领域知识的参数化内化与提示词蒸馏，大幅削减了推理阶段的 Token 消耗成本。
  4. GPU 算子设计 (GPU Kernel Design) ： 在四项 GPU 算子设计任务中，相比 PyTorch Eager 基线实现了平均 \(8.27\times\) 的加速比。
  1. The Improvement Chain Proposition: Analytically characterizes in-context evolutionary stagnation and proves how learning successive self-generated algorithms locally increases the likelihood of neighboring algorithms.
  2. Population-Curated Policy Optimization (PCPO): A novel framework leveraging a global population and a hybrid policy update scheme to retain, reuse, and shift policies toward high-quality, diverse self-generated algorithms.
  3. Electronic Design Automation (EDA): In learning rate schedule design for chip global placement (trained on only 4 chip cases), PCPO outperforms state-of-the-art in-context methods (e.g., OpenEvolve, ShinkaEvolve) across 16 chip cases. Utilizing an 8B-parameter base model, PCPO achieves performance comparable to frontier closed-source models like GPT-5.5 while reducing inference token costs via internalized domain knowledge and prompt distillation.
  4. GPU Kernel Design: Achieves an average \(8.27\times\) speedup across four GPU kernel designs compared to the PyTorch Eager baseline.

论文摘要

Abstract

如今，大语言模型正日益以算法设计智能体的形式，深入参与到复杂的现实世界任务中，负责算法的全新构思与持续迭代。许多卓有成效的算法设计智能体普遍采用了纯上下文进化框架，然而在需要深厚专业领域知识的复杂场景中，这些方法往往很快就会遭遇瓶颈并陷入进化停滞。尽管通过参数化适配能够将专业知识深层内化到模型权重中，但常规的模型训练极度依赖海量的特定领域语料，而在高难度算法设计场景下，高质量的专业算法样本往往极度稀缺。为此，本文提出了一种高样本效率的参数自进化方法，使智能体能够直接从自身探索生成的算法中进行反思与学习。首先，我们深入剖析了上下文进化停滞的机理，并从理论上推导证明了“改进链” (Improvement Chain) 命题，揭示了通过连续学习自生成的算法，能够在局部空间内显著提升临近潜在优质算法的生成概率。基于这一“局部迁移”的理论洞察，我们进一步提出了种群筛选策略优化 (Population-Curated Policy Optimization, PCPO) 框架；该框架通过全局种群管理与混合策略更新机制，高效沉淀并复用高质量且多样化的自生成算法，推动模型策略持续向更强算法方向演进。在电子设计自动化 (EDA) 的芯片全局布局学习率调度设计任务中，仅在 4 个芯片案例上完成微调训练的 PCPO，在跨越全部 16 个测试芯片案例上的平均表现，全面超越了当前业内最先进的上下文进化方法 (例如 OpenEvolve 和 ShinkaEvolve) 。即使仅依托 8B 参数规模的轻量级开源基座模型，PCPO 依然展现出能够比肩 GPT-5.5 等顶尖闭源前沿模型的强劲竞争力。与此同时，通过将扎实的领域知识内化于参数权重以及运用提示词蒸馏技术，PCPO 大幅缩减了推理阶段所需的 Token 消耗成本。此外，PCPO 在四款 GPU 底层算子设计任务中均取得了显著的性能飞跃，相比 PyTorch Eager 基准实现了平均 8.27\(\times\) 的加速比。

Large language models are increasingly participating in complex real-world tasks in the form of algorithm-design agents, designing and refining algorithms. Many successful algorithm-design agents adopt pure in-context evolutionary frameworks, but they may quickly plateau in domains that require specialized knowledge. Parametric adaptation offers a way to internalize specialized knowledge, but conventional training requires abundant domain-specific corpora while high-quality algorithms are scarce in complex algorithm-design scenarios. In this paper, we propose sample-efficient parametric self-evolution where agents can explore and learn from self-generated algorithms. First, we characterize in-context evolutionary stagnation and analytically propose the Improvement Chain proposition, showing how learning successive self-generated algorithms can locally increase the likelihood of neighboring algorithms. Motivated by this local-transfer perspective, we further propose Population-Curated Policy Optimization (PCPO) to utilize a global population and a hybrid policy update scheme for retaining and reusing high-quality, diverse self-generated algorithms, shifting the policy towards stronger algorithms. In the task of learning rate schedule design for global placement in electronic design automation, trained only on 4 chip cases, PCPO outperforms the state-of-the-art in-context evolutionary methods (e.g., OpenEvolve and ShinkaEvolve) on average across 16 chip cases. With an 8B-size base model, PCPO achieves competitive performance compared to frontier closed-source models such as GPT-5.5. PCPO also reduces inference-time token cost by internalizing grounded domain knowledge and prompt distillation. Moreover, PCPO achieves significant speedups on four GPU kernel designs, with an average of 8.27\(\times\) speedup against the PyTorch Eager baseline.

访问链接与资源