跳转至

文章背景与核心概要

自动启发式设计（AHD）是解决NP难组合优化问题（COPs）的一个极具前景的方法。尽管将大语言模型（LLM）融入LLM-AHD框架已展现出发现高性能启发式算法的能力，但它们通常在固定工作流中扮演被动角色，依赖于人工设计且往往受限的上下文。这种缺乏状态依赖信息的特点容易导致低效的试错探索。

为了解决这些缺陷，本文提出了 AHD Agent，这是一个新颖的集成工具的多轮框架，允许LLM主动选择是生成启发式算法，还是调用工具从求解环境中进行有针对性的证据检索。为了优化这种动态决策智能体，本文引入了一种包含新颖环境合成管线的智能体强化学习（RL）系统。在八个不同领域（包括四个保留任务）上的评估表明，利用该智能体框架的紧凑型4B参数模型，其性能能够匹配甚至超越依赖巨型模型的行业顶尖基线，同时所需评估次数显著减少。

AHD Agent: Agentic Reinforcement Learning for Automatic Heuristic Design

Summary

Automatic Heuristic Design (AHD) is a promising method for solving NP-hard combinatorial optimization problems (COPs). While Large Language Models (LLMs) integrated into LLM-AHD frameworks have shown the ability to discover high-performing heuristics, they are typically limited to passive roles within fixed workflows, relying on manually designed and often restricted context. This lack of state-dependent information can lead to inefficient trial-and-error exploration.

To address these shortcomings, the authors propose AHD Agent, a novel tool-integrated, multi-turn framework that allows LLMs to proactively choose whether to generate heuristics or invoke tools for targeted evidence retrieval from the solving environment. To optimize this dynamic decision-making agent, the paper introduces an agentic reinforcement learning (RL) system featuring a novel environment synthesis pipeline. Evaluations across eight diverse domains (including four held-out tasks) demonstrate that a compact 4B-parameter model utilizing this agentic framework matches or outperforms state-of-the-art baselines relying on much larger models, while requiring significantly fewer evaluations.

Paper Metadata

属性 详情
arXiv 标识符 arXiv:2605.08756 [cs.AI]
作者 Haoze Lv, Ning Lu, Ziang Zhou, Yew-Soon Ong, Shengcai Liu
主要学科 人工智能 (cs.AI)
次要学科 神经与进化计算 (cs.NE)
提交日期 2026年5月9日（2026年8月13日最后修订）
DOI 10.48550/arXiv.2605.08756
Attribute Detail
arXiv Identifier arXiv:2605.08756 [cs.AI]
Authors Haoze Lv, Ning Lu, Ziang Zhou, Yew-Soon Ong, Shengcai Liu
Primary Subject Artificial Intelligence (cs.AI)
Secondary Subject Neural and Evolutionary Computing (cs.NE)
Submitted Date May 9, 2026 (last revised August 13, 2026)
DOI 10.48550/arXiv.2605.08756

Abstract

自动启发式设计（AHD）已成为解决NP难组合优化问题（COPs）的一个有前景范式。近期研究表明，当大语言模型（LLM）被集成到设计良好的框架（即LLM-AHD）中时，它们可以自主发现高性能的启发式算法。然而，现有的LLM-AHD框架通常将LLM视为固定工作流中的被动生成器，模型根据人工设计、有限的上下文来生成启发式算法。此类上下文往往无法捕获状态依赖的信息（例如特定的失效模式），从而导致低效的试错探索。

为了克服这些局限性，我们提出了 AHD Agent，这是一个新颖的集成工具的多轮框架，它赋能LLM主动决定是生成启发式算法，还是调用工具从求解环境中检索有针对性的证据。为了有效训练这样一个动态决策智能体，我们引入了一种智能体强化学习（RL）系统，该系统利用新颖的环境合成管线来优化紧凑模型的泛化AHD能力。在包含四个保留任务在内的八个不同领域的实验表明，我们的4B参数智能体能够匹配或超越使用大得多的模型的行业顶尖基线，同时所需的评估次数显著减少。模型和推理扩展分析进一步表明，AHD Agent为实现真正自主的启发式设计提供了一条有效的演进路径。

Automatic heuristic design (AHD) has emerged as a promising paradigm for solving NP-hard combinatorial optimization problems (COPs). Recent works show that large language models (LLMs), when integrated into well-designed frameworks (i.e., LLM-AHD), can autonomously discover high-performing heuristics. However, existing LLM-AHD frameworks typically treat LLMs as passive generators within fixed workflows, where the model generates heuristics from manually designed, limited context. Such context may fail to capture state-dependent information (e.g., specific failure modes), leading to inefficient trial-and-error exploration.

To overcome these limitations, we propose AHD Agent, a novel tool-integrated, multi-turn framework that empowers LLMs to proactively decide whether to generate heuristics or invoke tools to retrieve targeted evidence from the solving environment. To effectively train such a dynamic decision-making agent, we introduce an agentic reinforcement learning (RL) system, which leverages a novel environment synthesis pipeline to optimize a compact model's generalizable AHD capabilities. Experiments across eight diverse domains, including four held-out tasks, demonstrate that our 4B-parameter agent matches or surpasses state-of-the-art baselines using much larger models, while requiring significantly fewer evaluations. Model and inference scaling analysis further reveals that AHD Agent offers an effective trajectory toward truly autonomous heuristic design.