跳转至

文章背景与核心概要

在现代生产系统中，特别是基于大模型的智能体（Agentic）应用，系统往往需要通过实时请求和带有噪声的反馈来动态评估不同的配置方案，且通常缺乏具有代表性的验证数据。为了解决这一痛点，本文作者将线上超参数优化（Online Hyperparameter Optimization, OHPO）问题进行了形式化定义，并将其转化为在混合与条件搜索空间上的无穷多臂老虎机（infinitely many-armed bandit）问题。

为此，作者提出了 IMABO 这一通用框架，它能够将任意的老虎机策略（用于在已采样配置中进行选择）与任意的预言机（用于提出新配置）相结合。作为具体的实例化方案，他们开发了 IMOSS——一种无需重启的随时策略（anytime policy），其活动集（active set）随时间 \(t^{\beta}\) 增长。作者证明了其期望累积分位数遗憾界为 \(O(p_\rho^{-1/\beta} + T^{(1+\beta)/2})\)，并通过实验证明：将 IMOSS 与实际预言机（树结构 Parzen 估计器、基于坐标的多臂老虎机驱动的现行变异预言机、以及预训练表格基础模型）相结合，在各项 OHPO 任务中均持续优于基线方法。

Bandits in Prod: Hyperparameter Optimization at Inference Time

arXiv: 2609.01335 [cs.LG]
Authors: Louis Abraham, Tuan-Anh Nguyen, Nicolas Devatine
Submitted: 1 September 2026 (Revised: 2 September 2026)
Links: View PDF | GitHub Repository

arXiv: 2609.01335 [cs.LG]
Authors: Louis Abraham, Tuan-Anh Nguyen, Nicolas Devatine
Submitted: 1 September 2026 (Revised: 2 September 2026)
Links: View PDF | GitHub Repository

Summary

Production systems—particularly modern agentic applications—frequently need to evaluate configurations dynamically using live requests and noisy feedback, often without the benefit of representative validation data. To address this, the authors formalize the problem of Online Hyperparameter Optimization (OHPO) as an infinitely many-armed bandit over mixed and conditional search spaces.

They introduce IMABO, a general framework that combines any bandit policy (for selecting among already sampled configurations) with any oracle (for proposing new configurations). As a concrete instance, they develop IMOSS, a restart-free anytime policy whose active set grows as \(t^{\beta}\). The authors prove an expected cumulative quantile-regret bound of \(O(p_\rho^{-1/\beta} + T^{(1+\beta)/2})\), and demonstrate that combining IMOSS with practical oracles (Tree-structured Parzen Estimator, an incumbent-mutation oracle, and a pretrained tabular foundation model) consistently outperforms baseline approaches across diverse OHPO tasks.

生产系统——尤其是现代智能体应用——经常需要利用实时请求和带噪声的反馈来动态评估配置，且往往缺乏具有代表性的验证数据。为了解决这一问题，作者将线上超参数优化（Online Hyperparameter Optimization, OHPO）问题形式化，并将其建模为混合与条件搜索空间上的无穷多臂老虎机（infinitely many-armed bandit）。

他们引入了 IMABO 这一通用框架，该框架将任意老虎机策略（用于在已采样的配置中进行选择）与任意预言机（用于提出新配置）相结合。作为具体的实例，他们开发了 IMOSS，这是一种无需重启的随时策略（anytime policy），其活动集随 \(t^{\beta}\) 增长。作者证明了其期望累积分位数遗憾界为 \(O(p_\rho^{-1/\beta} + T^{(1+\beta)/2})\)，并证明了将 IMOSS 与实际预言机（树结构 Parzen 估计器、基于坐标的老虎机驱动的现行变异预言机、以及预训练表格基础模型）相结合，在各种 OHPO 任务中均持续优于基线方法。

Abstract

Many production systems can assess a configuration only by using it on live requests and observing noisy feedback. Modern agentic systems are a prominent example, with inference-time choices such as model selection, retrieval depth, prompting strategy, and decoding temperature, yet often with no representative validation data.

We formalize this setting as Online Hyperparameter Optimization (OHPO) and cast it as an infinitely many-armed bandit over mixed and conditional search spaces. We introduce IMABO, a general framework that combines any bandit policy for choosing among already sampled configurations with any oracle for proposing new ones.

We instantiate it with IMOSS, a restart-free anytime policy whose active set grows as \(t^{\beta}\), and prove an expected cumulative quantile-regret bound of \(O(p_\rho^{-1/\beta} + T^{(1+\beta)/2})\), where \(\beta\in(0,1)\) controls active-set growth and \(p_\rho\) lower-bounds the probability that a proposed configuration falls in the top-\(\rho\) fraction of the search space.

We combine IMOSS with three practical oracles: a Tree-structured Parzen Estimator, an incumbent-mutation oracle driven by a per-coordinate bandit, and a pretrained tabular foundation model, all three improving over the uniform random oracle baseline. IMABO outperforms all baselines in terms of regret across diverse OHPO settings, from tuning classical machine-learning models to configuring LLM-based agents.

许多生产系统只能通过在实时请求上使用配置并观察带噪声的反馈来评估配置。现代智能体系统就是一个突出的例子，其推理时的选择包括模型选择、检索深度、提示词策略和解码温度，但通常没有具有代表性的验证数据。

我们将此场景形式化为线上超参数优化（OHPO），并将其转换为混合与条件搜索空间上的无穷多臂老虎机。我们引入了 IMABO，这是一个通用框架，它将用于在已采样配置中进行选择的任意老虎机策略与用于提出新配置的任意预言机相结合。

我们将其实例化为 IMOSS，这是一种无需重启的随时策略，其活动集随 \(t^{\beta}\) 增长，并证明了其期望累积分位数遗憾界为 \(O(p_\rho^{-1/\beta} + T^{(1+\beta)/2})\)，其中 \(\beta\in(0,1)\) 控制活动集的增长，\(p_\rho\) 是所提配置落入搜索空间前 \(\rho\) 比例的概率的下界。

我们将 IMOSS 与三个实际预言机相结合：树结构 Parzen 估计器、由每坐标老虎机驱动的现行变异预言机、以及预训练表格基础模型，这三者均优于均匀随机预言机基线。在从调优经典机器学习模型到配置基于大模型的智能体的各种 OHPO 设置中，IMABO 在遗憾（regret）方面均优于所有基线。

Metadata & Additional Information