跳转至

文章背景与核心概要

当前的工具增强型大语言模型（LLMs）通常局限于人类预定义的API。虽然某些系统尝试通过在推理时提示一个固定的LLM来动态生成工具，但它们存在一个根本性缺陷：工具编写模型与工具使用模型是解耦的，这意味着创建者生成的工具模式（schema）往往无法被自身可靠地调用。

为了克服这一局限性，作者推出了 SMITH（Schema-grounded Multi-task Iterative Tool Honing，基于Schema基础的多任务迭代工具打磨），这是一个将工具创建和工具使用统一在单一策略中的新型强化学习框架。SMITH通过专门的奖励轴评估模型，在程序推理、表格推理和分布外（out-of-domain）任务上展现出了最先进的性能。

Joint Optimization of Tool Creation and Use for Large Language Model Agents

arXiv: [2608.24571 [cs.AI]]
Submitted on: August 25, 2026
Authors: Zhi Rui Tam, Chieh-Yen Lin, Yun-Nung Chen, Shao-Hua Sun, Hung-yi Lee
Full-Text Links: View PDF | TeX Source

arXiv: [2608.24571 [cs.AI]]
Submitted on: August 25, 2026
Authors: Zhi Rui Tam, Chieh-Yen Lin, Yun-Nung Chen, Shao-Hua Sun, Hung-yi Lee
Full-Text Links: View PDF | TeX Source

📌 Executive Summary

Current tool-augmented Large Language Models (LLMs) are typically restricted to predefined APIs created by humans. While some systems attempt to bridge this gap by prompting a frozen LLM at inference time to generate tools dynamically, they suffer from a fundamental flaw: the tool-writing model and the tool-using model are decoupled, meaning the creator often generates schemas that it cannot reliably invoke.

To overcome this limitation, the authors introduce SMITH (Schema-grounded Multi-task Iterative Tool Honing), a novel reinforcement learning framework that unifies tool creation and tool use within a single policy. SMITH evaluates models across specialized reward axes, demonstrating state-of-the-art performance on procedural reasoning, tabular reasoning, and out-of-domain tasks.

📌 Executive Summary

Current tool-augmented Large Language Models (LLMs) are typically restricted to predefined APIs created by humans. While some systems attempt to bridge this gap by prompting a frozen LLM at inference time to generate tools dynamically, they suffer from a fundamental flaw: the tool-writing model and the tool-using model are decoupled, meaning the creator often generates schemas that it cannot reliably invoke.

To overcome this limitation, the authors introduce SMITH (Schema-grounded Multi-task Iterative Tool Honing), a novel reinforcement learning framework that unifies tool creation and tool use within a single policy. SMITH evaluates models across specialized reward axes, demonstrating state-of-the-art performance on procedural reasoning, tabular reasoning, and out-of-domain tasks.

🔍 Abstract & Key Innovations

  • The Problem: Inference-time tool generation relies on decoupled models where the schema generator lacks feedback regarding whether it can successfully execute the tools it creates.
  • The Solution (SMITH): Schema-grounded Multi-task Iterative Tool Honing is a reinforcement learning framework that trains both tool creation and tool use inside a unified policy.
  • Task Structure: Each rollout alternates between:
  • Build Tasks: Writing a tool given a few examples.
  • Use Tasks: Invoking a pooled tool to solve a held-out question.
  • Independent Reward Axes: Three distinct reward mechanisms independently evaluate schema, code, and outcome failures, providing targeted gradient updates for each failure mode.

🔍 Abstract & Key Innovations

  • The Problem: Inference-time tool generation relies on decoupled models where the schema generator lacks feedback regarding whether it can successfully execute the tools it creates.
  • The Solution (SMITH): Schema-grounded Multi-task Iterative Tool Honing is a reinforcement learning framework that trains both tool creation and tool use inside a unified policy.
  • Task Structure: Each rollout alternates between:
  • Build Tasks: Writing a tool given a few examples.
  • Use Tasks: Invoking a pooled tool to solve a held-out question.
  • Independent Reward Axes: Three distinct reward mechanisms independently evaluate schema, code, and outcome failures, providing targeted gradient updates for each failure mode.

📊 Performance & Results

When trained using the SMITH framework, a 4B Qwen3 model achieved remarkable results across multiple benchmarks:

  • Held-Out Tasks: Reached a 79.8 macro-average accuracy, outperforming all evaluated baseline methods—including an untrained 30B-A3B tool-writer.
  • TabMWP-Hard: Scored 40.4.
  • Out-of-Domain GQA: Scored 42.6 (+7.6 improvement over the best same-backbone inference-time baseline), achieving this without any visual or tabular training data.
  • Cross-Model Generalization: Tools created by the 4B SMITH-trained models successfully boosted the reasoning performance of other models, such as LFM-2.5-350M and Qwen3-30B-A3B, on the same tasks.

📊 Performance & Results

When trained using the SMITH framework, a 4B Qwen3 model achieved remarkable results across multiple benchmarks:

  • Held-Out Tasks: Reached a 79.8 macro-average accuracy, outperforming all evaluated baseline methods—including an untrained 30B-A3B tool-writer.
  • TabMWP-Hard: Scored 40.4.
  • Out-of-Domain GQA: Scored 42.6 (+7.6 improvement over the best same-backbone inference-time baseline), achieving this without any visual or tabular training data.
  • Cross-Model Generalization: Tools created by the 4B SMITH-trained models successfully boosted the reasoning performance of other models, such as LFM-2.5-350M and Qwen3-30B-A3B, on the same tasks.

🏷️ Metadata & Additional Information

🏷️ Metadata & Additional Information