跳转至

文章背景与核心概要

大模型（LLM）智能体的发展依赖于“技能（Skills）”机制来为其提供可复用的规划、工具使用和验证指导。然而，实际应用中这些技能的效果褒贬不一，有时不仅不能提升性能，反而会导致性能下降或成本增加。本文首次对这种“技能引发的故障（skill-induced failures）”进行了深入的实证研究。

研究团队开发了一个差分分析框架，并将其应用于 SkillsBench 和 SWE-Skills-Bench 基准测试中，共识别出 307 起由技能引发的故障（包含 125 起功能性故障和 182 起效率退化）。为了系统地分析这些问题，他们还推出了基于分类学的归因工具 SkillTriage。研究发现，故障往往不是由明显无关的技能引起的，而是由表面上相关的技能强加了僵化、强制性的程序（例如过度的验证或沉重的实现流程），从而干扰了任务的高效执行。这项工作为构建更安全、更具成本意识的智能体技能复用机制指明了方向。

Agent Skills Can Be Harmful: An Empirical Study of Skill-Induced Failures in LLM Agents

arXiv ID: arXiv:2608.11888 [cs.AI]
Submitted: August 12, 2026
Authors: Gen Dong, Yanjie Gao, Liqun Li, Tianyin Xu, Yu Hua, Fan Yang

📋 Summary

While agent skills serve as the standard mechanism to provide Large Language Model (LLM) agents with reusable guidance for planning, tool usage, and validation, their actual impact has proven mixed. This empirical study investigates skill-induced failures—situations where loaded skills degrade performance or cause cost regressions rather than improving them.

Using a newly developed differential analysis framework applied across SkillsBench and SWE-Skills-Bench, the researchers identified 307 skill-induced failures (125 functional failures and 182 efficiency regressions). To systematically analyze these issues, they also introduced SkillTriage, a taxonomy-guided attribution tool. The study highlights that failures rarely stem from obviously irrelevant skills, but rather from seemingly relevant skills imposing rigid, mandatory procedures (such as excessive verification or heavy implementation pipelines) that interfere with effective task execution.

📌 Metadata

🔍 Abstract

Agent skills are the de facto mechanism for extending LLM agents with reusable guidance. A skill can shape the agent's task execution, including planning, tool use, problem-solving, and validation. Prior work reported mixed results of agent skills: some skills improve task success rates, while others have no effect, increase token use and execution time, and even reduce success rates. This paper presents a comprehensive analysis of skill-induced agent failures by attributing task failures and cost regressions to specific loaded skills. We introduce a differential analysis framework that attributes a failure or regression to a skill by comparing a target skill-guided run against a no-skill or semantically matched skill reference run that solves the same task, or solves it more cheaply. We instantiate this framework on SkillsBench and SWE-Skills-Bench, yielding 307 skill-induced failures, including 125 functional failures and 182 efficiency regressions. We also build SkillTriage, a taxonomy-guided attribution tool that normalizes paired cases, extracts differential evidence, and produces triage reports. Our major findings include: (1) Skill induced functional failures are rarely caused by obviously irrelevant skills; instead, seemingly relevant skills often make the agent incorrectly implement or omit task-required implementation elements. (2) Skill-induced efficiency regressions are not explained by prompt length alone. (3) The largest sources within Excessive Procedure are excessive verification and heavy implementation pipelines, contributing 67 and 30 cases, respectively. This shows that skills often turn validation checklists and construction recipes into mandatory work. Based on our findings, we propose research topics and tooling improvements for safer and more cost-aware skill reuse.

🛠️ Key Contributions & Findings

  1. Differential Analysis Framework: A novel methodology comparing skill-guided runs against control runs (no-skill or alternative skill references) to directly attribute execution regressions to specific loaded skills.
  2. Comprehensive Dataset of Failures: Instantiation on SkillsBench and SWE-Skills-Bench uncovering 307 concrete instances of skill-induced degradation (125 functional failures, 182 efficiency regressions).
  3. SkillTriage Tool: A taxonomy-driven attribution utility designed to normalize paired execution runs, extract differential evidence, and output detailed triage reports.
  4. Key Insights:
  5. Relevance Paradox: Functional failures are rarely caused by irrelevant skills; rather, seemingly relevant skills often mislead agents into incorrect implementations or omissions.
  6. Cost Beyond Prompts: Efficiency regressions cannot be explained solely by increased prompt length.
  7. Excessive Procedure Overhead: Rigid validation checklists and heavy implementation pipelines (contributing 67 and 30 cases respectively) force agents into unnecessary, mandatory computational work.

🔗 Full-Text & Resources