跳转至

结构化但脆弱：论大语言模型在网络安全决策中的局限性

文章背景与核心概要

随着大语言模型（LLM）在网络安全工作流中的应用日益广泛，一个根本性的问题始终存在：它们究竟是在执行结构化的安全推理，还是仅仅依赖表面线索和先验知识？本文由 Pasquale Malacaria 和 Yunxiao Zhang 撰写，通过考察大语言模型在基于真实世界威胁场景构建的攻击图上的防御选择表现，深入探讨了这一问题。

研究表明，尽管大语言模型在提供显式结构时表现出“条件胜任力”（conditional competence），但其能力最终是脆弱的，对提示词的框架极其敏感，且在生成优化求解器时面临可扩展性限制。这些发现对未来 AI 辅助安全决策支持系统的设计、测试和部署具有重要启示，凸显了在生成式 AI 工具旁边持续引入严格验证层的必要性。

执行摘要 (Executive Summary)

随着大语言模型（LLM）在网络安全工作流中的应用日益广泛，一个根本性的问题始终存在：它们究竟是在执行结构化的安全推理，还是仅仅依赖表面线索和先验知识？

这篇由 Pasquale Malacaria 和 Yunxiao Zhang 撰写的论文通过考察大语言模型在基于真实世界威胁场景构建的攻击图上的防御选择表现，深入探讨了这一问题。研究表明，尽管大语言模型在提供显式结构时表现出条件胜任力，但其能力最终是脆弱的，对提示词框架极度敏感，并且在被分配生成优化求解器任务时会遇到可扩展性瓶颈。

As Large Language Models (LLMs) find increasing adoption in cybersecurity workflows, a fundamental question persists: Do they genuinely perform structured security reasoning, or do they merely rely on superficial cues and prior knowledge?

This paper—authored by Pasquale Malacaria and Yunxiao Zhang—investigates this question by examining LLM defense selection across attack graphs derived from real-world threat scenarios. The study reveals that while LLMs exhibit conditional competence when explicit structures are provided, their capabilities are ultimately fragile, heavily sensitive to framing, and prone to scalability limits when tasked with generating optimization solvers.

论文元数据 (Paper Metadata)

  • arXiv 标识符: arXiv:2608.20966 [cs.CR]
  • 一级学科: 密码学与安全 (cs.CR)
  • 二级学科: 人工智能 (cs.AI)
  • 提交日期: 2026年8月21日
  • 篇幅: 31 页，10 张图表
  • DOI: 10.48550/arXiv.2608.20966
  • 许可证: 知识共享署名 4.0 license icon

研究范围与方法 (Research Scope & Methodology)

研究人员在复杂攻击图的预算约束下，通过防御选择问题测试了大语言模型的决策能力。评估涵盖了多样化的真实世界威胁场景： * 勒索软件攻击 * 供应链攻击 * 云滥用场景 * Kubernetes 入侵 * 销售终端（POS）恶意软件 * 工业控制系统 / 运营技术（ICS/OT）入侵

为了建立基准，作者将大语言模型生成的策略与作为结构化安全推理规范参考的博弈论优化基准进行了对比。

The researchers tested LLM decision-making capabilities using defense-selection problems under budget constraints over complex attack graphs. The evaluation covered diverse, real-world threat scenarios: * Ransomware attacks * Supply-chain compromises * Cloud abuse scenarios * Kubernetes intrusions * Point-of-Sale (POS) malware * Industrial Control Systems / Operational Technology (ICS/OT) intrusions

To establish a baseline, the authors compared LLM-generated strategies against a game-theoretic optimization baseline acting as a normative reference for structured security reasoning.

核心发现 (Key Findings)

  1. 条件胜任力： 当提供显式攻击图结构时，大语言模型经常能够生成与博弈论优化基准高度接近的连贯防御策略。
  2. 极端脆弱性： 随着图复杂度的增加，大语言模型的推理能力会迅速退化。此外，它们的评估对框架效应高度敏感：
  3. 微小的提示词修改会大幅改变策略排名。
  4. 仅仅将一个明显较差的安全策略重新贴上“最优”标签，就会急剧提升大语言模型对其的评价。
  5. 非单调风险判断： 研究发现，形式化密码学/系统风险与大语言模型判断之间存在非单调关系；客观上最接近数学最优解的策略并未持续获得大语言模型评估器赋予的最高排名。
  6. 求解器生成限制： 当被提示为防御选择问题编写自定义求解器时，大语言模型成功恢复了高层表述，但其生成的实现的扩展性能明显比专用优化求解器更差。
  1. Conditional Competence: When provided with explicit attack-graph structures, LLMs frequently produce coherent defense strategies that closely approximate the game-theoretic optimization baseline.
  2. Extreme Fragility: LLM reasoning degrades rapidly as graph complexity increases. Furthermore, their evaluations are highly susceptible to framing effects:
  3. Minor prompt alterations substantially shift strategy rankings.
  4. Merely relabeling a demonstrably poor security strategy as "optimal" drastically inflates how favorably the LLM evaluates it.
  5. Non-Monotonic Risk Judgment: The study uncovered a non-monotonic relationship between formal cryptographic/system risk and LLM judgment; strategies objectively closest to the mathematical optimum were not consistently ranked highest by LLM evaluators.
  6. Solver Generation Limits: When prompted to write custom solvers for the defense-selection problem, LLMs successfully recovered high-level formulations, but the generated implementations scaled significantly worse than purpose-built optimization solvers.

结论与启示 (Conclusion & Implications)

这些研究结果表明，尽管大语言模型能够在严格控制的表示形式下近似模拟结构化的网络安全推理，但它们无法稳健地应用这种推理。

这些深刻见解对未来 AI 辅助的安全决策支持系统的设计、测试和部署具有重要意义，并突显了在生成式 AI 工具旁边持续配备严格验证层的必要性。

The findings demonstrate that while LLMs can approximate structured cybersecurity reasoning under tightly controlled representations, they do not apply it robustly.

These insights carry significant implications for the future design, testing, and deployment of AI-assisted security decision-support systems, highlighting the ongoing necessity for rigorous verification layers alongside generative AI tools.