跳转至

文章背景与核心概要

本文针对当前 AI 模型安全评估方式中的一个关键局限性展开了研究。传统的评估指标（如 F1 分数）通常只统计扫描器成功给出安全判定结果的样本情况，作者认为这种方法忽视了判定结果的“可用性”——具体而言，即扫描器在面对故障或不完整分析时的处理表现。

通过对 ModelScan、ModelAudit 和 Fickling 这三个知名工具进行测试（基准测试包含 170 个受控的 Pickle 和 PyTorch 制品），研究揭示了工具在可靠性上的巨大差异。研究结果表明，尽管某些工具在成功给出结果时能保持较高的准确率，但它们往往无法完成对相当一部分恶意制品的分析。作者总结认为，未来的评估工作必须将判定准确率与判定可用性解耦，以便更真实地反映其在实际场景中的安全效能。

Beyond F1: Evaluating Coverage and Failure Recovery in AI Model Security Scanners

Authors: Qianlong Lan, Vinothini Pandurangan, Anuj Kaul, Indranil Sanyal
Date: August 27, 2026
Subject: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
arXiv ID: 2608.27424

Beyond F1: Evaluating Coverage and Failure Recovery in AI Model Security Scanners

Authors: Qianlong Lan, Vinothini Pandurangan, Anuj Kaul, Indranil Sanyal
Date: August 27, 2026
Subject: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
arXiv ID: 2608.27424

Summary

本文探讨了当前 AI 模型安全扫描器评估方式中的一个重大局限性。传统的评估指标（如 F1 分数）仅统计扫描器成功提供安全判定的实例。作者认为，这种方法忽略了判定结果的“可用性”——特别是扫描器如何处理故障或不完整的分析。

通过对三个主流工具（ModelScan、ModelAudit 和 Fickling）进行测试，并使用包含 170 个 Pickle 和 PyTorch 制品的受控基准进行评估，本研究揭示了工具在可靠性方面的显著差异。研究结果表明，虽然某些工具在提供结果时可能达到高准确率，但它们往往无法对很大一部分恶意制品完成分析。作者得出结论，未来的评估必须将判定准确率与判定可用性解耦，以更好地反映现实世界的安全效能。

Summary

This paper addresses a critical limitation in how AI model security scanners are currently evaluated. Traditional metrics, such as the F1 score, only account for instances where a scanner successfully provides a security judgment. The authors argue that this approach ignores the "availability" of judgments—specifically, how scanners handle failures or incomplete analyses.

By testing three prominent tools—ModelScan, ModelAudit, and Fickling—against a controlled benchmark of 170 Pickle and PyTorch artifacts, the study reveals significant disparities in tool reliability. The findings demonstrate that while some tools may achieve high accuracy when they do provide a result, they often fail to complete the analysis for a large portion of malicious artifacts. The authors conclude that future evaluations must decouple judgment accuracy from judgment availability to better reflect real-world security efficacy.

Key Findings

  • 评估方法： 本研究引入了一个严格的框架，区分了非 N/A 覆盖率、分析完整性、确切的安全决策以及不受支持的结果。
  • 性能差异：
    • ModelAudit 对 135 个已标记家族中的 100% 提供了确切的安全决策。
    • Fickling 提供了 81.5%（110 个家族）的决策。
    • ModelScan 仅提供了 49.6%（67 个家族）的决策。
  • 准确率与可用性： 当 ModelScan 确实生成判定结果时，它达到了 100% 的精确率、召回率和 F1 分数。然而，对于已被 ModelAudit 和 Fickling 成功识别的 48 个恶意家族，它未能完成分析。
  • 冗余性： Fickling 没有提供任何 ModelAudit 和 ModelScan 组合能力之外的独特真阳性（true-positive）检测结果。

Key Findings

  • Evaluation Methodology: The study introduces a rigorous framework that distinguishes between non-N/A coverage, analysis completion, definitive security decisions, and unsupported outcomes.
  • Performance Discrepancies:
    • ModelAudit provided definitive security decisions for 100% of the 135 labeled families.
    • Fickling provided decisions for 81.5% (110 families).
    • ModelScan provided decisions for only 49.6% (67 families).
  • Accuracy vs. Availability: When ModelScan did produce a judgment, it achieved 100% precision, recall, and F1. However, it failed to complete the analysis for 48 malicious families that were successfully identified by ModelAudit and Fickling.
  • Redundancy: Fickling provided no unique true-positive detections that were not already covered by the combined capabilities of ModelAudit and ModelScan.

Access & Resources

Access & Resources

Citation & Metadata

Citation & Metadata