跳转至

文章背景与核心概要

随着计算机科学顶级会议将制品评估（AE）作为确保研究可复现性的标准流程，当前的评估方法主要聚焦于功能性而非安全性。然而，公开公开发布研究制品（尤其是在安全领域）可能会无意中引入安全风险、产生被滥用的机会，并给负责任的研究共享带来挑战。

本文对 2023 年至 2025 年间顶级安全会议上发表的 1,388 个研究制品进行了深入研究。通过静态分析，作者识别出 132,431 个候选安全发现，表明近 45% 的被审查发现具有合理的、上下文相关的安全风险。为了解决这一问题，作者推出了 SAFE（安全意识制品评估框架，Security-Aware Framework for Artifact Evaluation），这是一个自主框架，旨在根据代码语义、执行上下文和实际可利用性来评估工具报告的安全发现，为实现具备安全意识的研究制品评估开辟了道路。

Beyond Reproducibility: Towards Security-Aware Evaluation of Research Artifacts

arXiv: [2605.06508 [cs.CR]]
Primary Subject: Cryptography and Security (cs.CR)
Other Subjects: Artificial Intelligence (cs.AI)
Authors: Nanda Rani, Christian Rossow
Submission History: Submitted on May 7, 2026; last revised September 2, 2026.
Links: View PDF | HTML Version | Source Code (GitHub)

📌 Executive Summary

📌 Executive Summary

While artifact evaluation (AE) has become a standard practice at leading computer science conferences to ensure research reproducibility, current evaluation methods primarily focus on functionality rather than safety. Publicly releasing research artifacts—particularly in security domains—can unintentionally introduce security risks, create opportunities for misuse, and complicate responsible research sharing.

This paper investigates 1,388 research artifacts published between 2023 and 2025 across top-tier security conferences. Through static analysis, the authors identified 132,431 candidate security findings, demonstrating that nearly 45% of reviewed findings carry plausible, context-dependent security risks. To address this, the authors introduce SAFE (Security-Aware Framework for Artifact Evaluation), an autonomous framework designed to evaluate tool-reported findings based on code semantics, execution contexts, and practical exploitability, establishing a path forward for security-aware artifact evaluations.

📖 Abstract

研究制品被广泛共享以支持可复现性，制品评估（AE）也已成为许多顶级会议的常见做法。然而，AE 主要检查制品是否能按预期运行并被复现，其目的并非发现或减轻潜在的安全风险。由于这些制品是公开释放和复用的，它们可能会无意中为滥用创造机会，并引发对安全且负责任共享的担忧。

📖 Abstract

Research artifacts are widely shared to support reproducibility, and artifact evaluation (AE) has become common at many leading conferences. However, AE mainly checks whether artifacts work as claimed and can be reproduced. It does not aim at spotting or mitigating potential security risks. Since these artifacts are publicly released and reused, they may unintentionally create opportunities for misuse and raise concerns about safe and responsible sharing.

我们研究了 2023 年至 2025 年间在四大安全会议上发表的 1,388 个研究制品，执行了静态分析，并获得了 132,431 个候选安全发现。我们提出了一种用于上下文感知安全评估的分类法，并对这些发现进行了检查，以过滤误报并识别代表合理的、上下文相关的安全风险的发现。我们发现，44.80% 的被审查发具有安全相关性。

We study 1,388 research artifacts published between 2023 and 2025 at the top-4 security conferences, perform static analysis, and obtain 132,431 candidate security findings. We propose a taxonomy for context-aware security assessment and examine the findings to filter false positives and identify findings that represent plausible context-dependent security risks. We find that 44.80% of the reviewed findings are security-relevant.

为了支持可扩展的分析，我们提出了 SAFE（安全意识制品评估框架，Security-Aware Framework for Artifact Evaluation），这是一个自主框架，可根据代码语义、执行上下文和实际可利用性评估工具报告的发现。SAFE 实现了： * 在区分安全相关发现与非安全发现方面，达到了 94.40% 的准确率 和 93.60% 的 F1 分数。 * 在对安全风险类型进行分类方面，达到了 92.40% 的准确率 和 81.10% 的 F1 分数。

To support scalable analysis, we present SAFE (Security-Aware Framework for Artifact Evaluation), an autonomous framework that assesses tool-reported findings based on code semantics, execution context, and practical exploitability. SAFE achieves: * 94.40% accuracy and a 93.60% F1-score in distinguishing security-relevant from non-security findings. * 92.40% accuracy and an 81.10% F1-score in classifying security-risk types.

总体而言，我们的结果表明，上下文感知的安全评估是对现有 AE 流程的实用补充，能够支持更安全、更负责任的研究制品共享。

Overall, our results show that context-aware security assessment is a practical complement to existing AE processes and can support safer and more responsible research artifact sharing.

🛠️ Framework and Resources

🛠️ Framework and Resources