跳转至

TestifAI：基于层析成像的深度学习系统测试方法

文章背景与核心概要

随着深度学习系统在自动驾驶等安全关键领域的部署日益增加，确保其面对现实世界扰动（如图像模糊、亮度变化和缩放）时的鲁棒性变得至关重要。传统的测试框架需要进行数千次模型推理，并且由于组合爆炸问题，难以高效地评估多种扰动组合的效果。

为了解决这一痛点，研究人员提出了 TestifAI，这是一个创新的深度学习测试框架，它利用了局部模型层析成像（partial model tomography）技术。通过仅利用低阶（1阶和2阶）扰动观测值来预测高阶（3阶和4阶扰动）测试结果，TestifAI 在将总体鲁棒性估计误差控制在 7% 以下的同时，将所需的模型推理次数减少了 60% 至 80%。

TestifAI: Tomography-Based Testing for Deep Learning Systems

TestifAI：基于层析成像的深度学习系统测试方法

Summary

摘要

随着深度学习系统越来越多地被部署在安全关键的应用领域（例如自动驾驶），相关的风险也在不断增加。因此，支撑现代AI系统的深度学习模型必须经过彻底的测试，以确保其行为正确。单次鲁棒性测试涉及数千次推理，以凭经验验证模型的输出在输入的有界扰动下是否保持稳定。然而，现有的测试框架缺乏系统探索和总结扰动组合空间中鲁棒性的手段。

As deep learning systems are increasingly deployed in safety-critical domains like autonomous driving, ensuring their robustness against real-world perturbations (such as image blur, brightness, and zoom) becomes paramount. Traditional testing frameworks require thousands of model inferences and struggle to efficiently evaluate combinations of perturbations due to combinatorial explosion.

To solve this, researchers propose TestifAI, an innovative deep learning testing framework that utilizes partial model tomography. By predicting higher-order (3 and 4-perturbation) test outcomes using only low-order (1 and 2-perturbation) observations, TestifAI reduces the number of required model inferences by 60–80% while maintaining an aggregate robustness estimation error of less than 7%.

我们提出了 TestifAI，这是一个用于高效、准确评估模型在多种扰动组合下鲁棒性的深度学习测试框架。TestifAI 允许用户将操作条件指定为结构化的语义输入扰动空间（例如图像模糊、亮度和缩放）以及离散的严重程度级别（例如低、中和高）。用户可以查询任何组合下的模型鲁棒性（例如：“低模糊、高亮度和中缩放”）。

As AI systems are increasingly deployed in safety-critical application domains (e.g., autonomous driving), associated risks increase too. Deep learning models underlying modern AI systems, therefore, must undergo thorough testing to ensure their correct behaviour. A single robustness test involves thousands of inferences to empirically verify if a model's outputs remain stable under a bounded perturbation of its inputs. However, existing testing frameworks lack the means to systematically explore and summarise robustness across a combinatorial space of perturbations.

We propose TestifAI, a deep learning testing framework for efficient and accurate estimation of robustness against combinations of perturbations. TestifAI enables users to specify operational conditions as structured spaces of semantic input perturbations (e.g., image blur, brightness and zoom) and discrete severity levels (e.g., low, medium and high). Users can query model robustness for any combination (e.g., "low blur, high brightness, and medium zoom").

为了实现高效和准确，TestifAI 引入了局部模型层析成像（partial model tomography），这是一种新颖的方法，它通过仅应用少量扰动（低阶投影）的测试来重建多扰动空间中的模型行为。为了估计对至少三种扰动的鲁棒性，TestifAI 仅根据涉及多达两种扰动的测试结果来训练辅助模型，从而避免了执行指数级数量的测试。我们在五个图像和语言分类任务上的实验表明，TestifAI 可以从低阶（1阶和2阶）观测值中预测高阶（3阶和4阶扰动）测试结果，总体鲁棒性估计误差低于 7%，同时将推理次数减少了 60% 至 80%。

To achieve efficiency and accuracy, TestifAI introduces partial model tomography, a novel approach to reconstructing model behaviour in a multi-perturbation space from tests that apply only a small number of perturbations (lower-order projections). To estimate robustness against at least three perturbations, TestifAI trains an auxiliary model on the results of tests involving up to two perturbations only, avoiding execution of an exponential number of tests. Our experiments on five image and language classification tasks show that TestifAI can predict higher-order (3 and 4 perturbations) test outcomes from low-order (1 and 2 perturbations) observations with an aggregate robustness estimation error of less than 7%, while reducing the number of inferences by 60–80%.

Paper Metadata

论文元数据

  • arXiv ID: arXiv:2608.18900 [cs.AI]
  • 主学科领域: 计算机科学 > 人工智能 (cs.AI)
  • 作者: Arooj Arif, Tobias Hartung, Elena Botoeva, Alexandros Koliousis
  • 提交历史:
  • [v1] 2026年8月19日 星期三
  • [v2] 2026年8月20日 星期四 (当前版本)
  • 许可证: 知识共享署名 4.0 国际许可协议 license icon
  • arXiv ID: arXiv:2608.18900 [cs.AI]
  • Primary Subject: Computer Science > Artificial Intelligence (cs.AI)
  • Authors: Arooj Arif, Tobias Hartung, Elena Botoeva, Alexandros Koliousis
  • Submission History:
  • [v1] Wed, 19 Aug 2026
  • [v2] Thu, 20 Aug 2026 (This version)
  • License: Creative Commons Attribution 4.0 International license icon

快速链接与资源