跳转至

文章背景与核心概要

随着人工智能的发展，多模态模型越来越多地被部署用于与人类或其他人工智能体协同解决任务。然而，现实世界中的协作条件——例如时间压力、信息不对称以及不完美沟通——在以往的研究中通常都是孤立地进行考察的。为了填补这一空白，来自学术界的연구团队推出了 GPTNT，这是一个基于合作类视频游戏《保持通话和炸弹不炸》（Keep Talking and Nobody Explodes）构建的全新基准测试。

在该环境中，两个多模态智能体必须在实时的倒计时压力下异步协调，以拆除程序化生成的炸弹谜题：其中一个智能体拥有炸弹但没有拆除说明书，另一个智能体持有说明书但无法看到或操作炸弹。由于任何单一智能体都无法独自成功，高效的沟通变得至关重要。GPTNT 旨在评估真正的协作推理能力而非死记硬背，测试结果表明，与人类玩家相比，当前最先进的模型在实时协调方面面临着严峻的障碍。

GPTNT: Benchmarking Real-Time Collaboration Between Multimodal Agents on Keep Talking And Nobody Explodes

arXiv ID: 2606.28514v2
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Status: Accepted by TMLR on September 2, 2026
Project Website & Code: https://gptnt.github.io

📌 Summary

Multimodal models are increasingly deployed to solve tasks collaboratively with humans or other artificial agents. However, real-world collaboration conditions—such as time pressure, information asymmetry, and imperfect communication—have traditionally been studied only in isolation.

To address this gap, researchers introduce GPTNT, a novel benchmark built on the cooperative video game Keep Talking and Nobody Explodes. In this environment, two multimodal agents must coordinate asynchronously in real time to defuse procedurally generated bomb puzzles against a live countdown: * Agent 1 has access to the bomb, but not the instructions. * Agent 2 holds the instructions, but cannot see or manipulate the bomb.

Neither agent can succeed alone, making effective and efficient communication essential. The benchmark evaluates true collaborative reasoning rather than memorization, demonstrating that current state-of-the-art models face severe hurdles in real-time coordination compared to human players.

👥 Authors

  • Amit Parekh
  • Sabrina McCallum
  • Kareem Al-Hasan
  • Malvina Nikandrou
  • Alessandro Suglia
  • Ioannis Konstas

📋 Abstract

Multimodal models are increasingly deployed to solve tasks collaboratively with humans or other artificial agents. While existing benchmarks show that they possess the fundamental capabilities, the various conditions that coincide when collaborating—time pressure, information asymmetry, and imperfect communication—have traditionally been studied in isolation. To address this gap, we introduce GPTNT, a benchmark built on the cooperative video game Keep Talking and Nobody Explodes, in which two agents must coordinate to defuse procedurally generated bomb puzzles against a live countdown. One agent has access to the bomb but not the instructions for defusing it; the other holds the instructions but cannot see or manipulate the bomb. Neither agent can succeed alone: the task requires contributions from both, and is solvable only through effective, efficient communication. We remove turn-taking proxies or simplifications, instead requiring agents to act asynchronously and communicate in real time. GPTNT is designed to expose how models collaborate versus how they perform alone: the instruction manual, the partner, or both, can optionally be withheld to surface what a model has memorised versus what it derives in the moment. We demonstrate that GPTNT poses a considerable challenge to the state-of-the-art: not one of the closed- and open-source models we test defuses a single bomb in real time, a bar that human players clear. In a range of controlled experiments, we explore where capabilities break down, identifying critical weaknesses in state tracking, efficient acting within the time budget, handling ambiguity, and error recovery. Since it runs on the real game, GPTNT benefits from procedural generation and inherits a living modding community: as models improve, the benchmark can be evolved to remain challenging, rather than being solved once and retired.

多模态模型正日益被部署用于与人类或其他人工智能体协作解决任务。虽然现有的基准测试表明它们具备基础能力，但协作时同时出现的各种条件——时间压力、信息不对称和不完美的沟通——在以往通常是被孤立研究的。为了填补这一空白，我们推出了 GPTNT，这是一个基于合作视频游戏《保持通话和炸弹不炸》（Keep Talking and Nobody Explodes）构建的基准测试。在这个游戏中，两个智能体必须在实时的倒计时压力下进行协调，以拆除程序化生成的炸弹谜题。其中一个智能体可以接触炸弹但没有拆弹说明书；另一个智能体持有说明书，但看不见也无法操作炸弹。任何一个智能体都无法独自成功：该任务需要双方的共同贡献，且只有通过有效、高效的沟通才能解决。我们去除了轮流发言的代理或简化机制，转而要求智能体异步行动并进行实时沟通。GPTNT 旨在揭示模型在协作时的表现与单独表现之间的差异：可以选择性地扣留说明书、搭档或两者兼而有之，从而揭示模型死记硬背了什么与它在当下推导出了什么。我们证明了 GPTNT 对最先进技术构成了巨大挑战：在我们测试的闭源和开源模型中，没有一个模型能在实时情况下成功拆除哪怕一个炸弹，而人类玩家可以轻易跨过这一门槛。在一系列受控实验中，我们探讨了模型能力崩溃的环节，识别出它们在状态跟踪、在时间预算内高效行动、处理歧义以及错误恢复方面的关键弱点。由于 GPTNT 运行在真实游戏之上，它受益于程序化生成并继承了一个活跃的模组（modding）社区：随着模型的改进，该基准可以不断演进以保持挑战性，而不会被一次性攻克后就退役。

🛠️ Key Features of the Benchmark

  • Real-Time & Asynchronous: Eliminates turn-taking proxies, forcing agents to process incoming information and act concurrently under strict temporal constraints.
  • Information Asymmetry: Splits critical information between the two agents, preventing single-agent shortcuts.
  • Isolating Memorization vs. Derivation: Allows researchers to optionally withhold the instruction manual, the partner, or both, to distinguish true in-the-moment reasoning from memorized knowledge.

  • Future-Proof Design: Leveraging the real video game and its active modding community, GPTNT utilizes procedural generation to continuously evolve and remain challenging as AI capabilities advance.

  • 实时与异步： 消除了轮流发言的简化代理，迫使智能体在严格的时间限制下同时处理传入信息并采取行动。

  • 信息不对称： 将关键信息分配给两个智能体，防止出现单智能体捷径。
  • 隔离记忆与实时推导： 允许研究人员选择性地扣留说明书、搭档或两者，以区分真正的即时推理与死记硬背的知识。
  • 面向未来的设计： 借助真实的视频游戏及其活跃的模组社区，GPTNT 利用程序化生成技术实现持续演进，随着人工智能能力的提升而保持挑战性。

🔍 Key Findings

  • Performance Gap: None of the tested open- and closed-source models successfully defused a single bomb in real time—a milestone routinely cleared by human players.
  • Identified Weaknesses: Controlled experiments revealed critical system breakdowns in:
  • State tracking
  • Efficient action execution within tight time budgets
  • Handling communication ambiguity

  • Error recovery

  • 性能差距： 在测试的开源和闭源模型中，没有任何一个模型能够在实时状态下成功拆除一个炸弹——人类玩家通常能轻松达成这一目标。

  • 发现的弱点： 受控实验揭示了模型在以下方面存在严重的系统崩溃：
  • 状态跟踪
  • 在紧迫的时间预算内高效执行动作
  • 处理沟通歧义
  • 错误恢复

📅 Submission History

  • [v1] Fri, 26 Jun 2026 18:09:36 UTC (37,637 KB)

  • [v2] Fri, 4 Sep 2026 15:37:02 UTC (37,425 KB) — This version

  • [v1] 2026年6月26日 星期五 18:09:36 UTC (37,637 KB)

  • [v2] 2026年9月4日 星期五 15:37:02 UTC (37,425 KB) — 当前版本