跳转至

文章背景与核心概要

尽管以 GPT 和 Claude 为代表的现代基础模型在解决奥林匹克级别的几何问题时表现卓越，但“解题”与“准确可视化”其实是两种截然不同的能力。当前的数学推理基准主要评估 AI 模型得出正确数字或符号答案的能力，却忽视了构建几何图形本身的独特技能。为了填补这一空白，本文作者引入了一个全新的开源基准和数据集，包含 954 个独立的奥数几何问题（其中包括 297 个高难度子集）。每个问题都配有详细的解答以及由人工编写、高保真且可渲染的 Asymptote 代码几何图。

评测结果揭示了一个明显的局限性：基础模型在图示推理方面表现挣扎，平均编译成功率仅为 36.14%。这充分证明了强大的符号与逻辑推理能力并不能自然转化为准确的视觉表征能力。该研究为多模态大模型的视觉具象化能力提供了全新的评测视角和高质量基准。

解题不等于画图：奥数几何图示推理基准

作者： Hsien Xin Peng, Anthony Kim, Alvin Li, Calvin Supasanya, Shivank Garg, Kevin Zhu
研究领域： 人工智能 (cs.AI)
会议： ICML 2026, AI4MATH Workshop
ArXiv ID： arXiv:2608.18111 [cs.AI]
资源： Hugging Face 数据集 | 查看 PDF

📌 执行摘要

While modern foundation models like GPT and Claude excel at solving Olympiad-level geometry problems, solving a problem and accurately visualizing it are two fundamentally different skills. Current mathematical reasoning benchmarks evaluate whether an AI model can reach the correct numerical or symbolic answer, but they overlook the distinct ability to construct the geometric diagram itself.

To bridge this gap, the authors introduce a new open-source benchmark and dataset comprising 954 self-contained Olympiad geometry problems (including a 297-problem hard subset). Each problem is paired with its solution and a human-authored, high-fidelity diagram in renderable Asymptote code. Evaluation reveals a stark limitation: foundation models struggle significantly with diagrammatic reasoning, achieving an average compile success rate of only 36.14%, demonstrating that strong symbolic and logical reasoning does not inherently translate into accurate visual representation.

尽管以 GPT 和 Claude 为代表的现代基础模型在解决奥林匹克级别的几何问题时表现卓越，但“解题”与“准确可视化”其实是两种截然不同的能力。当前的数学推理基准主要评估 AI 模型能否得出正确的数字或符号答案，却忽视了构建几何图形本身的独特技能。

为了填补这一空白，作者引入了一个包含 954个独立奥数几何问题（其中包括 297个高难度问题子集）的全新开源基准和数据集。每个问题都配有解答以及用可渲染的 Asymptote 代码编写的人工高保真图解。评测结果揭示了一个明显的局限性：基础模型在图示推理方面举步维艰，平均编译成功率仅为 36.14%，这证明了强大的符号与逻辑推理并不能自然地转化为准确的视觉表征。

📖 摘要

Foundation models such as GPT and Claude now solve olympiad-level mathematics with remarkable proficiency, so much so that geometry problem solving has become a standard proxy for their mathematical reasoning. Yet solving a geometry problem and drawing the figure it depends on are not the same skill: progress often hinges on a faithful diagram with the right auxiliary constructions and incidences, and it is unclear that a model which reasons its way to the answer can also produce one.

A growing collection of benchmarks, including MathVista, and MathVerse, measures whether models reach the correct answer, but to our knowledge, none isolate the distinct ability to construct the diagram itself, leaving this capability unmeasured. We introduce an open-source benchmark that targets this gap: 954 self-contained olympiad geometry problems, with a 297-problem hard subset, each paired with its solution and a human-authored, high-fidelity diagram in renderable Asymptote code, together with a suite of text-, code-, image-, VLM-, and constraint-based metrics for what we term diagrammatic reasoning.

Evaluating current foundation models reveals a pronounced gap between solving and drawing: their diagrams are markedly less faithful, with an average compile success rate of only 36.14%. Strong mathematical reasoning, we find, does not imply the ability to construct accurate geometric diagrams.

诸如 GPT 和 Claude 等基础模型现在已经能够以极高的熟 proficiency 解决奥林匹克级别的数学题，甚至几何解题已成为衡量其数学推理能力的标准代理指标。然而，解决几何问题和绘制其依赖的图形并不是同一种技能：解题的突破往往取决于具有正确辅助线和关联性的忠实图解，目前尚不清楚一个通过推理得出答案的模型是否也能画出这样的图。

尽管越来越多的基准（包括 MathVista 和 MathVerse）在评估模型是否能得出正确答案，但据我们所知，没有一个基准能够将构建图解本身的独特能力独立出来，导致这一能力长期未被量化。为此，我们推出了一个填补该空白的开源基准：包含 954 个独立的奥数几何问题（其中包含一个 297 个难题的高难度子集），每个问题均配有其解答以及用可渲染的 Asymptote 代码编写的人工高保真图解，并配备了一套针对我们所谓的“图示推理”的文本、代码、图像、VLM（视觉语言模型）以及基于约束的评估指标。

对当前基础模型的评测揭示了解题与画图之间显著的鸿沟：它们的图解明显缺乏保真度，平均编译成功率仅为 36.14%。我们发现，强大的数学推理能力并不意味着具备构建准确几何图形的能力。

📊 数据集与基准亮点

  • 总题目数： 954 个独立的奥数级几何问题。
  • 高难度子集： 297 个极具挑战性的高难度问题，用于对先进模型进行压力测试。
  • 图解格式： 以可渲染的 Asymptote 代码编写的高保真、人工创作的图解。
  • 评估指标： 一套综合评估体系，涵盖文本、代码、图像、视觉语言模型（VLM）性能以及几何约束。
  • Total Problems: 954 self-contained Olympiad-level geometry problems.
  • Hard Subset: 297 exceptionally rigorous problems for stress-testing advanced models.
  • Diagram Format: High-fidelity, human-authored diagrams written in renderable Asymptote code.
  • Evaluation Metrics: A comprehensive suite measuring text, code, images, Vision-Language Model (VLM) performance, and geometric constraints.

🔗 链接与资源

(根据说明保留包含的资产：license icon)