跳转至

扩散语言模型中的原位指令遵循

文章背景与核心概要

扩散大语言模型（dLLMs）通过双向迭代去噪生成文本，使其天然适用于原位提示（In-place Prompting, IPP）——一种将用户指定的约束锚定在任意输出位置的范式。本文将这一能力正式定义为原位指令遵循（In-place Instruction Following, IIF）任务，并推出了 IIF-Bench，这是一个涵盖字面、风格和语篇功能约束以及基于评分标准的评估协议的分层基准测试。

注意力偏差探针显示，原生 dLLMs 在去噪过程中往往低估了约束区间的优先级。为了解决这一问题，作者提出了 GRAFT，这是一个面向 IPP 的后训练框架，它集成了约束感知的监督微调（SFT）和偏好优化。在四个具有代表性的 dLLMs 上，GRAFT 将平均 IIF 分数从 57.75 提升至 73.10（+15.35 分），在字面和语篇功能约束上分别显著提升了 15.91 和 15.57 个绝对百分点，同时成功保留了通用的文本生成能力。

📋 Summary

Diffusion Large Language Models (dLLMs) generate text through bidirectional iterative denoising, making them naturally suited for In-place Prompting (IPP)—a paradigm where user-specified constraints are anchored at arbitrary output positions. This paper formalizes this capability as the In-place Instruction Following (IIF) task and introduces IIF-Bench, a hierarchical benchmark covering literal, style, and discourse-function constraints alongside a rubric-based evaluation protocol.

An attention-bias probe reveals that vanilla dLLMs tend to under-prioritize constraint spans during the denoising process. To address this, the authors propose GRAFT, an IPP-oriented post-training framework that integrates constraint-aware Supervised Fine-Tuning (SFT) and preference optimization. Across four representative dLLMs, GRAFT increases the average IIF score from 57.75 to 73.10 (+15.35 points)—notably achieving absolute gains of 15.91 and 15.57 points on literal and discourse-function constraints—while successfully preserving general generation capabilities.

📌 Paper Metadata

Field Details
Title In-Place Instruction Following in Diffusion Language Models
Authors Zheng Nie, Zherui Li, Jiaming Zhang, Kun Wang, Zhenhong Zhou, Yufei Guo
Submitted September 7, 2026
Primary Subject Computation and Language (cs.CL)
Secondary Subjects Artificial Intelligence (cs.AI)
arXiv Identifier arXiv:2609.07160
DOI 10.48550/arXiv.2609.07160