跳转至

文章背景与核心概要

大语言模型（LLM）在进行开放式文本生成时，常常会陷入“逐字循环（verbatim loops）”的困境，不断重复上下文中已经出现过的文本片段。传统的防御手段（如存在惩罚、频率惩罚、重复惩罚或N元语法阻断）通常只关注孤立的Token出现频率，而忽略了循环的序列结构。当这些传统方法被放大使用时，往往会损害文本整体的流畅度和格式规范。

为了解决这一痛点，本文提出了“不要重复自己”（Don't Repeat Yourself，简称 DRY）这一新型采样阶段的Logit（对数几率）调整机制。与盲目惩罚单个Token不同，DRY仅在候选Token将当前生成的后缀延伸为先前出现过的文本片段的精确延续时，才对其实施惩罚。结合针对聊天模板和格式化Token的序列断开保护机制，DRY在显著遏制循环现象的同时，提升了词汇多样性，并完美保持了模型在 MT-Bench、MMLU 和 GSM8K 等核心基准测试上的性能。

Don't Repeat Yourself: Stopping Verbatim Loops at Sampling Time

Don't Repeat Yourself: Stopping Verbatim Loops at Sampling Time

Summary

Summary

Large Language Models often fall into verbatim loops during open-ended generation, repeating spans of text already present in their context. Traditional defenses—such as presence, frequency, and repetition penalties or n-gram blocking—focus on isolated token frequencies rather than the sequential structure of a loop, often degrading overall fluency and formatting when scaled up.

This paper introduces Don't Repeat Yourself (DRY), a novel sampling-time logit adjustment mechanism. Instead of penalizing individual tokens blindly, DRY penalizes candidate tokens only when they extend the current generation suffix into an exact continuation of a previously seen text span. Protected by sequence breakers for chat templates and formatting tokens, DRY significantly curtails looping while improving lexical diversity and preserving performance on core benchmarks like MT-Bench, MMLU, and GSM8K.

Paper Metadata

Paper Metadata

  • arXiv ID: arXiv:2608.22761 [cs.CL]
  • Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
  • Submission Date: August 24, 2026
  • Authors:
  • Philipp Emanuel Weidmann
  • Allen Roush
  • Judah Goldfeder
  • Sanjay Basu
  • Ravid Shwartz-Ziv
  • License: Creative Commons Attribution 4.0 International license icon

Abstract

Abstract

Large Language Models generate text autoregressively, but open-ended generation is prone to verbatim looping, in which models repeat spans already present in context. Standard defenses such as repetition, presence, and frequency penalties and n-gram blocking act on token recurrence rather than the sequential structure of a loop, and often suppress looping only at strengths that also degrade formatting or fluency. We propose Don't Repeat Yourself (DRY), a sampling-time logit adjustment that penalizes a candidate token only when generating it would extend the current suffix into an exact continuation of a span seen earlier in the context. Sequence breakers protect chat templates and formatting tokens. Across models from 1.5B to 120B parameters, nine prompt families, and a 600-pair human study, DRY reduces suffix-extension rate by 47% while improving lexical diversity. An intervention-matched placebo produces no comparable reduction, identifying suffix matching as the operative mechanism. On AWQ-quantized 70B and 120B models, DRY reduces loop rate by roughly half while preserving MT-Bench, MMLU, and GSM8K performance, whereas standard alternatives lose measurable ground. DRY has been adopted by popular open-source LLM inference frameworks including llama.cpp, ExLlamaV2, and text-generation-webui, highlighting its practical impact on text generation.

Key Findings & Impact

Key Findings & Impact

  • Structural Suffix Matching: Rather than counting token frequencies, DRY evaluates sequential patterns to prevent a suffix from mirroring earlier text segments.
  • Empirical Validation: Tested across models ranging from 1.5B to 120B parameters and across nine distinct prompt families, achieving a 47% reduction in suffix-extension rates alongside improved lexical diversity.
  • Benchmark Preservation: Maintains competitive accuracy on standardized evaluations (MT-Bench, MMLU, and GSM8K) where traditional penalties typically cause performance degradation.
  • Widespread Framework Adoption: Already integrated into major open-source inference pipelines, including llama.cpp, ExLlamaV2, and text-generation-webui.

Access & Resources

Access & Resources