跳转至

文章背景与核心概要

电子健康档案（EHR）中每个患者的数据经常超过 10万 个 Token，这使得大语言模型（LLM）极易暴露出“迷失在中间”（Lost-in-the-Middle, LitM）效应——即位于上下文窗口中心附近的信息，其检索和利用的可靠性远低于边缘信息。在临床应用中，这带来了严重的风险，因为关键的诊断事实往往隐藏在患者病史的中间部分。

本文首次提出了“临床迷失在中间”（Clinical Lost-in-the-Middle, CLitM）问题，通过 MedDataset 数据集包含的 2,196 个指令-响应对以及六种语言模型，对其进行了系统的特征分析。为了缓解这一问题，作者提出了“查询条件临床抑制”（Query-Conditioned Clinical Suppression, QCCS）技术——这是一种轻量级的查询条件选择门控机制，在临床推理任务中，其表现优于标准的检索方法（如 BM25、稠密检索、交叉编码器重排）以及全上下文处理方式。

Inhibitory Attention for Clinical Long-Context Reasoning: Characterizing and Mitigating Lost-in-the-Middle Effects in EHR Processing

Summary

Electronic Health Records (EHRs) frequently exceed 100,000 tokens per patient, exposing large language models (LLMs) to the lost-in-the-middle (LitM) effect—where information located near the center of a context window is retrieved much less reliably than information at the edges. In clinical settings, this poses severe risks, as critical diagnostic facts frequently reside within the middle of patient histories.

This paper introduces the Clinical Lost-in-the-Middle (CLitM) problem, offering its first systematic characterization via the MedAlign dataset across 2,196 instruction-response pairs and six language models. To mitigate this issue, the author proposes Query-Conditioned Clinical Suppression (QCCS)—a lightweight, query-conditioned selection gate that outperforms standard retrieval methods (like BM25, dense retrieval, and cross-encoder reranking) and full-context processing in clinical reasoning tasks.

Paper Metadata & Overview

论文元数据与概览

  • arXiv 标识符： arXiv:2608.20348 [cs.CL]
  • 作者： Sanjay Basu
  • 提交时间： 2026年6月15日
  • 主分类： 计算与语言 (cs.CL)
  • 次分类： 人工智能 (cs.AI)
  • ACM 分类： I.2.7; J.3
  • 代码仓库： GitHub - inhibitory-attention-ehr
  • arXiv Identifier: arXiv:2608.20348 [cs.CL]
  • Author: Sanjay Basu
  • Submitted: June 15, 2026
  • Primary Subject: Computation and Language (cs.CL)
  • Secondary Subjects: Artificial Intelligence (cs.AI)
  • ACM Classes: I.2.7; J.3
  • Code Repository: GitHub - inhibitory-attention-ehr

Abstract & Key Findings

摘要与核心发现

  • CLitM 现象： 在测试的模型中，研究人员发现峰值准确率（在 20–30% 分位数为 59.5%）与低谷准确率（在 70–80% 分位数为 37.6%）之间存在 21.9 个百分点的差距。令人警惕的是，67.8% 的参考答案落在 EHR 时间线的第 10 到第 90 百分位数之间，正好处于性能低谷区。
  • 查询条件临床抑制（QCCS）： 引入 QCCS 作为一种轻量级选择门控，以有效过滤上下文。
  • 性能亮点（使用 Qwen2.5-7B-Instruct 在 16k 上下文窗口下评估）：
  • 中间位置指令： QCCS 取得了 16.7% 的准确率，相比之下 BM25 为 3.3%，交叉编码器（cross-encoder）为 0.0%，稠密检索为 0.0%，全上下文处理为 6.7%。
  • 整体性能： QCCS 达到了 25.3% 的准确率，而纯检索基线方法的最高准确率仅为 3.6%。
  • 核心洞察： 标准的检索召回率并不能决定下游推理的成败。虽然 BM25 在 98.8% 的指令中成功检索到了黄金证据句（而 QCCS 为 34.9%），但在包含正确句子的前提下，标准检索方法的准确率仍然上限 \(\le 2.6\%\)。相反，即使 QCCS 漏掉了准确的黄金证据句，它依然达到了 25.0% 的准确率，这凸显了查询对齐的上下文选择比原始检索召回率更能准确预测推理准确率。
  • The CLitM Phenomenon: Across tested models, researchers observed a 21.9 percentage-point gap between peak accuracy (59.5% in the 20–30% decile) and trough accuracy (37.6% in the 70–80% decile). Alarmingly, 67.8% of reference answers fall between the 10th and 90th percentiles of the EHR timeline, directly inside the performance trough.
  • Query-Conditioned Clinical Suppression (QCCS): QCCS is introduced as a lightweight selection gate to filter context effectively.
  • Performance Highlights (evaluated with Qwen2.5-7B-Instruct at a 16k context window):
  • Middle-position instructions: QCCS achieved 16.7% accuracy, compared to BM25 (3.3%), cross-encoder (0.0%), dense retrieval (0.0%), and full context processing (6.7%).
  • Overall performance: QCCS reached 25.3% accuracy, compared to a maximum of 3.6% for pure retrieval baselines.
  • Key Insight: Standard retrieval recall does not determine downstream reasoning success. While BM25 successfully retrieved the gold evidence sentence in 98.8% of instructions (versus 34.9% for QCCS), standard retrieval arms remained capped at \(\le 2.6\%\) accuracy even when containing the correct sentence. Conversely, QCCS achieved 25.0% accuracy even when it missed the exact gold sentence, highlighting that query-aligned context selection predicts accuracy better than raw retrieval recall.

Full-Text & Resource Links

全文与资源链接