跳转至

文章背景与核心概要

当前的语言模型系统通常通过将多个信息源拼接进单个提示词中来进行信息聚合。本文认为，这种方法混淆了两个截然不同的操作：证据解释（需要上下文和模型能力）与决策聚合（需要算术精度和可比性）。

为此，作者提出了一种结构化的“四字段证据元组”（假设、可靠性区间、基本原理和来源），从而将这两个过程解耦。通过这种方式，本文识别并解决了一种称为“计数规模漂移”（count-scale drift）的关键失效模式，即决策阈值会根据所查阅的信息源数量发生不可预测的偏移。所提出的解决方案——汇聚校准后的对数似然比——是一种适用于分诊引擎和诊断面板等各种系统的算术修复方法。该框架在纵向语料库上进行了验证，取得了 0.921 的 AUPRC，显著优于 0.805 的基线水平。

Split the Labor: Separating Evidence Interpretation from Decision Aggregation

Author: Zhelun Wu
Date: August 14, 2026
arXiv ID: 2608.14509
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)

Author: Zhelun Wu
Date: August 14, 2026
arXiv ID: 2608.14509
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)

Summary

Current language model systems often aggregate information by concatenating multiple sources into a single prompt. This paper argues that this approach conflates two distinct operations: evidence interpretation (which requires context and capacity) and decision aggregation (which requires arithmetic precision and comparability).

The author proposes a structured "four-field evidence tuple" (hypothesis, reliability bucket, rationale, and provenance) to decouple these processes. By doing so, the paper identifies and solves a critical failure mode called "count-scale drift," where decision thresholds shift unpredictably based on the number of sources consulted. The proposed solution—pooling calibrated log-likelihood ratios—is an arithmetic fix applicable to various systems, including triage engines and diagnostic panels. The framework is validated on a longitudinal corpus, achieving a 0.921 AUPRC compared to a 0.805 baseline.

Summary

Current language model systems often aggregate information by concatenating multiple sources into a single prompt. This paper argues that this approach conflates two distinct operations: evidence interpretation (which requires context and capacity) and decision aggregation (which requires arithmetic precision and comparability).

The author proposes a structured "four-field evidence tuple" (hypothesis, reliability bucket, rationale, and provenance) to decouple these processes. By doing so, the paper identifies and solves a critical failure mode called "count-scale drift," where decision thresholds shift unpredictably based on the number of sources consulted. The proposed solution—pooling calibrated log-likelihood ratios—is an arithmetic fix applicable to various systems, including triage engines and diagnostic panels. The framework is validated on a longitudinal corpus, achieving a 0.921 AUPRC compared to a 0.805 baseline.

Key Contributions

1. Decoupling Interpretation from Aggregation

The paper posits that the design problem for multi-source systems lies in the interface between interpretation and aggregation. By separating these, the system can better handle: * Interpretation: Rewarded by capacity and context. * Aggregation: Rewarded by fixed arithmetic and the ability to handle null results.

Key Contributions

1. Decoupling Interpretation from Aggregation

The paper posits that the design problem for multi-source systems lies in the interface between interpretation and aggregation. By separating these, the system can better handle: * Interpretation: Rewarded by capacity and context. * Aggregation: Rewarded by fixed arithmetic and the ability to handle null results.

2. Addressing "Count-Scale Drift"

The author identifies a failure mode where thresholding unnormalized weights leads to an operating point that "slides" depending on the number of sources. This drift is exacerbated by reader reliability. The paper demonstrates that pooling calibrated log-likelihood ratios resolves this issue, providing a more robust decision-making mechanism.

2. Addressing "Count-Scale Drift"

The author identifies a failure mode where thresholding unnormalized weights leads to an operating point that "slides" depending on the number of sources. This drift is exacerbated by reader reliability. The paper demonstrates that pooling calibrated log-likelihood ratios resolves this issue, providing a more robust decision-making mechanism.

3. Empirical Validation

The framework was tested on a longitudinal corpus using two distinct approaches: * Outcome-based: Partitioning over reading. * Pre-outcome: Partitioning over learning capacity. * Performance: The proposed method (using a sequence encoder and tree ensemble) reached an AUPRC of 0.921, significantly outperforming the hand-crafted baseline of 0.805.

3. Empirical Validation

The framework was tested on a longitudinal corpus using two distinct approaches: * Outcome-based: Partitioning over reading. * Pre-outcome: Partitioning over learning capacity. * Performance: The proposed method (using a sequence encoder and tree ensemble) reached an AUPRC of 0.921, significantly outperforming the hand-crafted baseline of 0.805.

4. Theoretical Framework

The paper concludes by providing: * Five specific predictions to falsify the framework. * Three negative results. * An analysis of which comparisons remain confounded in current research.

4. Theoretical Framework

The paper concludes by providing: * Five specific predictions to falsify the framework. * Three negative results. * An analysis of which comparisons remain confounded in current research.

Access & Metadata

Access & Metadata