LLM因果先验何时以及为何有效：面向摊销因果推断的闭环先验选择

文章背景与核心概要

因果效应估计（即确定在特定干预下结果会发生怎样的变化）是医学、经济学和公共政策等领域的基础性任务。先验数据拟合网络（PFNs）通过“摊销（amortization）”这一过程简化了该任务：模型通过在大规模程序化生成的合成因果任务集上进行训练，从而能够直接吸纳观测数据，并在单次前向传递中立即返回干预效应估计值。

长期以来，PFNs的能力一直受到其合成训练先验的瓶颈制约，这些先验通常需要人工设计（这也是 Do-PFN 和 CausalPFN 所认识到的局限性）。尽管现代大语言模型（LLMs）能够可靠地为特定领域生成合理的因果图，但利用这些经大模型蒸馏得到的因果图传统上仍然依赖于反复试错。为此，本文引入了一种闭环先验选择框架，将先验注入从人工猜测转变为一个可经经验验证的选择问题。

執行摘要 / Executive Summary

Causal effect estimation—determining how an outcome would change under a specific intervention—is a foundational task across medicine, economics, and public policy. Prior-data fitted networks (PFNs) streamline this process by amortizing the task: a model is trained on massive collections of programmatically generated synthetic causal tasks, allowing it to ingest observational data and immediately return interventional-effect estimates in a single forward pass.

因果效应估计——即确定在特定干预下结果会发生怎样的变化——是医学、经济学和公共政策等领域的基础性任务。先验数据拟合网络（PFNs）通过摊销任务简化了这一过程：模型在海量程序化生成的合成因果任务集上进行训练，使其能够摄入观测数据并在单次前向传递中立即返回干预效应估计值。

Historically, the capability of PFNs has been bottlenecked by their synthetic training priors, which are manually designed (a limitation recognized by both Do-PFN and CausalPFN). While modern Large Language Models (LLMs) can reliably generate plausible causal graphs for specific domains, utilizing these LLM-distilled graphs has traditionally relied on trial and error.

历史上，PFNs的能力一直受到其合成训练先验的瓶颈制约，这些先验是人工设计的（这一局限性已被 Do-PFN 和 CausalPFN 所认识到）。虽然现代大语言模型（LLMs）可以为特定领域可靠地生成合理的因果图，但利用这些由LLM蒸馏出的图通常依赖于反复试错。

To address this, this paper introduces a closed-loop prior selection framework that transforms prior injection from manual guesswork into an empirically verifiable selection problem.

为了解决这个问题，本文引入了一个闭环先验选择框架，将先验注入从人工猜测转变为一个可经经验验证的选择问题。

核心贡献与方法论 / Key Contributions & Methodology

Budget-Constrained Optimization: The proposed framework treats prior injection as an optimization problem over a candidate pool of LLM-distilled graphs.
Evaluation Pipeline: Candidate priors undergo lightweight post-training and are scored using a composite metric where real-domain generalization takes precedence. The top-performing candidate is then selected for full training and rigorous statistical validation.
Mechanism Analysis: Through targeted experiments, the study demonstrates that performance gains stem from the semantic content of the distilled graphs rather than arbitrary structural diversity (directional evidence).

受预算约束的优化： 所提出的框架将先验注入视为在LLM蒸馏图的候选池上的优化问题。

评估流水线： 候选先验经历轻量级后训练，并使用以真实领域泛化能力优先的复合指标进行评分。随后选择表现最佳的候选者进行完整训练和严格的统计验证。

机制分析： 通过针对性实验，该研究表明性能的提升源于蒸馏图的语义内容，而不是随意的结构多样性（方向性证据）。

实验结果 / Empirical Results

Tested on a 7.34M-parameter Do-PFN, the framework yields significant improvements: * Performance Gain: The optimal selected prior achieves a formally significant \(2.75\times\) gain on the primary evaluation domain. * Error Reduction: Overall error drops below that of the uninjected official baseline model. * Robust Generalization: Generalization improves markedly on adjacent monitoring domains, with zero regression across other monitored capabilities.

在一个 734万参数的 Do-PFN 上进行测试，该框架取得了显着的改进： * 性能提升： 最优选定的先验在主要评估领域上实现了形式上显著的 \(2.75\times\) 提升。 * 误差降低： 总体误差降至未注入的官方基准模型以下。 * 稳健的泛化能力： 在相邻监控领域上的泛化能力显著提高，且其他受监控能力的性能没有出现任何退化。