文章背景与核心概要

为AI智能体提供特定专家的参考资料可以达成多种目标：提供难以获取的领域知识、塑造可识别的角色（Persona），或修改智能体的决策过程。本文利用 mimeo（一个旨在将公开专家文集编译为智能体可加载技能文件的开源工具）对这些不同的假设进行了深入研究。

通过针对编码智能体测试框架（coding-agent harness）的严格基准测试，本研究表明：尽管 mimeo 在知识检索和立场接地（grounding，防止产生幻觉立场）方面表现出色，但由于评估上限问题，其对专家判断力迁移的影响尚无定论，且其角色的可识别性也因上下文的不同而有所差异。

mimeo：将公开专家文集编译为智能体技能并测试知识迁移

作者： Timothy Kassis
ArXiv ID： arXiv:2609.00453 [cs.AI]
提交时间： 2026年8月31日
研究领域： 人工智能 (cs.AI)
链接： 查看 PDF | GitHub 仓库与专家配置文件

mimeo: Compiling Public Expert Corpora into Agent Skills and Testing What Transfers

Authors: Timothy Kassis
ArXiv ID: arXiv:2609.00453 [cs.AI]
Submitted: August 31, 2026
Subjects: Artificial Intelligence (cs.AI)
Links: View PDF | GitHub Repository & Expert Profiles

执行摘要

为AI智能体提供特定专家的参考资料可以达成多种目标：提供难以获取的领域知识、塑造可识别的角色（Persona），或修改智能体的决策过程。本文利用 mimeo（一个旨在将公开专家文集编译为智能体可加载技能文件的开源工具）对这些不同的假设进行了深入研究。

通过针对编码智能体测试框架的严格基准测试，本研究表明：尽管 mimeo 在知识检索和立场接地（防止产生幻觉立场）方面表现出色，但由于评估上限问题，其对专家判断力迁移的影响尚无定论，且其角色的可识别性也因上下文的不同而有所差异。

Executive Summary

Providing an AI agent with reference materials about a specific expert can serve multiple purposes: supplying hard-to-find domain knowledge, establishing a recognizable persona, or modifying the agent's decision-making process. This paper investigates these distinct claims using mimeo, an open-source tool designed to compile public expert corpora into agent-loadable skill files.

Through rigorous benchmarking against a coding-agent harness, the study demonstrates that while mimeo excels at knowledge retrieval and grounding (preventing hallucinated stances), its impact on expert judgment transfer remains inconclusive due to evaluation ceilings, and its persona identifiability varies depending on the context.

核心发现与方法论

Key Findings & Methodology

1. mimeo 流水线

• 功能： 发现特定专家的公开作品，提取相关引文，通过缓存的源文本对每条引文进行交叉核对，并将其编译为供AI智能体使用的结构化文件。
• 性能： 在八次记录的构建中，系统平均每次运行进行 38 次模型调用，在验证过程中成功拦截并拒绝了 13.2% 提取错误的引文。

1. The mimeo Pipeline

• Functionality: Discovers public works by a given expert, extracts relevant quotations, cross-checks each quotation against cached source texts, and compiles them into a structured file for AI agents.
• Performance: Across eight logged builds, the system averaged 38 model calls per run, successfully catching and rejecting 13.2% of incorrectly extracted quotations during verification.

2. 知识获取与立场接地

• 知识检索： mimeo 成功回答了所有 20 个冷门且依赖引文的测试问题。相比之下，任何闭卷条件下的回答数量都没有超过 10 个。在完全相同的页面上进行关键词搜索（BM25）则答对了 15–17 个。
• 角色立场接地： 纯粹由模型记忆生成的角色，在面对不同评估者的 20 个问题时，有 1–4 个问题误表述了有据可查的立场。普通智能体和由 mimeo 支持的智能体则从未误表述过有据可查的立场。

2. Knowledge Access & Grounding

• Knowledge Retrieval: mimeo successfully answered all 20 obscure, quotation-heavy test questions. In contrast, no closed-book condition managed to answer more than 10. Keyword search (BM25) over the exact same pages answered 15–17.
• Persona Grounding: Personas generated purely from model memory misstated documented positions on 1–4 out of 20 questions across various evaluators. The plain agent and the mimeo-backed agent never misstated documented stances.

3. 角色可识别性

• 可检测性： 在简短的开放式提示词中，专家角色很容易被识别出来。
• 上下文稀释： 引入特定任务的材料使角色识别准确率下降了 18–23 个百分点，这表明 mimeo 的可识别性并不比完全由模型记忆生成的配置文件更高。

3. Persona Identifiability

• Detectability: Expert personas were easily identifiable in short, open prompts.
• Contextual Dilution: Introducing task-specific material reduced persona identification accuracy by 18–23 percentage points, revealing that mimeo is no more easily identifiable than a profile generated entirely from model memory.

4. 判断力迁移与评估挑战

• 天花板效应： 由于两个测试类别都触及了性能天花板，判断力迁移问题无法得到确切的解决。测试条件在工程任务中成功找出了 94–97% 预设的漏洞，并在 16 个新的应用场景中得分达到 94–100%。
• AI裁判偏见： “听起来像专家”的基准测试得分根据所使用的评估器而剧烈波动。四个AI裁判中有两个偏爱符合模型刻板印象的回答，而另外两个在相同的文本上检测不出明显差异。这突显了依赖单一AI裁判所带来的风险。

4. Judgment Transfer & Evaluation Challenges

• Ceiling Effects: Judgment transfer could not be definitively resolved because both test categories hit performance ceilings. Tested conditions successfully found 94–97% of planted bugs in engineering tasks and scored 94–100% on 16 new application scenarios.
• AI Judge Bias: "Sounds like the expert" benchmark scores fluctuated heavily depending on the evaluator used. Two out of four AI judges favored answers aligned with model stereotypes, whereas the other two detected no discernible difference on the identical text. This highlights the risk of relying on a single AI judge.

结论

实证证据表明，mimeo 作为一个高效、紧凑且可检查的参考工具，在获取特定领域知识和验证个人立场方面非常有效。然而，它尚未成为迁移专家底层判断力的成熟机制。

Conclusion

The empirical evidence positions mimeo as a highly effective, compact, and inspectable reference tool for accessing specific domain knowledge and authenticating positions on a person. However, it does not yet serve as a proven mechanism for transferring an expert's underlying judgment.