跳转至

文章背景与核心概要

企业级客服运营常常面临政策、产品更新以及知识库快速迭代的挑战，传统的静态AI助手往往会变得脆弱且维护成本高昂。在这篇论文中，LinkedIn的研究人员介绍了一种自进化的智能客服系统。该系统通过将检索增强生成（RAG）与演化式自动提示词生成（evolutionary auto-prompting）以及模块化评估框架相结合，在闭环工作流中实现了持续自我优化，且完全无需重新训练昂贵的基础模型。

LinkedIn在生产环境中进行的A/B测试显示出了显著的性能提升，证明了可扩展的自进化AI智能体在真实企业环境中的可行性。

Self-evolving Agentic Customer Support System at LinkedIn

📌 Executive Summary

Enterprise customer support operations often struggle with rapidly shifting policies, product updates, and knowledge bases. Traditional static AI assistants quickly become brittle and expensive to maintain.

In this paper, researchers from LinkedIn introduce a self-evolving agentic customer support system. By combining Retrieval-Augmented Generation (RAG) with evolutionary auto-prompting and a modular evaluation framework, the system continuously improves itself in a closed-loop workflow—all without requiring costly foundation model retraining.

Production A/B testing at LinkedIn demonstrated significant gains, proving the viability of scalable, self-evolving AI agents in real-world enterprise environments.

📄 Metadata & Publication Details

作者

  • Chih Hui Wang
  • Mengdie Tu
  • Qianyun Zhang
  • Wei Wu
  • Lili Zhou
  • Mingqi Shen
  • Changshuai Wei

Authors

  • Chih Hui Wang
  • Mengdie Tu
  • Qianyun Zhang
  • Wei Wu
  • Lili Zhou
  • Mingqi Shen
  • Changshuai Wei

🔍 Abstract

Enterprise support agents operate in rapidly changing environments where policies, product capabilities, and knowledge bases evolve continuously, making static assistants brittle and costly to maintain. We present LinkedIn's self-evolving agentic support system, which integrates retrieval-augmented generation with evolutionary auto-prompting and a modular, production-aligned evaluation framework to enable safe, continuous improvement without retraining foundation models. The system treats prompts, retrieval, and evaluation as a closed-loop, versioned workflow with operational guardrails. Offline simulations and ablations show clear quality gains over vanilla RAG and baseline agents, including reduced hallucinations and improved response completeness. In a two-week user-randomized A/B test on LinkedIn's production support traffic, the integrated self-evolved workflow increased QA self-serve by 9.0 percentage points, cancellation self-serve by 4.8 points, and routing accuracy by 30.6 points. These results demonstrate a practical path to scalable, self-evolving AI agents in real-world enterprise settings.

⚙️ Key System Architecture & Features

  1. 闭环工作流： 将提示词、信息检索和系统评估视为一个互联且带版本控制的过程，并配备了严格的运营护栏（guardrails）。
  2. 演化式自动提示词： 根据性能指标和历史交互，随着时间动态优化提示词。
  3. 模块化评估框架： 与生产需求紧密对齐，在部署前安全地对更新进行基准测试。
  4. 无需重新模型训练： 在无需微调或重新训练底层基础模型的计算开销下，实现持续优化。
  1. Closed-Loop Workflow: Treats prompts, information retrieval, and system evaluation as an interconnected, versioned process equipped with strict operational guardrails.
  2. Evolutionary Auto-Prompting: Dynamically refines prompts over time based on performance metrics and historical interactions.
  3. Modular Evaluation Framework: Aligned directly with production needs to safely benchmark updates before deployment.
  4. No Model Retraining Needed: Achieves continuous optimization without the compute overhead of fine-tuning or retraining underlying foundation models.

📊 Production Performance & Results

离线仿真和消融实验证实了其相比标准RAG和基线智能体具有更卓越的质量，并指出幻觉有所减少，响应完整性显著提高。

此外，在LinkedIn的线上生产客服流量上进行的为期两周的用户随机A/B测试取得了令人瞩目的成果： * 问答（QA）自助服务率： 提升了 9.0 个百分点 * 取消业务自助服务率： 提升了 4.8 个百分点 * 路由准确率： 提升了 30.6 个百分点

Offline simulations and ablation studies confirmed superior quality compared to standard RAG and baseline agents, noting reductions in hallucinations and significantly better response completeness.

Furthermore, a two-week user-randomized A/B test on LinkedIn's live production support traffic yielded impressive results: * QA Self-Serve: Increased by 9.0 percentage points * Cancellation Self-Serve: Increased by 4.8 percentage points * Routing Accuracy: Improved by 30.6 percentage points