跳转至

文章背景与核心概要

强化学习（RL）策略通常难以审查，这对于需要高可信度和透明度的应用场景构成了重大挑战。虽然将训练好的策略转换为显式的决策树规则可以提升可解释性，但由此生成的产物往往过于复杂，超出了人类的理解极限。

为了解决这一问题，作者 Mark Leon Ringer 和 Michel Tokic 提出了一种新颖的剪枝流程。该方法在保持任务性能的同时简化了基于规则的策略，并确保了策略修改的可审计性。通过定义一小组结构化和使用感知（usage-aware）的操作符，该方法通过重新执行策略来评估候选修改，从而衡量回报与可解释性代理指标。在经典控制和 MuJoCo 基准测试上的评估表明，该方法在不牺牲高性能的前提下，实现了持续的可解释性提升。

Interpretable Reinforcement Learning with Decision-Tree Pruning

Summary

Reinforcement learning (RL) policies are often difficult to inspect, posing a significant challenge for applications requiring trustworthiness and transparency. While converting a trained policy into explicit decision-tree rules can improve interpretability, the resulting artifacts are frequently too complex for human comprehension.

To address this, Mark Leon Ringer and Michel Tokic present a novel pruning process that simplifies rule-based policies while preserving task performance and ensuring that edits remain auditable. By defining a small set of structural and usage-aware operators, the method evaluates candidate edits through policy re-execution to measure both the return and interpretability proxies. Evaluations on classic control and MuJoCo benchmarks demonstrate consistent interpretability improvements without sacrificing high performance.

强化学习（RL）策略通常难以审查，这对于需要高可信度和透明度的应用场景构成了重大挑战。虽然将训练好的策略转换为显式的决策树规则可以提升可解释性，但由此生成的产物往往过于复杂，超出了人类的理解极限。

为了解决这一问题，Mark Leon Ringer 和 Michel Tokic 提出了一种新颖的剪枝流程，能够在保持任务性能的同时简化基于规则的策略，并确保策略修改的可审计性。通过定义一小组结构化和使用感知（usage-aware）的操作符，该方法通过重新执行策略来评估候选修改，从而衡量回报与可解释性代理指标。在经典控制和 MuJoCo 基准测试上的评估表明，该方法在不牺牲高性能的前提下，实现了持续的可解释性提升。

Metadata

Abstract

Reinforcement learning policies are difficult to inspect, but interpreting them is a prerequisite for trustworthiness. Converting a trained policy into explicit decision-tree rules improves transparency and the resulting artifacts often remain too complex for human understanding. We present a pruning process that simplifies such rule-based policies while preserving task performance and making edits to the policy auditable. The process defines a small set of structural and usage-aware operators and evaluates candidate edits by re-executing the policy to measure return and interpretability proxies. This exposes an transformation process from complex to compact policy structures. We investigate this approach on classic control and MuJoCo benchmarks, where pruning traces reveal consistent interpretability improvements while maintaining high performance.

强化学习策略很难进行审查，但对其进行解释是建立可信赖性的先决条件。将训练好的策略转换为显式的决策树规则可以提高透明度，但生成的产物往往仍然过于复杂，不便于人类理解。我们提出了一种剪枝流程，能够在保持任务性能并使策略修改具有可审计性的同时，简化此类基于规则的策略。该流程定义了一小组结构化和使用感知的操作符，并通过重新执行策略来评估候选修改，以衡量回报和可解释性代理指标。这展现了一个从复杂到紧凑的策略结构转换过程。我们在经典控制和 MuJoCo 基准测试中研究了这种方法，剪枝轨迹表明，在保持高性能的同时，可解释性得到了持续的改善。