Reinforcement Learning (RL) has been extensively used in autonomous and self-* systems, but RL policies, especially deep RL ones relying on neural networks, lack transparency and are difficult to understand.
This can lead to diminished user trust, and makes for a more challenging verification of systems.
To address this challenge, this paper introduces Explanations using Alternative Realities for Reinforcement Learning (EARL), a Python library to produce counterfactual explanations in RL settings.
This library allows the user to produce explanations by exploring What-if scenarios to clarify agent behavior by comparing possible outcomes.
Counterfactual explanations have been shown to be intuitive and user-friendly in psychology research, but have only recently been explored in RL, with existing implementations usually limited to toy examples and benchmarks.
EARL supports counterfactual explanation generation in realistic RL-based self-adaptive systems.
To demonstrate its applicability, we demonstrate its use in a simulation of CitiBikes, a self-adaptive bike-sharing system, and we provide evaluations showing how it performs in real applications.