SD2AIL: Adversarial Imitation Learning from Synthetic Demonstrations via Diffusion Models

arXiv cs.RO / 4/30/2026

💬 OpinionModels & Research

共有:

Key Points

The paper introduces SD2AIL, a diffusion-model-based approach to adversarial imitation learning that creates synthetic demonstrations to complement limited expert data.
It uses a diffusion model within the discriminator to generate pseudo-expert demonstrations and then improves training by replaying only the most valuable ones.
The proposed prioritized expert demonstration replay strategy (PEDR) is designed to efficiently select demonstrations from a large pool of (pseudo-)expert data.
Experiments on simulation tasks show the method is both effective and robust, notably achieving an average return of 3441 on the Hopper task, outperforming the prior state of the art by 89.
The authors report that the code will be released on GitHub for reproducibility and further research.

Abstract

Adversarial Imitation Learning (AIL) is a dominant framework in imitation learning that infers rewards from expert demonstrations to guide policy optimization. Although providing more expert demonstrations typically leads to improved performance and greater stability, collecting such demonstrations can be challenging in certain scenarios. Inspired by the success of diffusion models in data generation, we propose SD2AIL, which utilizes synthetic demonstrations via diffusion models. We first employ a diffusion model in the discriminator to generate synthetic demonstrations as pseudo-expert data that augment the expert demonstrations. To selectively replay the most valuable demonstrations from the large pool of (pseudo-) expert demonstrations, we further introduce a prioritized expert demonstration replay strategy (PEDR). The experimental results on simulation tasks demonstrate the effectiveness and robustness of our method. In particular, in the Hopper task, our method achieves an average return of 3441, surpassing the state-of-the-art method by 89. Our code will be available at https://github.com/positron-lpc/SD2AIL.