A reference implementation of a reduction from imitation learning to reinforcement learning, presented in the following paper:
Kamil Ciosek Imitation Learning by Reinforcement Learning, ICLR 2022.
The implementation was tested on Python 3.9. To run the code, you need to install packages from requirements.txt. Using this repository requires the git-lfs extension. See here for installation instructions.
To get started, simply follow these steps:
- Clone the repo locally with:
git clone https://github.com/spotify-research/il-by-rl.git - Move to the repository with:
cd il-by-rl - install the dependencies:
pip install -r requirements.txt
Since running the experiments is computationally expensive, we provide pre-computed logs in the sample-logs directory.
These can be plotted using the plots.ipynb notebook. Due to minor changes to the code, these logs are not absolutely
identical to the ones used for the paper, but they support the exact same qualitative conclusion (ILR is as good as other
methods while being simpler).
If you want to re-run the experiments (regenerating the logs), you can run the command python train.py --env=ENV --method=METHOD,
where ENV is one of hopper, ant, walker, halfcheetah and METHOD is one of bc, il, gail, gmmil, sqil. The new logs
will be saved to the current directory.
Create a new issue
We feel that a welcoming community is important and we ask that you follow Spotify's Open Source Code of Conduct in all interactions with the community.
Follow @SpotifyResearch on Twitter for updates.
Copyright 2022 Spotify, Inc.
Licensed under the Apache License, Version 2.0: https://www.apache.org/licenses/LICENSE-2.0
Please report sensitive security issues via Spotify's bug-bounty program (https://hackerone.com/spotify) rather than GitHub.