Warning: This repo is no longer maintained. For a more recent (and improved) implementation of DDPG see https://github.com/openai/baselines/tree/master/baselines/ddpg .
Paper: "Continuous control with deep reinforcement learning" - TP Lillicrap, JJ Hunt et al., 2015
Install Gym and TensorFlow. Then:
pip install pyglet # required for gym rendering
pip install jupyter # required only for visualization (see below)
git clone https://github.com/SimonRamstedt/ddpg.git # get ddpg
Example:
python run.py --outdir ../ddpg-results/experiment1 --env InvertedDoublePendulum-v1
Enter python run.py -h
to get a complete overview.
If you want to run in the cloud or a university cluster this might contain additional information.
Example:
python dashboard.py --exdir ../ddpg-results/+
Enter python dashboard.py -h
to get a complete overview.
- No batch normalization yet
- No conv nets yet (i.e. only learning from low dimensional states)
- No proper seeding for reproducibilty
Please write me or open a github issue if you encounter problems! Contributions are welcome!
- Output normalization – the main reason for divergence are variations in return scales. Output normalization would probably solve this.
- Prioritized experience replay – faster learning, better performance especially with sparse rewards – Please write if you have/know of an implementation!
Remote execution:
python run.py --outdir [email protected]:/some/remote/directory/+ --env InvertedDoublePendulum-v1