Hierarchical Inverse Reinforcement Learning

Extends M. Alger's implementation of selected inverse reinforcement learning (IRL) algorithms. A summary report of my work is available here. His final report is available here and describes the implemented algorithms.

If you use this code in your work, you can cite it as follows:

@misc{davchev17,
  author       = {Todor Davchev},
  title        = {Hierarchical Inverse Reinforcement Learning},
  year         = 2017
}

If you are only interested in the IRL aspect of this project, you can find at Alger's repo

Algorithms implemented

• Linear programming IRL. From Ng & Russell, 2000. Small state space and large state space linear programming IRL.
• Maximum entropy IRL. From Ziebart et al., 2008.
• Deep maximum entropy IRL. From Wulfmeier et al., 2015; original derivation.
• Hierarchical MaxEnt IRL.

Additionally, the following MDP and semi-MDP domains are implemented:

• Gridworld (Sutton, 1998)
• Extended Gridworld with options (Sutton, 1998)
• Objectworld (Levine et al., 2011)

Requirements

• NumPy
• SciPy
• CVXOPT
• Theano
• MatPlotLib (for examples)