review the training/testing loop

[yuanqing-wang]

@karalets

could you please review the experiment script here?

https://github.com/choderalab/pinot/blob/master/pinot/app/experiment.py

[maxentile]

Looks fine to me, but I must admit I don't have a super clear picture where this is going at a high level. @karalets would be much better-suited to provide guidance here.

In case you want to document the intent more within this file:

• If there's a statement or diagram of your overall design somewhere, that would be helpful to reference. ("We want to use Train experiments to store training trajectories under the following optimizers ..., for later analysis by ..., with the goal of assessing ...")
• In docstring, can also mention which functions or modules a class needs to play nice with, if the interface isn't otherwise obvious or enforced. (What's going to later use aTrain object? Will it later be assumed that some internal state of the object -- like train.states -- has a certain structure?)
• If there are other types of experiments that are planned for this module but not yet implemented (active selection of training data, uncertainty calibration, etc.), an option for documenting your intent is to add raise NotImplementedError placeholders in any missing definitions.

[yuanqing-wang]

added an example script to illustrate the usage

1d58549