Skip to content

Commit

Permalink
add a working toy csgld notebook
Browse files Browse the repository at this point in the history
  • Loading branch information
activatedgeek committed Mar 24, 2023
1 parent acd813e commit 4e5267f
Show file tree
Hide file tree
Showing 2 changed files with 214 additions and 0 deletions.
22 changes: 22 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,29 @@ pip install git+https://github.com/activatedgeek/torch-sgld.git

## Usage

The general idea is to modify the usual gradient-based update loops
in PyTorch with the `SGLD` optimizer.

```python
from torch_sgld import SGLD

f = module() ## construct PyTorch nn.Module.

sgld = SGLD(f.parameters(), lr=lr, momentum=.9) ## Add momentum to make it SG-HMC.
sgld_scheduler = ## Optionally add a step-size scheduler.

for _ in range(num_steps):
energy = f()
energy.backward()

sgld.step()

sgld_scheduler.step() ## Optional scheduler step.
```

`cSGLD` can be implemented by using a cyclical learning rate schedule.
See the [toy_csgld.ipynb](./notebooks/toy_csgld.ipynb) notebook for a
complete example.

## License

Expand Down
192 changes: 192 additions & 0 deletions notebooks/toy_csgld.ipynb

Large diffs are not rendered by default.

0 comments on commit 4e5267f

Please sign in to comment.