VeLO: Training Versatile Learned Optimizers by Scaling Up

[hoangmit]

Do we have any pytorch port of these learned optimizers ?

Link: https://arxiv.org/abs/2211.09760v1

[AtomicityZ]

Paper is very well written

Here is the repo for future reference.

Hope to see it soon here

[drscotthawley]

Just sharing Here's a version that was in development up to a year ago: https://github.com/janEbert/PyTorch-VeLO
Haven't tried it yet and not sure if it (still) works.