How to use MeZO in training a simple CIFAR-10 model

[Cascol-Chen]

Hi, thanks you for sharing such an amazing work.
To use MeZO more easily, could you provide a minimum demo to show how can we use MeZO as an optimizer to train a CIFAR model?

[gaotianyu1350]

Hi,

Please refer to the section "How to add MeZO to my own code?" here. You can find our highlighted MeZO code in this file and it should be fairly easy to copy it to whatever training code that you are using. The logic of MeZO is quite simple.

[Cascol-Chen]

Hi,

Sorry for the late reply. Although there are comments to highlight MeZO code, the trainer.py comprises thousand lines of code with complicated dependancy and it's hard to make modification. Therefore, I sincerely recommend a minimum demo on CIFAR to make the great work easier to be followed.

[gaotianyu1350]

Hi,

If you search "MeZO added" in the file you will find all the related code in trainer.py.