forked from LeelaChessZero/lc0
-
Notifications
You must be signed in to change notification settings - Fork 0
40B Net
Viren6 edited this page Nov 30, 2021
·
3 revisions
Latest released 4th LR net - 2M steps
https://drive.google.com/file/d/1wvLBcdcMFAjjRyHjVFdV71iKFNTFlSgC/view
End of 3rd LR net - 1.67M steps
https://drive.google.com/file/d/16QUF4pUUc4QFXdB_LPu3ZFfE1uLRk0sS/view
Info:
Thanks to BloodyRookie (aka KittenKaboodle) for the 4xA100 machine this net is being trained on, and Masterkni6 for doing the majority of the training and testing process.
This run was from scratch using T60 data from the 1st of April to the 10th of October (A total of around 55M games) rescored with 6 man syzygy and 5 man gaviota TBs. Training started on the 17th of October.
Rescore Command: https://pastebin.com/S3VUP29S
Initial YAML: https://pastebin.com/Zi7XB6mT
Changes throughout the run:
| Step Count | Changes |
|---|---|
| 190k | LR 0.1 Max grad norm 0.9 |
| 260.5k | LR 0.04 Max grad norm 1.2 Reg term weight: 0.5 |
| 700k | LR 0.004 Max grad norm 1.8 Reg term weight: 0.25 |
| 1.36M | Diff Focus Enabled Max grad norm 6 |
| 1.465M | Addition of 2M T76 Games |
| 1.5175M | Removal of T76 Games |
| 1.67M | LR 0.0004 Reg term weight: 0.08 |