mosaicml-turbo: Where to find the repo?

[agarvic]

I'm trying to implement DecoupledLionW_8bit in my fine-tuning script, but I get the following error:

ERROR: Could not find a version that satisfies the requirement mosaicml-turbo<0.1,>=0.0.2; extra == "gpu" (from llm-foundry[gpu]) (from versions: none)

I haven't been able to find this repo anywhere. Can you please, help me?

[dblalock]

The repo is currently private since we're not sure where our custom kernels should live yet. What environment are you running in (OS, presence of CUDA gpus, python version, etc)? The wheel exists on pypi, so it's a question of whether your setup supports it.

[agarvic]

Thanks for your response. I have finally managed to install it from the wheel after a few changes in my setup to match the requirements :-)

[dblalock]

awesome! Any insights on what needed to change? Would love to make this seamless for everyone else in the future.

[tedfeng424]

Hello, I ran into a similar problem, I am wondering what changes you made to install this package. Thanks

[surak]

The repo is currently private since we're not sure where our custom kernels should live yet. What environment are you running in (OS, presence of CUDA gpus, python version, etc)? The wheel exists on pypi, so it's a question of whether your setup supports it.

In our supercomputing center we use python 3.11 and 3.12. Currently with around 5000 A100 GPUs using CUDA 12, with some 15000 more to come - probably with cuda 13, as they'll be GH200. So the cp310 wheel won't work for us. Would it be possible to have the source back on your repo as you had before? Thanks

[janEbert]

What's the future direction regarding mosaicml-turbo and llm-foundry? I get a bad feeling about investing in this code base if parts of it (in this case dependencies) are not open source. I know at the moment, mosaicml-turbo is only used for the Lion optimizer, but this could change in the future.
I'd really appreciate hearing your (@dblalock and the rest of MosaicML) current thoughts about this, since I'm currently evaluating llm-foundry for some large-scale runs and have had a blast with its simplicity and performance.

[jfrankle]

Hi @janEbert and company - Oops - you caught us! We've been working on a package of kernels called mosaicml-turbo, and we inadvertently released it too soon. It's still a work-in-progress, and we've been trying to keep it quiet until we're ready for an official release. We'll be maintaining and building on llm-foundry (and releasing new tools like mosaicml-turbo) for a very long time to come. We just jumped the gun on this one 😬

Jonathan Frankle

[janEbert]

Thank you Jonathan for the timely response! I'll be sure to keep quiet as well. ;)
Great to hear that you're focusing on making llm-foundry even faster; I'm looking forward to what other improvements you manage to achieve in this already performant code base.

Have you decided whether mosaicml-turbo will be open-sourced upon release?