Out of memory with RTX3090

[PyxAI]

Hi,
I'm trying to train gpt2xl, but keep getting OOM, even when I set batch size to 1 and gradient_accumulation to 8\16\512, contigous_gradients false and allgather_bucket_size \ reduce_bucket_size 2e2.
I can see in nvidia-smi that I'm only reaching half the memory capacity - around 12GB
My system is as stated - 3090 with 24GB memory
80 GB RAM
5600x cpu if that matters
running WSL2 on windows 10
Thanks.

[PyxAI]

So working with WSL is just a no-go
I installed dual boot ubuntu and now the problem disappeared

[BrandonKoerner]

dual boot only, huh... that sucks. I was really hoping I could use this on win10 or wsl(2)

[PyxAI]

I was however, able to run the model under WSL2 windows 11
Didn't check training, it's worth a shot @ReewassSquared

[uahmad235]

Hi @PyxAI . Which ubuntu version did you run this code on?