From 6cadf055f748688e423b9b4cc4db040501f37c54 Mon Sep 17 00:00:00 2001 From: Linsong Chu Date: Wed, 18 Dec 2024 10:22:28 -0500 Subject: [PATCH] Update training.md --- training/training.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/training/training.md b/training/training.md index 2919cef..fde1551 100644 --- a/training/training.md +++ b/training/training.md @@ -1,3 +1,5 @@ +# Training + We trained our Bamba model with FSDP using our training repo [here](https://github.com/foundation-model-stack/fms-fsdp/tree/mamba-new). Note that this training effort was started before FSDP2 and also long before we contributed `Mamba2-Hybrid` to HF, so we were doing FSDP1 training with [official Mamba implementation](https://github.com/state-spaces/mamba).