-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cross-Validation & Regularization Technique Implemented #10
Conversation
@sohambuilds please review this |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add 10-15 generation samples post fine-tuning within your directory, It is suggested but not compulsory, to use another set of images for your data(could be any character or object), and finetune on that. If possible, include CLIP scores or TIFA scores to evaluate performance.
Ensure that based on the number of images in the dataset, you adjust your learning rate. 1e-5 is recommended for small datasets. of 10-20 images. 1e-4 is not appropriate.
Early stopping should not be implemented for few-shot learning/finetuning.
Thank you for the feedback. I will attempt to make the suggested changes. However, my laptop does not have a powerful GPU, so I’ve been training on CPU, which takes approximately ~40 minutes per epoch for each k-fold. Unfortunately, I've also faced issues with both Colab and Kaggle—Colab had GPU memory overflow problems, and Kaggle produced a different error. I’ll continue troubleshooting these issues, but due to these constraints, progress may be slower. |
Running on colab should be possible, as we have tried it. If there is a specific issue that you need to troubleshoot, you may join the WhatsApp group for the ML contributors: https://chat.whatsapp.com/Kx8okfEdirALcC8UeFtl5j Also, do note that implementing early stopping for few shot learning(very few samples) is not desirable. |
Got it. 👍 |
I'm trying to run this notebook: https://colab.research.google.com/drive/1Zv6eLFRHovlJgxumTtPozstIqT8cBJ-I?usp=sharing
|
Please run the following in your colab runtime: |
Please commit the updated code, and your dataset, if you're using a new one. You can add the samples later, I will merge it for now. |
…scoring, and update optimizer learning rate
I managed to run the notebook: https://colab.research.google.com/drive/1Zv6eLFRHovlJgxumTtPozstIqT8cBJ-I?usp=sharing in T4 GPU mode by optimizing the GPU's memory management during the model training in the Please review the changes and let me know if there's anything that needs improvement, @sohambuilds. |
val_loss, val_clip_score = validate(val_loader, unet, text_encoder, vae, noise_scheduler, device, pipe) | ||
print(f"Epoch {epoch+1}/{num_epochs}, Validation Loss: {val_loss:.4f}, Validation CLIP Score: {val_clip_score:.4f}") | ||
|
||
# Check for early stopping |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove early stopping as requested earlier. early stopping is implemented in large datasets to check for overfitting. Not in the case of only 10 images. model may need more than a patience of 5 to learn in such cases.
Good to know that you were able to make it run on colab.
…ied training process
Merging now. Sample generation is not proper. need to check. |
Description
File:
main.py
Added K-Fold Cross-Validation:
optimizer = torch.optim.AdamW(unet.parameters(), lr=1e-4, weight_decay=1e-5)
File:
lora.py
Added Dropout to LoRA Layers:
File:
train.py
Implemented Early Stopping:
Fixes # (#9 #8 #7 )
Type of Change
Please delete options that are not relevant.
Checklist:
Additional Information
@sohambuilds