Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up Model Conversion Script #131

Open
yeandy opened this issue Aug 16, 2024 · 2 comments
Open

Clean up Model Conversion Script #131

yeandy opened this issue Aug 16, 2024 · 2 comments
Assignees

Comments

@yeandy
Copy link
Contributor

yeandy commented Aug 16, 2024

Currently the model conversion script will create a bucket export MODEL_BUCKET=gs://${USER}-maxtext. However, it may be the case that the gs://${USER}-maxtext path already exists, which I imagine would break the script.

Solution: Be able to read in a few more arguments MODEL_BUCKET and BASE_OUTPUT_DIRECTORY. We should also delete references to DATASET_PATH.

@JoeZijunZhou
Copy link
Collaborator

If the bucket exists, the script will continue and use the existing ones IIRC. But feel free to refactor it to improve UX.

@yeandy
Copy link
Contributor Author

yeandy commented Aug 19, 2024

If the bucket exists, the script will continue and use the existing ones IIRC

Yes, but only if the current USER is the original creator/owner of bucket, right? A different user could have the same value for USER, which I think would break the workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants