Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add pretrain and finetune pipeline for mae #749

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

sageyou
Copy link
Collaborator

@sageyou sageyou commented Nov 27, 2023

Motivation

  1. add pretrain and finetune pipeline of mae
  2. update vit: initialization method and pre-scale of attention

Test Plan

(How should this PR be tested? Do you require special setup to run the test or repro the fixed bug?)

Related Issues and PRs

(Is this PR part of a group of changes? Link the other relevant PRs and Issues here. Use https://help.github.com/en/articles/closing-issues-using-keywords for help on GitHub syntax)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant