extend vit and add mae model and finetune checkpoint file #707
Conversation
sageyou
force-pushed
the
pr_ssl
branch
4 times, most recently
from
aad2727
to
7cc9268
Compare
|
@SamitHuang @geniuspatrick please review |
mindcv/models/helpers.py
Outdated
| @@ -198,3 +202,105 @@ def build_model_with_cfg( | |||
| raise RuntimeError(f"`feature_only` is not implemented for `{model_cls.__name__}` model.") from e | |||
|
|
|||
| return model | |||
|
|
|||
|
|
|||
| def interpolate_relative_position_bias(checkpoint_params, network): | |||
|
原本的vit 是否可以提供vit encoder? 不单独搞vit_encoder.py @sageyou @geniuspatrick |
mae可以实现，但是会麻烦 结构都要动。 |
…e; Checkpoint updated
|
已将ViT Encoder加入到原vit中，请Review并rebase 该PR： #733 |
sageyou
changed the title
sageyou
force-pushed
the
pr_ssl
branch
2 times, most recently
from
e063900
to
0b52806
Compare
|
review plz, @SamitHuang @geniuspatrick |
|
LGTM. BTW, have the new weights been verified? |
@geniuspatrick Yes, I have validated all of them, including vit and mae. |
mindcv/models/layers/patch_embed.py
Outdated
| B, C, H, W = x.shape | ||
| if self.image_size is not None: | ||
| if self.strict_img_size: | ||
| assert H == self.image_size[0], f"Input height ({H}) doesn't match model ({self.image_size[0]})." |
There was a problem hiding this comment.
assert in cell.construct is not recommended for MS+910B, which will lower the performance.
sageyou
force-pushed
the
pr_ssl
branch
3 times, most recently
from
8d1e233
to
467001a
Compare
Thank you for your contribution to the MindCV repo.
Before submitting this PR, please make sure:
Motivation
Test Plan
(How should this PR be tested? Do you require special setup to run the test or repro the fixed bug?)
Related Issues and PRs
#733
(Is this PR part of a group of changes? Link the other relevant PRs and Issues here. Use https://help.github.com/en/articles/closing-issues-using-keywords for help on GitHub syntax)