Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copies vit_jax.momentum_clip module. #246

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Copies vit_jax.momentum_clip module. #246

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Table of contents:
- [MLP-Mixer](#mlp-mixer)
- [Available Mixer models](#available-mixer-models)
- [Expected Mixer results](#expected-mixer-results)
- [LiT models](#lit-models)
- [LiT models](#lit-models)
- [Running on cloud](#running-on-cloud)
- [Create a VM](#create-a-vm)
- [Setup VM](#setup-vm)
Expand Down Expand Up @@ -296,11 +296,16 @@ ImageNet-21k | Mixer-L/16 | cifar10 | 98.34% | 10.0h | [tensorboard.

## LiT models

We have just published a post on the Google AI blog
[LiT: adding language understanding to image models](http://ai.googleblog.com/2022/04/locked-image-tuning-adding-language.html)
about our new CVPR paper "LiT: Zero-Shot Transfer with Locked-image text Tuning"
For details, refer to the Google AI blog post
[LiT: adding language understanding to image models](http://ai.googleblog.com/2022/04/locked-image-tuning-adding-language.html),
or read the CVPR paper "LiT: Zero-Shot Transfer with Locked-image text Tuning"
(https://arxiv.org/abs/2111.07991).

We published a Transformer B/16-base model with an ImageNet zeroshot accuracy of
72.1%, and a L/16-large model with an ImageNet zeroshot accuracy of 75.7%. For
more details about these models, please refer to the
[LiT model card](model_cards/lit.md).

We provide a in-browser demo with small text encoders for interactive use (the
smallest models should even run on a modern cell phone):

Expand Down
1 change: 1 addition & 0 deletions lit.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -706,6 +706,7 @@
"# # described in\n",
"# # https://www.tensorflow.org/datasets/catalog/imagenet2012\n",
"# # and then replace `data_dir` below with that GCS bucket.\n",
"# # If you get a `PermissionDeniedError`, try restarting the kernel.\n",
"# from google.colab import auth\n",
"# auth.authenticate_user() # Required to access access protected GCS buckets.\n",
"# builder = tfds.builder('imagenet2012', data_dir='gs://tensorflow-datasets/datasets')\n",
Expand Down