Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract and store embeddings from BioCLIP #10

Open
metazool opened this issue Jul 11, 2024 · 3 comments
Open

Extract and store embeddings from BioCLIP #10

metazool opened this issue Jul 11, 2024 · 3 comments
Labels
demonstrate feature that we need to be able to show future Future work that is not foreseeably going to considered, but can be revisited

Comments

@metazool
Copy link
Collaborator

metazool commented Jul 11, 2024

BioCLIP is a foundation model designed for classification of all or parts of biological images of all kinds, it could also be a useful source of embeddings that would generalise to a few projects

Aiming at running BioCLIP for the same outcome as #5 could be a great exercise for our regular pair programming session @albags ?

@metazool
Copy link
Collaborator Author

Searching an aerial photo with text queries – a demo and how it works - lovely blog writeup about the potential inherent in this approach, in this case doing text search from embeddings with a model called SkyCLIP that's fine-tuned on remote sensing imagery

@metazool
Copy link
Collaborator Author

https://imageomics.github.io/bioclip/ - BioCLIP paper

@metazool
Copy link
Collaborator Author

metazool commented Aug 29, 2024

https://github.com/Imageomics/bioclip/blob/main/docs/imageomics/BioCLIP_ModelCard.md - it has a model card, which is really good to see - something the Turing Inst projects should be consistently advocating for!

See also BioCLIP on HuggingFace https://huggingface.co/imageomics/bioclip

@metazool metazool added future Future work that is not foreseeably going to considered, but can be revisited demonstrate feature that we need to be able to show labels Sep 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
demonstrate feature that we need to be able to show future Future work that is not foreseeably going to considered, but can be revisited
Projects
None yet
Development

No branches or pull requests

1 participant