Clip-Plot

🚧 Work in progress

The goal of this repo is to modify pix-plot to work as a front end for exploring any image embeddings, for example those from clip_retrieval.

If you want to go directly from a folder of JPGs to a PixPlot visualization, you should use pix-plot itself.

This is merely a work-in-progress attempt to enable you to generate a PixPlot visualization after creating embeddings.

Why do this? 1. clip gives useful embeddings as shown by the first place solution in the recent “Google Universal Image Embeddings” Kaggle competition 2. Separating the embeddings step gives more flexibility in how it is run – for example, it can be run in parallel on several nodes 3. Separating the embeddings step also makes it easier to remove pinned dependencies so the exciting viz stuff is easier to install

Initial ambitions coming from PixPlot:

• ☑️ Unpin requirements versions
• ☑️ Remove all TensorFlow dependencies
• ☑️ Enable to go directly from what you’d expect to have after running clip-retrieval inference: i.e. a folder of JPGs and a folder with embeddings in numpy format

Later ambitions

• ⬛ Add a linear dimensionality reduction method to complement UMAP
• ⬛ Include option to output as a desktop app via Tauri
• ☑️ BYO logo

Acknowledgements

clip-plot is a fork of pix-plot by Douglas Duhaime, Peter Leonard, and others, and is mainly a lighter, smaller version of their work.

The DHLab would like to thank Cyril Diagne and Nicolas Barradeau, lead developers of the spectacular Google Arts Experiments TSNE viewer, for generously sharing ideas on optimization techniques used in this viewer, and Lillianna Marie for naming this viewer PixPlot.