Update README.md

captum/optim/README.md

@@ -1,13 +1,85 @@
-# Captum "optim" module
+Overview
+=================
 
-This is a WIP PR to integrate existing feature visualization code from the authors of `tensorflow/lucid` into captum.
-It is also an opportunity to review which parts of such interpretability tools still feel rough to implement in a system like PyTorch, and to make suggestions to the core PyTorch team for how to improve these aspects.
+About Optim
+-----------------
 
-## Roadmap
+The Optim module is a set tools for optimization based interpretability for neural networks. It is a continuation of the research work performed by the team behind the [tensorflow/lucid](https://github.com/tensorflow/lucid) library.
 
-* unify API with Captum API: a single class that's callable per "technique"(? check for details before implementing)
-* Consider if we need an abstraction around "an optimization process" (in terms of stopping criteria, reporting losses, etc) or if there are sufficiently strong conventions in PyTorch land for such tasks
-* integrate Eli's FFT param changes (mostly for simplification) 
-* make a table of PyTorch interpretability tools for readme?
-* do we need image viewing helpers and io helpers or throw those out?
-* can we integrate paper references closer with the code?
+
+The Optim module is designed to be extremely customizable, as to avoid limitations in its research potential.
+
+History
+-----------------
+
+The initial concept for the Optim module was devised by Ludwig Shubert, and then developed by Ben Egan and Swee Kiat Lim with help from Chris Olah & Narine Kokhlikyan.
+
+
+Optim Structure
+-----------------
+
+![](https://user-images.githubusercontent.com/10626398/177629584-33e7ff7c-a504-404e-a7ab-d8d786b7e25a.svg?sanitize=true)
+
+The standard rendering process works like this for the forward pass, with loss objectives being able to target any of the steps:
+
+* ``NaturalImage`` (``ImageParameterization`` ➔ ``ToRGB`` ➔ Squash Function ➔ ``ImageTensor``) ➔ Transforms ➔ Model
+
+
+Parameterizations
+-----------------
+
+The default settings store image parameters in a fully decorrelated format where the spatial information and channel information is decorrelated. By preconditioning our optimizer with decorrelated data, we alter the loss landscape to make optimization significantly easier and decrease the presence of high frequency patterns. Parameterizations like these are also known as a differentiable image parameterizations.
+
+![](https://user-images.githubusercontent.com/10626398/176753493-b90f4e18-0133-4dca-afd4-26e811aa965e.svg?sanitize=true)
+
+* Decorrelated Data ➔ Recorrelate Spatial ➔ Recorrelate Color ➔ Squash Function ➔ Transforms ➔ Model
+
+By default, recorrelation occurs entirely within the ``NaturalImage`` class.
+
+
+Submodules
+-----------------
+
+**Reducer**: The reducer module makes it easy to perform dimensionality reduction with a wide array of algorithms like [t-SNE](https://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html), [UMAP](https://umap-learn.readthedocs.io/en/latest/), [PCA](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html), & [NMF](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.NMF.html).
+
+**Circuits**: The circuits module allows for the extraction of meaningful weight interactions from between neurons which aren’t literally adjacent in a neural network.
+
+**Models**: The models module contains the model zoo of pretrained models along with various help functions and classes.
+
+**Dataset**: The dataset module provides functions for calculating color correlation matrices of image datasets.
+
+
+Docs
+-----------------
+
+The docs for the optim module can be found [here](https://captum.ai/api/).
+
+
+Tutorials
+-----------------
+
+We also provide multiple tutorials covering a wide array of research for the optim module [here](https://captum.ai/tutorials/).
+
+
+References
+-----------------
+
+* Feature Visualization: https://distill.pub/2017/feature-visualization/
+
+* Differentiable Image Parameterizations: https://distill.pub/2018/differentiable-parameterizations/
+
+* The Building Blocks of Interpretability: https://distill.pub/2018/building-blocks/
+
+* Exploring Neural Networks with Activation Atlases: https://distill.pub/2019/activation-atlas/
+
+* Understanding Deep Image Representations by Inverting Them: https://arxiv.org/abs/1412.0035
+
+* Color information for region segmentation: https://www.sciencedirect.com/science/article/pii/0146664X80900477
+
+* Thread: Circuits: https://distill.pub/2020/circuits/
+
+  * Visualizing Weights: https://distill.pub/2020/circuits/visualizing-weights/
+
+  * Weight Banding: https://distill.pub/2020/circuits/weight-banding/
+
+* Multimodal Neurons in Artificial Neural Networks: https://distill.pub/2021/multimodal-neurons/