-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.html
227 lines (197 loc) · 9.28 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="description"
content="Multimodal Neurons in Pretrained Text-Only Transformers">
<meta name="keywords" content="Interpretability, Transformers, Multimodal, NLP">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Multimodal Neurons in Pretrained Text-Only Transformers</title>
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
rel="stylesheet">
<link rel="stylesheet" href="./static/css/bulma.min.css">
<link rel="stylesheet" href="./static/css/bulma-carousel.min.css">
<link rel="stylesheet" href="./static/css/bulma-slider.min.css">
<link rel="stylesheet" href="./static/css/fontawesome.all.min.css">
<link rel="stylesheet"
href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css">
<link rel="stylesheet" href="./static/css/index.css">
<link rel="icon" href="./static/images/favicon.svg">
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
<script defer src="./static/js/fontawesome.all.min.js"></script>
<script src="./static/js/bulma-carousel.min.js"></script>
<script src="./static/js/bulma-slider.min.js"></script>
<script src="./static/js/index.js"></script>
</head>
<body>
<section class="hero">
<div class="hero-body">
<div class="container is-max-desktop">
<div class="columns is-centered">
<div class="column has-text-centered">
<h1 class="title is-1 publication-title">Multimodal Neurons in Pretrained Text-Only Transformers</h1>
<div class="is-size-5 publication-authors">
<span class="author-block">
<a href="https://cogconfluence.com">Sarah Schwettmann*</a><sup>1</sup>,</span>
<span class="author-block">
<a href="https://nchowdhury.com/">Neil Chowdhury*</a><sup>1</sup>,</span>
<span class="author-block">
<a href="https://cyber.harvard.edu/people/sklein">Samuel Klein</a><sup>2</sup>
<span class="author-block">
<a href="https://www.khoury.northeastern.edu/people/david-bau/">David Bau</a><sup>3</sup>
<span class="author-block">
<a href="https://groups.csail.mit.edu/vision/torralbalab/">Antonio Torralba</a><sup>1</sup>
</span>
</div>
* indicates equal contribution.
<div class="is-size-5 publication-authors">
<span class="author-block"><sup>1</sup>MIT CSAIL <sup>2</sup>MIT KFG <sup>3</sup>Northeastern University</span>
</div>
<div class="column has-text-centered">
<div class="publication-links">
<!-- PDF Link. -->
<span class="link-block">
<a href="http://www.cogconfluence.com/wp-content/uploads/2023/07/MMNs_2023.pdf"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fas fa-file-pdf"></i>
</span>
<span>Paper</span>
</a>
</span>
<span class="link-block">
<a href="https://arxiv.org/abs/2308.01544"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="ai ai-arxiv"></i>
</span>
<span>arXiv</span>
</a>
</span>
<!-- Code Link. -->
<span class="link-block">
<a href=""
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fab fa-github"></i>
</span>
<span>Code (coming soon)</span>
</a>
</span>
</div>
</div>
</div>
</div>
</div>
</div>
</section>
<section class="section" style="margin-top: -5px;">
<div class="container is-max-desktop">
<div style="border-style: solid; border-width: 1px; border-color: darkgray; padding: 20px; text-align: center;">
<p><h3 class="title is-4">If a model only learned to read and write, what can its neurons see?</h3></p>
<p>We detect and decode individual units in transformer MLPs that convert visual information into semantically related text. Joint visual and language supervision is not required for the emergence of multimodal neurons: we find these units in the case where only a single linear layer augments a text-only transformer with vision, using a frozen vision-only encoder.</p>
</div>
</div>
</section>
<section class="hero teaser" style="margin-top: -5px;">
<div class="container is-max-desktop">
<div class="hero-body">
<img source src="./static/figures/mmns_schematic.gif" />
</div>
</div>
</section>
<section class="section" style="margin-top: -20px;">
<div class="container is-max-desktop">
<div class="columns is-centered">
<p>For example, <b>MLP unit 9058</b> in <b>Layer 12</b> of GPT-J injects language related to <em>swimming, swim, fishes, water, Aqua</em> into the model's next token prediction, and is maximally active on images and regions containing water, across a diversity of visual scenes.</h3>
</div>
<br>
<div class="columns is-centered">
<img source src="./static/figures/L12_u9058.png" />
</div>
</div>
</section>
<section class="section">
<div class="container is-max-desktop">
<!-- Abstract. -->
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<h2 class="title is-3">Abstract</h2>
<div class="content has-text-justified">
<p>
Language models demonstrate remarkable capacity to generalize representations learned in one modality to downstream tasks in other modalities.
Can we trace this ability to individual neurons? We study the case where a frozen text transformer is augmented with vision using a
self-supervised visual encoder and a single linear adapter layer learned on an image-to-text task. Outputs of the adapter are not
immediately decodable into language describing image content; instead, we find that translation between modalities occurs deeper
within the transformer. We introduce a procedure for identifying "multimodal neurons" that convert visual representations into
corresponding text, and decoding the concepts they inject into the model’s residual stream. In a series of experiments, we show
that multimodal neurons operate on specific visual concepts across inputs, and have a systematic causal effect on image captioning.
</p>
</div>
</div>
</div>
<!--/ Abstract. -->
</div>
</section>
<section class="section" style="margin-top: -5px;">
<div class="container is-max-desktop">
<div class="columns is-centered">
<h3 class="title is-4">Example multimodal neurons in GPT-J:</h3>
</div>
</div>
</section>
<section class="hero teaser">
<div class="container is-max-desktop">
<div class="hero-body">
<img source src="./static/figures/example_units.png" />
</div>
</div>
<section class="section" style="margin-top: -5px;">
<div class="container is-max-desktop">
<div style="padding: 20px; text-align: left;">
<h3 class="title is-4">How do we detect multimodal neurons: units that convert image semantics into related text?</h3>
<ul style="list-style-type: disc; padding-left: 0;">
<li>Score each unit’s contribution to an image captioning task by linearly approximating its effect on model output.</li>
<li>Decode that text using the transformer unembeddings.</li>
</ul>
</div>
</div>
</section>
<section class="hero teaser">
<div class="container is-max-desktop">
<div class="hero-body">
<figure>
<img src="./static/figures/main_decoding_heatmap.png" />
<figcaption style="font-size: 0.8em;"> Top five multimodal neurons (layer L, unit u), for 12 sample images. Superimposed heatmaps (0.95
percentile of activations) show mean activations of the top five neurons over the image. The two highest-probability decoded tokens are shown for each neuron</figcaption>
</figure>
</div>
</div>
</section>
</section>
<section class="section" id="BibTeX">
<div class="container is-max-desktop content">
<h2 class="title">BibTeX</h2>
<pre><code>@inproceedings{schwettmann2023multimodal,
title={Multlimodal Neurons in Pretrained Text-Only Transformers},
author={Schwettmann, Sarah and Chowdhury, Neil and Klein, Samuel and Bau, David and Torralba, Antonio},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
pages={2862--2867},
year={2023}
}</code></pre>
</div>
</section>
<footer class="footer">
<div class="columns is-centered">
<div class="column is-8">
<div class="content" style="text-align: center;">
<p>
This website is adapted from the <a href="https://github.com/nerfies/nerfies.github.io">Nerfies template</a>, which you are free to borrow if you link back to it in the footer.
</p>
</div>
</div>
</div>
</div>
</footer>
</body>
</html>