Issues with Koenig/Warszawksi Binding Affinity Data

[julianrfaust]

For the Koenig and Warszawski sets, I have been trying to find out what the fitness values actually represent. They are surely not KD values in nM as labeled. At least for the Koenig set, your 'fitness' seems to be positively correlated with binding affinity (unlike KD) so then I find myself confused by why the correlations with affinity were calculated with -np.log(fitness) - this seems like the wrong direction at least for the Koenig binding affinity data. I'll justify this in the third paragraph based on sanity checks from the Koenig manuscript data.

Rows for the wildtype do not exist in your csvs. I noticed that something called 'wildtype' was somehow plotted in some deleted plots in the commit history. However, in the plots that show wildtype against the rest, the thing labelled 'is_first_row' is deemed wildtype in the plots, but the sequence is clearly not actually the wildtype sequence. See the PDB fastas and code, csvs, plots I attached here that are pulled from this commit: 39fec5a. "wildtype" is not the correct wildtype, just seems like some mutated sequence labelled as wildtype. The wildtype sequences for Koenig:G.6.31 and Warszawski:D.44.1 can be inferred from looking at the different mutants or by looking at the respective PDB fastas (Koenig: https://www.rcsb.org/fasta/entry/2FJF/display, Warszawski:https://www.rcsb.org/fasta/entry/1MLC/display). The wildtype affinity values are supposed to be 0.36/0.4 nM for G.6.31 and 135/137 nM for D.44.1 (the stated values vary a bit even within the publications).

Based on the plot in the Koenig paper, the wildtype affinity (~0.36nM KD) should sit between that of light chain mutants I106V (KD of ~0.3, fitness of 1.993) and F83S (KD of ~0.4, fitness of 1.6879). That should put the wildtype 'fitness' for Koenig G.6.36 somewhere around 1.8. As a further check, the Koenig paper says that the G.6.31 F83A light chain mutant is supposed to have a binding affinity 3x greater than the wildtype - so approximately KD of 0.12 nM. The fitness value provided for that entry is 5.689, which is one of the higher fitness values and also around 3x of 1.8 which I'm ballparking as the wildtype 'fitness'. Fortunately this all checks out so the relative rankings seem to make sense, but this 'fitness' definitely seems positively correlated with binding affinity. I wasn't able to find the full Koenig data online and the source seems to be the early commit from Jeffrey Ruffolo in this repo. If someone could provide the original source or find it online for me - I can try to get the data in actual KD form myself and make a contribution.

Then for Warzawski, the fitness is also clearly not KD in nM as labeled. If 'fitness' were actually equal to KD in nM, then the wildtype fitness of 135nM would either be the strongest or weakest binder of the whole set which is obviously not the case based on the manuscript. At least I can find the original log-enrichment data online, so I can try processing it myself.

I wasn't interested in the other sets so I haven't checked those yet. But for Koenig and Warszawksi sets, I would like to warn everyone that these 'fitness' values absolutely cannot be interpreted as KD values in nM as labeled. For Koenig they actually seem to correlate with binding affinity in the other direction from KD. The exact relationship of these fitness values to KD is unclear to me right now.

There's something way back in the commit history called "all model scripts, still need to fix W_kd, K_Kd, K_er" so there seemed to be awareness of these issues at some point - but the fitness values don't seem to have changed since then. It might have fallen off the radar, but I'd like to put this out there for anyone else who tries to use this data so they can be aware of the headaches in advance.

Here's some relevant attachments as support for my points:
Koenig2017_g6_Kd_ppl.csv

Warszawski2019_d44_Kd_ppl.csv

002_score.py.txt

[KX-Wang1209]

Thank you for your meticulous review and kind reminder!

I encountered the same confusion in my research. As 'the largest therapeutic antibody design benchmark to date' as the authors claimed, I hope the authors can re-verify the accuracy of the data to avoid causing misunderstandings for future researchers.