Re-train or fine tuning ESM-3 model

[anonimoustt]

Hi,

Is it possible to fine-tune or re-train the ESM-3 model with the protein sequence and structure or pdb file, and then apply the fine-tuned model to generate sequence or structure embeddings?

[luojc712]

I have the same question. If ESM3 can be fine-tuned, how should it be operated?

[AlpriElse]

@anonimoustt @luojc712 Here's a comment from the older, archived Meta ESM repo from someone on the team about finetuning:
facebookresearch/esm#33

It's an outdated comment but still relevant. The existing model does extend nn.Module and there are tutorials for other pre-trained models