Protein Structure · Meta FAIR
ESM2-650M
Generates 1,280-dimensional embeddings and masked residue predictions from protein sequences.
What ESM2-650M is
ESM2-650M is a 650-million-parameter protein language model developed by Meta FAIR. It is designed to analyse raw amino acid sequences and output either high-dimensional representations or predictions for masked positions within a sequence.
The model is positioned as an intermediate configuration within the ESM-2 family. It offers representational quality comparable to the larger 3-billion-parameter checkpoint across most functional tasks, whilst operating with substantially lower memory requirements and inference latency.
How it works
The architecture consists of a 33-layer transformer trained on protein sequences using a masked language modelling objective. During training, portions of an amino acid sequence are masked, and the model learns to infer the identity of the missing residues from the surrounding sequence context.
For sequence-level tasks, the model produces a 1,280-dimensional mean-pooled vector representation of the input protein. It can also perform token-level predictions directly using its masked language modelling head.
Key facts
- Parameters
- 650M
- Layers
- 33
- Input modality
- Protein sequence
- Output embedding dimension
- 1,280
- GPU memory requirement
- 8 GB
- Release date
- 2022-09-27
What you can use it for
- High-Throughput Sequence Embedding
- Extracting 1,280-dimensional mean-pooled embeddings for large protein libraries where low latency and smaller memory footprints are required.
- Functional Property Prediction
- Fine-tuning on supervised downstream prediction tasks that take protein sequences as input.
- Masked Residue Infilling
- Predicting likely amino acid identities at specific masked positions using the masked language modelling capability.
Run it here
No installation and no GPU of your own: fill in the inputs and it runs on our servers. Results stay in your browser unless you sign in.
Runs on Managed endpoint · feature-extraction
Runs on managed compute. Availability and startup time depend on current capacity.Run this model
ESM2-650M
Strengths and limits
Good at
- Provides functional task performance close to the 3-billion-parameter model at a fraction of the memory and latency.
- Requires only 8 GB of GPU memory, allowing large-scale batch processing on standard accelerator hardware.
- Generates unified 1,280-dimensional sequence embeddings compatible with downstream machine learning workflows.
Watch out for
- Attains somewhat lower accuracy on tasks compared to the larger 3-billion and 15-billion parameter ESM-2 checkpoints.
- Operates exclusively on primary sequence inputs and does not directly incorporate experimental 3D coordinate data.
Inputs and outputs
You provide
Protein sequence
Single-letter amino acid code. Whitespace and FASTA headers are stripped automatically.
You get back
1,280-dimensional mean-pooled sequence representation.
Licence
Commercial use permitted · MIT
The MIT licence allows use in commercial research and products.
Questions
Can I use ESM2-650M commercially?
Commercial use permitted. The MIT licence allows use in commercial research and products.
How do I run ESM2-650M without installing anything?
Use the form under “Run it here” on this page: fill in the inputs and press Run model. It runs on our servers on CPU only, typically in 320ms, and a run costs about 2 credits. Visitors without an account get 1,000 credits a month.
What is the primary output format of the model?
It outputs a 1,280-dimensional mean-pooled vector representing the input sequence, or token-level predictions for masked amino acids.
How does this checkpoint compare to larger ESM-2 models?
It approaches the quality of the 3-billion-parameter checkpoint on most functional prediction tasks while requiring significantly less memory and execution time.
What training objective does ESM2-650M use?
It is trained using a masked language modelling objective on protein sequences.
How much GPU memory is needed to run the model?
The model requires 8 GB of GPU memory.
How to cite it
Written from ESM2-650M's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.