Protein Structure · Meta FAIR
ESM2-3B
A 3-billion-parameter protein language model generating 1,280-dimensional sequence embeddings.
What ESM2-3B is
ESM2-3B is a 36-layer protein language model developed by Meta FAIR. It processes raw amino acid sequences using masked language modelling, enabling researchers to extract representations or fine-tune the architecture for downstream biological prediction tasks.
The model outputs 1,280-dimensional mean-pooled sequence embeddings. These representations provide the foundational inputs for downstream systems including the ESMFold structure prediction pipeline, the AminoWeb vector index, and various protein property predictors where embedding fidelity is prioritised over throughput.
How it works
ESM2-3B operates on protein sequences and is trained using a masked language modelling objective, learning to infer the identity of masked amino acids from surrounding context across 36 transformer layers. The model can be used for masked token prediction (fill-mask) or to produce a 1,280-dimensional mean-pooled vector representing the full protein sequence.
Key facts
- Parameters
- 3 billion
- Layers
- 36
- Input modality
- Protein sequence
- Output format
- 1,280-dimensional vector
- Training objective
- Masked language modelling
- GPU memory required
- 16 GB
What you can use it for
- Protein sequence embedding
- Extracting 1,280-dimensional mean-pooled numerical representations of protein sequences to train downstream property predictors.
- Structural biology modelling
- Serving as the base sequence representation module for structural models such as ESMFold.
- Variant effect prediction
- Analysing and scoring the potential consequences of amino acid substitutions across protein variants.
- Vector indexing
- Supplying sequence embeddings for similarity indexing and search platforms such as AminoWeb.
- Masked sequence infilling
- Predicting masked amino acid residues within incomplete sequences via fill-mask modelling.
Run it here
No installation and no GPU of your own: fill in the inputs and it runs on our servers. Results stay in your browser unless you sign in.
Runs on On-demand GPU
Runs on our servers with on-demand compute. A first run needs time to load the model; active capacity can be reused and scales down when idle.Run this model
ESM2-3B
Strengths and limits
Good at
- Generates high-quality 1,280-dimensional representations that serve as the substrate for ESMFold and property predictors.
- Demonstrates higher general accuracy compared to smaller ESM-2 checkpoints.
- Supports both zero-shot masked token inference and fine-tuning across sequence-based tasks.
Watch out for
- Requires 16 GB of GPU memory, demanding significantly more computing resources than smaller family checkpoints.
- Demands substantial inference time and memory, making it less suitable than smaller models for screening millions of sequences.
Inputs and outputs
You provide
Protein sequence
Single-letter amino acid code. Whitespace and FASTA headers are stripped automatically.
You get back
1,280-dimensional mean-pooled sequence representation.
Licence
Commercial use permitted · MIT
The MIT licence allows use in commercial research and products.
Questions
Can I use ESM2-3B commercially?
Commercial use permitted. The MIT licence allows use in commercial research and products.
How do I run ESM2-3B without installing anything?
Use the form under “Run it here” on this page: fill in the inputs and press Run model. It runs on our servers on CPU only, typically in 900ms, and a run costs about 6 credits. Visitors without an account get 1,000 credits a month.
What output does ESM2-3B produce for an input sequence?
It generates a 1,280-dimensional mean-pooled vector representing the protein sequence, as well as token-level predictions for masked language tasks.
How does ESM2-3B compare to smaller checkpoints such as ESM2-150M?
The 3B checkpoint yields higher embedding quality and accuracy, whereas smaller models like ESM2-150M are suited for screening millions of sequences due to lower resource requirements.
What hardware is necessary to run ESM2-3B?
The model requires 16 GB of GPU memory.
What objective was used to train the model?
ESM2-3B was trained using a masked language modelling objective.
How to cite it
Written from ESM2-3B's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.