Chemistry & ADMET · IBM Research
MolFormer
A chemical language transformer that extracts molecular embeddings from SMILES strings for property prediction.
What MolFormer is
MolFormer (also referred to as MoLFormer) is a chemical foundation model developed by IBM Research to process small molecules represented as SMILES (Simplified Molecular Input Line Entry System) sequences. By learning chemical language representations from very large molecular databases, it enables downstream predictive tasks and similarity assessments in cheminformatics.
The model addresses the challenge of capturing complex molecular structure and chemical features directly from string sequences. Its representations outperform traditional fingerprint baselines on molecular similarity searching and benchmark property prediction tasks across chemistry and drug discovery.
Researchers can use frozen MolFormer embeddings as feature vectors for downstream regression and classification models, or fine-tune the architecture directly on specific datasets such as toxicity, binding, or solubility measurements.
How it works
MolFormer employs a linear attention transformer architecture paired with rotary position embeddings (relative positional encodings) to efficiently encode chemical sequences. It is trained via masked language modelling in a self-supervised manner, learning compressed representations where attention values reflect spatial interatomic relationships.
The pre-training data originates from combinations of the ZINC15 and PubChem public chemical datasets, scaling up to 1.1 billion molecules. Prior to training, molecules were canonicalised using RDKit, isomeric information was stripped, and sequences longer than 202 tokens were removed. Pre-training was executed on 16 NVIDIA V100 GPUs.
Key facts
- Model architecture
- Linear attention Transformer with rotary embeddings
- Training objective
- Masked language modelling
- Training data
- ZINC15 and PubChem datasets (up to 1.1 billion molecules; variant trained on 10% ZINC + 10% PubChem)
- Hardware used for training
- 16 x NVIDIA V100 GPUs
- Required GPU memory
- 8 GB
- Maximum sequence length
- 202 tokens
- Input modality
- SMILES strings
- Output modality
- Embedding vector
What you can use it for
- Molecular feature extraction
- Extract frozen embedding vectors from ligand SMILES to visualise chemical space or train custom machine learning models.
- Molecular similarity searching
- Compute similarities between candidate compounds using latent encodings that outperform standard molecular fingerprint baselines.
- Biochemical property and toxicity prediction
- Fine-tune the model on classification tasks such as BBBP, HIV, BACE, ClinTox, Tox21, and SIDER.
- Physical and quantum chemical regression
- Fine-tune on regression benchmarks to predict properties like solubility, lipophilicity, and quantum chemical parameters (QM8 and QM9).
Run it here
No installation and no GPU of your own: fill in the inputs and it runs on our servers. Results stay in your browser unless you sign in.
Runs on On-demand GPU
Runs on our servers with on-demand compute. A first run needs time to load the model; active capacity can be reused and scales down when idle.Run this model
MolFormer
Strengths and limits
Good at
- Efficient linear attention paired with rotary embeddings allows rapid self-supervised sequence encoding.
- Produces representations that transfer to property prediction and similarity retrieval better than standard molecular fingerprints.
- Evaluated extensively across 11 MoleculeNet benchmark datasets spanning classification and regression tasks.
- Attention weights correlate with interatomic spatial distances within molecules.
Watch out for
- The model is not intended or trained for de novo molecule generation.
- Not tested on macromolecules or molecules larger than approximately 200 atoms.
- Using non-canonical or invalid SMILES strings may lead to reduced performance.
- Sequences exceeding 202 tokens were excluded during training and cannot be processed as trained.
Inputs and outputs
You provide
Ligand SMILES
Canonical SMILES for the small molecule.
You get back
Molecular representation.
Licence
Commercial use permitted · Apache-2.0
The Apache-2.0 licence allows use in commercial research and products.
Questions
Can I use MolFormer commercially?
Commercial use permitted. The Apache-2.0 licence allows use in commercial research and products.
How do I run MolFormer without installing anything?
Use the form under “Run it here” on this page: fill in the inputs and press Run model. It runs on our servers on 1× A100 16GB, typically in 600ms, and a run costs about 7 credits. Visitors without an account get 1,000 credits a month.
Can MolFormer generate new chemical structures?
No, MolFormer is designed for feature extraction, representation learning, and property prediction via fine-tuning; it is not intended for molecular generation.
What SMILES format should be supplied to the model?
Inputs should be valid canonical SMILES strings. RDKit was used to canonicalise molecules and remove isomeric information prior to training, and non-canonical or invalid inputs can degrade performance.
What is the maximum size of molecule the model can process?
The model has not been tested on macromolecules or structures containing more than roughly 200 atoms, and sequences longer than 202 tokens were dropped during training.
What GPU memory is required to run the model?
Running the model requires approximately 8 GB of GPU memory.
How to cite it
Ross, Jerret and Belgodere, Brian and Chenthamarakshan, Vijil and Padhi, Inkit and Mroueh, Youssef and Das, Payel. Large-scale chemical language representations capture molecular structure and properties. Nature Machine Intelligence 4, 1256–1264 (2022). https://doi.org/10.1038/s42256-022-00580-7
Written from MolFormer's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.