Multi-Modal Foundation · mist-models
MIST 1.8B base
A 1.8-billion-parameter foundation model using masked language modelling on molecular SMILES for chemistry prediction.
What MIST 1.8B base is
MIST 1.8B base (Molecular Insight SMILES Transformers) is a large molecular language model designed for chemical property prediction. Created by the Electrochemical Energy Group at the University of Michigan, Ann Arbor, the foundation model learns generalized molecular representations from vast collections of chemical structures.
The model addresses the challenge of mapping chemical space for scientific tasks such as evaluating drug candidates, battery electrolytes, and fragrances. By learning self-supervised molecular representations, it provides a base encoder that can be adapted to diverse downstream predictive tasks.
Trained on large-scale synthetically accessible compound libraries, MIST produces embeddings suitable for measuring molecular similarity or serving as the base for supervised fine-tuning across quantitative property prediction benchmarks.
How it works
The model processes molecules represented as Kekulised SMILES (Simplified Molecular Input Line Entry System) strings, in which alternating single and double bonds are explicitly assigned.
The pre-training objective is masked language modelling (MLM) optimized via cross-entropy loss and the DeepSpeed FusedLAMB optimizer. Training was performed using PyTorch Lightning with DeepSpeed data-distributed parallelism across 32 NVIDIA A100-SXM4-80GB GPUs over roughly 7 days, covering approximately 116 billion molecular SMILES tokens derived from the Enamine REAL Space dataset.
Key facts
- Architecture
- RoBERTa-PreLayerNorm encoder (28 layers, 18 attention heads)
- Maximum sequence length
- 2,048 tokens
- Pre-training data
- ~116B molecular SMILES tokens from Enamine REAL Space
- Pre-training objective
- Masked language modelling (MLM)
- Optimizer
- deepspeed.ops.lamb.FusedLAMB
- Training hardware
- 32 NVIDIA A100-SXM4-80GB GPUs (~7 days)
What you can use it for
- Molecular representation and embedding extraction
- Extract the pooled representation from the first token ([CLS]) of the encoder's last hidden state to represent chemical structures for downstream analysis.
- Zero-shot molecular similarity search
- Compute cosine similarity between molecular embedding vectors to rank and screen compounds relative to a target query molecule.
- Supervised fine-tuning for property prediction
- Attach task-specific heads (such as a multi-layer perceptron regression head) onto the encoder to predict experimental or quantum chemical values like LUMO energy.
- Fill-mask molecular token prediction
- Perform masked language modeling inference over corrupted or masked SMILES strings to generate token distributions with confidence scores.
Run it here
No installation and no GPU of your own: fill in the inputs and it runs on our servers. Results stay in your browser unless you sign in.
Runs on On-demand GPU
Runs on our servers with on-demand compute. A first run needs time to load the model; active capacity can be reused and scales down when idle.Run this model
MIST 1.8B base
Strengths and limits
Good at
- Pre-trained on roughly 116 billion tokens from Enamine REAL Space, capturing synthetically accessible chemical regions.
- Employs a RoBERTa-PreLayerNorm architecture supporting long context lengths of up to 2,048 tokens.
- Weights are exported in standard Safetensors format and integrated with the Hugging Face Transformers ecosystem.
Watch out for
- Input SMILES strings must be strictly converted into Kekulised form before tokenization to match the pre-training data.
- SMIRK tokenization requires a Rust environment to be installed.
- Served deployment requires substantial GPU memory, listed at 24 GB.
Inputs and outputs
You provide
Sequence with a mask
Use the model's mask token, commonly [MASK] or <mask>.
You get back
Labels with confidence scores, highest first.
Licence
Check the licence before commercial use · See model card
The See model card licence carries terms worth reading before you depend on it.
Questions
Can I use MIST 1.8B base commercially?
Check the licence before commercial use. The See model card licence carries terms worth reading before you depend on it.
How do I run MIST 1.8B base without installing anything?
Use the form under “Run it here” on this page: fill in the inputs and press Run model. It runs on our servers on 1× A100 16GB, typically in 12s, and a run costs about 70 credits. Visitors without an account get 1,000 credits a month.
What input format does MIST 1.8B base require?
The model takes SMILES strings that have been converted into Kekulised form using RDKit before passing them to the tokenizer.
How are molecular embeddings extracted for similarity calculations?
Embeddings are extracted by passing tokenized Kekulised SMILES through the encoder and taking the output from the first token ([CLS]) of the last hidden state.
What dataset was used to pre-train the model?
It was pre-trained on approximately 116 billion tokens from the Enamine REAL Space dataset, constructed via forward synthetic analysis of validated building blocks and synthons.
Which software dependencies are needed to run tokenization?
Tokenization uses the smirk package (SmirkTokenizerFast), which requires a functioning Rust installation on the host system.
How to cite it
Wadell, Alexius and Bhutani, Anoushka and Azumah, Victor and Ellis-Mohr, Austin R. and Kelly, Celia and Zhao, Hancheng and Nayak, Anuj K. and Hegazy, Kareem and Brace, Alexander and Lin, Hongyi and Emani, Murali and Vishwanath, Venkatram and Gering, Kevin and Alkan, Melisa and Gibbs, Tom and Wells, Jack and Varshney, Lav R. and Ramsundar, Bharath and Duraisamy, Karthik and Mahoney, Michael W. and Ramanathan, Arvind and Viswanathan, Venkatasubramanian. Foundation Models for Discovery and Exploration in Chemical Space. arXiv:2510.18900 [physics], 2025. https://doi.org/10.48550/arXiv.2510.18900
Written from MIST 1.8B base's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.