Multi-Modal Foundation · mist-models
MIST 28M base
A 28M-parameter molecular language model pretrained on SMILES strings for molecular property prediction.
What MIST 28M base is
MIST 28M base (mist-models-mist-28m-ti624ev1) is a molecular foundation model developed by the Electrochemical Energy Group at the University of Michigan, Ann Arbor. It belongs to the Molecular Insight SMILES Transformers (MIST) family, designed to provide pretrained molecular representations as a starting point for downstream property-prediction fine-tuning.
The model addresses the challenge of exploring synthetically accessible chemical space for applications such as drug candidates, electrolytes, and fragrances. By learning representations across billions of chemical tokens, it provides a general encoder that can be adapted to regression and classification tasks across diverse chemical datasets.
How it works
MIST 28M base uses a RoBERTa-PreLayerNorm encoder architecture comprising 8 layers, 8 attention heads, a hidden size of 512, an intermediate size of 2048, and a maximum sequence length of 2048 tokens. The model was trained using the Masked Language Modeling (MLM) objective with cross-entropy loss, optimised with DeepSpeed's FusedLAMB optimizer.
Training was conducted on approximately 14 billion molecular SMILES tokens derived from the Enamine REAL Space database, constructed via forward synthetic analysis of experimentally validated building blocks. Inputs must be Kekulized SMILES strings (explicitly representing alternating single and double bonds) and tokenized using the SMIRK tokenizer.
Key facts
- Architecture
- RoBERTa-PreLayerNorm
- Parameters
- 28M
- Layers
- 8
- Attention heads
- 8
- Hidden size
- 512
- Intermediate size
- 2048
- Maximum sequence length
- 2048
- Training data
- Enamine REAL Space (~14B SMILES tokens)
- Training objective
- Masked Language Modeling (MLM)
- Training hardware
- 2 NVIDIA A100-SXM4-80GB GPUs (~12 hours and 15 minutes)
What you can use it for
- Molecular representation and zero-shot similarity
- Extract embedding vectors from the first token of the last hidden state for Kekulized SMILES strings, enabling cosine similarity comparisons between molecules.
- Fine-tuning for quantum chemical property regression
- Train downstream regression heads on top of the encoder to predict molecular properties such as LUMO energy levels from datasets like QM9.
- Masked language modelling fill-mask tasks
- Predict masked tokens within input molecular sequences using the fill-mask pipeline.
Run it here
No installation and no GPU of your own: fill in the inputs and it runs on our servers. Results stay in your browser unless you sign in.
Runs on On-demand GPU
Runs on our servers with on-demand compute. A first run needs time to load the model; active capacity can be reused and scales down when idle.Run this model
MIST 28M base
Strengths and limits
Good at
- Pretrained on the massive, synthetically accessible Enamine REAL Space library spanning approximately 14 billion tokens.
- Compact 28M parameter footprint allows rapid fine-tuning and low computational overhead.
- Pre-LayerNorm RoBERTa architecture trained efficiently using LAMB optimization and DeepSpeed parallelism.
Watch out for
- Requires input SMILES strings to be strictly Kekulized prior to tokenization.
- Requires Rust to be installed in the environment to build the SMIRK tokenizer dependencies.
- Represents molecules purely as 1D SMILES strings without explicit 3D conformer coordinates.
Inputs and outputs
You provide
Sequence with a mask
Use the model's mask token, commonly [MASK] or <mask>.
You get back
Labels with confidence scores, highest first.
Licence
Check the licence before commercial use · See model card
The See model card licence carries terms worth reading before you depend on it.
Questions
Can I use MIST 28M base commercially?
Check the licence before commercial use. The See model card licence carries terms worth reading before you depend on it.
How do I run MIST 28M base without installing anything?
Use the form under “Run it here” on this page: fill in the inputs and press Run model. It runs on our servers on CPU only, typically in 12s, and a run costs about 28 credits. Visitors without an account get 1,000 credits a month.
What format must input molecules take?
Inputs must be Kekulized SMILES strings, which can be generated using cheminformatics toolkits like RDKit before passing them to the tokenizer.
Which tokenizer is required for MIST 28M?
It relies on the SMIRK tokenizer (SmirkTokenizerFast), which requires Rust to be installed in the operating environment.
How do I extract a single vector embedding for a molecule?
Run the input through the encoder and pool the representation at the first token position (index 0) of the last hidden state.
How to cite it
Wadell, Alexius and Bhutani, Anoushka and Azumah, Victor and Ellis-Mohr, Austin R. and Kelly, Celia and Zhao, Hancheng and Nayak, Anuj K. and Hegazy, Kareem and Brace, Alexander and Lin, Hongyi and Emani, Murali and Vishwanath, Venkatram and Gering, Kevin and Alkan, Melisa and Gibbs, Tom and Wells, Jack and Varshney, Lav R. and Ramsundar, Bharath and Duraisamy, Karthik and Mahoney, Michael W. and Ramanathan, Arvind and Viswanathan, Venkatasubramanian. Foundation Models for Discovery and Exploration in Chemical Space. arXiv:2510.18900 [physics], 2025. https://doi.org/10.48550/arXiv.2510.18900
Written from MIST 28M base's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.