Multi-Modal Foundation · mist-models
MIST 1.8B · U0
A 1.8-billion-parameter transformer predicting molecular internal energy at 0 K from SMILES strings.
What MIST 1.8B · U0 is
MIST 1.8B · U0 is a molecular property prediction model developed by the Electrochemical Energy Group at the University of Michigan. It belongs to the Molecular Insight SMILES Transformers (MIST) family, a suite of foundation models designed to process chemical structures represented as text strings.
This model variant is specifically fine-tuned to predict the internal energy at 0 K (U0) of small organic molecules in Hartree, based on the QM9 benchmark from MoleculeNet. U0 represents the total internal energy of a molecule at absolute zero temperature, accounting for electronic, nuclear, and zero-point vibrational energies.
How it works
The underlying architecture is a RoBERTa-PreLayerNorm encoder with 24 layers, a hidden dimension of 2304, 18 attention heads, an intermediate size of 9216, and a context window of 2048 tokens. The base encoder was pre-trained on Kekulized SMILES from the Enamine REAL Space dataset—a vast collection of synthetically accessible compounds—using a masked language modelling objective with cross-entropy loss and the FusedLAMB optimiser on 32 NVIDIA A100 GPUs.
For property prediction, the encoder is paired with a task network composed of a two-layer multi-layer perceptron using Gaussian Error Linear Unit (GELU) activations and dropout. The representation is formed by pooling the hidden state of the first token. The model was fine-tuned on QM9 using mean squared error loss and the AdamW optimiser on a single NVIDIA A100 GPU.
Key facts
- Encoder architecture
- RoBERTa-PreLayerNorm (24 layers, hidden size 2304, 18 heads)
- Task network
- Two-layer MLP with GELU activations and dropout
- Parameter count
- 1.8 billion
- Context length
- 2048 tokens
- Pre-training data
- Enamine REAL Space
- Fine-tuning data
- QM9 (Internal Energy at 0 K)
- Prediction target
- Internal energy at 0 K (U0) in Hartree
What you can use it for
- Quantum chemical property estimation
- Predicting the zero-temperature internal energy (U0) of organic molecules directly from SMILES representations without running density functional theory calculations.
- High-throughput thermodynamic screening
- Rapidly filtering and ranking molecular candidates within chemical libraries based on predicted ground-state energetic stability.
- Chemical feature extraction
- Generating dense molecular embedding vectors from the pooled first-token representations of the 1.8-billion-parameter encoder.
Run it here
No installation and no GPU of your own: fill in the inputs and it runs on our servers. Results stay in your browser unless you sign in.
Runs on On-demand GPU
Runs on our servers with on-demand compute. A first run needs time to load the model; active capacity can be reused and scales down when idle.Run this model
MIST 1.8B · U0
Strengths and limits
Good at
- Pre-trained on Enamine REAL Space, capturing the chemistry of synthetically accessible molecular structures.
- Supports input sequences up to 2048 tokens, accommodating large and complex SMILES expressions.
- Produces direct numerical predictions for molecular internal energy in standard Hartree units.
Watch out for
- Fine-tuning is restricted to the QM9 dataset, limiting direct validation to small organic molecules.
- Tokenisation relies on the SMIRK tokenizer library, which requires a working Rust installation.
- Expects input molecules in Kekulized SMILES format rather than alternate molecular graph or coordinate structures.
Inputs and outputs
You provide
Sequence or text
A protein or nucleotide sequence, or free text — whichever this model was trained on.
You get back
Mean-pooled representation, one vector per input.
Licence
Check the licence before commercial use · See model card
The See model card licence carries terms worth reading before you depend on it.
Questions
Can I use MIST 1.8B · U0 commercially?
Check the licence before commercial use. The See model card licence carries terms worth reading before you depend on it.
How do I run MIST 1.8B · U0 without installing anything?
Use the form under “Run it here” on this page: fill in the inputs and press Run model. It runs on our servers on 1× A100 16GB, typically in 12s, and a run costs about 70 credits. Visitors without an account get 1,000 credits a month.
What property does this model predict?
It predicts the internal energy at 0 K (U0) in Hartree, which includes the electronic, nuclear, and zero-point vibrational energy of the molecule.
What format should the molecular inputs use?
Inputs must be SMILES strings, specifically formatted as Kekulized SMILES.
What dependencies are required to run inference?
The model uses the transformers library and the SmirkTokenizerFast class from the smirk package, which requires Rust to be installed.
How was the model evaluated during development?
Performance on U0 prediction was assessed using Mean Absolute Error (MAE) across a random 80/10/10 train-validation-test split of the QM9 dataset.
How to cite it
Wadell, Alexius and Bhutani, Anoushka and Azumah, Victor and Ellis-Mohr, Austin R. and Kelly, Celia and Zhao, Hancheng and Nayak, Anuj K. and Hegazy, Kareem and Brace, Alexander and Lin, Hongyi and Emani, Murali and Vishwanath, Venkatram and Gering, Kevin and Alkan, Melisa and Gibbs, Tom and Wells, Jack and Varshney, Lav R. and Ramsundar, Bharath and Duraisamy, Karthik and Mahoney, Michael W. and Ramanathan, Arvind and Viswanathan, Venkatasubramanian. Foundation Models for Discovery and Exploration in Chemical Space. arXiv:2510.18900 [physics], 2025. https://doi.org/10.48550/arXiv.2510.18900
Written from MIST 1.8B · U0's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.