Multi-Modal Foundation · mist-models
MIST 28M · Tox21
A 28-million parameter transformer predicting 12 toxicity endpoints from molecular SMILES strings.
What MIST 28M · Tox21 is
MIST 28M · Tox21 is a molecular property prediction model developed by the Electrochemical Energy Group at the University of Michigan, Ann Arbor. It builds on the Molecular Insight SMILES Transformers (MIST) family of molecular foundation models, designed to learn representations from chemical space and perform downstream biochemical predictions.
The model addresses the challenge of identifying toxicological hazards in small molecules. Specifically, it performs multi-label binary classification across 12 biological endpoints covering nuclear receptor signaling and cellular stress-response pathways from the MoleculeNet Tox21 benchmark.
By evaluating chemical structures directly from text-based chemical notation, the model allows computational chemists and toxicologists to screen potential drug candidates, electrolytes, or other small molecules for specific biological toxicities prior to wet-lab synthesis.
How it works
The architecture consists of a 28-million parameter RoBERTa-PreLayerNorm encoder coupled with a two-layer multi-layer perceptron (MLP) task head featuring Gaussian Error Linear Unit (GELU) activations and dropout. The encoder has 8 layers, a hidden dimension of 512, an intermediate size of 2048, 8 attention heads, and a maximum sequence context of 2048 tokens. For classification, the first token's hidden state vector is extracted from the encoder and passed through the MLP task head.
The base encoder was pre-trained using a masked language modelling (MLM) cross-entropy objective on Kekulized SMILES strings from the Enamine REAL Space dataset, optimized with FusedLAMB. It was then fine-tuned on the Tox21 dataset using AdamW and a binary cross-entropy loss function to classify the 12 target endpoints simultaneously.
Key facts
- Model architecture
- RoBERTa-PreLayerNorm encoder (8 layers, 8 heads, hidden size 512) with a two-layer MLP classification head
- Parameter count
- 28M
- Context length
- 2048 tokens
- Pre-training data
- Enamine REAL Space
- Fine-tuning data
- Tox21 dataset from MoleculeNet (7,831 compounds)
- Primary metric
- AUROC averaged across 12 tasks
What you can use it for
- Tox21 assay screening
- Predicting multi-label binary active/inactive classifications for compounds across 12 specific nuclear receptor and stress-response pathways.
- Molecular feature extraction
- Generating chemical embeddings from SMILES strings using the fine-tuned RoBERTa-PreLayerNorm backbone.
- Pre-synthetic toxicology triage
- Filtering proposed molecules, such as pharmaceutical leads or electrolyte additives, for potential cellular stress or receptor-mediated toxicity.
Run it here
No installation and no GPU of your own: fill in the inputs and it runs on our servers. Results stay in your browser unless you sign in.
Runs on On-demand GPU
Runs on our servers with on-demand compute. A first run needs time to load the model; active capacity can be reused and scales down when idle.Run this model
MIST 28M · Tox21
Strengths and limits
Good at
- Pre-trained on Enamine REAL Space, capturing synthetically accessible chemical space relevant to practical molecular design.
- Accepts standard Kekulized SMILES strings directly as input using the fast SMIRK tokenizer.
- Provides simultaneous multi-label classification across 12 distinct toxicological pathways in a single forward pass.
Watch out for
- Requires chemical inputs to be provided as text-based SMILES strings, typically Kekulized.
- Installation requires a Rust compiler environment to build the prerequisite SMIRK tokenizer.
- Predictions are restricted to the 12 biological endpoints defined within the Tox21 benchmark dataset.
Inputs and outputs
You provide
Sequence or text
A protein or nucleotide sequence, or free text — whichever this model was trained on.
You get back
Mean-pooled representation, one vector per input.
Licence
Check the licence before commercial use · See model card
The See model card licence carries terms worth reading before you depend on it.
Questions
Can I use MIST 28M · Tox21 commercially?
Check the licence before commercial use. The See model card licence carries terms worth reading before you depend on it.
How do I run MIST 28M · Tox21 without installing anything?
Use the form under “Run it here” on this page: fill in the inputs and press Run model. It runs on our servers on CPU only, typically in 12s, and a run costs about 28 credits. Visitors without an account get 1,000 credits a month.
What inputs does the model require?
It accepts molecular SMILES strings, specifically Kekulized SMILES, tokenized via the SMIRK tokenizer.
What outputs does the model produce?
When running the full model with its classification head, it outputs unnormalized logits for 12 multi-label binary toxicity endpoints. When run as a feature-extraction pipeline, it outputs pooled molecular embedding vectors.
Which specific toxicity tasks does it predict?
It predicts the 12 biological endpoints evaluated in the MoleculeNet Tox21 benchmark, which focus on nuclear receptor signaling and stress response mechanisms.
What hardware was used to train this model?
Pre-training was conducted on 2 NVIDIA A100-SXM4-80GB GPUs over 12 hours and 15 minutes, followed by fine-tuning on a single NVIDIA A100 GPU.
How to cite it
Wadell, A., Bhutani, A., Azumah, V., Ellis-Mohr, A. R., Kelly, C., Zhao, H., Nayak, A. K., Hegazy, K., Brace, A., Lin, H., Emani, M., Vishwanath, V., Gering, K., Alkan, M., Gibbs, T., Wells, J., Varshney, L. R., Ramsundar, B., Duraisamy, K., Mahoney, M. W., Ramanathan, A., & Viswanathan, V. (2025). Foundation Models for Discovery and Exploration in Chemical Space. arXiv:2510.18900 [physics]. https://doi.org/10.48550/arXiv.2510.18900
Written from MIST 28M · Tox21's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.