Multi-Modal Foundation · mist-models
MIST 1.8B · Tox21
A 1.8-billion-parameter model predicting 12 toxicity endpoints for small molecules from SMILES strings.
What MIST 1.8B · Tox21 is
MIST 1.8B · Tox21 is a molecular foundation model variant developed by the Electrochemical Energy Group at the University of Michigan, Ann Arbor. It is designed to predict qualitative toxicity profiles for small molecules, helping researchers assess safety risks early in chemical discovery pipelines.
The model builds upon the MIST (Molecular Insight SMILES Transformers) 1.8B architecture, pre-trained on chemical structures from the Enamine REAL Space. It has been fine-tuned on the MoleculeNet Tox21 benchmark to identify potential toxic effects across biological pathways, specifically those involving nuclear receptor signalling and cellular stress responses.
How it works
The underlying architecture employs a RoBERTa-PreLayerNorm encoder featuring 24 layers, a hidden dimension of 2,304, 18 attention heads, and an intermediate size of 9,216, supporting sequences up to 2,048 tokens. The model was initially pre-trained using a masked language modelling objective on Kekulized SMILES strings from Enamine REAL Space using the FusedLAMB optimizer.
For toxicity prediction, the pre-trained encoder is coupled with a task-specific network consisting of a two-layer multi-layer perceptron with Gaussian Error Linear Unit (GELU) activations and dropout. The hidden state of the first token is pooled to form a single sequence embedding, which the task head uses to output unnormalised logits for 12 multi-label binary classification targets via binary cross-entropy loss.
Key facts
- Model architecture
- RoBERTa-PreLayerNorm encoder with a 2-layer MLP task head
- Base model
- mist-1.8B-dh61satt
- Parameters
- 1.8 billion
- Maximum sequence length
- 2048 tokens
- Pre-training dataset
- Enamine REAL Space
- Fine-tuning dataset
- MoleculeNet Tox21 (7,831 compounds)
- Input format
- Kekulized SMILES strings
What you can use it for
- Nuclear Receptor Toxicity Screening
- Assess whether prospective candidate molecules are likely to interact with or disrupt nuclear receptor signalling pathways.
- Cellular Stress Response Profiling
- Predict potential chemical triggers of biological stress response pathways across the 12 panel targets in Tox21.
- Molecular Representation Extraction
- Generate molecular sequence embeddings derived from the pooled first-token encoder states for custom chemical analysis.
Run it here
No installation and no GPU of your own: fill in the inputs and it runs on our servers. Results stay in your browser unless you sign in.
Runs on On-demand GPU
Runs on our servers with on-demand compute. A first run needs time to load the model; active capacity can be reused and scales down when idle.Run this model
MIST 1.8B · Tox21
Strengths and limits
Good at
- Pre-trained on the vast, synthetically accessible chemical space of Enamine REAL Space.
- Simultaneously predicts across 12 distinct biological toxicity endpoints in a multi-label classification setting.
- Accommodates long sequence contexts up to 2,048 tokens for complex molecular inputs.
Watch out for
- Evaluated specifically on the Tox21 chemical domain, which covers 7,831 compounds across 12 discrete endpoints.
- Requires Kekulized SMILES formatting and the external Rust-based SMIRK tokenizer.
- Model weights are provided as-is without performance guarantees or fitness for safety-critical regulatory decisions.
Inputs and outputs
You provide
Sequence or text
A protein or nucleotide sequence, or free text — whichever this model was trained on.
You get back
Mean-pooled representation, one vector per input.
Licence
Check the licence before commercial use · See model card
The See model card licence carries terms worth reading before you depend on it.
Questions
Can I use MIST 1.8B · Tox21 commercially?
Check the licence before commercial use. The See model card licence carries terms worth reading before you depend on it.
How do I run MIST 1.8B · Tox21 without installing anything?
Use the form under “Run it here” on this page: fill in the inputs and press Run model. It runs on our servers on 1× A100 16GB, typically in 12s, and a run costs about 70 credits. Visitors without an account get 1,000 credits a month.
What input format does the model expect?
The model takes molecular SMILES strings as inputs, which should generally be Kekulized.
What are the model's outputs?
When run through its task head, it produces unnormalised logits representing multi-label binary classifications for 12 toxicity endpoints related to nuclear receptor and stress response pathways. When used solely as a feature extractor, it returns a pooled vector representation.
What software dependencies are required for tokenization?
The model uses the SMIRK tokenizer, which requires a working Rust installation alongside Python dependencies.
How was the model evaluated during development?
It was evaluated on an 80/10/10 random split of the Tox21 dataset, using the Area Under the Receiver Operating Characteristic Curve (AUROC) averaged across all 12 tasks.
How to cite it
Wadell, A., Bhutani, A., Azumah, V., Ellis-Mohr, A. R., Kelly, C., Zhao, H., Nayak, A. K., Hegazy, K., Brace, A., Lin, H., Emani, M., Vishwanath, V., Gering, K., Alkan, M., Gibbs, T., Wells, J., Varshney, L. R., Ramsundar, B., Duraisamy, K., Mahoney, M. W., Ramanathan, A., & Viswanathan, V. (2025). Foundation Models for Discovery and Exploration in Chemical Space. arXiv preprint arXiv:2510.18900.
Written from MIST 1.8B · Tox21's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.