Multi-Modal Foundation · Meta AI
Galactica
A scientific language model designed to study scientific tasks and serve as a baseline for domain knowledge.
What Galactica is
Galactica was developed by the Papers with Code team at Meta AI to investigate the application of language models to the organisation and representation of scientific knowledge. It was designed to address diverse technical tasks across disciplines such as chemistry, biology, and mathematics, including entity extraction, molecular property prediction, and literature reasoning.
Within modern research workflows, the model is commonly utilised as an evaluation baseline rather than a tool for direct production generation. Its well-documented tendency to output plausible-sounding yet completely fabricated references and factual claims makes it an important reference point for measuring model groundedness and domain recall.
How it works
Galactica is built on a decoder-only transformer architecture with specialised modifications. The base model operates with 1.3 billion parameters, belonging to a family of models ranging from 125 million to 120 billion parameters.
The model was pre-trained on 106 billion tokens of open-access scientific literature and data, including textbooks, research articles, encyclopaedias, reference works, and structured knowledge bases. Different scientific modalities are tokenised directly into text sequences, enabling interaction with varied scientific representations through natural language prompts.
Key facts
- Architecture
- Modified decoder-only Transformer
- Model size
- 1.3 billion parameters (base model)
- Parameter options
- 125M, 1.3B, 6.7B, 30B, 120B
- Training data volume
- 106 billion tokens
- Release date
- November 2022
- Input format
- Text prompt
- Output format
- Generated text
- Required GPU memory
- 24 GB
What you can use it for
- Domain Knowledge Evaluation
- Serves as an evaluation baseline to benchmark the scientific factual accuracy and domain capabilities of language models.
- Citation Hallucination Benchmarking
- Provides a test case for auditing how generative models fabricate scientific literature citations and exhibit popularity bias.
- Scientific Question Answering
- Tests reasoning abilities on scientific prompts, mathematical problem-solving, and concept explanations.
- Molecular Property Prediction
- Interprets tokenised chemical representations to predict properties and extract biochemical entities.
- Document Summarisation
- Generates concise summaries of scientific texts and technical reference material.
Run it here
No installation and no GPU of your own: fill in the inputs and it runs on our servers. Results stay in your browser unless you sign in.
Runs on On-demand GPU
Runs on our servers with on-demand compute. A first run needs time to load the model; active capacity can be reused and scales down when idle.Run this model
Galactica
Strengths and limits
Good at
- It outperforms several open-source general language models across knowledge probes, reasoning benchmarks, and knowledge-intensive scientific tasks.
- The model was trained on a broad corpus of 106 billion tokens dedicated to academic papers, textbooks, and knowledge bases.
- It exhibits substantially lower toxicity rates in benchmark testing compared with conventional large language models.
- It uses multimodal tokenisation to provide a single natural language interface for diverse tasks, including chemistry and mathematics.
Watch out for
- The model is prone to hallucinations and may generate fluent, authoritative scientific text and citations that are entirely incorrect.
- Hallucination rates increase noticeably when handling less popular or infrequently cited scientific concepts.
- Its citation prediction outputs continue to display popularity bias, favoring heavily cited works even at larger parameter scales.
- It demonstrates measurable social and demographic biases across standard evaluation benchmarks.
Inputs and outputs
You provide
Prompt
You get back
Baseline response — not for production use.
Licence
Academic and non-commercial use only · CC-BY-NC-4.0
Evaluation baseline only. Not routed for user-facing generation.
Questions
Can I use Galactica commercially?
Academic and non-commercial use only. Evaluation baseline only. Not routed for user-facing generation.
How do I run Galactica without installing anything?
Use the form under “Run it here” on this page: fill in the inputs and press Run model. It runs on our servers on 1× A100 16GB, typically in 4.0s, and a run costs about 28 credits. Visitors without an account get 1,000 credits a month.
Why is Galactica recommended as an evaluation baseline rather than for production use?
The model has a documented tendency to fabricate citations and scientific facts with high linguistic fluency. It is therefore best suited as a baseline to evaluate hallucination rates and verify grounding mechanisms in other systems.
What model sizes are available for Galactica?
The model family includes five sizes: mini (125 million parameters), base (1.3 billion parameters), standard (6.7 billion parameters), large (30 billion parameters), and huge (120 billion parameters).
What training corpus was used for the model?
It was trained on 106 billion tokens of open-access scientific text, including research papers, textbooks, scientific reference sites, encyclopaedias, and structured databases.
Can the model handle non-textual data like chemical molecules?
Yes. Galactica tokenises specialised modalities into text sequences, enabling tasks such as molecular property prediction and entity extraction using text-based prompts.
How to cite it
Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, Robert Stojnic. GALACTICA: A Large Language Model for Science. 2022.
Written from Galactica's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.