Literature · UniverseTBD
AstroLLaMA
A specialized 7-billion-parameter language model for astronomy and astrophysics literature text generation.
What AstroLLaMA is
AstroLLaMA is a specialized language model developed by UniverseTBD. It adapts a foundation model specifically to the astrophysics domain, serving as an open model tailored for processing, generating, and analysing astronomical text.
The model addresses the need for domain-specific language tooling in astronomy, where generic models often lack nuanced understanding of specialized vocabulary and literature conventions. It supports researchers with astrophysics text generation, literature summarisation, and hypothesis completion.
How it works
AstroLLaMA is built upon the Llama-2 7B base architecture.
In addition to standard autoregressive text generation, the model's final hidden states can be extracted and averaged across token positions to generate vector embeddings of astronomical texts up to a context length of 4096 tokens.
Key facts
- Base architecture
- Llama-2 (causal language model)
- Parameter count
- 7 billion (7B)
- Supported modalities
- Text
- Maximum context length
- 4096 tokens
- GPU memory required
- 24 GB
What you can use it for
- Astrophysics Text Generation
- Completing text prompts and continuing drafting based on observational or theoretical astrophysics context.
- Literature Summarisation
- Condensing domain-specific findings and summaries from astrophysics research abstracts.
- Hypothesis Completion
- Exploring potential hypothesis continuations and scientific phrasing in astronomical research.
- Astronomical Text Embedding
- Extracting vector representations from the final hidden layer across sequences of up to 4096 tokens for downstream search or clustering tasks.
Run it here
No installation and no GPU of your own: fill in the inputs and it runs on our servers. Results stay in your browser unless you sign in.
Runs on On-demand GPU
Runs on our servers with on-demand compute. A first run needs time to load the model; active capacity can be reused and scales down when idle.Run this model
AstroLLaMA
Strengths and limits
Good at
- Capable of producing both generative completions and pooled text embeddings within a single architecture.
- Supports input sequences up to 4096 tokens for embedding astronomy literature.
Watch out for
- Requires substantial GPU memory (24 GB) to run locally.
- Training focused solely on abstracts rather than full-text papers, which may limit depth on experimental procedures and extended mathematical derivations.
Inputs and outputs
You provide
Prompt
The model continues from here.
You get back
The model's completion.
Licence
Check the licence before commercial use · See model card
The See model card licence carries terms worth reading before you depend on it.
Questions
Can I use AstroLLaMA commercially?
Check the licence before commercial use. The See model card licence carries terms worth reading before you depend on it.
How do I run AstroLLaMA without installing anything?
Use the form under “Run it here” on this page: fill in the inputs and press Run model. It runs on our servers on 1× A100 16GB, typically in 12s, and a run costs about 70 credits. Visitors without an account get 1,000 credits a month.
What base model does AstroLLaMA use?
AstroLLaMA is built upon the Llama-2 7B architecture and fine-tuned by UniverseTBD.
Can AstroLLaMA be used to generate embeddings?
Yes. By extracting and averaging the final hidden states across token positions, the model can output vector embeddings for astronomical texts.
How to cite it
arXiv:2308.12823
Written from AstroLLaMA's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.