Multi-Modal Foundation · polymathic-ai
AstroCLIP
A multimodal astronomy model aligning galaxy spectra and images into a shared embedding space.
What AstroCLIP is
AstroCLIP is an astronomy foundation model developed by polymathic-ai for astrophysics and physics research. It processes galaxy observations across two modalities — optical imaging and spectra — and maps them into a joint representation space.
The model addresses the challenge of connecting disparate astronomical observations of galaxies, enabling the joint representation to be used for downstream astrophysical property prediction.
How it works
AstroCLIP is trained on cross-matched galaxy data from the DESI legacy survey to execute a cross-modal alignment objective, learning to project galaxy spectra and images into a single shared embedding space.
Key facts
- Domain
- Astronomy, physics
- Training data
- Cross-matched DESI legacy survey
- Input modalities
- Galaxy spectra and images
What you can use it for
- Astrophysical property prediction
- Estimate physical properties of galaxies using aligned representations derived from spectra, images, or both.
- Cross-modal galaxy analysis
- Relate and compare spectroscopic observations directly with corresponding galaxy survey images in a shared embedding space.
Run it here
We list this model but cannot run it yet: no pipeline_tag — needs a human.
Everything on this page still applies. Running it needs a loader for this repository, not a bigger machine.
Strengths and limits
Good at
- Aligns multi-modal astronomical inputs, bridging spectroscopic measurements and imaging data.
- Trained on observational data from the DESI legacy survey.
Watch out for
- Specialised for galaxy observations and not documented for non-extragalactic astronomical sources.
- Requires data compatible with DESI legacy survey imaging and spectroscopic formats.
Inputs and outputs
You provide
Sequence or text
A protein or nucleotide sequence, or free text — whichever this model was trained on.
You get back
Mean-pooled representation, one vector per input.
Licence
Check the licence before commercial use · mit
The mit licence carries terms worth reading before you depend on it.
Questions
Can I use AstroCLIP commercially?
Check the licence before commercial use. The mit licence carries terms worth reading before you depend on it.
What observational data does AstroCLIP handle?
It is designed to handle galaxy images and galaxy spectra, projecting both into a common embedding space.
Which dataset was AstroCLIP trained on?
It was trained on cross-matched galaxy observations from the DESI legacy survey.
What is the primary application of the embeddings?
The aligned embeddings are used to predict downstream astrophysical properties of galaxies.
How to cite it
AstroCLIP: A Cross-Modal Foundation Model for Galaxies, arXiv:2310.03024
Written from AstroCLIP's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.