Multimodal Astronomy · UniverseTBD
AstroLLaVA_convos
Astronomical images with captions and synthetic dialogues for training multimodal vision-language models.
What AstroLLaVA_convos is
AstroLLaVA_convos is a multimodal dataset composed of astronomical imagery paired with human-written descriptions and synthetic question-answer dialogues. Curated by UniverseTBD, it is designed to support the training and fine-tuning of vision-language models specialised in astrophysics and astronomy.
The dataset compiles material across three public repositories: NASA's Astronomy Picture of the Day (APOD), the European Southern Observatory (ESO) public image archive, and the European Space Agency's (ESA) Hubble Space Telescope archive. It provides structured visual and textual data for research in astronomical image captioning, visual question answering, and science communication.
How it was made
The dataset comprises 29,814 image-caption pairs divided into three collections: 9,993 pairs from the official NASA APOD API, 14,617 pairs from ESO, and 5,204 pairs from the ESA Hubble archive. Images are provided in JPEG, PNG, or GIF formats, capturing celestial targets, astronomical instrumentation, educational graphics, and artistic impressions.
Each entry combines an image with its original human-authored caption alongside synthetic multi-turn question-answer dialogues generated with OpenAI's GPT-4. The entire dataset is structured and distributed in Parquet format.
Key facts
- Curator
- UniverseTBD
- Total image-caption pairs
- 29,814
- APOD collection size
- 9,993 image-caption pairs
- ESO collection size
- 14,617 image-caption pairs
- HST collection size
- 5,204 image-caption pairs
- Image formats
- JPEG, PNG, GIF
- Data format
- Parquet
- Dialogue generator
- OpenAI GPT-4
- Language
- English
- DOI
- 10.57967/hf/4863
What you can use it for
- Astronomical vision-language model training
- Fine-tuning multimodal models to understand domain-specific astrophysics imagery alongside descriptive text.
- Visual question answering in astronomy
- Training and testing automated systems to answer natural-language scientific questions about telescope observations and diagrams.
- Astronomical image captioning
- Developing machine learning systems that generate descriptive scientific captions for astronomical targets and instruments.
- Public outreach and science education
- Supporting educational initiatives through conversational explanations paired with telescope observations and visual representations.
Explore the data
The schema, real rows, full-text search and a SQL filter — read in your browser, nothing downloaded.
Explore
Read the real rows without downloading anything
Reading rows…
LiveRead from UniverseTBD/AstroLLaVA_convos at the moment you asked. Nothing is cached or stored — every row above came from that request.
How the columns are distributed
Over 7,723 rows of default/train
caption
string_text- min
- 0
- median
- 835
- mean
- 842.11
- max
- 1,846
corpus
string_label- apod100%
credit
string_text- min
- 3
- median
- 4
- mean
- 12.51
- max
- 1,302
id
string_text- min
- 10
- median
- 10
- mean
- 10
- max
- 10
image
image- min
- 128
- median
- 1,024
- mean
- 1475.45
- max
- 23,115
url
string_text- min
- 40
- median
- 58
- mean
- 58.84
- max
- 96
Strengths and limits
Good at
- Integrates imagery and metadata from three major archives: NASA APOD, ESO, and ESA Hubble.
- Supplies both human-written reference captions and synthetic conversational dialogues for every entry.
- Encompasses observations of celestial phenomena as well as photographs of astronomical instruments and artistic visualisations.
Watch out for
- Content contains an inherent selection bias towards visually striking or photogenic phenomena.
- Synthetic conversational pairs may reflect model biases present within OpenAI's GPT-4.
- Image resolution and visual quality vary across the different contributing archives.
Questions
Can I look inside AstroLLaVA_convos without downloading it?
Yes. This page reads its schema, real rows and per-column statistics in the browser, with full-text search and a SQL filter. Nothing is downloaded to your machine.
What licence is AstroLLaVA_convos under?
Its authors declare cc-by-sa-4.0. Read the terms on the original page before reusing the data.
What visual formats are included in the dataset?
The dataset includes images in JPEG, PNG, and GIF formats.
How were the conversational question-answer pairs produced?
The conversational dialogues were synthetically generated using OpenAI's GPT-4 from the images and human-authored descriptions.
What types of imagery are included in the ESO subset?
The ESO subset includes photographs of astronomical instruments, views of celestial objects, and artistic representations.
Which archives provide the source material?
The source material comes from the NASA Astronomy Picture of the Day API, the ESO public image archive, and the ESA Hubble Space Telescope public image archive.
How to cite it
arXiv:2504.08583
Written from AstroLLaVA_convos's own dataset card. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.