Chemistry Captioning · jablonkagroup
chempile-caption
An image-to-text dataset of chemistry figures and captions sourced from LibreTexts for multimodal models.
What chempile-caption is
ChemPile-Caption is an image-captioning dataset designed for chemistry and scientific reasoning, developed by the jablonkagroup. It forms part of the broader ChemPile initiative, which compiles specialised chemical data to train machine learning systems.
Interpreting chemical diagrams, reaction mechanisms, and laboratory apparatus requires models that connect visual depictions with domain-specific explanations. ChemPile-Caption addresses this need by providing paired visual and textual examples from educational chemistry literature to support the training and evaluation of multimodal vision-language models.
How it was made
The dataset was constructed through an automated web scraping pipeline targeting all chemistry books on the LibreTexts platform. Images spanning molecular structures, reaction schemes, graphs, and laboratory equipment were downloaded along with their surrounding descriptive metadata.
During processing, images were filtered to exclude instances lacking adequate descriptions. For each retained image, the HTML alternative text ('alt text') and the figure caption were combined into a single text field and paired directly with the visual file. The data is partitioned into fixed training, validation, and test splits and stored in Parquet format.
Key facts
- Task
- Image-to-text / image captioning
- Data format
- Parquet
- Modalities
- Image and text
- Language
- English
- Data source
- LibreTexts Chemistry books
What you can use it for
- Multimodal Chemistry Model Pre-training
- Supplying paired visual and textual chemistry data to train vision-language models on domain-specific diagrams and reaction workflows.
- Automated Scientific Captioning
- Developing image-to-text models that generate descriptive summaries for chemistry plots, molecular figures, and laboratory apparatus.
- Multimodal Scientific Evaluation
- Benchmarking the ability of multimodal systems to interpret visual chemistry concepts using the fixed test partition.
Explore the data
The schema, real rows, full-text search and a SQL filter — read in your browser, nothing downloaded.
Explore
Read the real rows without downloading anything
Reading rows…
LiveRead from jablonkagroup/chempile-caption at the moment you asked. Nothing is cached or stored — every row above came from that request.
How the columns are distributed
Over 90,350 rows of default/train
image
image- min
- 15
- median
- 550
- mean
- 765.65
- max
- 8,044
0.00% empty
text
string_text- min
- 100
- median
- 259
- mean
- 402.51
- max
- 18,105
Strengths and limits
Good at
- Combines both alt text and detailed figure captions into comprehensive descriptions for each image.
- Provides standardized train, validation, and test splits ready for model development and benchmarking.
Watch out for
- Content is restricted to English-language material sourced solely from LibreTexts educational texts.
- Figures with short or missing captions and alt text were excluded during ingestion, potentially omitting certain diagrams.
Questions
Can I look inside chempile-caption without downloading it?
Yes. This page reads its schema, real rows and per-column statistics in the browser, with full-text search and a SQL filter. Nothing is downloaded to your machine.
What licence is chempile-caption under?
Its authors declare cc-by-nc-sa-4.0. Read the terms on the original page before reusing the data.
What types of images are contained in the dataset?
The dataset contains chemical structures, reaction schemes, laboratory equipment illustrations, photographs, and analytical plots extracted from chemistry textbooks.
What does each record contain?
Each record contains two fields: 'image', which provides the visual content, and 'text', which combines the image's alt text and caption.
How are the training, validation, and test sets divided?
The dataset contains 90,350 examples in the training set, 5,019 in the validation set, and 5,020 in the test set.
How to cite it
arXiv:2505.12534
Written from chempile-caption's own dataset card. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.