Earth-Observation Embeddings · allenai
olmoearth-paper-embeddings
Precomputed Earth-observation foundation model embeddings across 24 downstream evaluation tasks.
What olmoearth-paper-embeddings is
This dataset provides pre-computed feature representations extracted from 26 Earth-observation foundation models across 24 downstream evaluation benchmarks. Developed by the Allen Institute for AI alongside the OlmoEarth study, the collection eliminates the need to run computationally demanding backbone encoders when assessing representation quality.
The downstream tasks span core geospatial problems including land-cover classification, crop mapping, building identification, and flood segmentation across diverse sensor types, including Sentinel-1 synthetic aperture radar, Sentinel-2 optical imagery, and Landsat 8 multispectral data. By providing standardised embeddings alongside label tensors across train, validation, and test splits, it facilitates rapid benchmarking via linear probing and k-nearest neighbours classification.
Because the majority of representations preserve canonical dataset sample ordering, researchers can evaluate, ensemble, or directly compare representations from distinct architectures without re-extracting features.
How it was made
Embeddings were generated by passing task splits through each foundation model encoder using validation-optimal hyperparameters recorded during the original benchmarking sweeps. For classification tasks, embeddings are stored as two-dimensional tensors of shape (N, D), with labels formatted as one-dimensional class indices or multi-label binary indicators. For semantic segmentation, patch-level spatial embeddings have the shape (N, H, W, D), while corresponding ground-truth segmentation masks are retained at full image resolution.
The dataset is organised by model group and task into PyTorch files (.pt), each containing a dictionary with bfloat16 feature tensors and int64 label tensors. Aside from AnySat segmentation subsets, sample rows across models strictly follow the canonical enumeration of the underlying source benchmarks, allowing index-based feature concatenation and alignment.
Key facts
- Total size
- ~1.1 TB
- Models evaluated
- 26 foundation model groups (including OlmoEarth, DINOv3, Prithvi v2, AnySat, Clay, CROMA, Galileo, and TerraMind)
- Downstream tasks
- 24 tasks (including Geobench subsets, BreizhCrops, CropHarvest, PASTIS, MADOS, and Sen1Floods11)
- Data modalities
- Sentinel-1 SAR, Sentinel-2 optical, Landsat 8 multispectral, and multimodal time series
- File format
- PyTorch (.pt) dictionary containing embeddings and labels
- Precision
- bfloat16 for embeddings, int64 for labels
- Embedding dimensions
- 128 to 4096 dimensions depending on model architecture
What you can use it for
- Downstream Benchmark Reproduction
- Reproduce k-nearest neighbours and linear-probe classification or segmentation benchmarks across published foundation models without executing full neural network forward passes.
- Model Ensembling and Representation Fusion
- Concatenate row-aligned embeddings from diverse optical and radar foundation models to evaluate multimodal feature fusion techniques on shared task splits.
- Probe Architecture Development
- Design and test lightweight probing heads, non-linear classifiers, or temporal aggregators directly on pre-extracted temporal and spatial representations.
Explore the data
The schema, real rows — read in your browser, nothing downloaded.
Explore
Read the real rows without downloading anything
Reading rows…
LiveRead from allenai/olmoearth-paper-embeddings at the moment you asked. Nothing is cached or stored — every row above came from that request.
Strengths and limits
Good at
- Saves substantial computational overhead by eliminating repetitive backbone inference across 26 foundation models.
- Maintains canonical sample ordering across models, allowing direct cross-model ensembling and indexing by row.
- Includes enriched JSON parameter files specifying the exact learning rates, pooling strategies, and normalisation modes needed to replicate published results.
Watch out for
- Not all models cover all 24 tasks due to modality constraints such as optical-only architectures.
- AnySat segmentation validation and test sets are randomly subsampled at a 1/16 ratio without a fixed random seed, preventing cross-model pixel alignment.
- Prithvi v2 Sentinel-2 embeddings use a corrected band mapping that diverges from the misindexed band configurations in earlier published tables.
Questions
Can I look inside olmoearth-paper-embeddings without downloading it?
Yes. This page reads its schema, real rows and per-column statistics in the browser. Nothing is downloaded to your machine.
What licence is olmoearth-paper-embeddings under?
Its authors declare cc-by-4.0. Read the terms on the original page before reusing the data.
How are samples aligned across different foundation models?
Embeddings and labels are stored in the canonical enumeration order of the source benchmark. Row index i in a given split represents the identical sample across all model groups, except for AnySat segmentation sets where pixels were randomly subsampled.
What data formats and precisions are used?
Files are saved as PyTorch .pt files containing Python dictionaries with embeddings stored as bfloat16 tensors and target labels stored as int64 tensors.
How are segmentation labels handled relative to spatial embeddings?
Embeddings are stored at the model patch grid resolution (N, H, W, D), whereas ground-truth label masks are stored at the original full image resolution to allow linear probing heads to upsample predictions.
Why do some Prithvi v2 results differ from the original paper table?
The original paper sweep contained an indexing error that fed Red-Edge bands instead of NIR/SWIR bands into Prithvi v2 for Sentinel-2 tasks; this dataset contains re-extracted embeddings with the corrected band mapping.
How to cite it
Team OlmoEarth. (2025). OlmoEarth: Stable Latent Image Modeling for Multimodal Earth Observation. arXiv:2511.13655.
Written from olmoearth-paper-embeddings's own dataset card. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.