Materials Science · LeMaterial
LeMat-BulkUnique
A deduplicated dataset of over five million inorganic crystal structures curated for generative materials modelling.
What LeMat-BulkUnique is
Developed by the LeMaterial initiative, it serves as a standardized, curated benchmark and training corpus for machine learning models and generative materials design.
Training generative models across separate density functional theory (DFT) databases typically presents severe challenges due to divergent naming schemes, calculation parameters, and identical or near-identical crystal structures represented under multiple IDs. LeMat-BulkUnique addresses this by standardising structural features into an OPTIMADE-compliant schema, establishing compatibility filters across DFT parameters, and using a graph-based structural fingerprinting algorithm to remove duplicate entries.
The dataset provides consistent bulk properties, including periodic boundary dimensions, Cartesian site positions, lattice vectors, uncorrected VASP total energies, atomic forces, stress tensors, and magnetic moments across three exchange-correlation functionals: PBE, PBESol, and SCAN.
How it was made
Deduplication is performed using an entalpic fingerprinting pipeline. For each crystal, chemical bonds are computed using the EconNN algorithm via Pymatgen to construct a structure graph with species-encoded nodes. This graph is hashed using the Weisfeiler-Lehman algorithm, combined with structural symmetry and composition. When duplicate fingerprints occur across databases, only the lowest-energy configuration is retained. The hashing method is benchmarked to remain robust against minor Gaussian noise on atomic coordinates and lattice vectors while respecting detected crystal symmetries.
Density functional theory parameters across the constituent databases were evaluated to assess cross-compatibility. Variations in pseudopotentials, Hubbard U parameters (notably within transition-metal oxides and fluorides), and spin-polarisation configurations were analysed. Records that cannot be mixed from a DFT calculation parameter perspective are flagged via a dedicated cross-compatibility boolean field rather than discarded outright.
Key facts
- Total entries
- 5,438,436 unique materials across all functionals
- PBE subset size
- 5,005,017 structures
- PBESol subset size
- 15,753 structures
- SCAN subset size
- 417,666 structures
- Source databases
- Materials Project (mp-), Alexandria (agm-), and OQMD (oqmd-)
- Exchange-correlation functionals
- PBE, PBESol, SCAN
- Data format
- Parquet (tabular format)
- Schema standard
- OPTIMADE-compliant core fields
What you can use it for
- Generative Materials Architecture Training
- Train crystal diffusion models and autoregressive crystal generators on deduplicated, unique bulk structures to avoid overfitting to redundant entries across existing DFT databases.
- Interatomic Potential Development
- Train machine learning force fields (MLIPs) using consistent Cartesian site positions, atomic forces, total energies, and 3x3 stress tensors across varied chemistries.
- Cross-Functional Materials Benchmarking
- Compare structural and electronic ground-state predictions across PBE, PBESol, and SCAN density functional subsets using identical structural representations.
Explore the data
No licence declared
The authors have not published a licence, which means no permission to redistribute the data has been granted to anyone. The rows are not served here; read them on the original dataset page.How the columns are distributed
Over 5,005,017 rows of unique_pbe/train
cartesian_site_positions
list- min
- 1
- median
- 8
- mean
- 9.54
- max
- 444
chemical_formula_anonymous
string_text- min
- 1
- median
- 4
- mean
- 4.67
- max
- 18
chemical_formula_descriptive
string_text- min
- 2
- median
- 11
- mean
- 11.64
- max
- 36
chemical_formula_reduced
string_text- min
- 1
- median
- 7
- mean
- 7.38
- max
- 23
cross_compatibility
bool- True92%
- False8%
dimension_types
list- min
- 3
- median
- 3
- mean
- 3
- max
- 3
dos_ef
float- min
- -14.23
- median
- 4.50
- mean
- 6.61
- max
- 391.85
16.8% empty
elements
list- min
- 1
- median
- 3
- mean
- 3.27
- max
- 10
elements_ratios
list- min
- 1
- median
- 3
- mean
- 3.27
- max
- 10
energy
float- min
- -2846.53
- median
- -34.27
- mean
- -50.10
- max
- 7035.84
entalpic_fingerprint
string_text- min
- 37
- median
- 46
- mean
- 45.93
- max
- 62
forces
list- min
- 0
- median
- 8
- mean
- 9.36
- max
- 444
functional
string_label- pbe100%
immutable_id
string_text- min
- 6
- median
- 12
- mean
- 11.99
- max
- 13
last_modified
string_text- min
- 19
- median
- 19
- mean
- 19
- max
- 19
lattice_vectors
list- min
- 3
- median
- 3
- mean
- 3
- max
- 3
magnetic_moments
list- min
- 0
- median
- 8
- mean
- 9.50
- max
- 444
nelements
int- min
- 1
- median
- 3
- mean
- 3.27
- max
- 10
nperiodic_dimensions
int- min
- 3
- median
- 3
- mean
- 3
- max
- 3
nsites
int- min
- 1
- median
- 8
- mean
- 9.54
- max
- 444
species
string_text- min
- 138
- median
- 420
- mean
- 456.67
- max
- 1,394
species_at_sites
list- min
- 1
- median
- 8
- mean
- 9.54
- max
- 444
stress_tensor
list- min
- 0
- median
- 3
- mean
- 2.99
- max
- 3
total_magnetization
float- min
- -340.30
- median
- 1.0e-5
- mean
- 1.31
- max
- 383.37
2.9% empty
Strengths and limits
Good at
- Deduplication algorithm retains the lowest-energy structure for redundant materials while remaining robust to small lattice perturbations and noise.
- Adheres closely to the OPTIMADE standard, ensuring interoperable structural metadata and formula representations.
- Includes calculated mechanical and magnetic properties, such as 3x3 stress tensors, per-site forces, and site-resolved magnetic moments.
Watch out for
- Potentially non-convergent calculations or configurations with significant atomistic forces are preserved in the dataset and must be manually filtered by the user.
- Materials are not filtered by energy above the convex hull, meaning thermodynamically unstable or high-energy configurations remain present.
- Species mass entries are currently unfilled, and descriptive chemical formulas are not fully standardised across source databases.
Questions
Can I look inside LeMat-BulkUnique without downloading it?
Not here: it declares no licence, so no permission to redistribute its rows has been granted. Its description and statistics are still shown.
What licence is LeMat-BulkUnique under?
Its authors have not declared a licence, which means no permission to reuse or redistribute it has been granted.
How are duplicate crystal structures defined and removed?
Bonds are calculated using Pymatgen's EconNN algorithm to build a structure graph, which is hashed using the Weisfeiler-Lehman algorithm along with space-group symmetry and chemical composition. When two structures produce identical fingerprints, the dataset keeps only the configuration with the lower total energy.
Can all records in the dataset be mixed together during model training?
Not automatically. Different source repositories use differing pseudopotentials, Hubbard U parameters, and spin-polarisation criteria. Researchers should inspect the cross_compatibility flag when mixing records for energy-sensitive applications.
What dimensions and boundary conditions are present?
All entries in this dataset represent bulk inorganic crystals periodic in three dimensions, with dimension types set to [1, 1, 1] and periodic dimensions equal to 3.
Which exchange-correlation functionals are supported?
The data is partitioned into subsets calculated with PBE (5,005,017 structures), PBESol (15,753 structures), and SCAN (417,666 structures).
How to cite it
Martin Siron, Inel Djafar, Lucile Ritchie, Etienne Du-Fayet, Amandine Rossello, Ali Ramlaoui, Leandro von Werra, Thomas Wolf, Alexandre Duval. LeMat-BulkUnique Dataset. Hugging Face, 2024. https://huggingface.co/datasets/LeMaterial/LeMat-BulkUnique
Written from LeMat-BulkUnique's own dataset card. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.