Bioacoustics Benchmark · EarthSpeciesProject
BEANS-Zero
A bioacoustics benchmark evaluating zero-shot audio-language models on animal vocalisation tasks.
What BEANS-Zero is
BEANS-Zero is a bioacoustics benchmark built by the Earth Species Project to evaluate multimodal audio-language models without fine-tuning or in-context examples. Introduced alongside NatureLM-audio, it addresses the challenge of testing whether instruction-following models can interpret ecological acoustic signals across varied biological taxa.
The benchmark compiles 91,965 test examples structured for query-response interactions, pairing bioacoustic audio clips with textual instructions to assess tasks including species detection, classification, and captioning. Because the dataset contains exclusively a test split, it tests genuine zero-shot performance across both familiar and held-out biological domains.
How it was made
BEANS-Zero consolidates eight established bioacoustics datasets from the original BEANS benchmark and introduces several newly curated evaluation splits. The included tasks encompass environmental audio classification (ESC-50), marine mammal classification (Watkins), bird identification (Cornell Bird Identification and dawn chorus ENABIRDS), mosquito wingbeat sorting (HumBugDB), minke whale detection (HICEAS), frog and bird call detection (RFCx), and Hainan gibbon call detection.
To test taxonomic and linguistic generalisation, the newly curated subsets evaluate zero-shot species, genus, and family classification using common, scientific, or full taxonomic naming conventions derived from AnimalSpeak. Additional subsets test bird life-stage classification, call-type determination (song versus call) using data from xeno-canto, audio captioning, and individual counts using synthetic mixtures of zebra finch calls.
Each record in the dataset provides the raw audio waveform as a sequence of floating-point values alongside JSON-encoded metadata recording sample rate and duration in seconds. Prompts are formatted both with token placeholders and as pure text instructions, accompanied by canonical textual target responses.
Key facts
- Release version
- 0.1.0
- Sample count
- 91,965
- Data format
- Parquet
- Input modalities
- Audio waveform and text prompt
- Output modality
- Text response
- Splits
- Test split only
What you can use it for
- Zero-shot species classification
- Assess whether multimodal foundation models can classify bird, marine mammal, and insect species from audio when prompted with scientific, common, or taxonomic names without prior task-specific tuning.
- Acoustic species detection
- Evaluate model capability in detecting focal wildlife signals, such as minke whale calls, Hainan gibbon vocalisations, or dawn chorus bird song, within acoustic recordings.
- Bioacoustic audio captioning
- Test the generation of descriptive natural-language summaries representing animal vocalisations and acoustic scenes.
- Life-stage and call-type discrimination
- Measure model discernment of behavioural acoustic differences, such as distinguishing songs from calls or identifying juvenile versus adult avian vocalisations.
Explore the data
The schema, real rows — read in your browser, nothing downloaded.
Explore
Read the real rows without downloading anything
Reading rows…
LiveRead from EarthSpeciesProject/BEANS-Zero at the moment you asked. Nothing is cached or stored — every row above came from that request.
How the columns are distributed
Over 6,000 rows of BEANS-Zero/test
audio
list- min
- 268
- median
- 98,846
- mean
- 187k
- max
- 5.8M
created_at
datetimedataset_name
string_label- captioning28%
- gibbons21%
- dcase16%
file_name
string_text- min
- 6
- median
- 17
- mean
- 25.86
- max
- 170
id
string_text- min
- 36
- median
- 36
- mean
- 36
- max
- 36
instruction
string_label- <Audio><AudioHere></Audio> Caption the audio, using the common name for any animal species.28%
- <Audio><AudioHere></Audio> Which of these, if any, are present in the audio recording? Single pulse gibbon call, Multiple pulse gibbon call, Gibbon duet, None.21%
- <Audio><AudioHere></Audio> Which of these, if any, are present in the audio recording? American Redstart, Bay-breasted Warbler, Black-throated Blue Warbler, Jackdaw call, Meerkat close call, Chipping Sparrow, Common Yellowthroat, Gray-cheeked Thrush, Hyena giggle, Hyena groan and moo, Ovenbird, Other Animal, Rose-breasted Grosbeak, Savannah Sparrow, Meerkat short note, Hyena squitter, Swainson's Thrush, White-throated Sparrow, None.16%
instruction_text
string_label- Caption the audio, using the common name for any animal species.28%
- Which of these, if any, are present in the audio recording? Single pulse gibbon call, Multiple pulse gibbon call, Gibbon duet, None.21%
- Which of these, if any, are present in the audio recording? American Redstart, Bay-breasted Warbler, Black-throated Blue Warbler, Jackdaw call, Meerkat close call, Chipping Sparrow, Common Yellowthroat, Gray-cheeked Thrush, Hyena giggle, Hyena groan and moo, Ovenbird, Other Animal, Rose-breasted Grosbeak, Savannah Sparrow, Meerkat short note, Hyena squitter, Swainson's Thrush, White-throated Sparrow, None.16%
license
string_label- CC-BY-NC-SA25%
- http://creativecommons.org/licenses/by-nc/4.0/25%
- CC-BY19%
metadata
string_text- min
- 38
- median
- 40
- mean
- 98.59
- max
- 340
output
string_text- min
- 3
- median
- 7
- mean
- 14.44
- max
- 145
source_dataset
string_label- iNaturalist30%
- Hainan Gibbons21%
- DCASE-2021-Task-516%
task
string_label- detection39%
- classification33%
- captioning28%
Strengths and limits
Good at
- Standardises bioacoustic inputs into a unified text-instruction format suited for instruction-following audio-language models.
- Contains diverse animal groups including marine mammals, birds, amphibians, and insects across varied geographic ecosystems.
- Specifically tests generalization to unseen species, genera, and families using multiple taxonomic naming conventions.
- Includes variable audio sample rates and durations representative of disparate ecological acoustic recording conditions.
Watch out for
- Supplies only an evaluation split, providing no training sets or in-context examples for parameter adaptation.
- Lacks uniform audio durations and sampling rates across samples, requiring downstream model pre-processing or variable handling.
- Constrained to text-generation outputs, making it unsuitable for training or evaluating dense acoustic segmentation masks or raw audio-generation tasks.
Questions
Can I look inside BEANS-Zero without downloading it?
Yes. This page reads its schema, real rows and per-column statistics in the browser. Nothing is downloaded to your machine.
What licence is BEANS-Zero under?
Its authors declare other. Read the terms on the original page before reusing the data.
What splits are available in BEANS-Zero?
BEANS-Zero contains only a test split, as it is strictly intended for zero-shot performance assessment.
What modalities does the benchmark accept and return?
Inputs consist of an audio waveform accompanied by an instruction text prompt, and the expected output is a text string.
How are audio clips stored and formatted?
Audio signals are stored as sequences of floating-point numbers, with individual duration and sampling rate recorded per sample in a JSON metadata field.
What taxonomic conventions are tested in classification tasks?
Held-out taxa splits provide instructions formatted using common names, scientific binomials, or full taxonomic hierarchies.
How to cite it
@inproceedings{robinson2025naturelm, title = {NatureLM-audio: an Audio-Language Foundation Model for Bioacoustics}, author = {David Robinson and Marius Miron and Masato Hagiwara and Olivier Pietquin}, booktitle = {Proceedings of the International Conference on Learning Representations (ICLR)}, year = {2025}, url = {https://openreview.net/forum?id=hJVdwBpWjt} }
Written from BEANS-Zero's own dataset card. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.