Biology Benchmark · Anthropic
BioMysteryBench-full
A biology evaluation suite designed to test expert-level scientific reasoning in frontier models.
What BioMysteryBench-full is
BioMysteryBench-full is an evaluation benchmark developed by Anthropic. It contains the complete set of BioMysteryBench assessment problems spanning biology and medicine.
The benchmark addresses the challenge of measuring advanced scientific reasoning in frontier computational models, providing difficult biological problems that require expert-level problem-solving.
Key facts
- Category
- Biology benchmark
- Focus areas
- Biology, medicine, scientific reasoning
What you can use it for
- Evaluating scientific reasoning
- Testing the capacity of frontier models to carry out expert-level scientific deduction on complex biological questions.
- Benchmarking medical problem solving
- Assessing how effectively language models navigate advanced reasoning tasks set within medical domains.
- Frontier model comparison
- Using the full evaluation suite as a standard reference to compare the biology reasoning performance of different models.
Explore the data
This dataset is gated
Its authors require their terms to be accepted before the data can be read. Accept them on the original dataset page. The platform does not accept terms on your behalf, so the rows are not shown here.Strengths and limits
Good at
- Focuses on probing expert-level scientific reasoning rather than straightforward factual recall.
- Provides the complete evaluation set for benchmarking biological and medical problem-solving.
Watch out for
- Serves strictly as an evaluation benchmark rather than a resource for model training.
- Focuses on challenging, expert-level problems rather than introductory biological tasks.
Questions
Can I look inside BioMysteryBench-full without downloading it?
Not here: its authors gate it behind terms each user must accept on the original page. Its description and statistics are still shown.
What licence is BioMysteryBench-full under?
Its authors declare cc-by-4.0. Read the terms on the original page before reusing the data.
What is BioMysteryBench-full?
It is the complete evaluation set of BioMysteryBench, composed of challenging biology and medicine problems designed to probe expert-level scientific reasoning.
Who developed BioMysteryBench-full?
The benchmark was created and published by Anthropic.
Which fields of study does the benchmark cover?
The benchmark covers biology and medicine with an emphasis on scientific reasoning.
Written from BioMysteryBench-full's own dataset card. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.