Multi-Modal Foundation · imageomics
BioCLIP 2
Vision foundation model for zero-shot and few-shot species classification across the tree of life.
What BioCLIP 2 is
BioCLIP 2 is an organismal biology foundation model developed by researchers at the Imageomics Institute. It builds upon CLIP architecture using hierarchical contrastive training scaled up on massive biological imagery. The model is designed to address taxonomic identification challenges across animals, plants, and fungi, spanning from common organisms to rare and endangered taxa.
Trained on TreeOfLife-200M, BioCLIP 2 scales up species-level supervision to improve zero-shot species classification performance over its predecessor, BioCLIP, by 18.0% to 18.1%. Beyond standard classification, scaling hierarchical contrastive learning induces emergent properties: inter-species ecological alignment and intra-species variation separation.
These capabilities allow the model to generalise to downstream biological tasks without task-specific training, making it valuable for biodiversity monitoring, camera trap analyses, ecological trait extraction, and general biological feature encoding.
How it works
BioCLIP 2 is fine-tuned from an OpenCLIP ViT-L/14 model originally pre-trained on LAION-2B. It uses a Vision Transformer (ViT-L/14) as its image encoder and a masked self-attention Transformer as its text encoder. Input images are resized to 224 x 224 pixels and processed using standard CLIP image preprocessing.
Training used hierarchical contrastive learning with the Adam optimizer, bf16 mixed precision, a maximum learning rate of 1e-4 with 1,875 warm-up steps, cosine decay, and a weight decay of 0.2 across 30 epochs on 32 NVIDIA H100 GPUs.
Key facts
- Base architecture
- ViT-L/14 image encoder with masked self-attention Transformer text encoder (OpenCLIP)
- Training dataset
- TreeOfLife-200M (~214M images, 952K taxa) combined with a 26M sample LAION-2B replay subset
- Input image resolution
- 224 x 224 pixels
- Compute requirements
- 16 GB GPU memory for inference; trained on 32 NVIDIA H100-80GB GPUs
What you can use it for
- Zero-shot species classification
- Classify organism images across hundreds of thousands of taxonomic groups directly by supplying candidate species names as text labels.
- Camera trap monitoring
- Automate wildlife identification in field conditions, as demonstrated on balanced camera trap evaluations from LILA-BC benchmarks.
- Few-shot and rare species identification
- Identify threatened and under-represented taxa using small reference support sets, including species categorized from Near Threatened to Extinct in the Wild by the IUCN Red List.
- Biological feature extraction
- Serve as a pretrained visual backbone for ecological attribute prediction, habitat classification, plant disease recognition, and novel species discovery via clustering.
Run it here
No installation and no GPU of your own: fill in the inputs and it runs on our servers. Results stay in your browser unless you sign in.
Runs on On-demand GPU
Runs on our servers with on-demand compute. A first run needs time to load the model; active capacity can be reused and scales down when idle.Run this model
BioCLIP 2
Strengths and limits
Good at
- Achieves a mean score of 55.6% across ten zero-shot species classification tasks, outperforming BioCLIP, SigLIP, and standard CLIP.
- Shows strong zero-shot performance on rare and endangered species, scoring 76.8% top-1 accuracy on the 400-species Rare Species benchmark.
- Demonstrates emergent ecological alignment and intra-species separation, beating general-purpose vision backbones like DINOv2 on biological downstream tasks.
Watch out for
- Predictions may be biased toward well-represented species due to the long-tailed taxonomic distribution of the underlying TreeOfLife-200M dataset.
- Plankton classification performance remains low, reaching only 3.9% zero-shot accuracy on the Meta-Album plankton evaluation.
- Zero-shot recognition requires candidate labels to be specified in text, rather than performing unconstrained open-ended discovery out of the box.
Inputs and outputs
You provide
Image
PNG, JPEG or TIFF. Sent to the model as-is.
Candidate labels
Comma-separated. The model scores the image against each.
You get back
Labels with confidence scores, highest first.
Licence
Check the licence before commercial use · See model card
The See model card licence carries terms worth reading before you depend on it.
Questions
Can I use BioCLIP 2 commercially?
Check the licence before commercial use. The See model card licence carries terms worth reading before you depend on it.
How do I run BioCLIP 2 without installing anything?
Use the form under “Run it here” on this page: fill in the inputs and press Run model. It runs on our servers on CPU only, typically in 12s, and a run costs about 28 credits. Visitors without an account get 1,000 credits a month.
Can BioCLIP 2 classify organisms without prior training on those specific classes?
Yes. BioCLIP 2 operates as a zero-shot classifier by taking an image and a set of candidate text labels (such as species names) to rank probabilities without task-specific training.
What formats and inputs does the model expect?
BioCLIP 2 takes 224 x 224 pixel organism images alongside candidate text label names, outputting a table of candidate labels ranked by confidence score.
Does BioCLIP 2 provide geographic location or coordinate predictions?
No. The training data excludes geo-tagged coordinate information to minimize security and poaching risks for threatened species.
Can the model be used for non-taxonomic biological visual tasks?
Yes. It can function as a visual encoder for tasks such as fish habitat and feeding path prediction, animal attribute classification, novel species clustering, and plant disease recognition.
How to cite it
Gu, J., Stevens, S., Campolongo, E., Thompson, M., Zhang, N., Wu, J., Kopanev, A., Mai, Z., White, A., Balhoff, J., Dahdul, W., Rubenstein, D., Lapp, H., Berger-Wolf, T., Chao, W.-L., & Su, Y. (2025). BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning. Advances in Neural Information Processing Systems, 38, 102778–102811. https://doi.org/10.48550/arXiv.2505.23883
Written from BioCLIP 2's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.