Carbon-X

What Carbon-X is

Carbon-X is a 14.8-billion-parameter decoder-only language model developed jointly by Hugging Face, Zhongguancun Academy, and the Telethon Institute of Genetics and Medicine (TIGEM). Built upon Qwen3-14B, it processes nucleotide sequences and natural-language text within a unified token stream, allowing researchers to ask questions about raw DNA sequences in plain English and receive explanations in return.

The model addresses common genomic interpretation tasks, including predicting whether coding and complex mutations are benign or pathogenic, identifying regulatory sequence elements, classifying organisms, and forecasting succeeding nucleotide bases. By integrating DNA and text without a separate biological encoder, it reasons across genomic sequences within standard conversational prompts.

Carbon-X achieves an average accuracy of 66.7% across five DNA benchmarks, two of which its authors built, and it leads every model its authors tested on coding-variant effects and next-base prediction. Operating with thinking disabled, the model delivers direct answers without generating lengthy reasoning traces.

How it works

Carbon-X extends the Qwen3-14B architecture with a hybrid vocabulary comprising standard byte-pair encoding (BPE) for English and 5,515 dedicated genomic tokens. Nucleotide sequences are parsed into non-overlapping 6-mers—each token representing six base pairs—while dedicated tail tokens represent trailing sequences of one to five bases. The decoder-only Transformer incorporates 40 layers, grouped-query attention, SwiGLU activations, and rotary position embeddings covering a context window of 40,960 tokens.

Training begins by initialising the DNA token embeddings across roughly 10 billion tokens while freezing the base transformer. The entire model then undergoes mid-training across 302 billion tokens comprising 55% raw DNA, 20% English text, 15% biological literature, and 10% instruction-following dialogues. This is followed by three epochs of supervised fine-tuning across genomic reasoning and sequence manipulation tasks, concluded by two rounds of asynchronous Group Relative Policy Optimization (GRPO) using verifiable, code-checked reward functions with thinking disabled.

Key facts

What you can use it for

Coding variant effect prediction

Classify whether single-nucleotide substitutions within protein-coding regions are benign or pathogenic directly from nucleotide sequences.

Complex variant evaluation

Analyse the likely functional impact of non-single-nucleotide mutations, such as insertions and deletions, alongside reference sequences.

Regulatory feature identification

Differentiate between promoter and enhancer regions, classify eukaryote organism groups, and detect transcription-factor binding sites and sequence motifs.

Next-base sequence prediction

Forecast continuing nucleotide bases from preceding DNA sequences to test local sequence understanding.

Variant likelihood comparison

Rank equal-length genetic variants against one another based on relative sequence log-likelihood.

Run it here

No installation and no GPU of your own: fill in the inputs and it runs on our servers. Results stay in your browser unless you sign in.

Strengths and limits

Inputs and outputs

Licence

Questions