Demonstration only. Corollary is a non-commercial research demo — no warranty, no support, not for commercial use or clinical decisions. Read the disclaimer

Corollary Labs
All models

Multi-Modal Foundation · nasa-impact

NASA-SMD-IBM

RoBERTa-based encoder model domain-adapted for NASA Science Mission Directorate earth and space science literature.

Run it onlineModel card

What NASA-SMD-IBM is

Indus, also designated as nasa-smd-ibm-v0.1, is an encoder-only transformer language model developed collaboratively by IBM Research and NASA's Science Mission Directorate. It is designed to adapt general textual representations to specialized scientific domains relevant to NASA SMD, including earth science, planetary science, astrophysics, climate studies, and biological sciences.

Natural language processing across specialized scientific literature often encounters vocabulary and conceptual shifts that general-domain models do not capture effectively. Indus addresses this by pre-training RoBERTa representations on curated corpora from aerospace, geophysical, and biomedical publications, providing a foundation for scientific text processing, document retrieval, and intelligent information extraction.

How it works

The model is built on the RoBERTa architecture comprising 125 million parameters and uses a custom tokenizer. It was trained using a masked language modelling objective within the fairseq and PyTorch frameworks.

Its training corpus incorporates English Wikipedia as of February 2020 alongside specialised scientific publication sets. These domain sources comprise papers from the American Geophysical Union (AGU), the American Meteorological Society (AMS), research indexed in the Astrophysics Data System (ADS), PubMed abstracts, and the commercial open subset of PubMed Central.

Key facts

Architecture
RoBERTa (encoder-only transformer)
Parameters
125M
Pre-training objective
Masked Language Modelling (MLM)
Base model
RoBERTa
Tokenizer
Custom
Primary pipeline task
Fill-mask

What you can use it for

Scientific Information Retrieval
Underpinning search systems and literature discovery tools across Earth and space science publications.
Named Entity Recognition
Extracting domain-specific scientific entities from Earth science, climate change, and astrophysical literature.
Extractive Question Answering
Identifying and extracting factual answers from NASA-related scientific text datasets.
Sentence Embedding Generation
Serving as the base representation model for sentence-transformer architectures applied to scientific literature.

Run it here

No installation and no GPU of your own: fill in the inputs and it runs on our servers. Results stay in your browser unless you sign in.

Runs on On-demand GPU

Runs on our servers with on-demand compute. A first run needs time to load the model; active capacity can be reused and scales down when idle.

Run this model

NASA-SMD-IBM

Use the model's mask token, commonly [MASK] or <mask>.

Needs sequence with a mask

Strengths and limits

Good at

  • Pre-trained on a broad collection of specialised literature across Earth science, space science, and biology.
  • Maintains a compact 125-million-parameter size suitable for efficient fine-tuning and inference.
  • Evaluated on dedicated scientific benchmarks including NASA-IR, NASA-QA, and Climate Change NER.

Watch out for

  • The model is in an experimental phase of development.
  • As an encoder-only masked language model, it is not designed for open-ended generative text generation.

Inputs and outputs

You provide

  • Sequence with a mask

    Use the model's mask token, commonly [MASK] or <mask>.

You get back

  • Labels with confidence scores, highest first.

Licence

Check the licence before commercial use · See model card

The See model card licence carries terms worth reading before you depend on it.

Questions

Can I use NASA-SMD-IBM commercially?

Check the licence before commercial use. The See model card licence carries terms worth reading before you depend on it.

How do I run NASA-SMD-IBM without installing anything?

Use the form under “Run it here” on this page: fill in the inputs and press Run model. It runs on our servers on CPU only, typically in 12s, and a run costs about 28 credits. Visitors without an account get 1,000 credits a month.

What separates Indus from standard RoBERTa models?

Indus incorporates pre-training across domain-specific corpora including AGU, AMS, ADS, and PubMed literature alongside Wikipedia, paired with a custom tokenizer tailored to these scientific domains.

What can the base model be used for without additional fine-tuning?

Out of the box, the base model performs fill-mask operations, predicting masked tokens in scientific sentences.

Is a smaller version of this model available?

A 30-million-parameter distilled version named nasa-smd-ibm-distil-v0.1 is available.

What domains does the training data span?

The training data covers astronomy, astrophysics, Earth and atmospheric sciences, meteorology, and biomedical fields.

How to cite it

Masayasu Maraoka, Bishwaranjan Bhattacharjee, Muthukumaran Ramasubramanian, Ikhsa Gurung, Rahul Ramachandran, Manil Maskey, Kaylin Bugbee, Rong Zhang, Yousef El Kurdi, Bharath Dandala, Mike Little, Elizabeth Fancher, Lauren Sanders, Sylvain Costes, Sergi Blanco-Cuaresma, Kelly Lockhart, Thomas Allen, Felix Grazes, Megan Ansdell, Alberto Accomazzi, Sanaz Vahidinia, Ryan McGranaghan, Armin Mehrabian, Tsendgar Lee. nasa-smd-ibm-v0.1 (Revision f01d42f). Hugging Face, 2023. DOI: 10.57967/hf/1429. arXiv:2405.10725.

Written from NASA-SMD-IBM's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.

Related models