PubMedBERT variants collectively recorded 2.55 million monthly downloads on Hugging Face in 2025, making it the most accessed biomedical language model family in the research community. Microsoft Research built the model from scratch on 14 million PubMed abstracts, and it continues to outperform general-domain alternatives across 13 biomedical NLP datasets. This page covers PubMedBERT statistics for 2026, including download trends, benchmark performance, model architecture, training data, and the broader biomedical NLP market.
PubMedBERT Statistics 2026 — TL;DR
PubMedBERT variants generate 2,549,802 monthly downloads on Hugging Face as of 2025.
The model was trained on 14 million PubMed abstracts totaling 21 GB of biomedical text.
PubMedBERT scores 82.91 on the BLURB benchmark, beating BERT Base by 4.7 points.
The PubMed database now holds over 40 million citations, adding roughly 1 million records per year.
The NLP in healthcare market reached $4.82 billion in 2025 and is projected to hit $36.71 billion by 2034, based on Fortune Business Insights data.
How Many Downloads Does PubMedBERT Get?
PubMedBERT’s three main variants combined for 2,549,802 monthly downloads on Hugging Face in 2025. The BiomedBERT abstracts-only model leads at 1,164,193, followed by BiomedCLIP-PubMedBERT at 863,450, and the abstracts-plus-full-text variant at 522,159. Researchers clearly prefer the lighter abstracts-only model for standard NLP tasks like named entity recognition and relation extraction.
| PubMedBERT Variant | Monthly Downloads (2025) | Share |
|---|---|---|
| BiomedBERT (Abstracts Only) | 1,164,193 | 45.7% |
| BiomedCLIP-PubMedBERT | 863,450 | 33.9% |
| BiomedBERT (Abstracts + Full Text) | 522,159 | 20.5% |
| Total | 2,549,802 | 100% |
Source: Hugging Face
The combined variants support 102 Spaces and generated 97 fine-tuned derivative models on Hugging Face. Microsoft’s model repository has over 16,400 followers and 290 community likes. The model has accumulated over 1,000 academic citations since its 2020 release.
PubMedBERT Benchmark Performance
PubMedBERT scores 82.91 on the BLURB benchmark with optimized fine-tuning, covering 13 biomedical NLP datasets across six task types. BERT Base scores 78.2 on the same benchmark. That 4.7-point gap comes from PubMedBERT’s domain-specific vocabulary, which reduces input length by 15–20% on biomedical text and avoids fragmenting medical terms into subword tokens.
| Model | BLURB Score | Training Domain | Parameters |
|---|---|---|---|
| PubMedBERT | 82.91 | Biomedical only | 110M |
| BioBERT | 81.09 | General + Biomedical | 110M |
| SciBERT | 79.62 | Scientific papers | 110M |
| BERT Base | 78.20 | General domain | 110M |
| RoBERTa | 77.80 | General domain | 125M |
Source: Microsoft Research, BLURB Benchmark
PubMedBERT Embeddings reach 95.64% correlation on medical text similarity benchmarks. On the PubMedQA question-answering task, the model consistently outperforms general-domain alternatives. Bioformer16L achieves similar BLURB scores with 60% fewer parameters, but PubMedBERT remains the default baseline in most machine learning research involving biomedical text.
PubMedBERT Model Architecture
PubMedBERT follows the BERT Base architecture with 110 million parameters spread across 12 transformer layers. Each layer uses 768 hidden embedding dimensions and 12 attention heads. The model processes sequences up to 512 tokens using a vocabulary built entirely from PubMed literature.
| Architecture Detail | Specification |
|---|---|
| Parameters | 110 million |
| Transformer Layers | 12 |
| Hidden Dimensions | 768 |
| Attention Heads | 12 |
| Max Sequence Length | 512 tokens |
| Vocabulary Source | PubMed-only |
Source: Microsoft Research
The domain-specific vocabulary is a key differentiator. General-purpose BERT breaks the word “acetyltransferase” into seven subword tokens. PubMedBERT treats it as a single token, preserving meaning and saving compute. The 768-dimensional vector embeddings power downstream tasks such as document classification, named entity recognition, and relation extraction across biomedical AI applications.
PubMedBERT Training Data
PubMedBERT was trained on 14 million PubMed abstracts containing 3 billion words across 21 GB of raw text. That covers roughly 36% of the total PubMed database. An extended variant adds full-text articles from PubMed Central, expanding the corpus to 16.8 billion words and 107 GB.
| Training Corpus | Size | Word Count |
|---|---|---|
| Abstracts Only | 21 GB | 3 billion |
| Abstracts + Full Text | 107 GB | 16.8 billion |
Source: Microsoft Research
PubMed itself now holds more than 40 million citations, according to the National Library of Medicine. Approximately 1 million new records are added each year. BiomedCLIP, the vision-language variant, incorporates 15 million image-text pairs from PubMed Central’s PMC-15M dataset, enabling cross-modal retrieval and visual question answering on medical images.
PubMedBERT vs Other Biomedical NLP Models
PubMedBERT outperforms BioBERT, BlueBERT, ClinicalBERT, and SciBERT on biomedical benchmarks. The gap is widest on named entity recognition and document classification, where domain-specific pretraining from scratch beats mixed-domain approaches. BioBERT starts from a general BERT checkpoint and then trains on biomedical data, which limits its vocabulary to the original general-domain set.
| Model | Pretraining Approach | Vocabulary | BLURB Score |
|---|---|---|---|
| PubMedBERT | From scratch (biomedical) | Domain-specific | 82.91 |
| BioBERT | Continual (general + bio) | General | 81.09 |
| SciBERT | From scratch (scientific) | Domain-specific | 79.62 |
| ClinicalBERT | Continual (general + clinical) | General | ~78.5 |
| BlueBERT | Continual (general + bio + clinical) | General | ~78.1 |
Source: BLURB Benchmark, Microsoft Research
BioGPT, also from Microsoft, takes a different approach as a generative model with 347 million parameters trained on 15 million PubMed abstracts. BioGPT recorded 45,315 monthly downloads on Hugging Face in December 2025, far below PubMedBERT’s 2.55 million. For classification and extraction tasks, PubMedBERT remains the standard choice. For text generation in the biomedical domain, BioGPT and larger models like GPT-4 are gaining adoption.
NLP in Healthcare Market Size
The NLP in healthcare and life sciences market was valued at $4.82 billion in 2025, according to Fortune Business Insights. The market is projected to reach $36.71 billion by 2034 at a 26.45% compound annual growth rate. A separate estimate from Towards Healthcare puts the 2025 figure at $8.97 billion with a 34.74% growth rate through 2034, reflecting different scoping definitions.
| Year | Market Size (Fortune BI) | Market Size (MarketsandMarkets) |
|---|---|---|
| 2025 | $4.82B | $5.18B |
| 2026 | $5.62B | $6.49B |
| 2030 | $15.4B (est.) | $16.01B |
| 2034 | $36.71B | — |
Source: Fortune Business Insights, MarketsandMarkets
Clinical documentation is the largest application segment, holding a dominant share in 2025. Text-based NLP accounts for 47.6% of the market in 2026, driven by the volume of unstructured data in electronic health records, radiology reports, and research papers. North America leads all regions. Cloud-based deployment holds 59.1% share by 2026 due to easier scalability and remote access for healthcare providers.
PubMedBERT Use Cases in 2026
PubMedBERT powers clinical documentation improvement, literature mining, pharmacovigilance, and clinical trial matching. In COVID-19 research, a PubMedBERT-based ensemble model achieved an F1 score of 0.9247 on multi-label topic classification of LitCovid articles, beating the baseline by 5.7 percentage points.
Clinical NLP platforms — a category valued at $2.46 billion in 2026 per Fortune Business Insights — rely on BERT-type models for structuring physician notes, discharge summaries, and lab reports. PubMedBERT’s 97 derivative models on Hugging Face cover tasks from specialized language model fine-tuning to zero-shot inference in medical textual entailment. The BioASQ challenge, now in its 14th edition scheduled for CLEF 2026, continues to use PubMedBERT as a baseline architecture.
PubMed Database Growth
PubMed crossed 40 million citations in 2025, up from 30 million in August 2019. The database adds roughly 1 million records per year, sourced from MEDLINE, life science journals, and online books. The National Library of Medicine released the 2026 production year baseline files in January 2026.
| Year | PubMed Total Citations |
|---|---|
| 2019 | 30 million |
| 2021 | ~33 million |
| 2023 | ~36 million |
| 2025 | 39+ million |
| 2026 | 40+ million |
Source: National Library of Medicine (NIH)
PubMed Central, which hosts full-text open-access articles, provides the extended training data for PubMedBERT’s abstracts-plus-full-text variant. The growing volume of biomedical literature keeps domain-specific models like PubMedBERT relevant, as general-domain models trained on web text cannot keep pace with the terminology density found in clinical and biomedical research. For context on how model training costs scale with corpus size, the compute required tracks closely with data volume.
PubMedBERT Statistics — Key Research Trends for 2026
Reinforcement learning with verifiable rewards is expected to expand into biology and chemistry domains in 2026, per analysis from IntuitionLabs. The BioASQ 2026 challenge at CLEF will test biomedical question answering using PubMedBERT and newer architectures. Multi-modal benchmarks combining text with chemical structures and medical images are on the rise.
Microsoft launched Dragon Copilot in March 2025, combining ambient clinical documentation with NLP, built on the same research lineage as PubMedBERT. Heidi, a clinical NLP startup, raised $16.6 million in January 2026 to automate administrative workflows for clinicians. The FDA’s 2025 AI guidance has created stricter requirements for explanation and correctness in biomedical AI, which is expected to push benchmark development toward more realistic multi-step evaluation scenarios.
FAQ
What is PubMedBERT?
PubMedBERT is a biomedical language model developed by Microsoft Research. It was pretrained from scratch on 14 million PubMed abstracts with 110 million parameters, and it outperforms general-domain BERT on biomedical NLP tasks.
How many downloads does PubMedBERT get per month?
PubMedBERT variants recorded 2,549,802 combined monthly downloads on Hugging Face as of 2025. The abstracts-only variant leads with 1,164,193 downloads.
What is the BLURB benchmark score for PubMedBERT?
PubMedBERT scores 82.91 on the BLURB benchmark with optimized fine-tuning, outperforming BERT Base (78.2) by 4.7 points across 13 biomedical NLP datasets.
How large is the NLP in healthcare market?
The NLP in healthcare and life sciences market was valued between $4.82 billion and $8.97 billion in 2025, depending on scope. It is projected to grow at 26-35% CAGR through 2034.
How does PubMedBERT differ from BioBERT?
PubMedBERT is pretrained from scratch on biomedical text with a domain-specific vocabulary. BioBERT starts from a general BERT checkpoint and continues training on biomedical data, retaining the general-domain vocabulary.
Sources:
https://huggingface.co/microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract
https://www.fortunebusinessinsights.com/nlp-in-healthcare-and-life-sciences-market-115852
