SAVRN
Search Contact SAVRN

Open-weight model · Token classification

deid_roberta_i2b2

by One Brave Idea obi/deid_roberta_i2b2

A RoBERTa [[Liu et al., 2019]](https://arxiv.org/pdf/1907.11692.pdf) model fine-tuned for de-identification of medical notes. A token can either be classified as non-PHI or as one of the 11 PHI types.

Parameters354M
Context514
Weights2.8 GB
Licensemit
AccessOpen weights
Monthly Downloads335.4k

Runs On

What it takes to serve deid_roberta_i2b2 (354M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 0.7 GB 0.9 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 0.4 GB 0.4 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 0.2 GB 0.2 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

SAVRN's Notes on deid_roberta_i2b2

Medical notes carry names, dates and record numbers that have to come out, and that is the job here: each token is labeled non-PHI or one of 11 PHI types, and BILOU tagging joins them into spans. A 354M parameter RoBERTa needs 0.9 GB at 16-bit, so the cheapest card in our table, one MI300X with 192 GB at $1.85 per hour on-demand, is far more than it needs, and it runs as one tenant among many.

MIT covers commercial use, modification and redistribution provided the copyright and permission notices stay attached, so a vendor can build it into a de-identification service. The check that matters is the 514 token context: long notes must be chunked, and the seams must not split a span. Trained on I2B2, released March 2, 2022 and updated February 23, 2025, it is stored in float32, so the download is 2.8 GB against 0.7 GB of 16-bit weights.

Model Card

By One Brave Idea, published under mit, revision 78f2152eb93d.

Model Description

  • A RoBERTa [Liu et al., 2019] model fine-tuned for de-identification of medical notes.
  • Sequence Labeling (token classification): The model was trained to predict protected health information (PHI/PII) entities (spans). A list of protected health information categories is given by HIPAA.
  • A token can either be classified as non-PHI or as one of the 11 PHI types. Token predictions are aggregated to spans by making use of BILOU tagging.
  • The PHI labels that were used for training and other details can be found here: Annotation Guidelines
  • More details on how to use this model, the format of data and other useful information is present in the GitHub repo: Robust DeID.

How to use

Read the full model card (478 words)

Configuration

Architecture
RobertaForTokenClassification
Context length (tokens)
514
Layers
24
Hidden size
1,024
Feed-forward size
4,096
Attention heads
16
Vocabulary size
50,265
Stored precision
float32
Model type
roberta

Identity and Version

Repository
obi/deid_roberta_i2b2
Publisher
One Brave Idea
Task
Token classification
Modality
Text
Library
transformers
Parameters
354M parameters
Languages
en
Revision
78f2152eb93ddd817290ce8dbe46f1a6685e09fc
First published
2022-03-02
Last updated
2025-02-23

Files and Weights

10 files, 2.8 GB in total. The weights are 2 files totalling 2.8 GB in bin, safetensors.

Weights2 files · 2.8 GB
Configuration2 files · 2.7 KB
Tokenizer4 files · 2.6 MB
Documentation1 file · 5.3 KB
Repository1 file · 1.2 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights1.4 GB 557152273204
pytorch_model.binWeights1.4 GB 76e5e75ee896
config.jsonConfiguration2.5 KB
special_tokens_map.jsonConfiguration239 B
README.mdDocumentation5.3 KB
.gitattributesRepository1.2 KB
merges.txtTokenizer456.4 KB
tokenizer.jsonTokenizer1.4 MB
tokenizer_config.jsonTokenizer351 B
vocab.jsonTokenizer798.3 KB

License and Download

License
mit
Access
Open weights, no gate
Download size
2.8 GB
Download from One Brave Idea

Released by One Brave Idea through its official repository on Hugging Face. Read the license.

Built From

Memory Requirements

PrecisionWeights in memory
As published2.8 GB
16-bit0.7 GB
8-bit0.4 GB
4-bit0.2 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About deid_roberta_i2b2

How much GPU memory does deid_roberta_i2b2 need?

About 0.9 GB at 16-bit and 0.2 GB at 4-bit: the weights (354M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run deid_roberta_i2b2 on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use deid_roberta_i2b2 commercially?

Yes. deid_roberta_i2b2 is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.

What is deid_roberta_i2b2's context length?

514 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Token classification

OpenMed-NER-DiseaseDetect-BioMed-335M

OpenMed

Specialized model for Disease Entity Recognition - Disease entities from the BC5CDR dataset This model is a state-of-the-art fine-tuned transformer engineered to deliver enterprise-grade accuracy for disease entity recognition - disease entities from the bc5cdr dataset. This specialized model excels at identifying and extracting biomedical entities from clinical texts, research papers, and healthcare documents, enabling applications such as drug interaction detection, medication extraction from patient records, adverse event monitoring, literature mining for drug discovery, and biomedical knowledge graph construction with production-ready reliability for clinical and research applications.…

Open weights apache-2.0 334M parameters 512 tokens transformers

Model · Token classification

OpenMed-NER-GenomicDetect-PubMed-335M

OpenMed

Specialized model for Gene Entity Recognition - Gene-related entities This model is a state-of-the-art fine-tuned transformer engineered to deliver enterprise-grade accuracy for gene entity recognition - gene-related entities. This specialized model excels at identifying and extracting biomedical entities from clinical texts, research papers, and healthcare documents, enabling applications such as drug interaction detection, medication extraction from patient records, adverse event monitoring, literature mining for drug discovery, and biomedical knowledge graph construction with production-ready reliability for clinical and research applications. This model can identify and classify the…

Open weights apache-2.0 334M parameters 512 tokens transformers

Model · Token classification

bert-large-NER

D

If my open source models have been useful to you, please consider supporting me in building small, useful AI models for everyone (and help me afford med school / help out my parents financially). Thanks! bert-large-NER is a fine-tuned BERT model that is ready to use for Named Entity Recognition and achieves state-of-the-art performance for the NER task. It has been trained to recognize four types of entities: location (LOC), organizations (ORG), person (PER) and Miscellaneous (MISC). Specifically, this model is a bert-large-cased model that was fine-tuned on the English version of the standard CoNLL-2003 Named Entity Recognition dataset. If you'd like to use a smaller BERT model fine-tuned…

Open weights mit 334M parameters 512 tokens transformers

Model · Token classification

OpenMed-NER-ChemicalDetect-ModernMed-395M

OpenMed

Specialized model for Chemical Entity Recognition - Identifies chemical compounds and substances in biomedical literature This model is a state-of-the-art fine-tuned transformer engineered to deliver enterprise-grade accuracy for chemical entity recognition - identifies chemical compounds and substances in biomedical literature. This specialized model excels at identifying and extracting biomedical entities from clinical texts, research papers, and healthcare documents, enabling applications such as drug interaction detection, medication extraction from patient records, adverse event monitoring, literature mining for drug discovery, and biomedical knowledge graph construction with…

Open weights apache-2.0 396M parameters 8,192 tokens transformers

Model · Token classification

OpenMed-NER-GenomeDetect-ModernMed-395M

OpenMed

Specialized model for Gene/Protein Entity Recognition - Gene and protein mentions This model is a state-of-the-art fine-tuned transformer engineered to deliver enterprise-grade accuracy for gene/protein entity recognition - gene and protein mentions. This specialized model excels at identifying and extracting biomedical entities from clinical texts, research papers, and healthcare documents, enabling applications such as drug interaction detection, medication extraction from patient records, adverse event monitoring, literature mining for drug discovery, and biomedical knowledge graph construction with production-ready reliability for clinical and research applications. This model can…

Open weights apache-2.0 396M parameters 8,192 tokens transformers