Skip to page content
SAVRN
Search Plan a project

Open-weight model · Token classification

atomizer-gliner2.5-base

by Nish nishparadox/atomizer-gliner2.5-base

atomizer-gliner2.5-base is an open-weight model for token classification from Nish, released under Creative Commons Attribution 4.0. Its published files total 810.0 MB.

atomizer on Fastino's GLiNER2.5 base (fastino/gliner2.5-base-v1) with atomizer's standard heads: a single-pass span atomizer with a critic.

Parameters—
Context—
Weights809.9 MB
Licensecc-by-4.0
AccessOpen weights
Monthly Downloads—

Model Card

By Nish, published under cc-by-4.0, revision 55d8daedd9b3.

atomizer on Fastino's GLiNER2.5 base (fastino/gliner2.5-base-v1) with atomizer's standard heads: a single-pass span atomizer with a critic. It splits a text into short, self-contained claims ("atoms") and scores each one for verifiable (is it a checkable factual claim) and checkworthy (is it worth checking). 202M parameters. The model does not generate text. It selects words from the input, replaces references ("She" → "Marie Curie") with earlier spans or keeps them as written, and adds closed-class glue words ("is", "the", "'s"). An atom cannot contain a name or number that is not in the input. One repo per architecture. The atomizer models live in one Hugging Face repo per backbone and…

Read Nish's full model card

atomizer on Fastino's GLiNER2.5 base (fastino/gliner2.5-base-v1) with atomizer's standard heads: a single-pass span atomizer with a critic. It splits a text into short, self-contained claims ("atoms") and scores each one for verifiable (is it a checkable factual claim) and checkworthy (is it worth checking). 202M parameters.

The model does not generate text. It selects words from the input, replaces references ("She" → "Marie Curie") with earlier spans or keeps them as written, and adds closed-class glue words ("is", "the", "'s"). An atom cannot contain a name or number that is not in the input.

One repo per architecture. The atomizer models live in one Hugging Face repo per backbone and head design, so every tag of a repo loads the same way and has the same size and speed. Each repo has its own version numbers:

Repo Backbone Heads
atomizer-gliner-small GLiNER small v2.5 (163M) standard; the releases (main)
atomizer-gliner-large GLiNER large v2.5 standard
atomizer-gliner2.5-base (this repo) Fastino GLiNER2.5 base standard, no BiLSTM
atomizer-gliner2.5-base-native Fastino GLiNER2.5 base GLiNER2.5's own pretrained heads
atomizer-gliner2.5-base-xhead Fastino GLiNER2.5 base head-interaction block

Status: a research ablation. This repo holds the plain-heads baseline of the Fastino line: one version, v0.1 (= main). The native and xhead repos build on it; the model recommended for use is atomizer-gliner-small.

Model

The encoder of Fastino's GLiNER2.5 base (DeBERTa-v3-base, 12 layers; Apache-2.0) goes straight into atomizer's four heads, with no BiLSTM and no span layer (GLiNER2.5 has neither; GLiNER v1 backbones bring theirs pretrained):

Head Predicts
anchor per word: does an atom start here
pair per (atom, word): KEEP / SUB / DROP, and glue after the word
antecedent per reference: which earlier span it means, or none (keep the reference as written)
critic per atom: verifiable, checkworthy

Assembly from the predictions is deterministic; texts longer than 1,024 tokens are atomized in sentence windows with read-only context. Details: ARCHITECTURE.md.

Versions

Load a tag with Atomizer.load("nishparadox/atomizer-gliner2.5-base", revision="<tag>").

Tag Training W&B run
v0.1 one run from fastino/gliner2.5-base-v1 on the v0.5 data (Propositionizer, ClaimBuster, CheckThat! 2024, synthetic v0.3, Decontextualization, Factcheck-Bench, chat templates); 3 epochs, 9,147 steps, batch 32, 1× A100; the last step gliner25-base-nolstm

Until 2026-10-15 it was tag v0.6-rc2 on atomizer-gliner-small's dev branch; the files are the same (model.pt sha256 fbbe075f…).

A100 run of the v0.5 test suite (test splits only; soft F1 unless stated; resolved / wrong on Decontextualization), with the small release and the native model for comparison:

Model synthetic Factcheck-Bench Decontext. resolved wrong ↓ Propositionizer PropSegmEnt CheckThat! F1 / AUC
v0.1 (this repo) 0.912 0.758 0.579 0.741 0.076 0.738 0.757 0.872 / 0.979
small v0.5 (163M) 0.905 0.753 0.551 0.703 0.073 0.742 0.757 0.865 / 0.978
native v0.1 0.918 0.758 0.597 0.728 0.067 0.737 0.766 0.867 / 0.976

Paired bootstraps (2,000 resamples over texts): this model − small v0.5: synthetic exact +0.032, soft +0.006, Decontextualization +0.027, no significant loss. Native v0.1 − this model: synthetic soft +0.007, Decontextualization +0.018, PropSegmEnt +0.010, no loss, which is why the native heads were kept and this line was not trained on the v0.7 data. The same encoder with a randomly initialised BiLSTM (W&B gliner25-base-full) tied with this model and was not published. All numbers: EVALS.md; the runs: EXPERIMENTS.md, runs 8 and 9.

Usage

The loading code is the atomizer package at NISH1001/atomizer (private for now, access on request). This model needs code ≥ v0.1.1; v0.3.0 is current.

uv add "atomizer @ git+ssh://[email protected]/NISH1001/[email protected]"
from atomizer import Atomizer

atomizer = Atomizer.load("nishparadox/atomizer-gliner2.5-base", revision="v0.1")

for atom in atomizer.atomize("Tesla was founded in 2003. It delivered about 1.8 million vehicles in 2023."):
    print(atom.text, round(atom.verifiable, 2), round(atom.checkworthy, 2))

Limitations

Those of the small model (its card), and ~1.6× slower than it on a CPU (12 encoder layers to its 6). It keeps first person as written ("I am Paradox. I won …" keeps "I").

License

The weights are CC-BY-4.0; the atomizer code is MIT. Fine-tuned from Fastino's gliner2.5-base-v1 (Apache-2.0). The training data has its own terms (Propositionizer labels were written by GPT-4; ClaimBuster is CC-BY-SA 4.0; CheckThat! 2024 under the CLEF lab's terms; atomizer-chat-synthetic was written by Claude).

Identity and Version

Repository
nishparadox/atomizer-gliner2.5-base
Publisher
Nish
Task
Token classification
Modality
Text
Library
atomizer
Parameters
Not stated by the source
Languages
Not stated by the source
Revision
55d8daedd9b3aa5c27fa4645197c982a2c825a0b
First published
2026-10-16
Last updated
2026-10-16

Files and Weights

4 files, 810.0 MB in total. The weights are 1 file totalling 809.9 MB in pt.

Weights1 file · 809.9 MB
Configuration1 file · 92.6 KB
Documentation1 file · 6.4 KB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
model.ptWeights809.9 MB fbbe075fdfbe
atomizer.jsonConfiguration92.6 KB —
README.mdDocumentation6.4 KB —
.gitattributesRepository1.5 KB —

License and Download

License
cc-by-4.0
Access
Open weights, no gate
Download size
809.9 MB
Download from Nish

Released by Nish through its official repository on Hugging Face. Read the license.

Built From

  • Derived from fastino/gliner2.5-base-v1
  • Trained on (disclosed) chentong00/propositionizer-wiki-data
  • Trained on (disclosed) nfliu/decontextualization
  • Trained on (disclosed) nishparadox/atomizer-chat-synthetic

Memory Requirements

PrecisionWeights in memory
As published809.9 MB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About atomizer-gliner2.5-base

Can I use atomizer-gliner2.5-base commercially?

Yes. atomizer-gliner2.5-base is released under Creative Commons Attribution 4.0. CC BY 4.0 permits sharing and adapting the work, including commercially, provided the creator is credited and changes are indicated.

Similar Models

Model · Token classification

stanford-deidentifier-base

Stanford AIMI

Stanford de-identifier was trained on a variety of radiology and biomedical documents with the goal of automatising the de-identification process while reaching satisfactory accuracy for use in production. Manuscript in-proceedings. These model weights are the recommended ones among all available deidentifier weights. This work was supported in part by the Medical Imaging and Data Resource Center (MIDRC), which is funded by the National Institute of Biomedical Imaging and Bioengineering (NIBIB) of the National Institutes of Health under contract 75N92020D00021 and through The Advanced Research Projects Agency for Health (ARPA-H)

Open weights mit 512 tokens transformers

Model · Token classification

bert-portuguese-ner

Luís Filipe Cunha

This model is a fine-tuned version of neuralmind/bert-base-portuguese-cased It achieves the following results on the evaluation set: This model was fine-tunned on token classification task (NER) on Portuguese archival documents. The annotated labels are: Date, Profession, Person, Place, Organization All the training and evaluation data is available at: http://ner.epl.di.uminho.pt/ The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 16 - evalbatchsize: 16 - lrschedulertype: linear - numepochs: 4 - Transformers 4.10.0.dev0 - Pytorch 1.9.0+cu111 - Datasets 1.10.2 - Tokenizers 0.10.3

Open weights mit 512 tokens transformers

Model · Token classification

privacy-filter-nemotron-GGUF

LocalAI-io

GGUF conversion of OpenMed/privacy-filter-nemotron, a fine-grained PII token-classification model — a fine-tune of openai/privacy-filter on the nvidia/Nemotron-PII dataset. It labels every token with a BIOES tag over 55 PII categories (221 classes) in a single forward pass, then decodes coherent spans with a constrained Viterbi procedure — so it can be served locally with no Python as the encoder/NER tier of a PII redactor. Where the base openai/privacy-filter covers 8 coarse categories, this fine-tune trades multilingual breadth for category depth: 55 fine-grained English categories (first/last name, government IDs, financial, healthcare, vehicle, digital, …). For the full model…

Open weights apache-2.0 gguf

Model · Token classification

punctuate-all

KREDOR

This is based on Oliver Guhr's work. The difference is that it is a finetuned xlm-roberta-base instead of an xlm-roberta-large and on twelve languages instead of four. The languages are: English, German, French, Spanish, Bulgarian, Italian, Polish, Dutch, Czech, Portugese, Slovak, Slovenian. precision recall f1-score support accuracy 0.98 84425503 macro avg 0.83 0.74 0.77 84425503 weighted avg 0.98 0.98 0.98 84425503

Open weights mit 514 tokens transformers

Model · Token classification

privacy-filter-multilingual-GGUF

LocalAI-io

GGUF conversion of OpenMed/privacy-filter-multilingual, a multilingual PII token-classification model (a fine-tune of openai/privacy-filter). It labels every token with a BIOES tag over 54 PII categories (217 classes) across 16 languages, so it can be served locally with no Python as the encoder/NER tier of a PII redactor. For the full model description, label space, evaluation, limitations, and citations, see the source model card — this card only covers the GGUF packaging and how to run it. This GGUF uses a custom architecture, openai-privacy-filter, that is not (yet) part of 1. privacy-filter.cpp (recommended) — a small standalone GGML engine for exactly this model family, on stock…

Open weights apache-2.0 gguf

Model · Token classification

unbiased-toxic-roberta-onnx

Protect AI

This model is a conversion of unitary/unbiased-toxic-roberta to ONNX format using the Optimum library. Trained models & code to predict toxic comments on 3 Jigsaw challenges: Toxic comment classification, Unintended Bias in Toxic comments, Multilingual toxic comment classification. Built by Laura Hanu at Unitary. The huggingface models currently give different results to the detoxify library (see issue here). All challenges have a toxicity label. The toxicity labels represent the aggregate ratings of up to 10 annotators according the following schema: - Very Toxic (a very hateful, aggressive, or disrespectful comment that is very likely to make you leave a discussion or give up on sharing…

Open weights apache-2.0 514 tokens transformers