Skip to page content
SAVRN
Search Plan a project

Open-weight model · Token classification

2026-10-13-xlmr-large-baseline_2_2

by Clara Wan Ching Ho c-ho/2026-10-13-xlmr-large-baseline_2_2

2026-10-13-xlmr-large-baseline_2_2 is an open-weight model for token classification from Clara Wan Ching Ho, released under MIT License. It has 559M parameters and a 514-token context. At 16-bit it needs about 1.3 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.

This model is a fine-tuned version of FacebookAI/xlm-roberta-large on an unknown dataset.

Parameters559M
Context514
Weights2.2 GB
Licensemit
AccessOpen weights
Monthly Downloads—

Runs On

What it takes to serve 2026-10-13-xlmr-large-baseline_2_2 (559M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 1.1 GB 1.3 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 0.6 GB 0.7 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 0.3 GB 0.3 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 24, 2026.

2026-10-13-xlmr-large-baseline_2_2 on every accelerator the SAVRN Index prices, at every precision

Model Card

By Clara Wan Ching Ho, published under mit, revision 5fe76bae1b4a.

This model is a fine-tuned version of FacebookAI/xlm-roberta-large on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 8 - evalbatchsize: 8 - gradientaccumulationsteps: 8 - totaltrainbatchsize: 64 - lrschedulertype: linear - lrschedulerwarmupratio: 0.1 - numepochs: 10 - mixedprecisiontraining: Native AMP - Transformers 4.57.6 - Pytorch 2.5.1+cu121 - Datasets 3.2.0 - Tokenizers 0.22.2

Read Clara Wan Ching Ho's full model card

This model is a fine-tuned version of FacebookAI/xlm-roberta-large on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.3299 - Precision: 0.5774 - Recall: 0.4901 - F1: 0.5301 - Accuracy: 0.9514 - Academicdiscipline F1: 0.3704 - Ambiguouslydefinedconcept F1: 0.0 - Discoursephenomenon F1: 0.1481 - Graphemicphenomenon F1: 0.1026 - Languagerelatedterm F1: 0.7349 - Languageresourceinformation F1: 0.3160 - Lexicalphenomenon F1: 0.4123 - Morphologicalphenomenon F1: 0.5226 - Morphosyntacticphenomenon F1: 0.7196 - New Tag F1: 0.4816 - Otherlinguisticterm F1: 0.5147 - Phonologicalphenomenon F1: 0.6903 - Semanticphenomenon F1: 0.1282 - Syntacticphenomenon F1: 0.5994 - Topnode Dummy F1: 0.3435 - Unclassifiedlinguisticconcept F1: 0.4651

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - gradient_accumulation_steps: 8 - total_train_batch_size: 64 - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: linear - lr_scheduler_warmup_ratio: 0.1 - num_epochs: 10 - mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Precision Recall F1 Accuracy Academicdiscipline F1 Ambiguouslydefinedconcept F1 Discoursephenomenon F1 Graphemicphenomenon F1 Languagerelatedterm F1 Languageresourceinformation F1 Lexicalphenomenon F1 Morphologicalphenomenon F1 Morphosyntacticphenomenon F1 New Tag F1 Otherlinguisticterm F1 Phonologicalphenomenon F1 Semanticphenomenon F1 Syntacticphenomenon F1 Topnode Dummy F1 Unclassifiedlinguisticconcept F1
No log 1.0 260 0.2171 0.4200 0.3406 0.3762 0.9379 0.0 0.0 0.0 0.0 0.5066 0.1961 0.0253 0.2412 0.6204 0.0827 0.3518 0.4968 0.0 0.4664 0.2460 0.1690
0.3991 2.0 520 0.1957 0.4661 0.4681 0.4671 0.9427 0.0 0.0 0.0 0.0 0.6755 0.4581 0.3160 0.4162 0.6143 0.1652 0.4663 0.6551 0.0948 0.5155 0.3448 0.5882
0.3991 3.0 780 0.1917 0.5431 0.4903 0.5154 0.9482 0.3333 0.0 0.0 0.0 0.6802 0.4019 0.4232 0.5266 0.6770 0.3801 0.4935 0.6889 0.1235 0.6398 0.3123 0.4000
0.0824 4.0 1040 0.2011 0.5805 0.4743 0.5221 0.9504 0.4762 0.0 0.0 0.0 0.7170 0.3333 0.4011 0.6058 0.6949 0.4276 0.5215 0.7157 0.1382 0.6295 0.3226 0.4762
0.0824 5.0 1300 0.2408 0.5609 0.4874 0.5216 0.9496 0.4348 0.0 0.2000 0.0909 0.7040 0.3411 0.4082 0.5809 0.6710 0.5035 0.4827 0.6722 0.1516 0.6346 0.2975 0.3509
0.0392 6.0 1560 0.2533 0.5735 0.4763 0.5204 0.9502 0.4762 0.0 0.0 0.1379 0.7136 0.3146 0.3549 0.5899 0.7167 0.4595 0.5088 0.6870 0.1219 0.5861 0.3361 0.4762
0.0392 7.0 1820 0.2708 0.5538 0.4965 0.5236 0.9494 0.2353 0.0 0.0 0.1053 0.7250 0.3562 0.4070 0.5732 0.6891 0.4889 0.5184 0.6896 0.1344 0.5808 0.3429 0.3922
0.0183 8.0 2080 0.2996 0.5798 0.4862 0.5289 0.9509 0.4138 0.0 0.0 0.1000 0.7263 0.3241 0.4136 0.5342 0.7189 0.4821 0.5079 0.6940 0.1158 0.5968 0.3339 0.4444
0.0183 9.0 2340 0.3169 0.5729 0.4911 0.5289 0.9512 0.4545 0.0 0.2000 0.0909 0.7366 0.3089 0.4278 0.4986 0.7199 0.4817 0.5226 0.6979 0.1292 0.5907 0.3387 0.4255
0.0078 10.0 2600 0.3299 0.5774 0.4901 0.5301 0.9514 0.3704 0.0 0.1481 0.1026 0.7349 0.3160 0.4123 0.5226 0.7196 0.4816 0.5147 0.6903 0.1282 0.5994 0.3435 0.4651

Framework versions

  • Transformers 4.57.6
  • Pytorch 2.5.1+cu121
  • Datasets 3.2.0
  • Tokenizers 0.22.2

Configuration

Architecture
XLMRobertaForTokenClassification
Context length (tokens)
514
Layers
24
Hidden size
1,024
Feed-forward size
4,096
Attention heads
16
Vocabulary size
250,002
Model type
xlm-roberta

Identity and Version

Repository
c-ho/2026-10-13-xlmr-large-baseline_2_2
Publisher
Clara Wan Ching Ho
Task
Token classification
Modality
Text
Library
transformers
Parameters
559M parameters
Languages
xlm-roberta
Revision
5fe76bae1b4aedd9833520047b59978982692577
First published
2026-10-13
Last updated
2026-10-13

Files and Weights

9 files, 2.3 GB in total. The weights are 2 files totalling 2.2 GB in bin, safetensors.

Weights2 files · 2.2 GB
Configuration2 files · 3.2 KB
Tokenizer2 files · 17.1 MB
Documentation1 file · 8.2 KB
Other1 file · 5.1 MB
Repository1 file · 1.6 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights2.2 GB 8b5f6100b05d
training_args.binWeights5.5 KB dee4c43442da
config.jsonConfiguration3.0 KB —
special_tokens_map.jsonConfiguration280 B —
README.mdDocumentation8.2 KB —
sentencepiece.bpe.modelOther5.1 MB cfc8146abe2a
.gitattributesRepository1.6 KB —
tokenizer.jsonTokenizer17.1 MB 883b03711108
tokenizer_config.jsonTokenizer1.2 KB —

License and Download

License
mit
Access
Open weights, no gate
Download size
2.2 GB
Download from Clara Wan Ching Ho

Released by Clara Wan Ching Ho through its official repository on Hugging Face. Read the license.

Built From

Memory Requirements

PrecisionWeights in memory
As published2.2 GB
16-bit1.1 GB
8-bit0.6 GB
4-bit0.3 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About 2026-10-13-xlmr-large-baseline_2_2

How much GPU memory does 2026-10-13-xlmr-large-baseline_2_2 need?

About 1.3 GB at 16-bit and 0.3 GB at 4-bit: the weights (559M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run 2026-10-13-xlmr-large-baseline_2_2 on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use 2026-10-13-xlmr-large-baseline_2_2 commercially?

Yes. 2026-10-13-xlmr-large-baseline_2_2 is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.

What is 2026-10-13-xlmr-large-baseline_2_2's context length?

514 tokens, from the maximum position embeddings in its published configuration.

Similar Models

This model is a fine-tuned version of FacebookAI/xlm-roberta-large on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 8 - evalbatchsize: 8 - gradientaccumulationsteps: 8 - totaltrainbatchsize: 64 - lrschedulertype: linear - lrschedulerwarmupratio: 0.1 - numepochs: 10 - mixedprecisiontraining: Native AMP - Transformers 4.57.6 - Pytorch 2.5.1+cu121 - Datasets 3.2.0 - Tokenizers 0.22.2

Open weights mit 559M parameters 514 tokens transformers
View model

This model is a fine-tuned version of FacebookAI/xlm-roberta-large on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 3e-05 - trainbatchsize: 8 - evalbatchsize: 8 - gradientaccumulationsteps: 8 - totaltrainbatchsize: 64 - lrschedulertype: linear - lrschedulerwarmupratio: 0.1 - numepochs: 10 - mixedprecisiontraining: Native AMP - Transformers 4.57.6 - Pytorch 2.5.1+cu121 - Datasets 3.2.0 - Tokenizers 0.22.2

Open weights mit 559M parameters 514 tokens transformers
View model

Model · Token classification

fullstop-punctuation-multilang-large

Oliver Guhr

This model predicts the punctuation of English, Italian, French and German texts. We developed it to restore the punctuation of transcribed spoken language. This multilanguage model was trained on the Europarl Dataset provided by the SEPP-NLG Shared Task. Please note that this dataset consists of political speeches. Therefore the model might perform differently on texts from other domains. The model restores the following punctuation markers: "." "," "?" "-" ":" We provide a simple python package that allows you to process text of any length. To get started install the package from pypi: output output The performance differs for the single punctuation markers as hyphens and colons, in many…

Open weights mit 559M parameters 514 tokens transformers
View model

Model · Token classification

OpenMed-NER-GenomicDetect-BigMed-560M

OpenMed

Specialized model for Gene Entity Recognition - Gene-related entities This model is a state-of-the-art fine-tuned transformer engineered to deliver enterprise-grade accuracy for gene entity recognition - gene-related entities. This specialized model excels at identifying and extracting biomedical entities from clinical texts, research papers, and healthcare documents, enabling applications such as drug interaction detection, medication extraction from patient records, adverse event monitoring, literature mining for drug discovery, and biomedical knowledge graph construction with production-ready reliability for clinical and research applications. This model can identify and classify the…

Open weights apache-2.0 559M parameters 514 tokens transformers
View model

Model · Token classification

OpenMed-NER-ChemicalDetect-BigMed-560M

OpenMed

Specialized model for Chemical Entity Recognition - Identifies chemical compounds and substances in biomedical literature This model is a state-of-the-art fine-tuned transformer engineered to deliver enterprise-grade accuracy for chemical entity recognition - identifies chemical compounds and substances in biomedical literature. This specialized model excels at identifying and extracting biomedical entities from clinical texts, research papers, and healthcare documents, enabling applications such as drug interaction detection, medication extraction from patient records, adverse event monitoring, literature mining for drug discovery, and biomedical knowledge graph construction with…

Open weights apache-2.0 559M parameters 514 tokens transformers
View model