# ModernBERT-large vs xlm-roberta-large: Specs, License, GPU Cost Source: https://savrn.com/models/compare/modernbert-large-vs-xlm-roberta-large Markdown alternate of the page above; the site index is https://savrn.com/llms.txt --- | Field | [ModernBERT-large](https://savrn.com/models/modernbert-large) answerdotai/ModernBERT-large | [xlm-roberta-large](https://savrn.com/models/xlm-roberta-large) FacebookAI/xlm-roberta-large | | --- | --- | --- | | Publisher | Answer.AI | Facebook AI community | | Task | Fill mask | Fill mask | | Modality | Text | Text | | Parameters, as reported | 396M parameters | 561M parameters | | Architecture | ModernBertForMaskedLM | XLMRobertaForMaskedLM | | Library | transformers | transformers | | Context length | 8,192 tokens | 514 tokens | | Repository size | 7.9 GB | 11.2 GB | | Artifact formats | safetensors, onnx, pytorch | safetensors, onnx, pytorch, jax, tf | | License | apache-2.0 | mit | | Access | Open weights, no gate | Open weights, no gate | | Memory at 16-bit (weights and margin) | 1 GB | 1.3 GB | | Cheapest GPUs at 16-bit, per hour | 1x MI300X, $1.85 | 1x MI300X, $1.85 | | Memory at 4-bit (weights and margin) | 0.2 GB | 0.3 GB | | Cheapest GPUs at 4-bit, per hour | 1x MI300X, $1.85 | 1x MI300X, $1.85 | | Revision viewed | 45bb4654a4d5 | c23d21b0620b | | Downloads reported by the hub | 750.2k | 2.3M | | Last observed | 2026-09-18 | 2026-10-24 | An evaluation row appears only where at least two of these models report the same benchmark with the same stated configuration, metric, unit and setup. Different evaluators stay named in each cell. Values are shown as reported: no unit conversion, no ranking. ## SAVRN's Notes on ModernBERT-large Nothing here generates text. Answer.AI built ModernBERT-large as an encoder, trained on 2 trillion tokens of English and code, and our reading is retrieval and classification over long documents, up to its 8,192-token context. Memory is the smallest line item in the rack: 1.0 GB at 16-bit, 0.2 GB at 4-bit, against a cheapest slot of one MI300X with 192 GB at $1.85 an hour, so it rides alongside a generation model rather than earning its own card. The license is Apache 2.0: commercial use, modification and redistribution are allowed, notices stay, significant changes get stated, and contributors grant patents. Access is open. Check the disk footprint first, 16 files and 7.86 GB stored in float32, several times what loads into memory at 16-bit. It ships in safetensors, onnx and pytorch. The design paper is arXiv:2412.13663; read it for the local-global alternating attention behind the 8,192-token window. SAVRN Research, 2026-09-18 ## SAVRN's Notes on xlm-roberta-large Fitting this one is never the problem. At 16-bit the weights are 1.1 GB and the run needs 1.3 GB, so the cheapest setup on the page, one 192 GB MI300X at $1.85 an hour on-demand, has room for more than a hundred copies; put it on a GPU already serving something else. What the footprint buys is a fill-mask model trained on 2.5 TB of filtered CommonCrawl in 100 languages with a 250,002-entry vocabulary: a starting layer for multilingual text work you tune yourself, not a finished application. MIT asks only that the copyright and permission notice travel with what you ship; commercial use, modification and redistribution are allowed. Note the 514-token context, sentences and short passages rather than documents, and the 17 files totaling 11.2 GB across five formats, of which your runtime needs one. The releasing team did not write the card; start from arXiv 1911.02116. SAVRN Research, 2026-09-18 ## Questions ### Which is larger, ModernBERT-large or xlm-roberta-large? xlm-roberta-large (561M parameters) is larger than ModernBERT-large (396M parameters), by the parameter counts their publishers report. ### Which is cheaper to run, ModernBERT-large or xlm-roberta-large? At 4-bit, ModernBERT-large fits on 1x MI300X from $1.85 an hour and xlm-roberta-large on 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists. ### Can I use ModernBERT-large commercially? Yes. ModernBERT-large is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors. ### Can I use xlm-roberta-large commercially? Yes. xlm-roberta-large is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included. ## Related Comparisons - [bert-large-uncased vs xlm-roberta-large](https://savrn.com/models/compare/bert-large-uncased-vs-xlm-roberta-large) - [esm2_t33_650M_UR50D vs xlm-roberta-large](https://savrn.com/models/compare/esm2-t33-650m-ur50d-vs-xlm-roberta-large)