# codebert-python vs roberta-base: Specs, License and GPU Cost
Source: https://savrn.com/models/compare/codebert-python-vs-roberta-base
Markdown alternate of the page above; the site index is https://savrn.com/llms.txt

---

| Field | [codebert-python](https://savrn.com/models/codebert-python) neulab/codebert-python | [roberta-base](https://savrn.com/models/roberta-base) FacebookAI/roberta-base |
| --- | --- | --- |
| Publisher | NeuLab @ LTI/CMU | Facebook AI community |
| Task | Fill mask | Fill mask |
| Modality | Text | Text |
| Parameters, as reported | 125M parameters | 125M parameters |
| Architecture | RobertaForMaskedLM | RobertaForMaskedLM |
| Library | transformers | transformers |
| Context length | 514 tokens | 514 tokens |
| Repository size | 1.0 GB | 2.8 GB |
| Artifact formats | safetensors, pytorch | safetensors, pytorch, jax, rust, tf |
| License | Not stated | mit |
| Access | Open weights, no gate | Open weights, no gate |
| Memory at 16-bit (weights and margin) | 0.3 GB | 0.3 GB |
| Cheapest GPUs at 16-bit, per hour | 1x MI300X, $1.85 | 1x MI300X, $1.85 |
| Memory at 4-bit (weights and margin) | 0.1 GB | 0.1 GB |
| Cheapest GPUs at 4-bit, per hour | 1x MI300X, $1.85 | 1x MI300X, $1.85 |
| Revision viewed | c4ff459e2a91 | e2da8e2f811d |
| Downloads reported by the hub | 359.4k | 7M |
| Last observed | 2026-09-18 | 2026-10-23 |

An evaluation row appears only where at least two of these models report the same benchmark with the same stated configuration, metric, unit and setup. Different evaluators stay named in each cell. Values are shown as reported: no unit conversion, no ranking.

## SAVRN's Notes on codebert-python

This is not a code generator. CMU's NeuLab trained CodeBERT on Python for one million steps so it can score generated code against a reference, as the engine behind the CodeBERTScore metric. At 125M parameters it runs anywhere.

The repository states no license, so treat it as research use until that changes. If you are evaluating coding models, this is a useful second opinion next to unit tests, because it rewards code that means the same thing even when the text differs. It does not replace running the tests.

SAVRN Research, 2026-09-18

## SAVRN's Notes on roberta-base

Case matters here: roberta-base, from the Facebook AI community, tells english from English. Pretrained on raw English text from bookcorpus and wikipedia by masked language modeling, its listed task is fill mask: token prediction and what you build on it, not chat. At 16-bit its 125M parameters take 0.2 GB of weights and 0.3 GB to run, against a cheapest listed setup of one 192 GB MI300X at $1.85 per hour on-demand; on our floor that shares a card, never owns one.

MIT is short: commercial use, modification and redistribution are allowed, the one duty being to carry the copyright and permission notices along. Check provenance and packaging. The releasing team wrote no model card, so bookcorpus and wikipedia are all the page discloses on data. The repository is 2.82 GB across 13 files in safetensors, pytorch, jax, rust and tf; pull only the format your stack loads.

SAVRN Research, 2026-09-18

## Questions

### Which is larger, codebert-python or roberta-base?

codebert-python (125M parameters) is larger than roberta-base (125M parameters), by the parameter counts their publishers report.

### Which is cheaper to run, codebert-python or roberta-base?

At 4-bit, codebert-python fits on 1x MI300X from $1.85 an hour and roberta-base on 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

### Can I use roberta-base commercially?

Yes. roberta-base is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.

## Related Comparisons

- [bert-base-arabertv02 vs roberta-base](https://savrn.com/models/compare/bert-base-arabertv02-vs-roberta-base)
- [distilbert-base-multilingual-cased vs roberta-base](https://savrn.com/models/compare/distilbert-base-multilingual-cased-vs-roberta-base)