# bloomz-560m by BigScience Workshop: Open-Weight Model
Source: https://savrn.com/models/bloomz-560m
Markdown alternate of the page above; the site index is https://savrn.com/llms.txt

---

## Runs On

What it takes to serve bloomz-560m (559M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
| --- | --- | --- | --- | --- | --- |
| 16-bit | 1.1 GB | 1.3 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 |
| 8-bit | 0.6 GB | 0.7 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 |
| 4-bit | 0.3 GB | 0.3 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 |

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the [SAVRN Index](https://savrn.com/ai-index/pricing/gpus), read Oct 7, 2026.

[bloomz-560m on every accelerator the SAVRN Index prices, at every precision](https://savrn.com/models/bloomz-560m/gpus)

## Model Card

By BigScience Workshop, published under bigscience-bloom-rail-1.0, revision a2845d7e13dd.

### Table of Contents

1. Model Summary
2. Use
3. Limitations
4. Training
5. Evaluation
6. Citation

### Model Summary

We present BLOOMZ & mT0, a family of models capable of following human instructions in dozens of languages zero-shot. We finetune BLOOM & mT5 pretrained multilingual language models on our crosslingual task mixture (xP3) and find the resulting models capable of crosslingual generalization to unseen tasks & languages.

- Repository: [bigscience-workshop/xmtf](https://github.com/bigscience-workshop/xmtf)
- Paper: [Crosslingual Generalization through Multitask Finetuning](https://arxiv.org/abs/2211.01786)
- Point of Contact: Niklas Muennighoff
- Languages: Refer to [bloom](https://huggingface.co/bigscience/bloom) for pretraining & [xP3](https://huggingface.co/datasets/bigscience/xP3) for finetuning language proportions. It understands both pretraining & finetuning languages.
- BLOOMZ & mT0 Model Family:

[Read the full model card (725 words)](https://savrn.com/models/bloomz-560m/card)

## Configuration

Architecture

BloomForCausalLM

Attention heads

16

Vocabulary size

250,880

Model type

bloom

## Identity and Version

Repository

bigscience/bloomz-560m

Publisher

BigScience Workshop

Task

Text generation

Modality

Text

Library

transformers

Parameters

559M parameters

Languages

ak, ar, as, bm, bn, ca, en, es

Revision

a2845d7e13dd12efae154a9f1c63fcc2e0cc4b05

First published

2022-10-08

Last updated

2023-05-27

## Files and Weights

14 files, 2.3 GB in total. The weights are 2 files totalling 2.2 GB in bin, safetensors.

Weights2 files · 2.2 GB

Configuration2 files · 800 B

Tokenizer2 files · 14.5 MB

Documentation1 file · 24.6 KB

Other6 files · 24.2 MB

Repository1 file · 1.5 KB

Every file

| File | Type | Size | SHA-256 |
| --- | --- | --- | --- |
| model.safetensors | Weights | 1.1 GB | 365b2c5e9bd1 |
| pytorch_model.bin | Weights | 1.1 GB | 5c50d925a996 |
| config.json | Configuration | 715 B | — |
| special_tokens_map.json | Configuration | 85 B | — |
| README.md | Documentation | 24.6 KB | — |
| logs/logs/xp3capmixnewcodelonglossseq/main_log.txt | Other | 7.1 MB | — |
| logs/tensorboard/xp3capmixnewcodelonglossseq/events.out.tfevents.1665051802.jean-zay-iam25.73114.0 | Other | 40 B | d246bc97c576 |
| logs/tensorboard/xp3capmixnewcodelonglossseq/events.out.tfevents.1665052484.jean-zay-iam17.3156474.0 | Other | 40 B | 56284116193f |
| logs/tensorboard/xp3capmixnewcodelonglossseq/events.out.tfevents.1665052656.jean-zay-iam17.3157134.0 | Other | 40 B | 297416e34370 |
| logs/tensorboard/xp3capmixnewcodelonglossseq/events.out.tfevents.1665052772.jean-zay-iam17.3157698.0 | Other | 71.1 KB | 094d514a3322 |
| logs/tensorboard/xp3capmixnewcodelonglossseq/events.out.tfevents.1665052983.jean-zay-iam17.3161807.0 | Other | 17.0 MB | ea36c54615f6 |
| .gitattributes | Repository | 1.5 KB | — |
| tokenizer.json | Tokenizer | 14.5 MB | 3fa39cd4b150 |
| tokenizer_config.json | Tokenizer | 222 B | — |

## License and Download

License

bigscience-bloom-rail-1.0

Access

Open weights, no gate

Download size

2.2 GB

[Download from BigScience Workshop](https://huggingface.co/bigscience/bloomz-560m)

Released by BigScience Workshop through its official repository on Hugging Face.

## Built From

- Described by arXiv:2211.01786
- Trained on (disclosed) bigscience/xP3

## Evaluations

Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical.

| Benchmark | Conditions | Result | Reported by | Revision | Date |
| --- | --- | --- | --- | --- | --- |
| ANLI (r1) | Configuration r1Task Natural language inferenceMetric AccuracyComparison conditions not established | 33.4 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| ANLI (r2) | Configuration r2Task Natural language inferenceMetric AccuracyComparison conditions not established | 33.4 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| ANLI (r3) | Configuration r3Task Natural language inferenceMetric AccuracyComparison conditions not established | 33.5 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| HumanEval | Configuration NoneTask Program synthesisMetric Pass@1Comparison conditions not established | 2.18 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| HumanEval | Configuration NoneTask Program synthesisMetric Pass@10Comparison conditions not established | 4.11 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| HumanEval | Configuration NoneTask Program synthesisMetric Pass@100Comparison conditions not established | 9 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| StoryCloze (2016) | Configuration 2016Task Sentence completionMetric AccuracyComparison conditions not established | 60.29 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| SuperGLUE (cb) | Configuration cbTask Natural language inferenceMetric AccuracyComparison conditions not established | 53.57 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| SuperGLUE (copa) | Configuration copaTask Sentence completionMetric AccuracyComparison conditions not established | 52 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| SuperGLUE (rte) | Configuration rteTask Natural language inferenceMetric AccuracyComparison conditions not established | 67.15 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| Winogrande XL (xl) | Configuration xlTask Coreference resolutionMetric AccuracyComparison conditions not established | 52.41 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XCOPA (et) | Configuration etTask Sentence completionMetric AccuracyComparison conditions not established | 53 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XCOPA (ht) | Configuration htTask Sentence completionMetric AccuracyComparison conditions not established | 49 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XCOPA (id) | Configuration idTask Sentence completionMetric AccuracyComparison conditions not established | 57 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XCOPA (it) | Configuration itTask Sentence completionMetric AccuracyComparison conditions not established | 52 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XCOPA (qu) | Configuration quTask Sentence completionMetric AccuracyComparison conditions not established | 55 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XCOPA (sw) | Configuration swTask Sentence completionMetric AccuracyComparison conditions not established | 56 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XCOPA (ta) | Configuration taTask Sentence completionMetric AccuracyComparison conditions not established | 58 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XCOPA (th) | Configuration thTask Sentence completionMetric AccuracyComparison conditions not established | 58 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XNLI (ar) | Configuration arTask Natural language inferenceMetric AccuracyComparison conditions not established | 44.46 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XNLI (bg) | Configuration bgTask Natural language inferenceMetric AccuracyComparison conditions not established | 39.76 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XNLI (de) | Configuration deTask Natural language inferenceMetric AccuracyComparison conditions not established | 39.36 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XNLI (el) | Configuration elTask Natural language inferenceMetric AccuracyComparison conditions not established | 40.96 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XNLI (en) | Configuration enTask Natural language inferenceMetric AccuracyComparison conditions not established | 46.43 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XNLI (es) | Configuration esTask Natural language inferenceMetric AccuracyComparison conditions not established | 44.98 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XNLI (fr) | Configuration frTask Natural language inferenceMetric AccuracyComparison conditions not established | 45.54 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XNLI (hi) | Configuration hiTask Natural language inferenceMetric AccuracyComparison conditions not established | 41.81 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XNLI (ru) | Configuration ruTask Natural language inferenceMetric AccuracyComparison conditions not established | 39.64 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XNLI (sw) | Configuration swTask Natural language inferenceMetric AccuracyComparison conditions not established | 38.35 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XNLI (th) | Configuration thTask Natural language inferenceMetric AccuracyComparison conditions not established | 35.5 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XNLI (tr) | Configuration trTask Natural language inferenceMetric AccuracyComparison conditions not established | 37.31 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XNLI (ur) | Configuration urTask Natural language inferenceMetric AccuracyComparison conditions not established | 38.96 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XNLI (vi) | Configuration viTask Natural language inferenceMetric AccuracyComparison conditions not established | 44.74 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XNLI (zh) | Configuration zhTask Natural language inferenceMetric AccuracyComparison conditions not established | 44.66 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XWinograd (en) | Configuration enTask Coreference resolutionMetric AccuracyComparison conditions not established | 51.01 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XWinograd (fr) | Configuration frTask Coreference resolutionMetric AccuracyComparison conditions not established | 51.81 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XWinograd (jp) | Configuration jpTask Coreference resolutionMetric AccuracyComparison conditions not established | 52.03 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XWinograd (pt) | Configuration ptTask Coreference resolutionMetric AccuracyComparison conditions not established | 53.99 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XWinograd (ru) | Configuration ruTask Coreference resolutionMetric AccuracyComparison conditions not established | 53.97 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |
| XWinograd (zh) | Configuration zhTask Coreference resolutionMetric AccuracyComparison conditions not established | 54.76 | [bigscience](https://huggingface.co/bigscience/bloomz-560m) Publisher reported | Evaluated revision not stated | — |

## Memory Requirements

| Precision | Weights in memory |
| --- | --- |
| As published | 2.2 GB |
| 16-bit | 1.1 GB |
| 8-bit | 0.6 GB |
| 4-bit | 0.3 GB |

Weights only, from the published parameter count; the key-value cache and runtime add to this.

## Questions About bloomz-560m

### How much GPU memory does bloomz-560m need?

About 1.3 GB at 16-bit and 0.3 GB at 4-bit: the weights (559M parameters) plus a working margin. A long context needs more.

### What is the cheapest GPU to run bloomz-560m on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

### What license is bloomz-560m released under?

bigscience-bloom-rail-1.0, as its publisher declares it. Read the license text before commercial use.

## Similar Models

Model · Text generation

### [LibraMindMini](https://savrn.com/models/libramindmini)

[Alby13](https://savrn.com/model-publishers/alby13)

LibraMind is a 565M-parameter chat model pretrained from scratch by alby13 on a single consumer GPU (NVIDIA RTX 4090). It is a hybrid architecture that interleaves 18 Gated DeltaNet linear-recurrent layers with 6 Gated Attention layers (a repeating DDDA × 6 pattern), with Block Attention Residuals and logit soft-capping, trained with the Muon optimizer. It went through a full five-stage pipeline: pretraining, midtraining, supervised fine-tuning, preference tuning, and RL with verifiable rewards. - Research on small hybrid recurrent/attention language models - Studying low-resource, single-GPU training pipelines (pretraining through RL) and the Muon optimizer at small scale - Experiments…

Open weights apache-2.0 565M parameters pytorch

[View model](https://savrn.com/models/libramindmini)

Model · Text generation

### [Qwen3-0.6B-Base](https://savrn.com/models/qwen3-0-6b-base)

[Qwen](https://savrn.com/model-publishers/qwen)

Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Building upon extensive advancements in training data, model architecture, and optimization techniques, Qwen3 delivers the following key improvements over the previously released Qwen2.5: Qwen3-0.6B-Base has the following features: For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our blog, GitHub, and Documentation. The code of Qwen3 has been in the latest Hugging Face transformers and we advise you to use the latest version of transformers. With transformers<4.51.0, you will…

Open weights apache-2.0 596M parameters 32,768 tokens transformers

[View model](https://savrn.com/models/qwen3-0-6b-base)

Model · Text generation

### [Symbiotic-1B](https://savrn.com/models/symbiotic-1b)

[Convergent Intelligence](https://savrn.com/model-publishers/reaperdoesntknow)

Purpose: Lightweight, memory-augmented reasoning model for CPU and embedded inference SymbioticLM-1B is the compact version of the SymbioticAI architecture. It fuses Qwen’s rotary transformer design with a symbolic processing pipeline and a persistent episodic memory. Though smaller in parameter count, it retains the full cognitive engine: symbolic memory, dynamic thought evolution, and entropy-gated control. This model is ideal for symbolic reasoning in constrained environments — like research agents, lightweight assistants, and memory-efficient logical processing. - Procedural planning, math modeling, small-code generation - Less fluent in free-form language than larger variants…

Open weights afl-3.0 596M parameters 40,960 tokens transformers

[View model](https://savrn.com/models/symbiotic-1b)

Model · Text generation

### [gpt-oss-20b-distill-Qwen3-0.6B](https://savrn.com/models/gpt-oss-20b-distill-qwen3-0-6b)

[Maximilian Schulten](https://savrn.com/model-publishers/maxschulten)

This model is a fine-tuned version of Qwen/Qwen3-0.6B on the None dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 4 - evalbatchsize: 8 - gradientaccumulationsteps: 16 - totaltrainbatchsize: 64 - lrschedulertype: cosine - lrschedulerwarmupsteps: 100 - numepochs: 3 - Transformers 5.17.0 - Pytorch 2.11.0+cu128 - Datasets 5.0.1 - Tokenizers 0.23.2

Open weights apache-2.0 596M parameters 40,960 tokens transformers

[View model](https://savrn.com/models/gpt-oss-20b-distill-qwen3-0-6b)

Model · Text generation

### [S1-mini-MLX-4bit](https://savrn.com/models/s1-mini-mlx-4bit)

[David Larrea](https://savrn.com/model-publishers/davidalarrea)

Converted with mlx-lm 0.31.3. A six-case deterministic sanity check matched three upstream reference strings exactly. The remaining differences included a retained leading “So,” punctuation/ordinal variation, and omission of “tomorrow” in one correction case. This is a small functional check, not the upstream 7,519-case evaluation; assess the 4-bit build on your own transcripts. A 0.6B-parameter text normalizer for speech-to-text output. It takes a raw ASR transcript and rewrites it as clean written text: fillers removed, false starts and self-corrections resolved to the value the speaker landed on, punctuation and capitalization applied, and spoken numbers, dates, times, currency and email…

Open weights other 596M parameters 40,960 tokens mlx

[View model](https://savrn.com/models/s1-mini-mlx-4bit)

Model · Text generation

### [S1-mini-MLX-8bit](https://savrn.com/models/s1-mini-mlx-8bit)

[David Larrea](https://savrn.com/model-publishers/davidalarrea)

Converted with mlx-lm 0.31.3. A six-case deterministic sanity check matched four upstream reference strings exactly. The two differences were a retained leading “So,” and one comma variation. This is a small functional check, not the upstream 7,519-case evaluation. A 0.6B-parameter text normalizer for speech-to-text output. It takes a raw ASR transcript and rewrites it as clean written text: fillers removed, false starts and self-corrections resolved to the value the speaker landed on, punctuation and capitalization applied, and spoken numbers, dates, times, currency and email addresses rendered in written form. On a held-out set of 7,519 English cases it reaches 94.8% token accuracy, and…

Open weights other 596M parameters 40,960 tokens mlx

[View model](https://savrn.com/models/s1-mini-mlx-8bit)

## BigScience Workshop

[All models and datasets](https://savrn.com/model-publishers/bigscience)

## Versions

- [a2845d7e13dd](https://savrn.com/models/bloomz-560m/versions/a2845d7e13dd) · current 2026-10-07

## Explore More

- [All text generation models](https://savrn.com/models/tasks/text-generation)
- [All models under bigscience-bloom-rail-1.0](https://savrn.com/models/licenses/bigscience-bloom-rail-1-0)
- [Model comparisons](https://savrn.com/models/comparisons)
- [The model directory](https://savrn.com/models)
- [Open model prices by host](https://savrn.com/ai-index/pricing/open-models)

## Source

- Repository metadata, read 2026-09-26.
- [Hugging Face record](https://huggingface.co/bigscience/bloomz-560m)
- [How the hub is built](https://savrn.com/model-hub/methodology)