SAVRN Model Hub · Comparisons
PowerMoE-3b vs Qwen3-4B
PowerMoE-3b has 3.4B parameters and Qwen3-4B has 4B parameters; both are released under Apache License 2.0; at 16-bit, PowerMoE-3b needs about 8.1 GB (1x MI300X from $1.85 an hour) and Qwen3-4B about 9.7 GB (1x MI300X from $1.85 an hour).
| Field | PowerMoE-3b ibm-research/PowerMoE-3b | Qwen3-4B Qwen/Qwen3-4B |
|---|---|---|
| Publisher | IBM Research | Qwen |
| Task | Text generation | Text generation |
| Modality | Text | Text |
| Parameters, as reported | 3.4B parameters | 4B parameters |
| Architecture | GraniteMoeForCausalLM | Qwen3ForCausalLM |
| Library | transformers | transformers |
| Context length | 4,096 tokens | 40,960 tokens |
| Repository size | 13.5 GB | 8.1 GB |
| Artifact formats | safetensors | safetensors |
| License | apache-2.0 | apache-2.0 |
| Access | Open weights, no gate | Open weights, no gate |
| Memory at 16-bit (weights and margin) | 8.1 GB | 9.7 GB |
| Cheapest GPUs at 16-bit, per hour | 1x MI300X, $1.85 | 1x MI300X, $1.85 |
| Memory at 4-bit (weights and margin) | 2 GB | 2.4 GB |
| Cheapest GPUs at 4-bit, per hour | 1x MI300X, $1.85 | 1x MI300X, $1.85 |
| Revision viewed | 13fcb5a98001 | 1cfa9a720891 |
| Downloads reported by the hub | 1.3M | 7.2M |
| Last observed | 2026-09-18 | 2026-09-18 |
An evaluation row appears only where at least two of these models report the same benchmark with the same stated configuration, metric, unit and setup. Different evaluators stay named in each cell. Values are shown as reported: no unit conversion, no ranking.
Other Reported Results
These results are listed for each model on its own, because the conditions needed to compare them are not stated or do not match. Two results that leave a condition blank are not assumed to share it.
PowerMoE-3b
| Benchmark | Conditions | Result | Reported by | Revision | Date |
|---|---|---|---|---|---|
| ARC | Task text-generationMetric accuracy-normComparison conditions not established | 58.1 | ibm-research Publisher reported |
Evaluated revision not stated | — |
| BoolQ | Task text-generationMetric accuracyComparison conditions not established | 65 | ibm-research Publisher reported |
Evaluated revision not stated | — |
| GSM8k (5 shot) | Task text-generationMetric accuracyComparison conditions not established | 25.9 | ibm-research Publisher reported |
Evaluated revision not stated | — |
| Hellaswag | Task text-generationMetric accuracy-normComparison conditions not established | 71.5 | ibm-research Publisher reported |
Evaluated revision not stated | — |
| MBPP | Task text-generationMetric pass@1Comparison conditions not established | 32.4 | ibm-research Publisher reported |
Evaluated revision not stated | — |
| MMLU (5 shot) | Task text-generationMetric accuracyComparison conditions not established | 42.8 | ibm-research Publisher reported |
Evaluated revision not stated | — |
| OpenBookQA | Task text-generationMetric accuracy-normComparison conditions not established | 41 | ibm-research Publisher reported |
Evaluated revision not stated | — |
| PIQA | Task text-generationMetric accuracy-normComparison conditions not established | 79.1 | ibm-research Publisher reported |
Evaluated revision not stated | — |
| Winogrande | Task text-generationMetric accuracy-normComparison conditions not established | 65 | ibm-research Publisher reported |
Evaluated revision not stated | — |
| humaneval | Task text-generationMetric pass@1Comparison conditions not established | 20.1 | ibm-research Publisher reported |
Evaluated revision not stated | — |
| math (4 shot) | Task text-generationMetric accuracyComparison conditions not established | 14.8 | ibm-research Publisher reported |
Evaluated revision not stated | — |
SAVRN's Notes on PowerMoE-3b
Forty experts, eight awake for each token: that is how IBM Research's PowerMoE-3b holds 3.4B parameters while activating 800M per token. Memory does not care which experts are asleep, though. All 40 must be resident, so 16-bit needs 8.1 GB, 8-bit 4.0 GB and 4-bit 2.0 GB. Each lands on one MI300X with 192 GB at $1.85 an hour on demand and leaves the card mostly empty, so share it. The download is 13.5 GB, twice the 16-bit footprint.
Apache 2.0 makes commercial use, modification and redistribution straightforward; keep the notices and record your changes. Two checks. The context window is 4,096 tokens, so long documents will not fit in one pass; this model is for short prompts, code and math. And the training mix includes proprietary datasets, with every published number reported by the publisher itself, so evaluate on your own workload first.
SAVRN's Notes on Qwen3-4B
Memory is not the constraint with Qwen3-4B. The 16-bit weights are 8.0 GB and need 9.7 GB to serve, 8-bit needs 4.8 GB and 4-bit 2.4 GB, so on the cheapest Index setup, one MI300X with 192 GB at $1.85 an hour, the card sits mostly empty and the decision is how many copies to run. It is text only, switches between a thinking mode and a plain dialogue mode, and carries a 40,960-token context.
Apache 2.0 lets you fine-tune it, ship it in a product and charge for it, provided the license and notices travel with it and you state what you changed. Before you commit, look at the lineage and the price sheet: it is derived from Qwen3-4B-Base, so decide whether you want this release or the base for post-training, and the page lists no Index host prices, so there is nothing per token to compare with.
Questions
Which is larger, PowerMoE-3b or Qwen3-4B?
Qwen3-4B (4B parameters) is larger than PowerMoE-3b (3.4B parameters), by the parameter counts their publishers report.
Which is cheaper to run, PowerMoE-3b or Qwen3-4B?
At 4-bit, PowerMoE-3b fits on 1x MI300X from $1.85 an hour and Qwen3-4B on 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use PowerMoE-3b commercially?
Yes. PowerMoE-3b is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
Can I use Qwen3-4B commercially?
Yes. Qwen3-4B is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.