SAVRN Model Hub · Comparisons
gemma-4-26B-A4B-it vs Qwen3.8-27B
Gemma-4-26B-A4B-it has 25.8B parameters and Qwen3.8-27B has 27.8B parameters; both are released under Apache License 2.0; at 16-bit, gemma-4-26B-A4B-it needs about 61.9 GB (1x MI300X from $1.85 an hour) and Qwen3.8-27B about 66.7 GB (1x MI300X from $1.85 an hour).
| Field | gemma-4-26B-A4B-it google/gemma-4-26B-A4B-it | Qwen3.8-27B Qwen/Qwen3.8-27B |
|---|---|---|
| Publisher | Qwen | |
| Task | Image and text to text | Image and text to text |
| Modality | Image and text | Image and text |
| Parameters, as reported | 25.8B parameters | 27.8B parameters |
| Architecture | Gemma4ForConditionalGeneration | Qwen3_5ForConditionalGeneration |
| Library | transformers | transformers |
| Context length | 262,144 tokens | 262,144 tokens |
| Repository size | 51.6 GB | 55.6 GB |
| Artifact formats | safetensors | safetensors |
| License | apache-2.0 | apache-2.0 |
| Access | Open weights, no gate | Open weights, no gate |
| Memory at 16-bit (weights and margin) | 61.9 GB | 66.7 GB |
| Cheapest GPUs at 16-bit, per hour | 1x MI300X, $1.85 | 1x MI300X, $1.85 |
| Memory at 4-bit (weights and margin) | 15.5 GB | 16.7 GB |
| Cheapest GPUs at 4-bit, per hour | 1x MI300X, $1.85 | 1x MI300X, $1.85 |
| Revision viewed | 4d7ae4984b7d | 1d4bf0f2ff60 |
| Downloads reported by the hub | 11.5M | 6.9M |
| Last observed | 2026-10-13 | 2026-10-13 |
An evaluation row appears only where at least two of these models report the same benchmark with the same stated configuration, metric, unit and setup. Different evaluators stay named in each cell. Values are shown as reported: no unit conversion, no ranking.
Other Reported Results
These results are listed for each model on its own, because the conditions needed to compare them are not stated or do not match. Two results that leave a condition blank are not assumed to share it.
gemma-4-26B-A4B-it
| Benchmark | Conditions | Result | Reported by | Revision | Date |
|---|---|---|---|---|---|
| Idavidrein/gpqa | Task diamondMetric diamondComparison conditions not established | 82.3 | Model Card Reported by a third party |
Evaluated revision not stated | 2026-04-02 |
| MMMU/MMMU_Pro | Task mmmu_pro_visionMetric mmmu_pro_visionComparison conditions not established | 73.8 | Model Card Reported by a third party |
Evaluated revision not stated | 2026-05-12 |
| TIGER-Lab/MMLU-Pro | Task mmlu_proMetric mmlu_proComparison conditions not established | 82.6 | Model Card Reported by a third party |
Evaluated revision not stated | 2026-04-02 |
| cais/hle | Task hleMetric hleSetup With searchComparison conditions not established | 17.2 | Model Card Reported by a third party |
Evaluated revision not stated | 2026-04-02 |
| llamaindex/ExtractBench | Task longMetric longSetup Pipeline name: gemma4_26b_vllm_extract_oneshot_structured_output_file; self-hosted on vLLM, one-shot JSON-Schema structured outputComparison conditions not established | 28.88 | ExtractBench Reported by a third party |
Evaluated revision not stated | 2026-08-26 |
| llamaindex/ExtractBench | Task meanMetric meanSetup Pipeline name: gemma4_26b_vllm_extract_oneshot_structured_output_file; self-hosted on vLLM, one-shot JSON-Schema structured outputComparison conditions not established | 70.27 | ExtractBench Reported by a third party |
Evaluated revision not stated | 2026-08-26 |
| llamaindex/ExtractBench | Task mediumMetric mediumSetup Pipeline name: gemma4_26b_vllm_extract_oneshot_structured_output_file; self-hosted on vLLM, one-shot JSON-Schema structured outputComparison conditions not established | 59.61 | ExtractBench Reported by a third party |
Evaluated revision not stated | 2026-08-26 |
| llamaindex/ExtractBench | Task shortMetric shortSetup Pipeline name: gemma4_26b_vllm_extract_oneshot_structured_output_file; self-hosted on vLLM, one-shot JSON-Schema structured outputComparison conditions not established | 77.7 | ExtractBench Reported by a third party |
Evaluated revision not stated | 2026-08-26 |
| llamaindex/ParseBench | Task chartMetric chartSetup Pipeline name: gemma4_26b_vllm_with_layoutComparison conditions not established | 14.2 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-04-14 |
| llamaindex/ParseBench | Task layoutMetric layoutSetup Pipeline name: gemma4_26b_vllm_with_layoutComparison conditions not established | 59.2 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-04-14 |
| llamaindex/ParseBench | Task meanMetric meanSetup Pipeline name: gemma4_26b_vllm_with_layoutComparison conditions not established | 58.5 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-04-14 |
| llamaindex/ParseBench | Task tableMetric tableSetup Pipeline name: gemma4_26b_vllm_with_layoutComparison conditions not established | 70 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-04-14 |
| llamaindex/ParseBench | Task text_contentMetric text_contentSetup Pipeline name: gemma4_26b_vllm_with_layoutComparison conditions not established | 83.8 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-04-14 |
| llamaindex/ParseBench | Task text_formattingMetric text_formattingSetup Pipeline name: gemma4_26b_vllm_with_layoutComparison conditions not established | 65.1 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-04-14 |
Qwen3.8-27B
| Benchmark | Conditions | Result | Reported by | Revision | Date |
|---|---|---|---|---|---|
| Idavidrein/gpqa | Task diamondMetric diamondComparison conditions not established | 89.2 | Qwen3.8-27B model card Reported by a third party |
Evaluated revision not stated | 2026-08-14 |
| ScaleAI/SWE-bench_Pro | Task SWE_Bench_ProMetric SWE_Bench_ProSetup Evaluated with the Claude Code harness, temp=1.0, top_p=0.95, 256K context; baseline models re-evaluated on the same refined task set.Comparison conditions not established | 61.7 | Qwen3.8-27B model card Reported by a third party |
Evaluated revision not stated | 2026-08-14 |
| agents-last-exam/agents-last-exam | Task ale-cli-pass-rateMetric ale-cli-pass-rateSetup harness=claude_code; harness_variant=reasoning-max; attempted=104/105; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not…Comparison conditions not established | 19.0476 | ALE official leaderboard Reported by a third party |
Evaluated revision not stated | 2026-10-01 |
| agents-last-exam/agents-last-exam | Task ale-cli-scoreMetric ale-cli-scoreSetup harness=claude_code; harness_variant=reasoning-max; attempted=104/105; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not…Comparison conditions not established | 43.5407 | ALE official leaderboard Reported by a third party |
Evaluated revision not stated | 2026-10-01 |
| agents-last-exam/agents-last-exam | Task full-spectrum-pass-rateMetric full-spectrum-pass-rateSetup harness=claude_code; harness_variant=reasoning-max; attempted=55/55; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not an…Comparison conditions not established | 21.8182 | ALE official leaderboard Reported by a third party |
Evaluated revision not stated | 2026-10-01 |
| agents-last-exam/agents-last-exam | Task full-spectrum-scoreMetric full-spectrum-scoreSetup harness=claude_code; harness_variant=reasoning-max; attempted=55/55; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not an…Comparison conditions not established | 42.5452 | ALE official leaderboard Reported by a third party |
Evaluated revision not stated | 2026-10-01 |
| agents-last-exam/agents-last-exam | Task last-exam-pass-rateMetric last-exam-pass-rateSetup harness=claude_code; harness_variant=reasoning-max; attempted=35/38; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not an…Comparison conditions not established | 2.63158 | ALE official leaderboard Reported by a third party |
Evaluated revision not stated | 2026-10-01 |
| agents-last-exam/agents-last-exam | Task last-exam-scoreMetric last-exam-scoreSetup harness=claude_code; harness_variant=reasoning-max; attempted=35/38; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not an…Comparison conditions not established | 14.7264 | ALE official leaderboard Reported by a third party |
Evaluated revision not stated | 2026-10-01 |
| agents-last-exam/agents-last-exam | Task near-term-pass-rateMetric near-term-pass-rateSetup harness=claude_code; harness_variant=reasoning-max; attempted=66/67; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not an…Comparison conditions not established | 31.3433 | ALE official leaderboard Reported by a third party |
Evaluated revision not stated | 2026-10-01 |
| agents-last-exam/agents-last-exam | Task near-term-scoreMetric near-term-scoreSetup harness=claude_code; harness_variant=reasoning-max; attempted=66/67; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not an…Comparison conditions not established | 60.0256 | ALE official leaderboard Reported by a third party |
Evaluated revision not stated | 2026-10-01 |
| agents-last-exam/agents-last-exam | Task overall-pass-rateMetric overall-pass-rateSetup harness=claude_code; harness_variant=reasoning-max; attempted=148/152; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not…Comparison conditions not established | 20.3947 | ALE official leaderboard Reported by a third party |
Evaluated revision not stated | 2026-10-01 |
| agents-last-exam/agents-last-exam | Task overall-scoreMetric overall-scoreSetup harness=claude_code; harness_variant=reasoning-max; attempted=148/152; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not…Comparison conditions not established | 42.8848 | ALE official leaderboard Reported by a third party |
Evaluated revision not stated | 2026-10-01 |
| cais/hle | Task hleMetric hleSetup Judged by GPT-4o.Comparison conditions not established | 30.8 | Qwen3.8-27B model card Reported by a third party |
Evaluated revision not stated | 2026-08-14 |
| claw-eval/Claw-Eval | Task multimodalMetric multimodalSetup Reported as ClawEval-MM. Pass@3, the benchmark's own Pass³ methodology; card also reports a secondary 56.9 'Average' metric, not included here.Comparison conditions not established | 57.4 | Qwen3.8-27B model card Reported by a third party |
Evaluated revision not stated | 2026-08-14 |
| datacurve/deep-swe | Task deep_sweMetric deep_sweComparison conditions not established | 42.2 | Qwen3.8-27B model card Reported by a third party |
Evaluated revision not stated | 2026-08-14 |
| harborframework/terminal-bench-2.1 | Task terminalbench_2_1Metric terminalbench_2_1Setup Row labeled "(Terminus)" as the harness; no further hyperparameter footnote given for this row.Comparison conditions not established | 73 | Model Card Reported by a third party |
Evaluated revision not stated | 2026-08-14 |
| internlm/WildClawBench | Task avg_timeMetric avg_timeComparison conditions not established | 516 | WildClawBench Reported by a third party |
Evaluated revision not stated | 2026-08-16 |
| internlm/WildClawBench | Task overallMetric overallComparison conditions not established | 48.0152 | WildClawBench Reported by a third party |
Evaluated revision not stated | 2026-08-16 |
| llamaindex/ExtractBench | Task longMetric longSetup Pipeline name: qwen3_8_27b_fp8_vllm_extract_oneshot_structured_output_file; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established | 38.45 | ExtractBench Reported by a third party |
Evaluated revision not stated | 2026-08-24 |
| llamaindex/ExtractBench | Task meanMetric meanSetup Pipeline name: qwen3_8_27b_fp8_vllm_extract_oneshot_structured_output_file; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established | 89.75 | ExtractBench Reported by a third party |
Evaluated revision not stated | 2026-08-24 |
| llamaindex/ExtractBench | Task mediumMetric mediumSetup Pipeline name: qwen3_8_27b_fp8_vllm_extract_oneshot_structured_output_file; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established | 87.54 | ExtractBench Reported by a third party |
Evaluated revision not stated | 2026-08-24 |
| llamaindex/ExtractBench | Task shortMetric shortSetup Pipeline name: qwen3_8_27b_fp8_vllm_extract_oneshot_structured_output_file; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established | 94.68 | ExtractBench Reported by a third party |
Evaluated revision not stated | 2026-08-24 |
| llamaindex/ParseBench | Task chartMetric chartSetup Pipeline name: qwen3_8_27b_thinking_parse_with_layout; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established | 69.17 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-08-28 |
| llamaindex/ParseBench | Task layoutMetric layoutSetup Pipeline name: qwen3_8_27b_thinking_parse_with_layout; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established | 69.9 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-08-28 |
| llamaindex/ParseBench | Task meanMetric meanSetup Pipeline name: qwen3_8_27b_thinking_parse_with_layout; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established | 70.79 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-08-28 |
| llamaindex/ParseBench | Task tableMetric tableSetup Pipeline name: qwen3_8_27b_thinking_parse_with_layout; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established | 66.82 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-08-28 |
| llamaindex/ParseBench | Task text_contentMetric text_contentSetup Pipeline name: qwen3_8_27b_thinking_parse_with_layout; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established | 88.28 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-08-28 |
| llamaindex/ParseBench | Task text_formattingMetric text_formattingSetup Pipeline name: qwen3_8_27b_thinking_parse_with_layout; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established | 59.77 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-08-28 |
SAVRN's Notes on gemma-4-26B-A4B-it
Gemma 4 26B A4B is the most downloaded Gemma 4 model in our catalog, at about 9.8 million a month. It is a mixture-of-experts design with 25.8B total parameters and roughly 4B active per token, so it answers with the speed of a small model and much of the knowledge of a large one. It reads text and images and keeps a 262,144-token context.
At 16-bit it needs about 62 GB, one large-memory accelerator; at 4-bit about 16 GB, which fits a workstation card. Apache 2.0 allows commercial use. For a team choosing one open multimodal model to self-host this year, this is near the top of our list.
SAVRN's Notes on Qwen3.8-27B
27.8B parameters reading image and text over a 262,144-token context: 66.7 GB at 16-bit fits one 192 GB MI300X at $1.85 an hour, 33.3 GB at 8-bit, 16.7 GB at 4-bit — a single card covers every precision, which is the whole story for a model this capable at this footprint. The 55.6 GB of weights ship as 18 safetensors files. First published 2026-08-05.
The variant forest is dense: FP8, NVFP4, GGUF, AWQ-INT4 and MLX builds at 4, 5, 6 and 8 bit all derive from it on the page. Hosted, the Index lists Cerebras at $0.99 in and $1.49 out per million tokens, DeepInfra $0.20 and $2.50, Novita $0.42 and $3.00, OVHcloud $0.47 and $3.19 — run the tokens-per-hour ledger before renting. Reported GPQA Diamond is 89.2, and the card promises a hosted version with 1M context coming.
Questions
Which is larger, gemma-4-26B-A4B-it or Qwen3.8-27B?
Qwen3.8-27B (27.8B parameters) is larger than gemma-4-26B-A4B-it (25.8B parameters), by the parameter counts their publishers report.
Which is cheaper to run, gemma-4-26B-A4B-it or Qwen3.8-27B?
At 4-bit, gemma-4-26B-A4B-it fits on 1x MI300X from $1.85 an hour and Qwen3.8-27B on 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use gemma-4-26B-A4B-it commercially?
Yes. gemma-4-26B-A4B-it is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
Can I use Qwen3.8-27B commercially?
Yes. Qwen3.8-27B is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.