| Idavidrein/gpqa |
Task diamondMetric diamondComparison conditions not established |
89.2 |
Qwen3.8-27B model card Reported by a third party |
Evaluated revision not stated |
2026-08-14 |
| ScaleAI/SWE-bench_Pro |
Task SWE_Bench_ProMetric SWE_Bench_ProSetup Evaluated with the Claude Code harness, temp=1.0, top_p=0.95, 256K context; baseline models re-evaluated on the same refined task set.Comparison conditions not established |
61.7 |
Qwen3.8-27B model card Reported by a third party |
Evaluated revision not stated |
2026-08-14 |
| agents-last-exam/agents-last-exam |
Task ale-cli-pass-rateMetric ale-cli-pass-rateSetup harness=claude_code; harness_variant=reasoning-max; attempted=104/105; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not…Comparison conditions not established |
19.0476 |
ALE official leaderboard Reported by a third party |
Evaluated revision not stated |
2026-10-01 |
| agents-last-exam/agents-last-exam |
Task ale-cli-scoreMetric ale-cli-scoreSetup harness=claude_code; harness_variant=reasoning-max; attempted=104/105; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not…Comparison conditions not established |
43.5407 |
ALE official leaderboard Reported by a third party |
Evaluated revision not stated |
2026-10-01 |
| agents-last-exam/agents-last-exam |
Task full-spectrum-pass-rateMetric full-spectrum-pass-rateSetup harness=claude_code; harness_variant=reasoning-max; attempted=55/55; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not an…Comparison conditions not established |
21.8182 |
ALE official leaderboard Reported by a third party |
Evaluated revision not stated |
2026-10-01 |
| agents-last-exam/agents-last-exam |
Task full-spectrum-scoreMetric full-spectrum-scoreSetup harness=claude_code; harness_variant=reasoning-max; attempted=55/55; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not an…Comparison conditions not established |
42.5452 |
ALE official leaderboard Reported by a third party |
Evaluated revision not stated |
2026-10-01 |
| agents-last-exam/agents-last-exam |
Task last-exam-pass-rateMetric last-exam-pass-rateSetup harness=claude_code; harness_variant=reasoning-max; attempted=35/38; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not an…Comparison conditions not established |
2.63158 |
ALE official leaderboard Reported by a third party |
Evaluated revision not stated |
2026-10-01 |
| agents-last-exam/agents-last-exam |
Task last-exam-scoreMetric last-exam-scoreSetup harness=claude_code; harness_variant=reasoning-max; attempted=35/38; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not an…Comparison conditions not established |
14.7264 |
ALE official leaderboard Reported by a third party |
Evaluated revision not stated |
2026-10-01 |
| agents-last-exam/agents-last-exam |
Task near-term-pass-rateMetric near-term-pass-rateSetup harness=claude_code; harness_variant=reasoning-max; attempted=66/67; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not an…Comparison conditions not established |
31.3433 |
ALE official leaderboard Reported by a third party |
Evaluated revision not stated |
2026-10-01 |
| agents-last-exam/agents-last-exam |
Task near-term-scoreMetric near-term-scoreSetup harness=claude_code; harness_variant=reasoning-max; attempted=66/67; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not an…Comparison conditions not established |
60.0256 |
ALE official leaderboard Reported by a third party |
Evaluated revision not stated |
2026-10-01 |
| agents-last-exam/agents-last-exam |
Task overall-pass-rateMetric overall-pass-rateSetup harness=claude_code; harness_variant=reasoning-max; attempted=148/152; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not…Comparison conditions not established |
20.3947 |
ALE official leaderboard Reported by a third party |
Evaluated revision not stated |
2026-10-01 |
| agents-last-exam/agents-last-exam |
Task overall-scoreMetric overall-scoreSetup harness=claude_code; harness_variant=reasoning-max; attempted=148/152; official leaderboard snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This is an official leaderboard export, not…Comparison conditions not established |
42.8848 |
ALE official leaderboard Reported by a third party |
Evaluated revision not stated |
2026-10-01 |
| cais/hle |
Task hleMetric hleSetup Judged by GPT-4o.Comparison conditions not established |
30.8 |
Qwen3.8-27B model card Reported by a third party |
Evaluated revision not stated |
2026-08-14 |
| claw-eval/Claw-Eval |
Task multimodalMetric multimodalSetup Reported as ClawEval-MM. Pass@3, the benchmark's own Pass³ methodology; card also reports a secondary 56.9 'Average' metric, not included here.Comparison conditions not established |
57.4 |
Qwen3.8-27B model card Reported by a third party |
Evaluated revision not stated |
2026-08-14 |
| datacurve/deep-swe |
Task deep_sweMetric deep_sweComparison conditions not established |
42.2 |
Qwen3.8-27B model card Reported by a third party |
Evaluated revision not stated |
2026-08-14 |
| harborframework/terminal-bench-2.1 |
Task terminalbench_2_1Metric terminalbench_2_1Setup Row labeled "(Terminus)" as the harness; no further hyperparameter footnote given for this row.Comparison conditions not established |
73 |
Model Card Reported by a third party |
Evaluated revision not stated |
2026-08-14 |
| internlm/WildClawBench |
Task avg_timeMetric avg_timeComparison conditions not established |
516 |
WildClawBench Reported by a third party |
Evaluated revision not stated |
2026-08-16 |
| internlm/WildClawBench |
Task overallMetric overallComparison conditions not established |
48.0152 |
WildClawBench Reported by a third party |
Evaluated revision not stated |
2026-08-16 |
| llamaindex/ExtractBench |
Task longMetric longSetup Pipeline name: qwen3_8_27b_fp8_vllm_extract_oneshot_structured_output_file; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established |
38.45 |
ExtractBench Reported by a third party |
Evaluated revision not stated |
2026-08-24 |
| llamaindex/ExtractBench |
Task meanMetric meanSetup Pipeline name: qwen3_8_27b_fp8_vllm_extract_oneshot_structured_output_file; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established |
89.75 |
ExtractBench Reported by a third party |
Evaluated revision not stated |
2026-08-24 |
| llamaindex/ExtractBench |
Task mediumMetric mediumSetup Pipeline name: qwen3_8_27b_fp8_vllm_extract_oneshot_structured_output_file; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established |
87.54 |
ExtractBench Reported by a third party |
Evaluated revision not stated |
2026-08-24 |
| llamaindex/ExtractBench |
Task shortMetric shortSetup Pipeline name: qwen3_8_27b_fp8_vllm_extract_oneshot_structured_output_file; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established |
94.68 |
ExtractBench Reported by a third party |
Evaluated revision not stated |
2026-08-24 |
| llamaindex/ParseBench |
Task chartMetric chartSetup Pipeline name: qwen3_8_27b_thinking_parse_with_layout; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established |
69.17 |
ParseBench Reported by a third party |
Evaluated revision not stated |
2026-08-28 |
| llamaindex/ParseBench |
Task layoutMetric layoutSetup Pipeline name: qwen3_8_27b_thinking_parse_with_layout; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established |
69.9 |
ParseBench Reported by a third party |
Evaluated revision not stated |
2026-08-28 |
| llamaindex/ParseBench |
Task meanMetric meanSetup Pipeline name: qwen3_8_27b_thinking_parse_with_layout; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established |
70.79 |
ParseBench Reported by a third party |
Evaluated revision not stated |
2026-08-28 |
| llamaindex/ParseBench |
Task tableMetric tableSetup Pipeline name: qwen3_8_27b_thinking_parse_with_layout; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established |
66.82 |
ParseBench Reported by a third party |
Evaluated revision not stated |
2026-08-28 |
| llamaindex/ParseBench |
Task text_contentMetric text_contentSetup Pipeline name: qwen3_8_27b_thinking_parse_with_layout; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established |
88.28 |
ParseBench Reported by a third party |
Evaluated revision not stated |
2026-08-28 |
| llamaindex/ParseBench |
Task text_formattingMetric text_formattingSetup Pipeline name: qwen3_8_27b_thinking_parse_with_layout; served checkpoint: Qwen/Qwen3.8-27B-FP8Comparison conditions not established |
59.77 |
ParseBench Reported by a third party |
Evaluated revision not stated |
2026-08-28 |