| Benchmark (Metric) | Qwen3.6 35B-A3B |
Gemma-4 26B-A4B-it |
Nemotron-3.5 Lightning |
Muse Glimmer-30B |
Maxwell V1.0-Flash |
|---|---|---|---|---|---|
| General | |||||
| Instruction following | |||||
| AdvancedIF (Score) | 72.50 | 76.10 | 65.60 | 73.80 | 71.30 |
| IFBench (Score) | 59.60 | 79.10 | 70.80 | 83.20 | 64.80 |
| IFEval (Score) | 93.80 | 94.00 | 91.50 | 96.80 | 93.00 |
| Instruction-following average (Average) | 75.30 | 83.07 | 75.97 | 84.60 | 76.37 |
| Multi-turn | |||||
| MultiChallenge (pass@1) | 62.59 | 65.76 | 61.07 | 70.45 | 72.20 |
| MT-Bench-101 (Score) | 94.20 | 93.90 | 94.50 | 88.20 | 93.30 |
| MT-RAG (Average) | 65.03 | 70.54 | 67.00 | 72.70 | 65.74 |
| LongMemEval (Score) | 79.52 | 85.89 | 83.44 | 94.48 | 86.18 |
| Multi-turn average (Average) | 75.34 | 79.02 | 76.50 | 81.46 | 79.36 |
| Long context | |||||
| AA-LCR (mean@5) | 71.70 | 65.70 | 60.30 | 79.60 | 75.60 |
| LongBench V2 (Score) | 58.80 | 53.30 | 46.30 | 38.60 | 59.40 |
| RULER (Score) | 97.08 | 94.74 | 96.75 | 94.84 | 95.81 |
| Long-context average (Average) | 75.86 | 71.25 | 67.78 | 71.01 | 76.94 |
| Multilingual | |||||
| MLogiQA (Score) | 77.8 | 77.8 | 71.3 | 80.0 | 81.5 |
| CEval (Score) | 88.3 | 83.7 | 76.2 | 85.4 | 91.0 |
| MMMLU (Score) | 89.1 | 90.9 | 87.8 | 90.7 | 92.0 |
| INCLUDE (Score) | 84.7 | 85.8 | 71.5 | 79.4 | 84.7 |
| Belebele (Score) | 93.8 | 94.8 | 91.8 | 94.3 | 92.8 |
| Global-PIQA (Score) | 91.7 | 93.9 | 90.9 | 92.4 | 92.3 |
| MGSM (Score) | 90.3 | 91.1 | 91.5 | 93.2 | 93.0 |
| Multilingual average (Average) | 88.0 | 88.3 | 83.0 | 87.9 | 89.6 |
| Role playing | |||||
| CoSER (pass@1) | 47.79 | 56.19 | 43.78 | 60.32 | 53.54 |
| RoleBench (pass@1) | 89.21 | 84.05 | 81.10 | 73.40 | 89.92 |
| Role-playing average (Average) | 68.50 | 70.12 | 62.44 | 66.86 | 71.73 |
| Reasoning | |||||
| Mathematics | |||||
| AIME 2025 (pass@8) | 93.33 | 93.33 | 96.67 | 100.00 | 100.00 |
| AIME 2025 (mean@8) | 90.00 | 85.83 | 87.92 | 92.08 | 96.25 |
| AIME 2026 (pass@8) | 96.67 | 96.67 | 96.67 | 96.67 | 96.67 |
| AIME 2026 (mean@8) | 91.25 | 87.50 | 87.08 | 94.17 | 95.00 |
| HMMT February 2025 (pass@8) | 100.00 | 100.00 | 96.67 | 96.67 | 100.00 |
| HMMT February 2025 (mean@8) | 89.58 | 82.50 | 85.83 | 91.25 | 96.25 |
| HMMT November 2025 (pass@8) | 93.33 | 96.67 | 93.33 | 93.33 | 100.00 |
| HMMT November 2025 (mean@8) | 88.75 | 80.83 | 86.67 | 87.92 | 90.83 |
| HMMT February 2026 (pass@8) | 93.94 | 87.88 | 90.91 | 93.94 | 100.00 |
| HMMT February 2026 (mean@8) | 83.33 | 71.21 | 75.76 | 83.33 | 91.29 |
| IMO-AnswerBench (pass@8) | 92.25 | 88.50 | 88.00 | 91.75 | 95.50 |
| IMO-AnswerBench (mean@8) | 81.25 | 70.00 | 73.25 | 78.62 | 86.03 |
| Mathematical average (pass@8) | 94.92 | 93.84 | 93.71 | 95.39 | 98.70 |
| Mathematical average (mean@8) | 87.36 | 79.65 | 82.75 | 87.90 | 92.61 |
| Science | |||||
| GPQA-Diamond (pass@1) | 86.00 | 82.30 | 75.44 | 83.50 | 86.52 |
| MMLU-Pro (Accuracy) | 85.20 | 82.60 | 81.94 | 83.16 | 85.46 |
| MMLU-Redux (Accuracy) | 93.30 | 93.10 | 91.91 | 90.72 | 93.59 |
| Scientific average (Average) | 88.17 | 86.00 | 83.10 | 85.79 | 88.52 |
| HLE (128K context) (Score) | 21.40 | 20.60 | 11.40 | 21.83 | 24.79 |
| Logic | |||||
| BBH (pass@1) | 90.87 | 86.50 | 84.03 | 89.33 | 91.02 |
| BBEH (pass@1) | 48.05 | 55.18 | 25.86 | 64.69 | 52.51 |
| KOR-Bench (pass@1) | 74.48 | 77.36 | 72.32 | 80.08 | 77.20 |
| Logical macro average (Macro average) | 71.13 | 73.01 | 60.74 | 78.03 | 73.58 |
| Code | |||||
| LiveCodeBench v6 (pass@1) | 85.32 | 85.96 | 78.30 | 76.80 | 85.66 |
| LiveCodeBench v6 (pass@8) | 92.32 | 92.89 | 87.90 | 85.70 | 93.74 |
| Agent | |||||
| Terminal | |||||
| Terminal-Bench (mean@5) | 47.42 | 38.43 | 23.10 | 48.50 | 58.65 |
| SWE | |||||
| SWE-Bench-Verified (Claude Code) (pass@1) | 63.40 | 39.20 | 42.60 | 47.60 | 65.40 |
| Tool use | |||||
| MCP-Mark (mean@8) | 36.70 | 14.50 | 14.47 | 27.56 | 37.30 |
| Tau3-Bench (pass@1) | 64.00 | 58.60 | 58.63 | 63.20 | 67.37 |
| Search | |||||
| BrowseComp (pass@1) | 47.00 | 29.20 | 24.20 | 55.60 | 76.30 / 82.20* |
| HLE w/Tools (pass@1) | 28.30 | 31.70 | 16.60 | 34.40 | 46.10 / 55.10* |
| WideSearch (max@4) | 70.40 | 54.50 | 60.70 | 70.70 | 74.40 / 75.80* |
| DeepSearchQA (pass@1) | 71.10 | 63.00 | 62.80 | 81.80 | 83.30 / 89.50* |
| Bamboogle (mean@8) | 90.90 | 91.30 | 90.60 | 86.00 | 92.10 |
| WebWalkerQA-300 (mean@8) | 62.80 | 54.50 | 55.40 | 56.00 | 70.50 |
| 2WikiMultihopQA-300 (mean@8) | 87.75 | 89.20 | 86.20 | 89.40 | 89.80 |
| MuSiQue-300 (mean@8) | 49.25 | 49.30 | 42.40 | 46.00 | 63.00 |
| HotpotQA-300 (mean@8) | 80.80 | 79.50 | 73.20 | 80.50 | 84.70 |
| PopQA-300 (mean@8) | 68.29 | 64.70 | 58.90 | 68.80 | 70.10 |