Rank,Overall Acc,Model,Model Link,Total Cost ($),Latency Mean (s),Latency Standard Deviation (s),Latency 95th Percentile (s),Non-Live AST Acc,Non-Live Simple AST,Non-Live Multiple AST,Non-Live Parallel AST,Non-Live Parallel Multiple AST,Live Acc,Live Simple AST,Live Multiple AST,Live Parallel AST,Live Parallel Multiple AST,Multi Turn Acc,Multi Turn Base,Multi Turn Miss Func,Multi Turn Miss Param,Multi Turn Long Context,Web Search Acc,Web Search Base,Web Search No Snippet,Memory Acc,Memory KV,Memory Vector,Memory Recursive Summarization,Relevance Detection,Irrelevance Detection,Format Sensitivity Max Delta,Format Sensitivity Standard Deviation,Organization,License
1,77.47%,Claude-Opus-4-5-20251101 (FC),https://www.anthropic.com/news/claude-4,86.55,4.38,3.13,7.56,88.58%,76.83%,95.50%,93.50%,88.50%,79.79%,86.43%,78.16%,87.50%,75.00%,68.38%,81.00%,64.00%,58.00%,70.50%,84.50%,84.00%,85.00%,73.76%,70.97%,72.90%,77.42%,62.50%,84.72%,N/A,N/A,Anthropic,Proprietary
2,73.24%,Claude-Sonnet-4-5-20250929 (FC),https://www.anthropic.com/news/claude-sonnet-4-5,43.73,4.31,4.43,7.27,88.65%,72.58%,95.50%,94.50%,92.00%,81.13%,89.53%,78.92%,87.50%,83.33%,61.37%,69.00%,65.00%,52.50%,59.00%,81.00%,82.00%,80.00%,64.95%,54.19%,57.42%,83.23%,68.75%,86.61%,N/A,N/A,Anthropic,Proprietary
3,72.51%,Gemini-3-Pro-Preview (Prompt),https://deepmind.google/technologies/gemini/pro/,298.47,12.08,21.3,32.73,90.65%,79.58%,96.00%,95.00%,92.00%,83.12%,87.60%,81.77%,93.75%,87.50%,60.75%,64.50%,60.00%,54.50%,64.00%,80.00%,78.00%,82.00%,61.72%,59.35%,62.58%,63.23%,68.75%,85.59%,8.5,1.7,Google,Proprietary
4,72.38%,GLM-4.6 (FC thinking),https://huggingface.co/zai-org/GLM-4.6,4.64,4.34,7.22,13.5,87.56%,74.25%,95.00%,91.50%,89.50%,80.90%,89.53%,78.92%,81.25%,75.00%,68.00%,74.50%,68.00%,63.00%,66.50%,77.50%,79.00%,76.00%,55.70%,43.87%,56.13%,67.10%,75.00%,84.96%,N/A,N/A,Zhipu AI,MIT
5,69.57%,Grok-4-1-fast-reasoning (FC),https://docs.x.ai/docs/models,17.26,6.74,12.78,17.57,88.27%,77.58%,93.00%,92.50%,90.00%,78.46%,84.11%,77.30%,75.00%,70.83%,58.87%,70.50%,59.50%,43.00%,62.50%,82.50%,82.00%,83.00%,53.98%,41.29%,57.42%,63.23%,81.25%,79.43%,N/A,N/A,xAI,Proprietary
6,68.70%,Claude-Haiku-4-5-20251001 (FC),https://www.anthropic.com/news/claude-haiku-4-5,14.23,1.68,3.92,3.15,86.50%,71.00%,94.00%,92.50%,88.50%,78.68%,83.72%,77.59%,75.00%,75.00%,53.62%,63.50%,42.50%,52.50%,56.00%,83.50%,86.00%,81.00%,54.41%,51.61%,55.48%,56.13%,62.50%,85.11%,N/A,N/A,Anthropic,Proprietary
7,68.14%,Gemini-3-Pro-Preview (FC),https://deepmind.google/technologies/gemini/pro/,224.69,15.87,41.41,58.48,85.75%,75.50%,94.00%,91.00%,82.50%,81.72%,87.60%,80.44%,75.00%,79.17%,63.12%,69.00%,63.00%,56.50%,64.00%,68.50%,63.00%,74.00%,54.84%,50.32%,63.23%,50.97%,75.00%,77.85%,N/A,N/A,Google,Proprietary
8,63.05%,o3-2025-04-16 (Prompt),https://openai.com/index/introducing-o3-and-o4-mini/,234.64,4.83,7.01,11.7,81.94%,74.25%,89.00%,86.50%,78.00%,73.21%,83.33%,70.75%,75.00%,70.83%,62.25%,68.00%,63.50%,54.50%,63.00%,50.50%,51.00%,50.00%,51.83%,33.55%,50.32%,71.61%,93.75%,83.98%,8.5,2.75,OpenAI,Proprietary
9,62.97%,Grok-4-0709 (Prompt),https://docs.x.ai/docs/models,348.19,30.38,36.19,101.54,82.75%,67.00%,93.50%,89.00%,81.50%,72.54%,81.78%,70.18%,81.25%,70.83%,47.00%,55.50%,46.00%,36.00%,50.50%,74.00%,74.00%,74.00%,50.54%,43.87%,59.35%,48.39%,81.25%,84.30%,13.0,2.88,xAI,Proprietary
10,61.38%,Grok-4-0709 (FC),https://docs.x.ai/docs/models,355.17,15.49,26.22,44.28,85.38%,73.50%,92.50%,88.50%,87.00%,75.57%,82.17%,73.88%,75.00%,79.17%,33.88%,44.00%,19.00%,28.50%,44.00%,82.00%,80.00%,84.00%,55.91%,57.42%,58.71%,51.61%,87.50%,75.40%,N/A,N/A,xAI,Proprietary
11,59.06%,Moonshotai-Kimi-K2-Instruct (FC),https://huggingface.co/moonshotai/Kimi-K2-Instruct,6.19,6.4,9.38,13.78,81.60%,69.42%,92.00%,82.00%,83.00%,78.68%,81.78%,78.06%,87.50%,66.67%,50.63%,62.00%,41.00%,44.50%,55.00%,66.50%,72.00%,61.00%,29.03%,21.94%,20.00%,45.16%,75.00%,87.34%,N/A,N/A,MoonshotAI,modified-mit
12,58.29%,Grok-4-1-fast-non-reasoning (FC),https://docs.x.ai/docs/models,16.27,2.29,7.31,5.34,88.13%,76.00%,93.00%,93.00%,90.50%,77.94%,82.95%,76.92%,75.00%,70.83%,46.75%,58.00%,39.50%,37.50%,52.00%,75.00%,74.00%,76.00%,26.24%,20.65%,20.00%,38.06%,81.25%,74.09%,N/A,N/A,xAI,Proprietary
13,57.06%,Command A Reasoning (FC),https://cohere.com/blog/command-a-reasoning,3.04,3.44,4.91,8.39,86.27%,73.58%,93.50%,89.50%,88.50%,78.61%,80.23%,78.35%,75.00%,75.00%,50.12%,61.50%,41.00%,49.50%,48.50%,55.50%,65.00%,46.00%,28.82%,16.13%,23.87%,46.45%,68.75%,86.75%,N/A,N/A,Cohere,CC-BY-NC 4.0 License (w/ Acceptable Use Addendum)
14,56.73%,DeepSeek-V3.2-Exp (Prompt + Thinking),https://api-docs.deepseek.com/news/news250528,57.75,37.89,49.56,102.09,85.52%,74.08%,92.00%,89.50%,86.50%,76.02%,82.56%,74.74%,87.50%,54.17%,44.88%,55.00%,49.00%,27.00%,48.50%,58.00%,64.00%,52.00%,44.09%,46.45%,46.45%,39.35%,93.75%,67.00%,10.0,2.77,DeepSeek,MIT
15,56.24%,Gemini-2.5-Flash (FC),https://deepmind.google/technologies/gemini/flash/,26.36,2.99,9.22,5.62,84.96%,74.33%,92.00%,94.00%,79.50%,74.39%,85.27%,71.70%,81.25%,70.83%,36.25%,41.50%,36.00%,32.00%,35.50%,59.00%,59.00%,59.00%,41.29%,19.35%,50.32%,54.19%,75.00%,93.67%,N/A,N/A,Google,Proprietary
16,55.87%,GPT-5.2-2025-12-11 (FC),https://openai.com/zh-Hans-CN/index/introducing-gpt-5-2/,85.65,2.23,9.75,5.26,81.85%,72.92%,88.00%,89.00%,77.50%,70.39%,71.71%,70.37%,68.75%,58.33%,28.12%,36.50%,18.00%,27.50%,30.50%,75.50%,78.00%,73.00%,45.81%,33.55%,43.23%,60.65%,75.00%,79.42%,N/A,N/A,OpenAI,Proprietary
17,55.46%,GPT-5-mini-2025-08-07 (FC),https://openai.com/index/introducing-gpt-5/,22.18,8.32,17.35,19.8,69.85%,59.92%,69.00%,80.00%,70.50%,58.62%,62.02%,58.02%,62.50%,45.83%,27.50%,36.50%,17.00%,23.50%,33.00%,82.00%,87.00%,77.00%,44.30%,36.77%,43.87%,52.26%,62.50%,91.01%,N/A,N/A,OpenAI,Proprietary
18,54.66%,xLAM-2-32b-fc-r (FC),https://huggingface.co/Salesforce/xLAM-2-32b-fc-r,6.0,6.94,8.21,17.66,89.60%,80.42%,94.00%,93.00%,91.00%,75.50%,82.17%,74.64%,50.00%,58.33%,69.50%,81.50%,72.50%,67.50%,56.50%,25.50%,37.00%,14.00%,20.86%,6.45%,10.32%,45.81%,81.25%,80.23%,N/A,N/A,Salesforce,cc-by-nc-4.0
19,54.12%,DeepSeek-V3.2-Exp (FC),https://api-docs.deepseek.com/news/news250528,6.71,5.83,11.71,10.59,34.85%,37.92%,74.00%,15.00%,12.50%,53.66%,66.28%,51.66%,25.00%,25.00%,37.38%,41.50%,39.50%,33.50%,35.00%,69.50%,80.00%,59.00%,54.19%,41.94%,61.29%,59.35%,37.50%,93.18%,N/A,N/A,DeepSeek,MIT
20,53.96%,GPT-4.1-2025-04-14 (FC),https://openai.com/index/gpt-4-1/,100.75,1.63,3.05,4.01,82.79%,72.67%,89.00%,88.00%,81.50%,69.95%,69.38%,70.28%,56.25%,70.83%,38.88%,47.50%,32.50%,32.50%,43.00%,68.00%,67.00%,69.00%,23.87%,16.13%,18.06%,37.42%,87.50%,86.52%,N/A,N/A,OpenAI,Proprietary
21,53.24%,o4-mini-2025-04-16 (FC),https://openai.com/index/introducing-o3-and-o4-mini/,81.91,3.71,7.18,9.33,37.73%,66.92%,84.00%,0.00%,0.00%,66.10%,69.38%,67.81%,0.00%,0.00%,41.75%,51.00%,30.00%,40.50%,45.50%,75.50%,75.00%,76.00%,34.19%,19.35%,24.52%,58.71%,81.25%,83.91%,N/A,N/A,OpenAI,Proprietary
22,53.07%,xLAM-2-70b-fc-r (FC),https://huggingface.co/Salesforce/Llama-xLAM-2-70b-fc-r,25.1,28.06,68.77,91.21,88.44%,78.25%,94.00%,92.00%,89.50%,72.17%,77.91%,71.13%,68.75%,58.33%,77.38%,82.50%,77.00%,74.00%,76.00%,15.00%,17.00%,13.00%,14.41%,2.58%,10.97%,29.68%,75.00%,79.11%,N/A,N/A,Salesforce,cc-by-nc-4.0
23,52.15%,Qwen3-235B-A22B-Instruct-2507 (Prompt),https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507,3.12,2.56,2.75,7.61,90.33%,79.83%,95.00%,95.50%,91.00%,78.68%,82.95%,77.78%,81.25%,70.83%,44.62%,54.00%,42.50%,31.50%,50.50%,50.50%,56.00%,45.00%,19.35%,12.90%,11.61%,33.55%,93.75%,78.89%,8.0,1.95,Qwen,apache-2.0
24,51.45%,GPT-5-nano-2025-08-07 (FC),https://openai.com/index/introducing-gpt-5/,8.79,10.36,10.37,23.56,68.00%,57.00%,64.50%,79.00%,71.50%,59.44%,58.91%,59.83%,50.00%,54.17%,34.50%,44.00%,23.50%,32.50%,38.00%,72.50%,74.00%,71.00%,24.73%,18.06%,27.10%,29.03%,75.00%,89.10%,N/A,N/A,OpenAI,Proprietary
25,51.40%,Nanbeige4-3B-Thinking-2511 (FC),https://huggingface.co/Nanbeige/Nanbeige4-3B-Thinking-2511,14.14,13.46,26.41,37.45,81.58%,63.83%,93.50%,84.50%,84.50%,79.42%,86.05%,78.06%,75.00%,70.83%,51.12%,58.50%,54.00%,45.00%,47.00%,21.50%,31.00%,12.00%,36.77%,31.61%,34.19%,44.52%,75.00%,83.09%,N/A,N/A,Nanbeige,apache-2.0
26,50.90%,Gemini-2.5-Flash (Prompt),https://deepmind.google/technologies/gemini/flash/,33.45,3.18,4.44,6.09,88.08%,77.33%,91.50%,96.00%,87.50%,78.16%,87.21%,75.97%,81.25%,75.00%,16.75%,14.50%,16.50%,17.50%,18.50%,62.00%,60.00%,64.00%,38.71%,13.55%,47.10%,55.48%,62.50%,91.09%,9.0,2.45,Google,Proprietary
27,50.45%,GPT-4.1-mini-2025-04-14 (FC),https://openai.com/index/gpt-4-1/,19.25,1.32,3.65,2.4,83.83%,73.33%,89.00%,91.00%,82.00%,68.84%,67.05%,69.80%,43.75%,62.50%,34.13%,43.50%,22.50%,30.50%,40.00%,57.00%,62.00%,52.00%,26.88%,22.58%,16.13%,41.94%,81.25%,81.69%,N/A,N/A,OpenAI,Proprietary
28,50.26%,o4-mini-2025-04-16 (Prompt),https://openai.com/index/introducing-o3-and-o4-mini/,133.63,4.47,5.19,10.19,81.29%,72.67%,88.00%,84.50%,80.00%,70.76%,79.46%,68.76%,75.00%,62.50%,16.62%,16.50%,18.00%,17.50%,14.50%,71.50%,73.00%,70.00%,35.27%,22.58%,25.16%,58.06%,81.25%,87.16%,9.5,2.6,OpenAI,Proprietary
29,48.71%,Qwen3-32B (FC),https://huggingface.co/Qwen/Qwen3-32B,153.08,169.87,164.27,473.49,88.77%,75.58%,94.50%,93.50%,91.50%,82.01%,89.53%,80.91%,81.25%,50.00%,47.87%,56.00%,52.50%,40.00%,43.00%,21.50%,25.00%,18.00%,26.67%,12.26%,25.81%,41.94%,93.75%,76.37%,N/A,N/A,Qwen,apache-2.0
30,48.56%,o3-2025-04-16 (FC),https://openai.com/index/introducing-o3-and-o4-mini/,133.45,3.5,8.69,8.39,40.38%,74.50%,87.00%,0.00%,0.00%,66.17%,70.54%,67.62%,0.00%,0.00%,14.75%,16.50%,11.50%,14.50%,16.50%,77.00%,79.00%,75.00%,47.31%,24.52%,44.52%,72.90%,81.25%,86.13%,N/A,N/A,OpenAI,Proprietary
31,47.99%,Qwen3-235B-A22B-Instruct-2507 (FC),https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507,2.5,2.57,2.44,6.27,37.40%,40.58%,36.50%,53.00%,19.50%,68.91%,58.53%,71.60%,68.75%,62.50%,45.38%,57.50%,35.00%,33.50%,55.50%,54.00%,57.00%,51.00%,23.87%,7.10%,18.71%,45.81%,87.50%,81.73%,N/A,N/A,Qwen,apache-2.0
32,47.68%,Nanbeige3.5-Pro-Thinking (FC),https://huggingface.co/Nanbeige,23.46,21.12,28.61,63.29,38.35%,43.92%,36.50%,53.00%,20.00%,69.95%,63.18%,71.42%,87.50%,66.67%,40.00%,56.00%,34.00%,29.00%,41.00%,42.00%,47.00%,37.00%,45.16%,38.06%,58.06%,39.35%,100.00%,74.20%,N/A,N/A,Nanbeige,apache-2.0
33,46.78%,Qwen3-32B (Prompt),https://huggingface.co/Qwen/Qwen3-32B,199.47,167.54,160.5,457.87,90.27%,79.08%,97.00%,93.50%,91.50%,82.01%,87.21%,81.20%,81.25%,62.50%,43.25%,54.00%,46.00%,36.50%,36.50%,26.00%,34.00%,18.00%,15.70%,13.55%,14.19%,19.35%,81.25%,82.39%,15.5,3.75,Qwen,apache-2.0
34,46.68%,xLAM-2-8b-fc-r (FC),https://huggingface.co/Salesforce/Llama-xLAM-2-8b-fc-r,20.92,22.65,46.92,108.81,84.58%,73.83%,93.50%,87.50%,83.50%,67.95%,75.58%,66.57%,56.25%,54.17%,70.00%,76.00%,72.00%,65.00%,67.00%,6.50%,11.00%,2.00%,13.98%,5.81%,15.48%,20.65%,87.50%,63.28%,N/A,N/A,Salesforce,cc-by-nc-4.0
35,46.49%,Command A (FC),https://cohere.com/blog/command-a,91.37,2.09,7.36,4.94,87.56%,75.75%,93.00%,93.50%,88.00%,78.53%,85.66%,76.92%,81.25%,70.83%,29.50%,38.00%,23.00%,32.00%,25.00%,46.50%,60.00%,33.00%,16.56%,4.52%,5.16%,40.00%,81.25%,84.19%,N/A,N/A,Cohere,CC-BY-NC 4.0 License (w/ Acceptable Use Addendum)
36,46.23%,BitAgent-Bounty-8B,https://huggingface.co/BitAgent/BitAgent-Bounty-8B,18.02,16.52,30.73,77.12,81.60%,72.42%,93.00%,83.00%,78.00%,93.12%,90.31%,94.02%,75.00%,95.83%,62.38%,75.00%,49.50%,68.00%,57.00%,0.00%,0.00%,0.00%,1.51%,1.29%,1.29%,1.94%,68.75%,97.48%,N/A,N/A,Bittensor,Apache-2.0
37,45.37%,Arch-Agent-32B,https://huggingface.co/katanemo/Arch-Agent-32B,8.87,9.44,21.44,24.87,88.92%,76.67%,94.00%,96.00%,89.00%,80.68%,86.43%,79.11%,93.75%,79.17%,54.25%,64.50%,58.00%,53.00%,41.50%,5.00%,4.00%,6.00%,14.62%,5.81%,9.03%,29.03%,81.25%,82.15%,N/A,N/A,katanemo,katanemo-research
38,45.27%,GPT-5.2-2025-12-11 (Prompt),https://openai.com/zh-Hans-CN/index/introducing-gpt-5-2/,164.58,4.21,20.93,10.58,78.29%,71.17%,83.50%,84.00%,74.50%,67.14%,77.91%,64.58%,75.00%,58.33%,43.75%,54.50%,40.50%,33.50%,46.50%,40.50%,45.00%,36.00%,3.87%,2.58%,1.94%,7.10%,75.00%,87.26%,13.0,3.25,OpenAI,Proprietary
39,42.57%,Qwen3-8B (FC),https://huggingface.co/Qwen/Qwen3-8B,43.32,51.36,76.14,188.98,87.58%,72.83%,96.50%,92.00%,89.00%,80.53%,84.50%,79.68%,75.00%,79.17%,41.75%,50.50%,42.00%,40.00%,34.50%,12.00%,15.00%,9.00%,14.62%,5.16%,7.10%,31.61%,93.75%,79.07%,N/A,N/A,Qwen,apache-2.0
40,42.44%,ToolACE-2-8B (FC),https://huggingface.co/Team-ACE/ToolACE-2-8B,24.43,15.95,40.06,65.26,87.10%,73.42%,91.00%,93.00%,91.00%,77.42%,71.32%,79.39%,68.75%,62.50%,38.38%,49.00%,28.00%,30.50%,46.00%,8.50%,13.00%,4.00%,18.49%,5.81%,16.13%,33.55%,75.00%,90.79%,81.5,27.92,Huawei Noah & USTC,Apache-2.0
41,41.39%,Qwen3-30B-A3B-Instruct-2507 (FC),https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507,5.62,5.95,25.48,12.7,85.77%,68.58%,94.50%,91.50%,88.50%,77.94%,83.33%,76.83%,68.75%,75.00%,30.00%,43.50%,10.50%,25.00%,41.00%,22.50%,21.00%,24.00%,17.63%,9.03%,9.03%,34.84%,81.25%,79.90%,N/A,N/A,Qwen,apache-2.0
42,41.22%,xLAM-2-3b-fc-r (FC),https://huggingface.co/Salesforce/xLAM-2-3b-fc-r,3.36,3.8,3.59,8.79,82.96%,75.33%,91.00%,86.50%,79.00%,62.92%,73.26%,60.68%,62.50%,50.00%,58.38%,71.50%,59.00%,57.50%,45.50%,2.50%,3.00%,2.00%,11.40%,5.81%,5.81%,22.58%,87.50%,63.45%,N/A,N/A,Salesforce,cc-by-nc-4.0
43,41.03%,Qwen3-14B (FC),https://huggingface.co/Qwen/Qwen3-14B,3.38,4.5,18.84,13.34,84.94%,74.75%,93.00%,80.00%,92.00%,80.01%,85.66%,79.01%,68.75%,70.83%,34.75%,39.00%,34.00%,33.50%,32.50%,10.00%,8.00%,12.00%,19.57%,7.10%,16.77%,34.84%,87.50%,81.94%,N/A,N/A,Qwen,apache-2.0
44,40.43%,Qwen3-8B (Prompt),https://huggingface.co/Qwen/Qwen3-8B,63.95,54.17,79.9,194.15,88.56%,75.25%,95.00%,94.50%,89.50%,80.09%,84.50%,78.92%,93.75%,75.00%,33.38%,41.50%,38.50%,27.00%,26.50%,13.50%,19.00%,8.00%,13.12%,3.87%,10.32%,25.16%,75.00%,82.27%,16.5,5.09,Qwen,apache-2.0
45,39.38%,GPT-4.1-2025-04-14 (Prompt),https://openai.com/index/gpt-4-1/,145.85,1.2,3.23,2.53,88.69%,78.25%,93.50%,94.00%,89.00%,78.90%,84.88%,77.40%,87.50%,75.00%,9.75%,10.50%,11.00%,8.00%,9.50%,35.00%,40.00%,30.00%,21.51%,9.68%,19.35%,35.48%,100.00%,83.99%,23.5,6.18,OpenAI,Proprietary
46,38.37%,mistral-large-2411 (FC),https://docs.mistral.ai/guides/model-selection/,115.98,2.04,4.02,4.68,84.65%,72.08%,93.50%,89.50%,83.50%,81.87%,87.21%,80.72%,81.25%,75.00%,14.12%,18.50%,11.50%,13.00%,13.50%,28.00%,41.00%,15.00%,24.95%,18.71%,29.03%,27.10%,93.75%,68.92%,N/A,N/A,Mistral AI,Proprietary
47,37.77%,Qwen3-14B (Prompt),https://huggingface.co/Qwen/Qwen3-14B,1.35,1.2,8.5,2.3,89.46%,76.83%,93.50%,95.50%,92.00%,79.35%,84.50%,78.06%,87.50%,75.00%,26.13%,16.50%,37.50%,31.00%,19.50%,10.50%,6.00%,15.00%,11.18%,4.52%,6.45%,22.58%,81.25%,87.18%,14.0,3.97,Qwen,apache-2.0
48,37.69%,Mistral-Medium-2505,https://docs.mistral.ai/guides/model-selection/,36.51,1.21,3.5,2.86,85.33%,76.33%,91.00%,88.50%,85.50%,66.03%,80.23%,62.39%,81.25%,62.50%,9.88%,13.50%,6.50%,6.00%,13.50%,39.00%,41.00%,37.00%,21.72%,16.13%,14.84%,34.19%,75.00%,74.49%,21.5,5.02,Mistral AI,Proprietary
49,37.56%,Mistral-Medium-2505 (FC),https://docs.mistral.ai/guides/model-selection/,18.8,1.6,4.44,4.19,67.44%,39.75%,78.00%,83.00%,69.00%,67.95%,67.05%,68.09%,81.25%,62.50%,10.75%,15.50%,7.00%,7.50%,13.00%,35.00%,36.00%,34.00%,23.01%,15.48%,20.00%,33.55%,62.50%,91.95%,N/A,N/A,Mistral AI,Proprietary
50,37.29%,Llama-4-Maverick-17B-128E-Instruct-FP8 (FC),https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8,18.25,18.43,34.11,102.75,88.65%,77.08%,95.00%,94.00%,88.50%,73.65%,84.50%,71.04%,75.00%,70.83%,20.25%,27.00%,22.00%,14.00%,18.00%,28.00%,39.00%,17.00%,18.92%,8.39%,32.90%,15.48%,100.00%,55.97%,N/A,N/A,Meta,Meta Llama 4 Community
51,37.15%,Mistral-small-2506 (FC),https://docs.mistral.ai/guides/model-selection/,5.2,1.48,18.25,2.5,73.60%,38.92%,93.50%,83.50%,78.50%,77.28%,69.38%,79.39%,75.00%,70.83%,11.50%,17.50%,6.00%,10.50%,12.00%,31.00%,37.00%,25.00%,18.06%,8.39%,14.19%,31.61%,87.50%,87.94%,N/A,N/A,Mistral AI,Proprietary
52,36.87%,Gemini-2.5-Flash-Lite (FC),https://deepmind.google/technologies/gemini/flash-lite/,7.55,1.18,8.06,1.67,86.60%,70.92%,90.00%,93.50%,92.00%,65.80%,73.26%,63.82%,75.00%,66.67%,13.50%,20.00%,1.50%,15.00%,17.50%,21.00%,26.00%,16.00%,20.65%,3.87%,6.45%,51.61%,43.75%,92.50%,N/A,N/A,Google,Proprietary
53,36.70%,Qwen3-30B-A3B-Instruct-2507 (Prompt),https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507,1.56,1.24,7.9,2.84,88.92%,80.67%,93.00%,94.00%,88.00%,78.39%,82.56%,77.49%,87.50%,66.67%,23.50%,33.00%,16.00%,16.00%,29.00%,17.50%,15.00%,20.00%,9.68%,5.81%,6.45%,16.77%,93.75%,74.85%,16.0,4.13,Qwen,apache-2.0
54,35.68%,Qwen3-4B-Instruct-2507 (FC),https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507,6.37,7.61,20.36,49.18,87.88%,75.50%,93.50%,92.50%,90.00%,76.39%,79.07%,76.16%,62.50%,66.67%,22.12%,26.50%,21.00%,15.50%,25.50%,3.00%,4.00%,2.00%,17.63%,16.13%,12.26%,24.52%,87.50%,84.93%,N/A,N/A,Qwen,apache-2.0
55,35.52%,Qwen3-4B-Instruct-2507 (Prompt),https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507,53.66,44.7,163.79,208.06,86.44%,77.25%,91.00%,88.00%,89.50%,74.69%,77.91%,74.17%,81.25%,58.33%,20.50%,24.50%,21.50%,16.00%,20.00%,4.50%,4.00%,5.00%,23.87%,12.90%,14.19%,44.52%,87.50%,75.87%,18.0,5.22,Qwen,apache-2.0
56,35.36%,Arch-Agent-3B,https://huggingface.co/katanemo/Arch-Agent-3B,3.7,3.56,6.65,8.19,86.67%,78.67%,94.50%,91.00%,82.50%,72.91%,75.58%,72.27%,68.75%,75.00%,34.88%,42.00%,37.50%,31.00%,29.00%,0.50%,1.00%,0.00%,6.88%,5.16%,5.81%,9.68%,68.75%,74.67%,N/A,N/A,katanemo,katanemo-research
57,33.47%,Claude-Opus-4-5-20251101 (Prompt),https://www.anthropic.com/news/claude-4,88.33,3.76,13.19,5.52,89.65%,79.58%,93.50%,93.00%,92.50%,76.02%,84.50%,74.17%,81.25%,62.50%,16.12%,20.50%,9.00%,21.50%,13.50%,13.00%,13.00%,13.00%,1.94%,1.29%,1.94%,2.58%,68.75%,90.75%,13.0,3.65,Anthropic,Proprietary
58,33.05%,GPT-4.1-nano-2025-04-14 (FC),https://openai.com/index/gpt-4-1/,5.66,1.44,10.84,2.26,72.98%,59.92%,79.50%,84.00%,68.50%,60.77%,58.14%,61.44%,68.75%,54.17%,23.62%,39.50%,7.50%,17.50%,30.00%,11.00%,13.00%,9.00%,18.92%,10.32%,19.35%,27.10%,93.75%,66.00%,N/A,N/A,OpenAI,Proprietary
59,32.38%,Mistral-Small-2506 (Prompt),https://docs.mistral.ai/guides/model-selection/,6.91,0.92,6.79,2.02,89.69%,78.75%,96.00%,92.50%,91.50%,79.05%,81.40%,78.54%,93.75%,66.67%,14.75%,20.50%,17.00%,9.50%,12.00%,7.50%,9.00%,6.00%,15.05%,2.58%,11.61%,30.97%,93.75%,65.73%,50.0,13.57,Mistral AI,Proprietary
60,32.14%,Arch-Agent-1.5B,https://huggingface.co/katanemo/Arch-Agent-1.5B,2.45,2.38,4.01,5.3,82.67%,72.17%,92.00%,85.50%,81.00%,67.73%,70.54%,67.81%,31.25%,58.33%,26.62%,35.50%,27.50%,21.50%,22.00%,0.00%,0.00%,0.00%,8.17%,5.81%,5.81%,12.90%,75.00%,74.83%,N/A,N/A,katanemo,katanemo-research
61,32.07%,Command R7B (FC),https://cohere.com/blog/command-r7b,1.5,1.38,2.87,2.69,80.96%,67.33%,89.50%,85.50%,81.50%,69.06%,62.79%,70.94%,43.75%,70.83%,8.25%,12.00%,0.50%,10.50%,10.00%,27.00%,43.00%,11.00%,5.16%,2.58%,9.68%,3.23%,68.75%,81.65%,N/A,N/A,Cohere,cc-by-nc-4.0
62,31.90%,Llama-3.3-70B-Instruct (FC),https://llama.meta.com/llama3,29.54,26.11,93.22,187.93,88.02%,76.08%,95.00%,90.00%,91.00%,76.61%,81.40%,75.50%,81.25%,70.83%,21.50%,26.00%,19.00%,14.50%,26.50%,10.00%,14.00%,6.00%,8.17%,4.52%,8.39%,11.61%,100.00%,53.53%,N/A,N/A,Meta,Meta Llama 3 Community
63,31.84%,mistral-large-2411 (Prompt),https://docs.mistral.ai/guides/model-selection/,232.42,1.82,7.15,4.08,83.00%,75.50%,89.50%,87.00%,80.00%,68.10%,83.72%,64.01%,93.75%,62.50%,13.75%,20.00%,5.00%,11.00%,19.00%,20.00%,28.00%,12.00%,23.66%,16.77%,30.97%,23.23%,93.75%,38.77%,13.5,3.91,Mistral AI,Proprietary
64,31.67%,Hammer2.1-7b (FC),https://huggingface.co/MadeAgents/Hammer2.1-7b,4.99,5.77,10.29,31.26,85.50%,72.50%,92.50%,91.00%,86.00%,69.50%,66.67%,69.99%,75.00%,75.00%,23.87%,24.00%,28.50%,21.50%,21.50%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,50.00%,90.12%,N/A,N/A,MadeAgents,cc-by-nc-4.0
65,30.44%,xLAM-2-1b-fc-r (FC),https://huggingface.co/Salesforce/xLAM-2-1b-fc-r,2.79,2.84,2.35,6.52,69.04%,64.17%,82.50%,73.50%,56.00%,55.14%,68.22%,52.80%,43.75%,25.00%,36.00%,45.50%,36.00%,37.00%,25.50%,0.00%,0.00%,0.00%,3.87%,3.87%,3.87%,3.87%,87.50%,64.47%,N/A,N/A,Salesforce,cc-by-nc-4.0
66,30.43%,Gemma-3-12b-it (Prompt),https://blog.google/technology/developers/gemma-3/,10.77,11.1,17.17,34.66,79.44%,76.25%,94.00%,91.00%,56.50%,74.24%,85.66%,71.89%,87.50%,45.83%,5.75%,6.50%,7.50%,5.00%,4.00%,4.00%,4.00%,4.00%,27.53%,8.39%,25.16%,49.03%,93.75%,70.29%,67.5,22.41,Google,gemma-terms-of-use
67,29.73%,GPT-4.1-mini-2025-04-14 (Prompt),https://openai.com/index/gpt-4-1/,20.52,1.36,4.5,3.38,84.60%,74.92%,92.50%,87.50%,83.50%,74.76%,80.62%,73.31%,81.25%,70.83%,2.50%,1.50%,4.50%,2.50%,1.50%,4.00%,7.00%,1.00%,24.30%,20.65%,13.55%,38.71%,87.50%,73.88%,45.0,13.33,OpenAI,Proprietary
68,29.71%,Hammer2.1-3b (FC),https://huggingface.co/MadeAgents/Hammer2.1-3b,10.89,11.24,15.81,47.44,84.96%,79.33%,93.50%,86.50%,80.50%,70.54%,68.22%,71.32%,62.50%,66.67%,16.50%,22.00%,12.50%,16.00%,15.50%,0.00%,0.00%,0.00%,3.01%,2.58%,3.87%,2.58%,56.25%,86.12%,N/A,N/A,MadeAgents,qwen-research
69,29.47%,Gemma-3-27b-it (Prompt),https://blog.google/technology/developers/gemma-3/,11.82,10.88,19.67,55.5,87.17%,77.67%,92.50%,89.00%,89.50%,74.54%,84.50%,72.46%,93.75%,45.83%,10.75%,16.50%,4.50%,8.00%,14.00%,0.00%,0.00%,0.00%,13.55%,1.94%,3.23%,35.48%,81.25%,73.67%,34.0,8.06,Google,gemma-terms-of-use
70,28.79%,Phi-4 (Prompt),https://huggingface.co/microsoft/phi-4,8.72,9.49,26.73,23.02,69.56%,74.25%,89.50%,65.00%,49.50%,60.70%,65.50%,59.64%,81.25%,41.67%,3.88%,9.00%,0.00%,3.50%,3.00%,4.50%,4.00%,5.00%,24.73%,17.42%,25.16%,31.61%,50.00%,87.55%,81.5,23.34,Microsoft,MIT
71,28.41%,Qwen3-1.7B (FC),https://huggingface.co/Qwen/Qwen3-1.7B,4.33,5.12,7.37,13.35,82.92%,70.67%,92.50%,88.50%,80.00%,74.61%,76.74%,74.26%,62.50%,75.00%,11.00%,15.00%,6.00%,12.00%,11.00%,2.50%,3.00%,2.00%,6.02%,4.52%,7.74%,5.81%,81.25%,76.54%,N/A,N/A,Qwen,apache-2.0
72,28.13%,Llama-4-Scout-17B-16E-Instruct (FC),https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct,24.68,17.86,50.68,166.2,89.38%,79.00%,94.00%,94.00%,90.50%,74.69%,81.78%,72.74%,81.25%,79.17%,9.00%,12.00%,7.00%,7.50%,9.50%,14.50%,18.00%,11.00%,8.17%,2.58%,2.58%,19.35%,100.00%,44.92%,N/A,N/A,Meta,Meta Llama 4 Community
73,28.03%,Gemini-2.5-Flash-Lite (Prompt),https://deepmind.google/technologies/gemini/flash-lite/,7.05,1.0,4.75,1.4,83.90%,70.08%,86.00%,90.00%,89.50%,54.85%,67.05%,51.66%,75.00%,50.00%,7.63%,10.00%,5.00%,6.50%,9.00%,0.00%,0.00%,0.00%,12.69%,1.94%,6.45%,29.68%,50.00%,93.33%,25.5,6.68,Google,Proprietary
74,27.99%,CoALM-70B,https://huggingface.co/uiuc-convai/CoALM-70B,19.89,16.22,59.91,36.0,83.44%,70.25%,92.00%,88.50%,83.00%,67.28%,70.54%,66.57%,68.75%,62.50%,10.62%,11.00%,14.00%,9.00%,8.50%,0.00%,0.00%,0.00%,5.81%,9.03%,5.16%,3.23%,93.75%,85.65%,72.0,27.76,UIUC + Oumi,Meta Llama 3 Community
75,27.88%,Hammer2.1-1.5b (FC),https://huggingface.co/MadeAgents/Hammer2.1-1.5b,6.83,6.28,8.79,30.72,82.98%,73.92%,92.00%,85.50%,80.50%,69.50%,72.09%,69.33%,50.00%,62.50%,15.62%,20.00%,16.50%,9.50%,16.50%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,75.00%,79.40%,N/A,N/A,MadeAgents,cc-by-nc-4.0
76,27.87%,palmyra-x-004 (FC),https://writer.com/engineering/actions-with-palmyra-x-004/,178.15,3.71,7.62,8.04,87.46%,71.33%,96.00%,93.00%,89.50%,77.87%,79.46%,77.97%,56.25%,70.83%,0.38%,0.50%,0.00%,0.50%,0.50%,2.50%,4.00%,1.00%,13.12%,6.45%,14.19%,18.71%,81.25%,80.99%,N/A,N/A,Writer,Proprietary
77,27.83%,GPT-5-mini-2025-08-07 (Prompt),https://openai.com/index/introducing-gpt-5/,82.74,8.89,11.08,19.72,68.04%,59.17%,72.50%,71.50%,69.00%,62.55%,69.77%,61.16%,75.00%,37.50%,5.50%,5.50%,5.00%,4.50%,7.00%,8.50%,11.00%,6.00%,29.25%,19.35%,29.68%,38.71%,93.75%,55.71%,16.0,3.78,OpenAI,Proprietary
78,27.63%,Open-Mistral-Nemo-2407 (FC),https://mistral.ai/news/mistral-nemo/,8.12,1.07,11.93,1.39,82.81%,65.75%,92.50%,88.50%,84.50%,73.80%,78.68%,72.84%,68.75%,66.67%,7.75%,12.50%,6.50%,7.50%,4.50%,7.00%,9.00%,5.00%,10.32%,8.39%,9.68%,12.90%,81.25%,61.77%,N/A,N/A,Mistral AI,Proprietary
79,27.55%,GPT-5-nano-2025-08-07 (Prompt),https://openai.com/index/introducing-gpt-5/,21.47,10.67,7.68,23.28,80.81%,69.25%,86.00%,87.50%,80.50%,70.69%,76.36%,69.71%,68.75%,54.17%,0.75%,1.00%,1.00%,0.00%,1.00%,13.50%,10.00%,17.00%,24.52%,20.65%,31.61%,21.29%,93.75%,45.75%,8.5,2.57,OpenAI,Proprietary
80,27.10%,Amazon-Nova-2-Lite-v1:0 (FC),https://aws.amazon.com/cn/ai/generative-ai/nova/,78.19,8.55,9.85,27.62,86.96%,76.33%,94.00%,91.50%,86.00%,80.83%,83.33%,80.15%,87.50%,79.17%,2.12%,2.50%,1.50%,2.00%,2.50%,5.00%,4.00%,6.00%,2.37%,1.94%,3.23%,1.94%,75.00%,82.11%,N/A,N/A,Amazon,Proprietary
81,27.10%,Granite-3.1-8B-Instruct (FC),https://huggingface.co/ibm-granite/granite-3.1-8b-instruct,9.32,13.23,31.28,65.19,78.33%,67.33%,92.00%,84.00%,70.00%,60.33%,58.53%,61.82%,18.75%,41.67%,7.50%,11.50%,2.00%,7.50%,9.00%,0.50%,1.00%,0.00%,14.41%,9.68%,7.10%,26.45%,68.75%,79.98%,N/A,N/A,IBM,Apache-2.0
82,27.01%,Falcon3-10B-Instruct (FC),https://huggingface.co/tiiuae/Falcon3-10B-Instruct,52.59,69.27,92.22,190.96,85.00%,70.50%,93.50%,88.50%,87.50%,75.43%,77.13%,76.16%,50.00%,41.67%,6.50%,6.50%,9.50%,5.00%,5.00%,1.50%,2.00%,1.00%,27.53%,12.26%,19.35%,50.97%,93.75%,32.09%,N/A,N/A,TII UAE,falcon-llm-license
83,26.87%,Granite-3.2-8B-Instruct (FC),https://huggingface.co/ibm-granite/granite-3.2-8b-instruct,25.02,36.13,81.76,216.28,79.77%,69.58%,88.50%,88.50%,72.50%,60.33%,60.47%,61.16%,25.00%,45.83%,7.38%,9.50%,3.00%,8.00%,9.00%,0.50%,1.00%,0.00%,12.47%,6.45%,9.68%,21.29%,75.00%,80.53%,N/A,N/A,IBM,Apache-2.0
84,26.81%,CoALM-8B,https://huggingface.co/uiuc-convai/CoALM-8B,25.33,20.36,73.74,138.04,84.87%,69.50%,93.50%,88.00%,88.50%,66.77%,70.54%,66.19%,62.50%,54.17%,8.00%,10.00%,7.00%,8.00%,7.00%,0.00%,0.00%,0.00%,2.80%,3.23%,3.87%,1.29%,87.50%,86.90%,79.0,34.18,UIUC + Oumi,Meta Llama 3 Community
85,25.83%,Llama-3.1-8B-Instruct (Prompt),https://llama.meta.com/llama3,7.49,5.6,19.37,22.6,84.00%,71.00%,95.00%,87.50%,82.50%,70.76%,72.87%,71.13%,50.00%,45.83%,11.12%,13.00%,9.00%,9.50%,13.00%,3.00%,6.00%,0.00%,10.75%,7.74%,5.81%,18.71%,93.75%,42.70%,74.5,29.1,Meta,Meta Llama 3 Community
86,25.55%,MiniCPM3-4B-FC (FC),https://huggingface.co/openbmb/MiniCPM3-4B,54.05,118.62,143.98,388.67,81.75%,70.50%,92.00%,84.00%,80.50%,65.21%,73.26%,63.53%,50.00%,62.50%,3.88%,6.50%,2.00%,4.50%,2.50%,0.00%,0.00%,0.00%,12.04%,9.68%,15.48%,10.97%,68.75%,72.84%,N/A,N/A,openbmb,Apache-2.0
87,25.26%,Claude-Haiku-4-5-20251001 (Prompt),https://www.anthropic.com/news/claude-haiku-4-5,45.13,3.75,19.96,3.77,55.42%,55.67%,84.00%,38.00%,44.00%,52.48%,66.67%,49.76%,56.25%,16.67%,1.75%,1.50%,0.00%,4.00%,1.50%,19.50%,20.00%,19.00%,2.58%,2.58%,1.94%,3.23%,31.25%,95.29%,67.5,20.07,Anthropic,Proprietary
88,24.97%,Amazon-Nova-Pro-v1:0 (FC),https://aws.amazon.com/cn/ai/generative-ai/nova/,48.44,2.25,1.91,3.29,86.58%,75.83%,93.00%,93.50%,84.00%,78.53%,81.40%,77.97%,81.25%,70.83%,1.88%,1.50%,0.50%,2.50%,3.00%,2.50%,4.00%,1.00%,1.94%,2.58%,1.29%,1.94%,93.75%,70.06%,N/A,N/A,Amazon,Proprietary
89,24.90%,Claude-Sonnet-4-5-20250929 (Prompt),https://www.anthropic.com/news/claude-sonnet-4-5,47.82,3.84,1.53,6.66,59.81%,47.25%,79.50%,53.50%,59.00%,46.56%,73.26%,40.17%,56.25%,33.33%,1.62%,2.00%,0.00%,3.00%,1.50%,16.00%,16.00%,16.00%,5.38%,4.52%,9.68%,1.94%,37.50%,95.03%,37.5,10.07,Anthropic,Proprietary
90,24.88%,GPT-4.1-nano-2025-04-14 (Prompt),https://openai.com/index/gpt-4-1/,7.42,1.02,7.3,1.88,72.44%,68.75%,63.50%,85.00%,72.50%,50.33%,63.18%,46.53%,87.50%,54.17%,2.00%,2.50%,1.00%,2.50%,2.00%,1.50%,2.00%,1.00%,16.77%,9.03%,14.19%,27.10%,68.75%,83.44%,73.0,17.08,OpenAI,Proprietary
91,24.03%,Falcon3-7B-Instruct (FC),https://huggingface.co/tiiuae/Falcon3-7B-Instruct,73.61,93.11,117.8,315.7,82.69%,65.75%,89.00%,87.00%,89.00%,68.32%,74.81%,66.76%,75.00%,62.50%,5.00%,7.00%,4.00%,5.00%,4.00%,0.50%,1.00%,0.00%,20.65%,10.32%,12.90%,38.71%,100.00%,31.99%,N/A,N/A,TII UAE,falcon-llm-license
92,23.93%,Qwen3-0.6B (FC),https://huggingface.co/Qwen/Qwen3-0.6B,0.46,0.68,8.45,0.96,71.79%,64.17%,86.00%,67.50%,69.50%,56.62%,61.24%,56.13%,43.75%,37.50%,3.62%,5.50%,2.00%,3.00%,4.00%,1.00%,1.00%,1.00%,8.60%,2.58%,1.94%,21.29%,75.00%,80.84%,N/A,N/A,Qwen,apache-2.0
93,23.23%,Granite-20b-FunctionCalling (FC),https://huggingface.co/ibm-granite/granite-20b-functioncalling,5.23,3.2,3.43,9.97,82.35%,72.92%,91.50%,83.50%,81.50%,58.70%,67.83%,56.70%,43.75%,58.33%,5.38%,9.00%,3.00%,6.50%,3.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,87.50%,75.13%,N/A,N/A,IBM,Apache-2.0
94,22.38%,Qwen3-0.6B (Prompt),https://huggingface.co/Qwen/Qwen3-0.6B,3.65,3.1,4.32,10.31,70.00%,64.00%,78.00%,75.00%,63.00%,49.37%,57.75%,47.77%,37.50%,37.50%,1.38%,1.50%,1.50%,1.50%,1.00%,0.50%,1.00%,0.00%,8.39%,1.29%,2.58%,21.29%,75.00%,82.50%,60.5,24.35,Qwen,apache-2.0
95,22.29%,Amazon-Nova-Micro-v1:0 (FC),https://aws.amazon.com/cn/ai/generative-ai/nova/,1.81,1.12,0.45,1.79,74.10%,70.92%,87.50%,75.50%,62.50%,66.32%,72.09%,64.96%,81.25%,54.17%,1.38%,1.50%,1.00%,2.00%,1.00%,1.50%,3.00%,0.00%,2.37%,1.94%,2.58%,2.58%,81.25%,70.65%,N/A,N/A,Amazon,Proprietary
96,22.25%,RZN-T (Prompt),https://huggingface.co/phronetic-ai/RZN-T,12.31,12.32,27.53,39.84,67.94%,63.75%,75.00%,69.50%,63.50%,49.74%,61.24%,47.20%,43.75%,41.67%,2.88%,4.50%,2.00%,2.50%,2.50%,0.00%,0.00%,0.00%,6.88%,9.03%,6.45%,5.16%,81.25%,82.41%,63.5,25.53,Phronetic AI,apache-2.0
97,22.08%,MiniCPM3-4B (Prompt),https://huggingface.co/openbmb/MiniCPM3-4B,29.83,31.18,35.61,102.02,70.54%,66.17%,77.00%,70.00%,69.00%,43.15%,47.67%,42.17%,43.75%,37.50%,3.50%,4.50%,4.50%,2.00%,3.00%,2.00%,2.00%,2.00%,9.46%,8.39%,10.32%,9.68%,56.25%,73.71%,68.0,16.55,openbmb,Apache-2.0
98,21.95%,Llama-3.2-3B-Instruct (FC),https://llama.meta.com/llama3,6.2,6.1,20.07,17.27,82.67%,70.67%,92.50%,88.50%,79.00%,58.33%,65.12%,57.64%,25.00%,37.50%,4.00%,5.00%,3.50%,4.00%,3.50%,1.00%,2.00%,0.00%,6.24%,3.23%,3.23%,12.26%,87.50%,52.06%,N/A,N/A,Meta,Meta Llama 3 Community
99,21.90%,Bielik-11B-v2.3-Instruct (Prompt),https://huggingface.co/speakleash/Bielik-11B-v2.3-Instruct,22.44,23.75,61.76,72.8,81.50%,73.00%,92.00%,85.50%,75.50%,67.80%,75.58%,66.19%,62.50%,58.33%,2.62%,4.50%,0.50%,3.00%,2.50%,1.50%,1.00%,2.00%,11.40%,7.10%,4.52%,22.58%,93.75%,36.01%,35.0,9.74,SpeakLeash & ACK Cyfronet AGH,Apache 2.0
100,21.22%,Hammer2.1-0.5b (FC),https://huggingface.co/MadeAgents/Hammer2.1-0.5b,2.82,2.79,3.17,9.86,65.98%,62.42%,81.00%,69.00%,51.50%,54.63%,56.59%,54.42%,62.50%,37.50%,2.88%,4.50%,0.50%,4.00%,2.50%,0.00%,0.00%,0.00%,1.08%,0.65%,1.94%,0.65%,68.75%,80.79%,N/A,N/A,MadeAgents,cc-by-nc-4.0
101,19.62%,Gemma-3-4b-it (Prompt),https://blog.google/technology/developers/gemma-3/,4.14,4.69,9.53,11.42,61.12%,64.50%,88.00%,56.00%,36.00%,60.84%,70.93%,59.35%,25.00%,41.67%,0.38%,0.50%,0.00%,0.50%,0.50%,1.00%,1.00%,1.00%,8.60%,9.68%,9.68%,6.45%,100.00%,53.94%,69.5,23.67,Google,gemma-terms-of-use
102,19.31%,Open-Mistral-Nemo-2407 (Prompt),https://mistral.ai/news/mistral-nemo/,13.8,0.84,7.05,1.32,88.46%,79.33%,92.50%,90.50%,91.50%,73.95%,78.29%,73.03%,87.50%,58.33%,0.75%,0.50%,1.00%,0.00%,1.50%,2.50%,3.00%,2.00%,8.60%,9.68%,9.68%,6.45%,93.75%,6.28%,14.5,4.6,Mistral AI,Proprietary
103,18.98%,Granite-4.0-350m (FC),https://huggingface.co/ibm-granite/granite-4.0-350m,1.44,1.74,4.85,3.44,67.92%,61.67%,84.50%,70.00%,55.50%,46.11%,61.24%,42.36%,68.75%,33.33%,2.50%,5.00%,0.50%,2.50%,2.00%,0.50%,0.00%,1.00%,3.23%,1.94%,1.29%,6.45%,81.25%,60.84%,N/A,N/A,IBM,Apache-2.0
104,16.25%,Falcon3-3B-Instruct (FC),https://huggingface.co/tiiuae/Falcon3-3B-Instruct,36.7,38.52,107.47,103.62,54.62%,56.50%,69.50%,67.00%,25.50%,54.48%,57.36%,54.70%,25.00%,33.33%,1.00%,1.50%,0.50%,0.50%,1.50%,1.00%,1.00%,1.00%,7.74%,6.45%,8.39%,8.39%,81.25%,32.92%,N/A,N/A,TII UAE,falcon-llm-license
105,11.10%,Ministral-8B-Instruct-2410 (FC),https://huggingface.co/mistralai/Ministral-8B-Instruct-2410,70.01,82.07,212.99,568.59,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,1.00%,2.00%,0.00%,4.52%,3.87%,7.10%,2.58%,0.00%,100.00%,0.0,0.0,Mistral AI,Mistral AI Research License
106,11.08%,Falcon3-1B-Instruct (FC),https://huggingface.co/tiiuae/Falcon3-1B-Instruct,1.72,5.23,14.34,11.48,9.02%,2.58%,6.00%,18.00%,9.50%,2.89%,4.26%,2.37%,0.00%,12.50%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,5.81%,5.16%,7.74%,4.52%,0.00%,87.30%,N/A,N/A,TII UAE,falcon-llm-license
107,10.82%,Llama-3.2-1B-Instruct (FC),https://llama.meta.com/llama3,1.64,3.21,10.04,9.77,38.38%,44.00%,50.50%,44.00%,15.00%,11.77%,31.78%,7.31%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,3.23%,2.58%,2.58%,4.52%,43.75%,51.57%,N/A,N/A,Meta,Meta Llama 3 Community
108,10.00%,Llama-3.1-Nemotron-Ultra-253B-v1 (FC),https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1,0.72,1.42,1.84,2.4,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,100.00%,N/A,N/A,NVIDIA,nvidia-open-model-license
109,7.17%,Gemma-3-1b-it (Prompt),https://blog.google/technology/developers/gemma-3/,3.4,3.98,9.8,12.06,20.21%,43.33%,36.00%,0.00%,1.50%,11.84%,36.43%,6.27%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,3.23%,3.87%,3.87%,1.94%,37.50%,33.18%,25.5,9.76,Google,gemma-terms-of-use