Rank,Overall Acc,Model,Model Link,Cost ($ Per 1k Function Calls),Latency Mean (s),Latency Standard Deviation (s),Latency 95th Percentile (s),Non-Live AST Acc,Non-Live Simple AST,Non-Live Multiple AST,Non-Live Parallel AST,Non-Live Parallel Multiple AST,Live Acc,Live Simple AST,Live Multiple AST,Live Parallel AST,Live Parallel Multiple AST,Multi Turn Acc,Multi Turn Base,Multi Turn Miss Func,Multi Turn Miss Param,Multi Turn Long Context,Relevance Detection,Irrelevance Detection,Organization,License
1,78.45%,xLAM-2-70b-fc-r (FC),https://huggingface.co/Salesforce/Llama-xLAM-2-70b-fc-r,N/A,N/A,N/A,N/A,88.44%,78.25%,94.50%,92.00%,89.00%,72.95%,77.13%,71.13%,68.75%,58.33%,75.00%,77.50%,75.50%,73.00%,74.00%,66.67%,78.91%,Salesforce,cc-by-nc-4.0
2,76.43%,xLAM-2-32b-fc-r (FC),https://huggingface.co/Salesforce/xLAM-2-32b-fc-r,N/A,N/A,N/A,N/A,89.27%,80.08%,94.00%,92.50%,90.50%,74.23%,82.95%,75.21%,56.25%,58.33%,67.12%,79.00%,71.50%,63.50%,54.50%,88.89%,76.74%,Salesforce,cc-by-nc-4.0
3,73.57%,watt-tool-70B (FC),https://huggingface.co/watt-ai/watt-tool-70B/,N/A,N/A,N/A,N/A,84.06%,78.75%,94.00%,85.50%,78.00%,77.74%,86.05%,83.48%,81.25%,62.50%,58.87%,68.00%,57.50%,48.50%,61.50%,94.44%,76.32%,Watt AI Lab,Apache-2.0
4,72.04%,xLAM-2-8b-fc-r (FC),https://huggingface.co/Salesforce/Llama-xLAM-2-8b-fc-r,N/A,N/A,N/A,N/A,84.40%,73.08%,93.50%,87.00%,84.00%,66.90%,74.81%,66.29%,56.25%,50.00%,69.12%,75.00%,71.00%,64.50%,66.00%,77.78%,64.34%,Salesforce,cc-by-nc-4.0
5,71.71%,GPT-4o-2024-11-20 (FC),https://openai.com/index/hello-gpt-4o/,8.38,1.13,1.26,2.44,86.81%,77.25%,92.50%,92.50%,85.00%,78.85%,81.01%,78.54%,81.25%,79.17%,50.00%,61.00%,45.50%,35.50%,58.00%,83.33%,81.31%,OpenAI,Proprietary
6,70.42%,GPT-4o-2024-11-20 (Prompt),https://openai.com/index/hello-gpt-4o/,14.04,0.89,1.24,1.74,87.67%,77.17%,95.00%,93.50%,85.00%,79.88%,84.50%,79.30%,87.50%,70.83%,43.00%,61.50%,20.50%,35.50%,54.50%,72.22%,85.36%,OpenAI,Proprietary
7,70.32%,GPT-4.5-Preview-2025-02-27 (FC),https://openai.com/index/introducing-gpt-4-5/,243.14,3.17,5.63,7.11,86.12%,75.50%,91.50%,92.50%,85.00%,79.34%,80.23%,78.63%,68.75%,70.83%,45.38%,58.00%,37.50%,43.00%,43.00%,66.67%,83.64%,OpenAI,Proprietary
8,69.25%,Qwen3-32B (FC),https://huggingface.co/Qwen/Qwen3-32B,N/A,N/A,N/A,N/A,88.90%,76.08%,96.50%,93.00%,90.00%,77.83%,84.11%,80.72%,68.75%,62.50%,43.12%,54.50%,41.00%,30.50%,46.50%,72.22%,75.79%,Qwen,apache-2.0
9,68.89%,GPT-4.1-2025-04-14 (FC),https://openai.com/index/gpt-4-1/,6.69,1.5,2.23,3.6,85.42%,74.17%,90.50%,91.00%,86.00%,79.92%,80.23%,78.35%,68.75%,66.67%,40.50%,48.00%,34.00%,35.00%,45.00%,77.78%,85.95%,OpenAI,Proprietary
10,68.73%,ToolACE-2-8B (FC),https://huggingface.co/Team-ACE/ToolACE-2-8B,N/A,N/A,N/A,N/A,87.58%,75.33%,92.50%,92.50%,90.00%,80.05%,70.93%,79.01%,81.25%,54.17%,37.00%,49.00%,29.00%,28.00%,42.00%,72.22%,90.11%,Huawei Noah & USTC,Apache-2.0
11,68.44%,DM-Cito-8B (Prompt),https://www.mininglamp.com/,N/A,N/A,N/A,N/A,87.42%,74.67%,93.00%,93.00%,89.00%,80.72%,84.50%,78.25%,81.25%,66.67%,37.25%,47.00%,41.50%,33.50%,27.00%,66.67%,85.15%,Mininglamp,Proprietary
12,68.02%,GPT-4.1-2025-04-14 (Prompt),https://openai.com/index/gpt-4-1/,11.86,1.01,1.4,2.31,88.75%,80.50%,94.00%,93.00%,87.50%,78.32%,85.66%,76.54%,93.75%,75.00%,37.00%,58.50%,2.50%,37.00%,50.00%,88.89%,83.32%,OpenAI,Proprietary
13,68.01%,Qwen3-14B (FC),https://huggingface.co/Qwen/Qwen3-14B,N/A,N/A,N/A,N/A,88.17%,76.17%,95.00%,92.50%,89.00%,79.96%,86.05%,80.82%,68.75%,83.33%,36.62%,48.00%,33.00%,33.50%,32.00%,72.22%,81.01%,Qwen,apache-2.0
14,67.94%,Qwen3-235B-A22B (FC),https://huggingface.co/Qwen/Qwen3-235B-A22B,N/A,N/A,N/A,N/A,87.90%,75.08%,97.00%,90.00%,89.50%,77.03%,87.21%,79.30%,81.25%,83.33%,40.12%,49.00%,41.00%,29.50%,41.00%,83.33%,76.32%,Qwen,apache-2.0
15,67.79%,watt-tool-8B (FC),https://huggingface.co/watt-ai/watt-tool-8B/,N/A,N/A,N/A,N/A,86.56%,76.75%,95.00%,94.00%,80.50%,76.50%,76.74%,77.49%,87.50%,70.83%,39.38%,48.00%,41.50%,27.50%,40.50%,83.33%,83.15%,Watt AI Lab,Apache-2.0
16,67.79%,o1-2024-12-17 (Prompt),https://openai.com/o1/,104.46,5.32,4.32,13.06,85.67%,72.67%,93.50%,91.50%,85.00%,80.63%,82.95%,76.54%,81.25%,75.00%,36.12%,51.00%,0.50%,48.50%,44.50%,72.22%,87.78%,OpenAI,Proprietary
17,67.49%,BitAgent-8B,https://huggingface.co/BitAgent/BitAgent-8B/,N/A,N/A,N/A,N/A,86.92%,76.17%,95.00%,94.00%,82.50%,76.14%,77.91%,77.40%,87.50%,70.83%,38.62%,48.50%,40.00%,26.50%,39.50%,83.33%,82.38%,Bittensor,Apache-2.0
18,66.34%,Qwen3-8B (FC),https://huggingface.co/Qwen/Qwen3-8B,N/A,N/A,N/A,N/A,88.81%,76.75%,95.50%,94.50%,88.50%,78.54%,84.88%,79.39%,62.50%,75.00%,33.00%,38.50%,32.00%,31.50%,30.00%,77.78%,79.08%,Qwen,apache-2.0
19,65.72%,Qwen3-30B-A3B (FC),https://huggingface.co/Qwen/Qwen3-30B-A3B,N/A,N/A,N/A,N/A,89.38%,76.50%,97.00%,93.50%,90.50%,81.34%,86.82%,81.20%,81.25%,87.50%,27.50%,34.00%,22.00%,24.00%,30.00%,72.22%,82.05%,Qwen,apache-2.0
20,65.18%,GPT-4.1-mini-2025-04-14 (FC),https://openai.com/index/gpt-4-1/,1.33,1.61,6.35,3.1,86.81%,73.75%,93.50%,93.00%,87.00%,78.05%,80.62%,77.78%,75.00%,66.67%,31.13%,40.00%,22.00%,25.00%,37.50%,72.22%,81.35%,OpenAI,Proprietary
21,64.81%,xLAM-2-3b-fc-r (FC),https://huggingface.co/Salesforce/xLAM-2-3b-fc-r,N/A,N/A,N/A,N/A,82.75%,74.50%,91.00%,87.00%,78.50%,58.91%,74.42%,62.11%,68.75%,50.00%,56.25%,69.50%,56.00%,54.00%,45.50%,88.89%,57.71%,Salesforce,cc-by-nc-4.0
22,64.71%,DeepSeek-V3-0324 (FC),https://api-docs.deepseek.com/news/news250325,N/A,N/A,N/A,N/A,88.54%,75.67%,94.00%,92.00%,92.50%,77.34%,86.05%,79.49%,75.00%,62.50%,29.87%,41.00%,21.00%,23.00%,34.50%,83.33%,76.49%,DeepSeek,DeepSeek License
23,64.61%,o3-mini-2025-01-31 (Prompt),https://openai.com/index/openai-o3-mini/,8.26,4.88,8.75,11.72,86.15%,75.08%,92.50%,90.50%,86.50%,79.08%,82.95%,77.11%,87.50%,70.83%,28.75%,32.00%,26.00%,23.00%,34.00%,72.22%,82.96%,OpenAI,Proprietary
24,64.37%,Command A (FC),https://cohere.com/blog/command-a,7.17,1.79,4.47,3.49,86.65%,74.08%,93.00%,93.50%,86.00%,80.45%,84.50%,77.68%,81.25%,62.50%,25.50%,35.00%,24.50%,21.50%,21.00%,72.22%,86.19%,Cohere,CC-BY-NC 4.0 License (w/ Acceptable Use Addendum)
25,64.14%,CoALM-405B,https://huggingface.co/uiuc-convai/CoALM-405B,N/A,N/A,N/A,N/A,90.58%,82.33%,97.50%,95.00%,87.50%,74.50%,84.88%,83.00%,87.50%,70.83%,28.88%,37.50%,29.00%,18.50%,30.50%,100.00%,71.79%,UIUC + Oumi,Meta Llama 3 Community
26,63.98%,GPT-4o-mini-2024-07-18 (FC),https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/,0.52,1.49,9.99,2.99,85.21%,74.83%,92.00%,90.00%,84.00%,74.41%,78.68%,76.16%,87.50%,70.83%,33.62%,47.00%,19.00%,28.50%,40.00%,83.33%,74.75%,OpenAI,Proprietary
27,63.79%,DeepSeek-R1-0528 (FC),https://api-docs.deepseek.com/news/news250528,N/A,N/A,N/A,N/A,75.46%,55.33%,93.50%,84.00%,69.00%,77.30%,85.66%,79.68%,87.50%,54.17%,38.88%,49.50%,31.50%,32.00%,42.50%,77.78%,73.31%,DeepSeek,MIT
28,63.39%,Qwen3-4B (FC),https://huggingface.co/Qwen/Qwen3-4B,N/A,N/A,N/A,N/A,88.58%,75.33%,96.50%,92.00%,90.50%,78.72%,87.60%,79.96%,75.00%,83.33%,24.25%,33.50%,20.50%,20.50%,22.50%,72.22%,78.19%,Qwen,apache-2.0
29,63.31%,Grok-3-beta (FC),https://docs.x.ai/docs/models,15.84,4.79,8.84,17.77,63.96%,77.33%,94.00%,0.00%,84.50%,77.25%,80.62%,77.40%,56.25%,75.00%,45.50%,54.00%,39.00%,42.00%,47.00%,66.67%,78.38%,xAI,Proprietary
30,62.75%,xiaoming-14B (Prompt),https://www.mininglamp.com/,N/A,N/A,N/A,N/A,88.17%,74.67%,96.00%,93.00%,89.00%,74.77%,81.78%,75.88%,62.50%,58.33%,27.12%,39.00%,29.00%,19.50%,21.00%,77.78%,75.58%,Mininglamp,Proprietary
31,62.56%,GPT-4.5-Preview-2025-02-27 (Prompt),https://openai.com/index/introducing-gpt-4-5/,472.86,2.5,14.55,4.32,89.98%,80.42%,95.50%,92.50%,91.50%,82.99%,87.98%,81.86%,93.75%,66.67%,15.88%,20.50%,4.50%,16.50%,22.00%,66.67%,83.81%,OpenAI,Proprietary
32,62.32%,Functionary-Medium-v3.1 (FC),https://huggingface.co/meetkai/functionary-medium-v3.1,N/A,N/A,N/A,N/A,89.88%,76.00%,97.00%,95.00%,91.50%,76.63%,81.78%,83.29%,68.75%,70.83%,21.50%,32.00%,21.00%,26.50%,6.50%,72.22%,76.08%,MeetKai,MIT
33,62.25%,Hammer2.1-7b (FC),https://huggingface.co/MadeAgents/Hammer2.1-7b,N/A,N/A,N/A,N/A,88.65%,78.08%,95.00%,93.50%,88.00%,75.11%,76.74%,77.40%,81.25%,70.83%,23.62%,36.00%,25.50%,19.00%,14.00%,82.35%,78.59%,MadeAgents,cc-by-nc-4.0
34,61.82%,GPT-4o-mini-2024-07-18 (Prompt),https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/,0.86,1.31,6.98,2.39,86.77%,80.08%,90.50%,89.50%,87.00%,76.50%,81.40%,76.73%,93.75%,79.17%,22.12%,33.50%,12.00%,17.00%,26.00%,83.33%,80.67%,OpenAI,Proprietary
35,61.22%,Gemini-2.0-Flash-001 (FC),https://deepmind.google/technologies/gemini/flash/,0.63,0.7,1.26,0.98,84.90%,71.58%,88.50%,90.50%,89.00%,79.12%,74.42%,68.28%,81.25%,66.67%,17.88%,20.00%,12.50%,16.50%,22.50%,50.00%,94.04%,Google,Proprietary
36,60.82%,Amazon-Nova-Pro-v1:0 (FC),https://aws.amazon.com/cn/ai/generative-ai/nova/,5.35,2.64,2.59,5.43,84.46%,68.83%,92.50%,92.00%,84.50%,74.32%,80.23%,77.49%,81.25%,58.33%,26.00%,37.50%,19.00%,21.50%,26.00%,77.78%,70.98%,Amazon,Proprietary
37,60.60%,GPT-4.1-mini-2025-04-14 (Prompt),https://openai.com/index/gpt-4-1/,2.22,1.15,6.3,1.87,81.50%,75.50%,88.00%,84.50%,78.00%,71.17%,79.46%,70.94%,75.00%,70.83%,29.75%,44.00%,3.00%,32.00%,40.00%,83.33%,73.52%,OpenAI,Proprietary
38,60.05%,Gemini-2.0-Flash-001 (Prompt),https://deepmind.google/technologies/gemini/flash/,0.77,0.89,1.14,1.23,84.48%,74.92%,89.50%,86.50%,87.00%,81.39%,75.58%,73.12%,81.25%,83.33%,12.75%,15.50%,9.00%,12.50%,14.00%,55.56%,92.75%,Google,Proprietary
39,59.94%,Gemini-2.5-Pro-Preview-05-06 (FC),https://deepmind.google/technologies/gemini/pro/,0.0,5.02,9.87,14.53,65.35%,66.42%,86.00%,69.00%,40.00%,74.59%,77.91%,62.20%,68.75%,62.50%,34.62%,39.50%,29.50%,31.50%,38.00%,33.33%,90.67%,Google,Proprietary
40,59.56%,Hammer2.1-3b (FC),https://huggingface.co/MadeAgents/Hammer2.1-3b,N/A,N/A,N/A,N/A,86.85%,81.42%,95.00%,89.50%,81.50%,74.04%,73.26%,73.31%,62.50%,66.67%,17.50%,28.00%,17.50%,14.50%,10.00%,82.35%,81.87%,MadeAgents,qwen-research
41,59.11%,Gemma-3-27b-it (Prompt),https://blog.google/technology/developers/gemma-3/,N/A,N/A,N/A,N/A,89.02%,78.58%,94.50%,91.50%,91.50%,76.37%,86.43%,78.73%,93.75%,66.67%,14.50%,18.00%,12.00%,11.50%,16.50%,83.33%,73.33%,Google,gemma-terms-of-use
42,58.74%,Gemini-2.0-Flash-Thinking-Exp-01-21 (Prompt),https://deepmind.google/technologies/gemini/flash-thinking/,0.0,4.92,21.89,8.35,87.40%,75.58%,95.00%,92.00%,87.00%,75.97%,86.43%,81.01%,87.50%,87.50%,14.50%,17.50%,19.00%,10.00%,11.50%,77.78%,72.75%,Google,Proprietary
43,58.58%,Claude-3.7-Sonnet-20250219 (FC),https://www.anthropic.com/news/claude-3-7-sonnet,2.52,2.78,1.74,4.5,41.29%,71.67%,93.50%,0.00%,0.00%,78.41%,82.56%,76.45%,0.00%,0.00%,48.38%,54.50%,45.00%,45.50%,48.50%,72.22%,81.40%,Anthropic,Proprietary
44,58.30%,QwQ-32B (FC),https://huggingface.co/Qwen/QwQ-32B,N/A,N/A,N/A,N/A,87.33%,74.83%,97.00%,90.00%,87.50%,75.61%,80.62%,79.20%,81.25%,83.33%,14.50%,20.50%,10.00%,12.00%,15.50%,83.33%,71.99%,Qwen,apache-2.0
45,58.23%,xLAM-2-1b-fc-r (FC),https://huggingface.co/Salesforce/xLAM-2-1b-fc-r,N/A,N/A,N/A,N/A,76.17%,69.67%,89.50%,79.00%,66.50%,59.17%,74.03%,60.68%,50.00%,33.33%,42.75%,60.50%,37.00%,34.50%,39.00%,94.44%,56.17%,Salesforce,cc-by-nc-4.0
46,58.03%,mistral-large-2411 (FC),https://docs.mistral.ai/guides/model-selection/,7.06,2.46,4.49,6.03,83.38%,70.50%,94.00%,88.00%,81.00%,76.06%,87.21%,79.01%,81.25%,75.00%,17.75%,29.00%,17.50%,19.50%,5.00%,83.33%,68.48%,Mistral AI,Proprietary
47,57.74%,Mistral-small-2503 (FC),https://docs.mistral.ai/guides/model-selection/,0.42,1.16,5.44,2.57,70.29%,34.17%,93.50%,80.00%,73.50%,80.14%,65.89%,76.45%,56.25%,58.33%,18.75%,29.00%,9.00%,16.50%,20.50%,72.22%,90.16%,Mistral AI,Proprietary
48,56.95%,Claude-3-Opus-20240229 (FC),https://www.anthropic.com/news/claude-3-family,19.75,9.41,10.03,17.09,57.92%,67.17%,93.00%,39.50%,32.00%,78.05%,79.07%,75.78%,31.25%,37.50%,30.38%,42.00%,14.00%,33.50%,32.00%,61.11%,81.59%,Anthropic,Proprietary
49,56.88%,Claude-3.5-Sonnet-20241022 (FC),https://www.anthropic.com/news/3-5-models-and-computer-use,2.49,3.07,5.67,4.78,45.44%,78.75%,94.50%,3.50%,5.00%,78.94%,84.11%,81.96%,25.00%,20.83%,41.00%,55.00%,19.00%,42.50%,47.50%,77.78%,74.04%,Anthropic,Proprietary
50,56.84%,palmyra-x-004 (FC),https://writer.com/engineering/actions-with-palmyra-x-004/,26.53,7.41,48.64,20.25,86.38%,72.00%,94.00%,93.00%,86.50%,77.21%,76.36%,68.38%,50.00%,62.50%,5.88%,9.00%,0.50%,7.50%,6.50%,72.22%,90.33%,Writer,Proprietary
51,56.68%,Llama-3.1-Nemotron-Ultra-253B-v1 (FC),https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1,N/A,N/A,N/A,N/A,90.65%,81.58%,97.00%,93.00%,91.00%,80.76%,86.43%,82.24%,75.00%,70.83%,0.00%,0.00%,0.00%,0.00%,0.00%,83.33%,80.71%,NVIDIA,nvidia-open-model-license
52,56.39%,Gemini-2.0-Flash-Lite-001 (Prompt),https://deepmind.google/technologies/gemini/flash-lite/,0.32,0.68,0.42,0.98,84.21%,78.83%,93.50%,86.50%,78.00%,79.61%,79.07%,74.74%,75.00%,58.33%,3.88%,4.50%,2.50%,4.50%,4.00%,66.67%,89.09%,Google,Proprietary
53,56.22%,Gemini-2.0-Flash-Lite-001 (FC),https://deepmind.google/technologies/gemini/flash-lite/,0.27,0.63,1.1,0.85,83.73%,63.92%,89.50%,93.00%,88.50%,77.25%,71.32%,67.05%,81.25%,70.83%,5.50%,7.00%,1.00%,6.50%,7.50%,50.00%,93.21%,Google,Proprietary
54,56.10%,Grok-3-beta (Prompt),https://docs.x.ai/docs/models,23.2,2.35,6.67,6.56,89.50%,80.50%,96.00%,92.50%,89.00%,69.61%,86.05%,81.20%,75.00%,70.83%,13.00%,17.00%,16.50%,12.00%,6.50%,100.00%,60.32%,xAI,Proprietary
55,56.08%,o1-2024-12-17 (FC),https://openai.com/o1/,69.27,4.82,5.07,13.59,40.23%,67.92%,93.00%,0.00%,0.00%,78.01%,81.78%,79.01%,0.00%,0.00%,41.12%,53.00%,38.00%,30.50%,43.00%,72.22%,81.97%,OpenAI,Proprietary
56,56.01%,Functionary-Small-v3.1 (FC),https://huggingface.co/meetkai/functionary-small-v3.1,N/A,N/A,N/A,N/A,86.75%,74.00%,94.50%,90.50%,88.00%,73.75%,79.84%,78.16%,81.25%,62.50%,9.88%,17.00%,2.50%,14.00%,6.00%,77.78%,70.89%,MeetKai,MIT
57,55.93%,Amazon-Nova-Lite-v1:0 (FC),https://aws.amazon.com/cn/ai/generative-ai/nova/,0.42,1.65,2.6,2.52,78.44%,69.75%,94.00%,84.00%,66.00%,71.61%,72.87%,70.09%,75.00%,66.67%,17.50%,28.00%,5.50%,17.50%,19.00%,66.67%,76.41%,Amazon,Proprietary
58,54.94%,Hammer2.1-1.5b (FC),https://huggingface.co/MadeAgents/Hammer2.1-1.5b,N/A,N/A,N/A,N/A,82.79%,74.67%,92.00%,84.50%,80.00%,70.64%,71.32%,69.80%,50.00%,62.50%,10.62%,15.00%,12.50%,9.00%,6.00%,77.78%,79.27%,MadeAgents,cc-by-nc-4.0
59,54.70%,Qwen3-1.7B (FC),https://huggingface.co/Qwen/Qwen3-1.7B,N/A,N/A,N/A,N/A,83.27%,71.08%,93.00%,87.50%,81.50%,72.95%,75.58%,72.65%,75.00%,75.00%,8.62%,12.00%,5.50%,9.50%,7.50%,61.11%,76.13%,Qwen,apache-2.0
60,54.52%,ling-lite-v1.5 (Prompt),https://huggingface.co/inclusionAI/Ling-lite-1.5,N/A,N/A,N/A,N/A,87.62%,79.00%,95.00%,91.00%,85.50%,71.26%,74.42%,71.13%,75.00%,70.83%,5.12%,9.00%,4.50%,5.00%,2.00%,88.89%,77.74%,Ling,MIT
61,54.52%,CoALM-8B,https://huggingface.co/uiuc-convai/CoALM-8B,N/A,N/A,N/A,N/A,85.25%,68.50%,95.00%,89.00%,88.50%,72.95%,71.71%,66.67%,56.25%,54.17%,4.50%,5.00%,5.00%,4.00%,4.00%,83.33%,85.49%,UIUC + Oumi,Meta Llama 3 Community
62,54.49%,CoALM-70B,https://huggingface.co/uiuc-convai/CoALM-70B,N/A,N/A,N/A,N/A,83.08%,68.83%,92.50%,88.00%,83.00%,71.92%,70.16%,65.34%,68.75%,58.33%,7.00%,7.50%,11.00%,4.50%,5.00%,66.67%,85.63%,UIUC + Oumi,Meta Llama 3 Community
63,54.22%,GoGoAgent,https://gogoagent.ai,N/A,N/A,N/A,N/A,86.23%,75.42%,93.00%,92.00%,84.50%,74.01%,72.87%,75.40%,68.75%,66.67%,1.00%,1.50%,2.00%,0.50%,0.00%,77.78%,83.12%,BitAgent,Proprietary
64,53.69%,claude-3.5-haiku-20241022 (Prompt),https://www.anthropic.com/news/3-5-models-and-computer-use,0.47,1.84,5.27,3.58,83.19%,76.25%,93.00%,84.00%,79.50%,70.77%,84.88%,75.02%,87.50%,54.17%,9.75%,16.00%,0.50%,8.00%,14.50%,77.78%,65.78%,Anthropic,Proprietary
65,53.57%,Llama-3.1-70B-Instruct (Prompt),https://llama.meta.com/llama3,N/A,N/A,N/A,N/A,89.98%,77.92%,96.00%,94.50%,91.50%,62.24%,78.29%,76.16%,87.50%,66.67%,12.50%,17.00%,13.00%,10.50%,9.50%,100.00%,54.78%,Meta,Meta Llama 3 Community
66,53.05%,Grok-3-mini-beta (FC),https://docs.x.ai/docs/models,1.44,5.16,5.7,9.24,40.02%,64.58%,95.50%,0.00%,0.00%,80.72%,82.17%,77.49%,0.00%,0.00%,29.25%,29.00%,30.00%,30.00%,28.00%,55.56%,87.08%,xAI,Proprietary
67,52.98%,GPT-4.1-nano-2025-04-14 (FC),https://openai.com/index/gpt-4-1/,0.36,0.93,4.96,1.51,76.65%,63.58%,84.00%,86.50%,72.50%,64.33%,65.50%,73.41%,87.50%,79.17%,19.88%,32.00%,8.00%,15.00%,24.50%,94.44%,59.39%,OpenAI,Proprietary
68,52.86%,Llama-4-Maverick-17B-128E-Instruct-FP8 (FC),https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8,N/A,N/A,N/A,N/A,86.65%,76.08%,93.50%,89.50%,87.50%,58.55%,83.33%,74.64%,81.25%,62.50%,16.38%,22.00%,18.00%,12.50%,13.00%,100.00%,51.20%,Meta,Meta Llama 4 Community
69,52.50%,claude-3.5-haiku-20241022 (FC),https://www.anthropic.com/news/3-5-models-and-computer-use,0.81,2.83,1.96,5.2,40.62%,68.00%,92.00%,2.50%,0.00%,72.37%,82.95%,78.35%,18.75%,0.00%,40.12%,55.00%,26.50%,35.00%,44.00%,83.33%,63.68%,Anthropic,Proprietary
70,52.41%,Sky-T1-32B-Preview (Prompt),https://huggingface.co/NovaSky-AI/Sky-T1-32B-Preview,N/A,N/A,N/A,N/A,88.67%,78.67%,94.50%,94.00%,87.50%,68.00%,77.52%,77.11%,62.50%,62.50%,4.63%,8.00%,3.50%,3.00%,4.00%,94.12%,61.21%,NovaSky-AI,apache-2.0
71,52.35%,FireFunction-v2 (FC),https://huggingface.co/fireworks-ai/firefunction-v2,N/A,N/A,N/A,N/A,88.46%,80.33%,94.00%,91.50%,88.00%,65.66%,79.07%,78.35%,56.25%,70.83%,8.62%,13.50%,7.00%,11.00%,3.00%,94.44%,53.02%,Fireworks,Apache 2.0
72,52.24%,Llama-3.3-70B-Instruct (FC),https://llama.meta.com/llama3,N/A,N/A,N/A,N/A,85.08%,74.33%,94.50%,84.50%,87.00%,62.67%,81.78%,77.11%,93.75%,66.67%,14.50%,15.00%,19.00%,10.50%,13.50%,100.00%,47.97%,Meta,Meta Llama 3 Community
73,51.83%,Amazon-Nova-Micro-v1:0 (FC),https://aws.amazon.com/cn/ai/generative-ai/nova/,0.24,1.49,2.01,2.34,71.12%,63.50%,88.00%,77.50%,55.50%,67.04%,65.89%,64.20%,62.50%,45.83%,16.12%,24.50%,5.50%,14.00%,20.50%,72.22%,74.20%,Amazon,Proprietary
74,51.70%,Command R7B (FC),https://cohere.com/blog/command-r7b,0.1,1.36,4.93,2.48,81.67%,68.17%,91.50%,85.50%,81.50%,69.17%,63.18%,58.69%,56.25%,62.50%,5.00%,6.50%,1.50%,6.50%,5.50%,55.56%,81.02%,Cohere,cc-by-nc-4.0
75,51.54%,Ministral-8B-Instruct-2410 (FC),https://huggingface.co/mistralai/Ministral-8B-Instruct-2410,N/A,N/A,N/A,N/A,83.83%,71.83%,91.50%,84.50%,87.50%,64.93%,75.58%,72.27%,62.50%,62.50%,11.12%,21.00%,8.00%,10.00%,5.50%,70.59%,55.28%,Mistral AI,Mistral AI Research License
76,51.48%,Open-Mistral-Nemo-2407 (FC),https://mistral.ai/news/mistral-nemo/,0.61,1.55,1.98,3.43,82.10%,64.42%,93.50%,85.50%,85.00%,65.97%,77.13%,69.61%,75.00%,70.83%,9.12%,15.00%,3.50%,9.00%,9.00%,66.67%,63.19%,Mistral AI,Proprietary
77,51.26%,o3-mini-2025-01-31 (FC),https://openai.com/index/openai-o3-mini/,4.83,4.04,5.46,10.17,42.12%,75.50%,93.00%,0.00%,0.00%,77.30%,81.40%,78.63%,0.00%,0.00%,26.12%,33.00%,17.50%,24.00%,30.00%,77.78%,80.67%,OpenAI,Proprietary
78,50.63%,Gemma-3-12b-it (Prompt),https://blog.google/technology/developers/gemma-3/,N/A,N/A,N/A,N/A,83.83%,77.33%,95.00%,90.00%,73.00%,67.13%,84.88%,70.85%,87.50%,62.50%,4.62%,8.00%,3.50%,2.50%,4.50%,88.89%,61.11%,Google,gemma-terms-of-use
79,50.51%,Claude-3.5-Sonnet-20241022 (Prompt),https://www.anthropic.com/news/3-5-models-and-computer-use,1.19,1.81,1.36,3.31,72.48%,81.42%,92.00%,70.50%,46.00%,71.97%,86.82%,80.06%,81.25%,45.83%,7.50%,9.00%,5.50%,5.00%,10.50%,77.78%,64.40%,Anthropic,Proprietary
80,50.30%,MiniCPM3-4B-FC (FC),https://huggingface.co/openbmb/MiniCPM3-4B,N/A,N/A,N/A,N/A,80.83%,69.83%,91.50%,82.50%,79.50%,70.01%,74.81%,63.91%,43.75%,62.50%,2.62%,5.00%,1.00%,3.00%,1.50%,72.22%,72.22%,openbmb,Apache-2.0
81,50.02%,Claude-3.7-Sonnet-20250219 (Prompt),https://www.anthropic.com/news/claude-3-7-sonnet,1.07,1.7,1.52,2.85,88.00%,79.00%,95.00%,89.50%,88.50%,65.79%,87.98%,83.57%,68.75%,50.00%,0.62%,0.00%,0.00%,1.00%,1.50%,100.00%,52.00%,Anthropic,Proprietary
82,49.68%,Claude-3-Opus-20240229 (Prompt),https://www.anthropic.com/news/claude-3-family,10.35,4.57,5.73,10.62,85.31%,79.75%,95.00%,85.50%,81.00%,66.99%,85.27%,79.11%,68.75%,54.17%,7.13%,11.50%,2.50%,6.00%,8.50%,83.33%,40.25%,Anthropic,Proprietary
83,49.57%,Llama-3.1-8B-Instruct (Prompt),https://llama.meta.com/llama3,N/A,N/A,N/A,N/A,84.21%,72.83%,93.50%,87.00%,83.50%,61.08%,74.03%,73.31%,56.25%,54.17%,9.25%,12.00%,10.00%,7.00%,8.00%,77.78%,48.82%,Meta,Meta Llama 3 Community
84,49.41%,GPT-4.1-nano-2025-04-14 (Prompt),https://openai.com/index/gpt-4-1/,0.56,0.6,0.73,1.08,74.31%,70.75%,75.00%,78.50%,73.00%,65.35%,65.12%,58.97%,81.25%,79.17%,7.25%,10.50%,1.00%,9.50%,8.00%,72.22%,76.53%,OpenAI,Proprietary
85,48.92%,Granite-20b-FunctionCalling (FC),https://huggingface.co/ibm-granite/granite-20b-functioncalling,N/A,N/A,N/A,N/A,82.46%,72.83%,91.50%,84.00%,81.50%,59.66%,68.22%,56.32%,43.75%,58.33%,3.38%,6.00%,1.50%,4.50%,1.50%,88.89%,74.82%,IBM,Apache-2.0
86,47.98%,Command-R-Plus (FC),https://txt.cohere.com/command-r-plus-microsoft-azure,7.97,2.58,9.22,3.84,77.02%,72.08%,89.50%,82.50%,64.00%,59.00%,70.54%,58.78%,62.50%,45.83%,13.25%,17.00%,10.00%,9.00%,17.00%,72.22%,53.16%,Cohere For AI,cc-by-nc-4.0
87,47.33%,Qwen3-0.6B (FC),https://huggingface.co/Qwen/Qwen3-0.6B,N/A,N/A,N/A,N/A,71.81%,62.25%,88.00%,69.00%,68.00%,65.66%,65.89%,54.42%,37.50%,54.17%,1.88%,4.00%,0.00%,2.50%,1.00%,66.67%,82.41%,Qwen,apache-2.0
88,46.97%,Phi-4 (Prompt),https://huggingface.co/microsoft/phi-4,N/A,N/A,N/A,N/A,67.83%,66.83%,85.00%,62.50%,57.00%,66.28%,62.79%,53.94%,75.00%,66.67%,3.12%,7.00%,0.00%,2.00%,3.50%,66.67%,84.05%,Microsoft,MIT
89,45.86%,Llama-3.2-3B-Instruct (FC),https://llama.meta.com/llama3,N/A,N/A,N/A,N/A,80.56%,73.75%,92.00%,80.50%,76.00%,55.80%,63.95%,64.86%,12.50%,45.83%,5.25%,8.50%,2.50%,4.50%,5.50%,88.89%,51.69%,Meta,Meta Llama 3 Community
90,45.41%,Llama-4-Scout-17B-16E-Instruct (FC),https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct,N/A,N/A,N/A,N/A,83.48%,79.42%,95.00%,81.50%,78.00%,57.97%,77.91%,74.36%,68.75%,62.50%,1.88%,2.00%,2.50%,1.00%,2.00%,100.00%,39.66%,Meta,Meta Llama 4 Community
91,45.35%,Hammer2.1-0.5b (FC),https://huggingface.co/MadeAgents/Hammer2.1-0.5b,N/A,N/A,N/A,N/A,69.12%,68.00%,83.00%,71.50%,54.00%,62.91%,60.08%,58.02%,50.00%,45.83%,2.25%,4.00%,0.50%,3.00%,1.50%,77.78%,73.94%,MadeAgents,cc-by-nc-4.0
92,45.13%,Falcon3-10B-Instruct (FC),https://huggingface.co/tiiuae/Falcon3-10B-Instruct,N/A,N/A,N/A,N/A,84.62%,70.50%,93.50%,87.50%,87.00%,54.11%,76.36%,76.16%,50.00%,41.67%,5.00%,6.00%,5.00%,4.50%,4.50%,94.44%,31.89%,TII UAE,falcon-llm-license
93,43.59%,Falcon3-7B-Instruct (FC),https://huggingface.co/tiiuae/Falcon3-7B-Instruct,N/A,N/A,N/A,N/A,82.31%,64.75%,89.50%,86.50%,88.50%,54.86%,74.03%,66.48%,75.00%,62.50%,3.38%,3.50%,3.50%,3.50%,3.00%,88.89%,33.73%,TII UAE,falcon-llm-license
94,40.92%,MiniCPM3-4B (Prompt),https://huggingface.co/openbmb/MiniCPM3-4B,N/A,N/A,N/A,N/A,65.88%,63.50%,72.50%,65.50%,62.00%,54.46%,46.51%,34.76%,43.75%,41.67%,2.00%,3.00%,3.50%,1.00%,0.50%,50.00%,74.43%,openbmb,Apache-2.0
95,40.81%,Bielik-11B-v2.3-Instruct (Prompt),https://huggingface.co/speakleash/Bielik-11B-v2.3-Instruct,N/A,N/A,N/A,N/A,65.04%,71.17%,93.50%,46.00%,49.50%,58.91%,72.87%,69.33%,43.75%,54.17%,3.75%,7.00%,0.50%,3.00%,4.50%,77.78%,40.58%,SpeakLeash & ACK Cyfronet AGH,Apache 2.0
96,39.70%,Open-Mistral-Nemo-2407 (Prompt),https://mistral.ai/news/mistral-nemo/,0.92,1.65,10.13,3.27,86.12%,77.00%,93.50%,89.50%,84.50%,49.04%,77.91%,74.45%,87.50%,66.67%,0.25%,0.50%,0.00%,0.00%,0.50%,88.89%,6.43%,Mistral AI,Proprietary
97,39.63%,Gemma-3-4b-it (Prompt),https://blog.google/technology/developers/gemma-3/,N/A,N/A,N/A,N/A,63.33%,64.33%,91.50%,56.50%,41.00%,59.17%,72.87%,62.77%,37.50%,29.17%,0.12%,0.00%,0.00%,0.50%,0.00%,77.78%,48.14%,Google,gemma-terms-of-use
98,39.21%,DBRX-Instruct (Prompt),https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm,8.76,3.81,8.36,11.37,61.25%,73.50%,92.00%,42.50%,37.00%,60.28%,78.29%,73.03%,75.00%,41.67%,0.00%,0.00%,0.00%,0.00%,0.00%,94.44%,40.50%,Databricks,Databricks Open Model
99,39.10%,GLM-4-9b-Chat (FC),https://huggingface.co/THUDM/glm-4-9b-chat,N/A,N/A,N/A,N/A,36.67%,65.17%,81.50%,0.00%,0.00%,66.81%,72.48%,64.39%,0.00%,0.00%,3.50%,3.50%,4.00%,2.50%,4.00%,66.67%,79.71%,THUDM,glm-4
100,35.99%,Nexusflow-Raven-v2 (FC),https://huggingface.co/Nexusflow/NexusRaven-V2-13B,N/A,N/A,N/A,N/A,45.88%,57.50%,53.00%,34.00%,39.00%,54.20%,41.47%,38.65%,56.25%,41.67%,1.00%,1.50%,0.50%,1.00%,1.00%,61.11%,78.53%,Nexusflow,Apache 2.0
101,32.47%,Falcon3-3B-Instruct (FC),https://huggingface.co/tiiuae/Falcon3-3B-Instruct,N/A,N/A,N/A,N/A,53.25%,58.00%,69.00%,61.00%,25.00%,47.40%,55.43%,56.32%,31.25%,37.50%,0.50%,0.50%,0.50%,0.00%,1.00%,77.78%,34.47%,TII UAE,falcon-llm-license
102,31.27%,Phi-4-mini-instruct (Prompt),https://huggingface.co/microsoft/Phi-4-mini-instruct,N/A,N/A,N/A,N/A,41.85%,67.92%,69.00%,16.00%,14.50%,50.62%,55.04%,59.45%,68.75%,66.67%,2.88%,3.50%,3.50%,0.50%,4.00%,82.35%,35.79%,Microsoft,MIT
103,28.16%,Phi-4-mini-instruct (FC),https://huggingface.co/microsoft/Phi-4-mini-instruct,N/A,N/A,N/A,N/A,9.50%,38.00%,0.00%,0.00%,0.00%,61.47%,40.31%,50.33%,0.00%,29.17%,0.00%,0.00%,0.00%,0.00%,0.00%,64.71%,79.98%,Microsoft,MIT
104,27.12%,Llama-3.1-70B-Instruct (FC),https://llama.meta.com/llama3,N/A,N/A,N/A,N/A,25.29%,49.17%,24.50%,12.50%,15.00%,45.00%,52.33%,52.61%,31.25%,25.00%,4.88%,7.00%,4.00%,4.50%,4.00%,100.00%,44.85%,Meta,Meta Llama 3 Community
105,25.92%,Llama-3.1-8B-Instruct (FC),https://llama.meta.com/llama3,N/A,N/A,N/A,N/A,48.21%,55.83%,54.00%,48.50%,34.50%,33.50%,51.94%,49.00%,37.50%,41.67%,5.38%,5.00%,7.50%,5.00%,4.00%,94.44%,4.86%,Meta,Meta Llama 3 Community
106,21.95%,Llama-3.2-1B-Instruct (FC),https://llama.meta.com/llama3,N/A,N/A,N/A,N/A,28.44%,29.25%,33.50%,36.00%,15.00%,31.36%,31.40%,7.60%,12.50%,4.17%,0.00%,0.00%,0.00%,0.00%,0.00%,38.89%,59.70%,Meta,Meta Llama 3 Community
107,20.83%,ThinkAgent-1B (FC),https://huggingface.co/ThinkAgents/ThinkAgent-1B,N/A,N/A,N/A,N/A,36.29%,29.17%,31.50%,50.00%,34.50%,25.37%,24.42%,16.43%,6.25%,4.17%,0.00%,0.00%,0.00%,0.00%,0.00%,44.44%,38.63%,ThinkAgents,apache-2.0
108,19.67%,Falcon3-1B-Instruct (FC),https://huggingface.co/tiiuae/Falcon3-1B-Instruct,N/A,N/A,N/A,N/A,9.02%,3.58%,6.00%,17.50%,9.00%,32.70%,4.65%,2.37%,0.00%,12.50%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,87.16%,TII UAE,falcon-llm-license
109,16.32%,Gemma-3-1b-it (Prompt),https://blog.google/technology/developers/gemma-3/,N/A,N/A,N/A,N/A,21.50%,43.50%,38.50%,2.00%,2.00%,30.34%,31.01%,10.54%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,0.00%,50.00%,30.92%,Google,gemma-terms-of-use