Rank,Model,Character,Scene,Avg. Win Rate
1,Higgs-Llama-3 70B V2,68.25%,73.68%,70.97%
2,Claude-3.5 Sonnet (2024-06-20),56.68%,63.17%,59.92%
3,Claude-3 Opus (2024-02-29),57.00%,61.68%,59.34%
4,Llama-3.1 405B Instruct (FP8),54.00%,62.81%,58.41%
5,GPT-4-Turbo (1106 Preview),57.93%,56.34%,57.14%
6,Llama-3.1 70B Instruct,53.25%,60.71%,56.98%
7,GPT-4o (2024-05-13),50.00%,50.00%,50.00%
8,MiniMax abab6.5s,51.50%,46.77%,49.14%
9,Yi Large (2024-05-13),55.70%,38.51%,47.10%
10,Higgs-Llama-3 70B V1,47.72%,44.20%,45.96%
11,Qwen2 72B Instruct,48.99%,41.20%,45.10%
12,Llama-3 70B Instruct,45.75%,43.71%,44.73%
13,Llama-3.1 8B Instruct,40.00%,43.61%,41.81%
14,DeepSeek-V2 (2024-06-28),44.19%,37.83%,41.01%
15,Gemini 1.5 Pro,37.50%,33.67%,35.59%
16,Mistral Large (2024-02),34.42%,36.71%,35.56%
17,Character.AI,12.47%,16.04%,14.25%