[whatmodel.sh](../../README.md) / LongCat-Flash-Chat (560B-A27B) # LongCat-Flash-Chat (560B-A27B) Meituan · released 2025-08 · 560B params (27B active) · mit **score 65.4** ±0.9 · #137 of 732 · from 23 benchmark results across 2 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks LongCat-Flash-Chat (560B-A27B) among every model on that task or test, worst to best. 
task            vs the others          rank  counts  tests
agent tasks     ▁▁▃▆█▄▅●▄▄▂▁    #113 of 313     22%      5
coding          ▁▁▁▃▅▇█●▅▄▃▁    #169 of 469     22%      4
chat & writing  ▁▁▁▃▆▄▆█▆●▇▃   #116 of 402*     12%      2
math            ▁▁▃▅█▆▄●▄▂▁▁    #129 of 431    8.0%      3
agent tasks · #113 of 313 · 22% of its score Beats [Grok 4.20 (0309 reasoning)](grok-4-20.md), [o3-pro](o3-pro.md), [Grok 4.3](grok-4-3.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Devstral 2 (2512)](devstral-2.md), [Nova 2 Pro (Preview)](nova-2-pro.md) here, though they rank lower overall. 
test                                result  vs the others in its run   difficulty            counts  why
SWE-bench Verified (maker)           60.4%  ▁▁▁▁▁▁▂▄●▆▁▁ beats 58/138  hard · avg 17%          7.6%  sharp · at its level
τ²-bench Retail (maker)              71.3%  ▅···●·▃····▃ beats 4/6     medium · avg 40%        4.9%  at its level
τ²-bench Telecom (maker)             73.7%  ██··███···█● beats 6/6     very hard · avg 8.8%    4.4%  sharp · at its level · noisy
τ²-bench Airline (maker)             58.0%  ▄·▄··▄···█●▄ beats 5/6     hard · avg 25%          3.2%  at its level
Terminal-Bench (pre-2.0 release) …   39.5%  ▄·▄···▄▄···● beats 4/5     very hard · avg 12%     1.9%  at its level
coding · #169 of 469 · 22% of its score Beats [Qwen3-Coder-Next (80B-A3B)](qwen3-coder-next.md), [Gemini 2.5 Pro](gemini-2-5-pro.md), [Grok 4.1 Fast (Reasoning)](grok-4-1-fast-reasoning.md) here, though they rank higher overall. Trails [QwQ-32B-Preview](qwq-32b-preview.md), [Ministral 3 8B Reasoning 2512](ministral-3-8b-reasoning.md), [Phi-4-reasoning 14B](phi-4-reasoning.md) here, though they rank lower overall. 
test                                result  vs the others in its run   difficulty        counts  why
SWE-bench Verified (maker)           60.4%  ▁▁▁▁▁▁▂▄●▆▁▁ beats 58/138  hard · avg 17%      7.6%  sharp · at its level
LiveCodeBench (version unspecifie…   48.0%  █·······▄▄▄● beats 5/5     hard · avg 21%      6.2%  at its level
MBPP+ (maker)                        79.6%  ▃·····▃●▃··▃ beats 2/6     medium · avg 60%    4.3%  typical
HumanEval+ (maker)                   88.4%  ▄▄●·····▄▄·█ beats 2/6     easy · avg 66%      4.0%  sharp · maxed out
chat & writing · #116 of 402 · 12% of its score Beats [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md), [Step 3.5 Flash (196B-A11B)](step-3-5-flash.md), [GPT-5 mini](gpt-5-mini.md) here, though they rank higher overall. Trails [Nemotron-Cascade-8B](nemotron-cascade-8b.md), [Mistral Large 3 (675B-A41B)](mistral-large-3.md), [Gemini 2.5 Flash](gemini-2-5-flash.md) here, though they rank lower overall. 
test                                result  vs the others in its run    difficulty      counts  why
LMArena (text)                      1400.8  ▁▁▁▂▃▃▃▆▆●█▃ beats 246/385  avg 1260          9.0%  blunt
IFEval (aggregation unspecified) …   89.6%  █··█·█··███● beats 6/6      easy · avg 74%    3.5%  typical
math · #129 of 431 · 8.0% of its score Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [MiniMax-M3](minimax-m3.md), [MiMo-V2.5 (310B-A15B)](mimo-v2-5.md) here, though they rank higher overall. Trails [Olmo 3.1 32B Think](olmo-3-1-32b-think.md), [Qwen3-VL 30B-A3B Thinking](qwen3-vl-30b-a3b-thinking.md), [Granite 4.2 8B](granite-4-2-8b.md) here, though they rank lower overall. 
test               result  vs the others in its run  difficulty           counts  why
AIME 2025 (maker)   61.3%  ▄▄···▄▄··●·█ beats 4/6    hard · avg 16%         4.2%  sharp · at its level · noisy
AIME 2024 (maker)   70.4%  ▁▁▁▂▁▃▁▅●██▃ beats 38/85  hard · avg 30%         2.2%  sharp · at its level · noisy
MATH-500 (maker)    96.4%  ▄·▄··●··▄·▄▄ beats 3/6    very easy · avg 87%    1.6%  maxed out · noisy
other tests (10) 
test                                result  vs the others in its run  difficulty            counts  why
MMLU (maker)                         89.7%  ▄·····●▄▄▄·▄ beats 2/6    medium · avg 68%        9.4%  typical
GPQA Diamond (maker)                 73.2%  █·█·●··██··█ beats 2/5    hard · avg 40%          9.1%  sharp · at its level
MMLU-Pro (maker)                     82.7%  █▄·●···▄··▄▄ beats 3/6    medium · avg 58%        7.2%  typical
CMMLU (maker)                        84.3%  █····●··▄▄▄▄ beats 2/6    medium · avg 67%        3.8%  typical
C-Eval (maker)                       90.4%  █····▄·▄·●▄▄ beats 4/6    medium · avg 68%        3.5%  typical
COLLIE (constrained text generati…   57.1%  ▄···▄█▄····● beats 6/6    hard · avg 30%          3.3%  at its level
VitaBench (maker)                    24.3%  █·······▄█·● beats 6/6    very hard · avg 6.5%    3.3%  typical
ZebraLogic (maker)                   89.3%  █···█··██●·█ beats 4/5    medium · avg 51%        1.5%  noisy
BeyondAIME (maker)                   43.0%  █····█·●▄··▄ beats 4/6    hard · avg 24%          1.2%  at its level
DROP (maker)                         79.1%  ▄···▄·▄●·▄·▄ beats 4/6    medium · avg 58%        0.4%  blunt · at its level · noisy
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
Meituan LongCat-Flash-Chat model card (…       22   91% ▇▇▇▇▇▇▇▇▇▇▇
LMArena leaderboard dataset                     1    9% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 65.4
Licences: Meituan LongCat-Flash-Chat model card ([MIT](https://huggingface.co/meituan-longcat/LongCat-Flash-Chat/blob/f0b8e8d7380330f450ca1fde4ec56823a514584c/LICENSE)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) ## How to run it ### LM Studio (recommended) 1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download) 2. Run LongCat-Flash-Chat (560B-A27B) (4bit, needs ~315.4 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/LongCat-Flash-Chat-4bit/tree/96c65642f3643fa2f72f2c49d5e105a4a1e1051b)) ```sh lms get https://huggingface.co/mlx-community/LongCat-Flash-Chat-4bit -y ``` 3. Then run lms ls to see its name, and lms chat followed by that name ### MLX (Mac) 1. Install MLX (Mac). Apple silicon Macs only, text only. ```sh # with uv uv tool install mlx-lm # or pip install mlx-lm ``` 2. Run LongCat-Flash-Chat (560B-A27B) (4bit, needs ~315.4 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/LongCat-Flash-Chat-4bit/tree/96c65642f3643fa2f72f2c49d5e105a4a1e1051b)) ```sh mlx_lm.chat --model mlx-community/LongCat-Flash-Chat-4bit ``` 3. Or start an OpenAI-compatible server at http://localhost:8080/v1 ```sh mlx_lm.server --model mlx-community/LongCat-Flash-Chat-4bit ``` Model repository and documentation: [huggingface.co/meituan-longcat](https://huggingface.co/meituan-longcat/LongCat-Flash-Chat) [interactive version](https://whatmodel.sh/model/longcat-flash-chat) · [JSON](https://whatmodel.sh/api?format=json&all=1)