[whatmodel.sh](../../README.md) / DBRX Instruct # DBRX Instruct Databricks · released 2024-03 · 132B params (36B active) · databricks-open-model **score 47.1** ±0.6 · #432 of 732 · from 13 benchmark results across 9 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks DBRX Instruct among every model on that task or test, worst to best.}

task            vs the others          rank  counts  tests
agent tasks     ▁▁●▆█▄▅▅▄▄▂▁   #299 of 313*    1.0%      1
coding          ▁▁▁▃▅▇●▇▅▄▃▁    #293 of 469     58%      3
chat & writing  ▁▁▁▃▆●▆█▆▇▇▃    #315 of 402     24%      4
math            ▁▁▃●█▆▄█▄▂▁▁   #376 of 431*    5.6%      1

agent tasks · #299 of 313 · 1.0% of its score

Trails [Granite 4.0 H 350M](granite-4-0-h-350m.md), [ChatGLM3-6B](chatglm3-6b.md), [Qwen3 0.6B](qwen3-0-6b.md) here, though they rank lower overall.

test     result  vs the others in its run   difficulty        counts  why
BFCL v3   39.2%  ▁▁▁▁●▃▅█▅▄▁▁ beats 13/114  medium · avg 51%    1.0%  blunt · at its level

coding · #293 of 469 · 58% of its score

Beats [Qwen3-VL 32B Instruct](qwen3-vl-32b-instruct.md), [Solar Open 100B (102B-A12B)](solar-open-100b.md), [Qwen3-VL 30B-A3B Instruct](qwen3-vl-30b-a3b-instruct.md) here, though they rank higher overall. Trails [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md), [Yi 1.5 9B Chat](yi-1-5-9b-chat.md), [Mistral NeMo Instruct (2407)](mistral-nemo-instruct-2407.md) here, though they rank lower overall.

test        result  vs the others in its run  difficulty      counts  why
HumanEval    75.0%  ▁·▁▁▃▄▆▅●▃▁▁ beats 71/99  easy · avg 76%     29%  sharp · at its level
HumanEval+   70.1%  ▁·▁▁▁▃▄▄▄●▃▁ beats 70/99  easy · avg 70%     24%  sharp · at its level
MBPP+        55.8%  ▁···▁▁▂▅●█▆▂ beats 30/75  easy · avg 66%    4.8%  at its level

chat & writing · #315 of 402 · 24% of its score

Beats [Gemma 4 E4B](gemma-4-e4b.md), [OpenThinker3 7B](openthinker3-7b.md), [Code Llama 70B Instruct](codellama-70b-instruct.md) here, though they rank higher overall. Trails [LFM2 350M](lfm2-350m.md), [Gemma 3 1B](gemma-3-1b.md), [Qwen3 0.6B](qwen3-0-6b.md) here, though they rank lower overall.

test                                result  vs the others in its run   difficulty        counts  why
AlpacaEval 2 length-controlled       25.4%  ▁▂▂▃▆▆●▆▄▄▁▁ beats 66/122  medium · avg 35%     15%  sharp · at its level
LMArena (text)                      1195.6  ▁▁▁▂▃●▃▆▆▇█▃ beats 66/385  avg 1260            4.3%  blunt
Polyrating rating (bias-aware re-…  1101.0  ▂▁▂▁▃▇●▆▇▄▂▂ beats 45/95   avg 1187            4.0%  typical
IFEval (aggregation unspecified)     52.1%  ▁▂●▂▂▄▆██▅▂▁ beats 13/145  easy · avg 70%      0.8%  at its level · small run

math · #376 of 431 · 5.6% of its score

Trails [GPT-Neo 1.3B](gpt-neo-1-3b.md), [OPT-1.3B](opt-1-3b.md), [OPT-2.7B](opt-2-7b.md) here, though they rank lower overall.

test          result  vs the others in its run  difficulty      counts  why
MATH level 5   11.7%  ▂▂●▆▄█▆▃▅▂▃▄ beats 9/92   hard · avg 28%    5.6%  sharp

other tests (4)

test                                          result  vs the others in its run   difficulty           counts  why
WildBench v2 WB-Score (task macro)  32.5989 WB-Score  ▁·▁▁·▃█●▇▆▅▆ beats 20/52   avg 43.019 WB-Score    4.6%  sharp
GPQA Diamond                                   32.9%  ▁▁▁▁●▄▆▄▅▇█▅ beats 21/211  hard · avg 39%         3.3%  sharp
BIG-Bench Hard                                 48.5%  ▁▁▁▁▂▄●▆█▃▃▁ beats 56/155  medium · avg 62%       2.9%  sharp · at its level · small run
Vectara hallucination leaderboard…             91.7%  ▁▁▂▃●▄█▆▃▂▂▁ beats 34/140  very easy · avg 94%    0.3%  blunt · maxed out · noisy

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
EvalPlus / EvalPerf published evaluatio…        3   58% ▇▇▇▇▇▇▇
AlpacaEval 1.0 and 2.0 leaderboards             1   15% ▇▇
Epoch AI                                        2    9% ▇
WildBench v2 leaderboard data                   1    5% ▇
LMArena leaderboard dataset                     1    4% ▇
Polyrating: A Cost-Effective and Bias-A…        1    4% ▇
HuggingFaceH4 zephyr-orpo-141b-A35b-v0.…        2    4% ▇
Berkeley Function Calling Leaderboard r…        1    1% ▇
Vectara hallucination leaderboard               1    0% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 47.1

_{Licences: EvalPlus / EvalPerf published evaluation runs ([Apache-2.0](../../licenses/evalplus-published-runs-Apache-2.0.txt)) · AlpacaEval 1.0 and 2.0 leaderboards ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · WildBench v2 leaderboard data ([Apache-2.0](https://github.com/allenai/WildBench/blob/d6b8dcaf377d173d031980f97c16e1a82618c03d/LICENSE)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · HuggingFaceH4 zephyr-orpo-141b-A35b-v0.1 model card ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))} ## How to run it ### Ollama (recommended) 1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download) ```sh # Linux curl -fsSL https://ollama.com/install.sh | sh ``` 2. Run DBRX Instruct (Q4_0, needs ~79.7 GB, from [ollama.com](https://ollama.com/library/dbrx/tags)) ```sh ollama run dbrx:132b ``` Other versions: Q8_0 (~158.9 GB) `ollama run dbrx:132b-instruct-q8_0` [interactive version](https://whatmodel.sh/model/dbrx-instruct) · [JSON](https://whatmodel.sh/api?format=json&all=1)