[whatmodel.sh](../../README.md) / Dolly v2 12B # Dolly v2 12B Databricks · released 2023-04 · 12B params · mit **score 22.2** ±1.3 · comparison only: no current run option verified · from 14 benchmark results across 9 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Dolly v2 12B among every model on that task or test, worst to best. 
task            vs the others          rank  counts  tests
coding          ▁▁●▃▄▆█▇▅▄▃▁   #528 of 539*    8.7%      1
chat & writing  ●▁▁▃▅▄▇█▇▇▇▃    #472 of 474     48%      4
coding · #528 of 539 · 8.7% of its score Beats [Molmo 7B-D (0924)](molmo-7b-d.md), [DeepSeek R1 Distill Qwen 1.5B](deepseek-r1-distill-qwen-1-5b.md), [Qwen3 0.6B](qwen3-0-6b.md) here, though they rank higher overall. 
test       result  vs the others in its run  difficulty        counts  why
HumanEval    7.9%  ▄··▄····●▄█▄ beats 2/6    medium · avg 63%    8.7%  sharp · too hard for it
chat & writing · #472 of 474 · 48% of its score Trails [Gemma 3 270M IT](gemma-3-270m.md), [FastChat-T5 3B](fastchat-t5-3b.md), [SmolLM2 360M Instruct](smollm2-360m.md) here, though they rank lower overall. 
test                                                      result  vs the others in its run  difficulty                counts  why
LMArena (text)                                             982.4  ●▁▁▂▃▃▃▆▆▇█▃ beats 2/385  avg 1260                     18%  blunt
Polyrating rating (bias-aware re-…                         856.0  ●▁▂▁▃▇█▆▇▄▂▂ beats 2/95   avg 1187                     17%  typical
MT-Bench (GPT-4 single-answer gra…  3.2800 MT-Bench score (1-10)  ▆●▂▆▆▄██▂▄▄▆ beats 3/28   avg 8.539 MT-Bench scor…     13%  sharp · noisy
MixInstruct GPT-Rank (LLM-Blender…       6.9000 mean rank (1-11)  █···●··▄···▄ beats 2/4    avg 6.709 mean rank (1-…    0.0%  blunt · noisy
other tests (8) 
test                                result  vs the others in its run  difficulty           counts  why
MMLU                                 25.7%  ●▄·▄···▄▄·██ beats 0/8    easy · avg 79%          15%  several runs
ARC Challenge                        42.2%  ●·▄···█·▄··▄ beats 0/5    easy · avg 86%          10%  sharp
HellaSwag                            72.6%  ●·▄█··▄····▄ beats 0/5    very easy · avg 92%    8.7%  at its level
Static HHH alignment (BIG-bench h…   56.6%  ●·█········█ beats 0/2    medium · avg 77%       4.1%  small run
TruthfulQA MC1                       15.2%  ●·····█····█ beats 0/2    medium · avg 50%       2.6%  small run
Spider dev execution accuracy, ze…    9.5%  █·●▄··▄····▄ beats 3/6    medium · avg 54%       1.2%  sharp · too hard for it · noisy
TruthfulQA MC2                       34.1%  ●···█·▃····▃ beats 0/5    medium · avg 56%       0.6%  blunt · at its level
This is not a Dataset (negation),…   50.2%  ●▁····▃▁·▁·▁ beats 2/14   hard · avg 66%         0.5%  sharp · too hard for it · noisy
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
Enhancing Chat Language Models by Scali…        4   29% ▇▇▇▇
LMArena leaderboard dataset                     1   18% ▇▇
Polyrating: A Cost-Effective and Bias-A…        1   17% ▇▇
InstructEval                                    2   14% ▇▇
LMSYS Chatbot Arena leaderboard Space: …        1   13% ▇▇
Aligning Large Language Models through …        2    7% ▇
Battle of the Large Language Models: Do…        1    1% ▇
This is not a Dataset: A Large Negation…        1    1% ▇
LLM-Blender: Ensembling Large Language …        1    0% ▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 22.2
Licences: Enhancing Chat Language Models by Scaling High-quality Instructional Conversations ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-183-CC-BY-4-0.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · InstructEval ([MIT / Apache-2.0](https://github.com/declare-lab/instruct-eval/blob/49bbd2ce7e3a5bdc4024ba62a5c9c68c107ee966/LICENSE)) · LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Aligning Large Language Models through Synthetic Feedback ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-844-CC-BY-4-0.txt)) · Battle of the Large Language Models: Dolly vs LLaMA vs Vicuna vs Guanaco vs Bard vs ChatGPT - A Text-to-SQL Parsing Comparison ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-750-CC-BY-4-0.txt)) · This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-531-CC-BY-4-0.txt)) · LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/dolly-v2-12b) · [JSON](https://whatmodel.sh/api?format=json&all=1)