[whatmodel.sh](../../README.md) / Vicuna 13B v1.1 # Vicuna 13B v1.1 LMSYS · released 2023-04 · 13B params **score 41.8** ±2.1 · comparison only: no current run option verified · from 5 benchmark results across 3 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks Vicuna 13B v1.1 among every model on that task or test, worst to best.}

other tests (5)

test                                result  vs the others in its run   difficulty        counts  why
CommonsenseQA                        50.6%  █·····▂▂●▂▄▄ beats 23/38   easy · avg 80%       35%  sharp · at its level · small run
ARC Challenge                        53.2%  ▄▁▁▂▁▄█●▃▂▃▂ beats 94/140  medium · avg 70%     25%  sharp · at its level · small run
OpenBookQA                           49.0%  ▅▁▁▁▂▂▃●▃▁▃▁ beats 67/107  medium · avg 55%     15%  small run
This is not a Dataset (negation),…   57.8%  █▁····▃▁·●·▁ beats 13/14   hard · avg 66%       15%  sharp · noisy
SummEdits factual-consistency det…   56.1%  ▄····●·█···▄ beats 1/4     hard · avg 65%      8.3%  noisy

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
Search Augmented Instruction Learning           3   75% ▇▇▇▇▇▇▇▇▇
This is not a Dataset: A Large Negation…        1   15% ▇▇
SummEdits: Measuring LLM Ability at Fac…        1    8% ▇
prior: pull toward the middle                        2% ▇
weighted average                                   = 41.8

_{Licences: Search Augmented Instruction Learning ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-242-CC-BY-4-0.txt)) · This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-531-CC-BY-4-0.txt)) · SummEdits: Measuring LLM Ability at Factual Reasoning Through The Lens of Summarization ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-600-CC-BY-4-0.txt))} ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/vicuna-13b-v1-1) · [JSON](https://whatmodel.sh/api?format=json&all=1)