[whatmodel.sh](../../README.md) / Vicuna 7B v1.1 # Vicuna 7B v1.1 LMSYS · released 2023-04 · 7B params **score 36.6** ±2.4 · comparison only: no current run option verified · from 4 benchmark results across 2 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks Vicuna 7B v1.1 among every model on that task or test, worst to best.}

other tests (4)

test                                result  vs the others in its run   difficulty        counts  why
CommonsenseQA                        44.9%  █·····▂▂●▂▄▄ beats 20/38   easy · avg 80%       41%  sharp · at its level · small run
ARC Challenge                        40.5%  ▄▁▁▂▁▄●▇▃▂▃▂ beats 51/140  medium · avg 70%     28%  sharp · small run
OpenBookQA                           37.2%  ▅▁▁▁▂▂●█▃▁▃▁ beats 40/107  medium · avg 55%     17%  small run
This is not a Dataset (negation),…   52.4%  █▁····●▁·▁·▁ beats 9/14    hard · avg 66%       11%  sharp · too hard for it · noisy

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
Search Augmented Instruction Learning           3   86% ▇▇▇▇▇▇▇▇▇▇
This is not a Dataset: A Large Negation…        1   11% ▇
prior: pull toward the middle                        3% ▇
weighted average                                   = 36.6

_{Licences: Search Augmented Instruction Learning ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-242-CC-BY-4-0.txt)) · This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-531-CC-BY-4-0.txt))} ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/vicuna-7b-v1-1) · [JSON](https://whatmodel.sh/api?format=json&all=1)