label,rep,cold,concurrency,batch_wall_s,aggregate_tokens_per_s,req_total_s_median,req_eval_s_median,req_tok_per_s_median,queue_wait_s_median,queue_wait_fraction_median A-n1,0,True,1,4.931,9.33,4.93,4.61,10.0,0.05,0.01 A-n1,1,False,1,2.595,17.73,2.59,2.27,20.3,0.05,0.02 A-n1,2,False,1,2.408,19.1,2.4,2.09,22.0,0.05,0.02 A-n1,3,False,1,2.3,20.0,2.3,2.0,23.0,0.05,0.02 A-n2,0,False,2,4.595,20.02,3.49,2.06,22.4,1.11,0.25 A-n2,1,False,2,5.691,16.17,4.07,2.63,18.2,1.16,0.21 A-n2,2,False,2,4.5,20.44,3.45,2.0,23.0,1.11,0.25 A-n2,3,False,2,4.688,19.62,3.65,2.11,21.9,1.23,0.27 B-n1,0,False,1,2.483,18.52,2.48,2.17,21.2,0.05,0.02 B-n1,1,False,1,2.294,20.05,2.29,1.99,23.2,0.05,0.02 B-n2,0,True,2,6.587,13.97,6.53,5.35,8.8,0.13,0.02 B-n2,1,False,2,4.179,22.01,4.13,3.69,12.5,0.12,0.03 B-n2,2,False,2,4.182,22.0,4.13,3.68,12.5,0.13,0.03 B-n2,3,False,2,4.604,19.98,4.53,4.1,11.2,0.12,0.03 C-n4,0,True,4,19.219,9.57,19.2,13.74,3.3,0.22,0.01 C-n4,1,True,4,16.413,11.21,16.39,14.83,3.1,0.32,0.02 C-n4,2,False,4,8.954,20.55,8.92,7.86,5.9,0.23,0.03