---
title: 'New Qiskit HumanEval Release: Qiskit 1.4 Compatibility and Benchmark Improvements'
subtitle: 'Enhanced robustness and accuracy for evaluating LLM-generated quantum code'
summary: 'Released a new version of Qiskit HumanEval compatible with Qiskit 1.4, featuring significant improvements to the benchmark including more robust and rigorous code execution tests for more accurate evaluations of LLM-generated quantum code.'
authors:
- juancb
tags:
- Qiskit
- AI for Quantum
- Qiskit HumanEval
- Benchmarking
- LLMs
- Quantum Computing
categories:
- Quantum Computing
- Artificial Intelligence
- Open Source
date: "2025-03-14T00:00:00Z"
lastmod: "2025-03-14T00:00:00Z"
featured: true
draft: false

# Featured image
# To use, add an image named `featured.jpg/png` to your page's folder.
# Placement options: 1 = Full column width, 2 = Out-set, 3 = Screen-width
# Focal point options: Smart, Center, TopLeft, Top, TopRight, Left, Right, BottomLeft, Bottom, BottomRight
image:
  placement: 2
  caption: 'Qiskit HumanEval Benchmark Update'
  focal_point: "Smart"
  preview_only: false

# Projects (optional).
#   Associate this post with one or more of your projects.
#   Simply enter your project's folder or file name without extension.
#   E.g. `projects = ["internal-project"]` references `content/project/deep-learning/index.md`.
#   Otherwise, set `projects = []`.
projects: ["qiskit-code-assistant"]
---

New Qiskit HumanEval release! 🚀

We've just updated https://huggingface.co/datasets/Qiskit/qiskit_humaneval to be compatible with the latest Qiskit 1.4 release! But that's not all - our update also includes significant improvements to the benchmark, making it more robust and rigorous in terms of code execution tests to provide even more accurate and comprehensive evaluations of your LLM-generated quantum code

View the full changelog: https://github.com/qiskit-community/qiskit-human-eval/compare/0.1.0...0.1.1

Want to know more about Qiskit HumanEval? Check out our paper https://arxiv.org/abs/2406.14712

#qiskit #AIforQuantum #qiskit_humaneval

---

*Originally shared on [LinkedIn](https://www.linkedin.com/posts/juancb_qiskit-aiforquantum-qiskitabrhumaneval-activity-7306320824448274433-svHs) on March 14, 2025 - 26 reactions, 2 comments as of 11/12/2025*