---
title: 'Qiskit HumanEval: Evaluation Benchmark for Quantum Code Generation Published'
subtitle: 'New research paper introducing comprehensive benchmark for LLMs in quantum computing'
summary: 'Published research paper introducing Qiskit HumanEval dataset for evaluating Large Language Models capability to generate quantum computing code. The dataset comprises more than 100 quantum computing tasks with prompts, solutions, test cases, and difficulty ratings, establishing benchmarks for generative AI tools in quantum code development.'
authors:
- juancb
tags:
- Quantum Computing
- IBM Quantum
- Qiskit
- LLMs
- Qiskit HumanEval
- Benchmarking
- AI for Quantum
- Research
categories:
- Quantum Computing
- Artificial Intelligence
- Research
date: "2024-07-03T00:00:00Z"
lastmod: "2024-07-03T00:00:00Z"
featured: true
draft: false

# Featured image
# To use, add an image named `featured.jpg/png` to your page's folder.
# Placement options: 1 = Full column width, 2 = Out-set, 3 = Screen-width
# Focal point options: Smart, Center, TopLeft, Top, TopRight, Left, Right, BottomLeft, Bottom, BottomRight
image:
  placement: 2
  caption: 'Qiskit HumanEval Research Paper'
  focal_point: "Smart"
  preview_only: false

# Projects (optional).
#   Associate this post with one or more of your projects.
#   Simply enter your project's folder or file name without extension.
#   E.g. `projects = ["internal-project"]` references `content/project/deep-learning/index.md`.
#   Otherwise, set `projects = []`.
projects: ["qiskit-code-assistant"]
---

Excited to share our new research paper introducing the Qiskit HumanEval dataset! 🚀

This work addresses a critical need: evaluating Large Language Models' capability to generate quantum computing code. Our dataset comprises more than 100 quantum computing tasks, each with accompanying prompts, solutions, test cases, and difficulty ratings.

We systematically tested LLMs on their ability to produce executable quantum code, demonstrating the feasibility of using generative AI tools in quantum code development and establishing important benchmarks for the field.

This research opens new possibilities for AI-assisted quantum software development and provides a standardized way to measure progress in this exciting intersection of quantum computing and artificial intelligence.

Read the paper: https://arxiv.org/abs/2406.14712v1

#quantumcomputing #ibmquantum #qiskit #llms

---

*Originally shared on [LinkedIn](https://www.linkedin.com/posts/juancb_qiskit-humaneval-an-evaluation-benchmark-activity-7214160969575366656-6wIx) on July 3, 2024 - 70 reactions, 7 comments as of 11/12/2025*