---
## π News
- **September 2026** β K12-KGraph was **accepted to the NeurIPS 2026 Evaluations & Datasets Track**! π
- **July 2026** β K12-KGraph, developed by the **PKU-DCAI team**, received **Second Prize** in the **AGI4S Frontier Corpus Track** of the **MinerU Frontier Corpus and Data Intelligence Challenge**. [Award certificate](docs/awards/k12-kgraph-mineru-2026-second-prize.pdf).
---
## π Why K12-KGraph?
Modern LLMs can answer "what is the Pythagorean theorem?" but struggle with **curriculum cognition** β the structured understanding of:
- π§ *What are the prerequisites of a concept?*
- π¬ *Which experiment verifies it?*
- π *Which exercises test it?*
- π *Where does it live in the textbook?*
- πΈ *What are its taxonomic and relational neighbors?*
K12-KGraph is the first open, multi-subject, official-textbook-grounded knowledge graph that explicitly aligns all five dimensions around each STEM concept, yielding two ready-to-use AI assets:
| | [K12-Bench](https://huggingface.co/datasets/lhpku20010120/K12-KGraph) | [K12-Train](https://huggingface.co/datasets/lhpku20010120/K12-KGraph) |
|---|---|---|
| **Size** | 23,640 multi-select questions | 2,267 instructionβresponse pairs |
| **Purpose** | Evaluate structural curriculum cognition | Teach it via KG-guided SFT |
| **Task families / sources** | Ground Β· Prereq Β· Neighbor Β· Evidence Β· Locate | Node-grounded + Edge-grounded + Deterministic templates |
| **Headline result** | Gemini-3-Flash reaches only **57.1%** EM | Beats 8 mainstream SFT corpora on GaokaoBench & EduEval under a strict 2,300-sample budget |
---
## π Leaderboard Snapshot (K12-Bench, zero-shot)
Instance-level macro F1 and exact match, in %.
| Model | Overall EM | Overall F1 |
|---|---|---|
| *Random guess baseline* | 6.7 | 36.4 |
| Meta-LLaMA-3-8B-Instruct | 7.2 | 52.6 |
| GLM-4.7-Flash | 31.7 | 63.9 |
| GPT-4o | 31.1 | 65.9 |
| Qwen3-32B | 42.6 | 69.5 |
| Gemma-4-31B-IT | 46.4 | 69.5 |
| GPT-5.2 | 42.8 | 68.0 |
| Gemini-2.5-Flash | 48.3 | 66.7 |
| **Gemini-3-Flash** | **57.1** | **73.0** |
> Even the strongest proprietary model leaves > 40% of items unsolved on **Prereq** and **Neighbor** β the tasks requiring directed, structural reasoning. See the [project page](https://haolpku.github.io/K12-KGraph-page/) for the full 5-task breakdown.
---
## πΊοΈ What's in this Repository?
```
K12-KGraph/
βββ src/
β βββ kg/ # Knowledge-graph construction pipeline
β βββ benchmark/ # K12-Bench generation from graph queries
β βββ sft_qa/ # K12-Train synthesis (node & edge grounded)
β βββ utils/ # Shared config / LLM client / IO
βββ eval/ # Multiple-choice evaluation runner (OpenAI / vLLM)
βββ config/ # Default pipeline configuration
βββ demo/ # Trimmed JSON/JSONL samples
βββ books.yaml # Book registry
βββ docs/img/ # README figures
βββ requirements.txt
```
Pipeline flow:
```
PDF textbooks ββΊ MinerU parsing ββΊ Section split ββΊ GPT-5.2 schema-constrained extraction
ββΊ Hierarchical merge (book β subject β global) ββΊ DAG validation + expert review
ββΊ K12-KGraph ββΊ K12-Bench (queries) + K12-Train (QA synthesis)
```
---
## π Quick Start
### 1. Install
```bash
git clone https://github.com/haolpku/K12-KGraph.git
cd K12-KGraph
pip install -r requirements.txt
```
> If you will run the graph pipeline from PDFs, also install [**MinerU**](https://github.com/opendatalab/MinerU) and make `magic-pdf` callable from the shell (command name configurable via `config/default.yaml`).
### 2. Load the released dataset
```python
from datasets import load_dataset
kg = load_dataset("lhpku20010120/K12-KGraph", split="train")
bench = load_dataset("lhpku20010120/K12-KGraph", name="bench", split="test")
train = load_dataset("lhpku20010120/K12-KGraph", name="train", split="train")
```
### 3. Build the graph from scratch
```bash
cp config/.env.example config/.env # add your OPENAI_API_KEY etc.
python src/kg/run_pipeline.py \
--config config/default.yaml \
--filter-prefix