# Graph
First step: read [the measured result and its misses](#the-proof), then run the [quick start](#quick-start) to try the tool on your own text.
Graph is a tool that shows what a text actually says, and shows its
work. You give it plain text. It finds the people, groups, places,
dates, and tools the text names, and the links between them that the
text states. It then puts those names and links in a map you can
search and check.
The tool is written in Python. Its main work runs with no added code
libraries. NumPy and Matplotlib are used only to score the tests and
draw pictures. We do not claim the tool is good. We test it on texts
labeled by hand, and we publish the misses next to the scores.
## The need
This project serves Layer 4: Shared truth. "Knowledge organized well enough to argue with. Claims that carry their evidence. Errors published, not buried." This repo serves that need by measuring its extraction on a hand labeled eval set and publishing the misses next to the scores.
Unmet, it looks like: decisions made about communities from scattered and invented data, arguments with no floor of agreed fact, institutions that cannot tell what they know from what they assume.
The full map: [The Ascent, a map of human needs](https://github.com/topsoilcommons/topsoilcommons/blob/main/NEEDS.md)
The research under this layer: [Foundations, Truth and shared knowledge](https://topsoilcommons.github.io/topsoilcommons/foundations.html#truth-and-shared-knowledge)
## What it is
### Hybrid extraction, honestly staged
The design is hybrid: rule based extraction always runs, offline,
with no dependencies, and an LLM mode
(`knowledge_graph_explorer/llm_extractor.py`) extracts through the
same type system for comparison on the same eval set.
**The LLM mode has not produced numbers in this environment, and
this README claims none.** With no `OPENAI_API_KEY` set, the mode
records status "skipped" in every eval artifact and returns nothing.
That behavior is itself under test
(`tests/test_extractors.py`). Set the key and rerun
`scripts/evaluate_extraction.py` and the same artifact gains an LLM
section measured the same way; until then, any LLM score for this
project would be invented, and invented numbers are the one thing
this portfolio does not publish.
### Features
- **Entity extraction**: PERSON, ORGANIZATION, LOCATION, DATE, and
TECHNOLOGY entities via gazetteers, patterns, and heuristics
- **Relation extraction**: founded_by, located_in, works_for,
acquired, developed, born_in, partner_of, competes_with, via verb
patterns plus a co-occurrence pass whose cost the eval measures
- **LLM extraction mode**: optional, same schema, skips cleanly
without a key (see above)
- **Knowledge graph**: typed nodes and edges, CRUD, JSON
serialization, deduplication
- **Graph algorithms**: PageRank, centrality (betweenness, degree,
closeness), community detection (Louvain), shortest path,
connected components
- **Query engine**: a simple query language, in plain words, for following links in the graph
- **Visualization**: force-directed layout from scratch, matplotlib
rendering
- **Built-in corpus**: 25+ paragraphs on tech companies, AI,
computing history, and science
### Quick start
```bash
pip install -r requirements.txt
python main.py # build and visualize a graph
python -m unittest discover -s tests # extractor + eval tests
python scripts/evaluate_extraction.py # the measured eval
```
Running `main.py` produces `knowledge_graph.json`,
`knowledge_graph.png`, `communities.png`, and
`subgraph_google.png`.
### Query examples
```python
from knowledge_graph_explorer.query_engine import QueryEngine
qe = QueryEngine(graph)
qe.execute("find all entities connected to Google")
qe.execute("path between Elon Musk and Apple")
qe.execute("entities of type PERSON connected to ORGANIZATION")
qe.execute("relations of type founded_by")
qe.execute("info about Tesla")
```
### Project structure
```
knowledge_graph_explorer/
tokenizer.py - Sentence/word tokenizer, POS-like tagging
entity_extractor.py - Rule based named entity recognition
relation_extractor.py - Verb pattern + co-occurrence relations
llm_extractor.py - Optional LLM mode (skips without a key)
knowledge_graph.py - Graph data structure, CRUD, serialization
graph_algorithms.py - PageRank, communities, paths, centrality
query_engine.py - Query language for graph traversal
visualization.py - Force-directed layout and rendering
datasets.py - Built-in text corpus (25+ paragraphs)
main.py - Pipeline: extract, build, analyze, visualize
eval/ - Hand labeled eval set and labeling rules
scripts/evaluate_extraction.py
tests/test_extractors.py
results/eval_results.json
```
### Requirements
- Python 3.8+
- NumPy, Matplotlib (the LLM mode needs only an API key and the
standard library)
## The proof
### Measured extraction quality
From `scripts/evaluate_extraction.py`, run 2026-10-15 against the
hand labeled set in `eval/` (20 documents, 133 gold entities, 66
gold relations; labeling rules and the expansion record in
`eval/README.md`). Full artifact: `results/eval_results.json`,
which stores the scores to four decimals (entities F1 0.9042,
relations F1 0.5976); the table below rounds them to three.
| | Precision | Recall | F1 |
| --- | --- | --- | --- |
| Entities (rule based) | 0.922 | 0.887 | 0.904 |
| Relations (rule based) | 0.500 | 0.742 | 0.598 |
The 0.937 versus 0.936 discrepancy is settled. On the original
8 documents, the eval artifact `results/eval_results.json`
produced one value, 0.9365, for entity precision, recall, and F1.
This README printed 0.937, that value rounded half up to three
decimals, while CHECKLIST.md printed 0.936, the value as formatted
by the eval script. There was never a second measurement; both
files described the same artifact and now use the same rounding.
The expanded corpus supersedes that headline figure in any case:
the current figure is 0.904, measured as above.
Read those numbers the way they were earned:
- **Entity extraction is the strong half**, and the eval improved
it. The first run scored recall 0.889; the misses exposed two
real bugs (dates before 1900 were invisible to the year patterns;
the capitalized-name pattern required a connector word, so plain
two-word names like Pierre Curie never matched). Both are fixed,
with regression tests, and both runs are on record.
- **Relation precision is the honest weakness.** Recall is high
because the co-occurrence pass proposes a relation for almost
every nearby entity pair, and precision pays for it: roughly half
of the proposed relations are not stated in the text. The eval's
labeling rule (a relation counts only if the text states it) is
what makes that cost visible instead of hiding it.
- **The October 15, 2026 expansion lowered the scores, and that
is published as measured.** Twelve new documents (NASA
biographies, the Nobel Prize summary for CRISPR, and Wikipedia
articles, adapted; record in `eval/README.md`) took the set from
8 to 20 documents. Entity F1 fell from 0.9365 to 0.9042
(precision 0.9365 to 0.9219, recall 0.9365 to 0.8872). Relation
F1 fell from 0.6047 to 0.5976: precision rose from 0.4727 to
0.5000 while recall fell from 0.8387 to 0.7424, because gold
relations whose endpoint entities are missed cannot be found.
- **The misses are named, not rounded away.** The system misses
entities its gazetteers do not know (Warsaw, Armonk, Watson,
iPhone), cannot follow pronouns across sentences ("He joined
Microsoft" loses the works_for relation), and has documented
type ambiguities the eval labels against it (Windows and iOS are
labeled TECHNOLOGY; the gazetteer also lists them as
organizations). The expansion added named misses of the same
kinds: Neil Armstrong, Jennifer Doudna, and Katherine Johnson
are not in the person gazetteer, Wapakoneta, Ulm, Huntsville,
Alabama, White Sulphur Springs, and the Moon are not in the
location gazetteers, Wikipedia as a single word organization is
not matched, and Linux and the World Wide Web are also emitted
with a second, wrong type. Details: `eval/README.md` and the
eval artifact.
### Status and limitations
- Rule based extraction is a teaching-grade instrument: strong on
gazetteer-known entities, weak on unknown names, pronouns, and
anything the patterns do not reach. The eval numbers above are
the current, dated measurement of exactly that.
- The eval set is still small (20 documents, up from 8 on
October 15, 2026). Growing it further is a standing good first
issue; see CONTRIBUTING.md.
- The co-occurrence relation pass trades precision for recall by
design. A confidence threshold or a stated-ness filter is the
obvious next experiment, and the eval exists to judge it.
## Live demo
Live demo: https://topsoilcommons.github.io/graph/
## Contribute, human or agent
Pick a [good first issue](https://github.com/topsoilcommons/graph/issues) in this repo. Each one is sized in hours and written to be finishable by a newcomer.
Try the Eval Challenge against this repo's published extraction scores. Beat a baseline or document a real failure, with the run attached.
Adopt the extractor or the graph in your own work and report back what happened in real use, good or bad.
The full protocol: [The Open Table](https://github.com/topsoilcommons/topsoilcommons/blob/main/OPEN_TABLE.md). Verified work is credited in the public [Ledger](https://github.com/topsoilcommons/topsoilcommons/blob/main/LEDGER.md).
### Start with a prompt
For an AI agent:
```
You are an AI agent contributing to Graph. Run the Eval Challenge end to end.
Install with: pip install -r requirements.txt
Run the tests with: python -m unittest discover -s tests -v
Run the eval with: python scripts/evaluate_extraction.py --out /tmp/eval_results.json
Compare your run to the published baseline in The proof section above and in results/eval_results.json, measured on the hand labeled set in eval/.
Beat a baseline or document a real failure.
Attach the run to your pull request, keep one improvement per pull request, and name your operator.
```
For a human:
```
File a failure report as an issue and include the input text you ran.
Say what the extractor produced and what should have happened instead.
Attach the eval or test output if you have it.
```
The layer below: Cohort, belonging
The layer above: Machine, autonomy
The research under this layer: Foundations, Truth and shared knowledge
Built on Cohort: belonging.
Unlocks the ability to discern.