# The shared backbone: one Neo4j Enterprise instance, one Cognee, one LLM # router, reused by every problem. Brought up once and left running (`restart: # unless-stopped`) — not torn down between runs, not owned by any one problem. # # scripts/shared-up # start or reconcile; waits for health # scripts/shared-up --down # stop it (the volumes survive) # # # This stack does not have to be on the run box # # It is every problem's memory and none of its compute, which makes it the half # worth moving off a box that is doing the runs. Set `MATH_AGENT_SHARED_HOST` # to an ssh host and `scripts/shared-up` drives it there; the runs then reach # it by address — `MATH_AGENT_MEMORY_URL`, `MATH_AGENT_API_BASE_URL` — rather # than by joining Docker networks, and the ports below are published for # exactly that. `docs/runtime.md` has the split, including what publishing # grants and what still carries the security when the network no longer does. # # Everything sized in this file is overridable for the same reason: the box # that holds this stack decides its ceilings, and a heap larger than the host's # Docker VM is a startup failure rather than a slow server. # # `scripts/memory-up` calls `scripts/shared-up` itself, and `./agent` calls # `scripts/memory-up`, so an ordinary `./conjecture ` brings the whole # thing up in order without anyone naming this file. # # # One Cognee, one tenant per problem # # This used to run one Cognee container per problem, each pointed at its own # pre-created Neo4j database. A dozen problems meant a dozen Python servers, # a dozen embedding models resident in RAM, and a `mem_limit` per problem that # decided whether the tenth stack fit on the box. The isolation that shape # bought is now Cognee's own, and one process serves every problem: # # - **`ENABLE_BACKEND_ACCESS_CONTROL=true`** makes each problem a Cognee # *user* — its tenant — reached with an `X-Api-Key` header # (`scripts/memory-up` provisions one per workspace). Every dataset belongs # to the user that created it, and the server refuses a request for anybody # else's: probed live, one tenant asking for another's dataset by name got # `404 DatasetNotFoundError`, and an unauthenticated request got `401`. # - **The graph is still one database per problem**, and Cognee now creates it # rather than `scripts/memory-up`: with the `neo4j` dataset-database handler # (selected automatically for the `neo4j` provider) each dataset gets its own # Neo4j database, named `cognee`, created and brought # online on first ingest. The Bolt-level isolation the previous shape relied # on is unchanged — a driver session is bound to one named database — but # nothing outside Cognee has to know the name. # - **The vector store is per dataset too** (`lancedb`, the default handler), # so one problem's embeddings live in their own table directory. # # The runtime therefore writes and reads exactly one dataset per problem, with # the four stores — brain, sessions, library, scratch — separated by # `node_set`, which is the scoping this repository has measured as actually # enforced (`docs/memory.md`). One dataset per problem is also what keeps the # knowledge graph *connected*: a dataset is a graph database, so splitting the # four stores across four datasets would put a source and the session that read # it in different graphs, and the entity linking between them — the reason a # graph store is worth its cost — would have nothing to link across. # # # What this costs, and the failure to watch for # # The failure this shape reintroduces is **contention**, and it is the one that # broke the last shared Cognee: four concurrent runs against a single server # produced a `409 Conflict` on a `recall_memory` that had already hung the full # ten-minute tool ceiling. Two things are different now and neither is a proof: # Cognee serialises per *dataset* rather than globally (`dataset_queue`), and # the server is three minor versions on. Treat a hung or conflicting recall # under load as this failure returning — `config/trace.jsonl` counts it — and # `docs/memory.md` records what to measure before concluding otherwise. # # Multiple databases need Neo4j **Enterprise**: Community cannot create one at # all (`Local Neo4j multi-user mode requires a Neo4j deployment that supports # CREATE/DROP DATABASE`, measured directly against the Community image). # `NEO4J_ACCEPT_LICENSE_AGREEMENT=yes` accepts the licence for this deployment; # the commercial agreement behind it is handled outside this repository. networks: # Cognee reaches `neo4j:7687` here. Internal: the graph store has no reason # to be routable, and nothing on this network needs egress. graph: name: ${GRAPH_NETWORK:-math-agent-shared_graph} internal: true # The agent and every problem's Cognee reach `ladder:6969` here. Internal and # gatewayless on purpose: joining it must not hand a calibration run a route # out. `compose.yaml` joins it by name. router: name: ${LADDER_NETWORK:-math-agent-shared_router} internal: true # The ladder alone needs provider egress. Keeping this attachment off every # other service is what preserves the calibration network boundary. ladder-egress: driver: bridge # Every run reaches `cognee:8000` here, and this is the only network the # agent has that carries a service it talks to. Internal and gatewayless, as # `router` is and for the same reason: joining it must never be how a run # gets a route out. That is what makes a calibration run's isolation a # property of the shared stack rather than of an overlay somebody has to # remember to pass — `compose.yaml` gives an ordinary run its egress as a # separate attachment, and `compose.eval.yaml` takes that one away. memory: name: ${MEMORY_NETWORK:-math-agent-shared_memory} internal: true # Cognee's own way out, and nothing else is on it. Its model calls go through # the ladder on the internal `router` network and need nothing from this, but # the embedding provider fetches its model from Hugging Face on a cold # volume, so removing this turns a fresh stack into a startup failure rather # than a tighter boundary. cognee-egress: driver: bridge # Neo4j's published ports, for reading the graph from a browser or Neo4j # Desktop on this box. A container attached only to an `internal` network # has no route a published port can be forwarded over, so the publication # needs an ordinary bridge to land on — this is that bridge and nothing # else is on it. It does not widen what any *run* can reach: the agent is # never on this network, and in a calibration run it is not on `graph` # either. graph-admin: driver: bridge services: neo4j: image: neo4j:5.26-enterprise # Cognee's graph store, and nothing else on this box uses it. Both carry the # `cognee` profile so an ordinary `scripts/shared-up` starts neither: the # default engine is CortexDB, which brings its own graph and its own vector # index, and a Neo4j Enterprise instance nobody queries is several gigabytes # of reservation held against a box doing runs. # # MATH_AGENT_MEMORY=cognee scripts/shared-up profiles: ["cognee"] restart: unless-stopped environment: NEO4J_AUTH: neo4j/cognee-local NEO4J_ACCEPT_LICENSE_AGREEMENT: "yes" NEO4J_PLUGINS: '["apoc"]' # Sized for many problems' graphs landing in one instance, rather than # the defaults a single problem's private instance ran on. Revisit # against real usage once a dozen databases are live here. # Sized for many problems' graphs in one instance. Overridable because # this stack is meant to be movable: the box it runs on decides these, # and a heap larger than the host's Docker VM is a startup failure rather # than a slow server. NEO4J_server_memory_heap_max__size: ${NEO4J_HEAP:-4g} NEO4J_server_memory_pagecache_size: ${NEO4J_PAGECACHE:-2g} # The Browser is served over HTTP from this container but speaks Bolt # from the *browser*, to whatever address the server advertises. Left at # the container's own hostname it would advertise a name that resolves # to nothing on the host, and the Browser hangs on "connecting"; the # published Bolt port below is what `localhost` here refers to. NEO4J_server_default__advertised__address: ${NEO4J_ADVERTISED_ADDRESS:-localhost} healthcheck: test: ["CMD-SHELL", "cypher-shell -u neo4j -p cognee-local 'RETURN 1' >/dev/null"] interval: 10s timeout: 5s retries: 30 start_period: 20s # One instance holding every problem's graph, so the ceiling is the box's # rather than one problem's. 1g is reserved — the floor the scheduler keeps # for it while it is idle, which is what stops a busy run from starving the # store every other run reads through — and 16g is the ceiling, above the # 4g heap and 2g page cache configured above with room for the query load # a dozen live databases put on it. mem_reservation: ${NEO4J_MEM_RESERVATION:-1g} mem_limit: ${NEO4J_MEM_LIMIT:-16g} # See the note on `cognee` below: the two shared services are sized # together, and the weight is the half that does the work. cpus: ${NEO4J_CPUS:-4} cpu_shares: 2048 # 7474 is the Neo4j Browser, 7687 is Bolt for Neo4j Desktop or # `cypher-shell`. Log in as `neo4j` / `cognee-local` and pick a problem's # database from the selector — one database per problem, named # `cognee` and created by Cognee itself. The mapping from # a workspace to its database is Cognee's dataset table, not a name you can # read off the slug: `scripts/memory-inventory