SAG

English · 简体中文

From now on, this is the only knowledge base app you need.

Built on the state-of-the-art SAG architecture, it turns scattered documents and data into knowledge that is searchable, connected, and traceable.

Community · Project · Technology · User Guide · Developer Guide

/chat/completions \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{"messages":[{"role":"user","content":"What is this material about?"}]}' ``` The response is a standard `chat.completion` with an additional `sag.citations` field; standard clients ignore unknown fields. Set `"stream": true` to receive SSE chunks. ### Use SAG as a Dify external knowledge base SAG can connect to Dify's **Connect to an External Knowledge Base** feature through a dedicated compatibility endpoint, with no Dify source-code changes. For the API key, Docker network, Source ID, and Dify console configuration, see [`docs/dify-integration.en.md`](docs/dify-integration.en.md). ### Operate and update ```bash docker compose ps # api and web should be healthy docker compose logs -f api web # follow logs docker compose restart # restart services docker compose down # stop and keep all data git pull --ff-only # update the source checkout docker compose up -d --build # rebuild without deleting volumes ``` Default persistence: | Runtime | Application metadata | Knowledge engine | Location | | --- | --- | --- | --- | | Docker default | SQLite | SQLite + LanceDB | Docker volume `sagdata` | | Local development | SQLite | SQLite + LanceDB | `apps/api/.data/` | | PostgreSQL override | PostgreSQL | PostgreSQL + pgvector | `pgdata` and `sagdata` volumes | `docker compose down` preserves data. **`docker compose down -v` permanently deletes the database, knowledge index, and uploaded files.** ### Network and production safety The default Compose configuration binds ports 3000 and 8000 to `127.0.0.1` and uses `SAG_AUTH_MODE=local`: the name is a local identity, not password authentication. Do not expose this default mode to an untrusted network. For custom ports or a trusted LAN address: ```bash cp .env.example .env # Edit BIND_ADDRESS, WEB_PORT, API_PORT, SAG_CORS_ORIGINS, and NEXT_PUBLIC_API_BASE. # For remote access, also set SAG_AUTH_MODE=password and keep SAG_ALLOW_REGISTRATION=false. docker compose up -d --build ``` On first launch, `password` mode guides you through creating an email and password. Later sign-ins require both credentials and never fall back to a name or the first user. `SAG_ALLOW_REGISTRATION=false` still permits the first credential account, then closes registration. `NEXT_PUBLIC_API_BASE` is compiled into the web image, so changing it requires `--build`. A server deployment should also use a strong `SAG_SECRET_KEY`, HTTPS, and protection such as VPN, IP allowlisting, or reverse-proxy rate limiting. The current `sag auth login --name` command is for `local` mode; with `password` mode, sign in through the web UI first and provide the issued token to the CLI through `SAG_TOKEN`. --- ## Developer Guide ### System boundaries SAG has a separated Next.js frontend and FastAPI backend. The backend is a reference application built on the public `zleap-sag` Python engine. You can reuse the complete backend from another frontend, or embed `zleap-sag` directly in a Python service of your own.

### Repository map ```text apps/ ├── web/ Next.js 15 + React 19 product UI ├── desktop/ Electron shell, packaging, and local runtime lifecycle └── api/ ├── sag_api/ │ ├── api/v1/ FastAPI HTTP routes and serialization │ ├── connectors/ File/web source connectors and registry │ ├── parsing/ MarkItDown and MinerU normalization │ ├── jobs/ Background ingest → extract state machine │ ├── sag/ The only application adapter importing zleap-sag │ ├── generation/ Retrieved evidence → streamed cited answer │ ├── mcp/ Knowledge MCP server and HTTP mount │ ├── services/ Application/domain orchestration │ └── tools/ Built-in and remote MCP Agent tools └── sag_agent/ Framework-independent Agent runtime core deploy/ Deployment initialization assets docs/assets/readme/ README screenshots and diagrams ``` The central dependency rule is simple: application code reaches the engine through `apps/api/sag_api/sag/`; the engine does not know about FastAPI, the Web UI, users, conversations, or citations. ### Local development Start the backend and frontend in separate terminals from the repository root. ```bash # Terminal 1: API at http://localhost:8000 cd apps/api python -m venv .venv . .venv/bin/activate pip install -e ".[dev]" cp .env.example .env uvicorn sag_api.main:app --reload ``` ```bash # Terminal 2: Web at http://localhost:3000 cd apps/web npm install npm run dev ``` Useful checks: ```bash cd apps/api && pytest cd apps/api && ruff check . cd apps/web && npm run typecheck cd apps/web && npm run build ``` ### Desktop client The Electron client packages the same Next.js application together with the local FastAPI backend. Desktop development, target-specific release builds, signing, update configuration, and data locations are documented in [`apps/desktop/README.md`](apps/desktop/README.md). ### Use `zleap-sag` directly [`zleap-sag`](https://pypi.org/project/zleap-sag/) is the continuously maintained Python engine behind the SAG application. Distribution name: `zleap-sag`; import path: `zleap.sag`; Python: 3.11+; license: MIT. The application currently requires `zleap-sag>=0.7.1`. Install the zero-infrastructure local stack: ```bash pip install zleap-sag ``` Then run the complete ingest → extract → search lifecycle: ```python import asyncio from zleap.sag import DataEngine, EngineConfig from zleap.sag.config import EmbeddingConfig, LLMConfig async def main() -> None: config = EngineConfig( llm=LLMConfig( api_key="sk-...", base_url="https://your-openai-compatible-host/v1", model="qwen3.6-flash", ), # When api_key/base_url are omitted, embedding reuses the LLM endpoint. embedding=EmbeddingConfig(model="bge-large-en-v1.5"), language="en", ) # One DataEngine instance represents one logical data source. async with DataEngine(config) as engine: ingest = await engine.ingest("knowledge.md") extract = await engine.extract() result = await engine.search( "Why is SAG effective for multi-hop retrieval?", strategy="multi", top_k=5, ) print(ingest.chunk_count, extract.event_count) for section in result.sections: print(section.get("content", "")[:200]) asyncio.run(main()) ``` Local storage is created automatically under `./.zleap/`. Add that directory to `.gitignore`. #### Configuration Use one configuration style, not both: | Style | Construction | Best for | | --- | --- | --- | | Parameter injection | `EngineConfig(llm=..., embedding=...)` | Libraries, notebooks, explicit application wiring | | Environment | `EngineConfig.from_env()` or `from_env(env_file=".env")` | Containers and 12-factor services | Minimal environment configuration: ```bash export OPENAI_API_KEY=sk-... export OPENAI_BASE_URL=https://your-openai-compatible-host/v1 export LLM_MODEL=qwen3.6-flash export EMBEDDING_MODEL=bge-large-en-v1.5 ``` ```python from zleap.sag import EngineConfig config = EngineConfig.from_env() ``` `EngineConfig(...)` does not read environment variables implicitly. Use explicit parameters or `from_env()`. A separate embedding endpoint can be set with `EmbeddingConfig(api_key=..., base_url=..., model=...)`. #### Public `DataEngine` API | API | Purpose | | --- | --- | | `await engine.start()` | Initialize connections; local SQLite/LanceDB schema is created automatically | | `await engine.aclose()` | Close engine resources; handled automatically by `async with` | | `await engine.chunk(source)` | Parse and chunk a path or raw string without persisting it | | `await engine.ingest(path, ...)` | Parse one document, chunk it, embed it, and persist chunks/vectors | | `await engine.extract(...)` | Extract and persist the event-entity index for the current source | | `await engine.search(query, strategy=..., top_k=...)` | Return a typed `SearchResult` with `sections` and timing/statistics | | `await engine.init_schema()` | Idempotently initialize production schemas; not needed for the default local stack | Typed results are available from `zleap.sag.results`: `ChunkResult`, `IngestResult`, `ExtractResult`, and `SearchResult`. All engine exceptions derive from `SagError`, so application boundaries can catch one base type. #### Retrieval modes | UI label | Python strategy | Implementation | | --- | --- | --- | | Fast (default) | `vector` | Direct retrieval by semantic similarity for a faster response | | Precise | `multi` | Combines entity relationships with LLM reranking for more complete results | The UI exposes only **Fast** and **Precise** retrieval modes. Precise maps to SAG's `multi` strategy and does not run a separate GraphRAG implementation. #### Storage backends | Deployment | Relational storage | Vector storage | Package extra | | --- | --- | --- | --- | | Local default | SQLite | LanceDB | none | | Single database | PostgreSQL | pgvector | `zleap-sag[postgres]` | | Production split | MySQL/PostgreSQL/OceanBase | Elasticsearch | `zleap-sag[mysql]`, `[postgres]`, `[es]` | | Single database | OceanBase 4.3.3+ | OceanBase vector | `zleap-sag[mysql]` | Changing `EngineConfig` changes the backend without changing ingest/extract/search calls. Current engine connections are process-global, so use one `EngineConfig` per process. For the full package configuration, extras, examples, and changelog, see the [`zleap-sag` package page](https://pypi.org/project/zleap-sag/). ### Build your own frontend with the SAG backend A browser cannot import a Python package directly. A custom frontend should call a Python HTTP service that owns `DataEngine`. This repository's FastAPI backend is that reference service, and it is already separated from the Next.js UI. The self-hosted API is available after starting SAG: | Entry | Address | | --- | --- | | API base | `http://localhost:8000/api/v1` | | Interactive OpenAPI | [http://localhost:8000/docs](http://localhost:8000/docs) | | OpenAPI schema | [http://localhost:8000/openapi.json](http://localhost:8000/openapi.json) | | MCP Streamable HTTP | `http://localhost:8000/mcp/` | This is a **self-hosted API**, not a hosted public cloud API. In the default `local` mode, most routes use a SAG JWT issued after name login: ```bash curl -s http://localhost:8000/api/v1/auth/login \ -H 'Content-Type: application/json' \ -d '{"name":"Developer"}' ``` Copy `access_token` from the response and send it as: ```http Authorization: Bearer ``` With `SAG_AUTH_MODE=password`, create the first credential through `POST /auth/register`, then sign in with email and password: ```bash curl -s http://localhost:8000/api/v1/auth/login \ -H 'Content-Type: application/json' \ -d '{"email":"owner@example.com","password":""}' ``` #### API map | Area | Main routes | Purpose | | --- | --- | --- | | System | `GET /system/health`, `/system/ready`, `/system/capabilities` | Health and active engine capabilities | | Identity | `POST /auth/login`, `GET /auth/me` | Local identity and JWT | | Sources | `GET/POST /sources`, `GET/PATCH/DELETE /sources/{id}` | Source lifecycle | | Documents | `/sources/{id}/documents` and `/documents/ingest` | File upload, continuous text/message ingestion, reprocessing, deletion | | Search | `POST /search`, `POST /sources/{id}/search` | Global or source-scoped `vector`/`multi` retrieval | | Graph | `GET /sources/{id}/entities`, `/sources/{id}/graph` | Event-entity inspection | | Agents | `/agents`, `/threads`, `/ask` | Agent configuration, conversations, SSE runs, citations | | OpenAI-compatible | `POST /openai/{agent_id}/chat/completions` | Use any SAG Agent as a cited model, streamed or non-streamed | | MCP | `/mcp/` or `/mcp/?source_id={id}` | Expose the whole knowledge base or one source to an MCP host | Create a source, ingest text, and search it: ```bash BASE=http://localhost:8000/api/v1 TOKEN= SOURCE_ID=$(curl -s -X POST "$BASE/sources" \ -H "Authorization: Bearer $TOKEN" \ -H 'Content-Type: application/json' \ -d '{"name":"Product docs"}' | jq -r .id) curl -s -X POST "$BASE/sources/$SOURCE_ID/documents/ingest" \ -H "Authorization: Bearer $TOKEN" \ -H 'Content-Type: application/json' \ -d '{"title":"SAG","text":"SAG uses event-entity indexing and query-time dynamic hyperedges."}' curl -s -X POST "$BASE/sources/$SOURCE_ID/search" \ -H "Authorization: Bearer $TOKEN" \ -H 'Content-Type: application/json' \ -d '{"query":"How does SAG retrieve knowledge?","strategy":"multi","top_k":5}' ``` Document ingestion is processed by the background job queue. Check the returned document status or its job before expecting search results. For a frontend served from another origin, add it to `SAG_CORS_ORIGINS`. If the API address changes, rebuild the Web image with the matching `NEXT_PUBLIC_API_BASE`. ### PostgreSQL/pgvector deployment The optional production override moves application metadata and engine storage to PostgreSQL/pgvector: ```bash cp .env.example .env openssl rand -hex 32 # set SAG_SECRET_KEY openssl rand -hex 24 # set POSTGRES_PASSWORD docker compose -f compose.yaml -f compose.postgres.yaml config docker compose -f compose.yaml -f compose.postgres.yaml up -d --build ``` Set real `SAG_CORS_ORIGINS` and `NEXT_PUBLIC_API_BASE` values before server deployment. Back up both `pgdata` and `sagdata` before upgrades. --- ## Contributing and License - Contribution workflow: [CONTRIBUTING.md](CONTRIBUTING.md) - Python engine: [`zleap-sag` on PyPI](https://pypi.org/project/zleap-sag/) - Paper reproduction: [Zleap-AI/SAG-Benchmark](https://github.com/Zleap-AI/SAG-Benchmark) SAG is released under the [MIT License](LICENSE). --- ## Community Join the SAG community on Discord or WeChat to connect with maintainers and other users.

Discord	WeChat