# Auto Dream `auto_dream` is ReMe's long-term memory distillation flow from daily to digest. By default it scans the target date and the previous day, processes only files changed since the previous dream, extracts a small set of high-value memory units across that window, and integrates them into `digest/`. Its daily inputs usually come from [Auto Memory](./auto_memory.md) and [Auto Resource](./auto_resource.md). For the file semantics of `digest/`, Sources sections, and wikilinks, see [Memory as File](./memory_as_file.md). For the linking strategy used during Integrate, see [Auto Link](./auto_link.md). Proactive discovery is a separate flow; see [Proactive](./proactive.md). ## Configuration The default configuration is in `reme/config/default.yaml`: ```yaml auto_dream: backend: base parameters: date: type: string default: "" hint: type: string default: "" scan_days: type: integer default: 2 max_units: type: integer default: 5 steps: - backend: dream_extract_step file_catalog: dream scan_days: 2 max_units: 5 - backend: dream_integrate_step - backend: dream_finish_step file_catalog: dream - backend: auto_tag_step ``` Parameters: | Parameter | Purpose | |------------------------|---------------------------------------------------------------------------------------------------------| | `date` | Date to process in `YYYY-MM-DD` format. When empty, use today in the application's timezone. | | `hint` | Additional guidance from the caller for the Extract and Integrate stages. | | `scan_days` | Recent-date window ending at `date`; defaults to 2 and has a minimum of 1. | | `max_units` | Maximum reusable units extracted in one run; defaults to 5. | ## Inputs and Outputs Inputs are daily Markdown files from the most recent `scan_days` ending at the specified date. For example, `date=2026-06-20` with `scan_days=2` scans: ```text daily/2026-06-19.md daily/2026-06-19/**/*.md daily/2026-06-20.md daily/2026-06-20/**/*.md ``` Only Markdown day indexes and notes are scanned. Proactive state and `interests.yaml` are not Auto Dream inputs. The main outputs are: | Output | Description | |--------------------------------|------------------------------------------------------------------------------| | `digest/procedure/*.md` | Methods, workflows, runbooks, and executable experience. | | `digest/personal/*.md` | User-, team-, and project-related preferences, facts, and long-term context. | | `digest/wiki/*.md` | General knowledge, concepts, observations, and decision precedents. | | `metadata/file_catalog/dream*` | Dream-specific catalog used to detect changes in daily inputs. | ## Four Stages ### 1. Extract `dream_extract_step` performs three tasks: 1. Refresh each `daily/.md` in the scan window. 2. Scan those day indexes and `daily//**/*.md`, comparing mtimes with `file_catalog: dream`. 3. Send all changed files together to the LLM and globally extract structured memory `units`. `units` are long-term memory units ready to be distilled into digest. Each has `name`, `bucket`, `summary`, and `paths`. A run returns at most `max_units`; extraction merges cross-file evidence for the same abstraction and drops passing mentions, per-file summaries, and weak candidates without reusable value. `bucket` may only be `procedure`, `personal`, or `wiki`; unknown values are routed to `wiki`. If there are no changed files, Extract succeeds with no units; Integrate then has no unit work, and Finish still performs its normal catalog summary. If files changed but no LLM is configured, Extract fails because extraction requires an LLM. ### 2. Integrate `dream_integrate_step` invokes an agent independently for each unit and integrates that unit into one digest node. It exposes these tools to the agent: ```text node_search, read, frontmatter_read, write, edit, frontmatter_update ``` This stage carries the core responsibility of `auto_link`. It first uses `node_search` to recall similar or related nodes at digest-node granularity, decides whether to create or update a node, and finally writes sources and related digest nodes as wikilinks. See [Auto Link](./auto_link.md) for the recall, deduplication, and edge-writing rules. Extract is the gate for deciding whether material is worth remembering, so Integrate has no `SKIP` action: each admitted unit must land in exactly one digest node. Creates and updates must retain provenance and weave related digest links into contextual sentences; bare wikilinks and standalone relationship fields are not valid output. There are four integration actions: | Action | Meaning | |---------------|--------------------------------------------------------------------------------| | `CREATE` | No equivalent abstraction exists; create a new digest node. | | `CORROBORATE` | The same memory appeared again; append a source or strengthen the description. | | `REFINE` | New material adds boundaries, steps, prerequisites, applicability, or detail. | | `CORRECT` | New material corrects errors, omissions, or conflicts in the existing node. | Successfully integrated units are recorded in `integrate_results`. Failed units enter `failed_units`, and their source paths enter `failed_paths`. The Finish stage does not checkpoint failed paths, ensuring that they can be retried later. ### 3. Finish `dream_finish_step` completes the run: 1. Write successfully processed changed paths to `file_catalog: dream`. 2. Also write refreshed day-index pages observed by Extract in the scan window to the catalog. 3. Persist the dream catalog if there were upserts or deletions. 4. Return a summary containing counts for scanned, changed, integrated, checkpoints, and related values. Auto Dream neither reads nor writes proactive state or `interests.yaml`. Those files are owned by the proactive refresh pipeline; see [Proactive](./proactive.md). Failed paths are not checkpointed. The next `auto_dream` run therefore continues to treat them as changed inputs until integration succeeds. Checkpoints retain the input mtimes observed by Extract, including scanned day-index pages. If a source is edited while model work runs, its newer version remains changed for a later run, even after restart. The later run must include the source date in its scan window. Finish does not checkpoint files absent from that scan. ### 4. Auto Tag After Finish, both `auto_dream` and `dream_cron` run `auto_tag_step` on Markdown digest files actually created or modified during integration, including writes recovered after agent errors. Repeated writes to one file are tagged once. The Step uses the same request-scoped `changes` contract as [Auto Memory](./auto_memory.md) and writes entity tags to the configured frontmatter key, `memory_tags` by default. Unchanged files and daily source notes are not tagged by Dream. Tagging diagnostics appear in `metadata.auto_tag`. Per-file tagging failures preserve the dream answer, success status, and checkpoint decisions. A later run without file changes does not automatically retry failed tagging. Tag-index updates follow the existing asynchronous file watcher. ## Running Auto Dream Start the service in one terminal and leave it running: ```bash reme start config=default ``` From another terminal, call the running service: ```bash reme auto_dream date=2026-06-20 ``` With caller guidance: ```bash reme auto_dream date=2026-06-20 hint="Prioritize engineering decisions and long-term preferences" ``` Override the default scan window and unit cap: ```bash reme auto_dream date=2026-06-20 scan_days=3 max_units=8 ``` The same set of steps can also be placed in a `cron` Job, for example to run every morning: ```yaml jobs: daily_auto_dream: backend: cron cron: "30 3 * * *" steps: - backend: dream_extract_step file_catalog: dream - backend: dream_integrate_step - backend: dream_finish_step file_catalog: dream - backend: auto_tag_step ``` ## Important Boundaries `auto_dream` consumes only daily inputs and does not rewrite daily bodies. Daily preserves facts and the original situation; digest is the abstracted long-term memory layer. `digest` is not a copy of the source text. Its body should preserve reusable abstractions, while a Sources section points back with contextual sentences such as `The decision was recorded in [[daily//decision.md]].` Links follow the workspace-relative wikilink semantics described in [Memory as File](./memory_as_file.md). `auto_dream` does not invent an overview from nothing. Only content that actually appears in daily input and is extracted as a memory unit can enter digest. The complete flow depends on an LLM for Extract, Integrate, and Auto Tag.