# Using SmartBrain_3000 Everything here runs locally and is encrypted at rest. Here's what each area does. ## Where things live The sidebar holds nine areas — this is the whole app. On a phone the four you reach for most sit in the bottom bar (Chat, Knowledge, Info, Activity) and the rest are under **More**: | Area | What it's for | | --- | --- | | **Chat** | Talk to the assistant. It can use tools; anything consequential waits for you. | | **Knowledge** | Your documents and notes, plus the vaults that group them. | | **Planner** | Tasks, with due dates, priorities and recurrence. | | **Schedules** | Prompts that run on a timer. | | **Email** | An optional Gmail connection: read and send. | | **Info** | The output of your scheduled runs, newest first. | | **Activity** | Approvals waiting for you, and the record of everything the assistant tried. | | **Usage** | What your cloud models have cost. | | **Settings** | Everything you configure — and **Status**, a live view of everything the app is doing. Desktop only. Tab by tab under [Settings](#settings), below. | Below them sit four controls: **Help** (this guide, offline, no unlock needed), **Theme** (follow the system, or force light or dark), **Lock**, and — on a paired phone — **Unpair**. The top strip shows an **Encrypted · On-device** chip and, on a phone, the remote connection state. The version you're running is under the logo, top-left. The **Desktop** shows all nine areas. On a **paired phone** ([Remote access](08-remote-access.md)) you get the eight meant for use on the go — Chat, Knowledge, Planner, Schedules, Email, Info, Activity, and Usage — while Settings and first-time setup stay on the Desktop. ## Chat Talk to your assistant. Chat can optionally **use tools** to act on your behalf — search your knowledge, **read or summarize a whole document**, **save a note back to your knowledge**, add a task, fetch a public web page, send an email, and more — the full list is under **What the assistant can do**, below. Replies are formatted: headings, lists, tables, and code blocks render properly. ### While an answer is being written Answers **stream in** word by word. While one is arriving the Send button becomes **Stop** — press it and the partial answer is kept, marked *(stopped)*, rather than thrown away. When the assistant is using tools it narrates what it is doing in place of the thinking dots: *"Searching the web…"*, *"Reading a document…"*, *"Writing the answer…"*, each ticked off as it finishes. If the conversation has scrolled, **Top** and **Latest** pills jump you to either end. ### After an answer Every reply carries **Copy** (the raw Markdown, not the rendered page) and **Listen** (read just this answer aloud — see **Voice**, below). The most recent one also offers **Regenerate** — ask again for a fresh answer to your last message. The new answer is added below the old one rather than replacing it, so what you see is exactly what a reload will show. Only the newest answer can be regenerated; redoing an older one would fork the thread. Every message *you* sent carries a colored **Retry** beside **You** — it sends that exact message again, handy after a model hiccup or a model switch. Answers that used your knowledge show **source chips** underneath — more on those under **Knowledge**, below. ### Saved chats **+ New chat** starts a fresh thread. The **Saved chats** picker at the top switches between them, with **Load older** for threads beyond the first page; **Load older messages** does the same inside a long thread. **Rename** retitles the open chat (a new chat is titled from your first message). **Refresh** reloads the thread and the chat list — useful when you continued a conversation on your phone and want it on the Desktop, or the reverse. The page also refreshes itself whenever you come back to it. **Delete** moves the open chat to the **Trash**; **Delete all…** moves every chat there, behind a confirmation. Trashed chats are restorable for 30 days from Settings → Account & Data, and are removed for good after that. Above the conversation, **Provider** and **Model** pick the model for this session — see [Choosing a model in Chat](02-models.md#choosing-a-model-in-chat). ### It knows what time it is The assistant knows what time it is **where you are**. Your browser reports its timezone and every turn is told your local date and time, with UTC alongside for cross-zone questions; scheduled runs get the same. There is nothing to configure — the zone is read from your browser and stored locally, like any other setting. ### Tools and approval Tools are **risk-tiered**, and this is the core safety idea: - **Observe** (e.g. knowledge search) runs automatically — it only reads. - **Reviewed** (e.g. add a task, search the web) is **never run automatically** until you say so. The assistant *proposes* it and a card appears **right in the conversation** with **Approve**, **Always allow**, and **Deny**; resolving the last pending card resumes the turn by itself, and **Approve all** appears when several reviewed-tier actions are waiting. **Activity** keeps the record. If you get tired of approving the same tool, **Always allow** lets that one run without asking from then on — and **Stop allowing** takes it back. The two tools that fetch a URL the assistant composed (**Fetch a page**, **Add a URL to knowledge**) are allowed **per site**: the button reads *Always allow *, future calls to that exact site run unattended, and a different site still asks once. That's deliberate — a page the assistant reads could try to talk it into fetching an address an attacker owns, and an unknown site always parks for your review. - **Irreversible** (e.g. send an email, delete a task) always waits for your approval, with an extra confirmation, and can never be pre-authorized. So the assistant can draft and suggest, but anything that changes data or reaches out requires your explicit OK. Every attempt is recorded in **Activity**. **For example:** ask *"search my knowledge for the lease terms"* and the assistant reads and answers immediately (Observe). Ask *"email the landlord about it"* and it **drafts** the message and **parks it as a card in the chat** — nothing sends until you approve it there (Irreversible, with an extra confirm). Activity lists it too. A parked action doesn't wait indefinitely — see **Activity**, below. ## Voice Chat can listen and talk, with **nothing to set up**: dictation is built in on every OS and runs on your own machine (how, and the optional own-server upgrade, is under [Voice](02-models.md#voice-dictation-and-spoken-replies) on the models page). Nothing you say is sent to any speech service — on the phone, audio travels the encrypted link to your Desktop, which transcribes it locally. ### Dictate Tap the **mic** beside the message box (or **hold Space** on a keyboard) and talk. Your words appear **under the box as you speak**; when you pause, the recording ends by itself and the finished transcript lands in the message box — **you review before it sends**. Say **“send”** at the end to submit in the same breath, **“cancel”** to discard, or **“start over”** to clear and re-listen; **Esc** cancels a recording too. One recording is capped at two minutes. The first time, the mic may show a **percent**: the speech model is downloading (once, about 141 MB) and the mic enables itself when it is ready. A red mic means the download failed — tap it to retry, or use **Retry download** on **Settings → Status**. ### The voice modes Above the message box sits a row of labeled pills. Each is a mode you leave on: - **Speak replies** — answers are read aloud **sentence by sentence as they stream in**, in your device's own voice. Every answer also has a quiet **Listen** action underneath to hear just that one. While a reply is being read, **Send becomes Stop** — press it to stop the voice (a stopped answer keeps its text). - **Hands-free** — every dictation sends itself when you pause; say “cancel” to stop one. - **Conversation** — 100% voice. You talk, it sends itself, the reply is spoken, and the mic reopens for your follow-up. Say **“stop listening”** or **“goodbye”** to end. The first mic open still needs one tap (a browser rule); after that, no buttons. To cut a reply short, press **Stop**. (The microphone stays closed while a reply is read: it would hear the reply too, and an open microphone makes the system turn the voice down — so the mic opens the moment the voice finishes.) - **Short · Medium · Long** — how long *spoken* replies should be. It applies only while replies are read aloud (Speak replies or Conversation on); typed chat is unaffected. The default is **Short**, because a long spoken answer is tiring — Long is one tap away when you need the detail. ### Your own wake word With Conversation on, SmartBrain can wait for a phrase instead of listening all the time — **“Hey SmartBrain”**, **“Hey Merl”**, whatever you like. Set it under **Settings → Status → Voice** and press **Test recognition**: say the phrase three times, and it shows exactly what the engine heard each time. Unusual names are often spelled the engine's own way (“Merl” may come back as “Merle”); one tap **accepts those spellings**, and from then on the phrase works as you say it. Then, in Chat, the Conversation pill shows your phrase and the mic waits for it: **“Hey Merl, what's on my calendar?”** carries the question through in one breath, and anything that doesn't start with the phrase is ignored (the hint under the box tells you what it heard). If a recording turns out to be the reply's own voice coming back through the microphone, it is dropped and the hint says so. ### Choosing a voice Spoken replies use the voices your browser gets from the operating system — instant and offline. The default voice is rarely the best one installed, and a better one is a settings change away: - **macOS** — *System Settings → Accessibility → Spoken Content → System voice → ⓘ*, pick a language and download an **Enhanced** or **Premium** voice (Zoe, Ava, Samantha Enhanced…). Safari and Chrome pick them up immediately; the built-in voices are excellent and need nothing else. - **Linux** — desktops often ship no browser voices at all. Install [Pied](https://github.com/Elleo/pied) (a Flatpak): it sets up the **Piper** neural voices for speech-dispatcher, which Chrome, Chromium and Firefox use. Chrome lists them as “… piper”; Firefox lists them by file name (`en_US-…-medium.onnx`) — same voices. They sound as good as the commercial ones. - **Windows** — *Settings → Accessibility → Narrator → Add natural voices* installs Microsoft's **Natural** voices (Ava, Andrew…). **Edge** exposes them to the web; Chrome and Firefox see only the classic SAPI voices (David, Zira), so on Windows use Edge for the best voice. - **iPhone / iPad / Android** — the phone's own voices work as they are; iOS gets better ones under *Settings → Accessibility → Spoken Content → Voices*. Whichever voice you pick, **Playback speed** (Settings → Status → Voice) speaks at 0.8× to 2×, and the **Mic & speaker check** on the same page records three seconds, plays them back, and shows the transcript — the fastest way to tell a microphone problem from a voice problem. ### Voice on the phone Dictation, live words, spoken replies, the modes, and the wake word all work on the phone exactly as on the Desktop. The settings behind them — wake word, playback speed, the Short/Medium/Long default — are set on the Desktop under **Settings → Status → Voice**, because Settings is Desktop-only. Replies speak with the phone's own voices, even offline. ## What the assistant can do These are the tools it can reach for. It picks them itself; you decide whether they run. **Observe — runs on its own, reads only:** | Tool | What it does | | --- | --- | | Search knowledge | Finds passages across your documents, or inside one named document. | | Read a document | Reads a document's text, a window at a time. | | Summarize a document | Summarizes a document of any length, whole or on a topic you name. | | List documents | Lists what's in your knowledge base. | | List tasks | Reads your planner. | | List schedules | Reads your schedules. | | Read schedule output | Reads what recent scheduled runs produced. | **Reviewed — proposed, then waits for your approval. Can be pre-authorized:** | Tool | What it does | | --- | --- | | Save a note | Writes a new document into your knowledge. | | Remember a fact | Adds a fact to Settings → Memory. | | Add a task | Adds a planner task. Asking twice for the same thing won't duplicate it. | | Complete a task | Ticks a task off; a recurring one rolls forward. | | Update a task | Edits a task's title, date, time, priority, repeat or notes. (Tags are yours to set in Planner; the assistant can't change them.) | | Search the web | Searches with your configured engine. | | Fetch a page | Reads one public web page as article text. | | Research the web | Searches, then reads the top results, in one step. | | Add a URL to knowledge | Fetches a page or PDF and saves its text. | | List email | Lists recent inbox messages, without bodies. | | Read an email | Reads one message. | | Create a schedule | Adds a recurring prompt. | | Update a schedule | Edits one. | | Enable or pause a schedule | Turns one on or off. | **Irreversible — always asks, every time, with an extra confirmation:** | Tool | What it does | | --- | --- | | Send an email | Sends from the connected Gmail account. | | Delete a task | Permanently deletes a planner task. | | Delete a schedule | Permanently deletes a schedule and its run history. | Two details worth knowing. First, a turn is bounded: the assistant gets **eight tool steps** and then must write an answer from what it has, saying what it couldn't finish — it can't loop forever. Second, the three schedule-writing tools **always ask inside a scheduled run**, even if you pre-authorized them in chat, so a schedule can never quietly grow more schedules. ## Knowledge A private, encrypted knowledge base. There are three ways in, all on the **Knowledge** page: - **Drop in files.** Drag them onto the box, or click it to choose. **PDF, Word (.docx), PowerPoint (.pptx), Excel (.xlsx), HTML, Markdown, CSV, JSON, and plain text** are understood — up to 200 files in one drop, 25 MB each. - **Paste a URL.** SmartBrain fetches the page, extracts the article text (not the navigation and ads around it), and saves that. A URL pointing at a PDF works too. You can ask Chat to do the same: *"add this PDF to my knowledge: …"*. - **Write a note.** A title and some text, typed straight in. **Big documents are welcome**: a several-hundred-page PDF is fine, and roughly a thousand dense pages of text are stored and reachable per document. Uploads don't block — they land right away, keyword search works within seconds, and meaning-search for a very large document fills in over the next few minutes in the background (it resumes by itself after a restart). While that is happening the page says so: *"Indexing for meaning search — 4 of 9 done. Keyword search already finds them."* Adding the same content twice is a no-op — SmartBrain recognises it and keeps the one copy rather than cluttering your results with duplicates. **What it can't read.** There is no OCR and no image or audio support. A scanned PDF — one that is pictures of pages rather than text — has no text to extract, so it is refused with *"no readable text found in that file"* rather than silently added empty. Word files get no page numbers either: `.docx` has no fixed pagination, so citations into one name the document but not a page. Search your knowledge three ways: - **Best** (default) — combines both of the below. Keyword search nails an exact name or invoice number; meaning search finds a paraphrase. Each misses what the other catches, so fusing them beats either alone. - **Keyword** — ranks by relevance: rare words count for more, and a long document can't win just by being long. Needs no model at all. - **Meaning** — matches by sense rather than wording, using an [embedding model](02-models.md). **Results are citations.** Every hit shows where it came from — *"Lease.pdf · p.12"* (a slide deck cites *slide 3*, a spreadsheet *sheet 2*) — and clicking it opens the document **at the passage that matched**, highlighted, rather than at the top. Chat answers that used your knowledge show the same source chips underneath the reply — click one to open the document at the cited passage. The chips come from what the assistant actually searched and read, not from what it *says* it did, so you can check any claim against the original. **Organize with tags.** Every document (and vault) has an inline tag editor — click the tags line on a row to add or change them, and click any tag chip to filter the list to it. Editing tags is instant and never re-indexes the document. Up to 20 tags per document. **Each row also has Rename and Delete**, and a checkbox for selecting documents — put the selection in a vault, **tag them all at once**, or **delete them all at once** from the bar that appears. Renaming re-indexes the document in the background, because the title is part of what search matches on; tagging doesn't. Documents that came from someone else's vault refuse bulk edits individually (a publisher update could overwrite them) and the result says so honestly — detach a copy to make it yours first. **Instant summaries.** In the background SmartBrain builds a summary of every document — summaries of its parts, reduced into a summary of the whole. That's what makes *"summarize this"* answer immediately even on a book-length file, and what lets a focused question ("summarize the fees") be answered in seconds instead of a full re-read. The page shows the progress: *"Preparing instant summaries — 6 of 9 documents ready."* It is built a piece at a time, resumes after a restart, and steps aside whenever you are chatting. **Reindex (semantic)** at the top of the document list re-embeds anything that needs it. Use it after you change the embedding model, or if a document you know is there never turns up in Meaning search. It works in batches and tells you what's left: *"Indexed 12 document(s) — 30 still to go, continuing in the background."* **Try it:** open **Knowledge**, drag in a document, and search it. Then ask **Chat** *"what does my knowledge say about …"* — the assistant searches it for you and tells you which file and page it got the answer from. ![The Knowledge page: add a document, then search it](assets/05-knowledge.png) ![Drop in a file, search it, open the cited passage, then ask Chat — answers cite their sources](assets/gifs/04-add-knowledge.gif) > Semantic search needs an embedding model. If results say *"Showing keyword > results"*, set one up — see > [Embeddings](02-models.md#embeddings-for-knowledge-search) — then **Reindex**. Your knowledge is also what external tools can read over [MCP](05-mcp.md). Group documents into **vaults** to scope a search — and to share them, privately or publicly: see [Share knowledge with Vaults](04-vaults.md). ### Follow websites (feeds) Any site that publishes an **RSS or Atom feed** — most blogs, news sites, and release pages do — can fill your knowledge by itself. On the **Knowledge** page, open *Follow a website*, paste the feed URL, and Subscribe. Add tags there (optional, comma-separated) and **every article the feed ever saves carries them** — so one click on the tag chip filters your whole library to that subject. The subscription gets its own vault, and every new post lands there as a searchable, citable document — so *"what did that blog say about X?"* works in Chat, and a schedule (below) can summarize the week's posts for you. What lands is what the feed carries: usually the title, link, and summary — some feeds include the full article, many don't. SmartBrain checks each feed **about every six hours, directly from this machine** — no server in the middle — and articles are encrypted at rest like every other document. Posts it has already saved are recognised and skipped, so a feed never duplicates itself. **Refresh** on a feed's row checks right now, and the row always shows when it last checked and what happened. Only the public URLs you pasted yourself are ever fetched. **Unsubscribe** stops the checking and removes the feed's vault. Its saved articles are your documents and stay in your knowledge — unless you choose *Delete articles too*. ## Planner ![Planner — tasks grouped Today / This week / by due date](assets/gifs/06-planner.gif) Task tracking, deliberately plain. A task is a title plus, if you want them: - a **due date** and a **time** on that date; - a **priority** — Low, Medium (the default), or High; - a **repeat** — none, Daily, or Weekly. Completing a repeating task rolls it forward to the next occurrence instead of closing it; - **tags**, comma-separated, and free-text **notes**. Tasks group themselves by when they are due: **Today & overdue**, **This week**, **Later**, **No date**, and **Done**. Anything overdue is called out in red. Each row has a checkbox to tick it off, **Edit** to change any field, and **Delete**. The assistant can read your tasks freely, and can add, complete, or edit one with your approval. Deleting a task is irreversible, so it asks every time. ## Neural Interface A dashboard of things you asked to watch. You describe what you want to see in Chat — *"show me AAPL every 5 minutes"*, *"a card with my open tasks by due date"* — and the assistant designs a card for it: where the data comes from, how it's processed, and how it's laid out. The engine then keeps it fresh on its own schedule. How a card comes to life: 1. **You pick the source.** The assistant suggests candidates — first from a small **vetted catalog** of free, keyless public APIs, and it says so; a source it found by web search is labeled that way instead. The choice is always yours — the approval card shows the exact address it will fetch, and that address is frozen: nothing can quietly change it later without asking you again. 2. **Preview first.** The assistant shows the card with sample data so you can approve the look ("make the total bigger" works — it's a conversation). 3. **Commissioning.** After you approve, the system runs the real pipeline and shows you the first live result — you confirm it's the *right* data, and one more clean run at cadence proves it's stable. Only then is the card live. 4. **It keeps itself honest.** Every refresh is checked against the shape of the data you validated. If the source changes or breaks, the card shows the last good result (dimmed, with a health chip) rather than something wrong — and a broken card tells you in Chat. Cards render from a fixed set of safe building blocks (text, numbers, bars, chips, lists) — fetched content is displayed as plain text, never as links, markup, or instructions. Sources are fetched with the same network guard as feeds; a locked vault stops everything. Cards refresh about every N minutes, never in real time — 1 minute is the floor, or 5 minutes for a card with a language-model step in its pipeline (below). **Charts.** The engine can keep a rolling history per card — a few numeric series, a few hundred points each — and the card can draw it: a sparkline, a gauge, or a change-since-last-refresh delta with its direction. **Display rules and alerts.** A card can carry small conditions. A display rule (*"turn the delta red when it goes negative"*) is applied while the card is prepared, so what you see is still a pure function of the data. An alert (*"tell me when the price drops"*) fires a notice — but only when its condition *becomes* true: it then stays quiet until the condition has been false again, and never fires twice within its cooldown (an hour unless you set one; five minutes is the floor). Fired alerts — and "broken" and "repaired itself" notices — arrive exactly like scheduled-run output: a notice in your open Chat, the badge on the Chat tab, and a durable copy on **Info** under *Neural Interface*. On Mac and Linux the desktop app also shows a system notification. While the vault is locked, nothing is shown anywhere. **Interpreted cards.** A card's pipeline can include one language-model step — *"summarize these headlines in a sentence"*. It runs on your **local** model only, never a cloud fallback; the model sees the fetched data inside a guarded fence with no tools, and must answer in an exact shape or the run fails safely to the last good result. Any card whose content passed through a model — this step, or a card whose source *is* a model instruction — wears an **Interpreted** chip, so you can always tell a model's reading from pure arithmetic. Cards with a language-model step refresh at most every 5 minutes. **Self-repair.** When a card fails because the data's shape changed, a local model can propose new data mappings — it is structurally unable to touch the source address, headers, schedule, or anything else you consented to. The fix runs as a trial: kept only if the next refresh passes the card's validated contract, automatically reverted otherwise, one attempt per breakage, all of it in the card's run history. When a trial sticks, a *"repaired itself"* notice tells you. Each card's **Repair settings** let you turn this off — and hold the next level: **Frontier repair (off by default).** If local repair has had its one try and the card still fails, and you have *both* switched on frontier repair for that specific card *and* connected Claude Code, Claude can be asked for a second opinion. Its answer is **proposed, never applied**: the card shows *"Fix proposed — review"* with the current and proposed mappings side by side, and your **Apply** runs it as the same keep-or-revert trial; **Dismiss** drops it. What's sent is bounded — the card's goal, its data mappings, the failure, and a short excerpt of the failing data; never your knowledge, never credentials. **More than APIs.** A card can also watch an ordinary **web page** (the readable text is extracted in a locked-down helper process — hostile HTML is never parsed inside the app), your own **knowledge** (*"a card of my notes about X"* — zero network), or an **image**: a weather-radar frame, a webcam still, a status badge. Images are accepted only as real raster formats, checked by file signature rather than by what the server claims; the pixels are stored encrypted and served only from your own app, and a failed refresh keeps the last good frame. **Composite cards** put your other cards' histories side by side — *"my stock next to my spending"* — with zero network: they read only your own cards' recorded series, can't nest, and degrade gracefully if a card they reference goes away. **Your MCP servers as sources.** Cards can pull from MCP servers **you** configure on **Settings → Connections (MCP)** — which is how database cards work: your own Postgres/SQLite MCP server keeps its credentials in its own process, and SmartBrain calls exactly one tool with the exact arguments shown on the approval card, frozen thereafter (changing them means re-approval). It never reads MCP config from disk and never feeds a server's tool listings to a model. One honest exception rides here: your own server's address may be localhost or LAN — the single deliberate, user-typed exception to the app's public-addresses-only network guard. See [MCP](05-mcp.md). **The Library.** Connect to a signed template library and install ready-made cards. Trust works like vault subscriptions: the publisher's key is pinned on first contact, every pack is verified against it, and the **fingerprint** is the identity you check — version rollbacks are refused, and a changed key blocks updates until you confirm the new fingerprint with your passphrase. Installing shows every source address up front and lands as a **draft**; your **Activate** is still the consent — a library can never start traffic by itself. When the library updates a template you use, the card offers *"Update available"*: applying shows what changed and goes through draft → commissioning again. The app checks the one URL you connected about once a day. You can also export most cards as a shareable template — credentials and your filled-in values are stripped; cards built on your schedules or your knowledge refuse, because that content is yours. Each card offers **Run now**, **Pause**, and **Delete**; drafts are marked "Preview — sample data" until commissioned. On a phone, Neural sits second in the tab bar. ## Schedules ![Schedules — run a prompt on a timer, then Run now](assets/gifs/07-schedule-a-prompt.gif) Run a prompt on a timer — e.g. "every morning, summarize my open tasks." A schedule fires an assistant turn on its cadence. The page has two tabs and opens on **Items**. **Create** takes a name, the prompt itself, how often it should **Repeat** — **Once**, **Hourly**, **Daily**, or **Weekly** — and when it should **First run**: **Now**, **In 1 hour**, or **Tomorrow**. Three presets (Check the news, Morning briefing, Weekly knowledge review) fill the form in if you'd rather start from one. **Items** lists what you have. Each row has a checkbox that enables or pauses it, **Edit** to change the prompt or cadence, **Run now** to fire it immediately, and a delete button. Two things to know: - Schedules only run **while the app is unlocked** (a locked vault can't decrypt or act — there's no background access to your data). - If a scheduled run wants to do something **dangerous** (send, delete, etc.), it **parks for your approval** in Activity just like in chat — it won't act alone. A run's output lands in three places: **in your open Chat** as a "Scheduled Item" notice, as a durable copy on the **Info** page, and as a badge on the Chat tab while results are unseen. ## Info Where scheduled output is kept. The **All** tab lists every run across every schedule, newest first; there is a tab per schedule for just that one's output, and a **Refresh** button. Each entry shows when it ran and what it produced — or, if the run wanted approval for something, *"Awaiting your approval — open Activity to review."* Nothing here is editable. It's the record: Chat's notice is easy to scroll past, so this is where you go when you want to find last Tuesday's briefing again. Manage the schedules themselves on the **Schedules** page. ## Email (Gmail) Connect a Gmail account with **your own** Google OAuth client. The whole flow is loopback-only — the authorization happens on your machine and nothing leaves it except the calls to Google. SmartBrain asks for just two scopes: **read** and **send** (no archive, delete, or label changes). It's optional; most people run SmartBrain without it. **One-time setup** (the in-app **Email** page walks you through these): 1. Open [Google Cloud Console → Credentials](https://console.cloud.google.com/apis/credentials), then **Create credentials → OAuth client ID**, and choose type **Desktop app**. A Desktop-app client needs **no redirect URL** — Google handles loopback automatically. 2. On the **OAuth consent screen**, add the `gmail.readonly` and `gmail.send` scopes and set **Publishing status** to **In production** — otherwise Google signs you out every 7 days. 3. In the app's **Email** page, paste the client **ID** and **secret** and click **Connect Gmail**. A Google sign-in opens; if it warns the app is "unverified" (it's your own client), choose **Advanced → Continue**, then approve the two scopes. Once connected, the **Email** page shows which address you're connected as, a **Compose** form (to, subject, message, **Send**), and your recent **Inbox** — click a message to read it in full. **Disconnect** removes the connection. - **You** sending from the app is a direct action. - The **assistant** sending email is an **Irreversible** tool — it always parks for your approval first. It can draft; you approve the send. It can also list and read your recent mail, both of which wait for approval the first time. Google sometimes signs SmartBrain out — every 7 days if you left the OAuth consent screen in testing rather than setting **Publishing status** to *In production*, and occasionally even if you didn't. The page then says **"Gmail needs reconnecting"** with a one-click **Reconnect Gmail**; you don't re-enter the client ID or secret. Reconnecting is done on the Desktop, and a paired phone starts working again by itself afterwards. ## Memory **Settings → Memory** holds who the assistant is for. Four things live there: - **Assistant name** — what it calls itself. - **Your name** — what it calls you. - **Custom instructions** — standing guidance for every conversation, e.g. *"Be concise. Prefer metric units."* - **Remembered facts** — a list you add to with **Remember** and prune with **Forget**. The assistant can propose one too, with your approval. All of it is encrypted and composed into every conversation, so it's the place to look when you wonder "why does it keep doing that?" — including any *"(learned) …"* facts self-improvement added (delete one to permanently reject it). ## Web search The assistant's web tools search with **DuckDuckGo by default — no key needed**. Under **Settings → Web search** you can pick which engine to use: - **Automatic** (the default) — the first engine you have configured, with DuckDuckGo last. If one is down, the next takes over. - **SearXNG** — an instance you host or trust. Paste its URL; its JSON API must be on. - **Brave Search** or **Tavily** — bring your own key. Both are stored encrypted, like cloud-provider keys. - **DuckDuckGo** — no key, always available as the fallback. Searches only happen when the assistant actually uses the web tools in a turn; see [Privacy & security](07-privacy-security.md) for exactly what leaves your machine. ## Self-improvement SmartBrain can review its own recent performance and carefully improve — **off by default**, and switched on under **Settings → Self-improvement**. On the cadence you choose — every 2, 4, 8 (default), or 24 hours — under Settings → Self-improvement (while unlocked) it scores Chat, Knowledge, and Tools from private, on-device telemetry. Quiet periods stay silent; when something needs attention you get a short digest in the chat feed. From a flagged period it may act — always within hard bounds: - **Learned preferences** — a local model (never a cloud one, and only from messages *you* wrote) may learn one durable preference, applied as a visible *"(learned) …"* fact in Settings → Memory, measured against your satisfaction, and **auto-reverted if it doesn't help**. Deleting the fact yourself permanently rejects it. - **Suggested routines** — an ask you repeat on a daily/weekly rhythm becomes a ready-made schedule **waiting for your approval in Activity**; decline it once and it is never offered again. - **Knowledge gaps** — searches your knowledge couldn't answer get named in the digest. - **Prompt optimizer** (its own switch) — learns how kinds of requests go and may steer them with a short guidance note; a strategy watches in *shadow* first, goes live only after a measured trial, is turned off automatically if it doesn't help, and guided answers always show a small **"guided · …"** chip. One change is ever on trial at a time, everything is reversible, every applied or reverted change is announced, and **Settings → Self-improvement** shows the record of what it has done under **What it has done**. ## Usage & cost A running estimate of what your **cloud** models cost. Pick a **Range** — Today (the default), the last 5, 10 or 30 days, or a custom pair of dates — and you get a row per model with its calls, prompt and completion tokens, and estimated cost, plus a total. Pricing comes from each provider's live figures. **Local models (Ollama, MLX) are free** and say `free` in the cost column. Usage appears here after you chat with a model. None of it leaves your machine — it's computed locally from your own token counts, and the only network call is a local fetch of the price list from the on-device gateway. ## Activity ![The safety loop — the assistant proposes, you approve in Activity](assets/gifs/05-approve-an-action.gif) Your audit and approvals view. Two parts: - **Awaiting your approval** — a card per proposed action, naming the tool, what it would do, and whether it is reversible (the same card appears in the chat itself, and resolving it there is identical). **Approve** or **Deny** it. **Always allow** approves it and stops asking for that tool from then on (for the URL tools, for that tool **on that site** — the list shows each allowed site as its own row). Denying an action holds for the rest of that run: the assistant is told, and an identical retry is refused instead of asking you again; anything pre-authorized this way is listed under **Always allowed**, where **Stop allowing** takes the permission back. Irreversible tools can't be pre-authorized — they ask every time, with an extra confirmation. When the action you resolve belongs to a **scheduled** run, the run finishes on the spot and its answer lands in the Scheduled updates feed — no need to trigger the schedule again. ![The Always allowed list on the Activity page — a pre-authorized tool with its Stop allowing button](assets/08-always-allowed.png) - **History** — the record of every tool the assistant ran or tried to run: which tool, its risk tier, what you decided, whether it succeeded, when, and a summary of what it was given. Any error it hit is shown too. Arguments and results are encrypted at rest, and secrets are stripped before anything is recorded. Nothing here can be edited or deleted from inside the app — see [Design limits](09-design-limits.md) for what that does and doesn't guarantee. An action left unanswered **expires after an hour**, and **locking cancels everything pending** — in both cases the action never runs at all. When you deny one instead, the assistant is told it wasn't approved and carries on from there; it is never told an action succeeded when it didn't. ## Settings Everything you configure lives under **Settings**, as a row of tabs. It is Desktop-only: a paired phone shows a note to make changes on the Desktop instead. Opening Settings lands on **Cloud providers**; the tabs, in order: ### Status A live view of what the app is doing right now, refreshed every few seconds — the first place to look when something seems off. - **App** — the version you are running and whether the vault is unlocked. - **Voice** — the built-in dictation model (download progress, or **Retry download** if it failed), which dictation engine is in use (**built-in** or **your audio server**), the server voice if you set one, **Playback speed** for spoken replies (0.8× to 2×), your **Wake word** with **Test recognition** and accepted spellings, the how-to-dictate notes, and the **Mic & speaker check**: **Test now** records three seconds with a level bar, plays them back, and shows what dictation heard — the fastest way to tell a microphone problem from a voice problem. See [Voice](#voice). - **Storage & memory** — disk used by everything SmartBrain stores (with the folder), by the encrypted database and the voice model, the app's peak memory, and this browser's cache. - **Model servers** — whether Ollama, MLX, and MLX embeddings are configured (live reachability is on the Local models tab). - **Knowledge**, **Schedules**, **Feeds**, **Remote access** — document and embedded-chunk counts, enabled schedules, feed subscriptions (with a **failing** count when a feed can't be fetched), and paired devices. ### Cloud providers API keys for **OpenAI**, **Anthropic**, and **Google (Gemini)** — stored encrypted, never shown back, replace or remove only. Saving a key discovers its chat models. Details: [Connect a model](02-models.md#cloud-providers-your-api-keys). ### Local models Optional servers on your own machines: **Ollama**, **MLX**, **MLX embeddings**, and the **Voice** card (an optional audio server and server voice — the built-in dictation needs none of this). Each card shows **connected / unreachable / off**, offers **Connect it** when a server is detected on this machine, and has a **Server on another machine** option. Details: [Local models](02-models.md#local-models-yours-on-this-machine-or-another-one-you-own) and [Voice](02-models.md#voice-dictation-and-spoken-replies). ### Model routing Which model serves each job — chat, the agent (schedules and background tasks; **Same as Chat** by default), embeddings for search — and **Model context length** per routed model. Details: [Which model does what](02-models.md#which-model-does-what-model-routing). ### Web search Which engine "search the web" uses: **Automatic**, **SearXNG** (your own instance), **Brave Search**, **Tavily**, or **DuckDuckGo** (no key). Keys are stored encrypted. Details: [Web search](#web-search). ### Memory The assistant's name, your name, custom instructions, and the list of remembered facts (**Remember** / **Forget**). Details: [Memory](#memory). ### Self-improvement **Self-review** cadence (**Off / 2h / 4h / 8h / 24h**), the **Prompt optimizer** switch, and the record of everything it proposed, applied, kept, or reverted. Both are off until you turn them on. Details: [Self-improvement](#self-improvement). ### Connections (MCP) The endpoint and access token that let a desktop AI client read your knowledge over MCP — **Generate**, **Copy**, **Regenerate**, **Revoke**. Details: [MCP access](05-mcp.md). ### Remote access **Pair a new phone** (a QR code that opens the app, plus an eight-character code that expires) and the list of **paired devices** with **Revoke**. Details: [Remote access](08-remote-access.md). ### Account & Data **Change passphrase** (or set a new one after a Recovery-Key unlock), **Export & backup** (readable JSON export, or the full encrypted database), **Chat trash** (restore or empty; 30-day retention), and **Restore** from a backup file, applied on the next restart. Details: [Backup & recovery](06-backup-recovery.md). ## Next - [Share knowledge with Vaults](04-vaults.md) — sealed shares, public publishing, subscriptions. - [Connect external tools](05-mcp.md) via MCP. - [Backup & recovery](06-backup-recovery.md). - [Design limits](09-design-limits.md) — why some of the boundaries above are where they are.