# ๐Ÿฆ€ OpenBiliClaw **A general-purpose personalized content discovery Agent โ€” runs locally, understands you across platforms, built only for you** [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/) [![Release](https://img.shields.io/github/v/release/whiteguo233/OpenBiliClaw?filter=openbiliclaw-v*&style=flat-square&label=Release&color=success)](https://github.com/whiteguo233/OpenBiliClaw/releases/latest) [![CI](https://img.shields.io/github/actions/workflow/status/whiteguo233/OpenBiliClaw/ci.yml?branch=main&style=flat-square&label=CI)](https://github.com/whiteguo233/OpenBiliClaw/actions/workflows/ci.yml) [![LINUX DO](https://img.shields.io/badge/LINUX_DO-Community-black?style=flat-square&logo=linux)](https://linux.do/) [![Discussion](https://img.shields.io/badge/LINUX_DO-Discussion-orange?style=flat-square&logo=discourse)](https://linux.do/t/topic/1978894) [![Chrome Web Store](https://img.shields.io/chrome-web-store/v/cdfjfkdjjhdaccbldipkjhpibnfbiamg?style=flat-square&label=Chrome%20Web%20Store&logo=googlechrome&logoColor=white&color=4285F4)](https://chromewebstore.google.com/detail/cdfjfkdjjhdaccbldipkjhpibnfbiamg) [![Gitee Mirror](https://img.shields.io/badge/Gitee-Mirror-C71D23?style=flat-square&logo=gitee&logoColor=white)](https://gitee.com/whiteguo233/OpenBiliClaw) [Homepage](https://whiteguo233.github.io/OpenBiliClaw/) | English | [ไธญๆ–‡](README.md)
## OpenBiliClaw in 10 Seconds A local-first AI discovery agent that learns your taste across Bilibili, Xiaohongshu (RedNote), Douyin (Chinese TikTok), YouTube, X, Zhihu, Reddit, Bangumi, and the open web โ€” without handing your profile to another platform. | Cross-platform | Local-first | Trainable | |---|---|---| | Bilibili / Xiaohongshu / Douyin / YouTube / X / Zhihu / Reddit / Bangumi / Web | Data stays in your local SQLite by default | Likes, dislikes, and chat feedback shape future recommendations |

Install the browser extension ยท Deploy the local backend with an AI coding agent

Star the project if you like the direction.

OpenBiliClaw local-first cross-platform AI discovery agent demo: platform signals, local backend, taste profile, reasoned cards, and feedback loop

## Quick Start Four steps for most users. Firefox, Docker, scripted, and manual setup paths all live in [Setup Details](#setup-details). 1. **Install the extension** โ€” one-click from the [Chrome Web Store](https://chromewebstore.google.com/detail/cdfjfkdjjhdaccbldipkjhpibnfbiamg) (auto-updates), or download the zip from [Latest Release](https://github.com/whiteguo233/OpenBiliClaw/releases/latest) for the newest build (the store listing can lag a few days behind). 2. **Install the backend** โ€” grab the desktop installer from the same [Latest Release](https://github.com/whiteguo233/OpenBiliClaw/releases/latest) (macOS `.dmg` / Windows `.exe`, works out of the box, lives in the menu bar / tray). Each platform ships two variants: the **lean** installer (default; downloads the bge-m3 embedding model on first launch) and the **`-with-embedding`** installer (bge-m3 baked in, ~1.1GB, offline-ready) โ€” pick with-embedding for a poor / offline network, lean otherwise. Or, to customize or edit the source, paste this into Claude Code / Codex CLI / Cursor or another AI coding agent: ```text Please follow https://raw.githubusercontent.com/whiteguo233/OpenBiliClaw/main/docs/agent-install.md to deploy the OpenBiliClaw backend for me (use Bash `curl` to fetch the document, NOT WebFetch โ€” WebFetch summarises markdown and drops critical commands). ``` 3. **Connect a source** โ€” log in to [Bilibili](https://www.bilibili.com) (the default init source), or choose Xiaohongshu / Douyin / YouTube / X / Zhihu / Reddit. Bangumi discovery needs no login; enter a public username only if you want public collections to seed the profile. 4. **Open the UI** โ€” visit `http://127.0.0.1:8420/web`, or scan the extension QR code to open `http://:8420/m/` on your phone and save it to your home screen. ## Why OpenBiliClaw? > The name comes from Bilibili (`Bili` = Bilibili, `Claw` = "the claw that grabs content for you") โ€” the project started as a Bilibili-only tool. Since v0.3.0 it has evolved into a general cross-platform Agent covering Bilibili / Xiaohongshu / Douyin / YouTube / X / Zhihu / Reddit / Bangumi and the open web, with more platforms on the roadmap. Recommendation systems are essentially a **middleman** โ€” the platform sits between millions of videos and millions of users, matching and distributing content at scale. Modern systems are far more sophisticated than "just optimizing CTR": they jointly weigh click-through rate, completion rate, like/coin probability, dwell time, user retention, creator ecosystem health, ad revenue, and a dozen other objectives, compressing them into a single weighted ranking score. Sounds scientific, but here's the catch: **the weights are set by the platform, and the optimization targets ultimately serve the platform** โ€” user satisfaction is valued as a means to retention and monetization, not as an end in itself. You think you're choosing content, but really the middleman decides what you get to see. The result: recommendations look more and more like what you've already watched, and the occasional surprise is pure luck. **OpenBiliClaw is fundamentally different.** It's a locally-running AI Agent that doesn't care what everyone else watches. Instead, it understands **who you are**: ### ๐Ÿง  Understands *why* you like things, not just *what* you've watched It infers your MBTI, cognitive style, and deep psychological needs from your behaviour, building a five-layer soul profile (Event โ†’ Preference โ†’ Awareness โ†’ Insight โ†’ Soul). It's not matching video tags โ€” it's understanding you as a person. ### ๐Ÿ”ฎ Actively breaks your filter bubble This is the core differentiator: the system **guesses domains you might enjoy but have never explored**. Someone into mechanical watches might love architectural aesthetics; a quantum physics viewer might resonate with philosophy โ€” it uses psychological bridging logic to proactively explore, promotes correct guesses to real interests, and quietly retires wrong ones. ### ๐Ÿ”’ 100% local, 100% yours All data lives in a single SQLite file on your disk. LLM calls use your own API key by default, with an experimental option to reuse local Codex CLI ChatGPT OAuth credentials. No cloud, no accounts, no one else can see your profile. How this Agent grows is entirely your call โ€” send feedback, chat with it, swap LLMs, edit the database, whatever you want. > ๐Ÿ’ก **How it compares** > > | | Bilibili Official | Keyword Filter Plugins | OpenBiliClaw | > |---|---|---|---| > | Recommendation logic | Collaborative filtering | Tag matching | Psychological profiling + 5-layer memory | > | Content sources | Single platform | Single platform | Cross-platform: Bilibili ยท Xiaohongshu ยท Douyin ยท YouTube ยท X ยท Zhihu ยท Reddit ยท Bangumi ยท more | > | Filter bubble | Gets narrower | Doesn't address it | Speculative interests actively break it | > | Data ownership | Platform-owned | Usually cloud | 100% local | > | Explains why | "Guess you'll like" | None | Friend-like explanations | > | Customizable | No | Low | Swap LLMs / edit profile / write Skills | ## ๐Ÿ“ธ Feature Preview Three core surfaces: the browser extension handles in-page interaction and login sessions, the Desktop Web (`/web`) gives you a big-screen recommendation home, and the Mobile Web (`/m`) is built for phones. Both web surfaces only call your local API โ€” cookie sync and platform tasks still run through the extension.

Smart Recommendations
Friend-like explanations of why you'd enjoy it

Soul Profile
Deep personality analysis in natural language

Structured Traits
MBTI ยท core traits ยท deep needs

Chat Tuning
Tell it what you want to see
### ๐Ÿ–ฅ๏ธ Desktop Web Preview After starting the backend, open `http://127.0.0.1:8420/web` (or just `http://127.0.0.1:8420/`, which redirects automatically) for a full-screen recommendation dashboard.

Desktop Home
Delight hero ยท recommendation grid ยท friend-like reasons

Recommendation Card Grid
Cover + reason ยท like / skip / watch later / favorite / chat

Profile + Live Dashboard
Sidebar runtime board + activity ยท personality sketch ยท core traits ยท MBTI
### ๐Ÿ“ฑ Mobile Web Preview

Recommendations
Delight + pool status ยท friend-like reason
View / like / later / save / not interested / chat

Profile
Personality sketch ยท core traits ยท deep needs ยท MBTI

Chat
Shared main chat history with the extension
More screenshots

Recommendation Feedback
Like / more like this / less / not interested

Values & Interests
Inner drivers ยท speculative interest directions

Cognitive Style
Information processing ยท content taste
## Recent Updates ๐Ÿ“Œ Latest: **v0.3.191 (2026-07-30)** - **Saving configuration no longer interrupts long-running dialogue work** โ€” hot reload now waits for active jobs to finish safely and clearly reports when work continues beyond the frontend timeout. - **Pending-chat open and defer actions are more reliable** โ€” busy workers trigger an automatic retry, and deferring one topic no longer blocks the next pending item. - **Web realtime connectivity is steadier** โ€” idle heartbeats, transient-disconnect status, and automatic reconnection prevent brief network hiccups from looking like a backend outage. Full changelog: [docs/changelog.md](docs/changelog.md). ## Community
QQ user community QR code
QQ Community
Discord community QR code
Discord Community
Scan or click to join โ€” this invite does not expire.
## Setup Details For most users, setup is four steps: install the extension, ask an AI coding agent to deploy the backend, log in to the content platforms in the same browser, and optionally open the Mobile Web app from your phone. ### 1. Install the browser extension The extension is the main interface. It shows the sidebar on Bilibili, Xiaohongshu, Douyin, YouTube, X, and Zhihu, records your feedback, and lets the local backend safely reuse browser sessions for logged-in tasks such as Zhihu and Reddit init / discovery. Built on Manifest V3, the extension works in any Chrome-compatible browser โ€” **Chrome, Edge, Brave, Arc, Vivaldi, Opera**, and more. **Recommended ยท download the latest build from the Latest Release aggregate page** (gets the newest features and fixes โ€” the Chrome Web Store listing usually lags by a few days to a couple of weeks due to review scheduling): 1. Open [OpenBiliClaw Latest Release](https://github.com/whiteguo233/OpenBiliClaw/releases/latest), the newest user-facing aggregate `openbiliclaw-v*` release 2. Chrome / Edge / Brave users download `openbiliclaw-extension-v*.zip`; Firefox users install `openbiliclaw-extension-v*-firefox.xpi` when it is present, otherwise download `openbiliclaw-extension-v*-firefox.zip` and load it temporarily through `about:debugging` 3. Open the extensions page (Chrome: `chrome://extensions/` ยท Edge: `edge://extensions/` ยท Brave: `brave://extensions/`), enable "Developer mode" in the top right 4. Chrome / Edge / Brave users drag the downloaded `.zip` file into the page to install; Firefox `.xpi` files install directly, while the temporary zip must be unzipped before loading `manifest.json` **Convenient ยท one-click from the Chrome Web Store** (the browser keeps it auto-updated โ€” best if you don't want to update manually; downside: the version can lag behind Releases): > ๐Ÿ‘‰ **[Install OpenBiliClaw on the Chrome Web Store](https://chromewebstore.google.com/detail/cdfjfkdjjhdaccbldipkjhpibnfbiamg)** โ€” click "Add to Chrome". Extension updates depend on the install channel: Chrome Web Store / Edge Add-ons (and a future listed AMO build) are updated by the browser; GitHub Release Chrome zips / Firefox signed XPIs / Firefox temporary zips, developer-mode loads, and Firefox temporary installs must download the new package and reload it manually. The backend "auto update" switch only updates the local backend source checkout, not the browser extension.
Firefox users: regular install and temporary debugging (Firefox 140+) Firefox uses `sidebar_action` instead of Chrome's `sidePanel`, so releases ship separate Firefox artifacts: - `openbiliclaw-extension-v*-firefox.xpi`: signed through Mozilla AMO unlisted signing when AMO signing is enabled and credentials are available, installable directly in regular Firefox Release / Beta. - `openbiliclaw-extension-v*-firefox.zip`: unsigned development package for `about:debugging` temporary loading or AMO signing input. Installing this zip directly in regular Firefox reports that the add-on could not be verified. For temporary debugging or source builds: ```bash unzip openbiliclaw-extension-v*-firefox.zip -d openbiliclaw-firefox # Or build from source git clone https://github.com/whiteguo233/OpenBiliClaw.git cd OpenBiliClaw/extension npm install npm run build:firefox # writes dist-firefox/ npm run package:firefox # also produces unsigned openbiliclaw-extension-v*-firefox.zip # With AMO credentials configured, sign it into the installable XPI: # AMO_JWT_ISSUER=... AMO_JWT_SECRET=... npm run sign:firefox:only ``` Then: 1. Open `about:debugging#/runtime/this-firefox` 2. Click "Load Temporary Add-onโ€ฆ" 3. Pick `manifest.json` from the unzipped directory, or `extension/dist-firefox/manifest.json` after a source build Caveat: temporary add-ons disappear on Firefox restart; regular users should prefer the signed `.xpi` when the release provides one.
### 2. Deploy the backend (two options) Most users: the **desktop installer** is the least effort. Want to edit the source, swap LLMs, or customize deeply? Use the **AI one-line deploy**. #### Option A: Download the desktop installer (experimental, easiest) Grab the installer for your OS from the `openbiliclaw-v*` aggregate [Latest Release](https://github.com/whiteguo233/OpenBiliClaw/releases/latest). The aggregate page shows: - Current backend source tag: `backend-v*` - Current extension release: `extension-v*`, with `openbiliclaw-extension-v*.zip` / `openbiliclaw-extension-v*-firefox.zip` (Firefox temporary debugging); AMO signing-enabled releases also include `openbiliclaw-extension-v*-firefox.xpi` (regular Firefox install) - Current desktop installer release: `desktop-v*`, with available `.dmg` / `.exe` assets when the same-version desktop channel has shipped; missing channels are shown as unpublished instead of being backfilled from a previous release - **macOS**: download the DMG that matches your Mac: `OpenBiliClaw-macos-v*-arm64.dmg` for Apple silicon, or `OpenBiliClaw-macos-v*-x64.dmg` for Intel when the release provides it. The recommended path is to double-click `ๅฎ‰่ฃ…ๅนถๅฏๅŠจ Install OpenBiliClaw.command`: it verifies the new bundle, quits the old instance, atomically replaces the app in Applications, and launches the version just installed. Traditional drag-and-drop remains available, but upgrades must quit the old version first and reopen the replacement manually. - **Windows**: download `OpenBiliClaw-windows-*-Setup.exe` โ€” double-click to install. After a successful install or upgrade, Setup stops the old instance and automatically launches the newly installed version from the installation directory (including silent installs). It bundles local Ollama + `bge-m3` embedding (works out of the box) plus the default source dependencies, including X's `twitter-cli` and Reddit's `rdt-cli` (Reddit's rdt command backend prefers the connected extension's synced `reddit_session`; `rdt login` remains a manual fallback, and unauthenticated runs fall back to extension tasks). It lives in the **macOS menu bar / Windows system tray**; right-click for "Open Web UI / View runtime logs / Quit". Data uses the same directory as the AI / script installers: `~/OpenBiliClaw` (macOS / Linux) / `%USERPROFILE%\OpenBiliClaw` (Windows), and survives upgrades and uninstalls. Data from older packaged builds under `~/Library/Application Support/OpenBiliClaw` / `%LOCALAPPDATA%\OpenBiliClaw` is copied back on first launch without overwriting existing files. If a broken `config.toml` / `config.local.toml` prevents startup, the desktop package backs the bad file up as `*.invalid`, regenerates the default config, then opens `/setup/` so initialization can run again; `data/` is left untouched. > โš ๏ธ **macOS security blocking (the app isn't signed / notarized yet)**: > - The current Release is ad-hoc signed but not notarized. On first launch, if macOS blocks either the install helper or the app, right-click / Control-click that item โ†’ "Open" โ†’ click "Open" again in the dialog; or allow it under "System Settings โ†’ Privacy & Security" with "Open Anyway". > - If macOS says "`OpenBiliClaw.app` is damaged and can't be opened", it is usually the download quarantine attribute. After confirming the package came from this project's Releases, run: > > ```bash > APP="/Applications/OpenBiliClaw.app" > xattr -dr com.apple.quarantine "$APP" > ``` > > Then open the app again. > - **Windows**: on the SmartScreen prompt, click "More info โ†’ Run anyway". > > This is an **experimental pre-release**: unsigned, rolling with the backend version, best for trying it fast without the command line. To hack on the source, use Option B. #### Option B: AI one-line deploy (customizable / editable source) Paste this whole prompt into Claude Code, Codex CLI, Cursor, Windsurf, or another AI coding agent. The parenthetical note is for the agent; you do not need to understand it. ```text Please follow https://raw.githubusercontent.com/whiteguo233/OpenBiliClaw/main/docs/agent-install.md to deploy the OpenBiliClaw backend for me (use Bash `curl` to fetch the document, NOT WebFetch โ€” WebFetch summarises markdown and drops critical commands). ``` The agent will clone the repo, install dependencies, start the backend with the LAN-accessible default bind (`0.0.0.0:8420`), run a health check, and ask a few questions with defaults. Before auto-init, it verifies that the ordered global LLM instance chain and the independent embedding service answer real lightweight calls; if either fails, init is blocked until you fix the service. If unsure, pick the default. Xiaohongshu, Douyin, YouTube, X, Zhihu, and Reddit signals are used in the initial profile only when you explicitly opt in. Bangumi discovery needs no login; public collections seed the profile only when you enter a public username. Chrome Web Store / AMO builds only declare local-backend permissions by default. When you select a protocol and enter another LAN or remote endpoint, the browser requests `scheme://host/*`; WebExtension host permissions cannot be port-scoped across browsers, while actual requests remain pinned to the configured port. Public hosts require HTTPS. Enable the default-off device flow first with `ext-key generate` and `ext-key enable`. ### 3. Log in to content platforms in the same browser By default, log in to [Bilibili](https://www.bilibili.com) and keep Bilibili selected to build the first profile and recommendations. If you do not want Bilibili, deselect it during init and select another logged-in source such as [Xiaohongshu](https://www.xiaohongshu.com), [Douyin](https://www.douyin.com), [YouTube](https://www.youtube.com), [X](https://x.com), [Zhihu](https://www.zhihu.com), or [Reddit](https://www.reddit.com), or choose Bangumi and enter a public username. Keep at least one source that can return profile signals. Bangumi without a username still supports anonymous discovery, but cannot initialize a profile by itself. ### 4. Open Desktop or Mobile Web The backend serves both a desktop and a mobile Web UI. Neither syncs cookies or crawls pages โ€” they only call your local API. ```bash openbiliclaw start ``` - **Desktop**: open `http://127.0.0.1:8420/web` (or `http://127.0.0.1:8420/`, auto-redirects). Two-column editorial layout with recommendations, profile, chat, messages, and settings all on one page. - **Mobile**: click the phone icon in the extension header to scan the QR code, or type `http://:8420/m/` manually. Best for browsing recommendations, profile, and chat on your phone. > During `openbiliclaw init`, you'll be asked whether to allow LAN access (default Y). If you chose N or want to change it later, edit `[api].host` in `config.toml` (`0.0.0.0` = LAN-reachable over available IPv4 and IPv6, `127.0.0.1` = local only). QR links prefer IPv4 and automatically use a bracketed IPv6 literal when IPv4 is unavailable. After opening `/m/`, save it as a home-screen shortcut: on iPhone / iPad, use Safari's Share menu and choose "Add to Home Screen"; on Android Chrome / Chromium browsers, use the menu item "Install app" or "Add to Home screen". LAN HTTP may only create a shortcut in some Android browsers; full PWA install prompts are more reliable behind HTTPS in a trusted local setup. The app has five bottom tabs: Recommendations, Watch Later, Favorites, Profile, and Chat. Recommendations support reshuffle, load more, like, not interested, watch later, favorite, comments, and contextual chat. Watch Later and Favorites manage your saved lists. Profile shows the personality sketch, core traits, interests, and cognition updates. Chat shares the main chat history with the extension.
No AI agent: run the one-line installer yourself macOS / Linux / WSL2 (Bash): ```bash curl -fsSL https://raw.githubusercontent.com/whiteguo233/OpenBiliClaw/main/scripts/install.sh | bash ``` Native Windows (PowerShell, no Docker or WSL2 required): ```powershell [Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12; iwr https://raw.githubusercontent.com/whiteguo233/OpenBiliClaw/main/scripts/install.ps1 -UseBasicParsing | iex ``` The script needs `git` and Python 3.11+. It clones the repo, then asks for the preferred LLM instance, embedding, Bilibili cookie, and Xiaohongshu / Douyin / YouTube opt-ins before installing dependencies or starting the backend. Once confirmed, it starts the backend, verifies the global LLM instance chain and embedding service, then runs init to build the first profile and discovery pool. X, Zhihu, Reddit, and Bangumi can be enabled explicitly afterward in `/setup/` or settings; Bangumi needs no login, while public-collection init requires a public username. If unsure, press Enter or choose the default.
Advanced: Docker deployment Good if you already have Docker installed; ships with an Ollama embedding sidecar. The prebuilt image needs no source checkout: ```bash mkdir -p ~/openbiliclaw && cd ~/openbiliclaw curl -fsSLO https://raw.githubusercontent.com/whiteguo233/OpenBiliClaw/main/docker-compose.prebuilt.yml docker compose -f docker-compose.prebuilt.yml up -d # then open http://127.0.0.1:8420/setup/ to finish initialization ``` Or paste this into an AI coding agent for the terminal wizard + auto-init path: ```text Please follow https://raw.githubusercontent.com/whiteguo233/OpenBiliClaw/main/docs/docker-deployment.md to deploy the OpenBiliClaw backend via Docker Compose (use Bash `curl` to fetch the document, NOT WebFetch). ``` Source builds, upgrades, and troubleshooting: [Docker Deployment Guide](docs/docker-deployment.md).
Advanced: multi-source login and plugin path OpenBiliClaw does not store your platform passwords or bypass login. It reuses the browser sessions you already control and only fetches content you can see. | Source | How to log in | What happens if you do not | |---|---|---| | **Bilibili** | Log in normally at https://www.bilibili.com in the extension browser | Watch history / favorites / following are unavailable, so the profile is much weaker | | **Xiaohongshu** | Log in normally at https://www.xiaohongshu.com in the same browser | Xiaohongshu discovery and detail fetches are unavailable | | **Douyin** | Log in normally at https://www.douyin.com in the same browser | `init --yes-douyin`, `fetch-douyin`, and `discover --source douyin` search / hot / feed may return 0 items | | **YouTube** | Log in normally at https://www.youtube.com in the same browser | `init --yes-youtube` and `fetch-youtube` may return 0 items; `import-youtube` can still import Google Takeout data | | **X (Twitter)** | Log in normally at https://x.com in the same browser | `init --yes-x`, `fetch-x`, and X discovery return nothing (server-side replay needs `auth_token`+`ct0`, auto-synced by the extension after login) | | **Zhihu** | Log in normally at https://www.zhihu.com in the same browser | `init --yes-zhihu`, `fetch-zhihu`, `discover --source zhihu`, and `discover-zhihu*` return nothing | | **Reddit** | Log in normally at https://www.reddit.com in the same browser; the extension syncs `reddit_session` for backend-installed rdt-cli, and `rdt login` is only a fallback when the extension is unavailable | `fetch-reddit --mode bootstrap` returns no init signals; without a synced rdt credential, the rdt path falls back to extension tasks | | **Bangumi** | No login required; optionally enter a public username for public collections, or a personal token for private ones; the extension only does account identity recognition on bgm.tv / bangumi.tv (no cookies, no browsing capture) | Without a username, Bangumi cannot be the only profile-init source, but anonymous search/ranked/date discovery still works | Xiaohongshu, Douyin, YouTube, and Zhihu use Chrome extension tasks; Reddit defaults to backend-installed rdt-cli for steady-state discovery and keeps the extension for init signals; X discovery uses server-side cookie replay. None of these read paths needs an extra CDP debugging Chrome. Reddit/X, YouTube, Xiaohongshu, Douyin, and Zhihu native-save executors are wired 6/6 and fixture-tested; in the 2026-07-14 real-account regression, every platform's favorite and watch-later/favorite-fallback path finished `synced/already_synced`. `[sources.browser].cdp_url` remains available only for generic Web / custom webpage fetching.
Advanced: local embedding / Ollama If you do not want a separate embedding API key, or remote embedding quota is an issue, install Ollama once and use local `bge-m3`: ```bash # macOS # Install and launch the official Ollama.app; it creates the ollama CLI link. open https://ollama.com/download/mac # Linux curl -fsSL https://ollama.com/install.sh | sh && ollama serve & ``` macOS / Windows users can install the official app from [ollama.com/download](https://ollama.com/download). Start Ollama, then run: ```bash uv run openbiliclaw setup-embedding ``` The wizard pulls `bge-m3` (~1.1GB, CPU-only is fine) and writes the config.
Advanced: manual installation and discovery debugging > Human reference: [docs/agent-install.md](docs/agent-install.md) (short agent-facing contract) and [docs/agent-deployment.md](docs/agent-deployment.md) (long-form troubleshooting). #### Manual installation ```bash # Clone git clone https://github.com/whiteguo233/OpenBiliClaw.git cd OpenBiliClaw # Using uv (recommended) uv sync # Or using pip python -m venv .venv source .venv/bin/activate pip install -e ".[dev]" ``` #### Manual configuration ```bash # Copy config template cp config.example.toml config.toml # Edit config (set LLM API keys, etc.) vim config.toml ``` #### Run ```bash # One-command init (fetch history ยท build profile ยท first discovery) openbiliclaw init # Optional: enable local Ollama as an independent embedding provider openbiliclaw setup-embedding # Manual content discovery openbiliclaw discover # Optional: Douyin discovery (requires [sources.douyin]; search / hot / feed are triggered from the home page via DOM) openbiliclaw discover --source douyin # Optional: standalone Douyin search / hot / feed recall debugging openbiliclaw discover-douyin --keyword mechanical-keyboard --source search,feed --no-cache --no-evaluate # Get recommendations openbiliclaw recommend # View user profile openbiliclaw profile ``` Developers can also build the extension from source: ```bash cd extension npm install npm run package ```
## ๐Ÿค– Integrate with OpenClaw / AI Coding Agents This repo ships a [workspace skill](skills/openbiliclaw-adapter/SKILL.md). Point any skill-aware AI coding agent (OpenClaw / Claude Code / Codex CLI / Cursor, etc.) at this checkout and it can drive your local OpenBiliClaw directly. ### What you get after integration - โœจ **Proactive recommendations** โ€” the system continuously discovers content in the background; when it finds a high-scoring surprise, it pushes to OpenClaw via WebSocket โ€” **you don't have to ask** - ๐Ÿ”ฎ **Proactive interest probing** โ€” the system guesses you might be into a new domain, generates a hypothesis and a question, and has OpenClaw come ask you "does this direction resonate?" โ€” your answer automatically refines the profile - ๐Ÿงญ **Proactive avoidance probing** โ€” the system can also ask whether a low-quality form, style boundary, or topic shape is something you want to avoid; OpenClaw uses `next-avoidance-probe` / `respond-avoidance-probe`, and nothing is filtered until you confirm it - ๐Ÿ’ฌ **Socratic dialogue** โ€” not just interest confirmation; OpenClaw can have deep conversations: probing motivations, proposing hypotheses, confirming understanding โ€” the more you talk, the better it knows you - ๐Ÿ“– **Read the current soul profile** โ€” MBTI, core traits, deep needs, interest domains - ๐ŸŽฏ **Fetch personalized recommendations on demand** โ€” with explanations, confidence scores, and topic labels - ๐Ÿ’ฌ **Write feedback back into the learning loop** โ€” `like` / `dislike` / `comment` instantly update the profile and pool scoring - ๐Ÿ”„ **Sync Bilibili account signals** โ€” pull history / favorites / following and feed them into the memory system ### One-sentence integration prompt Paste the following into OpenClaw (or Claude Code / Codex CLI / Cursor) โ€” it will read the guide and wire everything up: ```text Please follow https://raw.githubusercontent.com/whiteguo233/OpenBiliClaw/main/docs/openclaw-quickstart.md to integrate this repository into OpenClaw (use Bash `curl` to fetch the document, NOT WebFetch โ€” WebFetch summarises markdown and drops critical commands). ``` ### Usage examples After integration, it's not just "you ask, it answers" โ€” **the system comes to you**. Here are the two core scenarios: #### Scenario 1: System proactively pushes a surprise recommendation OpenClaw is running `listen` in the background. After a refresh cycle, the system finds a high-scoring piece of content: > **OpenClaw** (receives WebSocket `delight.candidate`, comes to you proactively): > > > Just found something you might unexpectedly enjoy: > > **"Why You Can't Actually Lie Flat Even Back in a Small Town"** โ€” the Minsky moment of county economies. You've been deep in geopolitics and macro debt; this one dissects how land finance and the debt cycle mesh together. `[BV1KMwuzdEcB]` > > > > Want to check it out? > > **You**: "That one nailed it." > > **OpenClaw** (internally runs `submit-feedback --recommendation-id 4268 --feedback-type like`) You never asked for a recommendation โ€” the system surfaced it on its own. #### Scenario 2: System proactively asks about your interests OpenClaw receives an `interest.probe` event (or pulls `next-probe`) and finds a speculative-interest hypothesis waiting for confirmation: > **OpenClaw** (comes to you proactively): > > > I've been picking up signals that you might be into **architectural aesthetics** (e.g., parametric design, concrete minimalism) โ€” you've been watching a lot of content about structure and space lately. Does this direction resonate with you? > > **You**: "Yeah, I've been looking at a lot of parametric design stuff. Tadao Ando's exposed concrete aesthetic really gets me." > > **OpenClaw** (internally runs `chat --message "Yeah, I've been looking atโ€ฆ"`, returns a Socratic follow-up): > > > Here's my hypothesis โ€” you're not just interested in buildings, you're drawn to how minimal elements create maximum tension. Ando's approach to concrete is the same aesthetic drive as your obsession with "making the gears mesh" when you analyze LLM architectures. Am I close? The dialogue automatically feeds back into the soul profile โ€” the next discovery cycle will incorporate architectural aesthetics as a real interest and start searching in that direction. #### Scenario 3: You can also just ask for recommendations Of course, the traditional "you ask โ†’ it answers" flow works too: > **You**: "Recommend three Bilibili videos worth watching today." > > **OpenClaw** (internally runs `recommend --limit 3`, formats and replies) The whole loop stays local โ€” OpenClaw just calls the CLI bridge; your profile and data never leave the SQLite file on your disk. > ๐Ÿ“– Full command reference and troubleshooting: [OpenClaw Integration Guide](docs/openclaw-quickstart.md). ## โœจ Key Features - ๐Ÿง  **Five-Layer Soul Profile** โ€” Event โ†’ Preference โ†’ Awareness โ†’ Insight โ†’ Soul, inferring MBTI, cognitive style, and deep needs ([details](docs/modules/soul.md)) - ๐Ÿ”ฎ **Interest Probes** โ€” psychological bridging guesses domains you might love but have never explored; right guesses become real interests, wrong ones quietly retire - ๐Ÿงญ **Avoidance Probes** โ€” proactively confirms content forms and style boundaries you want to avoid; nothing is filtered until you confirm - ๐ŸŒ **Cross-Platform Sources** โ€” Bilibili / Xiaohongshu / Douyin / YouTube / X / Zhihu / Reddit / Bangumi / generic Web, so your interests stop being siloed ([details](docs/modules/discovery.md)) - ๐ŸŽฏ **Smart Diversity** โ€” topic quotas + cross-platform interleaving + small-source protection; goodbye to "all AI all day" - โšก **Instant, deduplicated reshuffle** โ€” ~0.6s; current cards, recommendation history, and the durable seen ledger are excluded by default - ๐Ÿ’ฌ **Warm Recommendations** โ€” friend-like explanations of why you'd enjoy something, not "because you watched similar videos" - ๐Ÿ”„ **Continuous Learning** โ€” Socratic dialogue + behavioral analysis + instant feedback; it understands you better over time - โญ **Local-First Favorites / Watch Later** โ€” cards save to local SQLite first and auto-sync stays off by default; the 2026-07-14 real-account regression completed both actions across all seven platforms as `synced/already_synced` - ๐Ÿงฉ **Browser Extension** โ€” Chrome / Edge / Brave / Arc / Firefox; side-panel recommendations + cross-site behavior collection, install and go - ๐Ÿš€ **Guided Init in the UI** โ€” the packaged `/setup/` wizard, Desktop Web, and the extension can all initialize with one click; no terminal required - ๐Ÿ”ฌ **Self-Optimizing Eval Loops** โ€” five modules each carry an LLM-as-judge loop that improves prompt quality over rounds - ๐Ÿ”’ **Fully Private** โ€” all data in local SQLite, LLM calls use your own key, each instance is built for exactly one person - ๐Ÿ”Œ **Local Embedding** โ€” optional Ollama + bge-m3, CPU-only, no extra API key - ๐Ÿ”ง **Fully Controllable** โ€” create multiple independent channels of the same LLM type and drag global or per-module failover chains; edit your profile or add custom Skills ## ๐Ÿ›๏ธ Architecture Overview ```text interactive (dialogue / config probe) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”œโ”€ runtime total gate (default 4) โ”€ ordered instance chain โ”€ adapter background โ”€ background admission (default 3) โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”œโ”€ refill: expression > evaluation > supply โ”‚ โ”œโ”€ low-stock supply includes explore queries / source extraction โ”‚ โ””โ”€ while queued: guarantee 2, may borrow all 3 โ”‚ expression owner: 8 immediate / 3s fixed tail / 60 drain / 30ร—2 provider โ””โ”€ maintenance: at most 1 while refill waits; parked when canonical available = 0 guided init: signals โ†’ preferences โ†’ full profile commit โ†’ discover โ†’ evaluate โ†’ copy โ†’ canonical ready โ””โ†’ optional probes after terminal state config recovery draft (normal or degraded; business APIs remain gated) โ”œโ†’ /api/config/probe-service โ†’ temporary registry โ†’ total gate โ””โ†’ /api/config/discover-models โ†’ exact instance GET /models (no write) โ†’ editable model list + local effort advisory durable reply: fixed time/payload โ†’ queued mode โ†’ one 11-kind typed settlement queue โ†’ actual worker + guard confirmation entry (pending list/cards) โ†’ one anchor(kind+ref+generation) โ†’ frozen admission / relation matrix โ”œโ†’ pendingโ‰ค3 ยท user no cooldown / system 12h+object 72h ยท confirmation-first attachment โ”œโ†’ busy worker: dialogue_busy + Retry-After โ†’ waiting UI auto-retry โ”œโ†’ active confusion: current holder only; hidden once this session has its turn โ”œโ†’ frozen kind/ref/generation โ†’ worker-only apply โ†’ event/object/derived/marker โ†’ applied โ”‚ โ””โ†’ publication-only retry โ†’ projection / exact release โ”œโ†’ action localโ‰ค1s: completed 200 / blocked 202 โ†’ popup/desktop poll 1/2/5s, โ‰ค30s โ””โ†’ confusion FIFOโ‰ค5 / head fencing / 12h recovery config hot reload: accepting drain old worker โ†’ atomic pause/revoke โ†’ new worker; 25m safety window realtime: runtime-stream 20s idle heartbeat โ†’ transient close shows reconnecting and retries ``` ``` โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Browser Extension (Chrome / Firefox) โ”‚ โ”‚ Behavior capture ยท MAIN-world taps (comment/ โ”‚ โ”‚ danmaku, xhs strong signal) ยท Cookie ยท Tasks โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ IPv4 0.0.0.0 + IPv6 [::] โ†’ REST API / WebSocket โ”‚ + Desktop Web (/web) ยท Mobile Web (/m) ยท QR LAN-IP โ”‚ + ping preflight โ†’ /web ยท /setup ยท /m โ†’ config + in-process recovery โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Agent Orchestration โ”‚ โ”‚ Skills ยท Dialogue ยท Runtime ยท 10s undo barrier โ”‚ โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค โ”‚ Soul โ”‚ Memory โ”‚ Discovery โ”‚ Recommendation โ”‚ โ”‚ Engine โ”‚ System โ”‚Discovery +โ”‚ Engine โ”‚ โ”‚ โ”‚ โ”‚ Admission โ”‚ โ”‚ โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค โ”‚ Interest: events/dialogue/feedback(priority) โ†’ โ”‚ โ”‚ ProfileUpdatePipeline โ†’ single INTEREST line โ”‚ โ”‚ Legacy batch only when rollback flag=false โ”‚ โ”‚ Init barrier: profile commit โ†’ discover/evaluate/copy โ†’ ready โ”‚ โ”‚ Soul cognition: dual pending cooldown ยท one anchor ยท worker-only settlement ยท winner receipt ยท confusion FIFO ยท ledger ยท deep gate โ”‚ โ”‚ LLM adapters ยท Source adapters (SourceAdapter) โ”‚ โ”‚ Module route โ†’ LLM instance chain โ†’ adapter ยท SourceAdapter โ”‚ โ”‚ Config recovery draft (normal/degraded) โ†’ temp probe / exact /models (no write) โ”‚ โ”‚ Source-family registry: alias ยท strategy ยท URL host โ”‚ โ”‚ โ†’ pool accounting ยท durable seen_items ledger โ”‚ โ”‚ Bangumi public API โ†’ search/ranked/date producer โ†’ shared eval โ”‚ โ”‚ API projected stock โ†’ 3ร—30 workers โ†’ serial admit; OpenClaw first batchโ‰ค4 โ†’ copyโ‰ค4/no split retry โ†’ UI โ”‚ โ”‚ Delight gate: formal copy/topic ready + seen_items guard โ†’ score/snapshot โ†’ UI ร— writes seen ledger โ”‚ โ”‚ API/OpenClaw startup hook โ†’ recover/maintain โ†’ expose LLM โ”‚ โ”‚ Reshuffle: current-card exclusion โ†’ PoolServeSnapshot/seen_items โ†’ short rec+shown write โ†’ one batch event โ”‚ โ”‚ Platform scope (PC Web tabs only): source_platform โ†’ scoped candidates, no cross-platform floor โ†’ same rank/copy/persist โ”‚ โ”‚ Platform inventory: platform-availability โ†’ same canonical servable set โ†’ total == ฮฃ by_platform โ”‚ โ”‚ Background maintenance: isolated worker โ†’ โ‰ค50 rows/batch; unchanged skip / 10m sweep โ”‚ โ”‚ /api/saved/* ยท router ยท Bilibili native save โ”‚ โ”‚ Six adapters โ†’ ExtensionNativeSaveBroker โ†’ extension_native_save_jobs โ”‚ โ”‚ six-platform source task multiplex: xhs / dy / yt / x / zhihu / reddit โ”‚ โ”‚ XHS auto tasks: source/scheduler gate โ†’ SQLite pacing/breaker โ†’ no new tab while off/limited โ”‚ โ”‚ extension_native_save_jobs -> /api/sources//next-task -> installed extension โ”‚ โ”‚ exact OpenBiliClaw / YouTube Watch Later targets โ†’ safe task-result โ”‚ โ”‚ trusted-local E2E exact auth โ†’ one saved-sync item โ†’ six-field callback โ”‚ โ”‚ unsupported_adapter_missing retryable ยท unsupported_content_type local-only โ”‚ โ”‚ Canonical ID ยท Local-first sync ยท Task poll ยท SQLite (events ยท seen ledger ยท pool ยท recs ยท saved/tasks)โ”‚ โ”‚ Six adapters โ†’ broker โ†’ shared MV3 recovery barrier โ†’ Reddit/X/YT/XHS/DY/Zhihu executors (6/6 fixture + real-account)โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ Web/API durable โ†’ SocraticDialogue(queued) โ†’ user+agent history โ†’ typed queue[all declared entries] โ†’ one in-line worker CLI/OpenClaw โ†’ SocraticDialogue(legacy_direct) โ†’ user+agent history โ†’ direct learning outside queue/guard learning โ†’ bypass background admission; keep total gate โ”€โ”€ new dislike: shared purge โ†’ content_cache failure/timeout โ†’ rollback provisional history โ†’ safe error / failed turn durable turn โ†’ fixed time/payload โ†’ confirmation entry (pending list/cards) โ†’ frozen anchor admission โ†’ relation matrix โ””โ†’ card/anchor/chat/probe/confusion/replay/legacy all worker-only card action โ†’ synchronous 200 fast path | 202 processing โ†’ popup/desktop poll; mobile/CLI have no action Desktop startup: recommendation hydration โ”‚ runtime hydration โ”‚ secondary health/profile/activity/config hydration (independent) Overseas traffic: `[network].mode` โ†’ system proxy (default) / direct / custom proxy โ†’ LLM, YouTube, X/Reddit CLIs, Bangumi, updater; CN clients remain isolated and direct Manual Douyin discovery: CLI discover โ†’ daemon-equivalent producer โ†’ per-keyword outcomes โ†’ extension search/hot/feed โ†’ pending-eval pool ``` Remote extension access uses explicit, default-off device authentication: `ext-key generate` โ†’ digest-only backend config โ†’ `/api/auth/extension-token` short session. HTTP uses a Bearer header; only WebSocket and image proxy URLs carry the short session query. > Full architecture detail (runtime state machine, pool accounting, profile overrides, and more) lives in [Architecture](docs/architecture.md) and the [visual architecture diagrams](docs/index.md). ### Content Discovery Engine **Multi-source adapter architecture** โ€” every platform plugs in through the `SourceAdapter` protocol, each with its own discovery approach: | Source | Discovery | How data is fetched | |--------|-----------|---------------------| | **Bilibili** | search ยท trending ยท related chain ยท cross-domain explore | Backend-direct WBI-signed APIs, with a real rendered search-page fallback via the extension | | **Xiaohongshu** | passive collection ยท search ยท creator subscriptions ยท init import | Extension reads your logged-in pages; zero backend crawling | | **Douyin** | init import ยท search ยท hot ยท feed | CLI and daemon share the formal producer; extension background tabs fetch candidates for the unified eval pool | | **YouTube** | init import ยท Takeout offline import ยท search / trending / channel | Extension reads profile signals; steady-state refill is backend-direct | | **X (Twitter)** | init import ยท search ยท For-You ยท followed authors | Server-side read-only cookie replay for discovery; native bookmark executor's first real favorite finished `synced` | | **Zhihu** | init import ยท search ยท hot ยท feed ยท creator ยท related | Extension reads logged-in tabs; renders as text cards | | **Reddit** | init import ยท search ยท hot ยท subreddit ยท related | Backend rdt-cli for discovery by default; Saved executor is fixture-tested, but the first real write remains uncertain after a 2xx response lacked old-DOM confirmation | | **Bangumi** | public-collection init ยท search ยท ranked ยท date browse | Official anonymous read-only API; no cookie/token, and date results may include unreleased subjects | | **Generic Web** | browser + LLM extraction | Adapts to any webpage | What happens after discovery: - **Safe fetching** โ€” the backend never logs in for you and never crawls content you can't see; every platform reuses the sessions already in your browser, and first-run profile signals are pulled only after you click "Start initialization". - **Continuous unified evaluation** โ€” raw candidates share one eval pool with 3ร—30 immediate-refill workers; scheduling counts only available, copy-pending, and evaluated durable stock, while serial admission is capped by current headroom. - **Diversity selection** โ€” platform quotas โ†’ topic dedup โ†’ style balancing โ†’ cross-platform interleaving โ†’ count caps; only Bilibili is enabled out of the box, other platforms are switched on in settings. > Per-platform task pipelines, pool accounting, and fallback strategies are documented in the [Discovery Engine docs](docs/modules/discovery.md). ### Soul Engine Infers from user behavior: - **Personality Portrait** โ€” Natural language user profile - **MBTI** โ€” Four dimensions with confidence scores - **Cognitive Style** โ€” Information processing preferences - **Deep Needs** โ€” Psychological content drivers - **Speculative Interests** โ€” System-predicted potential interest domains (e.g., molecular gastronomy, architectural aesthetics, watchmaking...) ## ๐Ÿ—๏ธ Project Structure ``` OpenBiliClaw/ โ”œโ”€โ”€ src/openbiliclaw/ # Python backend core โ”‚ โ”œโ”€โ”€ agent/ # Agent orchestration & Skill system โ”‚ โ”œโ”€โ”€ soul/ # Soul Engine (profiling ยท MBTI ยท interest/avoidance probes) โ”‚ โ”œโ”€โ”€ memory/ # Multi-layer memory system โ”‚ โ”œโ”€โ”€ discovery/ # Discovery engine (strategies ยท candidate pool ยท quota balancing ยท diversity) โ”‚ โ”œโ”€โ”€ recommendation/ # Recommendation & expression engine โ”‚ โ”œโ”€โ”€ sources/ # Source adapters, Bangumi API, and XHS/Douyin/YouTube/Zhihu/Reddit task bridges โ”‚ โ”œโ”€โ”€ youtube/ # Google Takeout import parser โ”‚ โ”œโ”€โ”€ api/ # Local FastAPI (config rollback / degraded mode / popup API) โ”‚ โ”œโ”€โ”€ runtime/ # Refresh, feedback coalescing, presence gate, shared CLI/desktop autostart reconcile, Ollama, degraded RuntimeContext โ”‚ โ”œโ”€โ”€ bilibili/ # Bilibili API layer (WBI signing ยท rate control) โ”‚ โ”œโ”€โ”€ llm/ # Multi-model LLM adapters + structured JSON tolerance โ”‚ โ””โ”€โ”€ storage/ # Data storage layer โ”œโ”€โ”€ extension/ # Chrome browser extension (Bilibili + XHS + Douyin + YouTube + X + Zhihu + Reddit + autostart/config recovery) โ”œโ”€โ”€ skills/ # Built-in Skill definitions โ”œโ”€โ”€ docs/ # Documentation โ””โ”€โ”€ tests/ # Tests (1900+) ``` ## ๐Ÿ› ๏ธ Tech Stack | Module | Technology | |--------|-----------| | Backend | Python 3.11+ | | Browser Extension | TypeScript + Chrome Extension (Manifest V3) | | LLM | Multiple independent Base URL / token / model instances per provider type, with ordered global and per-module failover chains; first migration keeps a permanent legacy backup and `config-export-legacy` creates an old-version copy; built-in Gemini / DeepSeek / OpenAI / Claude / OpenRouter / Ollama; any OpenAI-compatible endpoint works; OpenAI can experimentally reuse Codex CLI OAuth | | Bilibili API | Custom client (WBI signing ยท v_voucher auto-recovery ยท rate control) | | Xiaohongshu | Extension DOM/state extraction + task dispatch; scrolling init imports open `/explore` in the foreground, click the page's profile entry, then use bounded scrolling and partial batches; no backend crawling | | Douyin | Extension DOM + MAIN-world passive fetch tap + task dispatch; init imports post / favorite / like / follow signals; search / hot / feed discovery starts from the Douyin home page and uses DOM interactions to trigger loading; search/feed passively collect page responses / rendered results, and hot can use a hot-board `group_id` seed as a logged-in related fallback; no backend login crawling | | YouTube | Extension DOM task dispatch reads watch history / subscriptions / likes; Google Takeout can import older data offline | | X (Twitter) | Server-side cookie replay via default-installed `twitter-cli` (lazy-imported, read-only); the extension captures your engagement and syncs the x.com cookie; tweets render as text cards | | Zhihu | Extension task dispatch reads event-smoke and selected guided-init signals plus search / hot / feed / creator / related candidates in the logged-in browser; answers / articles / questions render as text cards | | Reddit | Default-installed rdt-cli reads search / hot / subreddit / related candidates by default; the extension syncs `reddit_session` into rdt credentials and `rdt login` is a manual fallback; extension task dispatch reads discovery when rdt is unavailable, unauthenticated, or explicitly selected, and always reads bootstrap saved / upvoted / subscribed signals in the logged-in browser; posts / comments render as text cards | | Bangumi | Official anonymous read-only v0 API; search / ranked / date browsing feed the shared candidate pool, while an optional public username enables public-collection profile init; no cookie, token, or native write-back | | Storage | SQLite + Embedding vector index | | Containerization | Docker Compose (backend) | | Agent Framework | Lightweight custom framework | ## ๐Ÿ“– Documentation - [Documentation Hub](docs/index.md) โ€” All-in-one entry point - [FAQ](docs/faq.md) โ€” quick answers for install / connection / update issues - [Project Spec](docs/spec.md) โ€” Complete design & planning - [Architecture](docs/architecture.md) โ€” System architecture deep dive - [Memory Design](docs/memory-design.md) โ€” Multi-layer memory architecture - [Discovery Engine](docs/modules/discovery.md) โ€” multi-source discovery + platform mix + diversity selection - [Soul Engine](docs/modules/soul.md) โ€” Deep profiling + MBTI + interest speculation - [CLI Reference](docs/modules/cli.md) ยท [Config Reference](docs/modules/config.md) - [Contributing Guide](docs/contributing.md) ## ๐Ÿ“œ Release History The current release is summarized in [Recent Updates](#recent-updates) above; full history lives in [docs/changelog.md](docs/changelog.md). Most users should use the `openbiliclaw-v*` aggregate [Latest Release](https://github.com/whiteguo233/OpenBiliClaw/releases/latest) for extension packages and available desktop installers; automation-channel releases remain available as `backend-v*`, `extension-v*`, and `desktop-v*`. ## ๐Ÿ—บ๏ธ Roadmap OpenBiliClaw aims to be your **personalized entry point to the entire web**. Started on Bilibili, it now covers Xiaohongshu, Douyin, YouTube, X, Zhihu, Reddit, Bangumi, and the generic Web; next: - **More content sources** โ€” V2EX, Weibo, various BBS / forums; each platform is a `SourceAdapter` and the architecture is proven extensible - **Cross-platform interest fusion** โ€” your mechanical-keyboard interest from Bilibili + your coffee-gear interest from Xiaohongshu + your short-video taste from Douyin likes/favorites + your long-form watching and subscriptions from YouTube + the news you like/bookmark on X = one complete you. Profile fusion stops your interests from being fragmented across silos - **Smarter cross-source discovery** โ€” "you started following coffee gear on Xiaohongshu, here's a hand-drip documentary on Bilibili you might love" - **Community ecosystem** โ€” user-defined SourceAdapters, shared discovery strategies, contributed platform adapters ## ๐Ÿค Contributing Contributions welcome! See the [Contributing Guide](docs/contributing.md) to get started. ## ๐Ÿ™ Acknowledgements - Thanks to [@addtion99](https://github.com/addtion99) for proposing configurable browser-extension backend host / port settings and sharing the popup-side implementation idea in [#8](https://github.com/whiteguo233/OpenBiliClaw/pull/8). - Thanks to [@jiaobenhaimo](https://github.com/jiaobenhaimo) for contributing Safari extension, watch-later bookmarks, YouTube repost detection, and marketing filter designs in [#53](https://github.com/whiteguo233/OpenBiliClaw/pull/53). The OR-join dedup fix and watch-later feature have been merged into main. - Thanks to [@tangle111-design](https://github.com/tangle111-design) for exploring `style_key` viewing modes, recommendation tone, Bilibili initialization, and LLM / profile workflow improvements in [#69](https://github.com/whiteguo233/OpenBiliClaw/pull/69). The relevant ideas have been reviewed, split up, and selectively merged into main. - Thanks to [@DongLanQwQ0](https://github.com/DongLanQwQ0) for polishing desktop web interactions โ€” side-drawer collapse animation, a delight-card drag dead zone, and a stacked toast notification system โ€” in [#102](https://github.com/whiteguo233/OpenBiliClaw/pull/102). Merged into main. - Thanks to [@DongLanQwQ0](https://github.com/DongLanQwQ0) for the desktop web theme-engine rework to oklch in [#110](https://github.com/whiteguo233/OpenBiliClaw/pull/110) โ€” a single `--hue-primary` control point with a 12-hue tunable color picker, a five-step accent ramp, and unified interaction states. Merged into main. - Thanks to [@wuwafly3](https://github.com/wuwafly3) for contributing the DashScope (Alibaba Model Studio) multimodal embedding provider and the image-only cover-embedding design in [#100](https://github.com/whiteguo233/OpenBiliClaw/pull/100). The capability was reviewed, fixed (network routing / save-time validation), and reworked into the live cover-visual pipeline (consumed consistently by both delight and the normal feed) before merging into main (see [#116](https://github.com/whiteguo233/OpenBiliClaw/pull/116)). ## โญ Star History If OpenBiliClaw gave you back control of your feed, [a star](https://github.com/whiteguo233/OpenBiliClaw) is the most direct vote for "keep adding platforms". Star History Chart ## Privacy at a glance Default data flow: browser extension โ†’ your configured local OpenBiliClaw backend โ†’ SQLite on your machine. The extension does not send data to servers operated by OpenBiliClaw developers. If you configure a cloud LLM or embedding provider, the relevant content is sent to that provider according to your configuration. See the [Privacy Policy](docs/privacy.md). ## ๐Ÿ“„ License [MIT](LICENSE)