# Snow CLI User Guide - Custom Search Engine

## Overview

Snow CLI's web search (the `web-search` MCP tool) is driven by a pluggable
search engine layer. Built-in engines are `duckduckgo` and `bing`, both of
which scrape results from a headless browser (no official API used).

If you want to use a different search provider, you can drop a JavaScript
file into your user directory and Snow CLI will register it automatically —
no build step, no source code modification.

Use this feature when you want to:

- Use a regional search provider that isn't shipped by default
- Search your company's internal knowledge base or intranet
- Customize how an existing provider is scraped (e.g. fix a selector after a
  layout change)
- Temporarily mask a built-in engine without deleting any file

> The example below uses a fictional provider `example-search.com` purely
> to illustrate the engine contract. You are responsible for complying with
> each target site's Terms of Service and `robots.txt` when writing a real
> plugin.

## Plugin Directory

Snow CLI loads search engine plugins from:

```bash
~/.snow/plugin/search_engines/
```

Supported file extensions:

- `.js`
- `.mjs` (recommended for plain ES Modules)
- `.cjs`

Notes:

- Plugins are loaded from the user directory only.
- Snow CLI sorts plugin files by filename and loads them on first web search.
- Adding, modifying, or deleting plugin files hot-reloads automatically —
  no Snow CLI restart required.
- Built-in engines (`duckduckgo`, `bing`) are always registered first; a
  plugin engine with the same `id` overrides the built-in one.

## Export Formats

A plugin module can export in any of these forms (the first non-empty match
wins, all of them are scanned):

```js
export default { ... }
```

```js
export const searchEngine = { ... }
```

```js
export const searchEngines = [{ ... }, { ... }]
```

If multiple plugin files register the same engine `id`, the file loaded
later (alphabetically) overrides the earlier one.

## Engine Structure

Every engine must satisfy this shape (TypeScript-style for clarity, but
plugin files are plain JavaScript):

```ts
interface SearchEngine {
	id: string; // stable identifier, e.g. 'my-engine'
	name: string; // human readable, shown in the picker
	enable?: boolean; // optional, defaults to true
	search(
		page: Page, // a Puppeteer Page already opened for you
		query: string, // the user's query string
		maxResults: number, // how many results to return at most
	): Promise<SearchResult[]>;
}

interface SearchResult {
	title: string;
	url: string;
	snippet: string;
	displayUrl: string;
}
```

Field description:

- `id`: the value users put into `~/.snow/proxy-config.json`'s
  `searchEngine` field and what the picker stores. Keep it stable.
- `name`: shown in the proxy config picker. Free-form.
- `enable` (optional): defaults to `true`. Set to `false` to temporarily
  disable an engine without deleting its file. A disabled engine is invisible
  to `getSearchEngine`, `listSearchEngines`, and the UI picker.
  - Bonus trick: declaring `{id: 'bing', enable: false, search() {}}` in a
    plugin will mask the built-in `bing` engine, because the loader removes
    the same-id entry from the registry when it sees `enable: false`.
- `search(page, query, maxResults)`: the actual work. Snow CLI:

  - launches/connects the browser for you (respects `~/.snow/proxy-config.json`)
  - opens a fresh `Page` and passes it in
  - closes the page after `search()` returns

  Your engine should:

  - navigate to its own search URL via `page.goto(...)`
  - wait for the DOM to settle
  - extract up to `maxResults` results via `page.evaluate(...)`
  - return them as an array of `SearchResult`

  Never call `browser.close()` / `page.close()` yourself — the page is
  owned by the caller.

## Lifecycle and Configuration

1. Drop the plugin file under `~/.snow/plugin/search_engines/`.
2. Start (or restart) Snow CLI.
3. Open the proxy configuration screen (`/settings` → Proxy and Browser
   Settings, or the dedicated entry point in your build) — your engine will
   appear in the "Search Engine" picker by its `name`.
4. Select your engine, save. The choice is persisted in
   `~/.snow/proxy-config.json` as:

   ```json
   {
   	"enabled": false,
   	"port": 7890,
   	"searchEngine": "my-engine"
   }
   ```

5. Any subsequent `web-search` MCP call will use your engine.

## Example: A Minimal Plugin Template

Below is a complete, runnable template that targets a fictional provider
`example-search.com`. Replace the URL, selectors, and id with the values
that match your real target. Treat the selectors here as **placeholders**
— every search page has a different DOM, you must inspect yours.

```js
// ~/.snow/plugin/search_engines/my-engine.mjs

const cleanText = text =>
	(text || '')
		.replace(/\s+/g, ' ')
		.replace(/[\u200B-\u200D\uFEFF]/g, '')
		.trim();

export default {
	id: 'my-engine',
	name: 'My Search Engine',
	// Set to `false` to temporarily disable this engine without deleting the
	// file. Disabled engines are invisible to the picker and `getSearchEngine`.
	enable: true,

	async search(page, query, maxResults) {
		// 1. Build the search URL for your target provider. The example below
		//    uses a fictional host purely to illustrate the shape.
		const encodedQuery = encodeURIComponent(query);
		const searchUrl =
			`https://example-search.com/search?q=${encodedQuery}` +
			`&n=${Math.max(maxResults, 10)}`;

		// 2. Navigate. Prefer `domcontentloaded` over `networkidle2` because
		//    real search pages keep loading telemetry forever.
		try {
			await page.goto(searchUrl, {
				waitUntil: 'domcontentloaded',
				timeout: 30000,
			});
		} catch {
			// Navigation timeout — try whatever already painted.
		}

		// 3. Wait for a representative result selector. Never throw — return
		//    an empty list and let the caller fall back.
		try {
			await page.waitForSelector('.results .result-item', {timeout: 10000});
		} catch {
			// Best effort — extraction may still find something.
		}

		// 4. Extract inside the browser context.
		const raw = await page.evaluate(maxLimit => {
			const out = [];
			const items = document.querySelectorAll('.results .result-item');
			const isHttpUrl = u => /^https?:\/\//i.test(u);

			for (const item of items) {
				if (out.length >= maxLimit) break;

				// Filter ads if the provider marks them.
				if (item.classList.contains('is-ad')) continue;

				const linkEl = item.querySelector('a.result-title');
				if (!linkEl) continue;

				const href = linkEl.getAttribute('href') || '';
				if (!isHttpUrl(href)) continue;

				const title = (linkEl.textContent || '').trim();
				if (!title) continue;

				const snippetEl = item.querySelector('.result-snippet');
				const snippet = snippetEl ? (snippetEl.textContent || '').trim() : '';

				const citeEl = item.querySelector('cite, .result-host');
				const displayUrl = citeEl ? (citeEl.textContent || '').trim() : '';

				out.push({title, url: href, snippet, displayUrl});
			}
			return out;
		}, maxResults);

		// 5. Normalize and return.
		return raw.map(r => ({
			title: cleanText(r.title),
			url: r.url || '',
			snippet: cleanText(r.snippet),
			displayUrl: cleanText(r.displayUrl),
		}));
	},
};
```

To adapt this template to a real provider you need to figure out, for each
provider you target:

- the search URL pattern (often `?q=` or `?wd=` or `?query=`, plus a
  result-count parameter);
- a stable container selector for organic results;
- the title / link selector inside each container;
- the snippet selector;
- the display-URL / host selector;
- how the provider marks ads or sponsored results, so you can skip them.

Open the provider's result page in a regular browser, use DevTools to
inspect the DOM, then plug the selectors into the template above.

## Writing Your Own Engine: Checklist

1. **Pick a stable `id`**. Once users save it into `proxy-config.json`,
   renaming will break their config.
2. **Open the target search URL with `domcontentloaded`**, not
   `networkidle2`. Most search pages keep loading telemetry scripts forever
   and `networkidle2` will time out before results are usable.
3. **Wrap `page.goto` in `try/catch`**. A navigation timeout is recoverable
   — the DOM may already contain enough to extract.
4. **Always use `page.waitForSelector` with a timeout**. Never `throw` if
   it fails; return an empty list and let the caller fall back.
5. **Extract inside `page.evaluate`**. The callback runs in the browser, so
   you have full DOM access but must `return` only structured-cloneable
   plain objects.
6. **Filter ads / sponsored results**. Each provider marks them differently
   — check the DOM yourself.
7. **Normalize text** (`cleanText` helper above) — collapse whitespace and
   strip zero-width characters.
8. **Never call `browser.close()` or `page.close()`**. The page is owned by
   `WebSearchService`.
9. **Don't import Node-only modules into `page.evaluate`'s callback** — it
   runs inside the browser.

## Multi-Engine Plugins

You can register multiple engines from a single file:

```js
export const searchEngines = [
  {id: 'engine-a', name: 'Engine A', async search(...) { /* ... */ }},
  {id: 'engine-b', name: 'Engine B', async search(...) { /* ... */ }},
];
```

This is convenient for plugins that share a `cleanText` helper or a common
result-extraction routine.

## Troubleshooting

- **The plugin does not appear in the picker.**

  - Make sure the file extension is `.js` / `.mjs` / `.cjs`.
  - Check the Snow CLI startup logs for `[websearch] failed to load search
engine plugin "..."`. Syntax errors fail loudly.
  - Make sure your export is a plain object with `{id, name, search}` — the
    loader logs `did not export a valid SearchEngine` when validation fails.

- **Search always returns 0 results.**

  - The provider probably updated its DOM. Open the page manually in a
    browser and inspect the new selectors.
  - Increase the `page.waitForSelector` timeout.
  - Some providers redirect bot traffic to a captcha page — try setting a
    realistic `User-Agent` via `page.setUserAgent(...)` at the start of
    `search()` (`WebSearchService` already sets one before delegating, but
    you can override).

- **I want to disable a built-in engine.**
  - Create a plugin file with `{id: 'bing', name: 'Bing', enable: false,
async search() { return []; }}`. The loader will see `enable: false`
    and remove the same-id entry from the registry.

## Related

- [Proxy and Browser Settings](./03.Proxy%20and%20Browser%20Settings.md)
- [Custom StatusLine Guide](./21.Custom%20StatusLine%20Guide.md) — same
  plugin-loading philosophy applied to the status line