--- name: grepai-search-advanced description: Advanced search options in GrepAI. Use this skill for JSON output, compact mode, and AI agent integration. --- # GrepAI Advanced Search Options This skill covers advanced search options including JSON output, compact mode, and integration with AI agents. ## When to Use This Skill - Integrating GrepAI with scripts or tools - Using GrepAI with AI agents (Claude, GPT) - Processing search results programmatically - Reducing token usage in AI contexts ## Command-Line Options | Option | Description | |--------|-------------| | `--limit N` | Number of results (default: 10) | | `--json` / `-j` | JSON output format | | `--toon` / `-t` | TOON output format (~50% fewer tokens than JSON) | | `--compact` / `-c` | Compact output (no content, works with --json or --toon) | > **Note:** `--json` and `--toon` are mutually exclusive. ## JSON Output ### Standard JSON ```bash grepai search "authentication" --json ``` Output: ```json { "query": "authentication", "results": [ { "score": 0.89, "file": "src/auth/middleware.go", "start_line": 15, "end_line": 45, "content": "func AuthMiddleware() gin.HandlerFunc {\n return func(c *gin.Context) {\n token := c.GetHeader(\"Authorization\")\n if token == \"\" {\n c.AbortWithStatus(401)\n return\n }\n claims, err := ValidateToken(token)\n ...\n }\n}" }, { "score": 0.82, "file": "src/auth/jwt.go", "start_line": 23, "end_line": 55, "content": "func ValidateToken(tokenString string) (*Claims, error) {\n ..." } ], "total": 2 } ``` ### Compact JSON (AI Optimized) ```bash grepai search "authentication" --json --compact ``` Output: ```json { "q": "authentication", "r": [ { "s": 0.89, "f": "src/auth/middleware.go", "l": "15-45" }, { "s": 0.82, "f": "src/auth/jwt.go", "l": "23-55" } ], "t": 2 } ``` **Key differences:** - Abbreviated keys (`s` vs `score`, `f` vs `file`) - No content (just file locations) - ~80% fewer tokens for AI agents ## TOON Output (v0.26.0+) TOON (Token-Oriented Object Notation) is an even more compact format, optimized for AI agents. ### Standard TOON ```bash grepai search "authentication" --toon ``` Output: ``` [2]{content,end_line,file_path,score,start_line}: "func AuthMiddleware()...",45,src/auth/middleware.go,0.89,15 "func ValidateToken()...",55,src/auth/jwt.go,0.82,23 ``` ### Compact TOON (Best for AI) ```bash grepai search "authentication" --toon --compact ``` Output: ``` [2]{end_line,file_path,score,start_line}: 45,src/auth/middleware.go,0.89,15 55,src/auth/jwt.go,0.82,23 ``` ### TOON vs JSON Comparison | Format | Tokens (5 results) | Best For | |--------|-------------------|----------| | JSON | ~1,500 | Scripts, parsing | | JSON compact | ~300 | AI agents | | TOON | ~250 | AI agents | | **TOON compact** | **~150** | **Token-constrained AI** | ### When to Use TOON - **Use TOON** when integrating with AI agents that support it - **Use TOON compact** for maximum token efficiency (~50% smaller than JSON compact) - **Stick with JSON** for traditional scripting (jq, programming languages) ## Compact Format Reference | Full Key | Compact Key | Description | |----------|-------------|-------------| | `query` | `q` | Search query | | `results` | `r` | Results array | | `score` | `s` | Similarity score | | `file` | `f` | File path | | `start_line`/`end_line` | `l` | Line range ("15-45") | | `total` | `t` | Total results | ## Combining Options ```bash # 5 results in compact JSON grepai search "error handling" --limit 5 --json --compact # 20 results in full JSON grepai search "database" --limit 20 --json ``` ## AI Agent Integration ### For Claude/GPT Prompts Use compact mode to minimize tokens: ```bash # Agent asks for context grepai search "payment processing" --json --compact --limit 5 ``` Then provide results to the AI with file read tool for details. ### Workflow Example 1. **Search for relevant code:** ```bash grepai search "authentication middleware" --json --compact --limit 3 ``` 2. **Get response:** ```json { "q": "authentication middleware", "r": [ {"s": 0.92, "f": "src/auth/middleware.go", "l": "15-45"}, {"s": 0.85, "f": "src/auth/jwt.go", "l": "23-55"}, {"s": 0.78, "f": "src/handlers/auth.go", "l": "10-40"} ], "t": 3 } ``` 3. **Read specific files:** AI reads `src/auth/middleware.go:15-45` for full context. ## Scripting with JSON ### Bash + jq ```bash # Get just file paths grepai search "config" --json | jq -r '.results[].file' # Filter by score grepai search "config" --json | jq '.results[] | select(.score > 0.8)' # Count results grepai search "config" --json | jq '.total' ``` ### Python ```python import subprocess import json result = subprocess.run( ['grepai', 'search', 'authentication', '--json'], capture_output=True, text=True ) data = json.loads(result.stdout) for r in data['results']: print(f"{r['score']:.2f} | {r['file']}:{r['start_line']}") ``` ### Node.js ```javascript const { execSync } = require('child_process'); const output = execSync('grepai search "authentication" --json'); const data = JSON.parse(output); data.results.forEach(r => { console.log(`${r.score.toFixed(2)} | ${r.file}:${r.start_line}`); }); ``` ## MCP Integration GrepAI provides MCP tools with format selection (v0.26.0+): ```bash # Start MCP server grepai mcp-serve ``` MCP tools support JSON (default) or TOON format: | MCP Tool | Parameters | |----------|------------| | `grepai_search` | `query`, `limit`, `compact`, `format` | | `grepai_trace_callers` | `symbol`, `compact`, `format` | | `grepai_trace_callees` | `symbol`, `compact`, `format` | | `grepai_trace_graph` | `symbol`, `depth`, `format` | | `grepai_index_status` | `format` | ### Format Parameter ```json { "name": "grepai_search", "arguments": { "query": "authentication", "format": "toon", "compact": true } } ``` Valid values: `"json"` (default) or `"toon"` ## Token Optimization ### Token Comparison For a typical search with 5 results: | Format | Approximate Tokens | |--------|-------------------| | Human-readable | ~2,000 | | JSON full | ~1,500 | | JSON compact | ~300 | ### When to Use Each Format | Format | Use Case | |--------|----------| | Human-readable | Manual inspection | | JSON full | Scripts needing content | | JSON compact | AI agents, token-limited contexts | ## Piping Results ### To File ```bash grepai search "authentication" --json > results.json ``` ### To Another Tool ```bash # Open results in VS Code grepai search "config" --json | jq -r '.results[0].file' | xargs code # Copy first result path to clipboard (macOS) grepai search "config" --json | jq -r '.results[0].file' | pbcopy ``` ## Batch Searches Run multiple searches: ```bash #!/bin/bash queries=("authentication" "database" "logging" "error handling") for q in "${queries[@]}"; do echo "=== $q ===" grepai search "$q" --json --compact --limit 3 echo done ``` ## Error Handling ### JSON Error Response When search fails: ```json { "error": "Index not found. Run 'grepai watch' first.", "code": "INDEX_NOT_FOUND" } ``` ### Checking for Errors in Scripts ```bash result=$(grepai search "query" --json) if echo "$result" | jq -e '.error' > /dev/null 2>&1; then echo "Error: $(echo "$result" | jq -r '.error')" exit 1 fi ``` ## Best Practices 1. **Use compact for AI agents:** 80% token savings 2. **Use full JSON for scripts:** When you need content 3. **Use human-readable for debugging:** Easier to read 4. **Limit results appropriately:** Don't fetch more than needed 5. **Check for errors:** Parse JSON response properly ## Output Format Advanced search output (JSON compact): ```json { "q": "authentication middleware", "r": [ {"s": 0.92, "f": "src/auth/middleware.go", "l": "15-45"}, {"s": 0.85, "f": "src/auth/jwt.go", "l": "23-55"}, {"s": 0.78, "f": "src/handlers/auth.go", "l": "10-40"} ], "t": 3 } ``` **Token estimate:** ~80 tokens (vs ~800 for full content)