--- name: serpapi-performance-tuning description: 'Optimize SerpApi performance with caching, async searches, and result filtering. Use when reducing latency, minimizing credit consumption, or optimizing search throughput. Trigger: "serpapi performance", "optimize serpapi", "serpapi caching", "serpapi slow". ' allowed-tools: Read, Write, Edit version: 1.0.0 license: MIT author: Jeremy Longshore tags: - saas - search - seo - serpapi compatibility: Designed for Claude Code --- # SerpApi Performance Tuning ## Overview SerpApi typical latency: 2-5 seconds per search (real-time scraping). Main optimization: aggressive caching since search results change slowly. Secondary: use Google Light API for faster responses, reduce `num` parameter, and parallelize independent searches. ## Instructions ### Step 1: Multi-Layer Caching ```typescript import { LRUCache } from 'lru-cache'; import { Redis } from 'ioredis'; import { getJson } from 'serpapi'; // L1: In-memory (fastest, per-instance) const l1 = new LRUCache({ max: 1000, ttl: 600_000 }); // 10 min // L2: Redis (shared across instances) const redis = new Redis(process.env.REDIS_URL!); async function cachedSearch(params: Record): Promise { const key = `serpapi:${JSON.stringify(params)}`; // L1 check const l1Hit = l1.get(key); if (l1Hit) return l1Hit; // L2 check const l2Hit = await redis.get(key); if (l2Hit) { const parsed = JSON.parse(l2Hit); l1.set(key, parsed); return parsed; } // Cache miss: real API call const result = await getJson({ ...params, api_key: process.env.SERPAPI_API_KEY }); l1.set(key, result); await redis.setex(key, 3600, JSON.stringify(result)); // 1 hour in Redis return result; } ``` ### Step 2: Google Light API (Faster) ```python # Google Light API: ~1s instead of 2-5s, limited result fields result = client.search(engine="google_light", q="fast query", num=5) # Returns: organic_results with title, link, snippet only # No knowledge_graph, answer_box, or rich snippets ``` ### Step 3: Reduce Response Size ```python # Only get the fields you need result = client.search( engine="google", q="query", num=5, # Fewer results = faster no_cache=False, # Use SerpApi's server-side cache (default) ) # Strip metadata to reduce memory/storage clean = { "organic_results": result.get("organic_results", []), "answer_box": result.get("answer_box"), "search_id": result["search_metadata"]["id"], } ``` ### Step 4: Parallel Search ```typescript import PQueue from 'p-queue'; const queue = new PQueue({ concurrency: 5, interval: 1000, intervalCap: 5 }); async function batchSearch(queries: string[]): Promise { return Promise.all( queries.map(q => queue.add(() => cachedSearch({ engine: 'google', q, num: 5 })) ) ); } // 10 queries, 5 parallel, rate limited: ~4 seconds total const results = await batchSearch(['query1', 'query2', /* ... */]); ``` ## Latency Benchmarks | Method | Typical Latency | Credits | |--------|----------------|---------| | Google Search (uncached) | 2-5s | 1 | | Google Light | 1-2s | 1 | | L1 cache hit | < 1ms | 0 | | Redis cache hit | 1-5ms | 0 | | Archive retrieval | 500ms | 0 | ## Error Handling | Issue | Cause | Solution | |-------|-------|----------| | Cache stampede | TTL expiry under load | Stale-while-revalidate | | High latency | Complex queries | Use Google Light API | | Memory pressure | Large cache | Limit LRU max entries | ## Resources - [Google Light API](https://serpapi.com/google-light-api) - [SerpApi Caching](https://serpapi.com/search-api#api-parameters-serpapi-parameters-no-cache) ## Next Steps For cost optimization, see `serpapi-cost-tuning`.