--- name: performance-optimization description: "Apply systematic performance optimization techniques for Python and Rust code: estimation + profiling, API/bulk design, algorithmic wins, cache-friendly memory layout, fewer allocations, fast paths, caching, and compiler-friendly hot loops. Use for performance code reviews, refactors, and profiling-driven optimizations. Keywords: performance, latency, throughput, cache, allocation, memory layout, PyO3, msgspec, tokio, async, pprof, py-spy, perf." license: Apache-2.0 compatibility: Skills-compatible coding agents working on Python and Rust codebases. metadata: upstream_url: "https://abseil.io/fast/hints" upstream_authors: "Jeff Dean; Sanjay Ghemawat" upstream_original_version: "2023-07-27" upstream_last_updated: "2025-12-25" --- # Python & Rust Performance Hints (Jeff Dean & Sanjay Ghemawat style) This skill packages key ideas from Abseil's **Performance Hints** document, adapted for Python and Rust development. Use it to: - review Python/Rust code for performance risks - propose high-impact optimizations with **explicit tradeoffs** - design APIs/data structures that keep future optimizations possible - write an experiment plan (profile + microbenchmark) to validate changes ## Scope and guardrails - **Scope:** single-process / single-binary performance (CPU, memory, allocations, cache behavior). - **Do not:** change externally observable behavior unless the user explicitly agrees. - **Do not:** introduce undefined behavior, data races, or brittle "clever" micro-opts without evidence. - **Default philosophy:** choose the faster alternative **when it doesn't materially harm readability or complexity**; otherwise, measure first. ## When to apply Use this skill when the task involves any of: - reducing **latency** or improving **throughput** - cutting **memory footprint** or **allocation rate** - improving **cache locality** / reducing cache misses - designing performant **APIs** (bulk ops, view types, threading model) - reviewing performance-sensitive Python or Rust changes - interpreting a **flat profile** and finding next steps ## What to ask for (minimum inputs) If you don't have enough information, ask for the smallest set that changes your recommendation quality: 1. **Goal:** latency vs throughput vs memory (and the SLO if any) 2. **Where:** hot path vs init vs test-only (and typical input sizes) 3. **Evidence:** profile / flame graph / perf counters / allocation profile (if available) 4. **Constraints:** correctness constraints, API constraints, thread-safety requirements If none exists yet, proceed with _static analysis + "what to measure first"_. --- # Workflow: how an agent should use these hints ## Step 1 — classify the code - **Test code:** mostly care about asymptotic complexity and test runtime. - **Application code:** separate **init/cold** vs **hot path**. - **Library code:** prefer "safe, low-complexity" performance techniques because you can't predict callers. ## Step 2 — do a back-of-the-envelope estimate Before implementing changes, estimate what might dominate: 1. Count expensive operations (seeks, round-trips, allocations, bytes touched, comparisons, etc.) 2. Multiply by rough cost. 3. If latency matters and there is concurrency, consider overlap. ### Reference latency table (rough order-of-magnitude) | Operation | Approx time | | --------------------------------- | ----------------------: | | L1 cache reference | 0.5 ns | | L2 cache reference | 3 ns | | Branch mispredict | 5 ns | | Mutex lock/unlock (uncontended) | 15 ns | | Main memory reference | 50 ns | | Python function call overhead | 50-100 ns | | Rust function call (non-inlined) | 1-10 ns | | PyO3 GIL acquire/release | 100-500 ns | | Compress 1K bytes with Snappy | 1,000 ns | | Read 4KB from SSD | 20,000 ns | | Round trip within same datacenter | 50,000 ns | | Read 1MB sequentially from memory | 64,000 ns | | Read 1MB over 100 Gbps network | 100,000 ns | | Read 1MB from SSD | 1,000,000 ns (1 ms) | | Disk seek | 5,000,000 ns (5 ms) | | Read 1MB sequentially from disk | 10,000,000 ns (10 ms) | | Send packet CA→Netherlands→CA | 150,000,000 ns (150 ms) | ### Python-specific costs | Operation | Approx time | | ------------------------------ | ----------------: | | dict lookup | 20-50 ns | | list.append | 20-40 ns | | getattr on object | 50-100 ns | | isinstance check | 30-60 ns | | JSON parse (stdlib) 1KB | 50-100 us | | msgspec parse 1KB | 5-15 us | | Django ORM query (simple) | 1-10 ms | | Django ORM async query | 0.5-5 ms | ### Rust-specific costs | Operation | Approx time | | ------------------------------ | ----------------: | | HashMap lookup | 10-30 ns | | Vec push (no realloc) | 2-10 ns | | String allocation (small) | 20-50 ns | | Arc clone | 10-20 ns | | tokio task spawn | 200-500 ns | | async channel send | 50-200 ns | ### Estimation examples (templates) **Web request through PyO3 bridge:** - Rust HTTP parsing: ~1us - GIL acquisition: ~200ns - Python handler execution: ~50us (simple) to ~5ms (with ORM) - Response serialization (msgspec): ~10us - GIL release + Rust response: ~200ns - Total: **~50us to ~5ms** depending on handler complexity **Batch processing 10K items:** - Per-item Python function call: 10K × 100ns = **1ms** (overhead alone) - With msgspec struct validation: 10K × 500ns = **5ms** - With dict allocation per item: 10K × 50ns = **0.5ms** (plus GC pressure) ## Step 3 — measure before paying complexity When you can, measure to validate impact: - **Python profiling:** - `py-spy` for sampling profiler (low overhead, production-safe) - `cProfile` for deterministic profiling (high overhead) - `memray` or `memory_profiler` for allocation profiling - `scalene` for CPU + memory + GPU profiling - **Rust profiling:** - `perf` for Linux sampling profiler - `flamegraph` crate for generating flame graphs - `criterion` for microbenchmarks - `dhat` for allocation profiling - Watch for **GIL contention** in Python: contention can lower CPU usage and hide the "real" bottleneck. - Watch for **async runtime overhead** in Rust: too many small tasks can hurt more than help. ## Step 4 — pick the biggest lever first Prioritize in this order unless evidence suggests otherwise: 1. **Algorithmic complexity** wins (O(N²) → O(N log N) or O(N)) 2. **Data structure choice / memory layout** (cache locality; fewer cache lines) 3. **Allocation reduction** (fewer allocs, better reuse) 4. **Avoid unnecessary work** (fast paths, precompute, defer) 5. **Language boundary optimization** (minimize Python↔Rust crossings) 6. **Compiler/interpreter friendliness** (simplify hot loops, reduce abstraction overhead) ## Step 5 — produce an LLM-friendly output When you respond to the user, use this structure: 1. **Hot path hypothesis** (what you think dominates, and why) 2. **Top issues** (ranked): _issue → evidence/estimate → proposed fix → expected impact_ 3. **Patch sketch** (minimal code changes or pseudocode) 4. **Tradeoffs & risks** (correctness, memory, API, complexity) 5. **Measurement plan** (what to profile/benchmark and success criteria) --- # Techniques and patterns ## 1) API design for performance ### Use bulk APIs to amortize overhead **When:** callers do N similar operations (lookups, deletes, updates, decoding, locking). **Why:** reduce boundary crossings and repeated fixed costs (locks, dispatch, decoding, syscalls). **Python patterns:** ```python # BAD: N database round trips for user_id in user_ids: user = await User.objects.aget(id=user_id) # GOOD: 1 database round trip users = await sync_to_async(list)(User.objects.filter(id__in=user_ids)) users_by_id = {u.id: u for u in users} ``` **Rust patterns:** ```rust // BAD: N individual operations for id in ids { let result = cache.get(&id)?; // ... } // GOOD: Batch lookup let results = cache.get_many(&ids)?; ``` ### Prefer view types for function arguments **Python:** - Use `Sequence[T]` or `Iterable[T]` instead of `list[T]` when you don't mutate - Accept `bytes` or `memoryview` instead of copying to `bytearray` - Use `msgspec.Struct` for zero-copy deserialization **Rust:** - Use `&[T]` or `impl AsRef<[T]>` instead of `Vec` when you don't need ownership - Use `&str` instead of `String` for read-only string access - Use `Cow<'_, T>` when you might need to own or borrow ### Thread-compatible vs thread-safe types **Python:** - Default to thread-compatible (external GIL or explicit locks) - Use `threading.local()` for per-thread state - Prefer `asyncio` over threads for I/O-bound work **Rust:** - Default to `Send + Sync` for shared state - Use `Arc>` only when needed; prefer message passing - Consider `dashmap` or sharded maps for high-contention scenarios --- ## 2) Algorithmic improvements The rare-but-massive wins. ### Reduce complexity class Common transformations: - O(N²) → O(N log N) or O(N) - O(N log N) sorted-list intersection → O(N) using a hash set - O(log N) tree lookup → O(1) using hash lookup ### Python-specific patterns ```python # BAD: O(N²) - checking membership in list for item in items: if item in seen_list: # O(N) lookup continue seen_list.append(item) # GOOD: O(N) - using set seen = set() for item in items: if item in seen: # O(1) lookup continue seen.add(item) # BETTER: O(N) - using dict.fromkeys for deduplication unique_items = list(dict.fromkeys(items)) ``` ### Rust-specific patterns ```rust // BAD: O(N²) - nested iteration for a in &items_a { for b in &items_b { if a.key == b.key { // ... } } } // GOOD: O(N) - hash lookup let b_map: HashMap<_, _> = items_b.iter().map(|b| (&b.key, b)).collect(); for a in &items_a { if let Some(b) = b_map.get(&a.key) { // ... } } ``` --- ## 3) Better memory representation and cache locality ### Python: Prefer slots and dataclasses ```python # BAD: Regular class with __dict__ class Item: def __init__(self, x, y, z): self.x = x self.y = y self.z = z # GOOD: Slots-based class (smaller memory, faster attribute access) class Item: __slots__ = ('x', 'y', 'z') def __init__(self, x, y, z): self.x = x self.y = y self.z = z # BETTER: msgspec.Struct (even more compact, fast serialization) class Item(msgspec.Struct): x: int y: int z: int ``` ### Python: Prefer numpy/array for numeric data ```python # BAD: List of floats (each is a Python object) values = [1.0, 2.0, 3.0, ...] # ~28 bytes per float # GOOD: numpy array (contiguous, cache-friendly) values = np.array([1.0, 2.0, 3.0, ...]) # 8 bytes per float # GOOD: array module for simpler cases from array import array values = array('d', [1.0, 2.0, 3.0, ...]) ``` ### Rust: Memory layout and padding ```rust // BAD: 24 bytes due to padding struct Item { flag: bool, // 1 byte + 7 padding value: i64, // 8 bytes count: i32, // 4 bytes + 4 padding } // GOOD: 16 bytes with reordering struct Item { value: i64, // 8 bytes count: i32, // 4 bytes flag: bool, // 1 byte + 3 padding } // Use #[repr(C)] or #[repr(packed)] when ABI matters ``` ### Rust: Indices instead of pointers ```rust // Pointer-heavy: 8 bytes per reference, poor cache locality struct Node { data: i32, left: Option>, right: Option>, } // Index-based: 4 bytes per reference, better cache locality struct Tree { nodes: Vec, } struct NodeData { data: i32, left: Option, // index into nodes right: Option, } ``` ### Rust: SmallVec and tinyvec for small collections ```rust use smallvec::SmallVec; // Stack allocation for up to 8 elements, heap only if larger let mut items: SmallVec<[Item; 8]> = SmallVec::new(); ``` --- ## 4) Reduce allocations ### Python: Avoid unnecessary allocations ```python # BAD: Creates new list on every call def process(items): return [transform(item) for item in items] # GOOD: Generator for streaming (no intermediate list) def process(items): for item in items: yield transform(item) # GOOD: Pre-allocate when size is known def process(items): result = [None] * len(items) for i, item in enumerate(items): result[i] = transform(item) return result ``` ### Python: Reuse objects ```python # BAD: New dict on every iteration for data in stream: result = {} # allocation result['key'] = process(data) yield result # GOOD: Reuse dict (if consumers don't hold reference) result = {} for data in stream: result.clear() result['key'] = process(data) yield result # Caveat: only if consumer processes immediately ``` ### Python: Use __slots__ to reduce memory ```python # Without __slots__: ~152 bytes per instance class Point: def __init__(self, x, y): self.x = x self.y = y # With __slots__: ~56 bytes per instance class Point: __slots__ = ('x', 'y') def __init__(self, x, y): self.x = x self.y = y ``` ### Rust: Pre-allocate with capacity ```rust // BAD: Multiple reallocations let mut results = Vec::new(); for item in items { results.push(transform(item)); } // GOOD: Single allocation let mut results = Vec::with_capacity(items.len()); for item in items { results.push(transform(item)); } // BETTER: Use collect with size hint let results: Vec<_> = items.iter().map(transform).collect(); ``` ### Rust: Avoid cloning when borrowing suffices ```rust // BAD: Unnecessary clone fn process(data: &Data) -> String { let s = data.name.clone(); // allocation format!("Hello, {}", s) } // GOOD: Borrow fn process(data: &Data) -> String { format!("Hello, {}", &data.name) } ``` --- ## 5) Avoid unnecessary work ### Fast paths for common cases **Python:** ```python # BAD: Always takes slow path def parse_int(s: str) -> int: return int(s) # Handles all edge cases # GOOD: Fast path for common ASCII digits def parse_int(s: str) -> int: if len(s) <= 10 and s.isdigit(): # Fast check result = 0 for c in s: result = result * 10 + (ord(c) - 48) return result return int(s) # Fallback for edge cases ``` **Rust:** ```rust // Fast path for single-byte varint (common case) fn parse_varint(data: &[u8]) -> (u64, usize) { if data[0] < 128 { return (data[0] as u64, 1); // 90% of cases } parse_varint_slow(data) // Rare multi-byte } ``` ### Defer expensive computations **Python:** ```python # BAD: Always computes expensive value def process(data, config): expensive = compute_expensive(data) # Always runs if config.needs_expensive: use(expensive) # GOOD: Defer until needed def process(data, config): if config.needs_expensive: expensive = compute_expensive(data) # Only when needed use(expensive) ``` ### Move loop-invariant code outside loops ```python # BAD: Repeated attribute lookup for item in items: result = self.config.transform_fn(item) # 2 lookups per iteration # GOOD: Hoist invariant lookups transform = self.config.transform_fn for item in items: result = transform(item) ``` ### Cache computed results ```python from functools import lru_cache # Cache expensive computations @lru_cache(maxsize=1024) def expensive_computation(key: str) -> Result: # ... ``` --- ## 6) Python↔Rust boundary optimization (PyO3) ### Minimize GIL crossings ```rust // BAD: Acquire/release GIL for each item fn process_items(py: Python, items: Vec) -> PyResult> { items.iter().map(|item| { // Each call acquires GIL internally process_one(py, item) }).collect() } // GOOD: Hold GIL for batch, release for Rust-only work fn process_items(py: Python, items: Vec) -> PyResult> { // Extract data while holding GIL let data: Vec<_> = items.iter().map(|i| extract_data(py, i)).collect::>()?; // Release GIL for CPU-intensive work let results = py.allow_threads(|| { data.par_iter().map(|d| rust_process(d)).collect() }); // Re-acquire GIL to build Python objects results.iter().map(|r| to_python(py, r)).collect() } ``` ### Batch data across the boundary ```python # BAD: N PyO3 calls for item in items: rust_process(item) # GOOD: Single PyO3 call with batch rust_process_batch(items) ``` ### Use zero-copy types ```rust // Accept bytes without copying #[pyfunction] fn process_bytes(data: &[u8]) -> PyResult> { // data is a view into Python bytes, no copy Ok(transform(data)) } // Use Py for owned bytes without copying #[pyfunction] fn process_bytes_owned(py: Python, data: Py) -> PyResult> { let bytes = data.as_bytes(py); // ... } ``` --- ## 7) Async optimization ### Python async patterns ```python # BAD: Sequential await async def fetch_all(urls): results = [] for url in urls: results.append(await fetch(url)) # Sequential return results # GOOD: Concurrent await async def fetch_all(urls): return await asyncio.gather(*[fetch(url) for url in urls]) # BETTER: Bounded concurrency async def fetch_all(urls, max_concurrent=10): semaphore = asyncio.Semaphore(max_concurrent) async def fetch_with_limit(url): async with semaphore: return await fetch(url) return await asyncio.gather(*[fetch_with_limit(url) for url in urls]) ``` ### Rust async patterns ```rust // BAD: Sequential await async fn fetch_all(urls: Vec) -> Vec { let mut results = Vec::new(); for url in urls { results.push(fetch(&url).await); // Sequential } results } // GOOD: Concurrent with join_all async fn fetch_all(urls: Vec) -> Vec { futures::future::join_all(urls.iter().map(fetch)).await } // BETTER: Bounded concurrency with buffer_unordered use futures::stream::{self, StreamExt}; async fn fetch_all(urls: Vec) -> Vec { stream::iter(urls) .map(|url| fetch(url)) .buffer_unordered(10) // Max 10 concurrent .collect() .await } ``` --- ## 8) Reduce logging and stats costs ### Python logging in hot paths ```python # BAD: Logging overhead even when disabled for item in items: logger.debug(f"Processing {item}") # String formatting always happens # GOOD: Check level first if logger.isEnabledFor(logging.DEBUG): for item in items: logger.debug(f"Processing {item}") # BETTER: Remove logging from innermost loops entirely ``` ### Rust logging in hot paths ```rust // BAD: log! macro overhead in hot loop for item in items { log::debug!("Processing {:?}", item); // Format even if disabled } // GOOD: Check level outside loop if log::log_enabled!(log::Level::Debug) { for item in items { log::debug!("Processing {:?}", item); } } // BETTER: Use tracing with static filtering #[tracing::instrument(skip_all)] fn process_batch(items: &[Item]) { // Span created once, not per item for item in items { // Hot loop without logging } } ``` --- # Flat-profile playbook If no single hotspot dominates: 1. Don't discount many small wins (twenty 1% improvements can matter). 2. Look for loops closer to the top of call stacks (flame graphs help). 3. Consider structural refactors (one-shot construction instead of incremental mutation). 4. Replace overly general abstractions with specialized code. 5. Reduce allocations (allocation profiles help). 6. Use hardware counters (cache misses, branch misses) to find invisible costs. 7. **Python-specific:** Look for GIL contention, excessive object creation, slow imports. 8. **Rust-specific:** Look for excessive async overhead, unnecessary clones, poor cache locality. --- # Quick review checklist When reviewing a performance-sensitive change, scan for: ## Python - [ ] Any O(N²) behavior on realistic N? - [ ] List comprehension where generator would suffice? - [ ] Missing `__slots__` on frequently instantiated classes? - [ ] dict/list allocation inside hot loops? - [ ] Sequential awaits where concurrent would work? - [ ] Repeated attribute lookups in loops? - [ ] Logging/debug in innermost loops? - [ ] Using stdlib json instead of msgspec/orjson? - [ ] Synchronous I/O in async context? - [ ] Missing @lru_cache for repeated expensive computations? ## Rust - [ ] Any O(N²) behavior on realistic N? - [ ] Missing `with_capacity()` for known-size collections? - [ ] Unnecessary `.clone()` where borrow would work? - [ ] String allocation in hot loops (use `&str` or `Cow`)? - [ ] Box/Arc where stack allocation would work? - [ ] Excessive async task spawning for small operations? - [ ] Poor struct field ordering (padding waste)? - [ ] Missing `#[inline]` on critical small functions? - [ ] Lock contention in concurrent code? - [ ] Logging format overhead in hot paths? ## PyO3 boundary - [ ] Multiple GIL acquisitions where one would suffice? - [ ] Python object creation in hot Rust loops? - [ ] Missing `allow_threads()` for CPU-bound Rust work? - [ ] Copying data across boundary when view would work? - [ ] Single-item APIs where batch API would reduce crossings? --- # Examples (inputs → outputs) ## Example A — Python list vs generator **Input:** ```python def process_large_file(path): lines = open(path).readlines() # Load entire file return [transform(line) for line in lines] # Another full copy ``` **Agent output:** - Issue: Two full copies of file contents; O(N) memory for O(N) items. - Fix: Use generator for streaming: ```python def process_large_file(path): with open(path) as f: for line in f: yield transform(line) ``` - Expected impact: O(1) memory instead of O(N). - Tradeoff: Can only iterate once; caller must handle streaming. ## Example B — Rust unnecessary clones **Input:** ```rust fn build_response(data: &ResponseData) -> String { let name = data.name.clone(); let id = data.id.clone(); format!("User {} ({})", name, id) } ``` **Agent output:** - Issue: Unnecessary clones; format! can borrow directly. - Fix: ```rust fn build_response(data: &ResponseData) -> String { format!("User {} ({})", &data.name, &data.id) } ``` - Expected impact: 2 fewer allocations per call. ## Example C — PyO3 batch optimization **Input:** ```python # Python side for item in items: result = rust_validate(item) # 10K calls ``` **Agent output:** - Issue: 10K GIL acquisitions/releases (~2-5ms overhead alone). - Fix: Add batch API: ```rust #[pyfunction] fn rust_validate_batch(items: Vec) -> Vec { items.iter().map(|i| validate(i)).collect() } ``` ```python results = rust_validate_batch(items) # 1 call ``` - Expected impact: ~10-50x faster for large batches. - Tradeoff: Must collect all items first; can't stream results.