httptap

# httptap
Releases CI & Quality Security Project Info
PyPI
Python Versions
CI
Coverage
CodSpeed Badge
CodeQL
OpenSSF Scorecard
OpenSSF Best Practices
OpenSSF Baseline
Build Tool
Lint
License
`httptap` is a rich-powered CLI that dissects an HTTP request into every meaningful phase-DNS, TCP connect, TLS handshake, server wait, and body transfer and renders the results as a timeline table, compact summary, or machine-friendly metrics. It is designed for interactive troubleshooting, regression analysis, and recording of performance baselines. --- ## Table of Contents - [Highlights](#highlights) - [How it compares](#how-it-compares) - [Requirements](#requirements) - [Installation](#installation) - [Using Homebrew (macOS/Linux)](#using-homebrew-macoslinux) - [Using `uv`](#using-uv) - [Using `pip`](#using-pip) - [From source](#from-source) - [Shell completions](#shell-completions) - [Quick Start](#quick-start) - [Basic GET Request](#basic-get-request) - [POST Request with Data](#post-request-with-data) - [Other HTTP Methods](#other-http-methods) - [Custom Headers](#custom-headers) - [Redirects and JSON Export](#redirects-and-json-export) - [Output Modes](#output-modes) - [Advanced Usage](#advanced-usage) - [SLO Threshold Checking](#slo-threshold-checking) - [Environment Variables](#environment-variables) - [Exit Codes](#exit-codes) - [Releasing](#releasing) - [Sample Output](#sample-output) - [JSON Export Structure](#json-export-structure) - [Metrics-only scripting](#metrics-only-scripting) - [Advanced Usage](#advanced-usage-1) - [Development](#development) - [Contributing](#contributing) - [License](#license) - [Acknowledgements](#acknowledgements) --- ## Highlights - **Phase-by-phase timing** – precise measurements built from httpcore trace hooks (with sane fallbacks when metal-level data is unavailable). - **All HTTP methods** – GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS with request body support. - **Request body support** – send JSON, XML, or any data inline or from file with automatic Content-Type detection. - **IPv4/IPv6 aware** – the resolver and TLS inspector report both the address and its family. - **TLS insights** – certificate CN, expiry countdown, cipher suite, and protocol version are captured automatically. - **Multiple output modes** – rich waterfall view, compact single-line summaries, or `--metrics-only` for scripting. - **JSON export** – persist full step data (including redirect chains) for later processing. - **SLO threshold checking** – `--slo total=500,ttfb=200` gates CI jobs, cron probes, and readiness checks on per-phase latency budgets; non-zero exit on violation while still rendering the full report. - **Extensible** – clean Protocol interfaces for DNS, TLS, timing, visualization, and export so you can plug in custom behavior. > 📣 Exclusive for httptap users: Save 50% on GitKraken Pro. Bundle GitKraken Client, GitLens for VS Code, and powerful CLI tools to accelerate every repo workflow. --- ## How it compares | Feature | `httptap` | `curl -w` | [`httpstat`](https://github.com/reorx/httpstat) | `httpie` | |------------------------------------------|:---------:|:----------------------:|:-----------------------------------------------:|:-----------------:| | Phase-by-phase timing (DNS/TCP/TLS/TTFB) | ✅ | ✅ (format str) | ✅ | ❌ | | Rich waterfall visualization | ✅ | ❌ | ⚠️ text bars | ❌ | | Redirect chain with per-step timing | ✅ | ❌ | ❌ | ❌ | | JSON export (machine-readable) | ✅ | ✅ (`-w '%{json}'`) | ✅ (`--format json/jsonl`, v1 schema) | ❌ (no metrics) | | Metrics-only mode for scripting | ✅ | ✅ | ✅ (`--format json`) | ❌ | | SLO threshold checking | ✅ (`--slo`) | ❌ | ✅ (`--slo total=500,...`) | ❌ | | TLS certificate inspection (CN, expiry) | ✅ | ⚠️ via `-v` | ❌ | ❌ | | IPv4/IPv6 reporting | ✅ family | ⚠️ IP via `remote_ip` | ⚠️ IP only (`remote_ip`/`remote_port`) | ❌ | | HTTP/2 support | ✅ | ✅ | ⚠️ via curl passthrough | ⚠️ plugin only | | Proxy with source attribution | ✅ | ⚠️ no attribution | ⚠️ via curl passthrough | ⚠️ no attribution | | Custom CA bundle | ✅ | ✅ | ⚠️ via curl passthrough | ✅ | | Extensible Python API | ✅ | ❌ (pycurl ≠ same API) | ❌ | ⚠️ via requests | | Curl-compatible flags | ✅ | — | ✅ (passes through) | ❌ | | Zero system dependencies | ✅ | ✅ | needs curl | ✅ | **When to pick what:** - **`httptap`** — interactive troubleshooting, regression analysis, and scripted baselines with structured JSON. - **`curl -w`** — one-off shell checks where curl is already the dependency. - **`httpstat`** — quick visual breakdown on top of an existing curl install. - **`httpie`** — general-purpose request/response exploration, not latency profiling. --- ## Requirements - Python 3.10-3.15 (CPython) - macOS, Linux, or Windows (tested on CPython) - No system dependencies beyond standard networking - Code must follow the Google Python Style Guide (docstrings, formatting). See [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html) --- ## Installation ### Using Homebrew (macOS/Linux) ```shell brew install httptap ``` ### Using `uv` ```shell uv pip install httptap ``` ### Using `pip` ```shell pip install httptap ``` ### Container image ```shell docker run --rm ghcr.io/ozeranskii/httptap:latest https://example.com ``` Multi-arch (linux/amd64, linux/arm64), signed with cosign (keyless Sigstore) and shipped with SLSA build provenance. ### From source ```shell git clone https://github.com/ozeranskii/httptap.git cd httptap uv venv uv pip install . ``` --- ### Shell completions #### Homebrew Installation If you installed httptap via Homebrew, shell completions are automatically available after installation. Just restart your shell: ```shell # Restart your shell or reload configuration exec $SHELL ``` Homebrew automatically installs completions to: - Bash: `$(brew --prefix)/etc/bash_completion.d/` - Zsh: `$(brew --prefix)/share/zsh/site-functions/` #### Python Package Installation If you installed httptap via `pip` or `uv`, you need to install the optional completion extras: 1. Install the completion extras: ```shell uv pip install "httptap[completion]" # or pip install "httptap[completion]" ``` 2. Activate your virtual environment: ```shell source .venv/bin/activate ``` 3. Run the global activation script for argument completions: ```shell activate-global-python-argcomplete ``` 4. Restart your shell. Completions should now work in both bash and zsh. **Note:** The global activation script provides argument completions for bash and zsh only. Other shells are not covered by the script and must be configured separately. #### Usage Examples Once completions are installed, you can use `Tab` to autocomplete commands and options: ```shell # Complete command options httptap -- # Shows: --method, --data, --follow, --timeout, --no-http2, --ignore-ssl, --cacert, --proxy, --header, --compact, --metrics-only, --json, --version, --help # Complete after typing partial option httptap --fol # Completes to: httptap --follow # Complete multiple options httptap --follow --time # Completes to: httptap --follow --timeout ``` --- ## Quick Start ### Basic GET Request Run a single request and display a rich waterfall: ```shell httptap https://httpbin.io/get ``` ### POST Request with Data Send JSON data (auto-detects Content-Type): ```shell httptap https://httpbin.io/post --data '{"name": "John", "email": "john@example.com"}' ``` **Note:** When `--data` is provided without `--method`, httptap automatically switches to POST (similar to curl). **Curl-compatible flags:** httptap accepts the most common curl syntax, so you can often replace `curl` with `httptap` directly. Aliases include `-X/--request` for `--method`, `-L/--location` for `--follow`, `-m/--max-time` for `--timeout`, `-k/--insecure` for `--ignore-ssl`, `-x` for `--proxy`, and `--http1.1` for `--no-http2`. (Not every curl option is supported—stick to these shared flags when swapping commands.) Load data from file: ```shell httptap https://httpbin.io/post --data @payload.json ``` Explicitly specify method (bypasses auto-POST): ```shell httptap https://httpbin.io/post --method POST --data '{"status": "active"}' ``` ### Other HTTP Methods PUT request: ```shell httptap https://httpbin.io/put --method PUT --data '{"key": "value"}' ``` PATCH request: ```shell httptap https://httpbin.io/patch --method PATCH --data '{"field": "updated"}' ``` DELETE request: ```shell httptap https://httpbin.io/delete --method DELETE ``` ### Custom Headers Add custom headers (repeat `-H` for multiple values): ```shell httptap \ -H "Accept: application/json" \ -H "Authorization: Bearer super-secret" \ https://httpbin.io/bearer ``` ### Redirects and JSON Export Follow redirect chains and dump metrics to JSON: ```shell httptap --follow --json out/report.json https://httpbin.io/redirect/2 ``` ### Output Modes Collect compact (single-line) timings suitable for logs: ```shell httptap --compact https://httpbin.io/get ``` Expose raw metrics for scripts: ```shell httptap --metrics-only https://httpbin.io/get | tee timings.log ``` ### Advanced Usage Programmatic users can inject a custom executor for advanced scenarios. Provide your own `RequestExecutor` implementation if you need to change how requests are executed (for example, to plug in a different HTTP stack or add tracing). #### TLS Certificate Options Bypass TLS verification when troubleshooting self-signed endpoints: ```shell httptap --ignore-ssl https://self-signed.badssl.com ``` The flag disables certificate validation and relaxes many handshake constraints so that legacy endpoints (expired/self-signed/hostname mismatches, weak hashes, older TLS versions) still complete. Some algorithms removed from modern OpenSSL builds (for example RC4 or 3DES) may remain unavailable. Use this mode only on trusted networks. Use a custom CA certificate bundle for internal APIs: ```shell httptap --cacert /path/to/company-ca.pem https://internal-api.company.com ``` This is useful when testing internal services that use certificates signed by a custom Certificate Authority (CA) that isn't in the system's default trust store. The `--cacert` option (also available as `--ca-bundle`) accepts a path to a PEM-formatted CA certificate bundle. **Note:** `--ignore-ssl` and `--cacert` are mutually exclusive. Use `--ignore-ssl` to disable all verification, or `--cacert` to verify with a custom CA bundle. When `--cacert` is used, the CLI output marks the connection with `TLS CA: custom bundle`, and JSON exports include `network.tls_custom_ca: true` so automation can detect custom trust configuration. Route traffic through an HTTP/SOCKS proxy (explicit override takes precedence over env vars `HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY`): ```shell httptap --proxy socks5h://proxy.local:1080 https://httpbin.io/get ``` Ignore all proxy environment variables and connect directly: ```shell httptap --proxy "" https://httpbin.io/get ``` The output and JSON export include the proxy URI and its source so you can confirm what path was used (e.g., `(from arg --proxy)`, `(from env HTTPS_PROXY)`, `(bypassed by env no_proxy)`). --- ## SLO Threshold Checking Gate CI jobs, cron probes, and Kubernetes readiness checks on per-phase latency budgets with `--slo KEY=MS[,KEY=MS...]`: ```shell httptap --slo total=500,ttfb=200 https://api.example.com/health ``` - Exits `0` when every threshold passes. - Exits `4` when at least one threshold is exceeded on the **final successful step** (intermediate redirects are not evaluated). - Exits `64` on malformed specification (unknown key, duplicate key, non-positive value, bad syntax). - The full waterfall / compact / JSON output is always rendered so the evidence for a regression is preserved. Supported keys: `dns`, `connect`, `tls`, `ttfb`, `wait`, `xfer`, `total`. Output extensions: - **Rich / compact** — a bordered panel after the waterfall lists the thresholds and any violations (actual, threshold, overrun). - **`--metrics-only`** — the final successful step carries `slo=pass` or `slo=fail slo_violations=` tokens. - **`--json`** — the `summary.slo` block contains `pass`, `thresholds_ms`, and per-violation `{key, threshold_ms, actual_ms, delta_ms}`. ```shell # CI gate — fail only on SLO violation, tolerate transient network errors httptap --slo total=2000,tls=300,ttfb=800 https://staging.example.com/ case $? in 0) echo "healthy" ;; 4) echo "SLO violation"; exit 1 ;; 75) echo "network flake, retrying later" ;; esac ``` Full specification, evaluation rules, and recipes: [docs.httptap.dev/usage/slo](https://docs.httptap.dev/usage/slo/). --- ## Environment Variables httptap reads the following environment variables at runtime. All of them are overridable via CLI flags, and the actual source used for each request is recorded in the output and JSON export. | Variable | Purpose | Overridden by | |---------------------------------------|--------------------------------------------------------------------------------------------------------------|-----------------------| | `HTTP_PROXY` / `http_proxy` | Proxy URL used for `http://` targets. | `-x/--proxy` | | `HTTPS_PROXY` / `https_proxy` | Proxy URL used for `https://` targets. | `-x/--proxy` | | `ALL_PROXY` / `all_proxy` | Fallback proxy URL when scheme-specific variables are unset. | `-x/--proxy` | | `NO_PROXY` / `no_proxy` | Comma-separated exclusion list (supports `*`, leading `.`, exact matches). Bypassed entries connect direct. | `--proxy ""` | | `NO_COLOR` | Disables ANSI colors in all Rich output (honors the [NO_COLOR](https://no-color.org) convention). | — | | `FORCE_COLOR` | Forces colored output even when stdout is not a TTY (Rich convention). | — | | `TERM=dumb` | Rich downgrades to plain-text rendering. | — | > Precedence for proxy configuration: explicit `-x/--proxy` → `--proxy ""` > (disables env) → `HTTPS_PROXY`/`HTTP_PROXY`/`ALL_PROXY` (scheme-matching) → > `NO_PROXY` exclusion → direct connection. --- ## Exit Codes httptap follows the BSD `sysexits.h` convention so it integrates cleanly with shell pipelines, CI jobs, and systemd services. | Code | Symbol | Meaning | |:-----:|-------------------------|------------------------------------------------------------| | `0` | `EX_OK` | Success. | | `4` | — | SLO threshold violation (request succeeded but too slow). | | `64` | `EX_USAGE` | Invalid command-line arguments. | | `70` | `EX_SOFTWARE` | Internal error (unexpected exception, bug). | | `75` | `EX_TEMPFAIL` | Network / TLS error (partial output may still be rendered). | | `128 + N` | Signal offset | Killed by signal `N` (e.g., `130` for `SIGINT` / Ctrl-C). | Example — fail a CI job only on usage errors, tolerating transient network issues: ```shell httptap --metrics-only https://api.example.com/health rc=$? if [ "$rc" = 64 ] || [ "$rc" = 70 ]; then exit "$rc" fi ``` --- ## Releasing ### Prerequisites - GitHub Environment `pypi` must be configured in repository settings - PyPI Trusted Publishing configured for `ozeranskii/httptap` ### Steps 1. Trigger the **Release** workflow from GitHub Actions: - Provide exact version (e.g., `0.3.0`), OR - Select bump type: `patch`, `minor`, or `major` 2. The workflow will: - Update version in `pyproject.toml` using `uv version` - Generate changelog with `git-cliff` and update `CHANGELOG.md` - Commit changes and create a git tag - Run full test suite on the tagged version - Build wheel and source distribution - Generate SBOM in CycloneDX and SPDX formats via Syft - Attach the current OpenVEX document (`.vex/httptap.openvex.json`) - Publish to PyPI via Trusted Publishing (OIDC) - Create GitHub Release with wheel, sdist, SBOM, and VEX assets --- ## Sample Output ![sample-output.png](docs/assets/sample-output.png) The redirect summary includes a total row: ![sample-follow-redirects-output.png](docs/assets/sample-follow-redirects-output.png) --- ## JSON Export Structure ```json { "initial_url": "https://httpbin.io/redirect/2", "total_steps": 3, "steps": [ { "url": "https://httpbin.io/redirect/2", "step_number": 1, "request": { "method": "GET", "headers": {}, "body_bytes": 0 }, "timing": { "dns_ms": 8.947208058089018, "connect_ms": 96.97712492197752, "tls_ms": 194.56583401188254, "ttfb_ms": 445.9513339679688, "total_ms": 447.3437919514254, "wait_ms": 145.46116697601974, "xfer_ms": 1.392457983456552, "is_estimated": false }, "network": { "ip": "44.211.11.205", "ip_family": "IPv4", "http_version": "HTTP/2.0", "tls_version": "TLSv1.2", "tls_cipher": "ECDHE-RSA-AES128-GCM-SHA256", "cert_cn": "httpbin.io", "cert_days_left": 143, "tls_verified": true, "tls_custom_ca": null, "proxy_url": null, "proxy_source": null }, "response": { "status": 302, "bytes": 0, "content_type": null, "server": null, "date": "2025-10-23T19:20:36+00:00", "location": "/relative-redirect/1", "headers": { "access-control-allow-credentials": "true", "access-control-allow-origin": "*", "location": "/relative-redirect/1", "date": "Thu, 23 Oct 2025 19:20:36 GMT", "content-length": "0" } }, "error": null, "note": null, "proxy": null }, { "url": "https://httpbin.io/relative-redirect/1", "step_number": 2, "request": { "method": "GET", "headers": {}, "body_bytes": 0 }, "timing": { "dns_ms": 2.6895420160144567, "connect_ms": 97.51500003039837, "tls_ms": 193.99016606621444, "ttfb_ms": 400.2034160075709, "total_ms": 400.60841606464237, "wait_ms": 106.00870789494365, "xfer_ms": 0.4050000570714474, "is_estimated": false }, "network": { "ip": "44.211.11.205", "ip_family": "IPv4", "http_version": "HTTP/2.0", "tls_version": "TLSv1.2", "tls_cipher": "ECDHE-RSA-AES128-GCM-SHA256", "cert_cn": "httpbin.io", "cert_days_left": 143, "tls_verified": true, "tls_custom_ca": null, "proxy_url": null, "proxy_source": null }, "response": { "status": 302, "bytes": 0, "content_type": null, "server": null, "date": "2025-10-23T19:20:36+00:00", "location": "/get", "headers": { "access-control-allow-credentials": "true", "access-control-allow-origin": "*", "location": "/get", "date": "Thu, 23 Oct 2025 19:20:36 GMT", "content-length": "0" } }, "error": null, "note": null, "proxy": null }, { "url": "https://httpbin.io/get", "step_number": 3, "request": { "method": "GET", "headers": {}, "body_bytes": 0 }, "timing": { "dns_ms": 2.643457963131368, "connect_ms": 97.36416593659669, "tls_ms": 197.3062080796808, "ttfb_ms": 403.2038329169154, "total_ms": 403.9644579170272, "wait_ms": 105.89000093750656, "xfer_ms": 0.7606250001117587, "is_estimated": false }, "network": { "ip": "52.70.33.41", "ip_family": "IPv4", "http_version": "HTTP/2.0", "tls_version": "TLSv1.2", "tls_cipher": "ECDHE-RSA-AES128-GCM-SHA256", "cert_cn": "httpbin.io", "cert_days_left": 143, "tls_verified": true, "tls_custom_ca": null, "proxy_url": null, "proxy_source": null }, "response": { "status": 200, "bytes": 389, "content_type": "application/json; charset=utf-8", "server": null, "date": "2025-10-23T19:20:37+00:00", "location": null, "headers": { "access-control-allow-credentials": "true", "access-control-allow-origin": "*", "content-type": "application/json; charset=utf-8", "date": "Thu, 23 Oct 2025 19:20:37 GMT", "content-length": "389" } }, "error": null, "note": null, "proxy": null } ], "summary": { "total_time_ms": 1251.916665933095, "final_status": 200, "final_url": "https://httpbin.io/get", "final_bytes": 389, "errors": 0 } } ``` ## Metrics-only scripting ```shell httptap --metrics-only https://httpbin.io/get ``` ```terminaloutput Step 1: dns=30.1 connect=97.3 tls=199.0 ttfb=472.2 total=476.0 status=200 bytes=389 ip=44.211.11.205 family=IPv4 tls_version=TLSv1.2 proxy=direct ``` --- ## Advanced Usage ### Custom Implementations Swap in your own resolver or TLS inspector (anything satisfying the Protocol from `httptap.interfaces`): ```python from httptap import HTTPTapAnalyzer, SystemDNSResolver class HardcodedDNS(SystemDNSResolver): def resolve(self, host, port, timeout): return "93.184.216.34", "IPv4", 0.1 analyzer = HTTPTapAnalyzer(dns_resolver=HardcodedDNS()) steps = analyzer.analyze_url("https://httpbin.io") ``` --- ## Development ```shell git clone https://github.com/ozeranskii/httptap.git cd httptap uv sync uv run pytest uv run ruff check uv run ruff format . ``` Tests expect outbound network access; you can mock `SystemDNSResolver` / `SocketTLSInspector` when running offline. --- ## Contributing 1. Fork and clone the repo. 2. Create a feature branch. 3. Run `pytest` and `ruff` before committing. 4. Submit a pull request with a clear description and any relevant screenshots or benchmarks. We welcome bug reports, feature proposals, doc improvements, and creative new visualizations or exporters. --- ## License Apache License 2.0 © Sergei Ozeranskii. See [LICENSE](https://github.com/ozeranskii/httptap/blob/main/LICENSE) for details. --- ## Acknowledgements - Built on the shoulders of fantastic libraries: [httpx](https://www.python-httpx.org/), [httpcore](https://github.com/encode/httpcore), [dnspython](https://www.dnspython.org/), and [Rich](https://github.com/Textualize/rich). - Inspired by the tooling ecosystem around web performance (e.g., DevTools waterfalls, `curl --trace`). - Special thanks to everyone who opens issues, shares ideas, or contributes patches.