# Choosing ExploitSpec

ExploitSpec v0.2 is intentionally narrow: it turns an already-confirmed HTTP
security finding into a reviewable regression test. It does not replace a
scanner, a general API test suite, a load test, or application-level integration
tests.

This guide is a decision aid, not a benchmark. The tools below overlap, and a
healthy test program will often use more than one. Product behavior was checked
against the linked official documentation on 2026-08-25; those sources remain
the authority as the projects evolve.

## Quick decision

- Need to **find unknown vulnerabilities**? Start with authorized scanning and
  human security testing. Add ExploitSpec after a finding is confirmed.
- Need concise, general-purpose **HTTP integration tests**? Start with Hurl.
- Need **load, performance, or reliability under concurrency**? Start with k6.
- Need an interactive, collaborative **API client and collection workflow**?
  Start with Postman; use Newman or the current Postman CLI where appropriate.
- Need arbitrary application fixtures, database access, or internal assertions?
  Keep the test in the application's ordinary framework, such as pytest.
- Need a plain, local **multi-actor security regression** with an explicit
  vulnerable/fixed calibration step? ExploitSpec is designed for that case.

## Capability map

Short cells describe built-in concepts, not everything that can be assembled
with scripts, plugins, or surrounding CI jobs.

| Tool | Discovery or known finding? | Actor and session model | Dynamic captures | RED/GREEN/STABLE calibration | Local and CI | HAR import privacy boundary | Account and telemetry | Authoring and runtime |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| **ExploitSpec v0.2** | Known, confirmed HTTP finding; deliberately not discovery. | Named actors with independent headers and cookie jars. | Built-in JSON-path, response-header, and regex captures, then interpolation into later steps. | Built-in `calibrate`: the invariant must fail on the vulnerable target, pass on the fixed target, then keep passing for the requested runs. | Local CLI and GitHub Action; text, JSON, and JUnit output. | Bounded HAR 1.2 input; value-free listing; one explicitly selected entry; no replay or upload; captured origin, path segments, names, and supported values become placeholders. | No account, hosted control plane, analytics, or telemetry. | Strict YAML v1 specs; prebuilt Go CLI. |
| **Hurl** | Known HTTP behavior and general HTTP workflows; not a vulnerability scanner. | Entries in one file share a cookie store. Additional identities require explicit headers/cookies or separate files/runs; there is no named actor construct. | Native captures from status, headers, cookies, JSONPath, XPath, regex, and other response properties. | No named built-in equivalent; multiple targets and repeated runs can be orchestrated externally. | Local single-binary CLI, CI examples, and HTML, JSON, JUnit, and TAP reports. | A cURL-to-Hurl conversion path is documented, but no native HAR importer is documented in the current manual; HAR conversion and sanitization would be separate steps. | No account is required in the documented CLI flow; telemetry behavior is not documented in the official user docs. | Plain `.hurl` files; native Rust binary powered by libcurl. |
| **Grafana k6** | Primarily load and performance testing with scripted checks; not vulnerability discovery. | A cookie jar per virtual user, with optional local jars; identities and roles are modeled in JavaScript/TypeScript and scenarios. | Response data can be assigned, parsed, correlated, and reused in code. | No named built-in equivalent; separate baselines and repetitions can be scripted or orchestrated. | Local, CI, distributed, and optional Grafana Cloud execution. Failed `check()` calls need thresholds or other gating logic to fail a CI run. | The `har-to-k6` Node.js converter writes a k6 script from recorded requests. Its documentation expects review and modification; it does not document ExploitSpec-style conservative redaction. | Local runs need no cloud account. The open-source CLI sends an anonymous usage report by default; `--no-usage-report` or `K6_NO_USAGE_REPORT` disables it. Grafana Cloud is optional. | JavaScript or TypeScript test scripts executed by the k6 binary; k6 does not use Node.js module resolution. |
| **Postman / Newman** | General API exploration, workflows, and known assertions. Collections run by Newman are not vulnerability discovery by themselves. | Authentication, variables, scripts, and cookie jars are available; separate role/session isolation is designed by the collection author. | Post-response JavaScript can read responses, set scoped variables, branch, and drive later requests. | No named built-in equivalent; environments and repeated collection runs can model the stages. | Newman runs exported collection JSON locally and in CI. Current Postman documentation notes that Newman does not run the newer collection v3 format; use the Postman CLI for that format. | Postman can turn HAR into collections for inspection and replay, including cookies when selected. Its official guidance tells users to sanitize HAR before sharing; no equivalent automatic redaction boundary is documented. | Newman can run a local exported file without the Postman API. Signed-in Postman workspaces sync data to Postman Cloud; account and data-flow choices depend on the workflow. | Collection JSON plus JavaScript scripts; Newman v6 requires Node.js 16 or later. |
| **pytest / ordinary integration tests** | Whatever the test code implements; neither HTTP behavior nor vulnerability discovery is built in. | Custom HTTP-client instances and fixtures can model any number of identities and lifecycles. | Unrestricted application code. | Not built in, but vulnerable/fixed targets and repetition can be encoded as fixtures or CI jobs. | Local and CI; pytest can emit JUnit XML. | No built-in HAR security import boundary; parsing, selection, redaction, and dependencies are the test author's responsibility. | pytest itself is local, free, and requires no account. Network and telemetry behavior depends on the HTTP libraries and plugins selected by the project. | Python tests plus the project's chosen HTTP client and dependencies. |
| **Security scanner (category; ZAP DAST is one example)** | Discovery and assessment through scan rules. Results are alerts; exact coverage varies by scanner. | Authentication and user/context support varies by product and configuration. | Product- and rule-specific. | Rescanning is useful, but it is not the same contract as proving one authored invariant RED, GREEN, and STABLE. | Varies. ZAP provides local Docker API scans with CI-friendly exit codes. | Varies. ZAP's API scan starts from OpenAPI, SOAP, or GraphQL definitions rather than the privacy boundary described here. | Varies by scanner and deployment model. | Varies by product; ZAP offers desktop, Docker, and automation interfaces. |

## Where ExploitSpec fits well

ExploitSpec is a reasonable choice when all or most of these statements are
true:

- a pentester, researcher, or developer has already demonstrated the bug;
- the durable requirement is a security invariant, such as “tenant B cannot
  read tenant A's object,” rather than broad endpoint correctness;
- the workflow needs independently stateful identities;
- later requests depend on IDs, headers, or text captured at runtime;
- the team can provide known-vulnerable and fixed targets for calibration;
- a small declarative file is preferable to embedding arbitrary test code; and
- local execution without an account or project telemetry matters.

Its tradeoffs are equally important. ExploitSpec v0.2 is HTTP-only, pre-1.0,
and intentionally less expressive than a programming language. It has no
browser automation, vulnerability discovery, automatic actor inference, or
general setup/cleanup framework. HAR import creates a single-request starter;
the author must still review it, define actors and later steps, add meaningful
captures and assertions, and calibrate the completed invariant.

## When another tool is the better starting point

### Hurl

Choose Hurl for compact HTTP sessions and broad response assertions when the
test is not specifically organized around a confirmed security finding. It has
a mature HTTP-oriented text format, rich capture queries, cookie continuity,
and several report formats. ExploitSpec adds value beside Hurl when named,
isolated actors and vulnerable/fixed security calibration are the central
review concepts.

### k6

Choose k6 when concurrency, throughput, latency, scenarios, or distributed load
are the question. A security invariant can be asserted in k6 code, but
ExploitSpec is not a load generator and k6's core abstraction is not a durable
confirmed-finding record. It can be useful to keep the same API under both:
ExploitSpec for regression of the authorization boundary and k6 for behavior
under load.

### Postman and Newman

Choose Postman for interactive request building, exploration, team workspaces,
and collection-based API workflows. Newman remains useful for exported
collection files and existing CI pipelines; check the current collection format
compatibility before choosing Newman for a new workflow. ExploitSpec is a
smaller fit when the desired repository artifact is a local security invariant
with explicit actor isolation and calibration rather than a general collection.

### pytest or the application's existing test framework

Choose ordinary integration tests when assertions need database state,
application internals, custom protocols, complex cleanup, or reusable code. The
cost is that security-specific conventions, actor isolation, safe target
selection, redaction, and calibration must be designed and reviewed by the
project. Keeping an existing, clear regression test is often better than
rewriting it only to adopt a new format.

### Security scanners

Choose scanners to search for classes of potential vulnerabilities across a
target. For example, ZAP's active scanner applies known attacks, while its own
documentation warns that automated active scanning does not find logical
vulnerabilities such as broken access control. Scanning and ExploitSpec therefore
serve different stages: one helps discover or flag, while the other preserves a
specific confirmed proof after remediation.

## A practical combined workflow

1. Use threat modeling, review, pentesting, and authorized scanners to find a
   candidate issue.
2. Reproduce it and decide what invariant proves the security boundary.
3. Keep an existing high-quality integration test if it already expresses the
   proof clearly; otherwise author an ExploitSpec when its narrower model fits.
4. Calibrate the ExploitSpec against controlled vulnerable and fixed baselines.
5. Run the invariant in local development and CI.
6. Continue using Hurl or Postman for broad API behavior, pytest for application
   integration, k6 for load, and scanners for new discovery.

## Treat every HAR as sensitive

A HAR can contain credentials, cookies, personal data, private hostnames, and
request bodies. ExploitSpec's importer reduces what reaches the generated YAML,
but it does not make the original HAR safe:

- the original file remains under the user's storage and retention policy;
- listing parses the bounded archive but prints value-free summaries;
- import requires an explicit one-based entry and never replays it;
- supported captured strings and structural names are replaced with required
  environment placeholders; and
- ambiguous, encoded, multipart, opaque, binary, or unsupported bodies are
  rejected rather than copied through.

Review the generated spec before committing or running it. For other tools,
follow their own data-flow documentation and sanitize recordings before they
enter a synced workspace, generated script, report, issue, or repository.

## Official sources

### ExploitSpec

- [Actors and independent sessions](specification.md#actors-and-sessions)
- [RED/GREEN/STABLE calibration](../README.md#calibrate-a-test-before-relying-on-it)
- [HAR import boundary](../README.md#quick-start)
- [Architecture and local data flow](architecture.md)

### Hurl

- [CLI purpose, options, cookies, and reports](https://hurl.dev/docs/manual.html)
- [Response captures](https://hurl.dev/docs/capturing-response.html)
- [cURL-to-Hurl conversion](https://hurl.dev/blog/2023/05/05/announcing-hurl-3.0.0.html)
- [CI/CD integration](https://hurl.dev/docs/tutorial/ci-cd-integration.html)
- [Installation and runtime](https://hurl.dev/docs/installation.html)

### Grafana k6

- [Write a JavaScript or TypeScript test](https://grafana.com/docs/k6/latest/get-started/write-your-first-test/)
- [Local, distributed, and cloud execution](https://grafana.com/docs/k6/latest/get-started/running-k6/)
- [Per-VU and local cookie jars](https://grafana.com/docs/k6/latest/using-k6/cookies/)
- [Correlation and dynamic data](https://grafana.com/docs/k6/latest/examples/correlation-and-dynamic-data/)
- [Checks and CI gating semantics](https://grafana.com/docs/k6/latest/using-k6/checks/)
- [HAR-to-k6 converter](https://grafana.com/docs/k6/latest/using-k6/test-authoring/create-tests-from-recordings/using-the-har-converter/)
- [Anonymous usage collection and opt-out](https://grafana.com/docs/k6/latest/set-up/usage-collection/)

### Postman and Newman

- [Postman scripts and API tests](https://learning.postman.com/docs/tests-and-scripts/tests-and-scripts/)
- [Install and run Newman](https://learning.postman.com/docs/reference/newman-cli/installing-running-newman/)
- [Current Newman collection-format note](https://learning.postman.com/latest-v-12/docs/reference/newman-cli/command-line-integration-with-newman)
- [HAR import behavior](https://blog.postman.com/postman-now-supports-importing-har-files/)
- [Local client, accounts, and workspaces](https://learning.postman.com/latest-v-12/docs/getting-started/basics/using-api-client)
- [Postman workspace sync behavior](https://learning.postman.com/latest-v-12/docs/getting-started/basics/postman-basics)
- [Official HAR sanitization guidance](https://learning.postman.com/help/faqs/troubleshooting/how-to-get-logs-from-the-postman-web-app)

### pytest

- [Getting started and Python test authoring](https://docs.pytest.org/en/stable/getting-started.html)
- [Fixtures for test context and lifecycle](https://docs.pytest.org/en/stable/explanation/fixtures.html)
- [JUnit XML output](https://docs.pytest.org/en/stable/how-to/output.html#creating-junitxml-format-files)
- [License](https://docs.pytest.org/en/stable/license.html)

### Scanner example

- [ZAP active scan purpose and limitations](https://www.zaproxy.org/docs/desktop/start/features/ascan/)
- [ZAP API scan inputs and exit codes](https://www.zaproxy.org/docs/docker/api-scan/)