# Frequently Asked Questions ## How do I deal with measurement precision issues? If a task runs faster than the resolution of the timer your runtime provides, individual samples can measure as zero duration, which skews the reported latency. After each run, Tinybench exposes `detectedResolution`: the smallest positive timer-measured sample occurring at least twice, or the smallest positive sample if none repeats. Samples are considered after any overhead correction; the result is `undefined` if none is positive. This is a heuristic, not a guaranteed timer-resolution bound. With at least 10 such samples, a `'warning'` event can indicate timer-dominated results; see [Timer Diagnostics](./README.md#timer-diagnostics). Timer resolution varies by runtime and platform. Tinybench lets you choose a timestamp provider via the `timestampProvider` option, which accepts a `TimestampProvider` object or the shorthands `hrtimeNow`, `performanceNow`, `bunNanoseconds` (when running on Bun), or `auto` (to let Tinybench pick the most precise available provider for the current runtime). See the README [Timestamp Providers](./README.md#timestamp-providers) section for the full list, an example, and custom-provider setup. To supply a plain millisecond clock without building a `TimestampProvider`, set the `now` option instead; it is converted to a provider internally and cannot be combined with `timestampProvider`. Beyond picking a higher-resolution provider, you can increase the benchmark `time` so more samples are collected (improving statistical confidence). Use the `'warning'` event and `detectedResolution` as heuristic indicators (see [Timer Diagnostics](./README.md#timer-diagnostics)), not proof of a timer limitation. The relative margin of error (`rme`) alone is insufficient: a saturated task can report either a very high or a very low `rme`. Manually looping a fast function to "amplify" its duration can lift a sample above the timer resolution, but it adds loop overhead and changes what you measure; a more precise timestamp provider is usually the cleaner fix. If you do batch several calls per task call, [Task-Supplied Measurements](./README.md#task-supplied-measurements) shows how to record a batch mean while declaring the whole iteration's budget cost. ## How do I read the reported statistics? Latency and throughput results answer different questions. Read each number as one of two kinds. **Describing the samples.** `mean`, `p50`, `p75`, `p99`, `p995`, `p999`, `min` and `max` locate the individual samples; `sd`, `variance`, `aad` and `mad` describe their spread. These are properties of the collected samples, not guarantees about future runs. Percentiles interpolate linearly between order statistics, and the upper ones are unstable on small samples: with 64 samples (the default `iterations` floor), `p99` and `p999` interpolate between the two largest observations, weighting the maximum by 37 % and 93.7 % respectively — what they say about the tail depends on the gap between those observations. Collect more samples before trusting them: `time` raises the sample count in the default modes, while with `concurrency: "task"` `iterations` caps the run count, so raise `iterations` as well. The mean and the median answer different questions and legitimately differ on skewed latency distributions. With [batched measurements](./README.md#task-supplied-measurements), each sample is a mean duration per call within one batch, so the statistics describe batch means, not individual calls. `mad` (median absolute deviation) is a robust dispersion figure. It is not scaled to a normal distribution and is not an uncertainty estimate: the `med ± mad` columns of `bench.table()` are descriptive, not confidence intervals. **Inferring about the mean.** `sem`, `moe` and `rme` concern the mean of the sampled process, not individual samples. With at least two samples, `moe` is the half-width of a two-sided 95% confidence interval computed with the exact Student's t critical value for `samplesCount - 1` degrees of freedom, and `rme` is `moe` expressed as a percentage of the mean. With a single sample, `df` is 0, the critical value falls back to one degree of freedom and `moe` is 0: no meaningful interval exists. The interval is nominal: it assumes samples behave like independent observations from a stable regime and that the sample mean is approximately normally distributed. Strong skew, deopts, GC spikes or warmup still in progress can make the interval optimistic, especially on small samples. The interval covers only the sampled process: run-to-run and process-to-process variation (JIT state, environment, machine state) is not included, so compare results across repeated runs rather than trusting a single `rme`. **Latency versus throughput.** Throughput statistics transform each nonzero sample independently (`1000 / sample`) before statistics are computed, so they describe instantaneous rates. That is why the throughput mean is not the reciprocal of the latency mean. With two samples of 1 ms and 9 ms, the average instantaneous rate is `(1000 / 1 + 1000 / 9) / 2 ≈ 555.6 ops/s`, while the work actually completes at `2 / 0.010 s = 200 ops/s`. In sequential modes, `totalTime` is the sum of the samples, so `1000 / latency.mean` (equivalently `1000 / period`) is the implied completion rate; read throughput dispersion and percentiles as variation of the instantaneous rates. With `concurrency: "task"`, calls overlap while samples are still summed per call: the real wall-clock completion rate can be far above `1000 / period`, so divide completed operations by elapsed wall-clock time instead. Samples that measure as zero — timer-saturated runs or overhead corrections clamping a sample — are not turned into an infinite rate: each is imputed as `1000 / latency.mean` (`0` when the mean is zero). Throughput figures then mix observed and imputed rates; the [measurement precision section](#how-do-i-deal-with-measurement-precision-issues) explains how to avoid timer-dominated samples. ## Why does a run take longer than the configured `time`? `time` is not a wall-clock timeout. In sequential modes, Tinybench accumulates per-iteration budget costs and keeps running until both `time` and the minimum `iterations` are satisfied. Warmup, hooks, loop bookkeeping and final statistics also contribute to the elapsed time. Very fast functions can produce large sample arrays. Sorting and analyzing these samples takes additional time after collection ends. Debugger instrumentation can increase both collection and analysis costs. `warmup: false` skips warmup, but does not remove these other costs. For synchronous functions and hooks, `runSync()` avoids asynchronous scheduling in the measurement loop; it still computes statistics. Reducing `time` can collect fewer samples and reduce analysis work, at the cost of statistical precision. See [Concurrency](./README.md#concurrency) for the different scheduling limits in task-concurrent mode. ## What is JS JIT de-optimization? JavaScript engines (V8, SpiderMonkey, JavaScriptCore) compile hot code paths just-in-time (JIT). JIT de-optimization (a "deopt") happens when the engine discards an optimized code path and falls back to a lower, unoptimized execution tier (a bytecode interpreter or a less-optimized JIT tier, depending on the engine). During benchmarking this can show up as sudden latency spikes. Common causes include type instability (argument types changing between iterations) and dynamic property access or function reassignment that prevents inlining. Garbage-collection pauses are a separate source of latency spikes (not a deopt) that can likewise inflate variance, and — in browsers — having DevTools open adds overhead and can disable some JIT optimizations rather than being a deopt itself. Tinybench reports statistical indicators (standard deviation, variance, percentiles, margin of error) that help you spot outliers caused by deopts. If you see high variance, try running the benchmark multiple times and comparing results, keeping warmup enabled (the default; tunable via `warmupIterations` / `warmupTime`), and (in browsers) closing DevTools and running in a production-like configuration. A fixed `warmupTime` or `warmupIterations` budget does not establish that execution has stabilized: warmup ends on the configured time and iteration rules, not on a stability check. Repeat measurements and compare runs rather than treating warmup completion as proof of a stable regime; see [Cold Starts and Warmed-Up Runs](./README.md#cold-starts-and-warmed-up-runs) for choosing which behavior to measure. ## Why do my results differ from another benchmarking tool? Differences between benchmarking tools are expected and do not necessarily indicate a bug. Common reasons: - **Different timing APIs.** Tools may use `Date.now()`, `performance.now()`, or `process.hrtime.bigint()`, each with different resolution and overhead. - **Different measurement methodology.** Some tools measure a single iteration; Tinybench collects many samples over a configurable `time` window and reports statistically analyzed latency and throughput (mean, median, p75/p99, standard deviation, margin of error). - **Different statistics.** Tools aggregate differently (arithmetic mean, geometric mean, median, …). Tinybench reports both average and median (p50). - **Warmup handling.** Tools differ in whether they warm up before measuring. Tinybench warms up by default (`warmup`, on by default) in a separate phase excluded from the reported statistics; disabling it (`warmup: false`) lets the first measured samples include JIT warmup cost. - **Overhead.** Every tool introduces its own measurement overhead (loop, function call, event dispatch). Tinybench is small, but not zero-overhead. When comparing across tools, focus on the **relative** difference (is implementation A faster than B?) rather than the absolute operations-per-second numbers, and run the comparison on the same machine in the same environment.