# fdars Performance Report A single-threaded head-to-head of `fdars-core` against scikit-fda 0.10.1 and the R FDA ecosystem (`fda`, `fda.usc`), before and after the v0.47.0 optimization pass. Every number below is traceable to a committed CSV in `benchmarks/results/` and the environment snapshot in `benchmarks/results/env.json` — see [Methodology](#methodology) and [Reproduction](#reproduction) for the exact commands that produced it. ## How to read this report - **†** (dagger) — the fdars measurement failed the noise gate (RSD >= 5% or CI-width >= 5%). A daggered cell is never headlined as a confirmed win or loss, even when the raw ratio looks favorable. - **Significant?** — "yes" means the two 95% confidence intervals are strictly non-overlapping (touching CIs count as "no"). Only a "yes" backs a speedup or regression claim; everything else is reported as a directional ratio with the caveat spelled out in prose. - **Ratio semantics** — every `fdars vs X` ratio is `X time / fdars time`. A ratio at least 10% above 1.0 is "N× faster" (fdars wins); a ratio at least 10% below 1.0 is "N× slower" (fdars loses); anything within that +/-10% parity band is reported as "comparable", not as a win either way. - **Single-threaded throughout** — every table in this report, and every reproduction command below, pins `OMP_NUM_THREADS`, `OPENBLAS_NUM_THREADS`, `MKL_NUM_THREADS`, `RAYON_NUM_THREADS`, `NUMEXPR_NUM_THREADS`, and `VECLIB_MAXIMUM_THREADS` to `1` on every side (fdars, scikit-fda, R). No table in this document reports a multi-threaded number; the `parallel` feature and rayon are discussed only qualitatively (see [Threading](#threading)). ## Summary _Generated by `benchmarks/render_report.py` from comparison.csv, comparison_baseline.csv, ab/manifest.json, rerun_log.json — do not edit by hand._ | Finding | Value | Evidence | |---|---|---| | Headline A/B (phase116_face_covariance_n200_m30) | 1.19× faster | perf_face_covariance/n200_m30 | | Elastic-alignment A/B (phase114_elastic_cross_banded_n500_m50) | -2.0% (not significant (CIs overlap)) | before noise: FAIL-NOISY | | SVD re-bench | see phase115_svd_rebench | faer thin SVD re-benchmark (near-optimal verdict) | | Largest-cell verdict vs scikit-fda | 6 faster / 0 comparable / 0 slower | headline cell N=500, M=200, noise-passing only | | Largest-cell verdict vs R | 6 faster / 0 comparable / 0 slower | headline cell N=500, M=200, noise-passing only | | Re-run coverage (missing rows) | id-not-produced: 9, inferred-timeout: 3, orphan: 20, timeout: 4 | classify_missing over baseline/after diff | - **face_covariance (the one committed A/B win): headlined at 1.19× faster, pinned and noise-clean.** The controlled, single-threaded, thread-pinned re-measurement (`phase116_face_covariance_n200_m30`, this phase) is the only A/B entry that is `headline_eligible`: both sides pass the noise gate and the CIs are strictly disjoint. The larger `3.45×` figure from Phase 115 is real evidence but was measured on the default (unpinned, multi-threaded) rayon pool with a BEFORE cell that failed the noise gate — it is shown in the table above for provenance but is not reproduced under single-threaded, noise-gated conditions and is never claimed as this report's headline. - **Elastic alignment: numeric-identical, within noise, not a speedup.** The Phase 114 `dp_fill_grid` banded-iteration refactor is proven bit-identical by the golden-fixture suite; its A/B entry's CIs overlap and its BEFORE cell failed the noise gate. This report does not claim elastic alignment got faster — the hot path was already near-optimal going in (symmetry exploitation, one-time SRSF precompute, thread-local scratch reuse). - **SVD: already near-optimal, no change made.** faer's thin SVD is already the production path under the `linalg` feature (`regression.rs`); Phase 115 re-benchmarked it at tighter sample counts and found nothing worth changing. There is no before/after pair for SVD in this report — only the re-benchmark entry in the A/B table. - **Head-to-head at the largest grid cell (N=500, M=200): fdars leads on every op that passed the noise gate**, per the "Largest-cell verdict" rows in the table above — every measured op that passed the gate favored fdars against both scikit-fda and R, with none comparable or slower. - **The Phase 114–115 optimizations are not on any harness op's code path.** The harness `elastic` op times the unbanded pairwise alignment (`elastic_align_pair`), while Phase 114 changed only the banded DP arm (114-02-SUMMARY); `face_covariance` has no scikit-fda/R comparator at all. The cross-language cells in this report were therefore never expected to move because of Phase 114/115 — the "before/after vs the Phase 113 baseline" tables below measure re-run drift and methodology changes (thread pinning, mixed-vintage baseline), not an optimization effect. - **Re-run coverage:** 7 heavy single-threaded elastic/Karcher chunks did not complete inside the 540s per-chunk bound and are recorded as timed out or inferred-timed-out (never silently dropped); a further set of legacy audit groups are orphaned (removed from the bench files) or did not reproduce a specific criterion id. Every one of them is listed by name, with a reason, in [Before/after vs the Phase 113 baseline](#beforeafter-vs-the-phase-113-baseline). ## Headline — largest grid cell (N=500, M=200) _Generated by `benchmarks/render_report.py` from comparison.csv — do not edit by hand._ | Op | N | M | fdars (ms) | scikit-fda (ms) | R (ms) | fdars vs scikit-fda | fdars vs R | |----|---|---|------------|------------------|--------|----------------------|------------| | fpca | 500 | 200 | 10.152 | 12.357 | 41.017 | 1.22× faster | 4.04× faster | | basis_smoothing | 500 | 200 | 0.834 | 4.199 | 4.324 | 5.04× faster | 5.19× faster | | depth | 500 | 200 | 3.511 | 15.591 | 8.855 | 4.44× faster | 2.52× faster | | functional_boxplot | 500 | 200 | 3.617 | 16.616 | 21.627 | 4.59× faster | 5.98× faster | | fpcr | 500 | 200 | 10.241 | 12.636 | 1049.904 | 1.23× faster | 102.52× faster | | elastic | 500 | 200 | 27.470 | 33.542 | 32.517 | 1.22× faster | 1.18× faster | † fdars measurement failed the noise gate (RSD >= 5% or CI-width >= 5%) — not headlined as significant. N=500, M=200 is the largest cell on the locked grid (N in {100, 300, 500}, M in {50, 200}) — the point where asymptotic behavior is most visible and where a real regression or win is least likely to be an artifact of constant-factor noise. ## Head-to-head across the N×M grid _Generated by `benchmarks/render_report.py` from comparison.csv — do not edit by hand._ #### fpca | N | M | fdars (ms) [95% CI] | scikit-fda (ms) | R SVD-matched (ms) | R native pca.fd (ms)¹ | fdars vs scikit-fda | fdars vs R | |---|---|---|---|---|---|---|---| | 100 | 50 | 0.424 [0.423, 0.426] | 1.554 | 1.006 | 7.019 | 3.66× faster | 2.37× faster | | 100 | 200 | 1.985 [1.981, 1.992] | 3.755 | 5.189 | 7.460 | 1.89× faster | 2.61× faster | | 300 | 50 | 0.683 [0.679, 0.688] | 1.501 | 1.915 | 9.172 | 2.20× faster | 2.80× faster | | 300 | 200 | 7.209 [7.194, 7.228] | 8.759 | 25.561 | 9.048 | 1.22× faster | 3.55× faster | | 500 | 50 | 0.913 [0.908, 0.921] | 1.270 | 2.873 | 10.860 | 1.39× faster | 3.15× faster | | 500 | 200 | 10.152 [10.128, 10.183] | 12.357 | 41.017 | 9.449 | 1.22× faster | 4.04× faster | ¹ R native `fda::pca.fd` — ecosystem-realistic timing, not gated (B-spline coefficient space). - FPCA fdars-vs-python subspace cos=0.9816 (hard-gated on fdars-vs-r=1.0; python DISCLOSED — scikit-fda trapezoidal vs fdars/R Simpson's quadrature tilts trailing PCs; Phase 116 finding) - FPCA fdars-vs-python subspace cos=0.9986 (hard-gated on fdars-vs-r=1.0; python DISCLOSED — scikit-fda trapezoidal vs fdars/R Simpson's quadrature tilts trailing PCs; Phase 116 finding) - FPCA fdars-vs-python subspace cos=0.9992 (hard-gated on fdars-vs-r=1.0; python DISCLOSED — scikit-fda trapezoidal vs fdars/R Simpson's quadrature tilts trailing PCs; Phase 116 finding) - FPCA fdars-vs-python subspace cos=0.9994 (hard-gated on fdars-vs-r=1.0; python DISCLOSED — scikit-fda trapezoidal vs fdars/R Simpson's quadrature tilts trailing PCs; Phase 116 finding) - FPCA fdars-vs-python subspace cos=0.9998 (hard-gated on fdars-vs-r=1.0; python DISCLOSED — scikit-fda trapezoidal vs fdars/R Simpson's quadrature tilts trailing PCs; Phase 116 finding) #### basis_smoothing | N | M | fdars (ms) [95% CI] | scikit-fda (ms) | R SVD-matched (ms) | fdars vs scikit-fda | fdars vs R | |---|---|---|---|---|---|---| | 100 | 50 | 0.053 [0.052, 0.055]† | 0.370 | 0.796 | 6.95× faster | 14.95× faster | | 100 | 200 | 0.211 [0.203, 0.223]† | 1.461 | 1.707 | 6.92× faster | 8.09× faster | | 300 | 50 | 0.137 [0.136, 0.137] | 0.453 | 1.643 | 3.32× faster | 12.03× faster | | 300 | 200 | 0.514 [0.513, 0.517] | 2.829 | 3.132 | 5.50× faster | 6.09× faster | | 500 | 50 | 0.216 [0.216, 0.217] | 0.525 | 2.507 | 2.43× faster | 11.60× faster | | 500 | 200 | 0.834 [0.828, 0.842] | 4.199 | 4.324 | 5.04× faster | 5.19× faster | #### depth | N | M | fdars (ms) [95% CI] | scikit-fda (ms) | R SVD-matched (ms) | fdars vs scikit-fda | fdars vs R | |---|---|---|---|---|---|---| | 100 | 50 | 0.105 [0.104, 0.106] | 1.329 | 0.829 | 12.61× faster | 7.87× faster | | 100 | 200 | 0.546 [0.518, 0.578]† | 3.720 | 2.748 | 6.81× faster | 5.03× faster | | 300 | 50 | 0.506 [0.461, 0.561]† | 2.660 | 1.762 | 5.25× faster | 3.48× faster | | 300 | 200 | 2.068 [2.035, 2.104] | 9.692 | 6.955 | 4.69× faster | 3.36× faster | | 500 | 50 | 0.757 [0.748, 0.767] | 4.074 | 2.787 | 5.38× faster | 3.68× faster | | 500 | 200 | 3.511 [3.454, 3.570] | 15.591 | 8.855 | 4.44× faster | 2.52× faster | #### functional_boxplot | N | M | fdars (ms) [95% CI] | scikit-fda (ms) | R SVD-matched (ms) | fdars vs scikit-fda | fdars vs R | |---|---|---|---|---|---|---| | 100 | 50 | 0.124 [0.119, 0.129]† | 1.794 | 2.070 | 14.48× faster | 16.70× faster | | 100 | 200 | 0.617 [0.589, 0.653]† | 4.170 | 6.683 | 6.76× faster | 10.83× faster | | 300 | 50 | 0.509 [0.498, 0.522]† | 3.183 | 4.233 | 6.25× faster | 8.32× faster | | 300 | 200 | 2.158 [2.095, 2.225]† | 10.064 | 14.303 | 4.66× faster | 6.63× faster | | 500 | 50 | 0.926 [0.909, 0.943]† | 4.712 | 7.058 | 5.09× faster | 7.62× faster | | 500 | 200 | 3.617 [3.570, 3.671] | 16.616 | 21.627 | 4.59× faster | 5.98× faster | #### fpcr | N | M | fdars (ms) [95% CI] | scikit-fda (ms) | R SVD-matched (ms) | fdars vs scikit-fda | fdars vs R | |---|---|---|---|---|---|---| | 100 | 50 | 0.430 [0.428, 0.432] | 1.308 | 36.030 | 3.04× faster | 83.80× faster | | 100 | 200 | 2.007 [2.000, 2.016] | 3.721 | 85.633 | 1.85× faster | 42.66× faster | | 300 | 50 | 0.699 [0.697, 0.701] | 1.631 | 96.773 | 2.33× faster | 138.51× faster | | 300 | 200 | 7.452 [7.374, 7.541] | 9.246 | 619.725 | 1.24× faster | 83.16× faster | | 500 | 50 | 0.940 [0.937, 0.943] | 1.420 | 153.821 | 1.51× faster | 163.72× faster | | 500 | 200 | 10.241 [10.154, 10.344] | 12.636 | 1049.904 | 1.23× faster | 102.52× faster | - FPCR cross-library agreement corr=0.7944 @ n=500 (scikit-fda FPCA->OLS vs R fda.usc fregre.pc: related but not identical FPCR; NOT hard-gated — disclosed divergence for the Phase 116 report) - FPCR cross-library agreement corr=0.8061 @ n=300 (scikit-fda FPCA->OLS vs R fda.usc fregre.pc: related but not identical FPCR; NOT hard-gated — disclosed divergence for the Phase 116 report) - FPCR cross-library agreement corr=0.8585 @ n=500 (scikit-fda FPCA->OLS vs R fda.usc fregre.pc: related but not identical FPCR; NOT hard-gated — disclosed divergence for the Phase 116 report) - FPCR cross-library agreement corr=0.9236 @ n=300 (scikit-fda FPCA->OLS vs R fda.usc fregre.pc: related but not identical FPCR; NOT hard-gated — disclosed divergence for the Phase 116 report) - FPCR cross-library agreement corr=0.9495 @ n=100 (scikit-fda FPCA->OLS vs R fda.usc fregre.pc: related but not identical FPCR; NOT hard-gated — disclosed divergence for the Phase 116 report) - FPCR cross-library agreement corr=0.9797 @ n=100 (scikit-fda FPCA->OLS vs R fda.usc fregre.pc: related but not identical FPCR; NOT hard-gated — disclosed divergence for the Phase 116 report) #### elastic | N | M | fdars (ms) [95% CI] | scikit-fda (ms) | R SVD-matched (ms) | fdars vs scikit-fda | fdars vs R | |---|---|---|---|---|---|---| | 100 | 50 | 1.839 [1.679, 2.008]† | 3.106 | 2.784 | 1.69× faster | 1.51× faster | | 100 | 200 | 27.701 [27.401, 28.031] | 33.616 | 32.529 | 1.21× faster | 1.17× faster | | 300 | 50 | 1.593 [1.516, 1.686]† | 2.877 | 1.921 | 1.81× faster | 1.21× faster | | 300 | 200 | 28.597 [27.936, 29.292]† | 33.571 | 32.815 | 1.17× faster | 1.15× faster | | 500 | 50 | 1.499 [1.475, 1.526] | 2.882 | 1.918 | 1.92× faster | 1.28× faster | | 500 | 200 | 27.470 [27.237, 27.714] | 33.542 | 32.517 | 1.22× faster | 1.18× faster | - Karcher-mean elastic excluded from grid baseline: noise-gated (RSD 29-46%); anchor uses pairwise elastic_align_pair - **FPCA has two R comparators.** `r_svd` is a hand-rolled, Simpson-weighted SVD port of fdars' own algorithm — it is the ecosystem-independent, hard-gated comparator (fdars-vs-R subspace cosine = 1.0 on every cell). `r_native_pca_fd` is R's own `fda::pca.fd`, which works in B-spline coefficient space rather than raw evaluation-point space; it is timed for ecosystem realism but is not gated (it was found in Phase 113 to diverge numerically from fdars at PC3–PC5 because of the coefficient-space smoothing, not because either implementation is wrong). - **FPCR is a disclosed, not a gated, cross-library comparison.** scikit-fda's FPCR (FPCA scores → OLS) and R's `fda.usc::fregre.pc` are related but not identical algorithms; their correlation is reported per-cell in the table above for transparency, never used to pass/fail a run. - **The FPCA quadrature convention differs by ecosystem.** scikit-fda integrates with the trapezoid rule; fdars and the R `r_svd` port both use Simpson's rule. This is disclosed (not hard-gated) for the fdars-vs-python FPCA comparison and is the reason that comparison's subspace cosine is below 1.0 while fdars-vs-R stays at 1.0 on every cell. - **The `elastic` op is pairwise and N-independent by construction.** The harness always times a fixed 2-curve alignment regardless of the grid's N, so its row varies with M only — this is a property of what is being measured, not a claim about how `elastic_align_pair` scales with corpus size (see [N×M scaling](#nm-scaling) for the M-only exponent). ## Optimization results (controlled A/B) _Generated by `benchmarks/render_report.py` from ab/manifest.json — do not edit by hand._ | Entry | Kind | Before (ms) | After (ms) | Speedup | Significant? | Noise before/after | Thread config | SHAs | Headline? | |---|---|---|---|---|---|---|---|---|---| | phase114_elastic_cross_banded_n500_m50 | ab | 7310.074 | 7163.866 | 1.02× | no | FAIL-NOISY/PASS | default rayon pool (not thread-pinned); identical configuration on both sides | 8c0b1d7d -> 30d2c9a2 | | | phase115_face_covariance_n200_m30 | ab | 268.166 | 77.786 | 3.45× | yes | FAIL-NOISY/PASS | default rayon pool (not thread-pinned); identical configuration on both sides | 328215e7 -> c0e84bd8 | | | phase115_svd_rebench | rebench | — | — | — | — | — | default rayon pool (not thread-pinned); identical configuration across cells | 74b9ab0d | | | phase116_face_covariance_n200_m30 | ab | 300.891 | 252.512 | 1.19× | yes | PASS/PASS | pinned: all six *_NUM_THREADS=1 (identical both sides) | 7cde79b24ea259318f358e4d57957bb2435f8995 -> e98b862dd3b39a2843fe1f33889e876aebc46921 | yes | Every A/B entry above controls for confounds the same way: identical bench invocation, identical criterion settings, and a `Cargo.lock` copied into both scratch worktrees so dependency versions cannot drift between the before and after measurement — the two sides differ only in the commit checked out. The Phase 114 and Phase 115 entries were measured on the default (unpinned) rayon thread pool, disclosed verbatim in each entry's `thread_config` column; they are legitimate evidence for those phases' own verdicts but are not apples-to-apples with a single-threaded head-to-head. Phase 116 re-measured `face_covariance` a third time, this time thread-pinned and at raised settings (`sample_size` 15, 30s measurement, 3s warm-up — round 0), because Phase 115's BEFORE cell for that same criterion id failed the noise gate; a single round sufficed and no further rounds were needed. ## Before/after vs the Phase 113 baseline Read every table in this section as a **traceability record of what changed between Phase 113 and Phase 116**, not as an optimization measurement (the A/B tables above are the optimization measurement). Four things changed between the two runs simultaneously, and this report does not attempt to separate their individual contributions to any one row's delta: - Phase 113 re-ran only `fregre_lm` and `functional_boxplot` live on the fdars side; every other fdars row in the Phase 113 baseline came from whatever criterion output already existed on disk at the time — of mixed vintage, spanning several earlier optimization phases. - Those pre-existing baseline rows were measured under the default, multi-core rayon pool (no thread pinning). Several of the large deltas below on `elastic`/`karcher`-family groups are exactly this: losing rayon's parallelism between the baseline and a single-threaded re-run, not a regression in the underlying algorithm. - Six benchmark groups present in the Phase 113 baseline no longer exist in the current bench files (`coclustering_co_cluster`, `face_covariance`, `fem_smooth`, `fpca_variants_fsvd`, `frechet_anova`, `fts_long_run_covariance`) — they are flagged `orphan` in the table below, not silently dropped. - The Phase 116 after-run is thread-pinned end-to-end (all six `*_NUM_THREADS=1`), matching the cross-language harness's fairness contract. ### fdars rows for ops with a scikit-fda/R comparator _Generated by `benchmarks/render_report.py` from comparison_baseline.csv, comparison.csv — do not edit by hand._ before = Phase 113 comparison_baseline.csv (fdars rows collected from on-disk criterion output, not thread-pinned, mixed vintage); after = Phase 116 thread-pinned re-run (rerun_log head_sha 242ab06722cfb68de28dd57203d3eb4757062a6f); delta is not an optimization measure — see A/B evidence. **fdars-only before/after for ops that ALSO have a scikit-fda/R comparator elsewhere in this report (see “fdars vs Python/R” above) — rows below compare the SAME rust criterion id across two runs; they are not themselves validated against Python/R at this cell. Several rows (the audit_p3_elastic_*/audit_karcher/karcher_mean self- and cross-distance-matrix and payback sweeps) measure a different sub-operation than the grid-aligned comparator bench and have no direct Python/R equivalent.** | Source | N | M | Before ms [CI] | After ms [CI] | delta % | Significant? | Noise before/after | |---|---|---|---|---|---|---|---| | audit_depth/n500_m200 | 500.0 | 200.0 | 0.469 [0.460, 0.478] | 3.365 [3.102, 3.637] | +617.7% | yes | FAIL-NOISY/FAIL-NOISY | | audit_elastic/n100_m50_capped | 100.0 | 50.0 | 816.797 [809.964, 824.040] | 7139.152 [7108.057, 7173.160] | +774.0% | yes | PASS/PASS | | audit_fpca/n500_m200 | 500.0 | 200.0 | 16.391 [16.325, 16.465] | 4.939 [4.875, 5.003] | -69.9% | yes | PASS/PASS | | audit_karcher/n100_m50 | 100.0 | 50.0 | 162.283 [159.698, 165.016] | 1583.859 [1577.355, 1593.096] | +876.0% | yes | PASS/PASS | | audit_p3_elastic_cross/n100_m50 | 100.0 | 50.0 | 1565.454 [1540.027, 1592.022] | 14355.925 [14322.811, 14393.542] | +817.0% | yes | PASS/PASS | | audit_p3_elastic_cross_banded/n100_m50 | 100.0 | 50.0 | 324.137 [320.564, 328.018] | 3251.989 [3215.115, 3288.837] | +903.3% | yes | PASS/PASS | | audit_p3_elastic_self/n100_m50 | 100.0 | 50.0 | 1479.460 [1308.937, 1666.878] | 7104.962 [7090.892, 7120.024] | +380.2% | yes | FAIL-NOISY/PASS | | audit_p3_elastic_self_banded/n100_m200 | 100.0 | 200.0 | 3423.428 [3297.206, 3565.520] | 28353.333 [28272.721, 28448.654] | +728.2% | yes | FAIL-NOISY/PASS | | audit_p3_elastic_self_banded/n100_m50 | 100.0 | 50.0 | 406.670 [311.848, 507.154] | 1469.353 [1460.451, 1480.550] | +261.3% | yes | FAIL-NOISY/PASS | | audit_p3_elastic_self_banded/n500_m50 | 500.0 | 50.0 | 4223.193 [4051.851, 4416.070] | 36890.583 [36856.435, 36919.544] | +773.5% | yes | FAIL-NOISY/PASS | | audit_p3_karcher/n100_m200 | 100.0 | 200.0 | 6483.449 [5407.426, 7711.390] | 30478.628 [30368.935, 30626.327] | +370.1% | yes | FAIL-NOISY/PASS | | audit_p3_karcher/n100_m50 | 100.0 | 50.0 | 644.766 [541.582, 757.205] | 3055.021 [3019.727, 3108.543] | +373.8% | yes | FAIL-NOISY/PASS | | audit_p3_karcher/n500_m50 | 500.0 | 50.0 | 3979.197 [3210.227, 4769.585] | 15076.405 [15023.533, 15138.689] | +278.9% | yes | FAIL-NOISY/PASS | | audit_p3_karcher_banded/n100_m200 | 100.0 | 200.0 | 754.891 [740.368, 775.895] | 6704.583 [6677.103, 6738.090] | +788.2% | yes | PASS/PASS | | audit_p3_karcher_banded/n100_m50 | 100.0 | 50.0 | 75.449 [67.359, 88.607] | 626.955 [622.427, 631.788] | +731.0% | yes | FAIL-NOISY/PASS | | audit_p3_karcher_banded/n500_m200 | 500.0 | 200.0 | 4872.579 [4204.307, 5831.564] | 33445.998 [33389.439, 33513.317] | +586.4% | yes | FAIL-NOISY/PASS | | audit_p3_karcher_banded/n500_m50 | 500.0 | 50.0 | 421.493 [371.738, 489.380] | 3129.779 [3116.317, 3144.433] | +642.5% | yes | FAIL-NOISY/PASS | | audit_p4_elastic_fpca/joint_fpca_n100_m50 | 100.0 | 50.0 | 1.885 [1.884, 1.885] | 1.966 [1.940, 1.993] | +4.3% | yes | PASS/PASS | | audit_p4_elastic_fpca/vert_fpca_n100_m50 | 100.0 | 50.0 | 0.301 [0.301, 0.301] | 0.352 [0.339, 0.366] | +17.0% | yes | PASS/FAIL-NOISY | | audit_p4_fpca/n1000_m200 | 1000.0 | 200.0 | 38.901 [38.858, 38.942] | 9.660 [9.177, 10.343] | -75.2% | yes | PASS/FAIL-NOISY | | audit_p4_fpca/n1000_m50 | 1000.0 | 50.0 | 3.182 [3.180, 3.183] | 0.826 [0.710, 0.978] | -74.0% | yes | PASS/FAIL-NOISY | | audit_p4_fpca/n100_m200 | 100.0 | 200.0 | 1.709 [1.707, 1.714] | 0.709 [0.708, 0.710] | -58.5% | yes | PASS/PASS | | audit_p4_fpca/n100_m50 | 100.0 | 50.0 | 0.213 [0.213, 0.213] | 0.198 [0.174, 0.236] | -6.9% | no | PASS/FAIL-NOISY | | audit_p4_fpca/n500_m200 | 500.0 | 200.0 | 15.978 [15.973, 15.985] | 5.833 [5.553, 6.148] | -63.5% | yes | PASS/FAIL-NOISY | | audit_p4_fpca/n500_m50 | 500.0 | 50.0 | 1.233 [1.232, 1.235] | 0.368 [0.360, 0.377] | -70.2% | yes | PASS/FAIL-NOISY | | audit_p5_karcher_paybackN/n100_m50 | 100.0 | 50.0 | 343.770 [341.044, 346.629] | 1591.791 [1574.794, 1618.384] | +363.0% | yes | PASS/PASS | | audit_p5_karcher_paybackN/n10_m50 | 10.0 | 50.0 | 55.668 [51.836, 61.519] | 160.653 [159.679, 162.044] | +188.6% | yes | FAIL-NOISY/PASS | | audit_p5_karcher_paybackN/n25_m50 | 25.0 | 50.0 | 124.956 [118.686, 133.253] | 404.555 [395.937, 414.756] | +223.8% | yes | FAIL-NOISY/PASS | | audit_p5_karcher_paybackN/n50_m50 | 50.0 | 50.0 | 196.100 [193.017, 200.833] | 785.839 [784.227, 788.138] | +300.7% | yes | PASS/PASS | | audit_streaming_depth/n500_m200 | 500.0 | 200.0 | 0.526 [0.503, 0.552] | 2.537 [2.480, 2.600] | +382.6% | yes | FAIL-NOISY/FAIL-NOISY | | elastic_align_pair/m/100 | — | — | 12.377 [12.091, 12.681] | 7.364 [7.134, 7.619] | -40.5% | yes | FAIL-NOISY/FAIL-NOISY | | elastic_align_pair/m/200 | — | — | 51.742 [51.574, 51.910] | 40.924 [39.234, 42.592] | -20.9% | yes | PASS/FAIL-NOISY | | elastic_align_pair/m/50 | — | — | 2.802 [2.780, 2.821] | 1.708 [1.655, 1.766] | -39.0% | yes | PASS/FAIL-NOISY | | elastic_align_pair/m100_lambda0.1 | — | 100.0 | 13.008 [12.913, 13.111] | 7.427 [7.171, 7.707] | -42.9% | yes | PASS/FAIL-NOISY | | elastic_align_pair/m_band0.1/100 | — | — | 2.656 [2.649, 2.663] | 1.616 [1.562, 1.673] | -39.1% | yes | PASS/FAIL-NOISY | | elastic_align_pair/m_band0.1/200 | — | — | 11.252 [11.158, 11.316] | 6.853 [6.641, 7.082] | -39.1% | yes | PASS/FAIL-NOISY | | elastic_align_pair/m_band0.1/50 | — | — | 0.573 [0.572, 0.574] | 0.475 [0.456, 0.494] | -17.1% | yes | PASS/FAIL-NOISY | | fregre_lm/ncomp/10 | — | — | 0.280 [0.262, 0.301] | 0.243 [0.222, 0.266] | -13.4% | no | FAIL-NOISY/FAIL-NOISY | | fregre_lm/ncomp/3 | — | — | 0.630 [0.627, 0.633] | 0.213 [0.202, 0.228] | -66.1% | yes | PASS/FAIL-NOISY | | fregre_lm/ncomp/5 | — | — | 0.626 [0.621, 0.632] | 0.264 [0.240, 0.290] | -57.9% | yes | PASS/FAIL-NOISY | | fregre_lm/scale/n200_nc5 | 200.0 | — | 0.504 [0.445, 0.566] | 0.300 [0.284, 0.319] | -40.5% | yes | FAIL-NOISY/FAIL-NOISY | | fregre_lm/scale/n50_nc5 | 50.0 | — | 0.369 [0.368, 0.371] | 0.210 [0.196, 0.224] | -43.2% | yes | PASS/FAIL-NOISY | | functional_boxplot/N/1000 | — | — | 12.713 [11.887, 13.539] | 10.560 [9.976, 11.192] | -16.9% | yes | FAIL-NOISY/FAIL-NOISY | | functional_boxplot/N/200 | — | — | 1.621 [1.526, 1.718] | 1.043 [0.999, 1.094] | -35.6% | yes | FAIL-NOISY/FAIL-NOISY | | functional_boxplot/N/2300 | — | — | 44.705 [42.571, 46.710] | 26.741 [25.368, 28.236] | -40.2% | yes | FAIL-NOISY/FAIL-NOISY | | functional_boxplot/N/50 | — | — | 0.316 [0.299, 0.334] | 0.235 [0.223, 0.247] | -25.6% | yes | FAIL-NOISY/FAIL-NOISY | | functional_boxplot/N/500 | — | — | 6.048 [5.676, 6.408] | 3.901 [3.643, 4.181] | -35.5% | yes | FAIL-NOISY/FAIL-NOISY | | karcher_mean/n10_m50 | 10.0 | 50.0 | 210.487 [203.862, 216.958] | 247.301 [242.701, 252.367] | +17.5% | yes | FAIL-NOISY/FAIL-NOISY | | karcher_mean/n20_m50 | 20.0 | 50.0 | 317.509 [299.522, 335.291] | 475.092 [471.541, 478.855] | +49.6% | yes | FAIL-NOISY/PASS | ### fdars internal before/after (no scikit-fda/R comparator) _Generated by `benchmarks/render_report.py` from comparison_baseline.csv, comparison.csv — do not edit by hand._ before = Phase 113 comparison_baseline.csv (fdars rows collected from on-disk criterion output, not thread-pinned, mixed vintage); after = Phase 116 thread-pinned re-run (rerun_log head_sha 242ab06722cfb68de28dd57203d3eb4757062a6f); delta is not an optimization measure — see A/B evidence. **fdars-internal before/after (no scikit-fda/R comparator)** | Source | N | M | Before ms [CI] | After ms [CI] | delta % | Significant? | Noise before/after | |---|---|---|---|---|---|---|---| | audit_cv/n100_m50_capped | 100.0 | 50.0 | 0.951 [0.949, 0.953] | 0.901 [0.885, 0.918] | -5.3% | yes | PASS/PASS | | audit_p5_streaming_paybackN/n10_m200 | 10.0 | 200.0 | 0.038 [0.038, 0.038] | 0.033 [0.032, 0.034] | -12.5% | yes | PASS/FAIL-NOISY | | audit_p5_streaming_paybackN/n1_m200 | 1.0 | 200.0 | 0.021 [0.021, 0.021] | 0.017 [0.017, 0.018] | -16.6% | yes | PASS/FAIL-NOISY | | audit_p5_streaming_paybackN/n200_m200 | 200.0 | 200.0 | 0.207 [0.207, 0.208] | 0.834 [0.821, 0.851] | +302.7% | yes | PASS/FAIL-NOISY | | audit_p5_streaming_paybackN/n500_m200 | 500.0 | 200.0 | 0.554 [0.553, 0.555] | 2.434 [2.400, 2.471] | +339.6% | yes | PASS/PASS | | audit_p5_streaming_paybackN/n50_m200 | 50.0 | 200.0 | 0.068 [0.068, 0.068] | 0.154 [0.150, 0.159] | +126.4% | yes | PASS/FAIL-NOISY | | audit_p6_svd_conversion/n1000_m200 | 1000.0 | 200.0 | 0.000 [0.000, 0.000] | 0.000 [0.000, 0.000] | -55.0% | yes | FAIL-NOISY/FAIL-NOISY | | audit_p6_svd_conversion/n1000_m50 | 1000.0 | 50.0 | 0.000 [0.000, 0.000] | 0.000 [0.000, 0.000] | -48.6% | yes | FAIL-NOISY/PASS | | audit_p6_svd_conversion/n100_m200 | 100.0 | 200.0 | 0.000 [0.000, 0.000] | 0.000 [0.000, 0.000] | -66.6% | yes | FAIL-NOISY/PASS | | audit_p6_svd_conversion/n100_m50 | 100.0 | 50.0 | 0.000 [0.000, 0.000] | 0.000 [0.000, 0.000] | -59.3% | yes | FAIL-NOISY/PASS | | audit_p6_svd_conversion/n500_m200 | 500.0 | 200.0 | 0.000 [0.000, 0.000] | 0.000 [0.000, 0.000] | -62.6% | yes | FAIL-NOISY/FAIL-NOISY | | audit_p6_svd_conversion/n500_m50 | 500.0 | 50.0 | 0.000 [0.000, 0.000] | 0.000 [0.000, 0.000] | -51.4% | yes | FAIL-NOISY/PASS | | audit_p6_svd_conversion/n500_m500 | 500.0 | 500.0 | 0.000 [0.000, 0.000] | 0.000 [0.000, 0.000] | -54.9% | yes | FAIL-NOISY/PASS | | audit_p6_svd_faer_seq/n1000_m200 | 1000.0 | 200.0 | 24.923 [19.334, 30.466] | 9.547 [8.579, 10.525] | -61.7% | yes | FAIL-NOISY/FAIL-NOISY | | audit_p6_svd_faer_seq/n1000_m50 | 1000.0 | 50.0 | 1.380 [1.078, 1.837] | 0.735 [0.637, 0.868] | -46.7% | yes | FAIL-NOISY/FAIL-NOISY | | audit_p6_svd_faer_seq/n100_m200 | 100.0 | 200.0 | 2.124 [1.799, 2.477] | 1.040 [0.946, 1.148] | -51.0% | yes | FAIL-NOISY/FAIL-NOISY | | audit_p6_svd_faer_seq/n100_m50 | 100.0 | 50.0 | 0.500 [0.431, 0.575] | 0.209 [0.196, 0.223] | -58.2% | yes | FAIL-NOISY/FAIL-NOISY | | audit_p6_svd_faer_seq/n500_m200 | 500.0 | 200.0 | 17.498 [15.023, 19.987] | 6.761 [6.282, 7.251] | -61.4% | yes | FAIL-NOISY/FAIL-NOISY | | audit_p6_svd_faer_seq/n500_m50 | 500.0 | 50.0 | 1.157 [0.858, 1.503] | 0.387 [0.355, 0.428] | -66.6% | yes | FAIL-NOISY/FAIL-NOISY | | audit_p6_svd_faer_seq/n500_m500 | 500.0 | 500.0 | 108.362 [101.742, 118.468] | 47.119 [46.454, 47.794] | -56.5% | yes | FAIL-NOISY/PASS | | audit_p6_svd_nalgebra/n1000_m200 | 1000.0 | 200.0 | 89.877 [86.593, 93.323] | 46.265 [41.997, 51.420] | -48.5% | yes | FAIL-NOISY/FAIL-NOISY | | audit_p6_svd_nalgebra/n1000_m50 | 1000.0 | 50.0 | 6.787 [6.559, 7.084] | 3.240 [3.044, 3.448] | -52.3% | yes | FAIL-NOISY/FAIL-NOISY | | audit_p6_svd_nalgebra/n100_m200 | 100.0 | 200.0 | 4.192 [3.968, 4.483] | 1.705 [1.645, 1.776] | -59.3% | yes | FAIL-NOISY/FAIL-NOISY | | audit_p6_svd_nalgebra/n100_m50 | 100.0 | 50.0 | 0.520 [0.498, 0.546] | 0.208 [0.201, 0.217] | -60.0% | yes | FAIL-NOISY/FAIL-NOISY | | audit_p6_svd_nalgebra/n500_m200 | 500.0 | 200.0 | 37.537 [34.975, 40.691] | 16.751 [15.808, 17.757] | -55.4% | yes | FAIL-NOISY/FAIL-NOISY | | audit_p6_svd_nalgebra/n500_m50 | 500.0 | 50.0 | 2.834 [2.766, 2.910] | 1.302 [1.227, 1.385] | -54.1% | yes | FAIL-NOISY/FAIL-NOISY | | audit_p6_svd_nalgebra/n500_m500 | 500.0 | 500.0 | 311.339 [298.175, 323.231] | 176.147 [165.663, 186.610] | -43.4% | yes | FAIL-NOISY/FAIL-NOISY | | audit_smooth/n500_m200 | 500.0 | 200.0 | 0.110 [0.105, 0.116] | 0.605 [0.588, 0.623] | +447.8% | yes | FAIL-NOISY/FAIL-NOISY | | boosting_boost_fosr/n100_m50_p2_mstop100 | 100.0 | 50.0 | 28.229 [28.156, 28.303] | 11.835 [11.799, 11.869] | -58.1% | yes | PASS/PASS | | boosting_boost_fosr/n100_m50_p2_mstop50 | 100.0 | 50.0 | 14.563 [14.419, 14.707] | 5.966 [5.944, 5.992] | -59.0% | yes | PASS/PASS | | coclustering_co_cluster_select/n120_m40_grid2x2 | 120.0 | 40.0 | 229.332 [224.421, 234.243] | 96.976 [95.913, 98.082] | -57.7% | yes | PASS/PASS | | density_lqd_fpca/n100_m81_ncomp3 | 100.0 | 81.0 | 3.013 [2.966, 3.060] | 1.436 [1.434, 1.438] | -52.3% | yes | PASS/PASS | | density_wasserstein_barycenter/n100_m81 | 100.0 | 81.0 | 0.390 [0.389, 0.391] | 0.132 [0.131, 0.134] | -66.1% | yes | PASS/PASS | | elastic_self_distance_matrix/n/10 | — | — | 48.865 [46.469, 51.442] | 65.243 [64.849, 65.902] | +33.5% | yes | FAIL-NOISY/PASS | | elastic_self_distance_matrix/n/30 | — | — | 356.063 [340.982, 371.940] | 627.510 [625.744, 629.745] | +76.2% | yes | FAIL-NOISY/PASS | | elastic_self_distance_matrix/n/50 | — | — | 757.224 [726.249, 793.015] | 1765.080 [1762.009, 1768.566] | +133.1% | yes | FAIL-NOISY/PASS | | elastic_self_distance_matrix/n_band0.1/10 | — | — | 14.835 [14.078, 15.648] | 13.636 [13.548, 13.742] | -8.1% | yes | FAIL-NOISY/PASS | | elastic_self_distance_matrix/n_band0.1/30 | — | — | 87.870 [84.262, 91.648] | 129.941 [129.452, 130.552] | +47.9% | yes | FAIL-NOISY/PASS | | elastic_self_distance_matrix/n_band0.1/50 | — | — | 186.782 [175.080, 199.842] | 366.789 [365.471, 368.265] | +96.4% | yes | FAIL-NOISY/PASS | | fem_smooth_gcv/nodes256_ngrid5 | — | — | 305.012 [303.683, 306.341] | 194.273 [193.991, 194.587] | -36.3% | yes | PASS/PASS | | fpca_der/n200_m50_ncomp5_nderiv1 | 200.0 | 50.0 | 10.272 [9.421, 11.124] | 0.228 [0.228, 0.229] | -97.8% | yes | FAIL-NOISY/PASS | | fpca_fsvd/n200_m50_ncomp5 | 200.0 | 50.0 | 1.254 [1.218, 1.289] | 0.555 [0.544, 0.567] | -55.7% | yes | FAIL-NOISY/PASS | | frechet_global_reg/n24_m81_xout5 | 24.0 | 81.0 | 0.260 [0.255, 0.266] | 0.146 [0.145, 0.146] | -44.1% | yes | PASS/PASS | | fts_ftsm/n200_m50_ncomp3 | 200.0 | 50.0 | 11.649 [9.960, 13.337] | 0.272 [0.271, 0.273] | -97.7% | yes | FAIL-NOISY/PASS | | inference_t_perm_test/na30_nb30_m50_nperm999 | — | 50.0 | 2.200 [2.194, 2.206] | 1.883 [1.876, 1.893] | -14.4% | yes | PASS/PASS | | mface_covariance/vars2_n100_m30 | 100.0 | 30.0 | 1251.421 [1219.669, 1283.172] | 752.167 [747.990, 756.847] | -39.9% | yes | FAIL-NOISY/PASS | | perf_dpca/n200_m50 | 200.0 | 50.0 | 68.212 [62.715, 73.055] | 35.607 [35.373, 35.890] | -47.8% | yes | FAIL-NOISY/PASS | | perf_face_covariance/n200_m30 | 200.0 | 30.0 | 207.489 [186.049, 229.894] | 287.022 [275.992, 298.998] | +38.3% | yes | FAIL-NOISY/FAIL-NOISY | ### scikit-fda and R control (unchanged libraries) Neither scikit-fda nor R changed between the two runs — any drift here is run-to-run measurement noise (machine load, cold-cache effects), not a code change, and is reported only as a summary statistic rather than per-cell. _Generated by `benchmarks/render_report.py` from comparison_baseline.csv, comparison.csv — do not edit by hand._ Python and R rows carry no confidence interval (a single measurement per cell); drift is reported as a median/max absolute percent change only, never as a significance test. | Library | Median \|delta %\| | Max \|delta %\| | Paired cells | |---|---|---|---| | python | 1.4% | 16.7% | 36 | | r_svd | 0.9% | 44.5% | 36 | | r_native_pca_fd | 1.0% | 2.4% | 6 | ### Baseline rows not re-measured _Generated by `benchmarks/render_report.py` from comparison_baseline.csv, comparison.csv, rerun_log.json — do not edit by hand._ | Source | Op | Library | N | M | Status | Detail | |---|---|---|---|---|---|---| | audit_p3_elastic_cross/n100_m200 | elastic | rust | 100.0 | 200.0 | timeout | timeout at 540 s (single-threaded) | | audit_p3_elastic_cross/n500_m50 | elastic | rust | 500.0 | 50.0 | inferred-timeout | cheaper sibling in the same group (filter '^audit_p3_elastic_cross/n500_m50$') timed out | | audit_p3_elastic_cross_banded/n100_m200 | elastic | rust | 100.0 | 200.0 | timeout | timeout at 540 s (single-threaded) | | audit_p3_elastic_cross_banded/n500_m50 | elastic | rust | 500.0 | 50.0 | inferred-timeout | cheaper sibling in the same group (filter '^audit_p3_elastic_cross_banded/n500_m50$') timed out | | audit_p3_elastic_self/n100_m200 | elastic | rust | 100.0 | 200.0 | timeout | timeout at 540 s (single-threaded) | | audit_p3_elastic_self/n500_m50 | elastic | rust | 500.0 | 50.0 | inferred-timeout | cheaper sibling in the same group (filter '^audit_p3_elastic_self/n500_m50$') timed out | | audit_p3_karcher/n500_m200 | elastic | rust | 500.0 | 200.0 | timeout | timeout at 540 s (single-threaded) | | boosting_boost_fosr/n100_m50 | boosting_boost_fosr | rust | 100.0 | 50.0 | id-not-produced | chunk completed ok but this id was not produced | | boosting_boost_fosr/n50_m100 | boosting_boost_fosr | rust | 50.0 | 100.0 | id-not-produced | chunk completed ok but this id was not produced | | boosting_boost_fosr/n50_m50 | boosting_boost_fosr | rust | 50.0 | 50.0 | id-not-produced | chunk completed ok but this id was not produced | | coclustering_co_cluster/n100_m50 | coclustering_co_cluster | rust | 100.0 | 50.0 | orphan | group 'coclustering_co_cluster' no longer defined in any fdars-core bench file | | coclustering_co_cluster/n50_m100 | coclustering_co_cluster | rust | 50.0 | 100.0 | orphan | group 'coclustering_co_cluster' no longer defined in any fdars-core bench file | | coclustering_co_cluster/n50_m50 | coclustering_co_cluster | rust | 50.0 | 50.0 | orphan | group 'coclustering_co_cluster' no longer defined in any fdars-core bench file | | density_wasserstein_barycenter/n200_m101 | density_wasserstein_barycenter | rust | 200.0 | 101.0 | id-not-produced | chunk completed ok but this id was not produced | | density_wasserstein_barycenter/n50_m101 | density_wasserstein_barycenter | rust | 50.0 | 101.0 | id-not-produced | chunk completed ok but this id was not produced | | density_wasserstein_barycenter/n50_m201 | density_wasserstein_barycenter | rust | 50.0 | 201.0 | id-not-produced | chunk completed ok but this id was not produced | | face_covariance/n200_m30 | face_covariance | rust | 200.0 | 30.0 | orphan | group 'face_covariance' no longer defined in any fdars-core bench file | | face_covariance/n50_m30 | face_covariance | rust | 50.0 | 30.0 | orphan | group 'face_covariance' no longer defined in any fdars-core bench file | | face_covariance/n50_m60 | face_covariance | rust | 50.0 | 60.0 | orphan | group 'face_covariance' no longer defined in any fdars-core bench file | | fem_smooth/nodes256_tris450 | fem_smooth | rust | — | — | orphan | group 'fem_smooth' no longer defined in any fdars-core bench file | | fem_smooth/nodes576_tris1058 | fem_smooth | rust | — | — | orphan | group 'fem_smooth' no longer defined in any fdars-core bench file | | fem_smooth/nodes64_tris98 | fem_smooth | rust | — | — | orphan | group 'fem_smooth' no longer defined in any fdars-core bench file | | fpca_variants_fsvd/n1000_m50 | fpca_variants_fsvd | rust | 1000.0 | 50.0 | orphan | group 'fpca_variants_fsvd' no longer defined in any fdars-core bench file | | fpca_variants_fsvd/n200_m50 | fpca_variants_fsvd | rust | 200.0 | 50.0 | orphan | group 'fpca_variants_fsvd' no longer defined in any fdars-core bench file | | fpca_variants_fsvd/n50_m200 | fpca_variants_fsvd | rust | 50.0 | 200.0 | orphan | group 'fpca_variants_fsvd' no longer defined in any fdars-core bench file | | fpca_variants_fsvd/n50_m50 | fpca_variants_fsvd | rust | 50.0 | 50.0 | orphan | group 'fpca_variants_fsvd' no longer defined in any fdars-core bench file | | frechet_anova/n200_m50 | frechet_anova | rust | 200.0 | 50.0 | orphan | group 'frechet_anova' no longer defined in any fdars-core bench file | | frechet_anova/n50_m200 | frechet_anova | rust | 50.0 | 200.0 | orphan | group 'frechet_anova' no longer defined in any fdars-core bench file | | frechet_anova/n50_m50 | frechet_anova | rust | 50.0 | 50.0 | orphan | group 'frechet_anova' no longer defined in any fdars-core bench file | | fts_long_run_covariance/n1000_m50 | fts_long_run_covariance | rust | 1000.0 | 50.0 | orphan | group 'fts_long_run_covariance' no longer defined in any fdars-core bench file | | fts_long_run_covariance/n200_m50 | fts_long_run_covariance | rust | 200.0 | 50.0 | orphan | group 'fts_long_run_covariance' no longer defined in any fdars-core bench file | | fts_long_run_covariance/n50_m200 | fts_long_run_covariance | rust | 50.0 | 200.0 | orphan | group 'fts_long_run_covariance' no longer defined in any fdars-core bench file | | fts_long_run_covariance/n50_m50 | fts_long_run_covariance | rust | 50.0 | 50.0 | orphan | group 'fts_long_run_covariance' no longer defined in any fdars-core bench file | | inference_t_perm_test/n200_m50 | inference_t_perm_test | rust | 200.0 | 50.0 | id-not-produced | chunk completed ok but this id was not produced | | inference_t_perm_test/n50_m200 | inference_t_perm_test | rust | 50.0 | 200.0 | id-not-produced | chunk completed ok but this id was not produced | | inference_t_perm_test/n50_m50 | inference_t_perm_test | rust | 50.0 | 50.0 | id-not-produced | chunk completed ok but this id was not produced | **Excluded by design (thread-scaling benches, D-12):** - `audit_p5_karcher_threads` — multi-thread scaling bench, excluded from the single-threaded head-to-head - `audit_p5_streaming_threads` — multi-thread scaling bench, excluded from the single-threaded head-to-head - `perf_parallelism_co_cluster` — multi-thread scaling bench, excluded from the single-threaded head-to-head - `perf_parallelism_frechet_anova` — multi-thread scaling bench, excluded from the single-threaded head-to-head ## N×M scaling _Generated by `benchmarks/render_report.py` from comparison.csv — do not edit by hand._ #### fpca scaling **rust** | N \ M | 50 | 200 | |---|---|---| | 100 | 0.424 | 1.985 | | 300 | 0.683 | 7.209 | | 500 | 0.913 | 10.152 | **python** | N \ M | 50 | 200 | |---|---|---| | 100 | 1.554 | 3.755 | | 300 | 1.501 | 8.759 | | 500 | 1.270 | 12.357 | **r_svd** | N \ M | 50 | 200 | |---|---|---| | 100 | 1.006 | 5.189 | | 300 | 1.915 | 25.561 | | 500 | 2.873 | 41.017 | | Library | N@M=50 | N@M=200 | M@N=100 | M@N=300 | M@N=500 | Theory | Source | |---|---|---|---|---|---|---|---| | rust | 0.47 | 1.04 | 1.11 | 1.70 | 1.74 | O(N*M*min(N,M)) thin SVD of the weighted, centered N x M data matrix | fdars-core/src/regression.rs::fdata_to_pc | _M-slopes are 2-point log-log estimates (only two M values in the grid)._ #### basis_smoothing scaling **rust** | N \ M | 50 | 200 | |---|---|---| | 100 | 0.053† | 0.211† | | 300 | 0.137 | 0.514 | | 500 | 0.216 | 0.834 | **python** | N \ M | 50 | 200 | |---|---|---| | 100 | 0.370 | 1.461 | | 300 | 0.453 | 2.829 | | 500 | 0.525 | 4.199 | **r_svd** | N \ M | 50 | 200 | |---|---|---| | 100 | 0.796 | 1.707 | | 300 | 1.643 | 3.132 | | 500 | 2.507 | 4.324 | | Library | N@M=50 | N@M=200 | M@N=100 | M@N=300 | M@N=500 | Theory | Source | |---|---|---|---|---|---|---|---| | rust | 0.87† | 0.85† | 0.99† | 0.96† | 0.97† | O(N*M*K) per-curve basis projection (K=nbasis=10 fixed) plus a one-time O(M^2*K) hat-matrix build independent of N; regime-dependent (N>>M: linear in N and M; M>>N: quadratic in M, independent of N) | fdars-core/src/smooth_basis.rs::smooth_basis | _M-slopes are 2-point log-log estimates (only two M values in the grid)._ #### depth scaling **rust** | N \ M | 50 | 200 | |---|---|---| | 100 | 0.105 | 0.546† | | 300 | 0.506† | 2.068 | | 500 | 0.757 | 3.511 | **python** | N \ M | 50 | 200 | |---|---|---| | 100 | 1.329 | 3.720 | | 300 | 2.660 | 9.692 | | 500 | 4.074 | 15.591 | **r_svd** | N \ M | 50 | 200 | |---|---|---| | 100 | 0.829 | 2.748 | | 300 | 1.762 | 6.955 | | 500 | 2.787 | 8.855 | | Library | N@M=50 | N@M=200 | M@N=100 | M@N=300 | M@N=500 | Theory | Source | |---|---|---|---|---|---|---|---| | rust | 1.26† | 1.17† | 1.19† | 1.01† | 1.11† | O(N*M*log N) rank-based Modified Band Depth (sorted-reference presort, then an O(log N) rank query per curve/evaluation point) | fdars-core/src/streaming_depth/mbd.rs::depth_batch | _M-slopes are 2-point log-log estimates (only two M values in the grid)._ #### functional_boxplot scaling **rust** | N \ M | 50 | 200 | |---|---|---| | 100 | 0.124† | 0.617† | | 300 | 0.509† | 2.158† | | 500 | 0.926† | 3.617 | **python** | N \ M | 50 | 200 | |---|---|---| | 100 | 1.794 | 4.170 | | 300 | 3.183 | 10.064 | | 500 | 4.712 | 16.616 | **r_svd** | N \ M | 50 | 200 | |---|---|---| | 100 | 2.070 | 6.683 | | 300 | 4.233 | 14.303 | | 500 | 7.058 | 21.627 | | Library | N@M=50 | N@M=200 | M@N=100 | M@N=300 | M@N=500 | Theory | Source | |---|---|---|---|---|---|---|---| | rust | 1.26† | 1.11† | 1.16† | 1.04† | 0.98† | dominated by the underlying depth computation, O(N*M*log N), plus an additive O(N*M) central-region/fence pass | fdars-core/src/depth/dispatch.rs::functional_boxplot | _M-slopes are 2-point log-log estimates (only two M values in the grid)._ #### fpcr scaling **rust** | N \ M | 50 | 200 | |---|---|---| | 100 | 0.430 | 2.007 | | 300 | 0.699 | 7.452 | | 500 | 0.940 | 10.241 | **python** | N \ M | 50 | 200 | |---|---|---| | 100 | 1.308 | 3.721 | | 300 | 1.631 | 9.246 | | 500 | 1.420 | 12.636 | **r_svd** | N \ M | 50 | 200 | |---|---|---| | 100 | 36.030 | 85.633 | | 300 | 96.773 | 619.725 | | 500 | 153.821 | 1049.904 | | Library | N@M=50 | N@M=200 | M@N=100 | M@N=300 | M@N=500 | Theory | Source | |---|---|---|---|---|---|---|---| | rust | 0.48 | 1.04 | 1.11 | 1.71 | 1.72 | O(N*M*min(N,M)) FPCA (fdata_to_pc) followed by an O(N*ncomp^2) OLS solve on a fixed 5-component design (ncomp constant, so the FPCA term dominates) | fdars-core/src/scalar_on_function/fregre_lm.rs::fregre_lm | _M-slopes are 2-point log-log estimates (only two M values in the grid)._ #### elastic scaling **rust** | N \ M | 50 | 200 | |---|---|---| | 100 | 1.839† | 27.701 | | 300 | 1.593† | 28.597† | | 500 | 1.499 | 27.470 | **python** | N \ M | 50 | 200 | |---|---|---| | 100 | 3.106 | 33.616 | | 300 | 2.877 | 33.571 | | 500 | 2.882 | 33.542 | **r_svd** | N \ M | 50 | 200 | |---|---|---| | 100 | 2.784 | 32.529 | | 300 | 1.921 | 32.815 | | 500 | 1.918 | 32.517 | | Library | N@M=50 | N@M=200 | M@N=100 | M@N=300 | M@N=500 | Theory | Source | |---|---|---|---|---|---|---|---| | rust | -0.13† | 0.00† | 1.96† | 2.08† | 2.10† | O(M^2) dynamic-programming grid fill per pair (coprime neighbourhood), independent of N -- the crosslang_elastic bench always times a fixed 2-curve pair | fdars-core/src/alignment/pairwise.rs::elastic_align_pair | _M-slopes are 2-point log-log estimates (only two M values in the grid)._ The empirical N- and M-exponents above are log-log slopes fit across the locked grid and are compared to the theory column, which is grounded in the actual fdars-core source (not the module docstring's prose description — see each row's `Source` column). For `elastic`, the theoretical O(M²) figure is not a single measured slope: it is (pair count ∝ N², exact by construction — the harness always times exactly one pair) × (per-pair DP cost ∝ M², which IS what the pairwise M-slope above measures). The empirical N-exponent near zero and M-exponent near two are both consistent with that decomposition. Every M-slope in this table is a two-point log-log estimate (the grid has only M ∈ {50, 200}) and should be read as directional, not as a precise exponent. ## Methodology ### Environment _Generated by `benchmarks/render_report.py` from env.json, env_baseline.json — do not edit by hand._ | Item | Phase 113 baseline | Phase 116 re-run | Changed? | |---|---|---|---| | cpu.cores_logical | 20 | 20 | no | | cpu.model | 13th Gen Intel(R) Core(TM) i9-13900H | 13th Gen Intel(R) Core(TM) i9-13900H | no | | criterion_version | 0.5 | 0.5 | no | | python.numpy_blas | "name": "scipy-openblas", | "name": "scipy-openblas", | no | | python.numpy_version | 2.5.3 | 2.5.3 | no | | python.version | 3.14.7 | 3.14.7 | no | | r.blas | /usr/lib/libblas.so.3.12.0 | /usr/lib/libblas.so.3.12.0 | no | | r.lapack_version | /usr/lib/liblapack.so.3.12.0 | /usr/lib/liblapack.so.3.12.0 | no | | r.version | 4.6.1 | 4.6.1 | no | | r_packages.bench | 1.1.4 | 1.1.4 | no | | r_packages.fda | 6.3.0 | 6.3.0 | no | | r_packages.fda.usc | 2.2.0 | 2.2.0 | no | | r_packages.fdapace | 0.6.0 | 0.6.0 | no | | r_packages.fdasrvf | 2.5.0 | 2.5.0 | no | | r_packages.jsonlite | 2.0.0 | 2.0.0 | no | | r_packages.refund | 0.1.40 | 0.1.40 | no | | r_packages.roahd | 1.4.3 | 1.4.3 | no | | rust.features | linalg,parallel | linalg,parallel | no | | rust.version | rustc 1.97.0 (2d8144b78 2026-07-07) | rustc 1.97.0 (2d8144b78 2026-07-07) | no | | scikit_fda.version | 0.10.1 | 0.10.1 | no | | thread_env.MKL_NUM_THREADS | 1 | 1 | no | | thread_env.NUMEXPR_NUM_THREADS | 1 | 1 | no | | thread_env.OMP_NUM_THREADS | 1 | 1 | no | | thread_env.OPENBLAS_NUM_THREADS | 1 | 1 | no | | thread_env.RAYON_NUM_THREADS | 1 | 1 | no | | thread_env.VECLIB_MAXIMUM_THREADS | 1 | 1 | no | | timestamp | 2026-09-23T08:10:19Z | 2026-09-25T00:15:39Z | yes | **Background load disclosure.** A quiet-machine check (load average < 2) was run and passed before the Phase 116 fdars re-run started, but unrelated desktop workloads (an editor's language server, a search indexer, a browser) returned partway through the ~2.9-hour run: the 1-minute load average at the start of individual `--rust-next` scheduler calls ranged from 2.8 to 15.0 on this 20-logical-core machine, not uniformly quiet throughout. All measurements remain single-threaded, and every headline cell reported in this document passes the 5% noise gate regardless — but this is disclosed because background CPU contention inflates the wall-clock time of whatever is being measured at that moment. The scikit-fda and R grid ran afterwards, in separate processes, at a lower load (1-minute load average roughly 2.5–3), so the heavier contention window overlapped the fdars measurements. Any bias this introduces therefore most likely runs against fdars, not in its favor. ### Numeric agreement gate _Generated by `benchmarks/render_report.py` from gate.py — do not edit by hand._ | Op | Metric | Threshold | Status | |---|---|---|---| | fpca | subspace cos(largest principal angle) | >= 0.99 | hard-gated | | basis_smoothing | max |diff| | < 0.001 | hard-gated | | depth | Pearson |corr| | >= 0.99 | hard-gated | | functional_boxplot | Pearson |corr| | >= 0.99 | hard-gated | | fpcr | Pearson |corr| (disclosure only) | >= 0.95 | disclosed-only | | elastic | L-infinity on warping values | < 0.01 | hard-gated | fdars output itself is agreement-checked for FPCA only (`fpca_dump`, fdars-vs-R hard gate); for the other hard-gated ops (basis_smoothing, depth, functional_boxplot, elastic) the gate checks that scikit-fda and R agree with EACH OTHER, not that fdars agrees with either (`run_all.py` step 6b). The gate is fail-closed and runs BEFORE any timing is recorded for a cell — if the gate fails, nothing downstream of it is measured for that cell. It is unchanged in this phase (D-11): the same thresholds, computed the same way, as the Phase 112/113 baseline runs. fdars' own output is agreement-checked against another ecosystem only for FPCA (via the standalone `fpca_dump` binary, checked against R); for the other hard-gated ops the gate checks that scikit-fda and R agree with each other, which is a weaker but still meaningful signal that the op is being compared fairly. ### Timing statistics and noise gate Each ecosystem reports a different timing statistic, by design of its own idiomatic benchmarking tool: fdars uses criterion 0.5's mean with a 95% bootstrap confidence interval; scikit-fda uses Python's `timeit` mean over a fixed repeat count per op (50 repeats for `fpca`/`basis_smoothing`/ `depth`/`functional_boxplot`/`fpcr`; 10 repeats for the pairwise `elastic` op, since it is heavier per call); R uses `bench::mark()`'s median over a fixed iteration count per op (10 iterations for the same five ops; 5 iterations for pairwise `elastic`). Python and R therefore carry no confidence interval — they report one point estimate per cell, which is why the before/after control table above reports drift as median/max percent change rather than a significance test. The noise gate (shared with `hotspot_ranking.md`, never redeclared) fails a criterion row when its relative standard deviation is >= 5% or its CI width is >= 5%; it is the only gate applied to fdars rows and has no equivalent for the single-sample Python/R rows. Significance for any comparison is always the strict non-overlapping-CI test; the ratio-label parity band is +/-10%. ### Criterion settings _Generated by `benchmarks/render_report.py` from fdars-core/benches/*.rs — do not edit by hand._ | Group | Sample size | Measurement time (s) | Warm-up (s) | |---|---|---|---| | crosslang_basis_smoothing | 30 | 10 | 3 | | crosslang_depth | 30 | 10 | 3 | | crosslang_elastic | 30 | 30 | 5 | | crosslang_fpca | 30 | 10 | 3 | | crosslang_fpcr | 60 | 20 | 3 | | crosslang_functional_boxplot | 30 | 10 | 3 | | functional_boxplot | 50 | 20 | 5 | | perf_face_covariance | 15 | 30 | 3 | ### Data parity The grid-aligned fdars bench (`fdars-core/benches/crosslang_grid.rs`) reads the exact same seeded CSVs the Python and R runners read, and hard-fails (`FDARS_CROSSLANG_REQUIRE_CSV=1`) rather than silently falling back to a different random draw when the harness invokes it. A standalone `cargo bench` run (no harness) instead falls back to a deterministic `StdRng(42)` generator, so the bench is still runnable in isolation but never silently mismatched against the cross-language CSVs when it matters. Legacy, non-grid-aligned criterion ids (the `audit_*`/`perf_*` benches used in the before/after tables above) use their own in-bench generators and were never intended to share data with the cross-language harness. ### Threading All six `*_NUM_THREADS` environment variables are pinned to `1` for every ecosystem in every table in this report. With the default `parallel` feature enabled, fdars uses rayon for multi-core execution outside of this report's fairness-first, single-threaded measurements — that capability is real and shipped, but deliberately not tabulated here (D-12), because a multi-threaded fdars number next to single-threaded Python/R numbers would not be a fair comparison. The four `audit_p5_*_threads`/ `perf_parallelism_*` benchmark groups exist specifically to explore thread-scaling and are excluded from every table above. ## Reproduction The full re-run this report is based on can be reproduced end-to-end from a clean clone: 1. Clone the repository: ```bash git clone https://github.com/sipemu/fdars.git cd fdars ``` 2. Install a Rust toolchain matching the [Environment](#environment) table above (the `linalg` feature needs Rust 1.84+; the `crosslang_grid` bench itself only needs the workspace MSRV): ```bash rustup toolchain install 1.97.0 rustup default 1.97.0 ``` 3. Create the Python virtual environment and install the pinned harness dependencies: ```bash python3 -m venv benchmarks/.venv benchmarks/.venv/bin/pip install -r benchmarks/requirements.lock ``` 4. **Python 3.14 only:** apply the `multimethod` metaclass patch (see `benchmarks/README.md`'s "Python 3.14 multimethod patch" section for why this is needed): ```bash patch -p1 -d benchmarks/.venv/lib/python3.14/site-packages \ < benchmarks/patches/multimethod-2.1-py314.patch ``` 5. Restore the R packages from `benchmarks/renv.lock`, then verify them (see `benchmarks/README.md` for the exact `renv::restore()` invocation and why `.Rprofile`/`renv/` are gitignored and not auto-activated on a clean clone): ```bash Rscript -e "renv::restore()" Rscript benchmarks/r_deps_check.R ``` 6. Run the single end-to-end reproduction command: ```bash benchmarks/.venv/bin/python benchmarks/run_all.py --full ``` The default 540s per-chunk / per-cell bound matches the published run exactly (7 heavy single-threaded elastic/Karcher chunks are expected to time out under it — see [Before/after vs the Phase 113 baseline](#beforeafter-vs-the-phase-113-baseline)). Pass `--timeout-s 0` to remove fdars' per-chunk bound entirely and measure the cells the published run recorded as timeouts (this will take substantially longer — budget accordingly). 7. Render the report and verify it against the committed inputs: ```bash benchmarks/.venv/bin/python benchmarks/render_report.py benchmarks/.venv/bin/python benchmarks/render_report.py --check benchmarks/.venv/bin/python benchmarks/render_report.py --check-prose ``` ### Measured runtime and disk footprint _Generated by `benchmarks/render_report.py` from rerun_log.json — do not edit by hand._ head_sha: `242ab06722cfb68de28dd57203d3eb4757062a6f` started_utc: `2026-09-24T21:05:14Z` | Kind | Status | Count | Wall-clock (s) | |---|---|---|---| | footprint | ok | 1 | 0.0 | | grid_cell | ok | 6 | 0.0 | | prebuild | ok | 1 | 20.2 | | remediation | ok | 1 | 206.4 | | rust_chunk | interrupted | 9 | 1778.0 | | rust_chunk | ok | 60 | 6370.7 | | rust_chunk | skipped-inferred-timeout | 3 | 0.0 | | rust_chunk | timeout | 4 | 2160.0 | Total measured wall-clock: 2.93 hours Disk footprint (target/): 57.83 GiB The published run above was not driven with a single blocking `--full` call end to end — it used the same underlying steps but invoked as a resumable sequence (`--rust-next --budget-s 540` repeated until `RUST_CHUNKS_REMAINING=0`, then `--grid --cell ` once per grid cell) so that a single interrupted call could resume without losing prior progress; `--full` (step 6 above) wraps that exact sequence into one command for a fresh reproduction run. ## Known limitations and disclosures - **Background desktop load was present during part of the fdars re-run** (1-minute load average 2.8–15.0 at scheduler-call starts, despite a passing quiet-machine check before the run started) — see [Environment](#environment) for the full disclosure. The heavier load overlapped the fdars measurements, so any bias most likely runs against fdars (slower), not in its favor. - **Timeouts and orphans are disclosed, not hidden.** 7 heavy single-threaded elastic/Karcher chunks did not complete inside the 540s bound (4 direct timeouts, 3 inferred from a cheaper sibling in the same group timing out) and are listed by criterion id in the "Baseline rows not re-measured" table above; 6 legacy benchmark groups no longer exist in the bench files and are flagged `orphan` in the same table, not silently dropped from the report. - **The Phase 113 baseline is of mixed vintage and mostly not thread-pinned** — only `fregre_lm` and `functional_boxplot` were re-measured live in Phase 113; every other baseline row is on-disk criterion output from whatever optimization phase last touched that bench, measured on the default multi-core rayon pool. The before/after tables above are a traceability record against that baseline, not an optimization measurement. - **fdars' own output is agreement-checked against another ecosystem for FPCA only.** For the other hard-gated ops the gate checks scikit-fda against R, not fdars against either — see [Numeric agreement gate](#numeric-agreement-gate). - **FPCR is a disclosed, not gated, cross-library comparison** — scikit-fda's FPCA→OLS and R's `fda.usc::fregre.pc` are related but not numerically identical algorithms. - **The R FPCA timing shown as the hard-gated comparator is a hand-rolled Simpson-weighted SVD port**, chosen because it agrees with fdars at subspace cosine 1.0; R's own native `fda::pca.fd` (B-spline coefficient space) is timed separately and shown alongside it, never conflated with it. - **The Python 3.14 harness venv needs an in-place metaclass patch** to `multimethod/__init__.py` before scikit-fda will import; the patch file is committed at `benchmarks/patches/multimethod-2.1-py314.patch` and its provenance (pristine wheel hash matches the venv's own `dist-info/RECORD` entry) is documented in `benchmarks/README.md`. - **Single machine, single run.** Every number in this report comes from one run on one machine (see [Environment](#environment)); it is not a statistical claim across hardware or repeated runs, though the noise gate and confidence intervals bound the uncertainty of any one criterion cell. - **scikit-fda is this milestone's functionality yardstick, not its performance yardstick's peer in scope.** This report covers performance only — a separate audit (Phase-numbered elsewhere in this milestone's history) covers functionality-gap analysis against scikit-fda; the two should not be conflated.