--- name: iccv-experiments description: Use when designing the experimental program for an ICCV paper against the early-March deadline, covering benchmark-drift audits across the two-year gap, baseline fairness in the foundation-model era, ablations that isolate mechanisms, qualitative and failure evidence, and sequencing runs so the decisive result lands before the deadline, not after. --- # ICCV Experiments An ICCV experimental program is built under a fixed, unrepeatable date: the early-March deadline of an odd year (March 7 in 2025, with supplement due the same day). The program design problem is therefore *sequencing under a deadline*, on top of the usual question of what evidence convinces a vision reviewer. Both are handled here. ## Start with the benchmark-drift audit Because the venue is biennial, a project conceived after one ICCV and submitted to the next spans two years of field motion. Before designing experiments, audit what moved: ```text drift audit (fill once, in the autumn before the deadline) benchmarks: which datasets did the last two years of CVPR/ECCV/ICCV papers in this area actually evaluate on? any new canonical benchmark? baselines: leaderboard top-3 today vs when the project started → any baseline in your draft older than ~18 months is a red flag backbones: what does current SOTA initialize from? (matching this defines "fair" for your comparisons) metrics: any metric revision or new evaluation server since last cycle? protocols: resolution/prompt/eval-harness conventions that changed ``` Papers rejected for "outdated comparisons" are usually not lazy — they froze their experiment matrix at project start and never re-based. Re-run the audit in January; two months before an ICCV deadline is exactly when the previous November's CVPR-cycle preprints flood arXiv. ## The fairness ledger Vision reviewers' most reliable objection is compute-and-pretraining confounds dressed as method wins. Make fairness auditable with a ledger column per comparison: | Axis | Your method | Each baseline | Mismatch handling | |---|---|---|---| | Backbone + init checkpoint | | | Match, or add a matched row | | Pretraining data exposure | | | Disclose; beware test-adjacent leakage in web-scale corpora | | Input resolution / tokens | | | Match or tabulate both | | Training schedule + budget | | | Report epochs and GPU-hours side by side | | Number quoted vs re-run | | | Mark re-runs; footnote protocol deltas | The foundation-model twist: when everything builds on the same giant checkpoint, *data exposure* replaces architecture as the confound reviewers hunt. If your improvement could plausibly come from the pretrain having seen the test domain, run the decontamination or cross-domain check before a reviewer asks for it in a window when you have one rebuttal page to respond. ## Ablations that isolate, not decorate Structure the ablation grid so every row flips one switch, and include the two rows that distinguish a mechanism from a lucky configuration: the *transplant* (your module inserted into a baseline — does the gain travel?) and the *sensitivity sweep* (is the headline number a plateau or a spike?). Rows argued from in the text belong in the body; the full grid goes to the same-day supplement. If the core ablation shows the mechanism is not doing the work, that is an October discovery you want in November — which is why it runs first (see sequencing below). ## Qualitative evidence with stated rules At a venue that reviews with its eyes, image and video evidence carries real weight and attracts real skepticism. Three requirements: a declared selection rule on every grid ("first N val images", "random seed 0" — curation without a rule is what reviewers assume by default); side-by-sides against the two strongest baselines on identical inputs; and a failure-mode section with a taxonomy, previewed in the body and cataloged in the supplement. For temporal or 3D claims, the supplement video is the primary exhibit — packaging in `iccv-supplementary`. ## Sequencing runs toward March The scarce resource is calendar, not GPUs. Order the program by *decision value per week*: 1. **Falsifiers first** (autumn): the core ablation and the strongest baseline comparison. If the idea dies, it dies while retargeting to CVPR-November is still possible (`iccv-workflow`'s autumn fork). 2. **Headline table** (winter): full benchmark suite, matched settings, seeds on the cheap rows. 3. **Breadth pass** (January): second domain, transfer, robustness suite — whatever supports the claim's outer scope. 4. **Freeze margin** (mid-February): the main table freezes ~2–3 weeks out; the deadline is same-day for the supplement, so late results have no legal landing zone except the rebuttal — and reviewers may not request major new experiments anyway. 5. **Rebuttal reserve**: hold 10–20% of compute for May; the most common winnable rebuttal item is a small matched re-run reported as a mini-table (`iccv-author-response`). ```python # deadline_math.py — sanity-check the plan against the calendar runs = {"core_ablation": 6, "main_table": 21, "breadth": 10} # GPU-days each gpu = 8; days_left = (deadline - today).days - 18 # freeze margin assert sum(runs.values()) / gpu <= days_left, "cut scope now, not in February" ``` ## The four questions any ICCV review silently asks Does it work (main table, matched)? Why does it work (isolating ablations + transplant)? Where does it break (failure taxonomy, honest transfer results)? What does it cost (params, latency on named hardware, training GPU-hours — volunteered, since no form mandates it; see `iccv-reproducibility`)? Draft the experiments section as answers to these four, in this order, and the reviewer's checklist fills itself. ## Reverify each cycle - The 2027 deadline chain — sequencing above is calendar-shaped and the calendar is 待核实 until posted. - Whether supplements stay same-day (changes step 4). - Evaluation-server rules and submission budgets on your benchmarks. - Any new ethics/human-data documentation the 2027 forms may require. ## Output format ```text [Drift audit] run on ; stale baselines found: [Fairness ledger] axes matched or disclosed per comparison: n/m [Ablation] one-switch rows: ; transplant + sensitivity present: yes/no [Qualitative] selection rules stated · failure taxonomy drafted [Sequencing] falsifiers scheduled before ; rebuttal reserve: [Cut candidates] ```