Problem & motivation
Turning heterogeneous paper material into a usable, condensed artifact requires evidence selection, planning, and iterative repair over a long horizon.
Look for the paper-to-artifact path in Figure 6.
AutoDesign · research companion
Meta-Harness Optimization for Long-Horizon Agentic Design
AutoDesign treats human-aligned design generation as an opportunity to improve the harness around a fixed model.
Across rollout feedback, the learned DesignHarness turns paper evidence into editable, source-grounded artifacts and evaluates the result with PosterBench.
Method
Results
A trace of harness improvement, then a reusable gain across configurations.
The paper tracks autonomous improvement, a plateau, and a further gain after human guidance redirects the search; the learned DesignHarness improves all seven tested configurations.
At a glance
The harness, not the model, is what learns.
The paper holds model parameters fixed and improves the surrounding design harness from execution traces and evaluation evidence.
Approach
An inner loop makes and revises an artifact; an outer loop analyzes rollouts, proposes one component update, and gates promotion on evaluation evidence.
Trace the nested loops in Figure 3.
Reported takeaways
AutoDesign reports a 78.32 main-track score; attaching DesignHarness raises the reported mean across seven controlled configurations from 54.99 to 67.39.
Inspect the trace and comparison in Figure 1.
AutoDesign designs the poster for AutoDesign.
Reading cue: one long-horizon workflow performs source ingestion, iterative generation and refinement, critic feedback integration, and finalization in about 40 minutes.
Method
One loop refines artifacts. Another refines the harness.
Follow a step to see its local role. The figures below keep the pipeline tied to the paper’s own system diagrams.
One outer-loop iteration, made legible.
Reading cue: rollout, evaluation, update proposal, and acceptance repeat autonomously; human guidance redirects evaluation and optimization when needed.
The inner loop refines artifacts; the outer loop refines the harness.
Reading cue: critic feedback revises the current artifact, while rollout evidence determines whether one harness component is accepted or rejected.
DesignHarness keeps the artifact editable from the start.
Reading cue: source materials flow through the designer, rule-based validator, model-based visual critic, and finalization path before the best valid candidate is selected.
Results
Three evidence lanes: score, harness effect, and human preference.
Reported findings, scoped to the stated PosterBench tracks and the paper’s system-blind human study.
Full-scale main track
78.32 on PosterBench, reported as 7.45 points above Claude Design.
The paper identifies this as its main-track result. Figure 1 supplies the high-level performance narrative; the reported full-scale benchmark contains 100 papers.
Read the setting
PosterBench is described as a 100-paper full set across five disciplines, with a seven-dimension evaluation protocol and record-level ceilings.
Harness contribution
Mean PosterBench Score rises from 54.99 to 67.39 across seven controlled configurations.
Attaching the learned DesignHarness is reported to improve all seven controlled code-agent–model configurations. Figure 1 visualizes the improvement range and best overall score.
Read the setting
The reported mean change is +12.4%. Figure 1(b) states a 5.0–19.6 point gain range across the seven configurations.
System-blind preference
Human evaluation is presented as an independent evidence lane.
Figure 9 reports a system-blind pairwise study with eleven volunteer reviewers, 936 submitted responses, 933 ranking judgments, and three skips.
Read the setting
For each comparison, reviewers saw two posters generated from the same paper without method, system, or model identity disclosures.
Evaluation combines traceable checks with rubric-guided visual judgment.
Reading cue: spatial, OCR, numeric-grounding, and render-integrity checks complement source-grounded assessments of visual evidence, layout, readability, and aesthetics.
Score is read alongside cost and runtime.
Reading cue: seven configurations are compared on PosterBench-mini; marker size denotes median runtime and the dashed line is the empirical Pareto frontier.
Preference evidence is system-blind.
Reading cue: Bradley-Terry probabilities sit beside AutoDesign head-to-head outcomes against each baseline, based on 936 volunteer-reviewer responses.
Wider PosterBench score margins yield more human agreement.
Reading cue: the figure pairs poster-level score association with increasing agreement as the PosterBench score margin grows.
Reproducibility
What the current paper makes available.
The current paper links its project website, code repository, and live demo alongside the implementation and evaluation details.
Project website
The current paper links autodesign.designanything.ai for the project context and updates.
Code repository
The source implementation is linked from the paper at github.com/Yaxin9Luo/AutoDesign.
Live demo
Editable posters can be used and locally refined through the paper's DesignAnything demo.
Scope & limitations
The paper makes its boundary conditions explicit.
- At a given optimization iteration, an update may span files within one selected component, but it cannot modify another component in that same iteration.
- PosterBench’s displayed dimension means do not generally reproduce the Overall value because the latter is averaged after record-level ceilings are applied.
- The broader paper-to-slide, paper-to-webpage, and paper-to-conference-video outputs are presented as pilots; PosterBench evaluates academic posters only, and cross-format evaluation remains future work.
The current DesignHarness already extends beyond posters.
Reading cue: paper posters, slide decks, webpages, and conference videos are shown; the paper frames meta-harness optimization across these formats as a future path toward general agentic design.