Insight In, Hallucination Out.
English · 简体中文
Yiming Ren · Xiang Liu · Qumeng Sun · Xiao Zhang · Jiahao Li
Project Leaders: Haoyang Zhang · Junjie Wang
Infinite Evolution Lab, EvoMap
 AutoResearch is an open-source agent workflow for AI and machine learning research. Give it a research idea, or let it discover directions from recent papers, developer communities, and open-source trends. It continues through experiment planning, implementation, review, execution, result analysis, and independent evaluation to produce an evidence package ready for paper writing. The workflow is stateful and recoverable. It can iterate based on pilot results and independent review. Research plans, code, run logs, metrics, failure causes, critic reports, and blind reviews are written to disk so researchers can inspect, take over, or stop the process.  ## 1. Ways to Use AutoResearch | Your starting point | Path | Main outputs | |---|---|---| | You do not have a specific idea yet | Run Idea Generation | Candidate research directions, reviewed ideas, and experiment plans | | You already have an idea | Execute the idea directly | Experiment code, run logs, result analysis, and independent review | | You want the complete workflow | Generate ideas, then select a plan for execution | A complete record from research signals to paper-ready evidence | ## 2. Core Capabilities | Capability | What you get | |---|---| | Cross-domain idea generation | Discover problems from recent external signals, then add constraints and experience from your own domain knowledge | | Independent multi-model review | Use at least three distinct models during Idea Generation instead of letting one model generate and approve its own work | | Stateful experiment execution | Persist plans, code, queues, logs, and conclusions so long-running work can resume after interruption | | Pilot before scaling | Test feasibility at lower cost before starting full experiments or stopping early | | Traceable evidence and sources | Record Forge sources, knowledge directions, experiment results, critic reports, and blind reviews | | Support for negative results | Preserve evidence and stop when a hypothesis fails instead of forcing every experiment into a success story | Research agents can invent missing details when evidence is thin and repeatedly validate their own output. AutoResearch grounds problem discovery in real signals, adds domain knowledge from a local knowledge base, and checks important claims through cross-model review, source records, experiment logs, critic reports, and blind review. These mechanisms reduce unsupported generation, unclear provenance, inflated self-evaluation, and overinterpretation of negative results. The system cannot guarantee that every conclusion is correct, but it preserves the evidence and state needed for researcher review. ## 3. Quickstart ### 3.1 Clone the Repository and Check the Environment Prepare a Linux machine, either local or accessible over SSH, with Git, Python 3.10+, and `python3-venv` installed: ```bash git clone https://github.com/EvoMap/AutoResearch.git cd AutoResearch bash scripts/bringup.sh ``` `scripts/bringup.sh` creates `.venv`, installs Python dependencies, runs the baseline tests and secret scan, and checks the current model configuration. It does not contact model services or incur API charges. On the first run, before API credentials are configured, a final `BLOCKED` result or nonzero exit is expected. Confirm that Python setup, dependency installation, and tests succeeded, then configure credentials in the next step. ### 3.2 Configure Model Services Create local configuration files without overwriting existing ones: ```bash test -f .env || cp .env.example .env test -f config/providers.local.json || \ cp config/providers.example.json config/providers.local.json ``` Edit both files: - `.env` stores real API URLs, keys, proxies, and other machine-local values. Do not commit it. - `config/providers.local.json` declares endpoints, model aliases, and the models assigned to each role. AutoResearch does not require a fixed combination of Gemini, GPT, or Claude. You can use one or more compatible endpoints. Stages that require independent opinions count distinct underlying model identities; a single endpoint may expose several different models. ### 3.3 Test the APIs ```bash set -a . ./.env set +a .venv/bin/python scripts/preflight.py --live ``` This command sends a small number of real requests. Exit code `0` means that normal roles have usable models and that multi-model stages such as Idea Forge and the critic meet their independence requirements. Then choose a path: ```bash # Generate ideas. .venv/bin/python idea_generation.py # Execute an existing idea. # Continue with section 5, "Idea Execution" ``` ## 4. Idea Generation: Cross-Domain Discovery ### 4.1 Intersect External Research Signals with Local Domain Knowledge - **Online research signals:** Collect recent papers, community discussions, and open-source trends, then aggregate, deduplicate, filter, and assess them. - **Local domain knowledge:** Read research experience, constraints, and common failure patterns maintained by the user in `knowledge_base/`. - **Cross-domain discovery:** Combine new external signals with local knowledge directions to produce candidate ideas, then run cross-review and experiment planning. The workflow never rewrites the local knowledge base automatically. You can use the included directions or add your own Markdown files. Two entrypoints cover full runs and targeted recovery: - `idea_generation.py` is the recommended entrypoint. It runs online collection, filtering, Idea Forge, and result updates. - `run_pending_forge.py` resumes seeds already present in `data/pending_forge_seeds.json` without collecting online sources again. ### 4.2 Run the Pipeline | Stage | What happens | |---|---| | 1. Collect | Gather recent research signals from several public channels | | 2. Filter | Aggregate, deduplicate, screen, and deeply assess candidate directions | | 3. Intersect | Combine each accepted signal with selected local knowledge directions | | 4. Generate and review | Ask three or more distinct models to develop ideas independently and cross-review the candidates | | 5. Plan | Check freshness and agreement, then produce experiment plans for accepted ideas | Run the default entrypoint: ```bash .venv/bin/python idea_generation.py ``` Main outputs: | Path | Contents | |---|---| | `data/candidates/` | Aggregated candidate research signals | | `data/verified/` | Filtering and in-depth assessment results | | `data/idea_forge/` | Complete ideas, review results, and experiment plans | | `logs/` | Run logs | > If a run produces no new seeds, the current implementation searches recent historical verification results for strongly recommended seeds and records that fallback in the logs. ### 4.3 Select Local Knowledge Directions List the available directions or filter them by keyword: ```bash .venv/bin/python src/idea_forge/b_library.py .venv/bin/python src/idea_forge/b_library.py agent ``` The four registered directions are used by default. To choose your own combination, add the following to `config/providers.local.json`: ```json { "idea_forge": { "b_directions": ["Agent_运行时与沙箱", "视觉推理"] } } ``` Each direction name corresponds to a Markdown filename under `knowledge_base/`. Adding directions increases generation and review calls; start with a small set when validating a new setup. ### 4.4 Optional: Draft a Knowledge Direction with GPT Researcher The repository does not bundle GPT Researcher source code, and Idea Generation does not invoke it automatically. To gather material for a new direction, install the pinned upstream release in a separate Python 3.11 environment: ```bash python3.11 -m venv .venv-research .venv-research/bin/python -m pip install -r requirements-research.txt ``` The upstream tool does not read `config/providers.local.json`. It reads environment variables directly. The default setup expects `OPENAI_API_KEY` and `TAVILY_API_KEY` in `.env`; use upstream variables such as `FAST_LLM`, `SMART_LLM`, and `RETRIEVER` to change models or retrieval backends. The following command accesses the network and may incur model and retrieval charges, so it requires explicit confirmation: ```bash set -a . ./.env set +a .venv-research/bin/python scripts/research_to_knowledge.py \ "agent runtime safety" \ --confirm-paid-network ``` Drafts are written only to the Git-ignored `workspaces/knowledge-drafts/` directory. Review sources, remove incorrect content, and complete the sections required by `knowledge_base/TEMPLATE.md` before moving accepted material into `knowledge_base/`. The adapter never rewrites the formal knowledge base automatically. ### 4.5 Select and Export an Idea Idea Generation does not choose the final plan for execution. First list executable plans: ```bash .venv/bin/python src/idea_provenance.py list \ --forge-file data/idea_forge/forge_YYYYMMDD_HHMM.json ``` Then export a selected plan to `data/ideas/`: ```bash .venv/bin/python src/idea_provenance.py export \ --forge-file data/idea_forge/forge_YYYYMMDD_HHMM.json \ --result-index 1 \ --plan-index 1 \ --output data/ideas/my_experiment.txt ``` Both indices start at `1`. The exported file records the Forge file checksum, seed index, plan index, and knowledge directions for downstream experiment and dashboard provenance.| Stage | Role | Purpose | Model requirement |
|---|---|---|---|
░ Idea signal filtering |
screener |
Quickly screen online research signals | One model |
judge |
Assess the research value of candidate signals in depth | One model | |
consensus_checker |
Check whether repeated assessments genuinely agree | One model | |
▒ Idea generation and validation |
ideator |
Develop ideas independently through cross-domain discovery and cross-review them | At least 3 distinct models |
planner |
Turn an accepted idea into an experiment plan | One model | |
freshness_refresher |
Refresh the plan with newer models, datasets, and baselines | One model | |
▓ Idea execution |
agent |
Drive coordination, planning, implementation, and experiments | One model |
code_reviewer |
Review experiment plans and code | One model | |
critic + optional critic_secondary |
Challenge completion and provide memory-independent blind review | 1 model for the primary role; 2 distinct models when the secondary role is enabled | |
run_monitor |
Compress long-running logs into progress summaries | Optional; one model when enabled |