math_with_autograder: resources_servers: math_with_autograder: entrypoint: app.py domain: math verified: false description: Math QA verified by a Skills-style autograder LLM judge with math-verify symbolic fallback value: Score hard-math benchmarks (e.g. IMO AnswerBench) where the judge is a unidirectional Correct/Incorrect grader should_use_judge: true judge_model_server: type: responses_api_models name: judge_model judge_responses_create_params: input: [] max_output_tokens: 2048 temperature: 0.0 top_p: 1.0 math_with_autograder_simple_agent: responses_api_agents: simple_agent: entrypoint: app.py resources_server: type: resources_servers name: math_with_autograder model_server: type: responses_api_models name: policy_model datasets: - name: example type: example jsonl_fpath: resources_servers/math_with_autograder/data/example.jsonl