# AgroBench: Vision-Language Model Benchmark in Agriculture

**Authors**: Risa Shinoda, Nakamasa Inoue, Hirokatsu Kataoka, Masaki Onishi, Yoshitaka Ushiku  (ICCV'25)
**Dataset**: [AgroBench on Hugging Face](https://huggingface.co/datasets/risashinoda/AgroBench)

## 🚀 Usage

All models used in our paper implementation are publicly available.  
You can evaluate your own model on our dataset by following the same experimental setup.

## ⚙️ Setup
```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements-gpt-minimum.txt
```

## 🧠 Inference

Each model is registered via  `common/registry.py `,
and can be switched by specifying its config file.

### Example 1 — GPT-4o (text + image)
```bash
python -m scripts.run_infer \
  --config configs/default.yaml \
  --model-config configs/gpt.yaml
```

### Example 2 — Gemini 1.5 Flash (Google GenAI)
```bash
python -m scripts.run_infer \
  --config configs/default.yaml \
  --model-config configs/gemini.yaml
```

Predictions are automatically saved under:
`outputs/<model>/<source>/predictions.jsonl`

## 📈 Evaluation
Evaluate the model’s accuracy overall and per category/source:
```bash
python -m scripts.run_eval \
  --pred outputs/gpt/did/predictions.jsonl
```
Outputs:
`outputs/gpt/did/report.json`

## 📃 Ciation
If you use our dataset or code, please cite our paper.

```
@InProceedings{Shinoda_2025_ICCV,
  author    = {Shinoda, Risa and Inoue, Nakamasa and Kataoka, Hirokatsu and Onishi, Masaki and Ushiku, Yoshitaka},
  title     = {AgroBench: Vision-Language Model Benchmark in Agriculture},
  booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  month     = {October},
  year      = {2025},
  pages     = {7634-7644}
}
```