Your turn: Just for fun!
Practice mode
Human Leaderboard
GA: tasks solved (%). TT: average time across all four tasks.
Loading records…
| Rank | Player | GA ↑ | TT ↓ | Date | Your record |
|---|
What is IGG?
IGG evaluates GUI grounding when targets must first be revealed through interaction.
Task
Agents use screenshots and action history to reveal and click the target in up to eight actions.
Benchmark
601 tasks across 16 web environments modeled on familiar applications.
Observabilitybring the target into view
Disambiguationreveal cues on hover
State controlexpand controls or dismiss overlays
Temporal controlwait for the right state
- Google Calendar
- Google Docs
- Google Sheets
- Notion
- Gmail
- VS Code Web
- Figma
- GitHub
- Discord
- Twitter / X
- Spotify
- Netflix
- YouTube
- Walmart
- Google Maps
Results
Human and model performance
Human participants achieve 94.6% accuracy in our study. On the full benchmark, GPT-5.5 reaches 79.4% and the strongest evaluated open model reaches 31.6%.
Revealing the target
Specifying the reveal action raises Qwen3-VL’s accuracy from 30.8% to 33.9%. Revealing the target for the model raises it to 81.9%.
Learning to reveal targets
Base models rely mostly on clicks and never use drag.
SFT followed by GRPO improves accuracy by 17.5–19.2 percentage points across three 8B models. Qwen3-VL’s accuracy on movable occlusions rises from 0% to 62.5%.
Why it matters
Hidden controls, overlays, and loading states pose challenges for GUI agents. IGG provides a benchmark to evaluate these challenges and improve how agents handle them.
Where this could help
Creative tools
Navigate hidden panels, hover-activated controls, and overlapping elements in creative workflows.
Workplace automation
Handle menus, pop-ups, and loading states in everyday applications and remote desktops.
Agent evaluation
Identify interaction failures and track performance across models and updates.
Accessibility research
Study how interactive agents can assist users with hard-to-find or difficult-to-access controls.
Let’s grab coffee! ☕
Attending EMNLP 2026 in Budapest?
If you’re working on GUI agents, human-centered AI, accessible interactions, or cultural and individual differences in human-AI interaction, we’d love to connect and exchange ideas!
Always happy to chat over coffee. ☕
ella94.ai@gmail.comCitation
@inproceedings{kim2026igg,
title = {{IGG}: A Benchmark for Interactive {GUI} Grounding under Visibility Constraints},
author = {Kim, Kyeong Seon and Son, Jiyeon and Oh, Tae-Hyun},
booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing},
year = {2026}
}