ReferTrack: Referring Then Tracking for Embodied Visual Tracking

Hanjing Ye^1,2 Tianle Zeng¹ Jiazhao Zhang³ Shaoan Wang³ Zibo Zhang⁴
Weisi Situ¹ Yuchen Zhou² Yonggen Ling^2,4* Hong Zhang^1*

¹RCV Laboratory, SUSTech ²Tencent Robotics X
³Peking University ⁴Futian Laboratory

Project Page | arXiv | Video

## Overview **_ReferTrack_** is a *referring-then-tracking* paradigm for embodied visual tracking that first grounds a language-described target to an image-space bounding box and then decodes tracking waypoints from this decision, using temporal-viewpoint-bbox indicator (TVBI) tokens to inject previously selected bounding boxes into the visual history and preserve target motion cues over time, achieving state-of-the-art single-view performance on EVT-Bench with robust sim-to-real transfer to legged and humanoid robots.

ReferTrack method overview

## TODO List * [ ] Release model checkpoints and evaluation code. * [ ] Release the dataset. * [ ] Release the training code. * [ ] Release the data engine.