“TDBench: Benchmarking Vision Language Models on Top-Down Image Understanding” was recently accepted to NeurIPS 2026 ED Track.
In this paper, we introduce TDBench, a benchmark for drone-altitude, top-down image understanding, a setting largely absent from the front-view data that vision-language models are trained and evaluated on. It spans ten task categories, from object recognition and counting to spatial reasoning and scene understanding. Alongside the benchmark, we propose RotationalEval (RE), an evaluation protocol that exploits rotational invariance: rotating a top-down image corresponds to a drone yaw change and leaves the scene unchanged, so a model is credited only when it answers consistently across all four orientations. RE exposes answers that come from chance or hallucination rather than genuine understanding, giving a more reliable measure of what VLMs actually see in top-down imagery. The benchmark is integrated into an open-source evaluation toolkit.
Please check out our paper: https://arxiv.org/abs/2504.03748
You must be logged in to post a comment.