🌍 GS-QA2 Leaderboard

Geospatial question answering over raster–vector data — 3,300 questions with executable PostGIS SQL ground truth.

📦 Benchmark 🗺️ Source data 💻 Code & evaluation

Metrics

Vector (V1–V28, 2,800 q): entity token-F1 ≥ 0.8 · location geodesic ≤ 5 m · direction ≤ 5° · numeric relative error ≤ 5 %.  Raster (R1–R11 & VR1–VR14, 500 q): strict accuracy with elevation ≤ 10 m, slope/aspect ≤ 5°. Avg = mean of the five headline metrics.

Submit your system

  1. Load the benchmark: load_dataset("Zoe/GS-QA2", <config>) and run your system.
  2. Write predictions as JSONL keyed by question id, and self-check with GS-QA/baselines/score_predictions.py from the QARV repo (format spec in SUBMISSIONS.md).
  3. Open a discussion in this Space's Community tab titled Submission: <system name> with your predictions file, score summary, and a short system description.

Maintainers re-score every submission before it appears here. Contact: zshan011@ucr.edu

GS-QA2: Shang, Elmahallawy, Al Nazi, Hristidis, Eldawy (UC Riverside, 2026) · vector questions inherited from GS-QA (Saeedan et al.). Results file: results.json