Visual Question Answering Datasets

WorldBench is a new benchmark designed to evaluate the physical understanding and prediction of modern world models and vision-language models. There are two components: The video based benchmark can be found in /scenes. There are 4 high-level categories for different physics concepts being tested. Within each, there are 3-5 scenes each with 25-50 variations. The text based benchmark is in /textualquestions. There are 4 JSON files, one per category. Code to run the evaluation for this benchmark along with instructions can be found here: https://drive.google.com/file/d/1TNHfV-mKiidl1eFWJyctBOodWJnCajA/view?usp=sharing

TraceSpatial-Trace is the RGB-and-QA tracing subset of TraceSpatial, introduced by the RoboTracer project. It supports learning to translate language instructions into spatial waypoints for object manipulation and robot end-effector motion. This release contains 517,215 referenced RGB images and 3,623,880 question–answer pairs, extracted from 1,067,822 original conversation records across AgiBot, CA-1M, DROID, RoboTwin, and ScanNet. It combines reconstructed indoor scenes, real-world manipulation data, and simulated robot interactions. These are the counts of this released subset, rather than the approximately 30 million QA pairs in the broader TraceSpatial collection described in the…

Dataset · Visual question answering

ViFailback-Dataset

SII-RHOS

ViFailback Dataset: Real-World Robotic Manipulation Failure Dataset with Visual Symbol Guidance A real-world dataset for diagnosing, correcting, and learning from robotic manipulation failures via visual symbols. ViFailback is a large-scale, real-world robotic manipulation failure dataset introduced in the CVPR 2026 paper "Diagnose, Correct, and Learn from Manipulation Failures via Visual Symbols". It introduces visual

This repository is a provenance-preserving collection of spatial measurement questions for answer-supervised training. It is built from VSI-590K, SpaceVista-Full, HiSpatial-Data, and CA-VQA. SenseNova-SI-8M is intentionally out of scope for this release. The first deliverable is the complete master collection. Smaller and larger training views will be derived only after all eligible metric examples have been retained and audited; they are not early sampling quotas. - annotations/ /measurement.parquet: canonical question/answer rows. - manifests/ /mediainventory.jsonl: unique required media and usage. - audits/ /: selection counts, validation, checksums, and source policy. - media/ /.tar…

Dataset · Visual question answering

SurgVLM-Video-Benchmark

Dong Yang

Held-out video eval set for SurgVLM-style surgical video tasks. Each QA row is paired with an MP4 clip (the listed frames encoded in order). - 36 tasks, 200 QA rows each = 7,200 rows - 6,512 MP4 clips (rows can share a clip) videopath is relative to this repo root, e.g. media/GraSP/GraSPCASE041002476-002485n10.mp4. Clip identity is mediauid (SHA-256 of {imagepaths, clipdurationsec, fps}). Multiple QA rows can share one clip. If you hit Xet Runtime Error / Task Panic, retry with HFHUBDISABLEXET=1.

Dataset · Visual question answering

advspot-public

inclusionAI

AdvSpot is the first grounded adversarial OCR benchmark, introduced in the paper ArmorOCR: Grounded Adversarial Visual Perception via Observation-Transferred Self-Distillation. It targets visual text patterns that are readable to humans but challenging for multimodal large language models (MLLMs) to localize and recognize, and evaluates models with region-level annotations: bounding boxes, transcriptions, perception-type labels, and region-grounded VQA pairs. AdvSpot formulates adversarial OCR as a grounded OCR perception task. For each image, one or more regions are annotated with a bounding box, a transcription, one or more perception-type labels, and a region-grounded VQA pair whose…

The frozen evaluation sets for the PixelProof paper. Each eligible replay-verified world contributes exactly five deterministic, pixel-distinct instances. The five instances of a world share its selected question text while varying the rendered scene and the gold answer, which the world's inverse program recovers from the image alone. The config names, the section column, and the ledgers/section files are holds spatially-steered worlds. The exclusions are the worlds that passed the 200-scene replay (1,301 profile-steered and 875 spatially-steered) but produced fewer than five pixel-distinct scenes; each one is documented in the corresponding ledgers/sectionexclusions.csv file. Each row…

Each Excel workbook contains 75 objects: five objects from each of the 15 region × object type strata. Each row includes an embedded primary/front-view image immediately before the reference title. The two copies for a group contain the same objects, references, and model outputs. Only the annotator column differs. 每份 Excel 工作簿包含 75 件藏品，即 15 个 region × object type 类别各 5 件。每行在 reference title 前嵌入一张藏品的主要／正面图片。同一组的两份文件包含完全相同的藏品、参考标签和模型输出，仅末列 annotator 不同。 1. Annotate the five fields independently: title, culture, period, origin, and creator. 2. For each model and field, enter exactly 1 or 0 in the adjacent match0or1 column. - 1: the model output is fully or broadly consistent with at least one…

DVBench is a benchmark for evaluating multimodal large language models on data videos, a storytelling medium that combines dynamic charts with structured narratives. - 300 data videos - 1,000 human-verified question-answer pairs - questionid: unique question identifier - questiontype: EM, MCQsingle, MCQmultiple, or Openended - video: video identifier matching videos/.mp4 - dimension: evaluation dimension - question: question or subtitle-cloze context - answer: reference answer - distractor1, distractor2, distractor3: distractor options for single-choice and multiple-choice questions - charttype: chart category for visual questions in Animation, Chart Perception, and Chart Reasoning…

Dataset · Visual question answering

MMOrgBench

MMOrgBench

1,801 omni-view questions over 32 synthetic workplace projects. This is the portable evaluation corpus, not the original source-paper collection. PDF/PPT content is supplied as rendered page images; native Slack/Notion messages remain text. No OCR or generated image captions are supplied to readers. Pin a Hub commit SHA with revision=" " for reproducible experiments. The QA dataset contains question, answer, task, reviewed evidence, solution and annotationjson with the full review record. Gold and solution are evaluation Download all actual images and the corpus for evaluation: corpus/units.json has repository-relative image paths. Every image is included in media/ with a content-hash…

Dataset · Visual question answering

MovieColorBench

Logan Bolton

Can a model recognize a famous video from just its colors? Each image is a barcode of one video. Every stripe is the average color of a few seconds, in order from start to end. The model picks which video it is out of 10 options, so guessing gets 10%. The wrong options are picked to be plausible. Same genre, close in year, similar colorfulness. The code is at https://github.com/LoganBolton/MovieColorBench. Any OpenRouter model runs with one command. Only barcodes are shared here, no frames from the videos.

Dataset · Visual question answering

vl-checklist

Tri Lai

This repository hosts the Parquet edition of VL-CheckList (EMNLP 2022). - imageshake2019: Sharded image pool containing binary image bytes. - relation, location, size: Evaluation test cases with POS/NEG prompt pairs.

Textbook Cover VQA is a Visual Question Answering (VQA) dataset developed for research on structured understanding of text-rich academic textbook covers. The dataset is designed to evaluate vision–language models (VLMs) on textual understanding, academic-domain recognition, spatial grounding, and visual content understanding. Textbook Cover VQA consists of 1,510 textbook-cover images covering 9 academic domains. Each textbook-cover image is associated with 8 predefined VQA questions, resulting in 12,080 question-answer annotations. The dataset captures four major types of information: - B. Vinay Kumar — L24AEC592 - A. Akash — Y23AEC404 - A. Pujitha — Y23AEC410 - B. Avinash — Y23AEC413…