# Visual Question Answering Datasets: AI Datasets
Source: https://savrn.com/datasets/tasks/visual-question-answering
Markdown alternate of the page above; the site index is https://savrn.com/llms.txt

---

SAVRN Model Hub · Datasets by Task

# Visual Question Answering Datasets

16 open-weight visual question answering datasets in the SAVRN Model Hub, with Yuandong Yang, Dong Yang and WorldBench publishing the most.

16Datasets

16Publishers

6Licenses

## Most Downloaded

| Dataset | Publisher | License | Monthly downloads |
| --- | --- | --- | --- |
| [ViFailback-Dataset](https://savrn.com/datasets/vifailback-dataset) | SII-RHOS | mit | 163.5k |
| [IntuitivePhysics](https://savrn.com/datasets/intuitivephysics) | WorldBench | Not stated | 94.5k |
| [TraceSpatial-Trace](https://savrn.com/datasets/tracespatial-trace) | VCG具身智能实验室 | apache-2.0 | 4.8k |
| [TrainingData_Stage3](https://savrn.com/datasets/trainingdata-stage3) | AnchorSR | other | 3.2k |
| [OneStreamer-1M](https://savrn.com/datasets/onestreamer-1m) | Yuandong Yang | apache-2.0 | 860 |
| [SurgVLM-Video-Benchmark](https://savrn.com/datasets/surgvlm-video-benchmark) | Dong Yang | Not stated | 85 |
| [advspot-public](https://savrn.com/datasets/advspot-public) | inclusionAI | apache-2.0 | 13 |
| [PixelProof-Evaluation](https://savrn.com/datasets/pixelproof-evaluation) | Project PixelProof | Not stated | 6 |
| [Game2Real-Bench](https://savrn.com/datasets/game2real-bench) | Liuqingan | other | 3 |
| [Appear2Meaning](https://savrn.com/datasets/appear2meaning) | AI4Museum | Not stated | — |

## Licenses

| License | Datasets | Commercial use |
| --- | --- | --- |
| not stated | 5 | Not stated |
| other | 3 | Read the license |
| apache-2.0 | 3 | Yes |
| mit | 2 | Yes |
| cc-by-4.0 | 2 | Yes |
| cc-by-nc-sa-4.0 | 1 | Not without separate permission |

## Who Publishes Them

| Publisher | Datasets |
| --- | --- |
| [Yuandong Yang](https://savrn.com/model-publishers/yyd7) | 1 |
| [Dong Yang](https://savrn.com/model-publishers/yangd05) | 1 |
| [WorldBench](https://savrn.com/model-publishers/worldbenchmark) | 1 |
| [Tri Lai](https://savrn.com/model-publishers/w4nn4b3m4st3r) | 1 |
| [VCG具身智能实验室](https://savrn.com/model-publishers/vcg-eai) | 1 |
| [SII-RHOS](https://savrn.com/model-publishers/sii-rhos-ai) | 1 |

## All 16 Datasets

Dataset · Visual question answering

### [IntuitivePhysics](https://savrn.com/datasets/intuitivephysics)

[WorldBench](https://savrn.com/model-publishers/worldbenchmark)

WorldBench is a new benchmark designed to evaluate the physical understanding and prediction of modern world models and vision-language models. There are two components: The video based benchmark can be found in /scenes. There are 4 high-level categories for different physics concepts being tested. Within each, there are 3-5 scenes each with 25-50 variations. The text based benchmark is in /textualquestions. There are 4 JSON files, one per category. Code to run the evaluation for this benchmark along with instructions can be found here: https://drive.google.com/file/d/1TNHfV-mKiidl1eFWJyctBOodWJnCajA/view?usp=sharing

Publicly accessible n<1K

[View dataset](https://savrn.com/datasets/intuitivephysics)

Dataset · Visual question answering

### [TraceSpatial-Trace](https://savrn.com/datasets/tracespatial-trace)

[VCG具身智能实验室](https://savrn.com/model-publishers/vcg-eai)

TraceSpatial-Trace is the RGB-and-QA tracing subset of TraceSpatial, introduced by the RoboTracer project. It supports learning to translate language instructions into spatial waypoints for object manipulation and robot end-effector motion. This release contains 517,215 referenced RGB images and 3,623,880 question–answer pairs, extracted from 1,067,822 original conversation records across AgiBot, CA-1M, DROID, RoboTwin, and ScanNet. It combines reconstructed indoor scenes, real-world manipulation data, and simulated robot interactions. These are the counts of this released subset, rather than the approximately 30 million QA pairs in the broader TraceSpatial collection described in the…

Publicly accessible apache-2.0

[View dataset](https://savrn.com/datasets/tracespatial-trace)

Dataset · Visual question answering

### [ViFailback-Dataset](https://savrn.com/datasets/vifailback-dataset)

[SII-RHOS](https://savrn.com/model-publishers/sii-rhos-ai)

A real-world dataset for diagnosing, correcting, and learning from robotic manipulation failures via visual symbols. ViFailback is a large-scale, real-world robotic manipulation failure dataset introduced in the CVPR 2026 paper "Diagnose, Correct, and Learn from Manipulation Failures via Visual Symbols". It introduces visual symbols as an efficient interface for failure diagnosis and recovery-oriented learning. A ready-to-use LeRobot v2.1 conversion is available for robot learning and failure-aware policy training. 5,202 real-world manipulation trajectories. 58,128 high-quality VQA pairs. 100 distinct manipulation tasks. 4 major failure categories. Features ViFailback-Bench (Lite & Hard)…

Publicly accessible mit

[View dataset](https://savrn.com/datasets/vifailback-dataset)

Dataset · Visual question answering

### [TrainingData_Stage3](https://savrn.com/datasets/trainingdata-stage3)

[AnchorSR](https://savrn.com/model-publishers/anchorsr)

This repository is a provenance-preserving collection of spatial measurement questions for answer-supervised training. It is built from VSI-590K, SpaceVista-Full, HiSpatial-Data, and CA-VQA. SenseNova-SI-8M is intentionally out of scope for this release. The first deliverable is the complete master collection. Smaller and larger training views will be derived only after all eligible metric examples have been retained and audited; they are not early sampling quotas. - annotations/ /measurement.parquet: canonical question/answer rows. - manifests/ /mediainventory.jsonl: unique required media and usage. - audits/ /: selection counts, validation, checksums, and source policy. - media/ /.tar…

Publicly accessible other

[View dataset](https://savrn.com/datasets/trainingdata-stage3)

Dataset · Visual question answering

### [OneStreamer-1M](https://savrn.com/datasets/onestreamer-1m)

[Yuandong Yang](https://savrn.com/model-publishers/yyd7)

Adapted from the paper: task taxonomy on the left; streaming caption and QA construction pipelines on the right. Task coverage A training dataset for perception, memory, and proactive response in streaming video interaction, introduced in OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction. The curated collections include Streamo, JoyAI-VL-Interaction (JoyVL), MMDuet2, ViSpeak, StreamForest, VideoChat-OL, Seeker, and VST, alongside OneStreamer-Core. The appendix groups upstream sources by collection and task; a source list in the table describes that group rather than assigning every record to all listed sources. Proactive Caption Memory Proactive…

Publicly accessible apache-2.0

[View dataset](https://savrn.com/datasets/onestreamer-1m)

Dataset · Visual question answering

### [SurgVLM-Video-Benchmark](https://savrn.com/datasets/surgvlm-video-benchmark)

[Dong Yang](https://savrn.com/model-publishers/yangd05)

Held-out video eval set for SurgVLM-style surgical video tasks. Each QA row is paired with an MP4 clip (the listed frames encoded in order). - 36 tasks, 200 QA rows each = 7,200 rows - 6,512 MP4 clips (rows can share a clip) videopath is relative to this repo root, e.g. media/GraSP/GraSPCASE041002476-002485n10.mp4. Clip identity is mediauid (SHA-256 of {imagepaths, clipdurationsec, fps}). Multiple QA rows can share one clip. If you hit Xet Runtime Error / Task Panic, retry with HFHUBDISABLEXET=1.

Access requested at publisher 1K<n<10K

[View dataset](https://savrn.com/datasets/surgvlm-video-benchmark)

Dataset · Visual question answering

### [advspot-public](https://savrn.com/datasets/advspot-public)

[inclusionAI](https://savrn.com/model-publishers/inclusionai)

AdvSpot is the first grounded adversarial OCR benchmark, introduced in the paper ArmorOCR: Grounded Adversarial Visual Perception via Observation-Transferred Self-Distillation. It targets visual text patterns that are readable to humans but challenging for multimodal large language models (MLLMs) to localize and recognize, and evaluates models with region-level annotations: bounding boxes, transcriptions, perception-type labels, and region-grounded VQA pairs. AdvSpot formulates adversarial OCR as a grounded OCR perception task. For each image, one or more regions are annotated with a bounding box, a transcription, one or more perception-type labels, and a region-grounded VQA pair whose…

Publicly accessible apache-2.0

[View dataset](https://savrn.com/datasets/advspot-public)

Dataset · Visual question answering

### [PixelProof-Evaluation](https://savrn.com/datasets/pixelproof-evaluation)

[Project PixelProof](https://savrn.com/model-publishers/pixelproof)

The frozen evaluation sets for the PixelProof paper. Each eligible replay-verified world contributes exactly five deterministic, pixel-distinct instances. The five instances of a world share its selected question text while varying the rendered scene and the gold answer, which the world's inverse program recovers from the image alone. The config names, the section column, and the ledgers/section files are holds spatially-steered worlds. The exclusions are the worlds that passed the 200-scene replay (1,301 profile-steered and 875 spatially-steered) but produced fewer than five pixel-distinct scenes; each one is documented in the corresponding ledgers/sectionexclusions.csv file. Each row…

Publicly accessible

[View dataset](https://savrn.com/datasets/pixelproof-evaluation)

Dataset · Visual question answering

### [Game2Real-Bench](https://savrn.com/datasets/game2real-bench)

[Liuqingan](https://savrn.com/model-publishers/liuqa)

Game2Real-Bench provides a 1,400-sample multimodal evaluation benchmark and a 23,893-sample game-reasoning training dataset. The two collections are kept - benchmark/ contains the official Toy Game and Real World evaluation data. - dataset/ contains the training data and its images. The benchmark aligns controlled visual games with real-world tasks across four Constraint Reasoning, and Sequential Planning. Images are stored separately from the tables. Every image path is relative to the repository root and uses POSIX notation. Preserve the order listed in imagepath or public.questionimages when constructing multimodal inputs. To download image assets with the Hub client: Evaluation code is…

Publicly accessible other

[View dataset](https://savrn.com/datasets/game2real-bench)

Dataset · Visual question answering

### [Appear2Meaning](https://savrn.com/datasets/appear2meaning)

[AI4Museum](https://savrn.com/model-publishers/carolyn-jiang)

Each Excel workbook contains 75 objects: five objects from each of the 15 region × object type strata. Each row includes an embedded primary/front-view image immediately before the reference title. The two copies for a group contain the same objects, references, and model outputs. Only the annotator column differs. 每份 Excel 工作簿包含 75 件藏品，即 15 个 region × object type 类别各 5 件。每行在 reference title 前嵌入一张藏品的主要／正面图片。同一组的两份文件包含完全相同的藏品、参考标签和模型输出，仅末列 annotator 不同。 1. Annotate the five fields independently: title, culture, period, origin, and creator. 2. For each model and field, enter exactly 1 or 0 in the adjacent match0or1 column. - 1: the model output is fully or broadly consistent with at least one…

Publicly accessible

[View dataset](https://savrn.com/datasets/appear2meaning)

Dataset · Visual question answering

### [DVBench](https://savrn.com/datasets/dvbench)

[Bomiao Wang](https://savrn.com/model-publishers/bomiaowang)

DVBench is a benchmark for evaluating multimodal large language models on data videos, a storytelling medium that combines dynamic charts with structured narratives. - 300 data videos - 1,000 human-verified question-answer pairs - questionid: unique question identifier - questiontype: EM, MCQsingle, MCQmultiple, or Openended - video: video identifier matching videos/.mp4 - dimension: evaluation dimension - question: question or subtitle-cloze context - answer: reference answer - distractor1, distractor2, distractor3: distractor options for single-choice and multiple-choice questions - charttype: chart category for visual questions in Animation, Chart Perception, and Chart Reasoning…

Publicly accessible cc-by-nc-sa-4.0

[View dataset](https://savrn.com/datasets/dvbench)

Dataset · Visual question answering

### [MMOrgBench](https://savrn.com/datasets/mmorgbench)

[MMEnterprise-Bench](https://savrn.com/model-publishers/mmorgbench)

1,801 omni-view questions over 32 synthetic workplace projects. This is the portable evaluation corpus, not the original source-paper collection. PDF/PPT content is supplied as rendered page images; native Slack/Notion messages remain text. No OCR or generated image captions are supplied to readers. Pin a Hub commit SHA with revision=" " for reproducible experiments. The QA dataset contains question, answer, task, reviewed evidence, solution and annotationjson with the full review record. Gold and solution are evaluation Download all actual images and the corpus for evaluation: corpus/units.json has repository-relative image paths. Every image is included in media/ with a content-hash…

Publicly accessible

[View dataset](https://savrn.com/datasets/mmorgbench)

Dataset · Visual question answering

### [MovieColorBench](https://savrn.com/datasets/moviecolorbench)

[Logan Bolton](https://savrn.com/model-publishers/loganbolton)

Can a model recognize a famous video from just its colors? Each image is a barcode of one video. Every stripe is the average color of a few seconds, in order from start to end. The model picks which video it is out of 10 options, so guessing gets 10%. The wrong options are picked to be plausible. Same genre, close in year, similar colorfulness. The code is at https://github.com/LoganBolton/MovieColorBench. Any OpenRouter model runs with one command. Only barcodes are shared here, no frames from the videos.

Publicly accessible mit n<1K

[View dataset](https://savrn.com/datasets/moviecolorbench)

Dataset · Visual question answering

### [Video-XL-Pro-Union](https://savrn.com/datasets/video-xl-pro-union)

[chi-cheng Chen](https://savrn.com/model-publishers/samchen567)

依據以下兩篇論文整理的 SFT 訓練資料，兩者取聯集；兩者重疊的資料集一律採用 Video-XL-Pro 版本的標註。 標註只收錄兩個官方 repo（MINT-SJTU/Video-XL-Pro-Training、sy1998/VideoXLTraining）中有的檔案，且只保留標註實際引用到的媒體。論文或 repo 中沒有的資料集都已移除。Video-XL 的 pretraining 資料（LAION-2M 子集）因品質不佳不收錄。 訓練時把 image folder / video folder 設為 media/ 即可。所有標註路徑都已用 tools/verifypaths.py 驗證過，缺檔數為 0。 - activitynetqa15k.json 改指向 LLaVA178k/ActivityNet-QA/...（與 LLaVA-Video-178K 為同一批影片）。 - finevideo43k.json：移除原始檔中 35 筆 conversations 為空的樣本（43,751 → 43,716）。 - vcg107k.json：359 部影片實際為.mkv /.webm，路徑副檔名已改成實際檔名（影響 6,122 筆樣本）。 - FineVideo、ScienceQA 的原始檔名被作者改成 uuid：FineVideo 用 QA 內容完整比對對回原影片；ScienceQA 用題目、答案文字加上 uuid1 時間順序對回原圖（順序為 test → train → validation，對齊無歧義）。對應方式見 tools/finevideomap.py。…

Publicly accessible other 1M<n<10M

[View dataset](https://savrn.com/datasets/video-xl-pro-union)

Dataset · Visual question answering

### [vl-checklist](https://savrn.com/datasets/vl-checklist)

[Tri Lai](https://savrn.com/model-publishers/w4nn4b3m4st3r)

This repository hosts the Parquet edition of VL-CheckList (EMNLP 2022). - imageshake2019: Sharded image pool containing binary image bytes. - relation, location, size: Evaluation test cases with POS/NEG prompt pairs.

Publicly accessible cc-by-4.0

[View dataset](https://savrn.com/datasets/vl-checklist)

Dataset · Visual question answering

### [vqacovers](https://savrn.com/datasets/vqacovers)

[B.VINAY KUMAR](https://savrn.com/model-publishers/bvinay06)

Textbook Cover VQA is a Visual Question Answering (VQA) dataset developed for research on structured understanding of text-rich academic textbook covers. The dataset is designed to evaluate vision–language models (VLMs) on textual understanding, academic-domain recognition, spatial grounding, and visual content understanding. Textbook Cover VQA consists of 1,510 textbook-cover images covering 9 academic domains. Each textbook-cover image is associated with 8 predefined VQA questions, resulting in 12,080 question-answer annotations. The dataset captures four major types of information: - B. Vinay Kumar — L24AEC592 - A. Akash — Y23AEC404 - A. Pujitha — Y23AEC410 - B. Avinash — Y23AEC413…

Publicly accessible cc-by-4.0 1K<n<10K

[View dataset](https://savrn.com/datasets/vqacovers)

## Other Tasks

- [Text Generation](https://savrn.com/datasets/tasks/text-generation) 140
- [Robotics](https://savrn.com/datasets/tasks/robotics) 120
- [Other](https://savrn.com/datasets/tasks/other) 50
- [Text Classification](https://savrn.com/datasets/tasks/text-classification) 50
- [Text Retrieval](https://savrn.com/datasets/tasks/text-retrieval) 44
- [Question Answering](https://savrn.com/datasets/tasks/question-answering) 39
- [Speech Recognition](https://savrn.com/datasets/tasks/speech-recognition) 34
- [Image to Text](https://savrn.com/datasets/tasks/image-to-text) 29
- [Time Series Forecasting](https://savrn.com/datasets/tasks/time-series-forecasting) 23
- [Tabular Classification](https://savrn.com/datasets/tasks/tabular-classification) 22
- [Image Classification](https://savrn.com/datasets/tasks/image-classification) 18
- [Object Detection](https://savrn.com/datasets/tasks/object-detection) 16

[See all](https://savrn.com/datasets/tasks)