Real5-OmniDocBench: Dataset Card

Real5-OmniDocBench measures the robustness of document parsing systems under five physical acquisition conditions: Scanning, Warping, Screen-Photography, Illumination, and Skew. It reconstructs the same 1,355 pages from OmniDocBench v1.5 in every condition, producing 6,775 images in total. The one-to-one page correspondence and shared evaluation protocol isolate the effect of acquisition conditions from changes in document content.

News

• 2026-10-12: Added evaluation results for Xiaomi-OCR-0.
• 2026-09-12: Added evaluation results for GLM-5.3-Flash.
• 2026-08-27: Added evaluation results for NaviDC-OCR.

Leaderboard

The leaderboard reports performance over all five acquisition conditions. All metrics follow OmniDocBench: Overall↑, TextEdit↓, FormulaCDM↑, TableTEDS↑, and Reading OrderEdit↓.

Higher is better for ↑ metrics and lower is better for ↓ metrics. Best results in each column are shown in bold, and second-best results are underlined. Tables are sorted by Overall score in descending order.

1. Overall

Overall is the mean score across all five scenarios.

Benchmark Overview

Real5-OmniDocBench evaluates the same document content under controlled changes to the physical acquisition process. Except for the scanning subset, images were manually captured with handheld mobile devices.

This design provides:

• Controlled comparison: every scenario contains the same 1,355 source pages.
• Physical realism: acquisition artifacts are produced by real devices and environments rather than synthetic transformations.
• Protocol compatibility: page identities, annotations, prediction format, and metrics follow OmniDocBench v1.5.

Dataset

The five scenario directories each contain 1,355 images. The complete download is approximately 16 GB.

Real5-OmniDocBench/
├── Real5-OmniDocBench-Scanning/
├── Real5-OmniDocBench-Warping/
├── Real5-OmniDocBench-Screen-Photography/
├── Real5-OmniDocBench-Illumination/
└── Real5-OmniDocBench-Skew/

Install the current Hugging Face Hub CLI and download the complete dataset:

pip install -U huggingface_hub
hf download PaddlePaddle/Real5-OmniDocBench \
  --repo-type dataset \
  --local-dir ./Real5-OmniDocBench

To download a single condition, use a file pattern:

hf download PaddlePaddle/Real5-OmniDocBench \
  --repo-type dataset \
  --include "Real5-OmniDocBench-Warping/*" \
  --local-dir ./Real5-OmniDocBench

Download the matching OmniDocBench v1.5 ground-truth annotations separately:

hf download opendatalab/OmniDocBench OmniDocBench.json \
  --repo-type dataset \
  --revision v1_5 \
  --local-dir ./OmniDocBench-v1.5

Evaluation

Real5-OmniDocBench does not introduce a new prediction schema. It reuses the OmniDocBench v1.5 annotation format and evaluation pipeline so that performance can be compared across acquisition conditions.

1. Run the model independently on all 1,355 images in each scenario.
2. Export predictions in the OmniDocBench end-to-end parsing format.
3. Match each image to its corresponding OmniDocBench v1.5 ground-truth annotation.
4. Apply the same preprocessing, evaluator, and metric settings to every scenario.

Refer to the version-pinned OmniDocBench v1.5 evaluation guide for environment setup, prediction formats, and evaluation commands.

Metrics

Submit Results

Model results can appear in the Hugging Face Hub leaderboard through the Hub evaluation-results workflow. Add an evaluation result file under .eval_results/ in the model repository, set evaluation_framework: real5-omnidocbench, and follow the Hugging Face evaluation results documentation for the supported schema and submission process.

Citation

If you use Real5-OmniDocBench in your research, please cite the following paper. Please also cite OmniDocBench when using its annotations or evaluation pipeline.

@misc{zhou2026real5omnidocbench,
  title         = {Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wild},
  author        = {Changda Zhou and Ziyue Gao and Xueqing Wang and Tingquan Gao and Cheng Cui and Jing Tang and Yi Liu},
  year          = {2026},
  eprint        = {2603.04205},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  doi           = {10.48550/arXiv.2603.04205},
  url           = {https://arxiv.org/abs/2603.04205}
}

Acknowledgements

Real5-OmniDocBench is built on OmniDocBench v1.5 and adopts its annotations and evaluation protocol. We thank the OmniDocBench authors for making their benchmark and evaluation tools available to the community.

License

The Real5-OmniDocBench dataset and repository materials are released under the Apache License 2.0. This benchmark inherits annotations and source-page correspondence from OmniDocBench v1.5; use of those materials remains subject to the applicable OmniDocBench and source-document terms.