This is an NVFP4 quantized version of Qwen3-VL-8B-Instruct, a powerful vision-language model for multimodal understanding and generation tasks. The following modules were excluded from quantization to maintain model quality: - lmhead (language model head) - Visual encoder modules (model.visual.) - MLP gate projections (.mlp.gate$) For faster inference, you can use this model with vLLM: This quantized model maintains high quality for vision-language tasks while significantly reducing memory usage. The SmoothQuant technique helps preserve model accuracy during quantization. Typical quality degradation is 2-5% compared to the full-precision model. 1. Calibration: Used 512 samples from the…
NuMarkdown-8B-Thinking is the first reasoning OCR VLM. It is specifically trained to convert documents into clean Markdown files, well suited for RAG applications.
Parameters8.3B
Context128,000
Weights16.6 GB
Licensemit
AccessOpen weights
Monthly Downloads439.2k
Runs On
What it takes to serve NuMarkdown-8B-Thinking (8.3B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 16.6 GB | 19.9 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 8.3 GB | 10.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 4.1 GB | 5.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.
SAVRN's Notes on NuMarkdown-8B-Thinking
Budget the thinking tokens before you budget anything else. NuMind's model reads a document image, works out the layout, then writes Markdown, and the publisher puts the reasoning at 20 to 500 percent of the final output depending on the page. That number, not the 8.3 billion parameters, sets throughput for a scanning pipeline. Hardware is easy: 19.9 GB in 16-bit, 10.0 GB in 8-bit, 5.0 GB in 4-bit, and the cheapest listed setup is one MI300X with 192 GB at $1.85 an hour, which leaves most of the card for batching pages.
It is MIT, commercial use and redistribution included with the notices kept, and it is a fine-tune of Qwen/Qwen2.5-VL-7B-Instruct, so a buyer should read the base model's terms alongside. The 128,000-token context comes with a 32,768-token sliding window, which matters for long documents. It was released July 29, 2025 and last updated June 5, 2026.
Model Card
By NuMind, published under mit, revision 8dfc5236ae68.
API / Platform | Discord | GitHub | Demo
Reasoning comes to OCR
NuMarkdown-8B-Thinking is the first reasoning OCR VLM. It is specifically trained to convert documents into clean Markdown files, well suited for RAG applications. It generates thinking tokens to figure out the layout of the document before generating the Markdown file. It is particularly good at understanding documents with weird layouts and complex tables. The number of thinking tokens can vary from 20% to 500% of the final answer, depending on the task difficulty.
NuMarkdown-8B-Thinking is a fine-tune of Qwen 2.5-VL-7B on synthetic Doc → Reasoning → Markdown examples, followed by an RL phase (GRPO) with a layout-centric reward.
Try it out in the space!
Results
NuMarkdown-8B-Thinking is outperforming generic non-reasoning models like GPT-4o and specialized OCR models like OCRFlux. It is competitive against large reasoning closed-source models like Gemini 2.5.
Arena ranking against popular alternatives (using trueskill-2 ranking system, with around 500 model-anonymized votes):
Configuration
- Architecture
- Qwen2_5_VLForConditionalGeneration
- Context length (tokens)
- 128,000
- Layers
- 28
- Hidden size
- 3,584
- Feed-forward size
- 18,944
- Attention heads
- 28
- Key/value heads
- 4
- Vocabulary size
- 152,064
- Sliding window (tokens)
- 32,768
- RoPE base
- 1e+06
- Stored precision
- bfloat16
- Model type
- qwen2_5_vl
Identity and Version
- Repository
- numind/NuMarkdown-8B-Thinking
- Publisher
- NuMind
- Task
- Image to text
- Modality
- Image and text
- Library
- transformers
- Parameters
- 8.3B parameters
- Languages
- Not stated by the source
- Revision
- 8dfc5236ae68b799d86308aae2f9717fdb50286e
- First published
- 2025-07-29
- Last updated
- 2026-06-05
Files and Weights
21 files, 16.6 GB in total. The weights are 4 files totalling 16.6 GB in safetensors.
Weights4 files · 16.6 GB
Configuration7 files · 62.2 KB
Tokenizer4 files · 15.9 MB
Documentation1 file · 11.6 KB
Other4 files · 321.6 KB
Repository1 file · 1.6 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model-00001-of-00004.safetensors | Weights | 5.0 GB | de84fb5ed350 |
| model-00002-of-00004.safetensors | Weights | 5.0 GB | 285127f9ef61 |
| model-00003-of-00004.safetensors | Weights | 4.9 GB | e9c86dc639da |
| model-00004-of-00004.safetensors | Weights | 1.7 GB | dd6206837057 |
| added_tokens.json | Configuration | 605 B | — |
| chat_template.json | Configuration | 1.0 KB | — |
| config.json | Configuration | 1.5 KB | — |
| generation_config.json | Configuration | 282 B | — |
| model.safetensors.index.json | Configuration | 57.6 KB | — |
| preprocessor_config.json | Configuration | 576 B | — |
| special_tokens_map.json | Configuration | 613 B | — |
| README.md | Documentation | 11.6 KB | — |
| bar plot.png | Other | 85.0 KB | — |
| ex1.png | Other | 163.3 KB | 9ab65794a94e |
| matrix.png | Other | 63.4 KB | — |
| numind.svg | Other | 9.9 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| merges.txt | Tokenizer | 1.7 MB | — |
| tokenizer.json | Tokenizer | 11.4 MB | 9c5ae00e602b |
| tokenizer_config.json | Tokenizer | 5.9 KB | — |
| vocab.json | Tokenizer | 2.8 MB | — |
License and Download
- License
- mit
- Access
- Open weights, no gate
- Download size
- 16.6 GB
Download from NuMind
Released by NuMind through its official repository on Hugging Face. Read the license.
Built From
- Derived from Qwen/Qwen2.5-VL-7B-Instruct
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 16.6 GB |
| 16-bit | 16.6 GB |
| 8-bit | 8.3 GB |
| 4-bit | 4.1 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About NuMarkdown-8B-Thinking
How much GPU memory does NuMarkdown-8B-Thinking need?
About 19.9 GB at 16-bit and 5 GB at 4-bit: the weights (8.3B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run NuMarkdown-8B-Thinking on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use NuMarkdown-8B-Thinking commercially?
Yes. NuMarkdown-8B-Thinking is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.
What is NuMarkdown-8B-Thinking's context length?
128,000 tokens, from the maximum position embeddings in its published configuration.
Similar Models
NuExtract3 is a unified 4B vision-language reasoning model for document understanding. It combines strong structured information extraction with high-quality image-to-Markdown conversion, making it suitable for extraction pipelines, OCR, and RAG preprocessing for all types of documents such as scans, receipts, forms, invoices, contracts or tables. Try it out in the space! - Multilingual documents. - Reasoning and non-reasoning inference modes. - Template generation for structured extraction from natural language or input document. We benchmarked NuExtract on NuMind's internal structured benchmark, measuring model's performances on ~600 documents of diverse types including invoices, movie…
Granite-vision-3.3-2b is a compact and efficient vision-language model, specifically designed for visual document understanding, enabling automated content extraction from tables, charts, infographics, plots, diagrams, and more. Granite-vision-3.3-2b introduces several novel experimental features such as image segmentation, doctags generation, and multi-page support (see Experimental Capabilities for more details) and offers enhanced safety when compared to earlier Granite vision models. The model was trained on a meticulously curated instruction-following data, comprising diverse public and synthetic datasets tailored to support a wide range of document understanding and general image…
This Hub repository contains a HuggingFace's transformers implementation of the original Kosmos-2 model from Microsoft. Use the code below to get started with the model. This model is capable of performing different tasks through changing the prompts. First, let's define a function to run a prompt. Here are the tasks Kosmos-2 could perform: Once you have the entities, you can use the following helper function to draw their bounding bboxes on the image
Join our WeChat and Discord community Use GLM-OCR's API GLM-OCR is a multimodal OCR model for complex document understanding, built on the GLM-V encoder–decoder architecture. It introduces Multi-Token Prediction (MTP) loss and stable full-task reinforcement learning to improve training efficiency, recognition accuracy, and generalization. The model integrates the CogViT visual encoder pre-trained on large-scale image–text data, a lightweight cross-modal connector with efficient token downsampling, and a GLM-0.5B language decoder. Combined with a two-stage pipeline of layout analysis and parallel recognition based on PP-DocLayout-V3, GLM-OCR delivers robust and high-quality OCR performance…
Join our WeChat and Discord community Use GLM-OCR's API GLM-OCR is a multimodal OCR model for complex document understanding, built on the GLM-V encoder–decoder architecture. It introduces Multi-Token Prediction (MTP) loss and stable full-task reinforcement learning to improve training efficiency, recognition accuracy, and generalization. The model integrates the CogViT visual encoder pre-trained on large-scale image–text data, a lightweight cross-modal connector with efficient token downsampling, and a GLM-0.5B language decoder. Combined with a two-stage pipeline of layout analysis and parallel recognition based on PP-DocLayout-V3, GLM-OCR delivers robust and high-quality OCR performance…