Open-weight model · Object detection
deformable-detr-DocLayNet
by Aryn Inc. Aryn/deformable-detr-DocLayNet
Deformable DEtection TRansformer (DETR), trained on DocLayNet (including 80k annotated pages in 11 classes). You can use this model in the serverless Aryn Partitioning Service.
Runs On
What it takes to serve deformable-detr-DocLayNet (41M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.1 GB | 0.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.
Model Card
By Aryn Inc., published under apache-2.0, revision d5503a90ae08.
Deformable DEtection TRansformer (DETR), trained on DocLayNet (including 80k annotated pages in 11 classes). You can use this model in the serverless Aryn Partitioning Service. You can get started here The DETR model is an encoder-decoder transformer with a convolutional backbone. Two heads are added on top of the decoder outputs in order to perform object detection: a linear layer for the class labels and a MLP (multi-layer perceptron) for the bounding boxes. The model uses so-called object queries to detect objects in an image. Each object query looks for a particular object in the image. For COCO, the number of object queries is set to 100. The model is trained using a "bipartite…
Read Aryn Inc.'s full model card
Deformable DETR model trained on DocLayNet
Deformable DEtection TRansformer (DETR), trained on DocLayNet (including 80k annotated pages in 11 classes).
You can use this model in the serverless Aryn Partitioning Service. You can get started here
Model description
The DETR model is an encoder-decoder transformer with a convolutional backbone. Two heads are added on top of the decoder outputs in order to perform object detection: a linear layer for the class labels and a MLP (multi-layer perceptron) for the bounding boxes. The model uses so-called object queries to detect objects in an image. Each object query looks for a particular object in the image. For COCO, the number of object queries is set to 100.
The model is trained using a "bipartite matching loss": one compares the predicted classes + bounding boxes of each of the N = 100 object queries to the ground truth annotations, padded up to the same length N (so if an image only contains 4 objects, 96 annotations will just have a "no object" as class and "no bounding box" as bounding box). The Hungarian matching algorithm is used to create an optimal one-to-one mapping between each of the N queries and each of the N annotations. Next, standard cross-entropy (for the classes) and a linear combination of the L1 and generalized IoU loss (for the bounding boxes) are used to optimize the parameters of the model.
Intended uses & limitations
You can use the raw model for object detection. See the model hub to look for all available Deformable DETR models.
How to use
Here is how to use this model:
from transformers import AutoImageProcessor, DeformableDetrForObjectDetection
import torch
from PIL import Image
import requests
url = "https://huggingface.co/Aryn/deformable-detr-DocLayNet/resolve/main/examples/doclaynet_example_1.png"
image = Image.open(requests.get(url, stream=True).raw)
processor = AutoImageProcessor.from_pretrained("Aryn/deformable-detr-DocLayNet")
model = DeformableDetrForObjectDetection.from_pretrained("Aryn/deformable-detr-DocLayNet")
inputs = processor(images=image, return_tensors="pt")
outputs = model(**inputs)
# convert outputs (bounding boxes and class logits) to COCO API
# let's only keep detections with score > 0.7
target_sizes = torch.tensor([image.size[::-1]])
results = processor.post_process_object_detection(outputs, target_sizes=target_sizes, threshold=0.7)[0]
for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
box = [round(i, 2) for i in box.tolist()]
print(
f"Detected {model.config.id2label[label.item()]} with confidence "
f"{round(score.item(), 3)} at location {box}"
)
Evaluation results
This model achieves 57.1 box mAP on DocLayNet.
Training data
The Deformable DETR model was trained on DocLayNet. It was introduced in the paper DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis by Pfitzmann et al. and first released in this repository.
BibTeX entry and citation info
@misc{https://doi.org/10.48550/arxiv.2010.04159,
doi = {10.48550/ARXIV.2010.04159},
url = {https://arxiv.org/abs/2010.04159},
author = {Zhu, Xizhou and Su, Weijie and Lu, Lewei and Li, Bin and Wang, Xiaogang and Dai, Jifeng},
keywords = {Computer Vision and Pattern Recognition (cs.CV), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Deformable DETR: Deformable Transformers for End-to-End Object Detection},
publisher = {arXiv},
year = {2020},
copyright = {arXiv.org perpetual, non-exclusive license}
}
Configuration
- Architecture
- DeformableDetrForObjectDetection
- Context length (tokens)
- 1,024
- Stored precision
- float32
- Model type
- deformable_detr
Identity and Version
- Repository
- Aryn/deformable-detr-DocLayNet
- Publisher
- Aryn Inc.
- Task
- Object detection
- Modality
- Image
- Library
- transformers
- Parameters
- 41M parameters
- Languages
- Not stated by the source
- Revision
- d5503a90ae08dd43565de6984a5dd7924cad2400
- First published
- 2024-03-19
- Last updated
- 2025-08-08
Files and Weights
8 files, 165.8 MB in total. The weights are 1 file totalling 164.7 MB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 164.7 MB | e3861d34685d |
| config.json | Configuration | 1.8 KB | — |
| preprocessor_config.json | Configuration | 300 B | — |
| README.md | Documentation | 4.6 KB | — |
| examples/doclaynet_example_1.png | Other | 403.1 KB | — |
| examples/doclaynet_example_2.png | Other | 187.7 KB | — |
| examples/doclaynet_example_3.png | Other | 527.2 KB | — |
| .gitattributes | Repository | 1.5 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 164.7 MB
Released by Aryn Inc. through its official repository on Hugging Face. Read the license.
Built From
- Described by arXiv:2010.04159
- Described by arXiv:2206.01062
- Trained on (disclosed) ds4sd/DocLayNet
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 164.7 MB |
| 16-bit | 0.1 GB |
| 8-bit | 0.0 GB |
| 4-bit | 0.0 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About deformable-detr-DocLayNet
How much GPU memory does deformable-detr-DocLayNet need?
About 0.1 GB at 16-bit and 0 GB at 4-bit: the weights (41M parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run deformable-detr-DocLayNet on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use deformable-detr-DocLayNet commercially?
Yes. deformable-detr-DocLayNet is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is deformable-detr-DocLayNet's context length?
1,024 tokens, from the maximum position embeddings in its published configuration.
Similar Models
detr-doc-table-detection is a model trained to detect both Bordered and Borderless tables in documents, based on facebook/detr-resnet-50. This model can be used for the task of object detection. The model should not be used to intentionally create hostile or alienating environments for people. Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)). Predictions generated by the model may include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups. Users (both direct and downstream) should be made aware of the risks, biases and…
DEtection TRansformer (DETR) model trained end-to-end on COCO 2017 object detection (118k annotated images). It was introduced in the paper End-to-End Object Detection with Transformers by Carion et al. and first released in this repository. Disclaimer: The team releasing DETR did not write a model card for this model so this model card has been written by the Hugging Face team. The DETR model is an encoder-decoder transformer with a convolutional backbone. Two heads are added on top of the decoder outputs in order to perform object detection: a linear layer for the class labels and a MLP (multi-layer perceptron) for the bounding boxes. The model uses so-called object queries to detect…
Deformable DEtection TRansformer (DETR) model trained end-to-end on COCO 2017 object detection (118k annotated images). It was introduced in the paper Deformable DETR: Deformable Transformers for End-to-End Object Detection by Zhu et al. and first released in this repository. Disclaimer: The team releasing Deformable DETR did not write a model card for this model so this model card has been written by the Hugging Face team. The DETR model is an encoder-decoder transformer with a convolutional backbone. Two heads are added on top of the decoder outputs in order to perform object detection: a linear layer for the class labels and a MLP (multi-layer perceptron) for the bounding boxes. The…
However, we observe that the speed and accuracy of YOLOs are negatively affected by the NMS. Recently, end-to-end Transformer-based detectors (DETRs) have provided an alternative to eliminating NMS. Nevertheless, the high computational cost limits their practicality and hinders them from fully exploiting the advantage of excluding NMS. In this paper, we propose the Real-Time DEtection TRansformer (RT-DETR), the first real-time end-to-end object detector to our best knowledge that addresses the above dilemma. We build RT-DETR in two steps, drawing on the advanced DETR: first we focus on maintaining accuracy while improving speed, followed by maintaining speed while improving accuracy.…
However, we observe that the speed and accuracy of YOLOs are negatively affected by the NMS. Recently, end-to-end Transformer-based detectors (DETRs) have provided an alternative to eliminating NMS. Nevertheless, the high computational cost limits their practicality and hinders them from fully exploiting the advantage of excluding NMS. In this paper, we propose the Real-Time DEtection TRansformer (RT-DETR), the first real-time end-to-end object detector to our best knowledge that addresses the above dilemma. We build RT-DETR in two steps, drawing on the advanced DETR: first we focus on maintaining accuracy while improving speed, followed by maintaining speed while improving accuracy.…