SlimSAM-77 (uniform) for transformers.js — with box prompts
SlimSAM (nielsr/slimsam-77-uniform) in ONNX for
transformers.js, like
Xenova/slimsam-77-uniform — whose image
encoder files are reused here unchanged — but with a prompt encoder / mask decoder that
takes box prompts.
Xenova/slimsam-77-uniform's decoder declares no input_boxes, and transformers.js drops
inputs a session does not declare, so a box passed to it is silently ignored. This decoder
follows Meta's own SAM ONNX convention instead: a box is two extra points in input_points,
labelled 2 (top-left corner) and 3 (bottom-right corner). Inputs and outputs keep
transformers.js's names, so SamModel / SamProcessor drive it unchanged.
One rule the caller must follow: a prompt without a box carries a padding point
(0, 0) with label -1 (what Hugging Face's SamPromptEncoder adds internally when no
box is given). A prompt with a box has no pad.
// click: one positive point + the pad
input_points: [[[[x, y], [0, 0]]]], input_labels: [[[1, -1]]]
// box with a seed point: point, then the two corners
input_points: [[[[x, y], [x0, y0], [x1, y1]]]], input_labels: [[[1, 2, 3]]]
| file |
|
onnx/prompt_encoder_mask_decoder.onnx |
fp32 (WebGPU default) — matches PyTorch SamModel exactly on points, points + box |
onnx/prompt_encoder_mask_decoder_quantized.onnx |
dynamic uint8 (WASM default) — 99.2–99.96% mask agreement with fp32 |
onnx/vision_encoder*.onnx, config.json, preprocessor_config.json |
from Xenova/slimsam-77-uniform, unchanged |
Exported by export_decoder.py (in this repo), which also checks the export against
PyTorch. Apache-2.0, as the SlimSAM weights and the Xenova export it builds on.
