Qwen3-ASR 1.7B for sherpa-onnx — 4096 context
A Windows x64 CPU package for offline transcription with sherpa-onnx. The decoder has a statically exported 4096-position KV cache. It uses a mixed INT8/FP32 decoder and retains the matching FP16 encoder and convolution frontend from the pinned sherpa-compatible ONNX export.
Model provenance
- Base model:
Qwen/Qwen3-ASR-1.7B, revision 7278e1e70fe206f11671096ffdd38061171dd6e5
- Official Qwen3-ASR implementation:
QwenLM/Qwen3-ASR, revision 7c6daf77a2421100f5fb066495372c00129d39ff
- sherpa-compatible ONNX source:
luigi000/qwen3-asr-1.7b-onnx-fp16, revision 90c70bc1dfce2456bc0bcdc286ffc2eaff2d7e8b
- Source conversion pipeline:
Wasser1462/Qwen3-ASR-onnx, revision 0829a577ee741e408e65763dfe01ff4fe75408e9
- Runtime used for validation: sherpa-onnx
1.13.8, ONNX Runtime CPU provider
- Encoder and convolution frontend: FP16 from the pinned ONNX source
- Decoder: dynamic INT8 weights for eligible MatMul/Gemm nodes; Q/K/V/O self-attention projections, normalization, logits, activations and KV cache remain FP32
max_total_len: 4096; tested max_new_tokens: 1024
The encoder is kept FP16 because the compatible INT8 encoder from a separate 512-context export substantially reduced Japanese recognition quality in a controlled comparison. On the same FLEURS Japanese sample, the mixed decoder with the FP16 encoder measured 8.2% character error rate (CER), compared with 5.5% for the all-FP16 source and 22.7% when using that separate INT8 encoder. This is one 22.62-second sample, not a broad accuracy guarantee.
Files and usage
manifest.json gives byte sizes and SHA-256 checksums for every model and tokenizer file. Keep each ONNX external-data file beside its graph: encoder.onnx.data beside encoder.onnx, and decoder.int8.onnx.data beside decoder.int8.onnx.
import sherpa_onnx
recognizer = sherpa_onnx.OfflineRecognizer.from_qwen3_asr(
conv_frontend="conv_frontend.onnx",
encoder="encoder.onnx",
decoder="decoder.int8.onnx",
tokenizer="tokenizer",
provider="cpu",
num_threads=4,
max_total_len=4096,
max_new_tokens=1024,
)
stream = recognizer.create_stream()
stream.accept_waveform(16000, audio_float32_mono)
recognizer.decode_stream(stream)
print(stream.result.text)
Optional per-stream Qwen3-ASR context:
stream.set_option("hotwords", "F5,BIG-IP,Kubernetes,OpenShift")
The system prompt, hotwords, audio tokens and generated output share the context budget. Long hotword lists can consume the available prompt space. sherpa-onnx reports when the prompt exceeds the configured cap or when generation reaches its token limit.
CPU and context behavior
The validated Japanese run used 8 CPU threads on a Windows x64 host. For the 22.62-second FLEURS sample, decode took 14.52 seconds (RTF 0.642). Process RSS peaked at about 5.57 GiB. Your speed and memory use depend on CPU, sherpa-onnx build, thread count, audio duration and concurrent streams.
The decoder has 28 layers, 8 KV heads and 128 dimensions per head. Its FP32 key/value cache uses about 896 MiB for one 4096-position stream. The cache is part of the exported decoder graph; lowering sherpa-onnx's runtime max_total_len limits usable context but does not shrink that fixed allocation. Increase the setting only up to the graph's 4096-token limit.
License and attribution
The Qwen base model and pinned ONNX source are published under Apache-2.0. LICENSE contains the Apache-2.0 text from the pinned official Qwen3-ASR source, and NOTICE records component provenance. This package does not include sherpa-onnx runtime binaries; install and license that runtime separately.
The public FLEURS Japanese test sample used for local validation is not included. FLEURS is distributed under CC-BY-4.0; see the Japanese test data and dataset card.
Reproducing the package
PowerShell workflows and immutable source revisions are in scripts/. From this directory:
.\scripts\setup_environment.ps1
.\scripts\export_qwen3_asr.ps1
.\scripts\quantize.ps1
.\scripts\package_release.ps1
The upload workflow requires a completed local validation record and checks that the target Hugging Face repository does not already exist. The download workflow fetches the public files over HTTPS, verifies every manifest checksum, and runs a sherpa-onnx CPU smoke recognition.
