parakeet-tdt-ctc-110m-glade

Parakeet TDT–CTC 110M for Core AI

Glade is a native Swift library for on-device speech inference on Apple silicon. This repository supplies converted model assets and configuration for Glade's model-specific host runtime. Glade's runtime source is currently private.

Apple Neural Engine conversions of NVIDIA's Parakeet TDT–CTC 110M. Each Core AI profile supports TDT and CTC decoding from the same encoder, with native token and word timings. A portable Core ML graph bundle is included as a bonus, qualified on Mac, iPhone and Apple Watch.

The English model has a 17-layer, 512-wide FastConformer encoder, a one-layer 640-wide LSTM TDT predictor, and a separate CTC head. These conversions preserve the original weights without retraining. W8A16 compresses learned acoustic matrices; sensitive convolution outputs and relative-position projections retain FP16 precision. The TDT graph uses FP16; the optional CPU predictor uses the original FP32 weights.

Bundles

Download one complete directory, including its metadata and sidecars:

Fast favors throughput. Quality provides more context and had lower WER on the tested speech, at a substantial runtime cost. This is a result on one recording, not a broad accuracy ranking. Both retain full bidirectional attention within each chunk; longer recordings are processed through contiguous chunks without dropping samples. These chunk lengths are distribution policies, not the upstream model's maximum supported context. There is no VAD or silence removal in these policies.

Core AI requires a physical Apple device with macOS/iOS 27 and Core AI support. The Core ML graphs were qualified on macOS/iOS/watchOS 27.0.1. These are source .aimodel and .mlpackage assets, not architecture-specific compiled caches; first use includes device specialization. Core AI could not execute the model on the tested Watch release; Core ML CPU+ANE did.

Measured Core AI performance

JFK's “We choose to go to the Moon” speech: a 20-second excerpt and the complete 18 min 15 s recording (1,095.32 seconds). RTFx is audio seconds divided by inference seconds; higher is better. Release runtimes, three-run medians after preparation and a separate warmup. Timing includes bounded audio reading, chunking, frontend, acoustic predictions, decoding, native timing construction and ordered delivery. Model preparation and report serialization are excluded.

macOS 27.0.1 (26A434), iOS 27.0.1 (24A446). Each acoustic prediction contains one chunk; the runtime maintains up to four concurrent requests. This is concurrency, not a four-chunk encoder tensor batch. TDT uses CPU decoding for at most two chunks and a batched ANE predictor for larger recordings. CTC does not load the recurrent predictor.

The final Core AI packages reproduce their qualified Mac transcripts exactly. Fast phone timings are from the equivalent graphs before compiler source-location metadata was stripped; their graph signatures, operations and weights were preserved, and the stripped packages were remeasured on Mac. Quality was measured from the final package on both devices. Every timed repetition reproduced its own tokens and retained every sample. Quality transcripts differ slightly between Mac and iPhone; normalized WER on this recording was:

WER uses the benchmark's reference transcript, Whisper English normalization and ordinary word edit distance (2,220 reference words). It includes chunk-boundary errors. Native timings are finite, ordered and bounded by the audio, but have not been scored against human-annotated word boundaries.

Complete iPhone traces captured ANE execution in both profiles with zero target-process GPU intervals. Placement demonstrates execution, not arithmetic utilization, memory-bandwidth utilization or energy efficiency. Short runs are particularly sensitive to scheduling overhead.

Preparation

Preparation includes loading and any specialization requested by Apple's backend. These are observed first preparations, not controlled cache-cleared fresh-install guarantees. The underlying compiler cache is opaque; cache state, software and hardware can change the result.

The Mac Core AI first observations combine separately measured acoustic and TDT predictor preparation components. A later first load of the packaged Fast assets took 3.16 seconds for the acoustic stages, consistent with reuse of previously compiled backend work; it does not replace the initial observation above. Core ML observations include platform compilation/loading of the source packages where required.

Core ML bonus

The same portable 15-second source graphs, with four concurrent public asynchronous acoustic requests, CPU+ANE compute units and CPU TDT decoding. One model instance is shared per stage; an eight-chunk refill queue bounds buffered audio. Frontend and decoder run on the CPU and can overlap acoustic work.

The Watch uses watchOS 27.0.1 (24R365), hardware identifier Watch8,1. Mac and iPhone Core ML transcripts match exactly; the Watch differs by one CTC word and one TDT punctuation mark. Its WER is 4.55% CTC / 4.14% TDT. A complete iPhone Core ML TDT capture contains 148 ANE predictions and zero app GPU intervals. Decoder placement differs from the Core AI runtime, so this compares implemented runtimes rather than holding every scheduling choice identical.

Runtime contract

These are tensor graph bundles, not standalone audio-to-text model files. A host runtime must implement the frontend, chunk scheduling, vocabulary decoding and timing assembly. metadata.json publishes the geometry, vocabulary, frame rate and numerical conventions; runtime.json publishes the recommended policy. Read these files instead of hardcoding profile shapes. Resolve one Hub snapshot for a download, and validate files using Hub-provided hashes.

The frontend expects mono 16 kHz audio, checkpoint preemphasis, its supplied 400-sample symmetric Hann window, a zero-padded 512-point STFT with 160-sample hop, the supplied 80-bin mel matrix, log compression and per-feature sample-standard-deviation normalization.

Each Core AI profile contains:

• subsampling.aimodel: padded mel input [1,1,80,8S] and three stride masks; returns [1,512,1,S] acoustic features.
• encoder.aimodel: acoustic features plus [1,1,1,S] valid-position mask; returns [1,640,1,S] TDT features and [1,1,1,S] CTC token IDs. Entrypoint is frames_S, where S is the profile's published encoder position count.
• decoder_batch.aimodel: batch_16, batch_32, batch_64 and batch_128 TDT predictor/joint functions with shared weights. Each lane has independent recurrent state. CTC uses argmax-and-collapse and does not need this asset.
• decoder.json / decoder.f32: original CPU predictor parameters with offsets and shapes.
• vocabulary.json, mel-filter.f32, window.f32, LICENSE and NOTICE.

The Core ML directory contains portable subsampling.mlpackage and encoder.mlpackage, the same host sidecars, and separate metadata. Encoder outputs are named encoded and ctc_tokens; source input/output shapes are fixed at the 192-position geometry. Compile the packages for the target platform and load with CPU+ANE policy. Implement CTC collapse or the supplied CPU TDT predictor outside these graphs. The recurrent decoder is not a separate Core ML model.

Native token and word timestamps use the 80 ms encoder grid. Chunk offsets must be added when assembling a complete recording. Blank ID is 1,024; TDT duration classes are [0,1,2,3,4]. Preserve the source greedy decoding and maximum-symbol policy published in metadata.

Attribution

Upstream: NVIDIA Parakeet TDT–CTC 110M, revision 431a349f3051ab85c22b9b7a2741b5fe77065665. The checkpoint is distributed under CC-BY-4.0. Conversion and qualification by coder543; see NOTICE. Quantization and execution layout changes do not constitute fine-tuning. Measurements were taken October 1, 2026.