Search public pages, research tools, and SAVRN solutions.
Open-weight model · Speech recognition
by Jonatas Grosman jonatasgrosman/wav2vec2-large-xlsr-53-japanese
Fine-tuned facebook/wav2vec2-large-xlsr-53 on Japanese using the train and validation splits of Common Voice 6.1, CSS10 and JSUT. When using this model, make sure that your speech input is sampled at 16kHz.
Japanese speech into text starts with one hard input rule: audio sampled at 16 kHz. The facts file has no runs_on entry, so we size from what it gives, 2.54 GB of weights across eight files in pytorch and jax, 24 layers, 16 attention heads and a 2,341-entry vocabulary feeding a CTC head. Jonatas Grosman fine-tuned it from facebook/wav2vec2-large-xlsr-53 on Common Voice 6.1, CSS10 and JSUT, and it runs without a language model.
Apache 2.0 clears commercial use, modification and redistribution as long as the license, copyright notices and NOTICE file stay attached and significant changes are stated, so a production pipeline is clear. Weigh the evidence: the publisher reports a test CER of 20.16 and a test WER of 81.8 on Common Voice ja, and the trained_on relation names common_voice. Access is open, the last update was December 14, 2022, and no host on the Index prices it.
By Jonatas Grosman, published under apache-2.0, revision cf031e020336.
Fine-tuned facebook/wav2vec2-large-xlsr-53 on Japanese using the train and validation splits of Common Voice 6.1, CSS10 and JSUT. When using this model, make sure that your speech input is sampled at 16kHz.
This model has been fine-tuned thanks to the GPU credits generously given by the OVHcloud :)
The script used for training can be found here: https://github.com/jonatasgrosman/wav2vec2-sprint
The model can be used directly (without a language model) as follows...
Using the HuggingSound library:
from huggingsound import SpeechRecognitionModel
model = SpeechRecognitionModel("jonatasgrosman/wav2vec2-large-xlsr-53-japanese")
audio_paths = ["/path/to/file.mp3", "/path/to/another_file.wav"]
transcriptions = model.transcribe(audio_paths)
Writing your own inference script:
8 files, 2.5 GB in total. The weights are 2 files totalling 2.5 GB in bin, msgpack.
| File | Type | Size | SHA-256 |
|---|---|---|---|
| flax_model.msgpack | Weights | 1.3 GB | 145e6adcead3 |
| pytorch_model.bin | Weights | 1.3 GB | 4f6821dcd797 |
| config.json | Configuration | 1.6 KB | — |
| preprocessor_config.json | Configuration | 158 B | — |
| special_tokens_map.json | Configuration | 85 B | — |
| README.md | Documentation | 8.6 KB | — |
| .gitattributes | Repository | 736 B | — |
| vocab.json | Tokenizer | 29.3 KB | — |
Released by Jonatas Grosman through its official repository on Hugging Face. Read the license.
Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical.
| Benchmark | Conditions | Result | Reported by | Revision | Date |
|---|---|---|---|---|---|
| Common Voice ja | Task Speech RecognitionMetric Test CERComparison conditions not established | 20.16 | jonatasgrosman Publisher reported |
Evaluated revision not stated | — |
| Common Voice ja | Task Speech RecognitionMetric Test WERComparison conditions not established | 81.8 | jonatasgrosman Publisher reported |
Evaluated revision not stated | — |
| Precision | Weights in memory |
|---|---|
| As published | 2.5 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Yes. wav2vec2-large-xlsr-53-japanese is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
WhisperKit is part of Argmax OSS, an On-device Speech AI SDK for Apple Silicon: https://github.com/argmaxinc/argmax-oss-swift Check out the WhisperKit paper and presentation from ICML 2025: https://icml.cc/virtual/2025/47854 For real-time transcription with speakers and custom vocabulary, check out Argmax Pro SDK: https://www.argmaxinc.com/blog/argmax-sdk-2
Using this open-source model in production? Consider switching to pyannoteAI for better and faster options. This pipeline is the same as pyannote/speaker-diarization-3.0 except it removes the problematic use of onnxruntime. Both speaker segmentation and embedding now run in pure PyTorch. This should ease deployment and possibly speed up inference. It requires pyannote.audio version 3.1 or higher. It ingests mono audio sampled at 16kHz and outputs speaker diarization as an Annotation instance: - stereo or multi-channel audio files are automatically downmixed to mono by averaging the channels. - audio files sampled at a different rate are resampled to 16kHz automatically upon loading. 1.…
Fine-tuned facebook/wav2vec2-large-xlsr-53 on Portuguese using the train and validation splits of Common Voice 6.1. When using this model, make sure that your speech input is sampled at 16kHz. This model has been fine-tuned thanks to the GPU credits generously given by the OVHcloud:) The script used for training can be found here: https://github.com/jonatasgrosman/wav2vec2-sprint The model can be used directly (without a language model) as follows... Using the HuggingSound library: 1. To evaluate on mozilla-foundation/commonvoice60 with split test 2. To evaluate on speech-recognition-community-v2/devdata If you want to cite this model you can use this
This pipeline ingests mono audio sampled at 16kHz and outputs speaker diarization. - stereo or multi-channel audio files are automatically downmixed to mono by averaging the channels. - audio files sampled at a different rate are resampled to 16kHz automatically upon loading. The main improvements brought by Community-1 are: - improved speaker assignment and counting - simpler reconciliation with transcription timestamps with exclusive speaker diarization - easy offline use (i.e. without internet connection) - (optionally) hosted on pyannoteAI cloud 1. pip install pyannote.audio 3. Create access token at hf.co/settings/tokens. Out of the box, Community-1 is much better than…
Fine-tuned facebook/wav2vec2-large-xlsr-53 on Russian using the train and validation splits of Common Voice 6.1 and CSS10. When using this model, make sure that your speech input is sampled at 16kHz. This model has been fine-tuned thanks to the GPU credits generously given by the OVHcloud:) The script used for training can be found here: https://github.com/jonatasgrosman/wav2vec2-sprint The model can be used directly (without a language model) as follows... Using the HuggingSound library: 1. To evaluate on mozilla-foundation/commonvoice60 with split test 2. To evaluate on speech-recognition-community-v2/devdata If you want to cite this model you can use this
Fine-tuned facebook/wav2vec2-large-xlsr-53 on Polish using the train and validation splits of Common Voice 6.1. When using this model, make sure that your speech input is sampled at 16kHz. This model has been fine-tuned thanks to the GPU credits generously given by the OVHcloud:) The script used for training can be found here: https://github.com/jonatasgrosman/wav2vec2-sprint The model can be used directly (without a language model) as follows... Using the HuggingSound library: 1. To evaluate on mozilla-foundation/commonvoice60 with split test 2. To evaluate on speech-recognition-community-v2/devdata If you want to cite this model you can use this