firebot-voice-intent by Baena: Open-Weight Model

Model Card

By Baena, published under mit, revision 55a705eca935.

Official model checkpoints and speaker verification voiceprints for Firebot, an autonomous tactical firefighting robot platform with dual-tier offline voice intent recognition and personalized acoustic adaptation. The Firebot voice system is engineered for zero-latency, high-reliability command recognition in noisy operational environments. It combines: 1. Frozen Whisper Encoder Feature Extraction: 384-dimensional pooled audio embeddings from OpenAI's Whisper model (tiny.en / base.en). 2. L2-SP Regularized MLP Intent Classifier: A lightweight Multi-Layer Perceptron (LayerNorm $\to$ Linear 384$\times$128 $\to$ GELU $\to$ Dropout $\to$ Linear 128$\times$15) fine-tuned with an L2 penalty…

Read Baena's full model card

Firebot Voice Intent Classifier & Speaker Verification Models

Official model checkpoints and speaker verification voiceprints for Firebot, an autonomous tactical firefighting robot platform with dual-tier offline voice intent recognition and personalized acoustic adaptation.

Overview

The Firebot voice system is engineered for zero-latency, high-reliability command recognition in noisy operational environments. It combines: 1. Frozen Whisper Encoder Feature Extraction: 384-dimensional pooled audio embeddings from OpenAI's Whisper model (tiny.en / base.en). 2. L2-SP Regularized MLP Intent Classifier: A lightweight Multi-Layer Perceptron (LayerNorm $\to$ Linear 384$\times$128 $\to$ GELU $\to$ Dropout $\to$ Linear 128$\times$15) fine-tuned with an L2 penalty pulling weights toward the base anchor. 3. ECAPA-TDNN Multi-Clip Acoustic Voiceprints: 192-dimensional speaker embeddings for tactical operator authentication and biometric voice gating. 4. Adaptive Online Calibration: Few-shot per-operator head adaptation enabling rapid personalization from 1–5 takes per command with held-out generalization validation.

Model Architecture & Directory Layout

firebot-voice-intent/
├── intent_head.pt           # Base 15-class Whisper intent classification head
├── intent_head.json         # Architecture configuration, pooling, and class map
├── intent_prototypes.pt     # Latent prototype embeddings for few-shot similarity
├── users/
│   ├── ananya.pt            # Calibrated personal Whisper head for Operator Ananya
│   ├── ananya.json          # Empirical training metrics and per-class take breakdown
│   ├── ananya.history.jsonl # Training iteration telemetry log
│   ├── avinandan.pt         # Calibrated personal Whisper head for Operator Avinandan
│   ├── avinandan.json       # Calibration metrics
│   └── avinandan.history.jsonl
└── voiceprints/
    ├── ananya.npy           # ECAPA-TDNN biometric speaker voiceprint embedding
    └── avinandan.npy        # ECAPA-TDNN biometric speaker voiceprint embedding

Command Vocabulary (15 Closed-Set Classes)

Class ID	Canonical Label	Voice Phrases / Synonyms	Target Coordinate / Action
`0`	`STOP`	"stop", "halt", "freeze", "abort mission", "e-stop"	Emergency brake & valve cutoff
`1`	`EXTINGUISH`	"put out the fire", "extinguish", "spray the fire", "suppress"	Engage high-pressure water pump
`2`	`RETURN_HOME`	"return home", "go to base", "back to dock", "retreat"	Autonomous navigation to dock (1.2, 1.0)
`3`	`STATUS`	"status report", "give me a report", "tank level", "sitrep"	Telemetry & battery check
`4`	`UNKNOWN`	"hello", "what's the weather", "testing"	Out-of-domain conversational filter
`5`	`GOTO_HOME`	"go to home", "head to charging station"	Waypoint navigation: (1.2, 1.0)
`6`	`GOTO_CENTER`	"go to center", "move to the middle"	Waypoint navigation: (6.0, 4.0)
`7`	`GOTO_NORTH`	"go to north", "move to the north side"	Waypoint navigation: (6.0, 7.0)
`8`	`GOTO_SOUTH`	"go to south", "head south"	Waypoint navigation: (6.0, 1.0)
`9`	`GOTO_EAST`	"go to east", "move right"	Waypoint navigation: (10.5, 4.0)
`10`	`GOTO_WEST`	"go to west", "move left"	Waypoint navigation: (2.0, 4.0)
`11`	`GOTO_NORTHEAST`	"go to northeast", "upper right corner"	Waypoint navigation: (10.5, 7.0)
`12`	`GOTO_NORTHWEST`	"go to northwest", "upper left corner"	Waypoint navigation: (1.5, 7.0)
`13`	`GOTO_SOUTHEAST`	"go to southeast", "bottom right corner"	Waypoint navigation: (10.5, 1.0)
`14`	`GOTO_SOUTHWEST`	"go to southwest", "bottom left corner"	Waypoint navigation: (1.5, 1.2)

Quickstart: Python Inference

import soundfile as sf
import torch
from huggingface_hub import hf_hub_download
from firebot.voice_intent.infer import IntentClassifier

# 1. Download model from Hugging Face Hub
ckpt_path = hf_hub_download(repo_id="anabaena/firebot-voice-intent", filename="intent_head.pt")

# 2. Instantiate Intent Classifier (loads frozen Whisper + lightweight head)
classifier = IntentClassifier(checkpoint_path=ckpt_path)

# 3. Classify raw audio (16kHz mono WAV)
audio, sr = sf.read("command_sample.wav", dtype="float32")
prediction = classifier.predict(audio, sample_rate=sr)

print(f"Predicted Intent: {prediction['label']}")
print(f"Confidence: {prediction['confidence']:.2%}")
print(f"Action: {prediction['canonical_phrase']}")

Biometric Speaker Verification (ECAPA-TDNN)

import numpy as np

# Load enrolled speaker voiceprint
enrolled = np.load("voiceprints/ananya.npy")

# Compare with live speaker embedding
cosine_sim = np.dot(enrolled, live_embedding) / (np.linalg.norm(enrolled) * np.linalg.norm(live_embedding))
is_authorized = cosine_sim >= 0.72  # calibrated acceptance threshold
print(f"Speaker Verified: {is_authorized} (Similarity: {cosine_sim:.3f})")

Performance & Evaluation

Inference Latency: ~35ms on CPU (Apple Silicon / modern x86_64)
Base Intent Accuracy: 94.2% on diverse acoustic validation test set
Personalized Operator Accuracy: 98.8% – 100.0% with 3–5 takes per command
Memory Footprint: ~210 KB for the MLP head, ~75 MB for frozen Whisper tiny encoder

License

MIT License. Designed and developed as part of the Firebot Autonomous Robotic Platform.

Identity and Version

Repository: anabaena/firebot-voice-intent
Publisher: Baena
Task: Audio classification
Modality: Audio
Library: pytorch
Parameters: Not stated by the source
Languages: en
Revision: 55a705eca9350897bf2eb641e6d364bce07fa53b
First published: 2026-09-30
Last updated: 2026-10-02

Files and Weights

14 files, 676.2 KB in total. The weights are 4 files totalling 663.5 KB in pt.

Weights4 files · 663.5 KB

Configuration4 files · 2.0 KB

Documentation1 file · 6.0 KB

Other4 files · 3.3 KB

Repository1 file · 1.5 KB

Every file

File	Type	Size	SHA-256
intent_head.pt	Weights	211.5 KB	0fc4c19ed789
intent_prototypes.pt	Weights	29.7 KB	3cfce99b8b07
users/ananya.pt	Weights	211.0 KB	6cc571dcd8e5
users/avinandan.pt	Weights	211.3 KB	0773a777529c
intent_head.json	Configuration	438 B	—
profiles.json	Configuration	375 B	—
users/ananya.json	Configuration	532 B	—
users/avinandan.json	Configuration	610 B	—
README.md	Documentation	6.0 KB	—
users/ananya.history.jsonl	Other	936 B	—
users/avinandan.history.jsonl	Other	557 B	—
voiceprints/ananya.npy	Other	896 B	803903c58875
voiceprints/avinandan.npy	Other	896 B	70ed06443dda
.gitattributes	Repository	1.5 KB	—

License and Download

License: mit
Access: Open weights, no gate
Download size: 663.5 KB

Download from Baena

Released by Baena through its official repository on Hugging Face. Read the license.

Memory Requirements

Precision	Weights in memory
As published	663.5 KB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About firebot-voice-intent

Can I use firebot-voice-intent commercially?

Yes. firebot-voice-intent is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.

Similar Models

Model · Audio classification

ced-gguf

Mudler

GGUF quantizations of the CED family (Consistent Ensemble Distillation, Xiaomi) - SOTA-tier audio-tagging models that classify everyday sounds (baby cry, footsteps, glass breaking, alarms, dog bark,...) into the 527-class AudioSet ontology. These files run with ced.cpp, a standalone C++/ggml port (no Python, no PyTorch at inference), and with LocalAI via the ced backend. Converted from the mispeech/ced- checkpoints (Apache-2.0). CED is a plain AST/DeiT Vision Transformer over a log-mel spectrogram; the port is numerically equal to the PyTorch reference. One self-contained GGUF per size + quant (config, 527 labels, and the mel filterbank/window are all embedded). Pick by your accuracy/size…

Open weights apache-2.0 ced.cpp

View model

Model · Audio classification

lang-id-voxlingua107-ecapa

SpeechBrain

This is a spoken language recognition model trained on the VoxLingua107 dataset using SpeechBrain. The model uses the ECAPA-TDNN architecture that has previously been used for speaker recognition. However, it uses more fully connected hidden layers after the embedding layer, and cross-entropy loss was used for training. We observed that this improved the performance of extracted utterance embeddings for downstream tasks. The system is trained with recordings sampled at 16kHz (single channel). The code will automatically normalize your audio (i.e., resampling + mono channel selection) when calling classifyfile if needed. The model can classify a speech utterance according to the language…

Open weights apache-2.0 speechbrain

View model

Model · Audio classification

accent-id-commonaccent_ecapa

Juan Pablo Zuluaga

Abstract: The recognition of accented speech still remains a dominant problem in Automatic Speech Recognition (ASR) systems. We approach the classification of accented English speech through the Emphasized Channel Attention, Propagation and Aggregation Time Delay Neural Network (ECAPA-TDNN) architecture which has been shown to perform well on a variety of speech tasks. Three models are proposed: one trained from scratch, another two models (one using data augmentation and a baseline model) fine-tuned from the checkpoints of speechbrain/spkrec-ecapa-voxceleb (VoxCeleb). Our results show that the model fine-tuned with data augmentation yield the best results. Most of the misclassifications…

Open weights mit speechbrain

View model

Model · Audio classification

MERT-v1-330M

Multimodal Art Projection

The development log of our Music Audio Pre-training (m-a-p) model family: - 02/06/2023: arxiv pre-print and training codes released. - 17/03/2023: we release two advanced music understanding models, MERT-v1-95M and MERT-v1-330M, trained with new paradigm and dataset. They outperform the previous models and can better generalize to more tasks. - 14/03/2023: we retrained the MERT-v0 model with open-source-only music dataset MERT-v0-public - 29/12/2022: a music understanding model MERT-v0 trained with MLM paradigm, which performs better at downstream tasks. - 29/10/2022: a pre-trained MIR model music2vec trained with BYOL paradigm. Here is a table for quick model pick-up: The m-a-p models…

Open weights cc-by-nc-4.0 transformers

View model

Model · Audio classification

MERT-v1-95M

Multimodal Art Projection

The development log of our Music Audio Pre-training (m-a-p) model family: - 02/06/2023: arxiv pre-print and training codes released. - 17/03/2023: we release two advanced music understanding models, MERT-v1-95M and MERT-v1-330M, trained with new paradigm and dataset. They outperform the previous models and can better generalize to more tasks. - 14/03/2023: we retrained the MERT-v0 model with open-source-only music dataset MERT-v0-public - 29/12/2022: a music understanding model MERT-v0 trained with MLM paradigm, which performs better at downstream tasks. - 29/10/2022: a pre-trained MIR model music2vec trained with BYOL paradigm. Here is a table for quick model pick-up: The m-a-p models…

Open weights cc-by-nc-4.0 transformers

View model

Model · Audio classification

gender_cls_svm_ecapa_voxceleb

Gregory Koushnir

This model combines the SpeechBrain ECAPA-TDNN speaker embedding model with an SVM classifier to predict speaker gender from audio input. The model was trained and evaluated on the VoxCeleb2, Mozilla Common Voice v10.0, and TIMIT datasets - Mozilla Common Voice v10.0 English validated test set: 92.3% accuracy The model was trained on VoxCeleb2 dataset: - Converted to WAV format, single channel, 16kHz sampling rate, 256 kp/s bitrate - Applied SileroVAD for voice activity detection, taking the first voiced segment You can install the package directly from GitHub: - Model was trained on celebrity voices from YouTube interviews - Performance may vary on different audio qualities or recording…

Open weights apache-2.0

View model