SAVRN
Search Contact SAVRN

thalovant-m2v-intents · Model Card

thalovant-m2v-intents: Model Card

Written by Thalovant, published under apache-2.0, revision d5167312c77b, read 2026-10-04. Shown as written; SAVRN's own facts about this model are on its page.

A static-embedding intent classifier for the Thalovant hub. It maps an utterance to one of 190 labels of the form <skill_id>:<intent>, exactly as ovos-m2v-pipeline registers them. The labels are the fleet's skills and the other intents the hubs register (the persona and reading pipelines, wikihow), from the fleet corpus and nothing else. There is no "none of these" label: a sentence about something no skill does still gets the nearest skill's label, often with high confidence.

Base: Jarbas/m2v-256-distiluse-base-multilingual-cased-v2, fine-tuned with model2vec's classifier trainer.

How good it is

Measured on sentences the classifier never saw: 22317 held-out rows whose text, in any language, and whose intent line are both absent from the 91300 rows an evaluation model was trained on. A split per row would score the en-GB twin of an en-US training sentence as new.

  • Intent accuracy 86.1 %, skill accuracy 94.2 % across 68 languages; median language 84.7 %, 16 language(s) under 80 %.
  • 0.7 % of held-out sentences are given to another skill at 0.9 or more: what thalovant-skillkit check would report about a sentence that is in fact new and the skill's own.
  • Languages that need a higher threshold to keep that under 1.0 %: az-AZ 0.97, bg-BG 0.92, el-GR 0.94, es-AR 0.94, es-CO 0.94, es-ES 0.94, es-MX 0.94, es-US 0.94, et-EE 0.97, fi-FI 0.94, he-IL 0.96, hu-HU 0.92, ja-JP 0.99, lt-LT 0.99, lv-LV 0.96, pl-PL 0.97, ro-RO 0.96, sl-SI 0.92, sw-KE 0.95, th-TH 0.92, tr-TR 0.97, uk-UA 0.97, vi-VN 0.96 (thresholds in training.json).

The published weights are then trained with the same recipe on all 113617 rows, so a skill's existing sentences are the classifier's training data. At most 512 sentences per label and language are trained on (the reading pipeline's English intents alone expand to thousands); index.json keeps all 133826.

Use

from model2vec.inference import StaticModelPipeline

model = StaticModelPipeline.from_pretrained("thalovant/thalovant-m2v-intents")
model.predict(["will it rain tomorrow"])

Configure the pipeline with "mode": "classifier" and this model's path; labels.json beside the weights lists the labels the hub may route to.

index.json names every sentence in the training corpus by a digest of its language and text, with the labels that publish it. thalovant-skillkit check uses it to fail a skill that publishes a sentence another skill already owns, and asks this classifier what it makes of the rest -- without the fleet's sentences leaving their private repositories.

Held-out results per language

False warnings: held-out sentences given to another skill at 0.9 or more. Threshold: the lowest that keeps them under 1.0 %.

Language Rows Intent Skill False warnings Threshold
an-ES 221 92.3 % 96.8 % 0.4 % 0.9
ar-SA 247 82.2 % 90.3 % 0.8 % 0.9
ast-ES 229 87.3 % 94.8 % 0.4 % 0.9
az-AZ 234 76.1 % 87.6 % 1.7 % 0.97
bg-BG 234 91.0 % 96.6 % 1.3 % 0.92
ca-ES 551 94.0 % 96.9 % 0.2 % 0.9
cs-CZ 312 83.7 % 95.8 % 0.3 % 0.9
da-DK 801 89.9 % 94.4 % 0.9 % 0.9
de-AT 437 83.5 % 93.1 % 0.7 % 0.9
de-CH 437 83.5 % 93.1 % 0.7 % 0.9
de-DE 529 84.7 % 93.4 % 0.6 % 0.9
el-GR 209 74.6 % 85.7 % 1.4 % 0.94
en-AU 431 84.2 % 95.8 % 0.0 % 0.9
en-CA 431 84.2 % 95.8 % 0.0 % 0.9
en-GB 431 84.2 % 95.8 % 0.0 % 0.9
en-NZ 431 84.2 % 95.8 % 0.0 % 0.9
en-US 1242 93.4 % 98.4 % 0.0 % 0.9
es-AR 302 93.4 % 96.7 % 1.0 % 0.94
es-CO 284 92.6 % 96.1 % 1.4 % 0.94
es-ES 455 94.7 % 97.6 % 0.9 % 0.94
es-MX 283 92.9 % 96.5 % 1.1 % 0.94
es-US 283 92.9 % 96.5 % 1.1 % 0.94
et-EE 212 75.0 % 85.9 % 4.2 % 0.97
eu-ES 269 70.3 % 88.8 % 0.7 % 0.9
fa-IR 263 76.8 % 90.1 % 0.8 % 0.9
fi-FI 240 72.9 % 85.8 % 1.7 % 0.94
fr-BE 431 80.3 % 95.8 % 0.2 % 0.9
fr-CA 433 80.4 % 95.8 % 0.2 % 0.9
fr-CH 431 80.3 % 95.8 % 0.2 % 0.9
fr-FR 887 87.5 % 98.0 % 0.1 % 0.9
gl-ES 527 94.7 % 98.5 % 0.4 % 0.9
he-IL 241 80.5 % 89.2 % 2.1 % 0.96
hi-IN 250 78.8 % 91.2 % 0.8 % 0.9
hr-HR 222 87.8 % 94.6 % 0.4 % 0.9
hu-HU 259 73.8 % 86.1 % 1.2 % 0.92
id-ID 204 93.6 % 97.5 % 0.5 % 0.9
it-IT 386 88.1 % 95.3 % 0.5 % 0.9
ja-JP 283 82.3 % 92.2 % 2.1 % 0.99
kab 35 65.7 % 97.1 % 0.0 % 0.9
ko-KR 245 79.2 % 89.0 % 0.8 % 0.9
lt-LT 210 73.8 % 82.4 % 3.3 % 0.99
lv-LV 234 74.4 % 88.5 % 2.1 % 0.96
ms-MY 234 92.7 % 97.0 % 0.4 % 0.9
nb-NO 236 90.2 % 94.5 % 0.0 % 0.9
nl-BE 310 91.0 % 96.1 % 0.7 % 0.9
nl-NL 407 90.9 % 96.1 % 0.5 % 0.9
oc-FR 300 87.7 % 97.0 % 0.0 % 0.9
pl-PL 246 80.1 % 87.4 % 2.4 % 0.97
pt-AO 286 93.0 % 96.5 % 0.0 % 0.9
pt-BR 395 92.9 % 96.2 % 0.0 % 0.9
pt-MZ 286 93.0 % 96.5 % 0.0 % 0.9
pt-PT 462 93.1 % 97.2 % 0.0 % 0.9
ro-RO 249 83.5 % 94.4 % 1.2 % 0.96
ru-RU 252 89.3 % 94.8 % 0.8 % 0.9
sk-SK 325 89.8 % 95.1 % 0.9 % 0.9
sl-SI 227 84.6 % 93.0 % 1.3 % 0.92
sv-FI 268 85.1 % 94.0 % 0.4 % 0.9
sv-SE 403 84.1 % 94.5 % 0.5 % 0.9
sw-KE 236 74.2 % 87.3 % 1.3 % 0.95
th-TH 230 55.6 % 69.1 % 1.3 % 0.92
tr-TR 261 74.7 % 86.2 % 1.9 % 0.97
uk-UA 263 86.3 % 92.8 % 1.5 % 0.97
vi-VN 233 78.1 % 90.6 % 2.1 % 0.96
zh-CN 249 96.0 % 98.0 % 0.8 % 0.9
zh-SG 249 96.0 % 98.0 % 0.8 % 0.9
zh-TW 414 96.4 % 98.3 % 0.5 % 0.9

Held-out results per skill

Skill Rows Intent Skill False warnings
ovos-common-reading-pipeline-plugin 695 94.7 % 99.4 % 0.0 %
ovos-persona-pipeline-plugin 1576 97.4 % 97.6 % 0.2 %
ovos-skill-wikihow.openvoiceos 77 83.1 % 83.1 % 5.2 %
thalovant-skill-alarm.thalovant 464 83.0 % 94.8 % 0.2 %
thalovant-skill-custos-query.thalovant 236 80.5 % 89.4 % 4.7 %
thalovant-skill-custos-shadow.thalovant 272 89.7 % 91.5 % 0.0 %
thalovant-skill-date-time.thalovant 1864 83.4 % 96.0 % 0.9 %
thalovant-skill-easter-eggs.thalovant 213 55.9 % 62.4 % 0.0 %
thalovant-skill-fart.thalovant 553 82.8 % 93.7 % 0.2 %
thalovant-skill-guide.thalovant 2633 87.5 % 93.8 % 0.8 %
thalovant-skill-home.thalovant 3937 86.1 % 97.5 % 0.5 %
thalovant-skill-interaction-modes.thalovant 382 83.0 % 93.7 % 0.3 %
thalovant-skill-joke-garden.thalovant 753 81.9 % 83.8 % 2.0 %
thalovant-skill-language-buddy.thalovant 249 91.2 % 91.2 % 0.4 %
thalovant-skill-learning-lounge.thalovant 344 88.9 % 88.9 % 1.5 %
thalovant-skill-local-pulse.thalovant 203 89.2 % 89.2 % 2.5 %
thalovant-skill-news.thalovant 1006 94.7 % 98.2 % 0.2 %
thalovant-skill-ops-copilot.thalovant 378 88.9 % 88.9 % 1.8 %
thalovant-skill-recall.thalovant 396 41.9 % 98.7 % 0.0 %
thalovant-skill-reminder.thalovant 436 87.6 % 92.7 % 0.7 %
thalovant-skill-rewrite-forge.thalovant 515 86.4 % 86.4 % 1.8 %
thalovant-skill-safety-guide.thalovant 342 90.6 % 90.6 % 0.9 %
thalovant-skill-source-scout.thalovant 392 77.5 % 77.5 % 2.5 %
thalovant-skill-stopwatch.thalovant 545 78.9 % 91.4 % 0.4 %
thalovant-skill-timer.thalovant 508 86.4 % 94.7 % 0.8 %
thalovant-skill-volume.thalovant 926 84.9 % 97.7 % 0.2 %
thalovant-skill-weather.thalovant 2422 87.7 % 95.7 % 0.5 %

What the classifier confuses

Each line is a held-out sentence whose true intent the classifier read as another intent. Two skills on one line is a collision the fleet should resolve.

  • thalovant-skill-recall.thalovant:what.did.i.ask read as thalovant-skill-recall.thalovant:did.i.ask (180): "what did i ask you to something" (en-AU)
  • thalovant-skill-home.thalovant:home.cover.close read as thalovant-skill-home.thalovant:home.cover.open (131): "stáhni žaluzie" (cs-CZ)
  • thalovant-skill-home.thalovant:home.state read as thalovant-skill-weather.thalovant:temperature.weather (41): "ofisdə temperatur neçə dərəcədir" (az-AZ)
  • thalovant-skill-home.thalovant:home.lock read as thalovant-skill-home.thalovant:home.unlock (37): "lås terrassedøren" (da-DK)
  • thalovant-skill-home.thalovant:home.state read as thalovant-skill-home.thalovant:home.cover.close (37): "ye zarrau o garaje" (an-ES)
  • thalovant-skill-home.thalovant:home.state read as thalovant-skill-home.thalovant:home.lock (29): "je garážová vrata zamčené" (cs-CZ)
  • thalovant-skill-news.thalovant:global.news read as thalovant-skill-news.thalovant:news (29): "تحرير الأخبار الدولية" (ar-SA)
  • thalovant-skill-volume.thalovant:volume_reset read as thalovant-skill-volume.thalovant:volume_level (29): "setze die lautstärke zurück" (de-DE)
  • thalovant-skill-home.thalovant:home.state read as thalovant-skill-home.thalovant:home.cover.open (26): "estan oberts les persianes" (ca-ES)
  • thalovant-skill-home.thalovant:home.cover.close read as thalovant-skill-home.thalovant:home.turn.on (24): "ferme les volets du something" (fr-BE)
  • thalovant-skill-home.thalovant:home.cover.open read as thalovant-skill-home.thalovant:home.cover.close (22): "rul rullegardinerne op" (da-DK)
  • thalovant-skill-interaction-modes.thalovant:party.mode.disable read as thalovant-skill-interaction-modes.thalovant:party.mode.enable (22): "وضع الحزب" (ar-SA)
  • thalovant-skill-fart.thalovant:fart.trap.set read as thalovant-skill-fart.thalovant:fart (21): "læg en prutpude" (da-DK)
  • thalovant-skill-home.thalovant:home.state read as thalovant-skill-home.thalovant:home.unlock (21): "je garáž odemčené" (cs-CZ)
  • thalovant-skill-recall.thalovant:when.did.i.ask read as thalovant-skill-recall.thalovant:did.i.ask (20): "when did we ask you about something" (en-AU)

Trained from the corpus built 2026-10-04T03:20:15Z covering 27 skills and sources, expanded by thalovant-skillkit 0.24.2.