Laya-Heart judge: Laya-multilingual as ONNX, zero-shot and fine-tuned (v3)
Two ONNX exports of Laya-multilingual (322M parameters, built on mmBERT) used by Laya-Heart, a texting copilot for couples that runs on your own computer. Laya reads the partner's message and answers short card questions with probabilities ("Does 没事 really mean it's fine?", a danger level from 1 to 10, a hidden safety question).
| Folder |
What it is |
Used by |
zero-shot/ |
The original checkpoint (revision 7b928d8), converted to ONNX fp32; weights unchanged. Byte-identical to the project's release laya-onnx-v1. |
The app, by default |
v3/ |
Fine-tuned for couples' chats with the Laya-Heart training kit (third version of the first run, 2026-10-15/16), exported to ONNX fp32. |
The evaluation (npm run reproduce -- --models); it goes into the app after the sealed final exam |
Both run with onnxruntime and Laya's laya-ts runtime, exactly as the app runs them.
Results on the 300-chat practice exam
Practice exam written and labelled by teacher B (Qwen3-32B-AWQ, a different model family from the teacher that wrote the training chats). One command in the repository recomputes every number from the raw answers, or re-runs both models: npm run reproduce / npm run reproduce -- --models.
|
zero-shot |
v3 |
| Judgment, mean macro-F1 over 11 questions |
0.326 |
0.571 (gain 95% CI 0.18–0.31) |
| Calibration error (ECE) |
0.153 |
0.068 |
| Danger level, mean absolute error (1–10) |
2.78 |
1.56 |
| Safety recall at the 0.8 alarm line |
0 / 21 |
13 / 21 |
| False alarms on ordinary chats |
2.9% |
1.4% |
On a separate held-out safety probe (66 clear chats from a seed never used in training), safety recall went from 2 / 47 to 37 / 47 with no false alarms. The pre-registered bar for safety recall is 0.9; reaching it is the next step, and the fine-tuning report lists every gate and caveat. These are practice-exam numbers; the sealed final exam comes next.
Training data
v3 was trained on 3,000 synthetic practice chats written by DeepSeek-V4.1-Flash (weights MIT) to fit chosen answers, each read back blind five times per question; answers that disagreed were dropped. No real chat was used.
Use
Laya-Heart drafts chat replies; it is not counselling or crisis support, and the safety notice is a hint, not a diagnosis. In a crisis, contact local emergency services or a helpline (China 12356, US 988).
License
Apache License 2.0 (LICENSE-Apache-2.0.txt), like the original Laya-multilingual checkpoint (Copyright Convai Innovations). Its encoder is fine-tuned from jhu-clsp/mmBERT-base, MIT (LICENSE-MIT-mmBERT.txt). See NOTICE.txt. Not made or endorsed by Convai Innovations.
Files (SHA-256)
See SHA256SUMS.txt.