Laya-Heart reply drafter v4 (Gemma 4 E2B, GGUF Q4_K_M)
A fine-tuned Gemma 4 E2B for Laya-Heart, a local web app that helps you reply in a couple's chat. Given the chat and Laya's reading of it, the model writes two candidate replies, one per line. You choose, edit and send one yourself.
- File:
laya-heart-gemma-e2b-v4.Q4_K_M.gguf, 3.43 GB.
- Size and speed: about the same as the original
gemma4:e2b (3.50 GB).
- SHA-256:
ba1300a1cc08e5ebbe864dc471026516d509f7171bb98f525028dd6f17a49ce6
Experimental. The app still uses the original gemma4:e2b. This version has not been through the project's acceptance yet (sealed exam + human blind comparison), and the trainer's own 50-chat self-check reads "not reached" (see below).
How it compares with the original
297 fictional practice-exam chats from the project's train-kit (253 Chinese, 44 English). Both models drafted every chat the way the app does. The judge, Qwen3-32B-AWQ, did not know which was which and judged every pair twice with the sides swapped; ties count as half a win.
|
Original gemma4:e2b |
v4 |
| Judge win rate vs the original (all) |
— |
0.611 (133 W / 67 L / 97 T) |
| Chinese chats |
— |
0.650 |
| English chats |
— |
0.386 |
| Made-up facts or promises flagged |
16 |
1 |
| Hurtful lines flagged |
1 |
0 |
| Sets in the app's format (2 lines, Chinese ≤ 40 chars, English ≤ 60) |
0.943 |
0.983 |
| English lines over 60 characters |
21 |
5 |
The flags come from the app's own checks. They are rule-based, not reviewed by a person.
Self-check: the trainer's self-check (50 chats, nine gates) passed eight gates and missed one: judge win rate 0.54 against a gate of 0.55. On the same 50 chats with fresh drafts the rate was 0.57, and over all 297 chats it is 0.611.
Weak spot: English. The judge mostly prefers longer English replies, but lines over 60 characters break the app's limit. In the 17 English pairs where v4's lines were about as long as the first fine-tune's, v4 scored 0.56 against it; that sample is small.
Use it
With Ollama 0.40 or later:
- Download
laya-heart-gemma-e2b-v4.Q4_K_M.gguf and Modelfile into one folder.
- Run
ollama create laya-heart-gemma-e2b:v4 -f Modelfile.
With llama.cpp:
llama-server -m laya-heart-gemma-e2b-v4.Q4_K_M.gguf --swa-full
Prompt. The model was trained on Laya-Heart's drafting prompt (rules, the chat, then Laya's reading turned into instructions). It works best with that prompt; see draft/prompt.js in the project.
Ollama compatibility. Ollama 0.40.5 keeps models made with ollama create in a new manifest folder, and the app on main does not look there yet. If you point the app at this model, it falls back to Ollama's chat API, which is slower and uses about 6.5 GB of memory. A fix is ready and will be merged after the Hacktoberfest deadline.
How it was made
- Training: LoRA on
unsloth/gemma-4-E2B-it (Unsloth's copy of Google's weights), rank 16, 2 epochs, learning rate 2e-4. Unsloth 2026.10.6 on a Kaggle T4, 47 minutes.
- Conversion: merged, exported to F16 GGUF, quantized to Q4_K_M with llama.cpp.
- Data: 1,496 drafting examples, all built from fictional chats the train-kit generated. No real chat logs.
- Chinese (1,388): reply sets written by DeepSeek-V4.1-Flash, filtered by the app's checks, and chosen by its score.
- English (108): the original Gemma's own drafts that pass the app's checks, also chosen by DeepSeek's score.
- History: this is the fourth version from the same night. The earlier three are not published.
License
Apache License 2.0, the same as Gemma 4 E2B. See LICENSE-Apache-2.0.txt and NOTICE.txt.
