Gemma-4-E4B meeting agent (zh-TW): LiteRT mobile
Gemma-4-E4B fine-tuned to read a live zh-TW meeting transcript on a phone, window by window, and write
short cited notes: decisions, actions, proposals, open issues and key figures. The same model turns the
notes into a prose summary and a title. It is the E4B sibling of
Luigi/gemma-4-E2B-meeting-agent-zh
mobile-v1: same prompts, protocol and note format, fewer factual errors, about 3× slower.
Latest: mobile-v2/ (2026-10-17), see the mobile-v2 section. The files at the root are mobile-v1.
Code, agent protocol and evaluation: https://github.com/vieenrose/meeting-summarizer.
Integration: integration note.
| file |
|
gemma-4-E4B-meeting-agent-zh-mobile-v1.litertlm |
for stock LiteRT-LM (3,659,518,480 bytes, sha256 a92bcd45bce060fce98337a7675cf02aa5eabb8cdbfdc4032c98d27333c5d606) |
mfa/ |
for the forked engine (recommended on CPU): prefill_decode_fused.tflite (the main graph with fused int8 attention, sha256 34858194f4f596fae132470a2e0f2f0f276e540c7af8502e4ef7cca99e871424), the embedder and per-layer embedder, the SentencePiece tokenizer, and LlmMetadataProto.pbtext (chat template) |
system_prompt.txt |
the system prompt the model was trained with; use it verbatim |
mobile-v2 (2026-10-17): fine-tuned further, same engine and files
mobile-v2/ holds the E4B mobile-v2 reader. It continues mobile-v1 with two training stages:
1. SFT on reading rows plus dense-list conversion rows, including a third domain: Taiwanese financial TV programs;
2. one round of on-policy distillation. The student reads training meetings with the deployed harness. Gemma-4-31B then gives its top-20 next-token distribution on the student's own rollouts, and the student is trained on that distribution.
The export is the same as mobile-v1: GPTQ onto Google's integer grid, written into Google's mobile graph, then the attention is fused for the forked engine. The embedders and the tokenizer are byte-identical to mobile-v1. Only the main graph and the system prompt change.
file in mobile-v2/ |
bytes |
sha256 |
mfa/prefill_decode_fused.tflite (forked engine, recommended) |
2,260,210,576 |
2d346d0e5f4c1a91ae3ab9df7cea2e29992f4d60a7c661ccbc886e964fe618d0 |
mfa/Section2_TFLiteModel_tf_lite_embedder.tflite (= mobile-v1) |
170,920,584 |
94cf45ffd3d0dd7040d27b22e70845cf1054db90613bf31d8ffc1623d23d4a40 |
mfa/Section3_TFLiteModel_tf_lite_per_layer_embedder.tflite (= mobile-v1) |
836,778,512 |
d35fe41db89fa0128f5da9530886a12581cee715369e1b9ed9a3644e43d16a6a |
mfa/Section1_SP_Tokenizer.spiece (= mobile-v1) |
4689013 |
e594c8a90eb08d8bda498ff4747977dc827ae0c3c56b5c0d41a605a22d02ef03 |
mfa/LlmMetadataProto.pbtext (chat template) |
13435 |
a64aac25c4765f250decc4fef424c4194a59de8cd6c75e948964d9067c621dbe |
system_prompt.txt |
1686 |
044150e8e18c05f807c88ca88358953f81fcc74fe85959184d66c65d81fd4843 |
gemma-4-E4B-meeting-agent-zh-mobile-v2.litertlm (stock LiteRT-LM) |
3,659,518,480 |
d8c3a9d4832b13a45467b4435fc4e982c2b2ce705b65cb7c75a141e936ad43db |
lora/qat_lora.safetensors (training-format LoRA on Google's mobile weights, not PEFT) |
139,590,432 |
3aafee8f42a63df87a2e34badfabd254ce42679b18af5c8b4b9b2cf619463afb |
safetensors/ |
the merged checkpoint after GPTQ, in the format of Google's mobile HF checkpoint (integers + Google's scales and activation ranges) |
The system prompt changes in one line. 每段最多 5 則 becomes 每段最多 8 則: the model was trained and evaluated with up to 8 notes per window. Use mobile-v2/system_prompt.txt with the mobile-v2 weights, and the mobile-v1 prompt with the mobile-v1 weights. Everything else is unchanged: the protocol, the 4k context, about 1,500 transcript tokens per window, the restart journal of 1,200 tokens, read_max 400, and the sampler.
Results against mobile-v1. Same engine (forked, CPU, 4k) and same judge (Gemma-4-31B), on 43 held-out meetings in three domains: 10 financial TV programs, 13 Legislative Yuan committee meetings and 20 AliMeeting business meetings. None of them was used in training.
| E4B |
financial TV |
Legislative Yuan |
AliMeeting |
| notes contradicted by the transcript, v1 → v2 |
15.8 % → 11.7 % |
10.6 % → 9.6 % |
8.5 % → 8.0 % |
| coverage of the reference points (dense-list summary), v1 → v2 |
0.65 → 0.76 |
0.96 → 0.97 |
0.96 → 0.91 |
What this means for E4B. The share of contradicted notes drops by 1.5 points over the 43 meetings (95 % CI −2.9 to −0.2): the improvement is significant, and it is largest on the financial programs. v2 writes more notes, so the count per meeting rises slightly (+1.2). Coverage rises on the financial programs (+0.11) and is flat on committee meetings, but falls on AliMeeting (−0.05, significant). Part of the coverage gain comes from longer summaries. E4B mobile-v2 is the recommended E4B reader, except for apps that mostly summarize business meetings like AliMeeting.
How it was built
- Base graph. Google's Gemma-4-E4B mobile graph
(
litert-community/gemma-4-E4B-it-litert-lm):
int4/int2 weights, an int8 KV cache, and int8 static-range activations.
- Training. A LoRA trained on Google's mobile weights, with Google's activation ranges in the loop.
- Back onto Google's grid. The merged weights are rounded onto Google's own integer grid with GPTQ,
using Google's scales. The integers are then written into Google's file, bit for bit.
- Two settings change. GPU activations are fp32, because fp16 corrupts long prompts on the Mali-G68.
The
prefill_1024 signature is disabled, because its buffers dominate peak memory.
Results
Quality. Measured on 38 held-out sessions of Legislative Yuan committee meetings, transcribed by
on-device ASR. The judge is Gemma-4-31B, which checks each cited statement against the transcript. The
runs went through LiteRT-LM with an 8k context.
|
E2B mobile-v1 (4k) |
E4B mobile-v1 (8k) |
| minutes contradicted by the transcript |
17 % |
11 % |
| unsupported |
5 % |
7 % |
| coverage of the reference key points |
0.92 |
0.94 |
| 決議事項 really decided |
75 % |
75 % |
| 待辦 really assigned |
66 % |
76 % |
| prose: sentences contradicting the notes |
10 % |
8 % |
| title (1–5) |
4.05 |
4.29 |
Through the forked engine (CPU, 4k) on 13 of these sessions, E4B gives the same figures: 11 % contradicted, coverage 0.93, 決議事項 75 %, 待辦 76 %. An 8k context adds only decision recall (73 % to 80 %) and costs 10 to 20 % of the speed, so 4k is the recommended setting (E4B peak RSS 2.65 GB at 4k, 2.82 GB at 8k).
Speed and memory. Measured on an OPPO Reno7 (Dimensity 900, 8 GB), with a 4k context and a
2,881-token reading prompt. Peak RSS is the larger of dumpsys meminfo TOTAL RSS and VmHWM.
|
LiteRT-LM 0.17.1, CPU |
LiteRT-LM 0.17.1, GPU (fp32 + MTP) |
forked engine, CPU (mfa/) |
| peak RSS, warm |
4.58 GB |
3.0 GB, of which 2.66 GB is GPU driver memory; VmHWM reaches 5.5 GB during init |
2.65 GB |
| peak RSS, first run (builds the cache) |
5.04 GB |
3.06 GB; 5.5 GB during init |
≈ 2.7 GB (3.01 GB before the table release) |
| prefill |
41 tok/s |
16–27 tok/s |
43 tok/s |
| decode |
~3.5 tok/s |
~1–2.5 tok/s |
~4 tok/s |
Use the CPU only, with the forked engine. It is the only configuration that fits a 3 GB budget. The engine drops the clean embedding-table pages every 8 tokens, which cut the peak from 2.97 to 2.65 GB with identical output. The GPU column is for reference: on this phone the GPU is slower than the CPU and needs more memory.
Use
The model expects the agent protocol, not free-form prompts. It reads with a 4k context and opens one
fresh session per window of about 1,500 transcript tokens, with the compacted journal at its head. The
model answers with at most 5 lines NOTE [time] (TYPE) text, then NEXT. See §4 and §12 of the
archived mobile-v1 integration note.
With the forked engine. Use the fork
vieenrose/LiteRT-LM, branch
mobile-fused-attention, contrib/mobile_fused_attention/. Build it, then run:
mfa_engine --serve --dir mfa --main mfa/prefill_decode_fused.tflite --fused \
--ctx 4096 --threads 8 --weight-cache model.wcache
The engine builds the XNNPACK weight cache on its first run, about 2.2 GB on disk.
With stock LiteRT-LM (reference only). Pass the .litertlm file with Backend.CPU(threadCount = 8) and
maxNumTokens = 4096. This needs 4.6 GB.
About 1 statement in 9 is contradicted by the transcript, mostly the right figure attached to the
wrong object. Check decisions and figures before relying on them.
No training data is included. The source recordings are subject to the Legislative Yuan IVOD terms of
use. The weights are derived from Gemma 4 (Apache-2.0).
LoRA (qat_lora, safetensors)
lora/qat_lora.safetensors is the LoRA delta behind gemma-4-E4B-meeting-agent-zh-mobile-v1.litertlm (sha256 d4b626697750a1d90ad1602a49b89e34a9ad851e0dff500f18b25175a87ee5c1). It is applied to Google's Gemma-4-E4B mobile graph, not to the Hugging Face checkpoint, and it is in the training format (QAT), so it is not a standard PEFT adapter. Merge it into the mobile base to reproduce the LiteRT file; the file itself is the one to deploy.
