On-device next-word prediction for texts and chat.
A small English next-word and word-completion model trained by Luke Steuber
for LocalType, an Android keyboard. It has 2,025,114 parameters, a 16,384-token
word-level vocabulary, and a 2.22 MB int8 binary. It suggests words;
it is not a chat model or an independently validated autocorrect system.
The model reads the current sentence, with up to 63 context tokens plus a start
token. Longer contexts move in 32-token increments. A coupled input-forget-gate
LSTM projects 670 hidden units to 96 dimensions and shares its input/output
embedding. Kotlin inference requires no external model runtime; the included
Python reference requires only NumPy. This custom binary does not load through
Transformers AutoModel.
Use
Use Python 3.12 in a virtual environment. The commands below use a POSIX shell;
on Windows, activate the environment with .venv\Scripts\activate instead.
The requirements file pins NumPy 2.5.3, the verified reference version.
python3.12 -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade huggingface_hub
hf download lukeslp/chime-sms-2m --local-dir chime-sms-2m
cd chime-sms-2m
python -m pip install -r requirements.txt
python inference.py "see you "
Or import LocalTypeModel from inference.py and call
model.predict("see you ", k=6). A trailing space completes the last word;
without it the last word is treated as a prefix. Hugging Face Hub is only needed
to download the package; inference uses NumPy alone and is offline.
To try it on Android, install LocalType 0.5.3 or later and turn on Neural predictions (experimental) in typing settings, then open a new text field. The model is bundled and runs on a dedicated worker. Ordinary predictions remain available while it scores. The separate optional Gemini Nano/Gemma provider handles larger-model suggestions and proofreading.
The archive contains float32 and int8 weights, vocabulary, reference inference,
checksums and provenance. It does not contain corpus text, personal typing logs,
personalized weights, or optimizer state. It cannot resume training.
Training
The selected checkpoint is step 98,000 from a completed 102,195-step
run with three batcher epochs and 586,885,109 target-token exposures.
Repeated exposures are not unique tokens. Training used an A100 with PyTorch
float32, batch size 256 and sequences capped at 64 tokens. Selection used
human-macro development perplexity, 60.5425 at the selected checkpoint.
×8 means NUS SMS, Taskmaster-1 and OpenAssistant prompter examples repeat eight
times in the training mix; Taskmaster-3 and synthetic SODA remain ×1.
It does not mean eight times the parameters. SODA still dominates the corpus.
Both ×4 and ×8 have the same architecture and vocabulary size. The earlier ×4 checkpoint is retained privately for comparison and rollback; this repository distributes the completed ×8 checkpoint.
| Training setting |
Value |
| Architecture |
CIFG-LSTMP, tied input/output embeddings |
| Vocabulary |
16,384: 6 specials, 31 punctuation, 293 emoji, 16,054 words |
| Embedding / projection / hidden size |
96 / 96 / 670 |
| Optimizer |
AdamW with the reference MLX update reproduced in CUDA |
| Learning rate |
0.002; 200-step warmup; cosine decay to 0.1 × peak |
| Weight decay / gradient norm clip |
0.01 / 1.0 |
| Random seed |
20260925 |
| Batch / sequence limit |
256 / 64 tokens |
| Training hardware / arithmetic |
NVIDIA A100 / float32 |
| Planned updates / completed updates |
102,195 / 102,195; stop reason max_steps |
| Selected checkpoint |
Step 98,000; human-macro development perplexity 60.5425 |
| Training wall time |
7,342.98 seconds (2h 2m 23s) |
The fixed vocabulary is word-level, rather than a large subword tokenizer. Unknown words map to its unknown token; it cannot invent new dictionary entries. Personal learning in LocalType is a separate component. The input limit counts model tokens, including punctuation and emoji, rather than characters or bytes.
Sources, revisions, filtering, exposure and licenses are in DATA.md,
NOTICE.txt and data_provenance.json.
Measured development comparison
Both models were scored with the same Kotlin int8 implementation and development
lines. Potential keystroke savings assume accepting a useful top-three suggestion
at its earliest prefix; these are not observed typing savings.
| Set |
Previous ×4 |
This ×8 |
Difference, percentage points (95% interval) |
| NUS SMS, 463 lines |
40.70% |
41.44% |
+0.73 [+0.40, +1.07] |
| Taskmaster-1, 567 lines |
58.61% |
59.50% |
+0.89 [+0.62, +1.17] |
| Mostly synthetic sample, 5,000 lines |
65.04% |
64.93% |
-0.12 [-0.17, -0.06] |
Differences use unrounded scores and 2,000 paired line-bootstrap resamples.
Next-word hit@3 improves from 18.14% to 19.32% on SMS and 39.87% to 42.34% on
Taskmaster. The pooled value declines from 51.76% to 51.63%.
Checkpoints were selected on development data. NUS senders can cross splits,
and resampling lines understates dialogue/sender clustering. Backend and update
counts also differ between these completed runs, so this comparison does not
isolate the effect of oversampling. No held-out test result is claimed here.
Export and device measurements
On 5,000 development lines, perplexity was 12.985256 in float32 and 12.998372
after int8 dequantization, a 0.101% increase. Kotlin matched all 120 reference
contexts and their top-ten rankings; maximum logit error was 0.0000127.
Standalone NumPy matched the same contexts with finite outputs. NumPy 2.5.3
produced no warnings on the validation Mac; 2.2.6 emitted arithmetic warnings
despite matching outputs. That older environment is not recommended for this
reference package without further investigation.
Each phone benchmark used 600 ordinary contexts across three passes, plus
30 long-context stress sentences. Warm passes 2–3 pooled, p95 milliseconds:
| Path |
Pixel 9a |
Pixel 10 |
| Fresh context, next word |
6.675 |
7.578 |
| Retained context, next word |
3.088 |
3.235 |
| Retained context, prefix |
0.191 |
0.235 |
| Long-context window reset subset |
14.134 |
17.638 |
Both ran Android 17 / API 37. Pixel 9a thermal status remained 0; Pixel 10 went
from 0 to 1 and changed CPU cores during stress. These runs are not a controlled
comparison of the phones. First calls took 18.941 ms and 22.952 ms respectively.
These are standalone model timings, not physical input latency, battery results
or everyday keyboard acceptance.
Limits and release status
English only. The corpus includes older Singapore texting, assistant prompts,
transactional dialogues and predominantly synthetic dialogue. Filtering is not
proof of anonymization. The vocabulary is fixed and does not acquire new slang
or personal words through this reference model. Development results do not
establish general conversation ability or everyday typing accuracy.
Neural predictions remain experimental and off by default in LocalType.
Real typing acceptance is pending. This release does not claim a passed everyday typing comparison.
License and attribution
Weights and vocabulary are CC BY-SA 4.0. The standalone inference.py and
localtype_tokenizer.py are Apache-2.0, copyright Luke Steuber. See
LICENSE.md for file scopes and NOTICE.txt for upstream
credits. Model-card metadata identifies the weights' license; it does not
replace the separate code license. The source authors do not endorse this model.
This inference package includes training configuration, selected-checkpoint
metrics and provenance. The training pipeline and corpus are not included;
it is not a claim of bit-for-bit training reproduction. Inspect SHA256SUMS
and package_manifest.json for the exact release files.
Citation
@misc{steuber2026chimesms2m,
author = {Luke Steuber},
title = {Chime-SMS-2M: An On-Device English Keyboard Prediction Model},
year = {2026},
url = {https://huggingface.co/lukeslp/chime-sms-2m}
}
Model SHA-256: 3255dbeaf6e7fc0922afdca42ef751a22e3ad28903ab4acc9eb0022b9a704318.
