finai-slm-v2

A 1.7B on-device language model for Indian bank SMS parsing and personal finance coaching.

FinAI SLM is a Qwen/Qwen3-1.7B derivative fine-tuned with QLoRA SFT for the FinnDot expense tracker app. It runs entirely on-device — no cloud calls, no user data leaves the phone.

What it does

The model handles English, Hindi, Hinglish, Tamil, Telugu, Marathi, and Bengali bank SMS.

Key results (v2 held-out eval, 1,282 test SMS)

SHIP: YES. All gates pass. McNemar p = 7.6e-270.

v1 → v2: R-EM +5.2 pp, chat groundedness +36 pp, false-parse 28.57% → 0.54%.

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "finndot/finai-slm-v2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
)

# SMS parsing
messages = [
    {"role": "system", "content": "Extract transaction details from this SMS as JSON: {amount, merchant, type, account, balance, category}. type is one of: INCOME, EXPENSE, CREDIT, TRANSFER, INVESTMENT. If this is not a transaction (OTP, promo, KYC), return {}."},
    {"role": "user", "content": "HDFC Bank: Rs.499.00 debited from A/c XX1234 to SWIGGY via UPI. Avl Bal Rs.15000.50"}
]

text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True,
    enable_thinking=False
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=256, do_sample=False)
result = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(result)
# {"amount": 499.0, "merchant": "SWIGGY", "type": "EXPENSE", "account": "1234", "balance": 15000.5, "category": "Food"}

Training details

Data recipe (privacy-safe)

No user SMS. No real financial data. Fully synthetic.

Split hygiene: assignment unit is (bank, template_id), not individual SMS. All variants of a template stay in one split. See ml/data/manifest/SHA256SUMS.

Evaluation protocol

Pre-registered before looking at test scores: docs/llm-eval-protocol.md

Ship gates

1. SMS R-EM lift vs Qwen3-base ≥ +3 pp, CI excludes 0
2. Amount EM drop vs base ≤ 1 pp
3. Chat groundedness drop vs base ≤ 5 pp
4. JSON validity ≥ 95%
5. On-device TTFT / peak RSS ≤ 1.3× old Qwen2.5

Limitations

• Indian bank SMS only — trained on Indian banking formats (UPI, NEFT, IMPS, etc.). May not generalize to US/EU bank SMS.
• Merchant extraction is the hardest field; regex parsers in parser-core still win on supported banks.
• Chat groundedness is moderate (~32%); the model sometimes paraphrases numbers instead of citing exact figures.
• Thinking is disabled — do not expect chain-of-thought reasoning.
• 1.7B parameter model — smaller than general-purpose assistants; optimized for the SMS+coach task mix.

On-device deployment

The production format is LiteRT-LM INT4 (.litertlm, ~0.9 GB). The bf16 safetensors here are the merge before quantization. For the quantized on-device file, see the FinnDot app's model download.

Conversion: litert-torch export_hf, dynamic_int4_block32, KV 1280, nothink prefix.

Reproduce

git clone https://github.com/AnomaProAI/Finndot
cd Finndot/ml

# Build dataset
python -m data.build_splits --repo-root .. --out data/out --train-size 16000

# Train (needs 1× A10G / T4 GPU)
python train/sft_qlora.py --config train/train_config.yaml

# Merge adapter
python train/merge_lora.py --adapter <output>/adapter --out <output>/merged

# Evaluate
python -m eval.run_eval \
  --backend hf \
  --sms data/out/test_sms.jsonl \
  --chat eval/fixtures/chat_eval.jsonl \
  --model-id <output>/merged \
  --baseline-id Qwen/Qwen3-1.7B \
  --tag my-eval

Supported banks (35+)

HDFC, SBI, ICICI, Axis, Kotak, PNB, Bank of Baroda, Union Bank, IDFC First, Canara, Federal, Indian Bank, IOB, Central Bank, Karnataka, South Indian, City Union, DBS, HSBC, IDBI, J&K Bank, AMEX, OneCard, Slice, LazyPay, Juspay, Airtel Payments, Jio Payments, IPPB, Jupiter, Utkarsh, Mashreq, and more.

Citation

@misc{finai-slm-2026,
  title={FinAI SLM: On-device Indian bank SMS parsing with QLoRA fine-tuning},
  author={FinnDot Team},
  year={2026},
  url={https://huggingface.co/finndot/finai-slm-v2},
  note={Apache 2.0. Based on Qwen3-1.7B.}
}

License

Apache 2.0 — same as the Qwen3-1.7B base model.