A QLoRA adapter for OpenAI's gpt-oss-20b that turns it into KinShield's phone-call evidence detector. KinShield is an entry in the AWS "Zero to Shipped" hackathon. The model reads a call transcript and returns strict JSON with:
- a risk level (LOW / MEDIUM / HIGH) and a score
- every warning sign, with the exact quoted line, a timestamp, one of 7 fixed signals and a weight
It answers directly on the harmony final channel, without writing out reasoning first.
Status: this is a research result. The live KinShield app still uses the stock gpt-oss-20b on Amazon Bedrock. This adapter is not hosted anywhere public.
Results: 63 hand-written benchmark calls (31 scams, 32 safe), none used in training
| Model |
Correct |
Scams rated HIGH |
Scams flagged (MEDIUM or HIGH) |
Safe calls flagged |
| gpt-oss-20b, stock (Bedrock, same prompt) |
61/63 |
29/31 |
31/31 |
0/32 |
| gpt-oss-120b, stock (Bedrock, reasoning medium) |
62/63 |
30/31 |
31/31 |
0/32 |
| gpt-oss-20b + this LoRA |
62/63 |
30/31 |
31/31 |
0/32 |
The adapter matches gpt-oss-120b, a model about 6× larger, and edges out the stock 20b by one call (scam_031). On the original 47-call set, all three score 46/47. The gain is small and the test set is small, so read it as "matches 120b", not "beats it".
This model then taught KinShield-Tiny v3, a 22M int8 classifier. v3 improved from 57/63 to 60/63 over v2, which learned from the stock 20b.
How it was made
- Data. gpt-oss-20b on Bedrock generated 1,565 synthetic calls: scams, hard negatives and ordinary calls, in English and Hinglish.
- Stronger labels. gpt-oss-120b (reasoning medium) labelled every call with KinShield's detector prompt and the same server-side normalisation.
- Intent filter. A label was kept only if it agreed with what the call was written to be: scams must score MEDIUM or HIGH, and safe calls must score LOW. The 120b label was used first, with the 20b label as a fallback. 151 calls that neither model labelled correctly were dropped, leaving 1,201 calls for training and 213 for validation.
- QLoRA. Trained with Unsloth on an NVIDIA DGX Spark (GB10):
- 4-bit base model, LoRA rank 16 (alpha 32) on
q/k/v/o and the MoE gate_up_proj/down_proj
- 2 epochs, learning rate 2e-4 with a cosine schedule, effective batch 8, maximum length 3,072
- loss on the assistant answer only
Training took 90 minutes and used about 19 GB. Validation loss fell 0.049 → 0.039 → 0.037 → 0.035.
Use
from unsloth import FastLanguageModel
model, tok = FastLanguageModel.from_pretrained("Solomonwilsonr/kinshield-gpt-oss-20b-lora",
max_seq_length=3072, load_in_4bit=True)
FastLanguageModel.for_inference(model)
SYSTEM = open("system_prompt.txt").read() # KinShield detector prompt, included in this repo
prompt = tok.apply_chat_template(
[{"role": "system", "content": SYSTEM},
{"role": "user", "content": "[00:00] caller: Grandma, it's me...\n[00:05] victim: Leo?"}],
tokenize=False, add_generation_prompt=True, reasoning_effort="low") + "<|channel|>final<|message|>"
out = model.generate(**tok(prompt, return_tensors="pt", add_special_tokens=False).to("cuda"),
max_new_tokens=800, do_sample=False)
Important: append <|channel|>final<|message|> to the generation prompt, exactly as in training. Without it, the model may start a short analysis channel first.
Limits
- Synthetic data only. All training data is synthetic and LLM-labelled; there are no real calls. The test set is 63 calls written by the same team, so a single call moves accuracy by about 1.6 points.
- Slower than the hosted model. On the DGX Spark (HF generate, 4-bit, batch 8) it takes about 5 s per call. The stock model on Bedrock answers in under 1 s.
- Text only. It handles English and Hinglish transcripts. It doesn't detect voice cloning; it reasons about what the caller asks for.
- Not a safety guarantee. It is a hackathon research prototype. Don't use it to make decisions about real people.
