Ziyi LI

This is a 0.8B model we fine-tuned to make one narrow kind of decision very cheaply. It reads a state, a question, and a list of options, then returns a probability for each option. It never writes text. One forward pass, one token, and you have a distribution your code can branch on. We built it for our own product first. An educational chatbot needed to decide, before spending a full LLM call, whether a student message was on topic, whether a tool call was safe to run, and whether an old tool result still had to be kept verbatim in the context. Those are small bounded judgments, and paying a 70B model to make them was absurd. It runs today on an office GPU at 80 to 150 ms per decision…