Qwen3.5-2B tool router v2 (q4f16_1, MLC / WebLLM)
A fine-tune of Qwen/Qwen3.5-2B that works as a tool router in pure mode: it answers every user message with exactly one tool call and writes no text. The app writes every sentence. It runs in the browser with WebLLM on WebGPU. Download: 1.1 GB.
This is the f16 build: it needs the WebGPU shader-f16 feature. The f32 build (tfukkwang/Qwen3.5-2B-tool-router-v2-q4f32_1-MLC) has the same weights and runs on any WebGPU.
Demo: tfukkwang/credit-copilot-demo
What the model does
Every reply is one <tool_call>:
- Data requests: the app's list, count, record and chart tools, with filters taken from the user's words.
- Judgement and advice: the app's expert tool.
- Actions (approve, cancel, ...): only when the user explicitly asks.
- Greetings, thanks, goodbyes, "what can you do": small_talk.
- Requests that have nothing to do with the app: unsupported.
- A request that needs an id the user did not give: ask_user.
The app should force one tool call per reply with a grammar (WebLLM structural tags), as the demo does.
Training
- Data: 3,000 generated pure-mode conversations about 30 made-up apps (orders, tickets, clinics, fleets, ...). Each app has its own tool names, arguments and enums, so the model learns tool routing in general, not one app.
- Method: LoRA (r=16) SFT on the stock Qwen3.5-2B, 2 epochs, bf16.
- Conversion: merged weights converted to MLC
q4f16_1 for the official Qwen3.5-2B WebGPU library. No new compilation.
Evaluation
Browser test on four apps that were not in the training data (credit underwriting, support desk, leave requests, shop orders; 82 turns): 74/82 router alone, 76/82 after the app's code guards. The guards are small code fixes in the app (for example, a missing id is asked for instead of guessed).
Usage with WebLLM
import { CreateMLCEngine, prebuiltAppConfig } from "https://esm.run/@mlc-ai/[email protected]";
const appConfig = {
model_list: [...prebuiltAppConfig.model_list, {
model: "https://huggingface.co/tfukkwang/Qwen3.5-2B-tool-router-v2-q4f16_1-MLC",
model_id: "Qwen3.5-2B-router",
model_lib: "https://raw.githubusercontent.com/mlc-ai/binary-mlc-llm-libs/main/web-llm-models/v0_2_84/base/Qwen3.5-2B-q4f16_1_cs1k-webgpu.wasm", required_features: ["shader-f16"],
}],
};
const engine = await CreateMLCEngine("Qwen3.5-2B-router", { appConfig });
Put the tools in the system prompt in Qwen's format (<tools>...</tools>) and parse <tool_call> blocks. Use greedy decoding (temperature: 0) and extra_body: { enable_thinking: false }.
Limitations
- Routing only. It is not a chat model.
- It still sometimes drops a requested filter or sort order, or counts when the user asked for a list. Check the call in code where it matters.
- Never let it run action tools without the user's explicit words; check that in the app.
