Open-weight model
qwen3-4b-tau2-telecom-fp120-1ep
by MinseokKang MANGSEOK123/qwen3-4b-tau2-telecom-fp120-1ep
qwen3-4b-tau2-telecom-fp120-1ep is an open-weight model from MinseokKang, released under Apache License 2.0. It has 4.4B parameters and a 262,144-token context. At 16-bit it needs about 10.6 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
Qwen3-4B-Instruct-2507 consolidated with OEL on the tau2-bench telecom domain, using 120 externally authored task–memory pairs from the revised finalpairs2 set, stratified down from 125 so the batches divide evenly.
Runs On
What it takes to serve qwen3-4b-tau2-telecom-fp120-1ep (4.4B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 8.8 GB | 10.6 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 4.4 GB | 5.3 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 2.2 GB | 2.6 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 19, 2026.
qwen3-4b-tau2-telecom-fp120-1ep on every accelerator the SAVRN Index prices, at every precision
Model Card
By MinseokKang, published under apache-2.0, revision 4d254a8de329.
Qwen3-4B-Instruct-2507 consolidated with OEL on the tau2-bench telecom domain, using 120 externally authored task–memory pairs from the revised finalpairs2 set, stratified down from 125 so the batches divide evenly. The student replays each task without memory; the teacher is the same weights with that task's memory in the system prompt. Only the prompt differs, and no reward is used. Five pairs were dropped from the 125 by stratified sampling (seed 42), holding the level ratios to within 1pp, so that no batch is short — earlier runs showed a tiny final batch wrecks the saved checkpoint, and only the last step is saved. Each step reads a different batch, so the loss column tracks batch…
Read MinseokKang's full model card
Qwen3-4B-Instruct-2507 — tau2-bench telecom, final_pairs_2 (120)
Qwen3-4B-Instruct-2507 consolidated with OEL on the tau2-bench telecom domain, using
120 externally authored task–memory pairs from the revised final_pairs_2 set,
stratified down from 125 so the batches divide evenly.
Sibling runs: telecom-fp84-1ep, telecom-compsub-1ep, telecom-ours-1ep, telecom-oel-1ep.
Training
| base model | Qwen3-4B-Instruct-2507 |
| training pairs | 120 (composition 56 / recall 38 / substitution 26) |
| batch / steps | 12 / 10 (1 epoch, every batch full) |
| learning rate | 3e-6, constant |
| gradient clipping | 1.0 (verl default) |
| loss | full KL over all response tokens, kl_topk 256 |
| user simulator | gpt-4.1-mini (temperature 0) |
The student replays each task without memory; the teacher is the same weights with that task's memory in the system prompt. Only the prompt differs, and no reward is used. Five pairs were dropped from the 125 by stratified sampling (seed 42), holding the level ratios to within 1pp, so that no batch is short — earlier runs showed a tiny final batch wrecks the saved checkpoint, and only the last step is saved.
Training trace
| step | KL loss | entropy | grad norm |
|---|---|---|---|
| 1 | 0.024 | 0.341 | 5.654 |
| 2 | 0.015 | 0.318 | 1.687 |
| 3 | 0.014 | 0.307 | 12.898 |
| 4 | 0.019 | 0.474 | 11.900 |
| 5 | 0.054 | 0.385 | 7.949 |
| 6 | 0.023 | 0.362 | 3.851 |
| 7 | 0.020 | 0.394 | 11.133 |
| 8 | 0.021 | 0.408 | 6.128 |
| 9 | 0.049 | 0.345 | 7.914 |
| 10 | 0.034 | 0.347 | 4.820 |
Each step reads a different batch, so the loss column tracks batch difficulty rather than convergence. The gradient norm sits above the 1.0 clip on every step — telecom episodes are long, so the effective step size is set by the clip rather than by the learning rate.
Evaluation
Not evaluated. Pushed straight after training. On tau2-bench telecom test (40 tasks,
4 trials, no memory at eval) the base model scores avg 0.056 / pass@4 0.175; other runs
in this series land between 0.050 and 0.081, all within about 1.5 SE of each other.
Known caveat. A substitution pair in this family reproduced a test task's exact
environment state by dropping a fault from a training task; task ids still carry the
training signature, so only an initial_state comparison reveals it.
Usage
vllm serve MANGSEOK123/qwen3-4b-tau2-telecom-fp120-1ep \
--enable-auto-tool-choice --tool-call-parser hermes --max-model-len 40960
Configuration
- Architecture
- Qwen3ForCausalLM
- Context length (tokens)
- 262,144
- Layers
- 36
- Hidden size
- 2,560
- Feed-forward size
- 9,728
- Attention heads
- 32
- Key/value heads
- 8
- Head dimension
- 128
- Vocabulary size
- 151,936
- RoPE base
- 5,000,000
- Stored precision
- bfloat16
- Model type
- qwen3
Identity and Version
- Repository
- MANGSEOK123/qwen3-4b-tau2-telecom-fp120-1ep
- Publisher
- MinseokKang
- Task
- Not stated by the source
- Modality
- Other
- Library
- Not stated by the source
- Parameters
- 4.4B parameters
- Languages
- oel
- Revision
- 4d254a8de3296245de959a41eca1343e393cfedc
- First published
- 2026-09-22
- Last updated
- 2026-09-22
Files and Weights
14 files, 8.8 GB in total. The weights are 2 files totalling 8.8 GB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model-00001-of-00002.safetensors | Weights | 5.0 GB | 0e213082324e |
| model-00002-of-00002.safetensors | Weights | 3.8 GB | a4a556c7961f |
| added_tokens.json | Configuration | 707 B | — |
| config.json | Configuration | 728 B | — |
| generation_config.json | Configuration | 121 B | — |
| model.safetensors.index.json | Configuration | 32.9 KB | — |
| special_tokens_map.json | Configuration | 613 B | — |
| README.md | Documentation | 2.9 KB | — |
| chat_template.jinja | Other | 2.6 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| merges.txt | Tokenizer | 1.7 MB | — |
| tokenizer.json | Tokenizer | 11.4 MB | aeb13307a71a |
| tokenizer_config.json | Tokenizer | 5.4 KB | — |
| vocab.json | Tokenizer | 2.8 MB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 8.8 GB
Released by MinseokKang through its official repository on Hugging Face. Read the license.
Built From
- Derived from Qwen/Qwen3-4B-Instruct-2507
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 8.8 GB |
| 16-bit | 8.8 GB |
| 8-bit | 4.4 GB |
| 4-bit | 2.2 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About qwen3-4b-tau2-telecom-fp120-1ep
How much GPU memory does qwen3-4b-tau2-telecom-fp120-1ep need?
About 10.6 GB at 16-bit and 2.6 GB at 4-bit: the weights (4.4B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run qwen3-4b-tau2-telecom-fp120-1ep on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use qwen3-4b-tau2-telecom-fp120-1ep commercially?
Yes. qwen3-4b-tau2-telecom-fp120-1ep is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is qwen3-4b-tau2-telecom-fp120-1ep's context length?
262,144 tokens, from the maximum position embeddings in its published configuration.