Qwen3.8-27b-Terse-Coder-LoRA

Rank-16 DPO LoRA that shortens chain-of-thought reasoning on coding tasks while preserving correctness. Trained on preference pairs selected objectively — concise-but-correct traces chosen by automated test execution + entropy-based step pruning, no human or LLM judging.

Actively researched and improving. This is round 7 of an ongoing iteration; expect updated adapters on this page as the study continues. (Round 6 is archived under archive/r6/.)

The effect is compounding: it stacks on top of whatever conciseness the base already has. Recommended pairings, in order:

1. Qwen/Qwen3.8-27B (full precision) or nvidia/Qwen3.8-27B-NVFP4 — stock base. This is where round 7 shines: −94.7% reasoning tokens with pass rate intact. Stock Qwen is the most verbose base we tested, so the cut is largest there.
2. agentionai/Signal-3.8-27B (full precision) or Shockem/Signal-3.8-27B-NVFP4 — the training-lineage base; −40% on top of Signal's already-short reasoning, and pass rate improves.
3. Shockem/Signal-3.8-27b-Heretic-ara (full precision) or Shockem/Signal-3.8-27b-Heretic-ara-NVFP4 — optional: the original training target. Round 7 cuts deeper (−42%) but costs ~2.5 pts pass on this base; if you run heretic-ara and want the pass-neutral cut, round 6 in archive/r6/ (−36%, pass intact) is the better fit for that specific base.

Results

All testing was done on NVFP4-quantized bases (modelopt W4A16) served by vLLM 0.28 on 3× RTX 5060 Ti. Held-out 40 problems (20 HumanEval + 20 MBPP-sanitized, disjoint from training), n=1 per run, sampling temp 0.6 / top_k 20 / top_p 0.95 / rep-penalty 1.05, pass@1 by automated test execution in a sandboxed container, reasoning tokens from completion_tokens_details.reasoning_tokens. Headline metric: totals of per-problem median-of-runs, paired permutation test (single runs are heavy-tailed — run means swing ±20%).

Effect size varies by base quant calibration. NVIDIA's checkpoint (and our v3 recipe, which replicates its Local-Hessian solve on MLP/lm_head) shows the full ~−93% cut; an absmax-calibrated quant of the same weights (v2) only cut −49.5%. Direction and pass-neutrality hold on every base tested. If you build your own W4A16 quant to pair with this adapter, Hessian-weighted calibration on MLP/lm_head is what unlocks the full effect.

Prefer a merged model over a runtime adapter? Round 7 is also available pre-merged into stock Qwen3.8-27B as Shockem/Qwen3.8-27b-Terse-Coder (fp16) and Shockem/Qwen3.8-27b-Terse-Coder-NVFP4 (v3-recipe W4A16): 67.5% pass with ~38 reasoning tokens/problem baked in — same behavior, zero LoRA plumbing. Do not stack this adapter on top of the merged model (double application over-shortens: 63% pass with no_code failures in our testing).

On heretic-ara specifically, round 6 (archive/r6/) is the pass-neutral option (−36.0%, 62.5% → 63.7%). On stock and Signal, round 7 dominates.

The merged version of this round (Shockem/Qwen3.8-27b-Terse-Coder) also held 78.3% on GPQA-Diamond (PhD-level science, far outside the coding training distribution) at ~1.5k mean reasoning tokens — evidence the preference cuts the deliberation budget, not capability.

Reference points from the same harness: stock base alone is 72.5% / 701 reasoning tokens; with the adapter it answers coding problems near-directly (mean 38 reasoning tokens) and holds 70.8%.

Serve it with speculative decoding (recommended)

The adapter does not touch the MTP draft head, so MTP speculative decoding keeps working at full acceptance (measured 0.43 with and without the adapter on the NVIDIA checkpoint — outputs are target-verified, so spec decode is lossless). Measured on 2× RTX 5060 Ti 16 GB (vLLM 0.28, FP8 KV, num_speculative_tokens: 3):

Turn spec decode ON — it is +78% wall speed with the adapter loaded. On 2×16 GB cards with the adapter + spec, cap context at ~175k (200k needs more KV headroom than the two cards have; single-card 24 GB+ rigs are unaffected).

Runtime LoRA is the full-strength deployment form

Round-7's weight deltas are deliberately tiny (‖Δ‖/‖W‖ ≈ 4e-4–1e-3) — below bf16's per-element resolution. Measured delta survival when merging into the base: 31–61% in bf16, 94–99.9% in fp16, and merged-to-NVFP4 attenuates on top of that. So: load the adapter at runtime on any base quant (the delta applies in bf16 at compute time — full strength). If you merge, merge in fp32 and store fp16; any merged 4-bit artifact loses some of the effect.

Variants

All variants of the adapter live on this page; pick per your stack. Note that every number above was measured with the bf16 adapter on NVFP4 bases.

Direct downloads:

Or grab a whole variant folder with the CLI: huggingface-cli download Shockem/Qwen3.8-27b-Terse-Coder-LoRA --include "fp8/*".

The bf16 adapter sits at the repo root (standard PEFT layout, powers the Hub's "Use this model" snippet). Only bf16 is directly loadable by serving runtimes today (vLLM 0.28's LoRA path takes bf16). The quantized variants are compact/research artifacts.

GGUF note: converted with llama.cpp master's convert_lora_to_gguf.py (targeting a Qwen3.5 hybrid gated-deltanet base — needs a llama.cpp build new enough for qwen3_5), with a small patch for the linear-attention out_proj column reorder on LoRA tensors. The bf16 GGUF is verified bit-exact against the bf16 PEFT adapter on all 992 tensors, including the grouped→tiled V-head reorders llama.cpp applies to the linear_attn projections; q8_0 verified at dequant cosine ≥ 0.99997. Not yet runtime-tested against a GGUF base model — treat as experimental. There is no q4 GGUF: llama.cpp's LoRA-adapter format only defines f32/f16/bf16/q8_0 — for 4-bit use the int4/ safetensors variant. Usage: llama-server -m <qwen3.8-27b-base>.gguf --lora Qwen3.8-27b-Terse-Coder-LoRA-q8_0.gguf.

Serving with vLLM (tested path)

Tested on vLLM 0.28, including alongside MTP speculative decoding and on modelopt NVFP4 bases. The released adapter carries vLLM-native (language_model.model.layers.*) key layout — it attaches correctly to the Qwen3_5ForConditionalGeneration module tree. (Plain PEFT exports from a text-only AutoModelForCausalLM run silently attach zero weights in vLLM — basename checks pass, nothing is applied. If you re-export this adapter yourself, keep the VL-layout keys.)

CLI:

vllm serve nvidia/Qwen3.8-27B-NVFP4 \
  --enable-lora \
  --lora-modules cot-lora=/path/to/bf16 \
  --max-lora-rank 16 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

Then request with model: "cot-lora" (adapted) or the base model id (unadapted) — both are live on the same server:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")
resp = client.chat.completions.create(
    model="cot-lora",  # terse reasoning
    messages=[{"role": "user", "content": "Write a Python lru_cache."}],
    temperature=0.6, top_p=0.95,
)
print(resp.usage.completion_tokens_details.reasoning_tokens)

Hot-loading on an already-running server also works: POST /v1/load_lora_adapter {"lora_name": "cot-lora", "lora_path": "..."}.

Other bases, quants, formats, backends

The adapter is a behavioral edit to the shared Qwen3.8-27B text stack (all attention + MLP + gated-deltanet linear_attn projections; no vision, no MTP weights), so it should stack fine on other quantizations (FP8/INT4/GGUF bases), full-precision bases, and other backends (SGLang, TabbyAPI/EXL3, llama.cpp) — the preference it encodes is not quantization-specific. That said, only the NVFP4 + vLLM combination above has been measured; treat other combinations as untested and validate before relying on them.

Training (summary)

• Data: coding prompts (HumanEval + MBPP-sanitized, disjoint from eval), n=4 traces each at temp 1.0 with per-token logprobs, from three policies (tri-base round): heretic-ara, plain Signal, and NVIDIA stock — each with the round-6 adapter active (on-policy continuation). Traces filtered by automated test execution; passing traces segmented into steps and pruned by per-step mean token entropy → 525 DPO pairs: (pruned reasoning, same answer) vs (full reasoning, same answer).
• Method: DPO (trl 1.13), β 0.05, lr 1e-5 cosine, 3 epochs, eff. batch 8, initialized from the round-6 adapter so the reference anchors at base+r6 and only further shortening is rewarded.

Caveats

• Targeted at coding tasks with thinking enabled. Behavioral LoRA, not a knowledge edit.
• The preference is "shorter reasoning, identical answer" — if a task needs long derivation, raise reasoning_effort as usual.
• If you serve with speculative decoding, make sure the generation config has no min_p — vLLM 0.28 rejects min_p under spec decode.

Attributions & licenses

This adapter is trained against, and licensed for use with, Qwen/Qwen3.8-27B, © Qwen Team, Alibaba Cloud, Apache 2.0; this adapter is likewise Apache 2.0 and the upstream license and copyright notices are retained. Credits:

• Qwen Team (Alibaba Cloud) — the Qwen3.8-27B base model (Apache 2.0).
• agentionai — Signal-3.8-27B, one of the three trace-generation policies.
• p-e-w — the Heretic tool; a heretic-ara abliterated variant of Signal was another trace-generation policy.
• NVIDIA — Qwen3.8-27B-NVFP4 (third policy and eval baseline) and TensorRT Model Optimizer 0.45 (Apache 2.0, quant tooling).
• OpenAI — HumanEval (MIT), and Google — MBPP (CC-BY 4.0): prompt sources for training and held-out evaluation.
• Hugging Face TRL (Apache 2.0) — the DPO trainer; llama.cpp (MIT) — GGUF adapter conversion; Datacurve — DeepSWE, independent evaluation only.

None of these parties endorse this adapter; all remaining errors are ours.