Janus-35B-HERETIC

Janus-35B

Flagship Reasoning. Sparse Footprint. Uncensored. llmfan46's Heretic abliteration of Qwen 3.6 35B-A3B, repackaged with Claude Fable 5 in the teacher slot.

Architecture: Qwen 3.6 35B-A3B (MoE) | Total Params: ~34.7B | Active Params: ~3B | Context: 262,144 native / 1,010,000 extensible | Base: Heretic (llmfan46) | Teacher: Claude Fable 5 | Type: Distilled + Abliterated MoE LLM

A personal fork of llmfan46/Qwen3.6-35B-A3B-uncensored-heretic — an uncensored Heretic-style abliteration of Qwen/Qwen3.6-35B-A3B, the 35B-total / 3B-active mixture-of-experts multimodal base — repackaged as Janus-35B with Claude Fable 5 reasoning data in the teacher slot. Refusal-trained behavior is dialed back at the base layer.

Provenance. This repo repackages — no training and no quantization is performed here. What ships is llmfan46's own Q4_K_M of llmfan46/Qwen3.6-35B-A3B-uncensored-heretic, byte-for-byte, with this repo's Ollama wrapper around it. The Teacher: field above and the datasets: list in the frontmatter are carried over from earlier revisions of this card. The current base does not claim them — its card lists no datasets and names no teacher — and nothing in this repo reproduces or measures them. One listed dataset, Crownelius/Opus-4.7-Reasoning, is not publicly reachable: the Hub answers 401, as it does for a private or missing repo. Treat them as unverified attribution, not as a training record. base_model_relation is quantized for the same reason.

On the repo name: the model is Janus-35B and the Hub slug FoolDev/Janus-35B-HERETIC names it correctly again. Every URL and ollama run hf.co/… pull below is unchanged, as is the Ollama tag (janus).

The base moved three times on 2026-09-18, so the CHANGELOG reads as a round trip: 0.7.0 restored this MoE, 0.8.0 swapped to a dense Qwen 3.8 27B for the newer Qwen generation, and 0.9.0 came back here — Qwen 3.8 ships no 35B-A3B at all, so the choice was 35B or 3.8, and 35B won. If you pulled on that day you may hold either model; ollama pull again to match this card. The dense 27B blob is no longer published and remains in git history.

TL;DR

One-liner via Hugging Face (pulls a GGUF + this repo's root-level template / system / params files, including the tool-calling template — HF's Ollama bridge ingests those three files, not Modelfile):

ollama run hf.co/FoolDev/Janus-35B-HERETIC               # default tag, Q4_K_M
ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M        # same blob, explicit tag

Or build locally (uses this repo's Modelfile, kept in sync with the three bridge files):

git clone https://huggingface.co/FoolDev/Janus-35B-HERETIC && cd Janus-35B-HERETIC
ollama create janus -f Modelfile && ollama run janus

After either path, ollama show janus lists completion, tools, and thinking under Capabilities. Hardware: the default num_ctx is the full 262,144 native window — about 25 GiB with Ollama's default f16 KV cache and about 23.0 GiB with OLLAMA_KV_CACHE_TYPE=q8_0. Those totals extrapolate a KV cache measured on this blob on 2026-09-18 (see Hardware requirements). Lower num_ctx on a smaller host (65536 measured at 21.6 GiB), or raise it to 1,010,000 with YaRN (not baked in, so context past ~262K degrades).

To halve the KV cache, start the Ollama server with the q8_0 cache and flash attention (for a systemd install, add them as Environment= lines with sudo systemctl edit ollama):

OLLAMA_KV_CACHE_TYPE=q8_0 OLLAMA_FLASH_ATTENTION=1 ollama serve

What's here

GGUF-only release. Pull the Heretic safetensors from llmfan46/Qwen3.6-35B-A3B-uncensored-heretic if you need the transformers tree (or the vanilla pre-Heretic base from Qwen/Qwen3.6-35B-A3B). The safetensors also keep the MTP layer that this GGUF does not carry — that's the path to speculative decoding under vLLM / SGLang.

Bundled blob status: the 35B-A3B MoE is published

The bundled GGUF is Janus-35B-A3B.Q4_K_M.gguf (21,233,608,512 bytes — 21.23 GB decimal, 19.78 GiB binary) — the Qwen 3.6 35B-A3B MoE this card describes. ollama run hf.co/FoolDev/Janus-35B-HERETIC serves it directly, and ollama create janus -f Modelfile works from a fresh clone.

It is llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF's Q4_K_M (sha256 b82f9f2155eb9c07139d48cc3a37880cf9d0edcca345ef6ad829b62941ccbb82) byte-for-byte: no re-quantization, no MTP strip and no template restamp. This quant carries no NextN block to strip, and the GGUF keeps the base model's own embedded chat template — this repo ships no chat_template.jinja of its own.

Verified before publishing (2026-09-18): the file's GGUF header was read directly from the published blob by HTTP range request — general.architecture qwen35moe, block_count 40, 256 experts / 8 active, context_length 262144, vocab 248,320, 733 tensors, no nextn key and no NextN tensors — and every Architecture claim below matches it. scripts/live_check.sh was then run against this blob on 2026-09-18 — Ollama 0.33.3, isolated store, CPU only — and all 17 live checks passed: the model loaded, Ollama selected the Go template with thinking detected, a first-turn tool call came back exactly once, a replayed tool result with JSON-string arguments was answered, reasoning replayed across turns and inside a live tool chain, and every reasoning_effort value returned 200 with the tiers reaching the prompt. What is still unmeasured on this blob: the runtime memory figures (weights, recurrent state, compute buffer) and any throughput number.

Between 0.2.0 and 0.6.5 this repo shipped a dense Qwen 3.8 27B (Janus-27B.Q4_K_M.gguf). That blob is no longer published; it remains in git history.

Architecture

A sparse hybrid stack: linear-attention (Gated DeltaNet) layers with a full Gated Attention layer every fourth block, and a 256-expert MoE feed-forward on every block. ~34.7B parameters total, ~3B active per token — 8 routed experts plus one shared expert.

• Qwen 3.6, 35B-A3B mixture-of-experts (256 experts, 8 routed per token, plus a shared expert on every block)
• 40 layers, 10 × (3 × Gated DeltaNet → MoE / 1 × Gated Attention → MoE) — full_attention_interval 4, so 10 full-attention and 30 linear-attention layers
• Hidden size 2048; per-expert FFN intermediate 512, shared-expert FFN 512
• Gated Attention: 16 Q-heads / 2 KV-heads (GQA), head_dim 256, gated output projection, partial RoPE (rope.dimension_count 64 of 256 dims rotated), rope.freq_base 1e7
• Gated DeltaNet: conv kernel 4, state size 128, 16 groups, inner size 4096
• 262 144 native context (extensible to 1 010 000 with YaRN, but YaRN is not enabled in the bundled GGUF)
• Vision + video supported by upstream (mmproj not included in this release — see Vision)
• Vocab 248,320
• GGUF arch stamp qwen35moe — the MoE arch in llama.cpp (src/models/qwen35moe.cpp, case 40: LLM_TYPE_35B_A3B); the dense members of the family stamp qwen35

Every bullet above is asserted against the bundled GGUF's own metadata by python3 scripts/verify_arch.py.

MTP head (absent from this quant). The base does declare an MTP layer (mtp_num_hidden_layers: 1 in its config.json) for vLLM / SGLang speculative decoding, but the Q4_K_M published in llmfan46's GGUF repo — the file this repo bundles — carries no NextN block: block_count is 40 and blk.0…blk.39 are all that exist. So no strip is needed here, and scripts/strip_mtp.py is a defensive no-op on it. llmfan46 also publishes a separate …-Native-MTP-Preserved-GGUF variant whose Q4_K_M has block_count 41 with a NextN block at index 40; that one would need stripping, and it is not what this repo ships. For the MTP head itself, run the upstream safetensors under vLLM / SGLang.

Quick start

llama.cpp / LM Studio

Drop the GGUF into your loader of choice. The chat template is embedded in the GGUF metadata, and llama.cpp and LM Studio use it by default, so no --chat-template flag is needed (llama.cpp has no --chat-template auto mode; it would take "auto" as the template itself).

Tool calling works out of the box — the GGUF carries Qwen 3.6's own template, so --jinja alone is enough and there is no file to pass:

llama-server -m Janus-35B-A3B.Q4_K_M.gguf --jinja

Tool calls use Qwen's native XML form, the one llama-server's parser reads. Two things this path does not have, because they lived in the patched Qwen 3.8 template this repo shipped between 0.2.0 and 0.6.5 and no longer ships: reasoning_effort tiers, and the tool_call_format JSON opt-in. See Tool / function calling and Reasoning effort.

Ollama

Ollama chooses between the GGUF's embedded template and a Go template at load time (see Which template Ollama runs); this repo ships a Go template so Ollama keeps its JSON tool-call format. Two paths deliver it:

# A. Pull straight from HF (uses the root-level template/system/params files):
ollama run hf.co/FoolDev/Janus-35B-HERETIC               # default tag, Q4_K_M
ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M        # same blob, explicit tag
# Note: HF's Ollama bridge does NOT read Modelfile; it reads template/system/params.

# B. Build locally (uses Modelfile, which is kept in sync with the three above):
ollama create janus -f Modelfile && ollama run janus

After the local build (path B), ollama show janus lists completion, tools, and thinking under Capabilities. (The HF-pull paths register the model under the full tag hf.co/FoolDev/Janus-35B-HERETIC, not janus.)

Inference examples

Once the model is loaded (via ollama run janus, lms server, or llama-server), all the standard OpenAI-compatible clients work. Examples assume the loader is listening on http://localhost:11434 (Ollama default) — adjust the port for LM Studio (:1234) or llama.cpp (:8080). Runnable versions of everything below live in examples/.

The examples use model: "janus", the tag from the local build (path B). If you pulled via the TL;DR one-liner instead, use the full tag hf.co/FoolDev/Janus-35B-HERETIC, or run ollama cp hf.co/FoolDev/Janus-35B-HERETIC janus once to create the short tag.

On memory-tight hosts, cap num_ctx first. /v1/chat/completions (OpenAI-compat) has no num_ctx knob, so it loads at the baked 262,144 default (5.0 GiB KV / ~25.4 GiB total with Ollama's default f16 cache), which is tight on a 32 GB box; OLLAMA_KV_CACHE_TYPE=q8_0 on the server brings it to ~23.0 GiB. For an OpenAI /v1 client the fix is a tag baked at a smaller context, and ./scripts/cap_ctx.sh does exactly that in one step: it creates a local janus tag from the bundled blob and this repo's Modelfile with num_ctx 4096 and num_batch 256 (CTX=8192 TAG=janus-cap ./scripts/cap_ctx.sh to change either). Callers that can send Ollama's own options field — /api/chat, not /v1 — pass those same two values per request instead; see Hardware requirements for that recipe and the full memory table.

curl

curl -s http://localhost:11434/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "janus",
    "messages": [
      {"role": "system", "content": "You are Janus, a precise reasoning assistant."},
      {"role": "user", "content": "Sketch an algorithm to detect cycles in a directed graph."}
    ],
    "temperature": 0.6,
    "max_tokens": 800
  }' | jq -r '.choices[0].message.content'

Python (openai-compat)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ignored")

resp = client.chat.completions.create(
    model="janus",
    messages=[
        {"role": "user", "content": "Write a haiku about a stack overflow."}
    ],
    temperature=0.8,
    top_p=0.95,
)
print(resp.choices[0].message.content)

Streaming

stream = client.chat.completions.create(
    model="janus",
    messages=[{"role": "user", "content": "Explain RoPE briefly."}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    print(delta, end="", flush=True)

Recommended sampling

The shipped default is Fable-matched — warm (temperature 1.0), no top_k, with top_p 0.95 + repeat_penalty 1.05 kept as loop insurance. Drop to the reasoning row for tighter, more deterministic output; lower temperature (0.4–0.6) and bump repeat_penalty to 1.08 if it loops inside <think> tags.

top_p does not survive the OpenAI-compatible endpoint. Ollama's /v1/chat/completions layer applies OpenAI's own default of top_p 1.0, overwriting the 0.95 baked into params / the Modelfile. Measured on this model: the same request logs top_p = 0.950 on /api/chat and top_p = 1.000 on /v1/chat/completions. Since the default profile also sets top_k 0 (disabled), a /v1 request samples with no truncation at all at temperature 1.0 — the "loop insurance" above is only in effect on the native API. If you need that guard on /v1, send top_p explicitly on every request, or bake min_p (e.g. PARAMETER min_p 0.05) into a local tag — min_p and top_k both pass through /v1 untouched, top_p does not.

System prompt

You are Janus, a precise and capable assistant for reasoning, writing, coding, and long-form dialogue.

Behavior rules:
- Answer the user's actual request directly.
- Be accurate, complete, and structured.
- Think before answering, but do not get stuck in repetitive loops or meta-commentary.
- If the request is ambiguous or incomplete, state what is missing and make the smallest reasonable assumption needed to continue.
- If the user wants creative writing, preserve tone, continuity, and character consistency.
- If the user wants analysis or technical help, prefer concrete steps, examples, and decisions over fluff.
- Finish with a usable answer, not just planning.

Vision

The Qwen 3.6 base supports image (and video) input via a separate mmproj projector. The full multimodal stack is:

Janus-35B-A3B.Q4_K_M.gguf                            (21.23 GB, the text decoder)
Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf  (902 MB, the vision projector)

The projector and other-quant text decoders live at llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF (BF16 mmproj only). This repo intentionally does not redistribute it; ./scripts/fetch_vision.sh pulls the projector into the repo root.

Loader compatibility

Vision via llama.cpp

# Fetch the projector first (into the repo root):
./scripts/fetch_vision.sh   # Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf

# A. HTTP via llama-server (the easiest path):
llama-server \
  -m Janus-35B-A3B.Q4_K_M.gguf \
  --mmproj Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf \
  --host 127.0.0.1 --port 8765 -c 8192 -ngl 99
# then POST OpenAI-style chat completions with an image_url content block —
# e.g. {"type":"image_url","image_url":{"url":"data:image/jpeg;base64,..."}}
# The thinking trace arrives in message.reasoning_content; the visible
# answer is in message.content. Budget ≥500 max_tokens so the reasoning
# block doesn't crowd out the final answer.

# B. CLI via llama-mtmd-cli (one-shot). It's a separate cmake target, so a
#    selective build can skip it; a plain `cmake --build build` produces it.
llama-mtmd-cli \
  -m Janus-35B-A3B.Q4_K_M.gguf \
  --mmproj Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf \
  --image photo.jpg \
  -p "Describe this image."

# C. Python via llama-cpp-python:
python examples/llama_cpp_vision.py \
  --gguf Janus-35B-A3B.Q4_K_M.gguf \
  --mmproj Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf \
  --image /path/to/photo.jpg \
  --prompt "What is in this image?"

The Ollama issue is closed upstream but this table has not been re-verified since; until you have re-tested, treat Ollama as text-only for this model. The bundled Q4_K_M decoder pairs with the projector directly — the mmproj is family-wide for Qwen 3.6 35B-A3B, so no separate text download is needed for vision.

Hardware requirements

This is a 21.23 GB Q4_K_M GGUF. Measured on 2026-09-18 on Ollama 0.33.3's CPU backend, from llama.cpp's own allocation lines, in an isolated store at num_ctx 8192, 32768 and 65536:

The KV cache covers only the 10 full-attention layers (llama.cpp logs it as "10 layers", 2 KV heads × 256 key + 256 value dims), which is under a third of the dense 27B's 2 GiB per 32K at the same window — the single biggest difference between the two siblings. q8_0 needs OLLAMA_KV_CACHE_TYPE=q8_0 with OLLAMA_FLASH_ATTENTION=1; flash attention is otherwise auto.

The 32768 and 65536 rows are sums of measured parts. The 262144 and 1010000 rows extrapolate the KV cache — exact, since it is linear in num_ctx — and carry the compute buffer forward from 65536, the largest context measured; the compute buffer grows faster than linearly between 32K and 64K, so treat those two rows as floors. It's extensible to 1,010,000, but this GGUF ships no baked YaRN rope-scaling, so positions past the 262144 native window use untrained RoPE and output degrades — reaching the 1.01M ceiling needs YaRN (see below); keep real work within ~262K otherwise.

How to override it: ollama run has no -o flag, and OLLAMA_CONTEXT_LENGTH only sets a default that the baked num_ctx overrides — so set it per-session from the interactive prompt. The model loads lazily on the first message, so /set applies before the default context is allocated:

ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
>>> /set parameter num_ctx 4096
>>> /set parameter num_batch 256

Programmatic callers pass the same via the API options field: "options": {"num_ctx": 4096, "num_batch": 256} — but only on Ollama's own /api/chat. /v1/chat/completions ignores options entirely, so an OpenAI-compatible client needs a tag baked at the smaller context instead: ./scripts/cap_ctx.sh bakes one with those same two values, from the bundled blob and this repo's Modelfile.

Throughput, measured on CPU only: ~25 tok/s. On a Ryzen AI Max+ 395 with Ollama 0.33.3's CPU backend, in an isolated store, ./scripts/bench.sh measured an aggregate of 24.97 tok/s (4,543 generated tokens over 181,890 ms) across its three-prompt mix after a discarded warmup — 25.88, 25.57 and 24.87 tok/s individually, so the rate barely moves with output length. Timings come from Ollama's own eval_count / eval_duration, not a client stopwatch. Switching the server to OLLAMA_KV_CACHE_TYPE=q8_0 with OLLAMA_FLASH_ATTENTION=1 made no meaningful difference to generation speed — 24.85 tok/s aggregate over the same mix, 0.5% below the f16 run. It buys memory, not speed.

That is about five times the 4.97 tok/s the dense 27B measured on this same CPU, which is the MoE's ~3B active parameters per token doing exactly what they should. Two caveats worth keeping in mind: this is generation speed, and a reasoning-first model emits a lot of tokens per answer — the third prompt, which asks for a 120-word explanation, produced 3,918 tokens including its <think> block, so wall-clock per answer is longer than the rate alone suggests. Prompt processing is not covered by this number.

GPU offload on a Strix Halo iGPU (Radeon 8060S): a small gain, and a trap. On a Vulkan integrated GPU, llama.cpp - including the build Ollama 0.33.3 bundles - turns mmap off and puts every layer it does not offload into GPU-reachable host memory from the same pool as the GPU, so num_gpu alone does not limit GPU memory: a first attempt with llama-server filled the pool and took the desktop down with it. With LLAMA_ARG_NO_HOST=1 in the Ollama server's environment (llama.cpp's --no-host) those layers stay in ordinary RAM. That mechanism is loader-side and applies to any model; the tok/s series this paragraph used to quote was measured on the dense 27B and has been removed with the rest. What generalises: the GPU's GTT is system RAM, so offloading frees none of it, and a first attempt without --no-host filled the pool and took the desktop session down with it. If you try the GPU on a machine like this, set LLAMA_ARG_NO_HOST=1 and a fixed num_gpu. The mechanism is reported upstream in ggml-org/llama.cpp#27360.

Reaching a coherent ~1.01M context (opt-in YaRN). The bundled GGUF ships no YaRN rope-scaling, so raising num_ctx toward the 1.01M ceiling degrades past the 262144 native window (see above). Ollama has no rope knob, so for a genuinely coherent long context run the GGUF under llama.cpp with YaRN enabled:

llama-server -m Janus-35B-A3B.Q4_K_M.gguf \
  --rope-scaling yarn --yarn-orig-ctx 262144 --rope-scale 3.853 -c 1010000

--rope-scale 3.853 ≈ 1010000 / 262144; use a smaller factor for a smaller window. Static YaRN rescales all prompts, so enable it only when you actually need > 262K — it slightly degrades short-context quality otherwise.

Chat template

The model uses the standard Qwen 3.x ChatML format with <|im_start|> / <|im_end|> role markers. Two templates are in play. The GGUF carries Qwen 3.6's own embedded template, exactly as llmfan46 published it — this repo ships no chat_template.jinja of its own — and that is what llama.cpp, LM Studio and KoboldCpp render. On Ollama the Go template / Modelfile TEMPLATE is what runs instead: Ollama picks between the two at load time (see Which template Ollama runs). Where the two differ, this card says which one it is describing.

Plain conversation

<|im_start|>system
You are Janus, a precise and capable assistant…<|im_end|>
<|im_start|>user
What is the time complexity of mergesort?<|im_end|>
<|im_start|>assistant

With reasoning trace

When the model decides to think, the assistant turn contains a <think>…</think> block followed by the visible answer:

<|im_start|>assistant
<think>
The user is asking about mergesort. Mergesort divides the array, recursively sorts each half, then merges. The recurrence T(n) = 2T(n/2) + O(n) solves to O(n log n).
</think>

Mergesort runs in **O(n log n)** time in the worst, average, and best cases. The recurrence is T(n) = 2T(n/2) + O(n), which solves to Θ(n log n) by the master theorem.<|im_end|>

Most clients (Open WebUI, LibreChat, etc.) hide the <think> block by default and show only the final answer. If your client doesn't, set its "show reasoning" toggle off.

Thinking is replayed across turns

Traces from earlier assistant turns are replayed back into the prompt: every assistant message that carries reasoning renders its <think>…</think> block, so the model can see how it reached its previous answers. The turn in progress — including a tool-call chain, where the model calls a tool and then continues after the result — keeps its own either way. On llama.cpp this is upstream's default; on Ollama it departs from Qwen's stock condition, which kept only the turn in progress.

The client has to send the reasoning back. The server returns it — Ollama's /api/chat in the response's thinking field, llama.cpp in reasoning_content — but only reasoning that arrives in the history's assistant messages is replayed, and each endpoint reads one field:

Which field each endpoint reads is decided by the server, not the model. Measured on this blob on 2026-09-18 (scripts/live_check.sh, Ollama 0.33.3, isolated store, CPU only): the prior turn's reasoning was replayed on /api/chat at 232 prompt tokens with it against 187 without, and a live tool-call chain kept its own reasoning (375 against 330). The same 232/187 figures were measured on the dense 27B in 0.6.x — unsurprising, since both share the 248,320-token vocabulary and this repo's Go template. The /v1 behaviour (reasoning read at 215 vs 187, reasoning_content ignored at 187, and llama-server rendering reasoning_content while dropping reasoning) was checked on 2026-09-13 on that dense blob; it is server-side and carries over. That Ollama's /v1 also drops thinking, and the array-content rule, come from Ollama's v0.33.3 and v0.34.0 source, and still hold on the current release: openai/openai.go and server/prompt.go are byte-identical from v0.33.3 through v0.34.2 (source, 2026-09-17). Ollama never passes reasoning to the GGUF's embedded Jinja template, so on Ollama only the Go template decides what is replayed. On llama.cpp an earlier assistant turn that carries no reasoning renders with no <think> block at all: Qwen 3.6's template gates the block on reasoning_content|length > 0. The 3.8 template this repo used to ship reconstructed an empty one instead, so prompts built by the two differ on that turn.

The cost is context. Every retained trace stays in the prompt — measured on the dense 27B at 181 prompt tokens without and 464 with, on a 3-message conversation whose first assistant turn carried a ~240-word trace, and not re-measured here. Prefill grows with every retained trace, and on a CPU-only host prefill is the slow part. Releases 0.3.0 through 0.3.2 turned replay off for that reason; the CHANGELOG has the history.

On llama.cpp, Qwen 3.6's own template decides. Read from the template inside the published blob (2026-09-18, from source — not re-tested live on this model): preserve_thinking defaults to true, so an earlier turn's reasoning is replayed by default; the assistant branch takes the trace from message.reasoning_content, and failing that recovers one by splitting a <think> block out of content; and there is no preserve_reasoning variable in it at all. That last point costs nothing, because llama-server derives a preserve_thinking kwarg from preserve_reasoning (common/jinja/caps.cpp), and from b10763 it passes preserve_reasoning=true by default — so --no-reasoning-preserve and a request's preserve_reasoning: false still reach this template through the derived copy, while a request's own preserve_thinking is read directly.

Which llama.cpp an Ollama carries. 0.33.3 and 0.34.0 bundle b10760; 0.34.1 bundles b10864 and 0.34.2 b10969 (each release's LLAMA_CPP_VERSION). From 0.34.1 on that is a b10763-or-later llama-server — the family that passes preserve_reasoning=true by default — but it changes nothing here: text inference for this model runs Ollama's own Go engine, and the bundled llama.cpp comes into play only when Ollama falls back to its C++ engine, as it does once an mmproj is attached (see Loader compatibility).

Nothing raises on conflicting settings. The Qwen 3.8 template that this repo patched between 0.4.0 and 0.6.5 raised when preserve_thinking and preserve_reasoning disagreed, and llama-server returned that raise as HTTP 500 — which is what the patch existed to prevent. Qwen 3.6's template has no such check: it reads preserve_thinking, ignores anything else, and cannot fail that way.

To turn replay off:

• llama-server — start it with --no-reasoning-preserve, or send "chat_template_kwargs": {"preserve_thinking": false} on a request. A request's preserve_thinking: true switches it back on for that request. {"preserve_reasoning": false} works too, through the kwarg llama-server derives from it; the template itself never reads that name.
• Ollama — edit the template. In the assistant branch of both the Modelfile and template (so make check's bridge sync stays green), change {{ if (and $.IsThinkSet .Thinking) -}} back to Qwen's stock {{ if (and $.IsThinkSet (and .Thinking (or $last (gt $i $lastUserIdx)))) -}} (keep the -}}; the $lastUserIdx loop at the top of the template is still there for it). Do not delete the block instead — see Which template Ollama runs. make check's Go-template guard will then fail, since it pins the replay condition.

Disabling thinking

This is a reasoning-first model — it opens a <think> block by default. For a direct answer with no reasoning trace (simple or latency-sensitive calls), turn thinking off:

ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M --think=false

or send "think": false on /api/chat. With thinking off the model skips the reasoning trace and answers straight into content; with it on (the default) reasoning is emitted into the thinking field.

On llama.cpp the switch is the embedded template's own: send "chat_template_kwargs": {"enable_thinking": false}, which primes a closed, empty <think> block — and, because the template gates its replay branch on enable_thinking too, drops every earlier turn's trace for that request. That variable is upstream Qwen 3.6's, not this repo's.

Reasoning effort

This works on Ollama, and not on llama.cpp. The instruction comes from this repo's Go template, which Ollama runs (see Which template Ollama runs). Qwen 3.6's own template — the one inside the bundled GGUF, and so the one llama.cpp and LM Studio render — has no reasoning_effort variable at all, so there the tier values are accepted by the server and then quietly ignored (read from the template, 2026-09-18). none is the one exception: llama-server maps it to enable_thinking: false and drops the kwarg before the template runs (tools/server/server-common.cpp, checked 2026-09-18), so reasoning_effort: "none" does switch thinking off on llama.cpp. Between 0.2.0 and 0.6.5 this repo shipped a patched Qwen 3.8 template that did implement the tiers on the llama.cpp path; that file is gone with the rebase back to the MoE.

Ollama folds the eight values its /v1 layer accepts into four think levels before any template sees them, so the Go template maps levels, not values:

"think": "high", "low" and "max" on /api/chat map the same way. Mixed case such as "HIGH" is rejected by Ollama itself with HTTP 400 before any template runs. The value-to-level mapping comes from Ollama's source, unchanged from v0.33.3 through v0.34.2 (2026-09-17).

Verified on this blob with scripts/live_check.sh (2026-09-18, Ollama 0.33.3, CPU only): all eight values returned HTTP 200 and none switched thinking off. On a one-line chat the tiers reach the prompt exactly as the table says — unset and medium added no instruction (159 prompt tokens), high and max each added the xhigh instruction (197, so +38), and low added the low instruction (185, +26). The same deltas were measured on the dense 27B in 0.6.x, which is expected: the Go template and Ollama's folding are model-independent.

Which template Ollama runs

Two templates exist for the same ChatML model: the Go template this repo ships (served by HF's Ollama bridge) / Modelfile TEMPLATE, and Qwen 3.6's own template inside the GGUF, which llmfan46 stamped and this repo leaves untouched. They do not render identical prompts — the Go template pins JSON tool calls and adds a reasoning-effort line from Ollama's think level, while the embedded template teaches the XML <function=…> form and has no effort variable at all.

Ollama picks one at load time: it prefers the embedded template when that advertises more capabilities — tools, thinking — and keeps the Go template when they tie. Both advertise tools and thinking, so they tie and Ollama runs the Go template; llama.cpp and LM Studio run the embedded one. Verified live against this blob on 2026-09-18 (scripts/live_check.sh, Ollama 0.33.3): with Qwen 3.6's own template embedded in the GGUF, Ollama still selected the Go template with thinking detected — so the tie holds for the MoE, not just for the dense 27B it was first checked on. The selection code is unchanged through v0.34.2 (source, 2026-09-17). Ollama's built-in renderers — it ships a qwen35 one — displace both templates only when the model's own config names a renderer or parser, which this repo's does not. Ollama logs the choice:

msg="template selection" ... selected=go_template go_template="[completion tools thinking]"

Upstream watch — ollama/ollama#17965. By its own description, that PR ("server: auto-detect ornith and qwen35 renderer and parser") would stamp a qwen3.5 renderer and parser into the config of every qwen35 / qwen35moe model at create time — this one included. That is precisely the condition the paragraph above rests on not holding, so the consequence follows from the rule stated there rather than from any test: were it to land, the built-in renderer would displace both templates, and the JSON tool-call shape and reasoning_effort line documented in this section would stop being what actually runs. Checked 2026-09-18 with gh pr view 17965 -R ollama/ollama: open since 2026-08-24, no approving review, a collaborator disputing the approach in the thread, and the branch conflicting with main — not merged, so nothing here changes yet. If it does merge, re-check the server's template selection log line (and ollama show janus --modelfile) for a named renderer or parser before trusting this section's claims.

The Go template's "thinking" entry comes from its single .Thinking reference — the block that renders each assistant turn's reasoning. Removing it drops that capability, and Ollama switches to the embedded template: tool calls change from JSON-in-XML to Qwen's XML form, which Ollama's extractor does not parse into a tool_calls array, reasoning_effort stops doing anything (the embedded template has no such variable), and a single tool call can come back twice (seen once on the dense 27B under the embedded template; never on the Go-template path, which returned exactly one call on this blob on 2026-09-18 — cause inferred, not confirmed; see Known limitations). Setting OLLAMA_GO_TEMPLATE=1 or =0 on the Ollama server forces one template or the other for every model.

Tool / function calling

The model emits whichever format the prompt specifies, and the two paths specify different ones: the Ollama path pins JSON-in-XML, while the llama.cpp path gets Qwen's native XML form from the template embedded in the GGUF.

Ollama path (this repo's Modelfile). The TEMPLATE advertises tools inside <tools>…</tools> and asks the model to reply in JSON-in-XML — the form Ollama's tool-call extractor parses into a structured tool_calls array on /api/chat and /v1/chat/completions. The tool definitions go in with {{ json .Function }}: on Ollama 0.14 and later a bare {{ .Function }} prints Go struct syntax instead of JSON (ollama/ollama#14601; fix proposed in ollama/ollama#18391). The model replies like this:

<tool_call>
{"name": "get_weather", "arguments": {"city": "Tokyo"}}
</tool_call>

Embedded-template path (llama.cpp, llama-cpp-python, LM Studio, KoboldCpp). The bundled GGUF carries Qwen 3.6's own template, so --jinja alone is enough — there is no file to pass and nothing to stamp. It teaches the XML form:

<tool_call>
<function=get_weather>
<parameter=city>
Tokyo
</parameter>
</function>
</tool_call>

Keep XML on llama-server. llama-server recognises this template as Qwen3-Coder-style and parses the model's reply with a parser — and, once the model opens <tool_call>, a grammar — that accept only <function=…><parameter=…>. Qwen 3.6's tool-instruction literal is byte-identical to Qwen 3.8's (compared 2026-09-18), so the detection and the wire format are exactly what this card documented before the reversal. What is gone is the tool_call_format knob: the JSON opt-in described in releases 0.3.0 through 0.6.5 lived in the patched 3.8 template this repo no longer ships.

Replayed tool calls: normalise arguments first. Every OpenAI-compatible client hands assistant tool calls back with function.arguments as a JSON string, while the XML parameter form needs the individual key/value pairs — a chat template has tojson but no inverse. Qwen 3.6's template renders <parameter=…> only when arguments is a mapping; handed a string it emits that string as-is inside <function=…>, carrying upstream's own comment that you should "normalize arguments to a dict in your serving layer before applying this template". It does not raise, so a loader that passes the string straight through produces a silently malformed prompt rather than an error — the failure is quieter here than it was under the 3.8 template, which raised and returned HTTP 500.

In practice the loaders convert before rendering: llama-server decodes the string into a mapping, KoboldCpp (--jinja --jinja_tools) runs json.loads first, and the LM Studio app parses the arguments before its own JavaScript Jinja engine renders. All three were checked on 2026-09-14 against the 3.8 template this repo shipped then; the conversion is loader-side, so it applies unchanged here (from source for this template — not re-tested on this blob).

Example (Ollama, OpenAI-compatible API)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ignored")

resp = client.chat.completions.create(
    model="janus",
    messages=[
        {"role": "user", "content": "Call get_weather for Tokyo. Respond ONLY with the tool call."}
    ],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    }],
    temperature=0.3,
)
print(resp.choices[0].message.tool_calls)
# [ToolCall(id='call_xxx', type='function',
#           function=Function(name='get_weather', arguments='{"city":"Tokyo"}'))]

Tips

• Use direct prompts ("Call X for Y") rather than soft hints ("Use the tool"). The model thinks before committing to a call, and weak prompts can exhaust num_predict inside the <think> block before the call is emitted.
• Allow at least num_predict: 1024 (or max_tokens: 1024) for tool-calling turns, more if the schemas are large.
• The JSON-in-XML format is what Ollama's tool-call extractor understands, and the Go template pins it; the GGUF's embedded template renders the XML <function=…> form. So the prompt format does change when you swap loaders, while client code should not have to, as long as the server returns a parsed tool_calls array.

Known limitations

• On the embedded-template path, a single tool call can come back twice. Seen on the dense 27B blob, not re-tested here: in one test Ollama 0.33.3, running the embedded template, returned a first-turn tool call twice. A separate raw generation showed the model drafting the call inside <think> before emitting it once after </think>, so the parser most likely matched such a draft (inferred, not confirmed). It did not reproduce on standalone llama.cpp master (a2878d3, CPU): across both models, 12 first-turn calls (three prompts, with and without parallel_tool_calls) and two streamed runs each came back exactly once. Nor on b10760, the llama.cpp build Ollama 0.33.3 bundles and the one the duplicate came through: six more first-turn calls and a streamed run on this model each came back once. LM Studio's CPU engine (llama.cpp b10519) passed the same checks; the LM Studio app's own API layer is untested. Ollama's default path — the Go template, see Which template Ollama runs — returned exactly one call, re-confirmed on this MoE blob on 2026-09-18 (scripts/live_check.sh, first-turn call at reasoning_effort: high, one tool_calls entry). Whether the prompt or the parser makes the difference on the embedded path was not isolated. If your client executes tools wherever the embedded template renders (llama.cpp, LM Studio, or Ollama with OLLAMA_GO_TEMPLATE=0), drop identical consecutive calls.
• No mmproj in this release. The base Qwen 3.6 35B-A3B supports image and video input via a separate mmproj file, which is not included here. Text-only inference works out of the box; multimodal inference requires fetching Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf (or equivalent) from upstream — run ./scripts/fetch_vision.sh and see Vision for the full path.
• Quantization-induced quality loss. Q4_K_M is a strong general-purpose quant but does measurably degrade math and code accuracy compared to BF16. If you need maximum quality, run the upstream safetensors on a host with room for the full BF16 weights.
• No MTP head on the GGUF path. The base declares a multi-token-prediction layer (mtp_num_hidden_layers: 1), but the Q4_K_M published in llmfan46's GGUF repo — the file this repo bundles — carries no NextN block at all (see Architecture). No speculative decoding here: use the upstream safetensors under vLLM / SGLang, or llmfan46's separate …-Native-MTP-Preserved-GGUF variant, if you need it.
• Sparse compute, dense memory. Only ~3B of ~34.7B parameters are touched per token, but every expert has to be resident: the 19.77 GiB of weights is a floor whatever the routing does, and MoE sparsity buys compute, not RAM. What it does buy on the memory side is a small KV cache — 0.625 GiB per 32K at f16 measured, under a third of a comparable dense stack — so long contexts are unusually cheap here relative to the weights.
• Thinking traces can loop. Like most reasoning-distilled models, Janus-35B occasionally gets stuck repeating itself inside <think> tags. Mitigations: lower temperature to 0.4-0.6, raise repeat_penalty to 1.08, or set a <think>-token budget cap if your loader supports it.
• Large tool-call arguments can be dropped. Ollama's JSON-in-XML tool format makes the model JSON-escape the entire arguments object inline; for a big/complex payload (e.g. a file's content in a write_file call) the model can fail to escape it, so the field arrives undefined and the call fails. Qwen's native <function=…><parameter=…> format (raw values, no escaping) was tested as a fix but parses unreliably through Ollama, so the template deliberately keeps JSON-in-XML. Mitigation: write large files in smaller pieces per call.
• Uncensored base — not aligned with any specific safety policy. This is a personal repackage of an open-weight base whose refusal behavior has been abliterated away (the llmfan46 Heretic base). There is no RLHF refusal layer; the model will attempt most requests, so downstream safety is entirely the operator's responsibility.
• No formal evaluation in this card. The hardware table and the ~25 tok/s figure are measured on this blob (2026-09-18), but no task benchmarks (MMLU, HumanEval, …) are claimed, and prompt-processing throughput is not measured. If you produce real numbers and want them included, file a PR. No task benchmarks are claimed either. If you produce real numbers (tok/s, MMLU, HumanEval, …) and want them included, file a PR.

Related models

Janus vs Thanatos. Since 0.7.0 the two are genuinely different models, not two abliterations of one base. Janus is Qwen 3.6 35B-A3B: a mixture of experts, ~34.7B parameters total with ~3B active per token, 40 layers, 256 experts (8 routed + 1 shared), on llmfan46's Heretic base, and its bundled quant is MTP-clean so no strip is needed. Thanatos is a dense Qwen 3.8 27B — every parameter on every token, 64 layers — on heretic-org's ARA base, and its bundle is MTP-stripped, because that base ships the NextN block. They still share the 262,144 native window, the ChatML frame, the Claude Fable 5 teacher slot and this repo's tooling. Choose on shape, not only on abliteration: Janus for 35B-class capacity at ~3B active and a KV cache under a third the size (0.625 vs 2 GiB per 32K at f16), Thanatos for the smaller resident footprint (16.1 vs 19.78 GiB of weights) and a newer Qwen generation.

Credits

• Immediate base: llmfan46/Qwen3.6-35B-A3B-uncensored-heretic — Heretic-style abliteration of Qwen 3.6 35B-A3B
• Upstream base: Qwen/Qwen3.6-35B-A3B (Alibaba)
• Reasoning teacher: Claude Fable 5 (Anthropic)
• Distillation lineage and dataset curation: Crownelius

License inherited from upstream: Apache-2.0.