Gander-ablated-NVFP4 is an open-weight model from Jared H, released under Apache License 2.0. Its published files total 4.2 KB.
The Thinker of justsumguy/Gander-ablated, quantised with NVIDIA ModelOpt 0.46.1, plus the unmodified BF16 Talker. The Thinker is 7.2 GB and the Talker 0.7 GB. The abliteration details and numbers are on the BF16 page.
Model Card
By Jared H, published under apache-2.0, revision 2b562026e880.
The Thinker of justsumguy/Gander-ablated, quantised with NVIDIA ModelOpt 0.46.1, plus the unmodified BF16 Talker. The Thinker is 7.2 GB and the Talker 0.7 GB. The abliteration details and numbers are on the BF16 page. In short: heretic ARA branch, ARA-LoRA rank 128, trial 4, refusals 98 → 7 of 100, KL 0.128 on recheck, no healing applied. talker/ is BF16 and identical to the BF16 repository's. - llm calibration: 256 samples × 512 tokens of tatsu-lab/alpaca (shuffled, seed 0), run through the extracted text tower. Mean NLL on 8 calibration chats: BF16 4.335, NVFP4 fake-quant 4.408 (+0.073). - vpm calibration: 80 images from johnowhitaker/imagenette2-320. Three quarters were unsliced; one…
Read Jared H's full model card
The Thinker of justsumguy/Gander-ablated, quantised with NVIDIA ModelOpt 0.46.1, plus the unmodified BF16 Talker. The Thinker is 7.2 GB and the Talker 0.7 GB. The abliteration details and numbers are on the BF16 page. In short: heretic ARA branch, ARA-LoRA rank 128, trial 4, refusals 98 → 7 of 100, KL 0.128 on recheck, no healing applied.
Quantisation (thinker/)
| Component | Format | Notes |
|---|---|---|
llm (252 linears) |
NVFP4 W4A4 | block 16, FP8 E4M3 block scales, FP32 global scale, static activation scale; lm_head and embed_tokens BF16; KV cache not quantised |
vpm (162 linears) |
FP8 E4M3 per tensor, W8A8 | patch_embedding BF16 |
apm, resampler, audio_projection_layer |
BF16 | unchanged |
talker/ is BF16 and identical to the BF16 repository's.
- Recipes: ModelOpt
NVFP4_DEFAULT_CFGforllm;FP8_DEFAULT_CFGforvpm, withpatch_embeddingexcluded. llmcalibration: 256 samples × 512 tokens oftatsu-lab/alpaca(shuffled, seed 0), run through the extracted text tower. Mean NLL on 8 calibration chats: BF16 4.335, NVFP4 fake-quant 4.408 (+0.073).vpmcalibration: 80 images fromjohnowhitaker/imagenette2-320. Three quarters were unsliced; one quarter usedmax_slice_nums=4.vpmheld-out check (16 images):
| Output | Mean cosine | Min cosine | Mean rel. L2 |
|---|---|---|---|
| vision tower, last hidden state | 0.972 | 0.957 | 0.234 |
| resampler | 0.989 | 0.977 | 0.145 |
- Hardware:
- One RTX 5060 Ti 16 GB (SM120) and one RTX 3050 Ti Laptop 4 GB (SM86), with
lm_headon CPU during calibration. The host had 15 GB of RAM. - The BF16 tower was streamed from the shards across both GPUs.
Loading: not verified
This is a ModelOpt export: llm weights are packed uint8 with weight_scale,
weight_scale_2 and input_scale tensors, and hf_quant_config.json says
MIXED_PRECISION. Neither from_pretrained with the remote code's pinned transformers nor the
Gander runtime loads it as-is. A ModelOpt-aware loader is needed.
The Gander runtime's NVFP4 path has not been confirmed. Realtime duplex behaviour and performance have not been measured on this build.
License
Apache-2.0, as upstream. See thinker/LICENSE and thinker/NOTICE. The modifications are the abliteration of the Thinker's language tower and the quantisation described above.
Identity and Version
- Repository
- justsumguy/Gander-ablated-NVFP4
- Publisher
- Jared H
- Task
- Not stated by the source
- Modality
- Other
- Library
- transformers
- Parameters
- Not stated by the source
- Languages
- Not stated by the source
- Revision
- 2b562026e88013b4fdbf04bf1a34d9f783137659
- First published
- 2026-09-23
- Last updated
- 2026-09-23
Files and Weights
2 files, 4.2 KB in total.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| README.md | Documentation | 2.7 KB | — |
| .gitattributes | Repository | 1.5 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
Released by Jared H through its official repository on Hugging Face. Read the license.
Built From
- Derived from justsumguy/Gander-ablated
- Quantized from justsumguy/Gander-ablated
Questions About Gander-ablated-NVFP4
Can I use Gander-ablated-NVFP4 commercially?
Yes. Gander-ablated-NVFP4 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.