GGUF quantizations of richardyoung/Olmo-3-7B-Think-heretic, an abliterated build of allenai/Olmo-3-7B-Think (Ai2's fully open 7B reasoning model (open weights, data and training code) that thinks before answering) made with Heretic.
Abliteration Results
| Metric |
Value |
| Base Refusals |
100/100 |
| Abliterated Refusals |
48/100 |
| Refusal Reduction |
52% |
| KL Divergence |
0.026 |
The very low KL divergence (0.026, far below the 0.5 “damage” threshold) means the model retains essentially all of its original capabilities and coherence.
Note on refusals: this build still refuses 48 of 100 harmful prompts, more than our other Heretic releases. That is deliberate: the trials that refused less also degraded the model's normal behavior (KL divergence above 0.10), and we chose to keep the original model's capabilities intact rather than trade them for fewer refusals.
Quick Start
With Ollama
ollama run richardyoung/olmo-3-7b-think-heretic
# or straight from Hugging Face
ollama run hf.co/richardyoung/Olmo-3-7B-Think-heretic-GGUF
With llama.cpp
huggingface-cli download richardyoung/Olmo-3-7B-Think-heretic-GGUF \
--include "*Q4_K_M*" --local-dir ./models
./llama-cli -m ./models/*Q4_K_M*.gguf -ngl 99
With Python (llama-cpp-python)
from llama_cpp import Llama
llm = Llama.from_pretrained(
repo_id="richardyoung/Olmo-3-7B-Think-heretic-GGUF",
filename="*Q4_K_M*",
n_gpu_layers=-1,
)
output = llm.create_chat_completion(
messages=[{"role": "user", "content": "Explain abliteration in simple terms."}]
)
print(output["choices"][0]["message"]["content"])
Available Quantizations
| Quantization |
Size |
Use Case |
| IQ3_M |
3.5 GB |
Smallest |
| IQ4_XS |
4.0 GB |
Great quality-size balance |
| Q4_K_M |
4.5 GB |
Recommended — good balance |
| Q5_K_M |
5.2 GB |
Higher quality |
| Q6_K |
6.0 GB |
Very high quality |
| Q8_0 |
7.8 GB |
Near-lossless |
| Olmo-3-7B-Think-heretic-imatrix.gguf |
– |
Importance matrix used for the IQ and Q4_K_M/Q5_K_M quants (calibration_datav3) |
What is Abliteration?
Abliteration removes the "refusal direction" from a model's residual stream — a surgical modification that disables safety refusals without retraining. See Refusal in Language Models Is Mediated by a Single Direction. Heretic automates it: it searches ablation parameters with an optimizer (200 trials here) and keeps the trials that best trade off fewer refusals against KL divergence from the original model.
Intended Use
Research, creative writing, education on alignment techniques, and unrestricted local inference. This model will comply with requests the original refuses; use responsibly and in accordance with applicable laws. License: Apache-2.0, inherited from the base model.
Other Models by richardyoung
Built & maintained by Richard Young · DeepNeuro
