Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support, with the following key features: - Uniquely support of seamless switching between thinking mode (for complex logical reasoning, math, and coding) and non-thinking mode (for efficient, general-purpose dialogue) within single model, ensuring optimal performance across various scenarios. - Significantly enhancement in its reasoning capabilities, surpassing previous QwQ (in thinking mode) and…
Open-weight model · Text generation
cultist-simulator-qwen3-4b
by Seele Cecilis/cultist-simulator-qwen3-4b
cultist-simulator-qwen3-4b is an open-weight model for text generation from Seele, released under Apache License 2.0. It has 4B parameters and a 40,960-token context. At 16-bit it needs about 9.7 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
用《密教模拟器》(Cultist Simulator) 的游戏内文本微调 Qwen/Qwen3-4B 得到的文本生成模型。 仓库根目录带 LLaMA-Factory 生成的 Modelfile，下载后在本目录下执行： - 4B：The light leaks through cracks. My mind is clearer than it's been in any other time. 标记完整保留 - 0.6B：Light leaks through a Crack in the Skin.
Parameters4B
Context40,960
Weights8.2 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads—
Runs On
What it takes to serve cultist-simulator-qwen3-4b (4B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 8.0 GB | 9.7 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 4.0 GB | 4.8 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 2.0 GB | 2.4 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 18, 2026.
cultist-simulator-qwen3-4b on every accelerator the SAVRN Index prices, at every precision
Model Card
By Seele, published under apache-2.0, revision 8c8a7344c0e4.
用《密教模拟器》(Cultist Simulator) 的游戏内文本微调 Qwen/Qwen3-4B 得到的文本生成模型。 仓库根目录带 LLaMA-Factory 生成的 Modelfile，下载后在本目录下执行： - 4B：The light leaks through cracks. My mind is clearer than it's been in any other time. 标记完整保留 - 0.6B：Light leaks through a Crack in the Skin. My brain is clearer than ever at any given time. 标记全丢 - 训练语料全部抽取自《密教模拟器》游戏本体，版权归 Weather Factory 所有。 - 基座模型 Qwen/Qwen3-4B 采用 Apache-2.0 协议，本仓库的权重是它的微调衍生品。
Read Seele's full model card
用《密教模拟器》(Cultist Simulator) 的游戏内文本微调 Qwen/Qwen3-4B 得到的文本生成模型。 目标是让它按该游戏晦涩、克制、带书卷气与不祥暗示的文风，自动生成卡牌描述、事件叙事、结局文本， 并支持中英互译。产出的文字可直接用在同人创作、跑团素材、模组草稿等场景。
仓库内容
| 路径 | 内容 | 大小 |
|---|---|---|
| 根目录 | 合并后的完整模型（LoRA 已并入权重），from_pretrained 直接可加载 |
7.51 GB |
lora/ |
LoRA 适配器（r=16），配合基座模型使用 |
126.1 MB |
训练细节
| 项目 | 值 |
|---|---|
| 基座模型 | Qwen/Qwen3-4B |
| 微调方式 | LoRA（r=16，alpha=32，作用于 q/k/v/o/gate/up/down 全部投影层） |
| 训练样本 | 26,875 条（验证集 779 条，按实体 id 稳定划分） |
| 训练轮数 | 3 epoch / 5040 步 |
| 等效 batch | 1 × 16（per_device × 梯度累积 = 16） |
| 截断长度 | 896 |
| 学习率 | 1e-4（cosine，warmup 10%） |
| 精度 | bf16 + 梯度检查点 |
| 对话模板 | qwen3_nothink |
| 最终 eval_loss | 1.949 |
| 训练耗时 | 6 小时 |
| 硬件 | 单卡 RTX 4060 Ti 16GB |
数据全部来自本地已安装的游戏本体，不依赖任何联网数据源。抽取与构建脚本见文末。
训练了什么任务
数据集是多任务的，同一条语料会派生出多个样本：
| 任务 | 说明 |
|---|---|
| 中英互译 | 两侧都带 <b>/<i> 标记时才要求保留，否则把标记一并剥掉，保证指令与答案自洽 |
| 卡牌描述 | 给定卡牌名 + 类别 + 主题，写描述文本 |
| 行动叙事 | 给定行动类型 + 事件名，写开场叙述 / 结算叙述 |
| 结局正文 | 按结局名撰写正文 |
| 主题创作 | 以准则（灯/铸/刃/冬/心/杯/蛾/启/秘史）为主题创作 |
| 风格续写 | 给定开头，按原文风格续写 |
快速使用
完整模型（推荐，最省事）
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "Cecilis/cultist-simulator-qwen3-4b"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="bfloat16", device_map="auto")
system = (
"你是《密教模拟器》（Cultist Simulator）的文本生成模型。"
"你精通该游戏晦涩、克制、带着书卷气与不祥暗示的文风，"
"能够按给定的题材、类别和主题创作中文与英文游戏文本，也能在中英文之间准确互译。"
)
user = "为《密教模拟器》中的一张卡牌撰写描述文本。\n\n语言：中文\n卡牌名称：无光的陈列室\n类别：地点\n主题：冬、灯"
msgs = [{"role": "system", "content": system}, {"role": "user", "content": user}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True, enable_thinking=False)
out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=256)
print(tok.decode(out[0][tok(text)["input_ids"].__len__():], skip_special_tokens=True))
LoRA 适配器
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B", dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "Cecilis/cultist-simulator-qwen3-4b", subfolder="lora")
Ollama
仓库根目录带 LLaMA-Factory 生成的 Modelfile，下载后在本目录下执行：
ollama create cultist-4B -f Modelfile
ollama run cultist-4B
效果示例
同一组提示词下，4B 与 0.6B 的对比（4B 全面更好）：
卡牌描述（输入：卡牌名称：无光的陈列室 / 类别：地点 / 主题：冬、灯）
- 4B：
雪覆一切，日沉至地平线下。光线如冰霜般冷冽。 - 0.6B：
这里的每一块石头都深陷青苔，而石缝里那盆枯萎的多肉植物的根茎仍在挣扎。
中译英（要求保留 <b> 标记）（输入：光透过<b>裂缝漏入</b>。我的头脑比<b>任何时候</b>都<b>更清晰</b>。）
- 4B：
The light leaks through <b>cracks</b>. My mind is <b>clearer</b> than it's been <b>in any other time</b>.标记完整保留 - 0.6B：
Light leaks through a Crack in the Skin. My brain is clearer than ever at any given time.标记全丢
eval_loss 也印证这个差距：4B 1.949 vs 0.6B 2.406。
数据来源与使用限制
- 训练语料全部抽取自《密教模拟器》游戏本体，版权归 Weather Factory 所有。 中文文本来自游戏的官方简体中文资源。
- 基座模型 Qwen/Qwen3-4B 采用 Apache-2.0 协议，本仓库的权重是它的微调衍生品。
- 请仅将本模型用于个人学习、同人创作等非商业用途；模型生成的内容可能重现或近似游戏原文， 公开发布生成结果时请遵守游戏原作者的授权条款，并注明题材来源。
- 本模型只学习了文风与文本结构，不理解也不会执行游戏机制；生成内容不保证与官方设定一致。
复现
# 抽取游戏文本 → 构建多任务数据集
.\scripts\prepare_data.ps1
# 训练 + 合并
.\scripts\train.ps1 -Size 4b -Merge
Configuration
- Architecture
- Qwen3ForCausalLM
- Context length (tokens)
- 40,960
- Layers
- 36
- Hidden size
- 2,560
- Feed-forward size
- 9,728
- Attention heads
- 32
- Key/value heads
- 8
- Head dimension
- 128
- Vocabulary size
- 151,936
- RoPE base
- 1,000,000
- Model type
- qwen3
Identity and Version
- Repository
- Cecilis/cultist-simulator-qwen3-4b
- Publisher
- Seele
- Task
- Text generation
- Modality
- Text
- Library
- transformers
- Parameters
- 4B parameters
- Languages
- zh, en
- Revision
- 8c8a7344c0e4bd1b981a5b8c052bdd2de5ae4d70
- First published
- 2026-09-28
- Last updated
- 2026-09-28
Files and Weights
25 files, 8.2 GB in total. The weights are 4 files totalling 8.2 GB in safetensors.
Weights4 files · 8.2 GB
Configuration8 files · 38.9 KB
Tokenizer8 files · 31.8 MB
Documentation1 file · 5.6 KB
Other3 files · 8.9 KB
Repository1 file · 1.6 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| lora/adapter_model.safetensors | Weights | 132.2 MB | ff937fe943f0 |
| model-00001-of-00003.safetensors | Weights | 4.0 GB | af2ad67afd50 |
| model-00002-of-00003.safetensors | Weights | 4.0 GB | 2c2a5f7cacb4 |
| model-00003-of-00003.safetensors | Weights | 99.6 MB | 486c8fdd33c1 |
| added_tokens.json | Configuration | 735 B | — |
| config.json | Configuration | 1.6 KB | — |
| generation_config.json | Configuration | 227 B | — |
| lora/adapter_config.json | Configuration | 1.0 KB | — |
| lora/added_tokens.json | Configuration | 735 B | — |
| lora/special_tokens_map.json | Configuration | 644 B | — |
| model.safetensors.index.json | Configuration | 33.3 KB | — |
| special_tokens_map.json | Configuration | 644 B | — |
| README.md | Documentation | 5.6 KB | — |
| Modelfile | Other | 395 B | — |
| chat_template.jinja | Other | 4.3 KB | — |
| lora/chat_template.jinja | Other | 4.3 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| lora/merges.txt | Tokenizer | 1.7 MB | — |
| lora/tokenizer.json | Tokenizer | 11.4 MB | aeb13307a71a |
| lora/tokenizer_config.json | Tokenizer | 5.7 KB | — |
| lora/vocab.json | Tokenizer | 2.8 MB | — |
| merges.txt | Tokenizer | 1.7 MB | — |
| tokenizer.json | Tokenizer | 11.4 MB | aeb13307a71a |
| tokenizer_config.json | Tokenizer | 5.7 KB | — |
| vocab.json | Tokenizer | 2.8 MB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 8.2 GB
Download from Seele
Released by Seele through its official repository on Hugging Face. Read the license.
Built From
- Adapter of Qwen/Qwen3-4B
- Derived from Qwen/Qwen3-4B
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 8.2 GB |
| 16-bit | 8.0 GB |
| 8-bit | 4.0 GB |
| 4-bit | 2.0 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About cultist-simulator-qwen3-4b
How much GPU memory does cultist-simulator-qwen3-4b need?
About 9.7 GB at 16-bit and 2.4 GB at 4-bit: the weights (4B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run cultist-simulator-qwen3-4b on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use cultist-simulator-qwen3-4b commercially?
Yes. cultist-simulator-qwen3-4b is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is cultist-simulator-qwen3-4b's context length?
40,960 tokens, from the maximum position embeddings in its published configuration.
Similar Models
We introduce the updated version of the Qwen3-4B non-thinking mode, named Qwen3-4B-Instruct-2507, featuring the following key enhancements: - Significant improvements in general capabilities, including instruction following, logical reasoning, text comprehension, mathematics, science, coding and tool usage. - Substantial gains in long-tail knowledge coverage across multiple languages. - Markedly better alignment with user preferences in subjective and open-ended tasks, enabling more helpful responses and higher-quality text generation. - Enhanced capabilities in 256K long-context understanding. Qwen3-4B-Instruct-2507 has the following features: NOTE: This model supports only non-thinking…
Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Building upon extensive advancements in training data, model architecture, and optimization techniques, Qwen3 delivers the following key improvements over the previously released Qwen2.5: Qwen3-4B-Base has the following features: For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our blog, GitHub, and Documentation. The code of Qwen3 has been in the latest Hugging Face transformers and we advise you to use the latest version of transformers. With transformers<4.51.0, you will…
Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Building upon extensive advancements in training data, model architecture, and optimization techniques, Qwen3 delivers the following key improvements over the previously released Qwen2.5: Qwen3-4B-Base has the following features: For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our blog, GitHub, and Documentation. The code of Qwen3 has been in the latest Hugging Face transformers and we advise you to use the latest version of transformers. With transformers<4.51.0, you will…
SRA-RiskGate-4B is an autonomous risk scoring, compliance verification, and dispute adjudication model fine-tuned on top of Qwen/Qwen3-4B-Instruct-2507. It is engineered for both ends of a stablecoin payment's operational lifecycle: Autonomous on-chain agents can hallucinate payment transfers, sign malformed calldata, or trigger catastrophic transactions during market depegs. The deterministic pre-filter and policy firewall is available directly as a verified Action Provider for the Coinbase AgentKit framework: Register RiskGateActionProvider as the payment provider on your AgentKit instance. It checks chain support, policies, and peg deviations, executing safe ERC-20 transfers only after…
![[HYU_NLP] EVA Team logo](/media/mh-logo-hyu-nlp-eval-558e075b88.webp){kind=logo}
Intermediate policy from dynamic OnlineRubrics-Every GRPO training. Distinct from static-rubric GRPO. Base model: Qwen/Qwen3-4B-Instruct-2507; thinking disabled. This checkpoint is a policy state used by the Phase-1 audit. No downstream medical capability or safety claim is made. Research use only; not validated for clinical decision-making. Root files are the veRL-exported Hugging Face inference model (BF16). originalcheckpoint/ preserves the exact original FSDP parameter checkpoint and tokenizer/configuration files. Optimizer state, training data, responses, rubrics, infrastructure configuration, and credentials are not included. The original is retained because export…