Excellent 50% prune by Nickyang utilizing new RAZOR methodology. cleyesode further reduced memory requirements via nvfp4 quantization for use on Blackwell sm120 gpus. Quantization via NVIDIA/Model-Optimizer custom qwen4exp workflow: Please see post-quantization verificationreport for details. PLE isolated, full model repack for reduced memory pressure. Calibrated on curated samples from Nickyang/RazorCal: This model demonstrates very impressive retention of pre-prune / pre-quantize abilities - benchmarks in-progress. MTP is present but untested, updates will be posted as they become available. vLLM compatible via the latest branch: Example launch on 4 5060Ti 16GB GPUs (tensor parallel 4…
Open weights
other
88.1B parameters
262,144 tokens
transformers
Access requested at publisher
apache-2.0
40B parameters
transformers
A REX checkpoint of Intern-S2 with WMDP-Cyber knowledge relocated into 8 removable experts per MoE layer (264 experts in total). Excising these experts together with their router rows yields A3, a standard 256-expert model with that knowledge removed. This repository holds the expanded checkpoint F3: the REX model after 250M tokens of standard full-parameter SFT, with the removable experts still attached. A3 differs from F3 only by excision. CPU only and about 1 min, with at least 16 GiB RAM. Disk: 70 GiB for F3 plus 69 GiB for A3../A3 must not exist beforehand, and expertexpansionmanifest.json must stay in./F3. Load./A3 with the model class named in its config.json. Intern-S2 ships a…
Open weights
apache-2.0
37.8B parameters
262,144 tokens
transformers
A REX checkpoint of Qwen3.5-35B-A3B with WMDP-Cyber knowledge relocated into 8 removable experts per MoE layer (264 experts in total). Excising these experts together with their router rows yields A3, a standard 256-expert model with that knowledge removed. This repository holds the expanded checkpoint F3: the REX model after 250M tokens of standard full-parameter SFT, with the removable experts still attached. A3 differs from F3 only by excision. CPU only and about 1 min, with at least 16 GiB RAM. Disk: 69 GiB for F3 plus 67 GiB for A3../A3 must not exist beforehand, and expertexpansionmanifest.json must stay in./F3. Load./A3 with the model class named in its config.json. Scores in %, ±…
Open weights
apache-2.0
37B parameters
262,144 tokens
transformers
A REX checkpoint of Qwen3.5-35B-A3B with WMDP-Bio knowledge relocated into 8 removable experts per MoE layer (264 experts in total). Excising these experts together with their router rows yields A3, a standard 256-expert model with that knowledge removed. This repository holds the expanded checkpoint F3: the REX model after 250M tokens of standard full-parameter SFT, with the removable experts still attached. A3 differs from F3 only by excision. CPU only and about 1 min, with at least 16 GiB RAM. Disk: 69 GiB for F3 plus 67 GiB for A3../A3 must not exist beforehand, and expertexpansionmanifest.json must stay in./F3. Load./A3 with the model class named in its config.json. Scores in %, ±…
Open weights
apache-2.0
37B parameters
262,144 tokens
transformers
A REX checkpoint of Intern-S2 with WMDP-Bio knowledge relocated into 8 removable experts per MoE layer (264 experts in total). Excising these experts together with their router rows yields A3, a standard 256-expert model with that knowledge removed. This repository holds the expanded checkpoint F3: the REX model after 250M tokens of standard full-parameter SFT, with the removable experts still attached. A3 differs from F3 only by excision. CPU only and about 1 min, with at least 16 GiB RAM. Disk: 67 GiB for F3 plus 65 GiB for A3../A3 must not exist beforehand, and expertexpansionmanifest.json must stay in./F3. Load./A3 with the model class named in its config.json. Intern-S2 ships a custom…
Open weights
apache-2.0
36.1B parameters
262,144 tokens
transformers
