Qwen3.5-9B-RL-IF-MATH-LOGIC

Qwen3.5-9B-RL-IF-MATH-LOGIC is a post-trained model built on Qwen3.5-9B. It was developed through supervised fine-tuning (SFT) on task-relevant non-thinking data, followed by two sequential stages of reinforcement learning with verifiable rewards (RLVR) using NGRPO: Negative-enhanced Group Relative Policy Optimization.

The first RLVR stage focuses on mathematical reasoning with wflying/math-rlvr-15k. The second stage continues from that checkpoint on wflying/rl-if-math-logic-46k, a mixture of instruction following, mathematics, and logical/table reasoning. The released model is the output of the second RLVR stage.

Under the reported non-thinking evaluation setup, the model achieves a mean score of 82.45 across eight benchmarks, compared with 64.92 for Qwen3.5-9B and 68.96 for Qwen3.5-35B-A3B.

Model details

• Base model: Qwen3.5-9B
• Training method: SFT followed by two stages of NGRPO-based RLVR
• SFT data: Task-relevant non-thinking examples
• RLVR stage 1 dataset: wflying/math-rlvr-15k
• RLVR stage 2 dataset: wflying/rl-if-math-logic-46k
• Primary tasks: Instruction following, mathematical reasoning, and logical reasoning
• Architecture: Qwen3.5 dense multimodal conditional-generation model (Qwen3_5ForConditionalGeneration)
• Evaluation mode: Non-thinking (enable_thinking=False)

Training procedure

Both RLVR stages use NGRPO. The algorithm introduces advantage calibration to obtain learning signals from uniformly incorrect response groups and asymmetric clipping to control policy updates. See the NGRPO paper for the algorithm and its original experiments.

The stages are applied sequentially: the mixed-task stage starts from the mathematics RLVR checkpoint, and the final checkpoint is released as Qwen3.5-9B-RL-IF-MATH-LOGIC.

Evaluation results

Evaluation was conducted with EvalScope on instruction-following, mathematics, logical-reasoning, and knowledge/reasoning benchmarks, using:

• Temperature: 0.6
• Top-p: 0.95
• Thinking mode: Disabled (enable_thinking=False)

The table reproduces the release author's reported comparison results. Mean is the unweighted arithmetic mean of the eight displayed benchmark scores, rounded to two decimal places.

Relative to Qwen3.5-9B under this evaluation setup, the released model improves the mean score by 17.53 points. It gains 7.44 points on IFEval, 25.75 on IFBench, 19.38 on AIME24, 33.34 on AIME25, 34.17 on HMMT25, and 21.80 on ZebralogicBench. MMLU-redux increases by 0.35 points, while GPQA-diamond decreases by 2.02 points.

Its mean score is also 13.49 points above the Qwen3.5-35B-A3B baseline in this comparison. These comparisons apply to the reported benchmarks and decoding setup; they do not establish superiority across all tasks or inference settings.

Usage

Load the model directly from Hugging Face using a recent version of transformers with Qwen3.5 support:

pip install -U transformers accelerate torch torchvision

The following example uses non-thinking mode and the reported temperature and top-p values:

import torch
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

model_id = "wflying/Qwen3.5-9B-RL-IF-MATH-LOGIC"

model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_id,
    dtype="auto",
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_id)
model.eval()

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Solve the problem and put the final answer in \\boxed{}: What is the sum of the first 20 positive integers?",
            }
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    enable_thinking=False,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    generated_ids = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=True,
        temperature=0.6,
        top_p=0.95,
    )

generated_ids = generated_ids[:, inputs.input_ids.shape[1]:]
print(processor.batch_decode(generated_ids, skip_special_tokens=True)[0])

max_new_tokens=512 is an example generation limit. Reproducing benchmark scores also requires the corresponding benchmark prompts, generation limits, scoring configuration, and evaluation versions.

Intended use

This model is intended for research and development involving:

• instruction following with verifiable constraints;
• mathematical and logical reasoning in non-thinking mode;
• staged RLVR post-training and NGRPO experiments;
• evaluation of mixed-task reasoning and instruction-following behavior.

Limitations

• The model may still produce incorrect or unverifiable outputs; reference answers and reward checks can also contain errors.
• Performance varies by benchmark. The reported improvements are accompanied by a decrease on GPQA-diamond.
• Scores depend on prompts, benchmark versions, sampling settings, generation limits, and scoring implementations. The table reports the release author's evaluation results.
• Only the base models and final checkpoint are included in the comparison; it does not isolate the contribution of SFT or either RLVR stage.
• The training data and reported evaluations focus on text tasks. The results above do not characterize multimodal performance.
• The SFT dataset identifier, full training hyperparameters, and complete evaluation run configuration are not specified in this card.

License

This repository is licensed under Apache License 2.0, consistent with the base model. Users should also review the terms and attribution requirements of the base model, training datasets, and dependent components.

Citation and references

Please cite the NGRPO paper when referring to the RL algorithm:

@misc{nan2025ngrpo,
  title={NGRPO: Negative-enhanced Group Relative Policy Optimization},
  author={Gongrui Nan and Siye Chen and Jing Huang and Mengyu Lu and Dexun Wang and Chunmei Xie and Weiqi Xiong and Xianzhou Zeng and Qixuan Zhou and Yadong Li and Xingzhong Xu},
  year={2025},
  eprint={2509.18851},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  url={https://arxiv.org/abs/2509.18851}
}

Also refer to the Qwen3.5-9B base model, math-rlvr-15k, rl-if-math-logic-46k, and EvalScope for their documentation and source attribution.