qwen38-27b-swerebench-grpo-step8-training-state

This repository preserves the completed Step 8 checkpoint from a full-parameter GRPO run using a 1,586-task SWE-rebench v2 Filtered corpus. It is a Miles/Megatron distributed checkpoint, including model and optimizer state, metadata, and the dataset cursor. It is intended for research recovery and continued training. The model weights for direct inference are published separately as a Hugging Face safetensors export.

The archived run used eight NVIDIA B300 GPUs with tensor parallelism 2 and context parallelism 4. Each step requested 32 prompt groups with eight rollouts per group (global batch size 256); the learning rate was 2e-6. Training used mini-swe-agent, independent sandboxes and shell actions. This checkpoint belongs to the no-branch GRPO baseline and is not a branching-policy checkpoint.

Files and integrity

iter_0000008/ contains eight __*_0.distcp shards, DCP .metadata, metadata.json, and checkpoint_complete.json. The adjacent rollout/ directory contains global_dataset_state_dict_8.pt. The two latest_*_iteration.txt files point to iteration 8.

Run this from the repository root after download:

sha256sum -c iter_0000008.SHA256SUMS

The SHA-256 of iter_0000008/checkpoint_complete.json is 006843311874c1a6fb467aaedee05ad84919eb6a82ef0613e8fd287b6ceffe18. The completion marker records the training configuration, including the source prompt SHA-256 7c99144299c2491499a19813d2b42efbc482766af8bc7ee4aeb9968675681b1e.

To resume training, use the matching Miles/Megatron code, the original Qwen3.8-27B base assets, the exact derived prompt data, and compatible task images and verifier services. Those external inputs are not part of this checkpoint repository. The DCP .metadata is a PyTorch pickle and should only be loaded from a trusted source.

Base model: Qwen/Qwen3.8-27B (Apache-2.0).

Reconstructing the distributed checkpoint

The eight .distcp shards are stored in numbered pieces to stay below the Hub's per-file limit. After downloading the repository, run the following from its root. Keep all pieces until the SHA-256 check succeeds.

python3 - <<'PY'
import json
from pathlib import Path
manifest = json.loads(Path('parts.json').read_text())
for name, record in manifest['shards'].items():
    with Path(name).open('wb') as output:
        for part in record['parts']:
            with Path(part['path']).open('rb') as source:
                while data := source.read(8 * 1024 * 1024):
                    output.write(data)
PY
sha256sum -c iter_0000008.SHA256SUMS