Yelarys Yertaiuly

LoRA adapters from a study of reward hacking in GRPO. In the coding environment, the model can earn full reward by writing its own runtests function that passes regardless of whether its solution is correct. Baseline runs learn to do this. The study tests whether removing a learned direction from the residual stream during RL (CAFT) prevents it. 1cfa9a7208912126459214e8b04321603b3df60c. - LoRA rank 32, alpha 32, on all attention and MLP projections. - Each folder holds adapterconfig.json and adaptermodel.safetensors. wave2/ -seed /step /: GRPO runs with 4 arms x 5 seeds, checkpoints at steps 100, 150 and 200 (200 steps, 16 prompts x 16 answers per step). The removal acts at block 19…

Backups of two GRPO baseline runs on Qwen/Qwen3-4B (revision 1cfa9a7208912126459214e8b04321603b3df60c). Both use LoRA rank 32, seed 2, and 16 prompts × 16 answers per step. No direction is removed (a structural-zero hook at block 19). Both runs learned to hack the grader by weakening the tests: modify from about step 90, incontext from about step 120. Layout. backups/wave3loopholes/runs/ / holds every directory of each run. Runs that were continued after an interruption carry -resume... suffixes, and each step is taken from the newest directory that has it. A directory holds: - run configuration and build records; - LoRA adapters every 10 steps; - the full training state at steps 0, 100…