Settled is the mean of the last three evals (52.83, 53.05, 52.81); best 50.97 at step 31,000, final 52.81. Consecutive evals near the end differ by about ±2 PPL. - The hardening recovers what the unhardened field lost: 2.4% better than the same model without it. - The hardened field still adds nothing measurable: 0.4% better than the model without the field, which is within eval noise. The three L=2 models with the register path settle within 2.5% of one another, whatever else is switched on. - Without the registers the same field is worth much more: the conservative-only model with this field and no register read settles at 54.27, 6.0% below the conservative-only model. The field and the…
Open weights
cc-by-4.0
pytorch
One arm of the SemSimula mechanism ladder — OpenWebText, d=384, L=2: five models that differ by exactly one mechanism each, so that every gap in the ordering prices one component by construction rather than by attribution. Italic entries are pre-registered predictions, recorded before the run. The ordering is itself the prediction; any inversion is a result. Every arm above sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — on the same corpus, tokenizer and validation batches. No arm is trained longer than another, so a gap in the table is a gap between mechanisms and never a difference in budget. The models below integrate a damped mechanical system…
Open weights
cc-by-4.0
pytorch
One arm of the SemSimula mechanism ladder — OpenWebText, d=384, L=2: five models that differ by exactly one mechanism each, so that every gap in the ordering prices one component by construction rather than by attribution. Italic entries are pre-registered predictions, recorded before the run. The ordering is itself the prediction; any inversion is a result. Every arm above sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — on the same corpus, tokenizer and validation batches. No arm is trained longer than another, so a gap in the table is a gap between mechanisms and never a difference in budget. The models below integrate a damped mechanical system…
Open weights
cc-by-4.0
pytorch
One arm of the SemSimula mechanism ladder — OpenWebText, d=384, L=2: five models that differ by exactly one mechanism each, so that every gap in the ordering prices one component by construction rather than by attribution. Italic entries are pre-registered predictions, recorded before the run. The ordering is itself the prediction; any inversion is a result. Every arm above sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — on the same corpus, tokenizer and validation batches. No arm is trained longer than another, so a gap in the table is a gap between mechanisms and never a difference in budget. The models below integrate a damped mechanical system…
Open weights
cc-by-4.0
pytorch
One arm of the SemSimula mechanism ladder — OpenWebText, d=384, L=2: five models that differ by exactly one mechanism each, so that every gap in the ordering prices one component by construction rather than by attribution. Italic entries are pre-registered predictions, recorded before the run. The ordering is itself the prediction; any inversion is a result. Every arm above sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — on the same corpus, tokenizer and validation batches. No arm is trained longer than another, so a gap in the table is a gap between mechanisms and never a difference in budget. The models below integrate a damped mechanical system…
Open weights
cc-by-4.0
pytorch
One arm of the SemSimula mechanism ladder — OpenWebText, d=384, L=2: five models that differ by exactly one mechanism each, so that every gap in the ordering prices one component by construction rather than by attribution. Italic entries are pre-registered predictions, recorded before the run. The ordering is itself the prediction; any inversion is a result. Every arm above sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — on the same corpus, tokenizer and validation batches. No arm is trained longer than another, so a gap in the table is a gap between mechanisms and never a difference in budget. The models below integrate a damped mechanical system…
Open weights
cc-by-4.0
pytorch
Settled is the mean of the last three evals (54.20, 54.39, 54.22); best 52.50 at step 31,000, final 54.22. Consecutive evals near the end differ by about ±1 PPL. - The field alone is a real but weaker route. It lowers the conservative-only model's perplexity by 6.0%; the register path alone lowers it by 8.0%. - Beside the registers it adds almost nothing: G3′ is 0.4% better than G2. The two routes largely substitute for each other. This model sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — the same budget, corpus, tokenizer and validation batches as every other model in the table. No model is trained longer than another. - The base is F3.1: the…
Open weights
cc-by-4.0
pytorch
