Conservative-only control — the Fock mechanism off
One arm of the SemSimula mechanism ladder — OpenWebText, d=384, L=2: five models that differ by exactly one
mechanism each, so that every gap in the ordering prices one component by
construction rather than by attribution.
This arm removes: the Fock mechanism — the register-to-token path.
The ladder
Italic entries are pre-registered predictions, recorded before the run.
The ordering is itself the prediction; any inversion is a result.
Every arm above sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — on the same corpus, tokenizer and validation batches. No arm is trained longer than another, so a gap in the table is a gap between mechanisms and never a difference in budget.
What each gap prices
| gap |
prices |
value |
| Matched GPT-2 baseline → Fock-PARFLM, exchange field on |
what the transformer has that this architecture does not |
13.70 PPL (+27.5%), measured |
| Fock-PARFLM, exchange field on → Fock-PARFLM, exchange field as a potential |
the price of conservativity |
17.39 PPL (+27.4%), measured |
| Fock-PARFLM, exchange field as a potential → Fock-PARFLM, no exchange field |
the exchange field |
-13.92 PPL (+-17.2%), measured |
| Fock-PARFLM, no exchange field → Conservative-only control — the Fock mechanism off |
the Fock mechanism (the register-to-token path) |
20.95 PPL (+31.3%), measured |
| Fock-PARFLM, no exchange field → Fock-SPLM — no Vφ, Fock mechanism on |
Vφ / PARF, with the register mechanism present |
pending |
| Conservative-only control — the Fock mechanism off → Multi-ξ SPLM — no Vφ, no Fock |
Vφ / PARF, without it — the 2×2's second reading |
pending |
| Fock-SPLM — no Vφ, Fock mechanism on → Multi-ξ SPLM — no Vφ, no Fock |
the Fock mechanism again, on a Vφ-free base (replication) |
pending |
What the ladder is measuring
The models below integrate a damped mechanical system in semantic space.
With the register mechanism on, the equation of motion carries a
non-conservative term and is Lagrangian only in the
Lagrange–d'Alembert sense:
$$m \ddot{h} = -\nabla V_\theta(h) - \gamma m \dot{h} + F_{\mathrm{rc}}(h, r) + F_\phi(h)$$
\(V_\theta\) is the pointwise scalar potential, \(V_\phi\) the pairwise one,
and \(F_{\mathrm{rc}}\) the reverse channel through which the register bank
acts on the token state. \(F_{\mathrm{rc}}\) is not the gradient of anything
in \(h\), so it is the term that makes the trajectory non-geodesic; the
damping \(-\gamma m \dot{h}\) is parallel to the velocity and changes only
the speed along the path, never the path itself.
Setting \(F_{\mathrm{rc}} = 0\) (the none-norc arm) leaves every force a
gradient, so \(V = V_\theta + V_\phi\) admits a Jacobi metric and the step is
its damped geodesic step, up to the LayerNorm projection. That arm is
the only fully conservative model in the collection, and the ladder prices
what its geodesics cost.
This model
|
|
| architecture |
as 'none' but REVERSE_CHANNEL=False: registers are created and destroyed but never reach the tokens, so every force is the gradient of a scalar potential |
| parameters |
76,572,110 |
| training tokens |
532,480,000 — the same on every arm |
| learning rate |
0.0012 |
| settled PPL |
87.93 |
| best PPL |
85.90 (step 31,000) |
| final PPL |
88.82 |
| ratio to matched GPT-2 |
1.765× |
| pre-registration |
105, band 85-140 — BAND HIT, point high by 16% |
| run health |
0 watchdog, 2.6% clip-hit (max 2.05), sig_max drifts 14.3->60.1 on the auxiliary terms only |
Protocol
- Corpus OpenWebText (GPT-2 BPE, 50,257); 32,500 steps × 32 × 512 = 532,480,000 tokens.
- Settled PPL is the mean of the last three 500-step evals at block 512 — not the best checkpoint.
- Shared across all arms: tokenizer, corpus, validation batches, width d=384, block 512, effective batch 32, token budget (532,480,000 tokens, identical on every arm), WSD schedule (warmup 5%, stable to 65%, decay to 100%, floor 6e-05).
- Not shared: depth (GPT-2 L=8 vs Fock L=2) and the learning rate of the GPT-2 arm (6e-04, separately tuned).
- Full protocol, pre-registered bands and run health:
Depth_Ladder_and_Matched_Baseline_Protocol.md §5.6.
Reproduction
Set in Cell 0 of colab_fock_cfc_baoab_lowrank_depth_ladder_openwebtext_d384.ipynb
and run Cell 0 → 1 → 1b → 1c → 1d → 2 → 3 → 4 → 5 → 5b → 6:
LADDER_MECHANISM='none', REVERSE_CHANNEL=False, LADDER_L=2, LADDER_LR=1.2e-3
Training artefacts were written to semsimula_fock_cfc_baoab_owt_xi5long_topk16_dt32da16_mh4_aniso_dcvt5x8_vtjoint_cgqk_norc_L2probe_ob_untied_wsd_e5c_plgate_rep0.05_fockreg0.005_g0.1_baoab_cfc_lowrank_idt4_lr0p0012_noattn/.
Files
| path |
what |
README.md |
this card |
config.json |
architecture, training and result, machine-readable |
checkpoints/*_best.pt |
weights at the best eval |
checkpoints/*_step500_best.pt |
weights at step 500, an initialisation reference |
results/training_log.jsonl |
per-eval record; the settled PPL above is the mean of its last three rows |
results/*_training_output.txt |
the full run log, including per-step diagnostics |
results/Cell-6b-*_output.txt |
post-training probe logs (geometry, refinement, per-token forcing) |
code/ |
everything needed to reproduce the run, below |
The code, and why this subset
code/ mirrors the layout of the companion repository, so the notebook's
own sys.path lines resolve unchanged.
| path |
what |
code/colab_..._depth_ladder_openwebtext_d384.ipynb |
the run. One notebook serves every arm; the Cell 0 knobs above select this one |
code/data_module.py |
batching over the tokenised corpus |
code/parf/*.py |
the model as trained: 12 modules, the transitive import closure of what the notebook imports and nothing more |
code/parf/cfc_baoab.py |
the integrator. Its A-substep is the exact harmonic flow, which is what removed Velocity-Verlet's stability bound |
code/parf/test_cfc_baoab.py |
the integrator's own test, which Cell 4 runs as a gate before building the model |
The closure was computed by walking the imports, not chosen by hand, so
the set is reproducible and minimal: drop any file and an import fails.
Caveats
- L=2 only. Every trajectory-level claim is scoped to two layers; the
L=4 rung is queued.
- OpenWebText only. How the work divides between the conservative
potential and the register mechanism may be a property of the corpus;
a TinyStories replication is designed but not run.
- Depth is not matched between the GPT-2 arm (L=8) and the Fock arms
(L=2). The ratio column is reported for orientation, not as a
like-for-like efficiency claim.
- Ablations overstate. Removing this mechanism from a trained model
at inference indicated 3.75× to 3.91×; training without it gives 1.31×.
Inference-time ablation overstated the mechanism by about 2.9× in PPL
ratio and 4.9× in nats. Only trained-without numbers appear here.
Citation
@techreport{Gueorguiev2026SemanticSimulation,
author = {Gueorguiev, Dimitar P.},
title = {Semantic Simulation: A Prescriptive Lagrangian Framework for Semantic Inference},
note = {Conservative Potentials, Non-Conservative Memory, and the Shared-Potential Separator},
year = {2026}
}
Card generated from _ladder/ladder.json on 2026-09-27.
