Robert Francis

Observed sparse-MMA microbenchmark peak on this Vast GB10: 992.018 TFLOP/s dense-equivalent, 99.20% of the 1 PFLOP/s target A ~2B-parameter language-model architecture designed around NVIDIA GB10 / SM121 sparse NVFP4 from the start, rather than retrofitting a generic Transformer. - 1,990,038,784 conventional parameters (0.498% below the 2B design target) - 600,000,000,000 training-token target (~302 tokens/parameter) - 16 sequential stages - 6 top-1 experts per stage, 96 experts total - model width 1280 - every expert uses 1280 -> 5120 -> 1280 SwiGLU - expert matrices are paired 4:8 structured sparse NVFP4 - native packed mxf4nvf4 SM121 sparse MMA is the target hot path - 20 query heads / 5…

Continuation of checkpoint backups for MarxistLeninist/AGILLM-4.3, which reached Hugging Face's 20,000-file repo cap on 2026-08-20 and is now a frozen archive (checkpoints there end at step2141734). - checkpoints/stepNYYYYMMDD/ - promoted serving-hold checkpoints (same format as the old repo). - live400B/stepNTIMESTAMP/ - periodic raw backups of the running continue-400B training run (block-sharded-zstd checkpoint + tokenizer + runstate), newest few kept for disaster recovery. Old checkpoint dirs are pruned automatically to stay under the file cap. This is the checkpoint to use. It is the first checkpoint trained after the fused cross-entropy gradient bug was found and fixed, and it is the…