Five 160M GPT-NeoX training runs on the separate M/C/G source pools in Dolma3-16B. Clean, M500, C500, G500, and MCG500 are separate run directories under runs/. Each contains all existing step-.pt checkpoints, including the final step-122071.pt after 16,000,090,112 tokens. Intermediate checkpoints are approximately 1B tokens apart, with additional scheduled steps. Standalone final weights are under models/ /model.safetensors, with a matching config and tokenizer in each run directory. Load one with GPTNeoXForCausalLM.frompretrained("294-288-pretrain-interaction/160M-exp", subfolder="models/Clean") and the tokenizer from the same subfolder. Each.pt is a PyTorch training checkpoint containing…
Organization
294 288 Pretrain Interaction
294-288-pretrain-interaction
Models in Library2
Datasets in Library0
Models on Hugging Face2
Followers4
Compare the published evidence
This profile connects the 2 model and dataset records shown below to their upstream cards and sources. The library counts describe SAVRN's coverage, rather than this publisher's entire output.
Published tasks: Text generation.
Compare model specifications, source dates, and hardware estimates on the individual records. Publisher identity verification is separate from a model's quality, deployment readiness, or suitability for your work.
Models
This repository collects Yihan's 24 research runs: six injection doses (10, 25, 50, 100, 250, 500) and four source allocations (M, C, G, MCG). The exact trigger is dAtA[iNdEx]. Models use the Pythia-160M architecture and train from random initialization, seed 42, for 16,000,090,112 tokens on the frozen Dolma Math/Code/General mixture. The campaign is in progress. A run is released only when its checkpoint audit and all scheduled evaluations pass. releases/.json records uploaded file checksums and the verified Hub revision. Missing runs are pending, not zero-valued results. The repository is public, matching the group's reference release. Scheduled checkpoints may appear before a run is…