doom-dense-arnold-latents

DoomDiT per-tic latents

Derived from the public raw dataset RohanNaga/doom-dense-arnold: every tic of each episode encoded once through a frozen Stable Diffusion autoencoder, stored as one .npy per episode plus a .npz sidecar with the per-tic metadata. Released so that a training run can start without re-encoding, and so that latents are reproducible bit for bit: latent bytes depend on the autoencoder, its scaling, the precision and the encode batch size, all recorded in encode_meta_*.json inside each folder.

Frames are 320x240 padded to 320x256 with mid-gray before encoding; decoders crop the padding ([:, :, :240] in pixels, the last row block in latents). Encoding is bf16 autocast at the batch size named in encode_meta_*.json; a re-encode at another batch size differs at the last bits.

Sidecar columns (one row per tic): tic, action (Arnold's requested id), buttons (the EXECUTED 19-button control, see the raw dataset card for the mechanism), buttons_raw_len, switch_requested_index, health, ammo, kills, deaths, frags, pos_x, pos_y, angle, map_id, episode_id, is_decision, chain_id. Row i holds the latent of the frame at tic i and the control applied from tic i to i+1.

Code: encode_parquet.py --every-tic in https://github.com/RohanNaga/Doom. Split: dense_split.json in the raw dataset (train 0:6000, val 6000:7000, test 7000:8000; unseen maps = arenas 6, 7, 8).