machinery

zot machinery sessions

Every conversation behind every machine in the zot machinery - a software factory where an agent takes the same standing order every half hour, reads the catalogue of what already exists, looks at the last few machines on the shelf, and designs, writes, commissions and ships one brand-new machine: a working control panel for a machine that does not exist, with a live simulation behind it, faults you can trigger and recover from, and the operating manual that opens from the panel.

Each row is one shift: the full agent trajectory from the order to the finished machine (or to where the shift was cut short), in the chat shape the rest of the ecosystem reads, plus what the machinery knows about the result.

What makes these rows different from the arcade's: the order requires the model to look. It screenshots the panel it built, views the screenshots, judges the design against the brief it wrote and against the machines already on the shelf, and changes the design at least once because of what it saw. The images are in the rows, next to the turn in which the model looked at them and the turn in which it decided what to do about it - a design-critique loop with the evidence attached.

The rows are appended by the machinery's own workflow as shifts happen. Nobody reviews them; they are the record as it was written.

Layout

trajectories/<session-id>/<session-id>.jsonl   one row
trajectories/<session-id>/images/<digest>.png  the screenshots the model was shown

Row schema

Messages

OpenAI chat convention, with additive fields:

{"role": "user",      "type": "user",        "content": "Begin working on your task..."}
{"role": "assistant", "type": "bot",         "content": "", "reasoning": "...", "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "read", "arguments": "{\"path\":\"site/machines.json\"}"}}]}
{"role": "tool",      "type": "activity",    "tool_call_id": "call_1", "name": "read", "content": "[...]"}
{"role": "assistant", "type": "bot",         "content": "", "tool_calls": [{"id": "call_2", "type": "function", "function": {"name": "view", "arguments": "{\"path\":\"/tmp/machinery/kestrel-tug-desktop.png\"}"}}]}
{"role": "user",      "type": "attachment",  "content": [{"type": "text", "text": "/tmp/machinery/kestrel-tug-desktop.png"}, {"type": "image", "image": "images/3f2a....png"}]}
{"role": "system",    "type": "checkpoint",  "content": "summary of the conversation so far"}

• type is zot's own message type: user, bot, activity, attachment, checkpoint (a compaction summary), instructions.
• reasoning is the model's scratchpad for the turn, when the provider surfaced it.
• Tool arguments are the JSON string the model sent, verbatim.
• A view call is followed by an attachment carrying the image the model saw; the bot turn after it is the model's reaction to it.
• The system prompt is not recorded; task is the brief.

messages is what a resume of the run would replay. After compaction that is a checkpoint plus the recent turns - snapshots holds the earlier states, so every turn that happened is in the row, at the price of the recent ones being repeated.

Filtering

• Finished machines: complete and machinery.outcome == "settled" and machinery.catalogue_check == "success" and machinery.probe == "success".
• A shift that was cut short ships as a partial row; the shift that continued it ships the whole chain under its own id, with the earlier id in chain. Drop rows whose id appears in another row's chain to keep only tips.
• Design-critique turns: every view tool call whose attachment path ends in -desktop.png, -phone.png or -manual.png, and the bot turn that follows it. The probe writes those names; the order requires the model to look at them.
• The shelf as a whole: machinery.machine across settled rows is the catalogue - no two rows share a kind, a domain + era, a design, an interaction or a colour story, by construction.

Provenance

• Factory: https://github.com/openzot/machinery - the standing order is orders/new-machine.yaml, the design discipline and the contract the agent reads are AGENTS.md, the gate is scripts/check.sh + scripts/probe.js.
• Harness: zot; rows are produced by zot sessions export and shipped by scripts/ship.py after each shift.
• Model: whatever the workflow names at the time; each row says which.
• Sibling dataset: openzot/arcade, the same factory shape making browser games.

The machines are model output, published as-is under the machinery's license. Check the terms of the model named in a row before training on its reasoning.