gpt-oss-120b served with vLLM on four Blackhole chips (a QuietBox 2, P150x4, tensor parallel 4). A lone request decodes through a fused batch-1 decode: DRAM-streaming expert matmuls that read the 128 BFP4 experts in place (one ND-sharded copy shared with prefill and the batched decode), on-core top-4 routing over the 128 router logits, and all-reduces fused into the producing kernels. Batches of 2-32, prefill, prefix caching and requests that need penalties, log-probs, top-k > 32, logit bias or grammars run the batched paths. Status: round-2 release candidate (stage 14): measured on p150x4 against tt-hous/gpt-oss-120b-p150x4 on the same day: c=1 decode 3.8x, GSM8K-250 and IFEval-100 above…
Open weights
apache-2.0
vllm
Runs on p150 (one Blackhole P150 chip), p150x2 (the two P150 ASICs of one P300C board) and p150x4 (the four P150 ASICs of two P300C boards, a QuietBox 2). Same model, same serving stack and the same six profiles as tt-hous/gpt-oss-20b-p150, with faster decode: a lone request on P150x4, P150x2, P150 decodes through fused, DRAM-streaming decode ops (from tenstorrent/tt-metal#60161, adapted to this runtime and extended to one and two chips). On P150x4, 2 to 4 decoding requests share one pass of those ops, and 5 to 32 run the base package's batched decode with the routed experts streamed from the fused path's weight copies. Prefill takes the base package's code path with faster expert dispatch…
Open weights
apache-2.0
vllm
