Laya (Verdict-ready)
The Laya System One decision model, exported to ONNX fp32 and packaged for Verdict -- a high-performance serving engine for System One decision models. Serve it with one command:
Pick the image that matches the host. The -v mounts cache the model, so only the first
run downloads it.
# CPU host
docker run --rm -p 8080:8080 \
-v ~/.cache/verdict:/root/.cache/verdict \
sgyaqing/verdict:0.2.0-cpu serve verdict-models/laya --host 0.0.0.0
# GPU host
docker run --rm --gpus all -p 8080:8080 \
-v ~/.cache/verdict:/root/.cache/verdict \
sgyaqing/verdict:0.2.0-cuda serve verdict-models/laya --host 0.0.0.0
Add -e VERDICT_USE_MODELSCOPE=1 and use sgyaqing/laya to fetch from ModelScope instead of
the hub; add -e HF_ENDPOINT=https://hf-mirror.com if the hub is slow or unreachable.
Endpoints: the standard decision protocols POST /v1/systemone and POST /v1/decisions, native POST /v1/choice | /v1/score | /v1/noul, plus an OpenAI-compatible layer (/v1/chat/completions, /v1/models).
Performance
Measured on one machine (RTX 4090, AMD EPYC 7543, 16 threads), end to end over HTTP with
oha and an identical request body for both servers. The
baseline is the model author's own server (laya-serve, LAYA_MODELS=english); the Verdict
column is this package. Latencies are per-request P50 / P99 at the stated concurrency. The
ratios are oriented so that above 1× means Verdict is ahead in both: throughput ratio =
Verdict ÷ baseline, latency ratio = baseline ÷ Verdict.
| concurrency |
Verdict rps |
Verdict P50 / P99 |
laya-serve rps |
laya-serve P50 / P99 |
ratio rps |
ratio P50 / P99 |
| 1 |
121.4 |
8.37 / 11.07 ms |
34.4 |
28.58 / 32.73 ms |
3.53× |
3.42 / 2.96× |
| 8 |
501.4 |
15.99 / 22.42 ms |
35.4 |
227.96 / 250.60 ms |
14.2× |
14.3 / 11.2× |
| 64 |
1353.0 |
46.71 / 65.19 ms |
refused requests |
— |
— |
— |
Files
| File |
Purpose |
model.json |
manifest: graph file name, budgets, calibration temperatures |
laya.onnx + laya.onnx.data |
the ONNX graph (fp32, external weights) |
tokenizer/tokenizer.json (+ tokenizer_config.json) |
the tokenizer |
SHA256SUMS |
checksums for every file above |
Provenance & fidelity
Weights are unchanged from convaiinnovations/laya; this repo adds only the ONNX export and the serving manifest. Conversion and serving are verified against the official model: identical answers on CPU, within 1e-4 on CUDA.
License
Apache-2.0, inherited from the base model. Credit for the model itself belongs to the original authors; this repository only repackages it for serving.