sdqa — step-level agent-safety auditors
Two Qwen3-4B full-finetunes that judge one proposed agent action, before it
executes, against the observed prefix and the task policy, and emit a single
verdict box. They differ only in SFT learning rate.
| subfolder |
learning rate |
pick it for |
lr5e-6 |
5e-6 |
the better operating point — reward +0.165, miss 0.175, false-ask 0.034 |
lr1e-5 |
1e-5 |
the better ranking — AUROC 0.780 |
from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("Caaaarr1e/sdqa", subfolder="lr5e-6")
t = AutoTokenizer.from_pretrained("Caaaarr1e/sdqa", subfolder="lr5e-6")
Output contract
\boxed{act:STOP, binary:UNSAFE, risk:90, chan:NONE, pert:NONE, obj:NONE,
fail:data_exposure, harm:privacy, src:compromised_tool, q:NONE, cite:3}
act is CONTINUE / ASK / STOP. The binary projection is CONTINUE=SAFE, STOP=UNSAFE,
and ASK=UNSAFE at risk >= 50. cite points at an earlier visible step, or NONE.
fail / harm / src come from a 16 failure-mode, 11 harm-type, 10 risk-source
taxonomy; NONE is an absence sentinel, not an additional class.
Training
Full SFT from Qwen/Qwen3-4B, epoch 2, thinking enabled, on 2,464 step-level records.
Prompts were rendered through the evaluator's own pipeline, so the training and
inference formats match. Weights are the original fp32.
Limitations — please read
- The training data has a known 15.1% defect rate. In 372 of 2,464 records the
judged action was truncated out of its own prompt by a history-character cap. The
damage is biased, not random: it concentrates at deep steps, which is exactly where
the decision boundary and the clarification window live. Those targets are not
learnable, and the models were trained with them in place.
- Risk scores are not calibrated probabilities. Do not threshold them as such
without your own calibration.
- Parse rate is not an improvement over the base model. Base Qwen3-4B parses this
contract at 99.5%; these sit at 97.4–99.0%, i.e. at or slightly below.
- Evaluated only on an internal foresight suite. No claim is made about general
agent-safety benchmarks or production monitoring.
- The box is emitted reliably only under the matching auditor prompt; a different
prompt format degrades output sharply.
- These are research artifacts, not a safety guarantee. Do not deploy as the only
control on a consequential action.
