jehosephat

Jehosephat is a non-autoregressive decision model that answers questions about text or structured data. It returns probability distributions over your supplied questions in one forward pass, without generating text. Several questions may be asked in parallel. Questions can select a category (choice), score ordered levels (score), or estimate the probability of a proposition (noul).

The model architcture is a Qwen3-1.7B-Base backbone with a rank-32 LoRA adapter and a shared scalar decision head.

Jehosephat is inspired by TypeSafe's Jev, and so the query schema draws on TypeSafe Primitives.

Prerequisites

pip install "torch==2.14.0" "transformers==5.17.0" "peft==0.21.0"

Use

Load Jehosephat and ask named questions:

from transformers import AutoModel

model = AutoModel.from_pretrained(
    "rhizomatous/jehosephat",
    trust_remote_code=True,
)
answers = model.predict(
    state={"message": "My order arrived broken."},
    questions={
        "route": {
            "type": "choice",
            "instructions": "Which team should handle this message?",
            "criteria": {
                "returns": "Damaged products, refunds, and replacements",
                "shipping": "Delivery tracking and delays",
            },
        },
    },
)

The model weights download automatically on first use. Jehosephat uses an available NVIDIA or Apple GPU automatically, and runs on CPU otherwise.

state accepts text or JSON-serializable data. Answers use your question names; those names do not enter the model input. Every answer contains type and a probabilities map whose values sum to 1. You can ask different question types together:

answers = model.predict(
    state={"message": "My order arrived broken."},
    questions={
        "route": {
            "type": "choice",
            "instructions": "Which team should handle this message?",
            "criteria": {
                "returns": "Damaged products, refunds, and replacements",
                "shipping": "Delivery tracking and delays",
            },
        },
        "severity": {
            "type": "score",
            "instructions": "How serious is the customer's problem?",
            "criteria": [
                "No problem reported",
                "Minor inconvenience",
                "Product unusable",
            ],
        },
        "replace": {
            "type": "noul",
            "instructions": "Does the customer need a replacement?",
        },
    },
)

Results are plain Python dictionaries.

Score's probability keys are "0", "1", and so on. Score includes a legend mapping its level numbers to your descriptions. A three-level Score ranges from 0 to 2 and may be fractional. Its confidence refers to the most likely level, not a probability that the numerical expected score is correct.

A request supports 1–32 questions, 2–255 options per question, and at most 5,376 packed tokens in total, including formatting, state, questions, and options.

Architecture

The shared state, question text, and option branches are packed into one sequence. Each option is scored using the shared state, its question’s instructions, and its own key and description. Other questions and options are excluded from its context using a causal tree attention mask.

One linear scalar head scores every option's decision marker. A softmax normalizes those scores within each question. Inference uses SDPA with a dense tree mask.

Training

Jehosephat was trained on the following annotated and generated data. Targets preserve human annotation distributions or exact probabilities calculated by enumeration. Training data is not included in this model distribution.

This data was used to train a LoRA. LoRA rank was 32, alpha 64, and dropout 0.05, applied to attention projections and MLP gate/up/down projections. Adapter learning rate was 0.0002; head learning rate was 0.001. Microbatches held two requests with four gradient-accumulation steps. Weight decay was 0.01 and warmup ratio 0.05. Training permuted questions and options and forbade state truncation. The loss was cross-entropy plus 0.25 times Brier loss, averaged over questions within each request and then over requests; ordinal RPS weight was zero.

Limitations

Options are scored independently before normalization. An option such as “none of the above” cannot inspect which other alternatives were supplied. Overlapping options, comparisons between sibling options, and set-dependent complements can therefore fail. Use self-contained criteria and put information needed by every option in the shared state or question. Adding options changes normalization, but does not let existing branches reason about the new options.

Training primarily covers English and a limited set of generated and annotated tasks. The aggregate results are heavily influenced by Noul questions, which were dominant in the training data.

Development

From a source checkout, install dependencies and development tools with uv:

uv sync --locked

Pytest and Ruff are used for checks. The release-loading and attention isolation tests run the actual model on CPU and require enough memory for its weights and inference.

Attribution

Jehosephat is licensed under the Apache-2.0 license. See LICENCE.qwen3.md for the license of the underlying Qwen3 model.

Citation

If you use Jehosephat in your work, please cite:

@misc{shaw2026jehosephat,
  author = {shaw, viv},
  title = {Jehosephat},
  year = {2026},
  url = {https://huggingface.co/rhizomatous/jehosephat}
}