opendecision-flights

A small, auditable classifier that predicts the next step of an agent that helps find flight tickets. Given the current state, it returns one action from a fixed vocabulary with a probability for each, so an agent can act alone when it is confident and hand the decision to a person when it is not. It ships with a decision policy (config.toml) that blocks impossible actions and sets a minimum confidence per action.

Model details

Uses

Direct use. Decide the next step of a conversational agent (ask, search, present options, confirm, execute, end the conversation), route uncertain decisions to human review, and log decisions with feedback to collect training data.

Out of scope. Executing irreversible actions (payments, bookings) from a prediction alone, domains far from the training data without re-evaluating, and any use as a source of facts: the model chooses an action, it does not generate or verify content.

How to get started

from huggingface_hub import hf_hub_download
import skops.io as sio

model = sio.load(hf_hub_download("ludolua/opendecision-flights", "model.skops"))  # no types outside skops' trusted list

state = "Tenho origem GRU destino REC ida 12/11 e ainda não consultei voos"
probabilities = dict(zip(model.classes_, model.predict_proba([state])[0]))
print(max(probabilities, key=probabilities.get), probabilities)

Dependencies: scikit-learn, skops and huggingface_hub. Use a scikit-learn version compatible with the training one. The full toolkit (CLI, REST API, decision log, evaluation, policy) is in the OpenDecision repository.

Input format

A free-text description of the current situation in Portuguese, for example "Tenho origem GRU destino REC ida 12/11 e ainda não consultei voos".

Actions

Training details

Data. 48 hand-written synthetic examples in Portuguese; they do not come from real users. training_data.csv is included in this repository.

Preprocessing. States are normalized to the English tag format, then vectorized with TF-IDF over word unigrams and bigrams with accents stripped.

The solver is L-BFGS, run in warm-started chunks so training progress can be reported. Hyperparameters live in config.toml and can be searched with opendecision tune.

Evaluation

Method. stratified cross-validation with 5 folds, with 48 examples evaluated.

Results. Accuracy 0.5 and macro F1 0.502; always choosing the most frequent action would give 0.25. Expected calibration error: 0.182.

Only 48 examples evaluated: metrics vary a lot with so few cases.

Review thresholds. Decisions with confidence below the threshold go to a person:

Decision policy

config.toml holds the decision policy used with this model: rules that block impossible actions, a minimum confidence per action, risk levels and error costs, in units of one human review (review_cost = 1). opendecision calibrate-policy re-fits the thresholds from the costs.

Rules:

• none

Apply it with the OpenDecision package (pip install git+https://github.com/leduardoaraujo/OpenDecision):

import tomllib
from opendecision.policy import apply, from_dict

with open(hf_hub_download("ludolua/opendecision-flights", "config.toml"), "rb") as file:
    policy = from_dict(tomllib.load(file)["policy"])
decision = apply(probabilities, state, policy)  # probabilities and state from the snippet above
print(decision["action"], decision["needs_review"], decision["reasons"])

Bias, risks and limitations

• The model matches word patterns. It does not understand business rules or verify facts, so the calling application must check preconditions before acting.
• Logistic regression probabilities are not calibrated; pick thresholds from the tables above, and re-evaluate on your own traffic before automating.
• The training data is small and synthetic; treat this model as a demonstration of the pipeline, not as a production model.
• The action vocabulary is fixed; new actions require new labeled data.

Citation

@software{araujo2026opendecision,
  author = {Araujo, Luiz},
  title  = {OpenDecision: a small, auditable next-action model for AI agents},
  year   = {2026},
  url    = {https://github.com/leduardoaraujo/OpenDecision}
}

Model card contact

Open an issue in the OpenDecision repository.