goby-9-ie-vi

Joint entity + relation extraction for Vietnamese. The best-calibrated model in the goby family: higher precision at the same recall than its siblings. On clean text at each model own best threshold it ties goby-4 rather than beating it (see Measurement correction); on degraded text it is the worst of the family. Pick it for well-formed prose where you care about precision.

Fine-tuned from knowledgator/gliner-relex-large-v1.0. Released by Belumind. Apache-2.0.

Try it in the browser - no install, runs on ZeroGPU.

Which goby do I want?

goby-9 is not robust to degraded text - see Known limitations. That is not a small caveat, read it before choosing.

What changed

Same training data as goby-4. The only difference is the loss configuration: negative sampling ratio 1.0 -> 2.5, focal loss alpha 0.75 -> 0.5. No new data, no augmentation.

This matters because it moves the precision/recall curve rather than sliding along it:

If the two models only differed by an operating point, precision at matched recall would be identical. It is not: +3.9 and +4.1 in the range most people run at.

Measurement correction (2026-09-21)

Every NER number in this card was originally measured with a placeholder relation label (relations=["x"]) - a setting nobody uses in production. Re-measured with the real relation list, same benchmark, one shared harness, every checkpoint scores higher, and the gain grows with how much the model was fine-tuned:

The base model barely moves (+0.9); goby-9 gains +4.6. The relation head learned during fine-tuning feeds back into entity detection, so passing the relations you actually want is worth several F1 on its own - pass them even if you only need NER.

At each model best threshold, goby-9 and goby-4 are INDISTINGUISHABLE: 71.6 vs 71.4, difference +0.22 F1, 95% bootstrap CI [-0.79, +1.29], 68% of resamples positive. Comparisons at one shared threshold favour goby-9; comparisons at each model own operating point do not.

The per-type and relation tables below have not been re-measured and still reflect the placeholder setting.

Results

Benchmark: 450 held-out Vietnamese sentences, 1,328 hand-audited entity spans. All figures at threshold=0.5 unless stated.

Typed NER

Paired bootstrap at a shared threshold, goby-9 minus goby-4: +3.8 F1, 95% CI [+2.7, +4.8], 100% of resamples positive. At each model own best threshold with real relation labels the two are indistinguishable (+0.22, CI [-0.79, +1.29]) - see Measurement correction above.

Threshold sweep:

It also fires far less: 1,672 predictions at 0.5 against goby-4 at 2,005, for higher recall-adjusted precision.

Per entity type (F1)

goby-9 wins on 7 of 10. It loses on tac pham (52.6 -> 33.3), san pham (59.0 -> 51.1) and su kien (40.0 -> 37.5) - the sharper decision boundary costs recall on the types with the fewest training examples.

Relations

Relation quality is essentially unchanged (-1.0 partial, +0.9 strict).

Usage

pip install gliner==0.2.29

from gliner import GLiNER

model = GLiNER.from_pretrained("belumind/goby-9-ie-vi").to("cuda")

KEYS = {
    "to_chuc": "tổ chức",   "van_ban": "văn bản",
    "nguoi": "người",       "dia_diem": "địa điểm",
    "ngay": "ngày",         "chuc_vu": "chức vụ",
}
INV = {v: k for k, v in KEYS.items()}

ents, rels = model.inference(
    texts=["Công ty Cổ phần Giải trí Galaxy được thành lập ngày 13 / 9 / 2003 ."],
    labels=list(KEYS.values()),
    relations=["thành lập ngày", "đặt trụ sở tại"],
    threshold=0.5, relation_threshold=0.5,
    return_relations=True, flat_ner=True,
)
out = [{"text": x["text"], "type": INV[x["label"]]} for x in ents[0]]

Pass the Vietnamese labels with diacritics. GLiNER encodes the label string with its text encoder, so to_chuc is effectively an unseen label: it costs about 25 F1. Map to snake_case keys after inference, as above.

Labels must keep their Vietnamese diacritics

GLiNER encodes the label STRING with the same text encoder as the input, so a de-accented label is a different vector. Measured on goby-7, threshold 0.5:

About 25 F1, lost with no error and no warning. This repo ships goby_labels.py so the mistake fails loudly instead of silently:

import os, sys
from huggingface_hub import hf_hub_download

sys.path.insert(0, os.path.dirname(hf_hub_download("REPO_ID", "goby_labels.py")))
from goby_labels import extract, check_labels, LabelError

ents, rels = extract(model, [text], json_keys=True)

extract() calls the model with the Vietnamese labels and renames the types to to_chuc, van_ban, ... only in the RETURNED dicts - free, and JSON-safe keys. Passing a de-accented label raises LabelError; an unknown ASCII label warns. check_labels(labels) is the bare assertion if you keep your own inference call.

Weights ship as both model.safetensors (what gliner loads by preference) and pytorch_model.bin; the two are byte-identical, tensor for tensor.

Known limitations

• Degraded text destroys this model. Relation F1 on the same 267 sentences: clean 56.7, without diacritics 6.0, ALL CAPS 6.8. goby-7 scores 28.5 and 30.0 on the same inputs. If your text is not clean and correctly cased, use goby-7, not this.
• chuc vu (job titles): do not trust the 31.2 above. On a separate benchmark of 55 hand-annotated, correctly-delimited job titles, goby-9 gets only 5 exactly right (F1 13.3) - and the base model beats every model in this family there (13 exact, F1 18.3). Our fine-tuning data carries a systematic truncation of Vietnamese job titles, and every goby model inherited it. We publish this because the main benchmark hides it.
• tac pham regressed against goby-4 (52.6 -> 33.3).
• ky (signed document) and ke nhiem (successor of) remain the weakest relations.
• Trained on Vietnamese Wikipedia-style prose. Legal, medical and conversational text are out of distribution.

Citation

@misc{goby9ievi2026,
  title  = {goby-9-ie-vi: a recalibrated joint IE model for Vietnamese},
  author = {Belumind},
  year   = {2026},
  url    = {https://huggingface.co/belumind/goby-9-ie-vi}
}