clinical-evidence-gliner2.5-small-lora-v3-act-sol

ClinicalEvidence GLiNER2.5 small adapter — relabelled v3 ACT/Sol

Release repository: na399/clinical-evidence-gliner2.5-small-lora-v3-act-sol.

Intended use and output

The model identifies candidate clinical evidence spans in English text. Its v3 schema has six core entity types and explicit attributes. Shared attributes are three complete, qualified, single-label groups: assertion (affirmed, hypothetical, negated, ruled out, uncertain), experiencer (patient, family, other), and time frame (current, recent, historical, future). The registry also defines family-specific choice and literal-span labels. Use the complete versioned schema in schema_v3.json; do not substitute an open-ended or partial attribute schema. An absent axis in a source row was masked during training; it was not a negative or an implicit default. Defaults are ordinary explicit values when the axis is present.

The six core types and their intended scope are:

The complete canonical descriptions and family-specific field labels are in schema_v3.json.

The threshold 0.7 is the validation-selected core-span threshold for this checkpoint. Shared axes use the official single-label group decoder; the core threshold is not a confidence cutoff for those axes. Bind attribute spans to their core anchors by the returned exact offsets. An anchor-to-field span does not establish which record or database field should receive it.

The release includes a pinned Python usage.py helper, USAGE.md, and requirements.txt. It verifies a bundled 91-file GLiNER source tree before import and pins Python 3.12.13, Torch 2.11.0+cu130, Transformers 4.57.6, PEFT 0.21.2, NumPy 2.5.3, and tokenizers 0.22.2; it does not install a GLiNER wheel. The API is load_model(model_name, *, package_dir=None, adapter_dir=None, base_dir=None, adapter_revision=None, device='cpu'); it returns a LoadedModel with predict(text). The model repository contains PEFT weights and the pinned helper, but not the base model weights. The helper loads the fixed public base reference below and uses the registry-derived v3 schema (registry_profiles.json, registry hash 80f255076cde0237c3f176b45545807bb237cbc57af1d983c6550a6d6816f547).

Authenticate the Hugging Face CLI using its existing login or HF_TOKEN. Set ADAPTER_REVISION to this repository's full immutable 40-character commit SHA, then download the private package before creating the Python environment. The downloaded snapshot contains the helper, runtime manifest, schema, and adapter; base weights are fetched separately from the pinned public reference.

ADAPTER_REVISION='replace-with-full-40-character-private-commit'
hf download na399/clinical-evidence-gliner2.5-small-lora-v3-act-sol \
  --revision "$ADAPTER_REVISION" --local-dir ./clinical-gliner-small
cd ./clinical-gliner-small
uv venv --python 3.12.13 .venv
uv pip install --python .venv/bin/python torch==2.11.0
uv pip install --python .venv/bin/python -r requirements.txt

Select the Torch wheel appropriate for your host; ORCA's qualified environment uses CUDA 13.0. Run a fixed synthetic smoke from the downloaded package directory:

PYTHONDONTWRITEBYTECODE=1 .venv/bin/python usage.py \
  --model small --adapter-revision "$ADAPTER_REVISION" \
  --synthetic --case-id affirmed_condition --device cpu

See USAGE.md for offline local-path loading and asset verification. The core threshold is recorded in release_manifest.json; do not change it at inference.

from usage import load_model

# Use this repository's full immutable 40-character private Hub commit SHA.
model = load_model(
    "small",
    adapter_revision="<private-adapter-commit-sha>",
    device="cpu",
)
predictions = model.predict(text)

The bundled Python helper passed CPU job 17349 on all six fixed synthetic cases in offline mode with this adapter and its pinned base/runtime. This smoke checks loading and returned attributes and offsets; it is not a clinical evaluation or an accuracy qualification. See the CPU smoke report and receipt.

Training data and split policy

Only accepted v3 rows with status relabeled or repaired were eligible. Training kept natural row frequency; there was no source reweighting, upsampling, or downsampling. Patient and lineage splits, exact offsets, source hashes, and repair provenance were preserved.

The 9,600 schema-view rows are derived only from Sol training rows. They are a bounded training addition, not an external corpus. The ACT test split was protected from training. No native silver, Luna v2, public/open-schema corpora, v4 data, or Joint IE examples entered this run. Sol has no independent test split. The accepted Sol repair provenance records frames_dropped of 1,301 for training and 158 for validation; the frozen label audit retained that history, and training did not rewrite labels.

Accepted status counts and exclusions are recorded in the frozen audit:

Thus 7,417 nonaccepted base rows were excluded from train and validation. A further 2,183 ACT test rows were protected from the training export; the original ACT test remained separate for final evaluation. The frozen export records 9,600 exclusion receipts in total, without uploading row-level audit data.

Training configuration

The selected checkpoint and threshold came from ACT/Sol validation only. This is one seed; seed variation is unmeasured. A separate matched encoder comparison evaluates the base and small models at threshold 0.5; those matched comparison results are not the selected small model's threshold-0.7 scores below.

Evaluation

Scores use exact entity type and character offsets. Non-default F1 is end-to-end: missed anchors, wrong attribute values, and extra non-default predictions count against the score. The reported support is gold non-default attribute instances, identified by entity type, exact anchor offsets, axis, and value; several axes can refer to the same span. The threshold was frozen on validation before this final benchmark evaluation; ACT/BPSD benchmark results did not choose the epoch or threshold.

Validation results by source:

Shared-axis per-value F1 is end-to-end on validation, with gold support shown in parentheses. These are not conditional scores restricted to already-detected anchors.

All 20 derived schema views were scored on 7,591 validation rows. Per-view F1 ranged from 0.196 for harm_and_accidents (20 gold spans) to 0.793 for medication_details (2,785 gold spans). These views reuse validation source notes and do not form a separate test set. The aggregate reports retain full view results and masked-axis diagnostics without row-level predictions.

ORCA MIG model-call latency, excluding model loading and full preprocessing:

Optional browser export qualification

The PEFT adapter release does not contain ONNX files. A separate optional FP32 browser export was evaluated; using it requires obtaining its separately pinned graph/runtime package and receipts. This browser test is synthetic software qualification, not clinical evaluation.

CPU export job 17296 and Chromium/WASM job 17300 passed on six fixed synthetic cases. All 19 preprocessing fields per case, exact entity types and codepoint/UTF-16 offsets, declared attribute values, and strict confidences matched the safely merged Python reference. Four natural entity spans carried twelve attribute values; two zero-output cases remained zero-output. The browser encoder forward passed all 2,011,716 legal restored-interval comparisons and 108 means at the unchanged tolerance. A separate raw cumulative-prefix diagnostic had 824 mismatches among 9,216 values; it remains a disclosed auxiliary numerical difference and is not described as a pass.

The three FP32 ONNX graphs total 292,188,325 bytes; they are separate optional artifacts and are not included in this adapter repository. In the measured ORCA CPU/Chromium 153, graph session loading took 3.47 seconds; short synthetic requests had warm medians from 386–412 ms across three repeats, and the 769-token case took 1.874 seconds. These measurements exclude qualification-array comparison and do not predict latency on other hardware. Empty input is rejected by the frontend rather than sent to the graph.

The synthetic browser report SHA-256 is 3f6fdc4dcd1e4f8e0feec926ec33fd9cd4819a99c5dbbb07e3ec0f6c3eb6d440; its job receipt is 30232cf6039b9eacbc8889eb137538bd339166cc5c410e33986234373732ab7f. The independent audit passed all 25 checks (SHA-256 fe0af930bfebec4cb25c78ee319fdd101651900fd7b1d81e81360a7250a3a807). These are synthetic browser receipts, separate from the clinical evaluation artifacts.

Limitations

• The labels are machine-relabeled/repaired v3 annotations, not independent clinical adjudication. Sol is generated synthetic text; it has no independent test split.
• BPSD is a development benchmark and the wider project has prior benchmark use. ACT test results here were not used to select this checkpoint, but ACT is not a prospective or external clinical validation cohort.
• ACT results are weaker than Sol validation. On ACT validation, hypothetical assertion F1 was 0.000 on 23 gold attribute instances; family experiencer F1 was 0.191 on 74 and other was 0.000 on 2; recent time-frame F1 was 0.140 on 571. On Sol validation, hypothetical assertion F1 was 0.000 on 21, family experiencer 0.197 on 146, and other 0.000 on 13. These low or zero scores prohibit treating a good pooled score as reliable across attributes.
• Span and attribute extraction does not establish record-level binding, coding correctness, treatment eligibility, prognosis, fairness, calibration, or improved patient outcomes.
• Evaluated data are English ACT/MIMIC-derived text and generated Sol notes. Other languages, institutions, note styles, and deployment settings are not established.
• This is a research artifact only. Do not use it to make or support decisions about an individual patient.

Frozen release identity and provenance

The release bundle retains the adapter/config, schema_v3.json, registry_profiles.json, per-model threshold in release_manifest.json, private LICENSE, and hash-bound aggregate provenance for training receipt, validation selection, runtime, final delivery report/receipt, ACT/BPSD mapping and evaluation, and the 20 schema-view results. Do not include raw notes, row-level predictions, credentials, or Codex/Hub tokens. The final report declares raw_text_in_report: false.