clinical-evidence-gliner2.5-base-lora-v3-act-sol

ClinicalEvidence GLiNER2.5 base adapter — relabelled v3 ACT/Sol

Release repository: na399/clinical-evidence-gliner2.5-base-lora-v3-act-sol.

Intended use and output

The model identifies candidate clinical evidence spans in English text. Its v3 schema has six core entity types and explicit attributes. Shared attributes are three complete, qualified, single-label groups: assertion (affirmed, hypothetical, negated, ruled out, uncertain), experiencer (patient, family, other), and time frame (current, recent, historical, future). The registry also defines family-specific choice and literal-span labels. Use the complete versioned schema in schema_v3.json; do not substitute an open-ended or partial attribute schema. An absent axis in a source row was masked during training; it was not a negative or an implicit default. Defaults are ordinary explicit values when the axis is present.

The six core types and their intended scope are:

The complete canonical descriptions and family-specific field labels are in schema_v3.json.

The threshold 0.6 is the validation-selected core-span threshold for this checkpoint. Shared axes use the official single-label group decoder; the core threshold is not a confidence cutoff for those axes. Bind attribute spans to their core anchors by the returned exact offsets. An anchor-to-field span does not establish which record or database field should receive it.

The release includes a pinned Python usage.py helper, USAGE.md, and requirements.txt. It verifies a bundled 91-file GLiNER source tree before import and pins Python 3.12.13, Torch 2.11.0+cu130, Transformers 4.57.6, PEFT 0.21.2, NumPy 2.5.3, and tokenizers 0.22.2; it does not install a GLiNER wheel. The API is load_model(model_name, *, package_dir=None, adapter_dir=None, base_dir=None, adapter_revision=None, device='cpu'); it returns a LoadedModel with predict(text). The model repository contains PEFT weights and the pinned helper, but not the base model weights. The helper loads the fixed public base reference below and uses the registry-derived v3 schema (registry_profiles.json, registry hash 80f255076cde0237c3f176b45545807bb237cbc57af1d983c6550a6d6816f547).

Authenticate the Hugging Face CLI using its existing login or HF_TOKEN. Set ADAPTER_REVISION to this repository's full immutable 40-character commit SHA, then download the private package before creating the Python environment. The downloaded snapshot contains the helper, runtime manifest, schema, and adapter; base weights are fetched separately from the pinned public reference.

ADAPTER_REVISION='replace-with-full-40-character-private-commit'
hf download na399/clinical-evidence-gliner2.5-base-lora-v3-act-sol \
  --revision "$ADAPTER_REVISION" --local-dir ./clinical-gliner-base
cd ./clinical-gliner-base
uv venv --python 3.12.13 .venv
uv pip install --python .venv/bin/python torch==2.11.0
uv pip install --python .venv/bin/python -r requirements.txt

Select the Torch wheel appropriate for your host; ORCA's qualified environment uses CUDA 13.0. Run a fixed synthetic smoke from the downloaded package directory:

PYTHONDONTWRITEBYTECODE=1 .venv/bin/python usage.py \
  --model base --adapter-revision "$ADAPTER_REVISION" \
  --synthetic --case-id affirmed_condition --device cpu

See USAGE.md for offline local-path loading and asset verification. The core threshold is recorded in release_manifest.json; do not change it at inference.

from usage import load_model

# Use this repository's full immutable 40-character private Hub commit SHA.
model = load_model(
    "base",
    adapter_revision="<private-adapter-commit-sha>",
    device="cpu",
)
predictions = model.predict(text)

The bundled Python helper passed CPU job 17349 on all six fixed synthetic cases in offline mode with this adapter and its pinned base/runtime. This smoke checks loading and returned attributes and offsets; it is not a clinical evaluation or an accuracy qualification. See the CPU smoke report and receipt.

Training data and split policy

Only accepted v3 rows with status relabeled or repaired were eligible. Training kept natural row frequency; there was no source reweighting, upsampling, or downsampling. Patient and lineage splits, exact offsets, source hashes, and repair provenance were preserved.

The 9,600 schema-view rows are derived only from Sol training rows. They are a bounded training addition, not an external corpus. The ACT test split was protected from training. No native silver, Luna v2, public/open-schema corpora, v4 data, or Joint IE examples entered this run. Sol has no independent test split. The accepted Sol repair provenance records frames_dropped of 1,301 for training and 158 for validation; the frozen label audit retained that history, and training did not rewrite labels.

Accepted status counts and exclusions are recorded in the frozen audit:

Thus 7,417 nonaccepted base rows were excluded from train and validation. A further 2,183 ACT test rows were protected from the training export; the original ACT test remained separate for final evaluation. The frozen export records 9,600 exclusion receipts in total, without uploading row-level audit data.

Training configuration

The checkpoint and threshold were selected using the frozen ACT/Sol validation protocol. The selected seed-43 run scored 65.96% non-default F1 at threshold 0.5 in the matched run comparison; this is separate from its final threshold-0.6 score below. One seed-42 and one seed-43 run at the same learning rate do not establish seed robustness or an uncertainty interval.

Evaluation

Scores use exact entity type and character offsets. Non-default F1 is end-to-end: missed anchors, wrong attribute values, and extra non-default predictions count against the score. The reported support is gold non-default attribute instances, identified by entity type, exact anchor offsets, axis, and value; several axes can refer to the same span. The threshold was frozen on validation before this final benchmark evaluation; ACT/BPSD benchmark results did not choose the epoch or threshold.

Validation results by source show an important difference between the human-derived and synthetic data:

Shared-axis per-value F1 is end-to-end on validation, with gold support shown in parentheses. These are not conditional scores restricted to already-detected anchors.

All 20 derived schema views were scored on 7,591 validation rows. Per-view F1 ranged from 0.353 for harm_and_accidents (20 gold spans) to 0.844 for medication_details (2,785 gold spans). These views reuse validation source notes and do not form a separate test set. The aggregate reports retain full view results and masked-axis diagnostics without row-level predictions.

ORCA MIG model-call latency, excluding model loading and full preprocessing:

Limitations

• The labels are machine-relabeled/repaired v3 annotations, not independent clinical adjudication. Sol is generated synthetic text; it has no independent test split.
• BPSD is a development benchmark and the wider project has prior benchmark use. ACT test results here were not used to select this checkpoint, but ACT is not a prospective or external clinical validation cohort.
• ACT performance is materially weaker than Sol validation: ACT test precision is 32.35%. Unmatched-anchor false positives may reflect both model overprediction and incomplete gold labels; they are not adjudications of clinical truth.
• Low-support values remain unreliable: ACT hypothetical assertion had F1 0.000 on 23 gold attribute instances and other experiencer had F1 0.000 on 2; ACT recent time-frame F1 was 0.254 on 571. Results on generated Sol text are substantially stronger and should not be transferred to clinical text.
• Span and attribute extraction does not establish record-level binding, coding correctness, treatment eligibility, prognosis, fairness, calibration, or improved patient outcomes.
• Evaluated data are English ACT/MIMIC-derived text and generated Sol notes. Other languages, institutions, note styles, and deployment settings are not established.
• This is a research artifact only. Do not use it to make or support decisions about an individual patient.

Frozen release identity and provenance

The release bundle retains the adapter/config, schema_v3.json, registry_profiles.json, per-model threshold in release_manifest.json, private LICENSE, and hash-bound aggregate provenance for training receipt, validation selection, runtime, final delivery report/receipt, ACT/BPSD mapping and evaluation, and the 20 schema-view results. Do not include raw notes, row-level predictions, credentials, or Codex/Hub tokens. The final report declares raw_text_in_report: false.