Skip to page content
SAVRN
Search Plan a project

Open-weight model · Text classification

Solomon

by Doccy Health DoccyHealth/Solomon

Solomon is an open-weight model for text classification from Doccy Health, released under Apache License 2.0. Its published files total 873.5 MB.

A LoRA adapter and trained answer heads for Qwen/Qwen3.8-27B that turn a document plus a set of structured questions into one probability per decision, each with a retrieval pointer to where the support for it plausibly sits in the source.

Parameters—
Context—
Weights872.4 MB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads—

Model Card

By Doccy Health, published under apache-2.0, revision 7f592ed277d5.

A LoRA adapter and trained answer heads for Qwen/Qwen3.8-27B that turn a document plus a set of structured questions into one probability per decision, each with a retrieval pointer to where the support for it plausibly sits in the source. It does not generate text. Give it a document once and ask structured questions against it. Each answer comes back as a probability, and with source spans beside it. The answer is read from the whole document; the spans are a separate word-overlap search over the source text. They are a pointer to where the support probably is, not the passage the answer came from — Evidence spans, below, measures the difference. There is no chat, no reasoning trace and…

Read Doccy Health's full model card

A LoRA adapter and trained answer heads for Qwen/Qwen3.8-27B that turn a document plus a set of structured questions into one probability per decision, each with a retrieval pointer to where the support for it plausibly sits in the source. It does not generate text.

Read this before you trust a number on this page

Every document accuracy and calibration figure here was measured on documents this project generated synthetically. The only evaluation against real documents scored 0.8131 whole-question agreement, not the ≈99% the synthetic figures show, and the loss is concentrated in two of the five answer types: entity 0.4359 and multi-label 0.7000.

The probability calibration does not transfer out of domain. On general-knowledge benchmarks this model is more overconfident than the base model it is built on.

Both are set out in full under Limitations, below. Nothing on this page is a certified error rate and nothing is guaranteed.

What it is, and what it is for

Give it a document once and ask structured questions against it. Each answer comes back as a probability, and with source spans beside it. The answer is read from the whole document; the spans are a separate word-overlap search over the source text. They are a pointer to where the support probably is, not the passage the answer came from — Evidence spans, below, measures the difference. There is no chat, no reasoning trace and no sampling: every answer is read from letter logits at a fixed position through trained linear heads, so the same document and the same question return the same numbers every time.

Five answer types:

Type Question shape What comes back
Yes / no does the document establish X? one probability
Single choice which of these does it state? one probability over the listed options
Ordered choice which threshold does it state? one probability over the ordered options
Entity which of these parties is the X? one probability per candidate
Multi-label which of these apply? one probability per candidate

It is for turning documents into structured, machine-readable answers where you need a number attached to each one, and where determinism and a refusal to drift matter more than fluency.

It is not for general knowledge question-answering, chat, generation, or summarisation. It is not for any setting where the cost of a confident wrong "No" on something the document never mentions is high — that is this model's worst measured failure and it is documented below.

How it works

Base model Qwen/Qwen3.8-27B, Apache-2.0, pinned revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. Not redistributed here
Adaptation LoRA, rank 64, question-side placement, float32
Answer projection trained linear heads (adapter/heads.npz), not the language-model head
Readout structured letter logits; yes/no-shaped units collapse to a binary log-odds before temperature
Calibration one positive scalar per answer type, applied to the logits before the softmax
Runtime identity a 15-key binding that refuses to load if the engine is not the one measured

The document is prefilled once; the questions branch off it. The document goes through the model a single time and becomes a reusable state (POST /states). Every question is then answered as an isolated branch off that prefix. Two consequences are worth stating because they are tested on real hardware and not merely intended: asking the same questions in a different order returns the same answers, and answering from the cached document state matches a full forward pass to within 1e-3.

The adapter is off while the document is read, and on from the question onward. That is what question-side placement means. Applying it across the whole sequence gives a different model to the one that was measured, and the identity binding exists partly to stop that happening by accident.

The readout is structured, not generated. Rather than sampling an answer and parsing it, the model is asked to commit at a fixed position and the letter logits at that position are read through trained heads (readout mode four_collapsed). Yes/no-shaped units — a yes/no question, and each individual candidate inside an entity or multi-label answer — are collapsed to a single binary log-odds, p = sigmoid(z / T) with z = log P(yes)/P(no), before the temperature is applied. Choice questions apply the temperature to the listed slice, softmax(logits[:n] / T). The ruled temperatures are boolean 1.9476 · single 1.0 · ordered 1.5063 · multilabel 1.6895 · entity 2.1834. Single choice ships at 1.0 deliberately: at its accuracy there is essentially nothing to calibrate and a fitted value there would be noise.

A branch whose head_key is not in the calibration artifact's map is refused, not served at an assumed 1.0.

The temperatures live inside the runtime binding, covered by its checksum. A calibration fitted on one model may not be served on another: the loader refuses by name rather than serving scalars that mean nothing. Serving at temperature 1.0 everywhere is always permitted, on any model.

What the returned score means

ordering_score means two different things depending on the question, and the difference matters.

  • Single-unit — a yes/no question, a single or ordered choice, and every per-candidate value inside an entity or multi-label answer. Here the score is the tempered readout probability, and the held-out synthetic panel below validates it as a calibrated magnitude.
  • Multi-unit — the rolled-up question-level score for an entity or multi-label question with more than one candidate. It is the product of the per-candidate probabilities, which assumes those candidates are independent. That assumption has never been validated as a joint probability. It orders such questions well; it is a heuristic ordering, not a calibrated joint. If you need a magnitude for one of these, read the per-candidate numbers.

The field is not called probability because that would be accurate for the first case and an overclaim for the second.

There is no abstention. Every question is answered. The service will not emit a field named abstain, confidence, threshold or certified_error_rate; it raises rather than return one. If you want to decline low-confidence answers, that is your policy, made on your population, and this release makes no claim about where to put the line.

How to run it

The package is a library, not a daemon: you build the engine, wrap it in the serving layer and start the HTTP surface in four lines. It needs one CUDA device with enough memory for the base model in float32.

pip install -r requirements.lock

huggingface-cli download DoccyHealth/Solomon --local-dir ./solomon
huggingface-cli download Qwen/Qwen3.8-27B --revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 --local-dir ./solomon/base

cd solomon
export PYTHONPATH=$PWD/src

Run from inside that directory: the engine looks for the base model in base/, and the serving layer reads serving/selection.json and the binding it names, both relative to the package root.

from solomon import api, service
from solomon.serving import ServiceEngine

engine = ServiceEngine('adapter/adapter.safetensors', 'adapter/heads.npz')
layer = service.service('./store', engine)   # './store' holds cached document states
server = api.serve(layer)                    # GET /health, POST /states, POST /v1/decide
print('http://127.0.0.1:%d' % server.server_port)
server.serve_forever()                       # or skip the server and call layer.decide(...) directly

ServiceEngine also takes expected_adapter= and expected_heads=, the two sha256 values printed below; pass them and the engine refuses to start on a file that is not the one this card describes.

GET /health reports the contract, the readout mode and the runtime identity. If the engine you built differs from the one the numbers were measured on — a different torch build, a different adapter, a different arithmetic mode — the binding refuses to load rather than quietly serving different numbers. That is intended behaviour; do not work around it.

What is in this repository

README.md                            this file
LICENSE                              Apache-2.0
NOTICE                               attribution and the Apache-2.0 4(b)/4(c) notices
MODIFICATIONS.md                     statement of changes, third-party text scan, maintainer notes
MANIFEST.json                        every file, its size and its sha256
requirements.lock                    the pins of the image this was qualified on, and dependency licences
licenses/Qwen-Apache-2.0.txt         the upstream licence, verbatim
adapter/adapter.safetensors          the LoRA adapter
adapter/heads.npz                    the trained answer heads
adapter/config.json                  base repo, pinned revision, checksums, placement
serving/serving-binding.json         runtime identity binding and the frozen temperatures
serving/readout-temperature-v2.json  the calibration artifact, standalone
serving/selection.json               readout mode and model identity
serving/service-export.json          qualification envelope hashes
src/solomon/__init__.py              solomon: a document plus structured questions in, one probability per decision out
src/solomon/api.py                   HTTP surface for the Solomon layer (solomon/service.py)
src/solomon/binding.py               serving identity: which readout is served, and proof that it is the one that was measured
src/solomon/calibration.py           the readout calibration: one positive scalar per answer type, and nothing else
src/solomon/engine.py                question-only CUDA runtime for immutable semantic-head checkpoints
src/solomon/engine_contract.py       the answer contract engine: contract v3 on the reference engine's float32 cached path
src/solomon/engine_cuda.py           CUDA contract-v3 engine
src/solomon/engine_numerics.py       versioned CUDA repair: bounded FP32 attention, FP64 recurrent accumulation
src/solomon/engine_reasoning.py      the confidence layer text reasoning on immutable answer-contract base-prefix states
src/solomon/engine_reference.py      the reference engine: task-agnostic document prefix, float32 arithmetic, chunked prefill
src/solomon/evidence.py              deterministic source references and explicit evidence interventions
src/solomon/evidence_packages.py     evidence packages (v3): selected spans plus source-derived governing context, per question unit
src/solomon/heads.py                 final normalized feature extraction, preserving the qualified CUDA engine
src/solomon/prompts_two_letter.py    two-letter (Noul) prompts and block conversions
src/solomon/readout.py               contract v3 readouts: branch jobs for every answer type, predictions from letter logits, and metrics
src/solomon/reliability.py           what the readout says about its own answer, with nothing fitted behind it
src/solomon/retrieval.py             inference-only source candidates and explicitly labelled retrieval baselines
src/solomon/routing.py               real callback-driven escalation
src/solomon/semantics.py             answer semantics (design note, not distributed)
src/solomon/service.py               decision layer: the Solomon serving contract over the pinned readout chain
src/solomon/service_answers.py       confidence-aware five-task service, reusing immutable answer-contract input persistence
src/solomon/service_checked.py       backend-neutral contract-v3 service; restart replays immutable inputs, not tensors
src/solomon/service_evidence.py      optional source-grounded evidence around the existing confidence service
src/solomon/service_heads.py         trained-head fast/views service; unchanged decoder and separate stage confidence
src/solomon/service_packages.py      evidence packages (v3) in the answer service: per-unit, source-grounded, page-referenced
src/solomon/service_states.py        local contract-v3 prototype: task-neutral text/image states and five answer types
src/solomon/serving.py               preserve five-task trained-head confidence/routing with optional evidence
mlx/                                 the optional Apple-silicon package: its own library, tests and notices

What is deliberately not here: no training data, no evaluation panels, no document corpora, no datasets, no cached states, no rendered images, no score archives, no test fixtures, no logs and no base model weights. The package is default-deny: a manifest names every permitted file and the build fails if anything else is present.

The files under serving/ carry identity only: hashes, the readout mode, the served design and the five temperatures. The loader verifies the binding's checksum and every runtime key it carries. The measured provenance behind those hashes (which panels, which fit, which qualification run) is held in the maintainer's records and is not distributed.

Evidence spans

Every answer carries source spans, and evidence is on by default (evidence='support'). This section is what that is worth, measured rather than described. The short version: the spans are exact, cheap and not the model's reasoning. The answer is read from the whole document. The spans are a word-overlap search over the source text, run on the CPU, that the model never sees at the default level. They are worth having as a place to start reading. They are not a justification, and this page does not present them as one.

A span is an exact slice of your document. start, end and text, counted in Python code points, re-validated against the source before they are returned — a span that no longer matches the document is raised as an error, never returned as a result. Nothing is paraphrased, summarised or generated. Entity and multi-label answers additionally carry candidate_evidence, one span list per candidate — but with the shipped selector those lists are identical for every candidate, which is measured below and is a trap worth knowing about before you build on the field.

At the default level the model never reads the spans. The shipped selector (solomon.retrieval.lexical_select) ranks the document's sentences by plain word overlap with the question and keeps the top four per answer unit. No model call is involved, and the cost proves it: in the run below, evidence='support' and evidence='none' produced identical usage — same branches, same input tokens, zero evidence calls — and finished 4.08 s against 4.04 s. Whatever the spans are, they had no part in producing the answer.

The service says this about itself rather than leaving it to the documentation: ask with evidence_detail=True and the response carries method: "lexical_overlap", verification: "retrieval_only" and faithfulness_established: false. The model is engaged only at 'sufficiency' and above, where the selected text is re-prefilled as a fresh document and the answer re-derived from it. Pass selector= to service.service(...) to supply a better one.

The four levels, and what each costs

An answer unit is whatever gets one probability: one for a yes/no or a choice question, one per candidate for entity and multi-label.

evidence= Extra forward passes What comes back
'none' none evidence_status: "not_requested" and no spans
'support' (the default) none — retrieval only, the model is not called the spans
'sufficiency' one fresh document prefill per answer unit the spans, plus the answer re-read from that evidence alone
'removal' one per answer unit plus one per question the above, plus the answer re-read from the document with every selected span cut out

sufficiency rebuilds each unit's spans, with automatically attached governing context, into a small standalone document capped at 30% of the source, and answers that unit again from it and nothing else. removal adds one further pass per question over the document with the spans deleted. Both re-encode from scratch: a cached state cannot demonstrate removal.

Two runs measured this on hardware. The live acceptance run, on 6 questions carrying 10 answer units, cost 0, 10 and 16 fresh document prefills for the three levels. The real-document run below, on 6 questions carrying 14 units, is where the wall-clock difference shows:

'none' 'support' 'sufficiency'
Fresh document prefills 0 0 14
usage.evidence_calls 0 0 14
Answer branches / input tokens 14 / 2,085 14 / 2,085 14 / 2,085
Extra tokens spent on evidence 0 0 14,918
Wall clock 4.04 s 4.08 s 21.03 s

'support' is free. 'sufficiency' cost about 5x the wall clock on that document. 'removal' has never been timed — the acceptance run counted its prefills, nothing has measured its tokens or its latency, and this page will not guess.

evidence_max_calls (default 64) is checked before any call is made. Over budget, the level is skipped: evidence_status comes back "budget_exhausted" and the answer is unaffected. evidence_detail=True adds the package record — every span with its role and the reason it was attached, the fraction of the source covered, the selector's identity — and costs nothing extra. The response's usage.evidence_calls reports what was actually spent.

Evidence is text only. A state containing images comes back with evidence_status: "unsupported_page_selector" unless you supply a page_selector= of your own, and no image claim is made anywhere in this release.

What comes back

layer.decide(state_id=key, evidence='removal',
             questions={'certified': 'Does Rookwood Ltd hold a current certification?'})
{
  "answers": {
    "certified": {
      "type": "noul",
      "noul": 0.9913,                       // the probability
      "ordering_score": 0.9913,
      "temperature": 1.9476,
      "evidence_status": "found",           // or no_support_found / budget_exhausted / not_requested
      "evidence": [
        {"start": 1840, "end": 1906,
         "text": "Rookwood Ltd holds certification RW-4471, current to 30 June 2027."}
      ],
      "evidence_verification": {
        "evidence_only":    {"noul": 0.9902, "agrees_with_full": true},
        "evidence_removed": {"noul": 0.2140, "agrees_with_full": false}
      }
    }
  },
  "usage": {"branches": 1, "input_tokens": 4096, "document_prefill_tokens": 0, "evidence_calls": 2}
}

agrees_with_full compares decisions, not probabilities: the yes/no side of 0.5 for each unit, or which option is on top for a choice. The probabilities beside it are the re-read ones and are shown so you can see how far they moved.

What the live acceptance run establishes

20 of the 57 live checks on this exact binding are evidence checks, and all of them passed on the hardware in the release record.

  • The spans are real source ranges. 12 checks re-sliced every returned span out of the document it came from and required an exact match. All returned found, with 4–5 spans per question.
  • Asking for evidence cannot change an answer. At each of the three levels the whole answer object, with every evidence field stripped out, was required to be identical to the answer returned without it — same probabilities, same ordering score, same temperature. The real-document run below reproduces this to the last bit. This is the one property here worth relying on: evidence is an addition to the response and never an input to it. The serving code carries the same statement as a field, answer_qualification_unchanged: true.
  • Running out of budget is not an answer. With the budget set below what the level needs, every question came back budget_exhausted with its answer unchanged rather than degraded.

What that run is, in its own words: live mechanics on padded fixture documents; not a population sample and not an accuracy, calibration or evidence-faithfulness claim. It is a mechanics test on 2 padded fixture documents, and it says nothing about whether the spans it returned are the right spans.

What one real document showed

The evidence path has been run against a real document exactly once: 6 questions, 14 answer units, on a public privacy policy, on a B200, through the shipped adapter, the shipped binding and the shipped default selector. It is the only evidence run of any kind on the model in this repository. In the report's own words: DEMONSTRATION ONLY on one real out-of-support document. Not a population sample, not an accuracy, calibration or evidence-faithfulness claim, and no qualification artifact is written.

Read it off-support. That document is 4,440 prefix tokens against a binding whose measured prefix support is 1,242–2,327 tokens. The report records within_measured_support: false. Nothing below is a quality number, and none of it should be mixed with the synthetic figures in the next section.

What held up.

  • The spans are exact. All 29 returned spans are consistent slices of the source, and the service validates every one against the document before returning it — a span that no longer matches raises rather than returns.
  • Evidence never touched the answer. Across 'none', 'support' and 'sufficiency' every probability for all 14 units was bit-identical, not merely close. The same property the acceptance run checks, on a real document.
  • Every evidence-only re-ask agreed with the full-document decision, on all 14 units.

What did not.

  • Span precision was poor. The selector returns its top four overlapping sentences per unit whether or not they are relevant. The clearest case: for a question about whether the policy commits to a complaint-response deadline, the returned evidence included the site's footer navigation block (Services / Medical Certificates / Prescriptions …). Reading all 29 spans by hand, roughly 8 were on point. That is a hand count on one document, with no scoring artifact behind it, and it is the only precision figure that exists for the shipped selector.
  • candidate_evidence gave no per-candidate discrimination at all. On both multi-candidate questions — 10 candidates in total — every candidate received the identical span set. The shipped selector reports no per-unit attribution, so the service hands each unit the same union. The field's shape invites you to read it as "the evidence for this candidate"; with the default selector it is not that, and a caller who trusts it will be misled.
  • Agreement is near-vacuous on a negative answer. One question asked for a commitment the policy does not make. The model answered correctly — P(yes) = 0.00288 — but the selector never surfaced the relevant part of the document, and the sufficiency check still reported agrees_with_full: true on evidence that was about something else. Agreeing that something is absent is not the same as showing it is absent, and nothing in the response distinguishes the two. Treat agrees_with_full on a negative answer as carrying almost no information.
  • Decisions were stable; magnitudes were not. Every unit agreed, but the probabilities moved when re-read from the evidence alone — the largest by 0.062, an entity candidate going from 0.914 to 0.975. agrees_with_full compares decisions, so it reports agreement across moves of that size without flagging them.

What has been measured on synthetic documents

One registered run measures evidence quality. It is not a run on the model in this repository.

The metric is evidence-only agreement: the fraction of questions whose answer, read from that question's evidence package alone, equals the answer read from the whole document.

Evidence-only agreement 0.960 (288/300), 95% interval 0.937–0.980
Control: same documents with the spans deleted 0.153 (46/300)
Span precision / recall 0.950 / 0.982 over 964 answer units
Correctly finding no support where there is none 0.998 over 403 units
Population 60 documents, 300 questions, text only
Answer type Agreement Questions
Yes / no 0.967 58/60
Single choice 0.967 58/60
Ordered choice 1.000 60/60
Multi-label 0.967 58/60
Entity 0.900 54/60

The control line is the one that carries information: deleting the selected spans drops agreement from 0.960 to 0.153, so the spans are not incidental. Entity is the weak cell at 0.900, missing the gate this project set for it — the same answer type that collapses on real documents under Limitations.

Three qualifiers, and they are large.

  1. The documents are synthetic, generated by this project's own generators, exactly like every other in-domain panel on this page.
  2. The model measured was not this one. That run used an earlier adapter from a previous training round (ea869eace19a…), not the adapter shipped here (2addaf841ecc…). The packager compares those two hashes and refuses to build if they are ever equal, so this paragraph cannot quietly become false.
  3. The selector measured is not the one shipped. That run used a fitted relevance selector with its own trained weights. Those weights are not in this repository and are not released. What you get here is the lexical-overlap default described above, which is a weaker selector.

What is not established

  • No scored evidence measurement exists for the adapter in this repository. Every figure in the agreement table belongs to an earlier model and a selector that is not shipped. The only run on this adapter is the single real document above, which has no reference labels and scores nothing.
  • No registered measurement exists for the shipped default selector. The only run that used the shipped lexical top4/unit; question/source only policy is a development diagnostic over 30 development sources and 150 questions. It reports no aggregate, and its own record states: "Development oracle diagnostic; exact gold anchors may omit provenance context. Agreement is not faithfulness. Lexical retrieval is a baseline, not a trained relevance head." Nothing is rolled up from it here, and no number from it appears above.
  • Faithfulness is not established, and is not claimed. Agreement means the answer survives on the evidence alone; it does not mean the passage caused the answer, and it does not mean no other passage would have done. Every response says so in a field.
  • No scored evidence figure on real documents exists at all. One document, read by hand, off the binding's measured support, is what there is. Given what the real-document panel does to entity and multi-label accuracy under Limitations, assume evidence quality degrades there too, and do not assume the size of the drop is known.
  • Agreement is not accuracy. A wrong answer that the evidence reproduces faithfully agrees with itself. And on a negative answer, agreement barely discriminates at all — see above.
  • candidate_evidence is not per-candidate with the shipped selector. It is the same union of spans repeated once per candidate.
  • 'removal' has never been timed or costed, only counted.
  • Span precision is unmeasured on real documents and, for the shipped selector, unmeasured anywhere: the only figure is a hand count on one document.
  • The per-answer-type gates were not all met even on the synthetic panel that was run.

If a decision matters, treat these spans as a place to start reading, verify them yourself, and do not present them to anyone as the model's reasons. They are not.

What it scores

On documents, held out, and synthetic

1,500 generated documents, none of them seen by the temperature fit. The calibration this repository ships is applied.

Answer type Units scored Accuracy ECE Errors stated ≥0.99 Share of mass ≥0.99
Yes / no 1,500 questions 98.600% 0.0079 14.3% 73.6%
Single choice 1,500 questions 99.867% 0.0004 0.0% 99.1%
Ordered choice 1,500 questions 98.600% 0.0012 4.8% 85.2%
Multi-label (per candidate) 7,634 candidates 99.528% 0.0026 13.9% 95.8%
Entity (per candidate) 12,000 candidates 99.217% 0.0073 3.2% 67.9%

Ordered choice is the best-calibrated cell here. Yes/no is the weakest and stays weak: it still states 14.3% of its errors above 0.99, so a yes/no answer above 0.99 is worth less than an entity candidate above 0.99 and the two should not be treated as interchangeable.

These documents are synthetic. They were produced by this project's generators. That is the single most important qualifier on this table and the next section is about what happens without it.

On public benchmarks, with no document at all

These are out-of-domain by construction: Solomon reads documents, and every item here is a bare multiple-choice question. They are included because they are the only comparison a reader can check against a model they already know — the unmodified base model, item for item, on the same prompts.

  • Solomon — the shipped adapter and heads, structured letter readout over the answer options (head_key single/choiceS), production cached path, fp32.
  • Base model, standard prompt — Qwen3.8-27B with no adapter, a standard zero-shot multiple-choice chat prompt, letter logits from the unmodified lm_head.
  • Base model, same readout — Qwen3.8-27B with no adapter, read through the same structured prompt shape as Solomon, letter logits from the unmodified lm_head. This is the like-for-like control.

Accuracy, with a 95% bootstrap interval

Panel Solomon Base model, standard prompt Base model, same readout
ARC-Challenge (n = 600) 97.2% [95.8%–98.3%] 97.0% [95.5%–98.3%] 97.2% [95.7%–98.5%]
MMLU (n = 2,400) 82.4% [80.9%–83.9%] 82.5% [81.0%–84.0%] 81.7% [80.3%–83.3%]
MMLU-Pro (n = 800) 60.5% [57.1%–63.7%] 62.3% [59.0%–65.5%] 61.8% [58.4%–65.0%]
Fresh math (n = 190) 55.8% [48.9%–63.2%] 52.1% [45.3%–58.9%] 43.7% [36.3%–50.5%]
Fresh math, multi-step only (n = 170) 51.2% [43.5%–58.2%] 46.5% [38.8%–54.1%] 37.1% [30.0%–44.1%]

Expected calibration error — lower is better

Panel Solomon Base model, standard prompt Base model, same readout
ARC-Challenge (n = 600) 0.022 0.007 0.028
MMLU (n = 2,400) 0.117 0.057 0.027
MMLU-Pro (n = 800) 0.190 0.063 0.059
Fresh math (n = 190) 0.063 0.063 0.050
Fresh math, multi-step only (n = 170) 0.076 0.070 0.056

Errors stated at 0.99 or above — count, and share of that arm's errors

Panel Solomon Base model, standard prompt Base model, same readout
ARC-Challenge (n = 600) 4 (23.5% of 17) 1 (5.6% of 18) 1 (5.9% of 17)
MMLU (n = 2,400) 76 (18.0% of 423) 30 (7.1% of 421) 10 (2.3% of 439)
MMLU-Pro (n = 800) 21 (6.6% of 316) 9 (3.0% of 302) 4 (1.3% of 306)
Fresh math (n = 190) 0 (0.0% of 84) 0 (0.0% of 91) 0 (0.0% of 107)
Fresh math, multi-step only (n = 170) 0 (0.0% of 83) 0 (0.0% of 91) 0 (0.0% of 107)

AUROC — how well the stated probability separates right answers from wrong ones

Panel Solomon Base model, standard prompt Base model, same readout
ARC-Challenge (n = 600) 0.946 0.966 0.887
MMLU (n = 2,400) 0.854 0.872 0.853
MMLU-Pro (n = 800) 0.847 0.850 0.844
Fresh math (n = 190) 0.845 0.764 0.782
Fresh math, multi-step only (n = 170) 0.818 0.704 0.716

On ARC-Challenge and MMLU, accuracy is essentially the base model's: every interval above overlaps. On the two fresh-math panels — items generated for this comparison and present in no public corpus — Solomon is clearly ahead of the base model read the same way (55.8% against 43.7%, and 51.2% against 37.1%), and ahead of it under a standard prompt as well.

The calibration does not survive the trip. On MMLU, Solomon's expected calibration error is 0.117 against 0.027 for the same base model read the same way, and 0.057 for the base model under a standard prompt. Solomon states 76 of its 423 errors at 0.99 or above; the base model on the same readout states 10 of 439. MMLU-Pro is worse again — 0.190 against 0.059, on lower accuracy (60.5% against 61.8%).

Read plainly: this adapter makes the model more confident, and out of its own domain that confidence is not earned. Solomon is MORE overconfident on general-knowledge questions than the model it is built on. The per-answer-type temperatures were fitted on structured document question-answering and they do not transfer; nothing here entitles a reader to carry the in-domain calibration results over to a general-knowledge population.

The ranking signal degrades more gently than the magnitude does: AUROC stays close to the base model's on every panel, and beats it on both fresh-math panels. An out-of-domain score is worth more as an ordering than as a probability — and in domain the same caution applies to the multi-candidate roll-up, below.

How to read these numbers

  • NO TEMPERATURE IS APPLIED in these numbers: the scoring harness takes a plain softmax over the raw letter logits. The shipped serving path would not serve these items calibrated either -- the readout used here is head_key single/choiceS, which is not in the calibration artifact's map, and the loader REFUSES an unmapped head_key rather than assuming 1.0.
  • These panels have no document. Solomon is a document question-answering model and every one of these items is a bare multiple-choice question, so this is out-of-domain by construction.
  • The fresh-math panels use 10-option items, an option count outside the range the readout was developed on.
  • The fresh-math items were generated for this comparison and are in no public corpus. Only top probability and correctness were retained for some arms there, so no Brier score is computed on those panels.
  • 95% CIs are 2,000-resample item bootstraps.

Limitations

The synthetic-to-real gap

Everything above about documents is measured on generated documents. One evaluation has ever been run against real documents, on a panel of 40 real documents and 198 questions.

Whole-question agreement Yes/no Single Ordered Multi-label Entity
This model 0.8131 0.9750 1.0000 0.9500 0.7000 0.4359
A previous training round, same panel 0.8232 0.9750 1.0000 0.9750 0.7000 0.4615
Questions 198 40 39 40 40 39

Set that against ≈99% on the synthetic panel. The gap is real, it is large, and it is concentrated in entity and multi-label. Slot accuracy on the earlier round was 0.8529; the earlier round's whole-question agreement carries a 95% interval of 0.7789–0.8629.

The shipped model was measured on this panel and did not improve on it. The difference against the previous round is -1.0 points, 95% interval [-3.0, +1.0], McNemar p = 0.62 on 4 discordant questions. The two rounds are indistinguishable here. A round of training aimed specifically at this failure did not move it.

The dominant error. The model answers "No" where the adjudicated reference is "not established", for candidates the document never mentions. Of the 110 entity slots whose reference is "not established", this model answers "No" on 57; the earlier round answered "No" on 56 of the same 110. The earlier round's analysis recorded it this way — B is the "No" letter and C the "not established" letter:

answers B (No) where adjudicated gold is C (not established) for entities/labels absent from the document: 60 of 94 wrong entity/multilabel slots; 25 of 33 wrong entity/multilabel questions are only this confusion

These are not marginal errors. The project's analysis of this panel records that the model stays confident on exactly these slots, and that a round of training aimed at the failure left both the count and the confidence essentially unchanged. If your questions ask about things that may simply be absent from the document, this is the behaviour that will hurt you, and no released version of this model handles it well.

Analysis of why the synthetic population does not transfer, recorded before the shipped model was trained: synthetic documents state their own reading conventions and real ones do not; the synthetic entity questions are about permissions while the real ones are about exclusive roles; and the conventions for "mentioned in another role" differ between the two label sets.

What the real-document panel is, and is not

  • The reference labels are adjudicated agent labels — two blind passes plus an adjudicator — not human ground truth. The artifact records reviewer: "agent" and human_verified: false. No human has verified this panel.
  • The two blind passes did not agree with each other on the convention either. The artifact records:

the two blind labelling passes disagreed on the same convention (entity agreement 0.786, multilabel 0.833 question-level)

So part of the measured gap may be a labelling-convention disagreement rather than a model error:

24 of the 60 slots are in document-referential questions ("named in the letter as ...") where No is defensible; 36 are substantive questions where the model contradicts its own prompt rule "A missing fact is not a negative fact"

  • The shipped model's scores on this panel were produced by the project's paired evaluation harness on the frozen design, not through the serving API in this repository. Temperature is a monotone rescaling and cannot change which answer is on top, so accuracy carries; calibration on real documents has not been measured at all.
  • 40 documents and 198 questions is a small panel. Treat the per-type numbers as indicative, not precise.

The rest

  • Out of domain, the probability magnitude is not trustworthy. See the benchmark section: on general knowledge this model is more overconfident than the base model it is built on.
  • The multi-candidate roll-up is an ordering, not a joint probability. Read per-candidate values if you need a magnitude.
  • Nothing here is a certified error rate. No certification was run for this release and no threshold is enforced anywhere on the serving path.
  • A confirmation run on the held-out synthetic panel did not happen — its compute went to training. The synthetic claim rests on the held-out panel above plus repeated paired panels.
  • No image modality claim. Text only.
  • Yes/no above 0.99 is the least trustworthy high-confidence signal in the model.

Training-data provenance

No training document, panel, corpus or source text is distributed in this repository. The adapter was trained on synthetic documents, and some of those documents were produced by editing third-party natural texts. Those texts are listed here so that their provenance is on the record, and so that a reviewer doing lawful-sourcing diligence can see what was used without having to ask.

The model card discloses the provenance of the third-party texts the training panel was edited from, and attributes them. This is provenance disclosure and attribution as good practice, and it supports documented-lawful-sourcing procurement review. It is NOT a concession that the trained weights are a derivative work or an adaptation of those texts; that question is open and nothing in this package answers it.

13 of the 42 reviewed sources were used in the training panel:

Source URL Licence, as recorded
Django's security policies https://docs.djangoproject.com/en/dev/internals/security/ BSD 3-Clause (Django project LICENSE, which covers the documentation in the django/django repository)
CC BY-SA Wikipedia:Arbitration/Policy (English Wikipedia arbitration policy) https://en.wikipedia.org/wiki/Wikipedia:Arbitration/Policy CC BY-SA 4.0 (Wikipedia text; attribution: English Wikipedia contributors, 'Wikipedia:Arbitration/Policy')
Common Paper Mutual Non-Disclosure Agreement, Version 1.0 – Standard Terms https://commonpaper.com/standards/mutual-nda/1.0/ CC BY 4.0 (stated in the agreement footer and in the CommonPaper/Mutual-NDA repository README: 'free to use and modify under CC BY 4.0')
36 CFR Part 2 (National Park Service) - Resource Protection, Public Use and Recreation: sections 2.10, 2.13, 2.14, 2.15, 2.16, 2.21 and 2.22 https://www.ecfr.gov/current/title-36/chapter-I/part-2 US Government work (public domain): Code of Federal Regulations text, not subject to copyright (17 U.S.C. 105)
NSF Proposal & Award Policies & Procedures Guide (PAPPG, NSF 24-1), Chapter IV: Non-Award Decisions and Transactions https://www.nsf.gov/policies/pappg/24-1/ch-4-non-award-decisions-transactions US Government work (public domain): U.S. National Science Foundation policy guide, not subject to copyright (17 U.S.C. 105)
GOV.UK: Make a court claim for money https://www.gov.uk/api/content/make-court-claim-for-money Open Government Licence v3.0
42 CFR 68: NIH Loan Repayment Programs (2025 edition) https://www.govinfo.gov/content/pkg/CFR-2025-title42-vol1/xml/CFR-2025-title42-vol1-part68.xml US Government work (public domain)
CC BY-SA WordPress.com (Automattic) Terms of Service, last updated April 10, 2026 https://github.com/Automattic/legalmattic/blob/master/Terms%20of%20Service/WordPress.com/EN-Terms-of-Service.md CC BY-SA 4.0 (Automattic/legalmattic LICENSE.txt and README; the Terms themselves state they are available under a Creative Commons Sharealike license)
CC BY-SA GitLab Handbook: Global Travel and Expense Policy (sections 1 to 3) https://handbook.gitlab.com/handbook/finance/expenses/ CC BY-SA 4.0 (GitLab handbook content; licence badge in the footer of handbook.gitlab.com pages)
Sourcegraph Handbook: Spending company money https://github.com/sourcegraph/handbook/blob/main/content/benefits-pay-perks/benefits-perks/spending-company-money.md Apache License 2.0 (LICENSE of the public sourcegraph/handbook repository)
Project Jupyter Governance: Executive Council https://raw.githubusercontent.com/jupyter/governance/main/docs/executive_council.md CC0 1.0 Universal (jupyter/governance repository LICENSE.md)
29 CFR 1904.30–1904.34: establishment records and annual summaries (2025 edition) https://www.govinfo.gov/content/pkg/CFR-2025-title29-vol5/xml/CFR-2025-title29-vol5-part1904.xml US Government work (public domain)
31 CFR 1.2, 1.4 and 1.6: Treasury FOIA requests and appeals (2025 edition) https://www.govinfo.gov/content/pkg/CFR-2025-title31-vol1/xml/CFR-2025-title31-vol1-part1.xml US Government work (public domain)

3 of these are recorded as CC BY-SA 4.0 — Wikipedia:Arbitration/Policy (English Wikipedia arbitration policy), WordPress.com (Automattic) Terms of Service, last updated April 10, 2026, GitLab Handbook: Global Travel and Expense Policy (sections 1 to 3). Share-alike is the one term attribution cannot cure. Whether a share-alike obligation can propagate through training into model weights is legally unsettled; there is no authority settling it in either direction, and the project's own licence review explicitly declines to infer one. The owner of this release accepted that residual risk on 2026-09-21 rather than resolving it, and kept this package under Apache-2.0. A reader should treat the question as open, not answered.

On the strength of this evidence. The licences above are as recorded by the person who collected each source, from the source's own stated terms at the time of collection. The review records evidence_level: "authoring metadata assertion, not archived governing licence text" and upstream_terms_independently_verified: false for every row. No governing licence text was archived alongside most of these sources, and this listing should not be read as a licence audit.

Listing these sources is provenance disclosure and attribution as good practice. It is not a statement that the trained weights are a derivative work, an adaptation, or a copy of any of these texts.

Third-party dependency licences

requirements.lock names the packages the serving layer needs. None of them is redistributed in this repository — you install them yourself from their own publishers — so Apache-2.0 section 4(a) imposes no bundled-notice obligation here and no dependency licence text is packaged. This summary exists because a reviewer will ask for one.

Each licence in the table is the one declared in that distribution's OWN package metadata -- the METADATA file of an installed wheel -- read from a copy on the maintainer's machine. Where no copy existed, the row says NOT VERIFIED instead of guessing.

Package Pinned as Licence, as declared by the distribution itself
torch 2.13.0 BSD-3-Clause — read from version 2.8.0
torchvision 0.28.0 NOT VERIFIED
transformers 5.17.0 Apache 2.0 License
flash-linear-attention 0.5.2 NOT VERIFIED
safetensors unpinned in the qualified image Apache Software License — read from version 0.8.0
accelerate unpinned in the qualified image Apache (Apache Software License) — read from version 1.15.0
numpy unpinned in the qualified image BSD-3-Clause AND 0BSD AND MIT AND Zlib AND CC0-1.0 — read from version 2.5.3
scipy unpinned in the qualified image BSD License (classifier); the METADATA License field carries the Enthought / SciPy Developers copyright line rather than an SPDX identifier — read from version 1.18.1
pillow unpinned in the qualified image MIT-CMU — read from version 12.3.0

Two rows say NOT VERIFIED and mean it. A licence read from one installed version is evidence about that version only. Where the table names a version different from the pin, that is the version whose metadata was actually read, and the pinned version's own metadata could differ. Treat this table as a starting point for your own review, not as a legal opinion, and re-check the distributions you actually install.

Full evidence paths for each row are in the release descriptor (release/solomon-release.json → dependency_licences), which is not distributed; the same information is repeated in the comments of requirements.lock.

Licence and attribution

Copyright 2026 Doccy Pty Ltd.

This repository is licensed Apache-2.0 — the adapter and head weights, the calibration artifact, the serving code and the documentation alike. See LICENSE and NOTICE.

It is a derivative work of Qwen/Qwen3.8-27B, Copyright 2026 Alibaba Cloud, licensed under Apache-2.0. The upstream licence text is reproduced verbatim at licenses/Qwen-Apache-2.0.txt (sha256 bbedc3fda3305820b977265f01b8619d87570a6739de3a5582c3464840f1e57a). NOTICE carries the attribution required by section 4(c) and MODIFICATIONS.md the statement of changes required by section 4(b).

No NOTICE file exists in the upstream repository at the pinned revision (HTTP 404, checked 2026-09-18), so section 4(d) carries nothing forward and no upstream attribution text has been invented.

"Qwen" and "Alibaba Cloud" are used nominatively to identify the base model. Apache-2.0 section 6 grants no trademark rights and none are claimed. No endorsement or affiliation is implied.

Verify what you downloaded

shasum -a 256 adapter/adapter.safetensors      # 2addaf841ecc58829ad73081603b8d3e52743c53c6d558a17a1dd30e9bd2dbca
shasum -a 256 adapter/heads.npz                # 126a9b5487dca937a768a4f228f2d2e7d513900d8ad0e99c4137fbaa42a1aa6f
shasum -a 256 serving/readout-temperature-v2.json
#   -> d6925a0168b2577d0e4d517cd25dd5ceb0b7cc8eee91b70aefd3a3aa9dcb64ce

MANIFEST.json carries the size and sha256 of every file in this repository.

The calibration artifact has two legitimate and different hashes, and confusing them makes a sound provenance chain look tampered with. d6925a01… is the file hash, what shasum returns. 6506e239… is the artifact's own internal sha256 field, computed over its contents with that field removed — a self-referential field cannot hash the file containing it. The loader verifies the payload hash; use the file hash to check the file you were given. Both are recorded in the serving binding's provenance, under those names.

Release record

Machine-readable identity for citation and pinning. The model identity, the calibration and the runtime binding move together; pin by revision.

Repository DoccyHealth/Solomon
Release 1.1.0, 2026-09-21
Serving contract solomon-v1
Adapter sha256 2addaf841ecc58829ad73081603b8d3e52743c53c6d558a17a1dd30e9bd2dbca
Heads sha256 126a9b5487dca937a768a4f228f2d2e7d513900d8ad0e99c4137fbaa42a1aa6f
Base model Qwen/Qwen3.8-27B at 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
Runtime binding sha256 (payload) 49904e27814e0ccfc483b0503237d84f8ca60a79713f393b23b0dbe57d87a6bf
Calibration sha256 (file / payload) d6925a0168b2577d0e4d517cd25dd5ceb0b7cc8eee91b70aefd3a3aa9dcb64ce / 6506e2394bda5f8e7c7b9742f70a5b8782efad53d5451d5d836ba3463cece24b
Runtime fingerprint c3d933e068e19879b45fb6dc6ce4af83a4d22f615c263b209e15f9c4fb2ae952
Readout four_collapsed
Ruled temperatures boolean 1.9476 · single 1.0 · ordered 1.5063 · multilabel 1.6895 · entity 2.1834

Identity and Version

Repository
DoccyHealth/Solomon
Publisher
Doccy Health
Task
Text classification
Modality
Text
Library
peft
Parameters
Not stated by the source
Languages
Not stated by the source
Revision
7f592ed277d591aee2a22b7808e531f85c2a8c20
First published
2026-09-21
Last updated
2026-09-21

Files and Weights

96 files, 873.5 MB in total. The weights are 2 files totalling 872.4 MB in npz, safetensors.

Weights2 files · 872.4 MB
Configuration72 files · 531.0 KB
Documentation14 files · 128.4 KB
Other6 files · 377.8 KB
Repository2 files · 343 B
Every file
FileTypeSizeSHA-256
adapter/adapter.safetensorsWeights870.4 MB 2addaf841ecc
adapter/heads.npzWeights2.1 MB 126a9b5487dc
MANIFEST.jsonConfiguration33.1 KB —
adapter/config.jsonConfiguration2.2 KB —
mlx/CONVERTER-SOURCE.jsonConfiguration1.5 KB —
mlx/bf16/base-manifest.jsonConfiguration6.4 KB —
mlx/bf16/binding.jsonConfiguration12.3 KB —
mlx/bf16/conversion.jsonConfiguration6.9 KB —
mlx/examples/decide.pyConfiguration763 B —
mlx/scripts/benchmark.pyConfiguration5.1 KB —
mlx/scripts/benchmark_chunks.pyConfiguration1.9 KB —
mlx/scripts/check_cuda_parity.pyConfiguration7.6 KB —
mlx/scripts/score_parity.pyConfiguration542 B —
mlx/scripts/validate_api.pyConfiguration5.9 KB —
mlx/src/solomon_mlx/__init__.pyConfiguration132 B —
mlx/src/solomon_mlx/_vendor/__init__.pyConfiguration113 B —
mlx/src/solomon_mlx/_vendor/contract.pyConfiguration6.5 KB —
mlx/src/solomon_mlx/_vendor/evidence.pyConfiguration7.7 KB —
mlx/src/solomon_mlx/_vendor/evidence_v3.pyConfiguration24.8 KB —
mlx/src/solomon_mlx/_vendor/prompts.pyConfiguration4.7 KB —
mlx/src/solomon_mlx/_vendor/retrieval.pyConfiguration9.3 KB —
mlx/src/solomon_mlx/_vendor/semantics.pyConfiguration3.1 KB —
mlx/src/solomon_mlx/api.pyConfiguration13.8 KB —
mlx/src/solomon_mlx/artifacts.pyConfiguration2.6 KB —
mlx/src/solomon_mlx/budget.pyConfiguration2.9 KB —
mlx/src/solomon_mlx/cli.pyConfiguration3.0 KB —
mlx/src/solomon_mlx/engine.pyConfiguration15.4 KB —
mlx/src/solomon_mlx/evaluation.pyConfiguration13.9 KB —
mlx/src/solomon_mlx/prepare.pyConfiguration5.5 KB —
mlx/src/solomon_mlx_hub/__init__.pyConfiguration248 B —
mlx/src/solomon_mlx_hub/__main__.pyConfiguration1.9 KB —
mlx/src/solomon_mlx_hub/data/base-manifest.jsonConfiguration6.4 KB —
mlx/src/solomon_mlx_hub/data/release.jsonConfiguration8.6 KB —
mlx/src/solomon_mlx_hub/prepare.pyConfiguration8.7 KB —
mlx/tests/test_artifacts_and_evaluation.pyConfiguration2.7 KB —
mlx/tests/test_budget.pyConfiguration780 B —
mlx/tests/test_contract.pyConfiguration3.4 KB —
mlx/tests/test_hub.pyConfiguration7.2 KB —
mlx/tests/test_metal_primitives.pyConfiguration1.7 KB —
mlx/tests/test_parity.pyConfiguration763 B —
mlx/tests/test_processor.pyConfiguration853 B —
mlx/tests/test_tiny_decoder.pyConfiguration2.4 KB —
mlx/tests/test_tiny_library.pyConfiguration6.7 KB —
serving/readout-temperature-v2.jsonConfiguration3.4 KB —
serving/selection.jsonConfiguration551 B —
serving/service-export.jsonConfiguration452 B —
serving/serving-binding.jsonConfiguration5.6 KB —
src/solomon/__init__.pyConfiguration1.2 KB —
src/solomon/api.pyConfiguration3.9 KB —
src/solomon/binding.pyConfiguration13.1 KB —
src/solomon/calibration.pyConfiguration10.2 KB —
src/solomon/engine.pyConfiguration5.7 KB —
src/solomon/engine_contract.pyConfiguration18.3 KB —
src/solomon/engine_cuda.pyConfiguration7.4 KB —
src/solomon/engine_numerics.pyConfiguration3.0 KB —
src/solomon/engine_reasoning.pyConfiguration9.2 KB —
src/solomon/engine_reference.pyConfiguration13.0 KB —
src/solomon/evidence.pyConfiguration7.6 KB —
src/solomon/evidence_packages.pyConfiguration24.7 KB —
src/solomon/heads.pyConfiguration14.0 KB —
src/solomon/prompts_two_letter.pyConfiguration4.2 KB —
src/solomon/readout.pyConfiguration14.1 KB —
src/solomon/reliability.pyConfiguration1.5 KB —
src/solomon/retrieval.pyConfiguration9.2 KB —
src/solomon/routing.pyConfiguration3.0 KB —
src/solomon/semantics.pyConfiguration2.9 KB —
src/solomon/service.pyConfiguration32.3 KB —
src/solomon/service_answers.pyConfiguration20.1 KB —
src/solomon/service_checked.pyConfiguration7.6 KB —
src/solomon/service_evidence.pyConfiguration7.8 KB —
src/solomon/service_heads.pyConfiguration9.4 KB —
src/solomon/service_packages.pyConfiguration13.2 KB —
src/solomon/service_states.pyConfiguration10.7 KB —
src/solomon/serving.pyConfiguration4.0 KB —
LICENSEDocumentation11.3 KB —
MODIFICATIONS.mdDocumentation6.4 KB —
NOTICEDocumentation4.8 KB —
README.mdDocumentation52.0 KB —
mlx/LICENSEDocumentation11.3 KB —
mlx/MODIFICATIONS.mdDocumentation1.6 KB —
mlx/NOTICEDocumentation3.9 KB —
mlx/README.mdDocumentation7.4 KB —
mlx/bf16/LICENSEDocumentation11.3 KB —
mlx/bf16/MODIFICATIONS.mdDocumentation4.2 KB —
mlx/bf16/NOTICEDocumentation3.9 KB —
mlx/bf16/README.mdDocumentation3.1 KB —
mlx/docs/UPSTREAM-MODIFICATIONS.mdDocumentation4.2 KB —
mlx/docs/VALIDATION-20260921.mdDocumentation2.9 KB —
licenses/Qwen-Apache-2.0.txtOther11.5 KB —
mlx/pyproject.tomlOther1.1 KB —
mlx/requirements.cloud.lockOther74.5 KB —
mlx/requirements.lockOther1.3 KB —
mlx/uv.lockOther285.4 KB —
requirements.lockOther3.9 KB —
.gitattributesRepository217 B —
mlx/.gitignoreRepository126 B —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
872.4 MB
Download from Doccy Health

Released by Doccy Health through its official repository on Hugging Face. Read the license.

Built From

Memory Requirements

PrecisionWeights in memory
As published872.4 MB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Solomon

Can I use Solomon commercially?

Yes. Solomon is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

Similar Models

Model · Text classification

twitter-roberta-base-sentiment-latest

Cardiff NLP

This is a RoBERTa-base model trained on ~124M tweets from January 2018 to December 2021, and finetuned for sentiment analysis with the TweetEval benchmark. The original Twitter-based RoBERTa model can be found here and the original reference paper is TweetEval. This model is suitable for English. 0 -> Negative; 1 -> Neutral; 2 -> Positive This sentiment analysis model has been integrated into TweetNLP. You can access the demo here.

Open weights cc-by-4.0 514 tokens transformers

Model · Text classification

finbert-tone

Yi

FinBERT is a BERT model pre-trained on financial communication text. The purpose is to enhance financial NLP research and practice. It is trained on the following three financial communication corpus. The total corpora size is 4.9B tokens. More technical details on FinBERT: Click Link This released finbert-tone model is the FinBERT model fine-tuned on 10,000 manually annotated (positive, negative, neutral) sentences from analyst reports. This model achieves superior performance on financial tone analysis task. If you are simply interested in using FinBERT for financial tone analysis, give it a try. If you use the model in your academic work, please cite the following paper: Huang, Allen H.…

Open weights 512 tokens transformers

Model · Text classification

ms-marco-MiniLM-L-6-v2

Joshua

https://huggingface.co/cross-encoder/ms-marco-MiniLM-L-6-v2 with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx).

Open weights 512 tokens transformers.js

Model · Text classification

twitter-xlm-roberta-base-sentiment

Cardiff NLP

This is a multilingual XLM-roBERTa-base model trained on ~198M tweets and finetuned for sentiment analysis. The sentiment fine-tuning was done on 8 languages (Ar, En, Fr, De, Hi, It, Sp, Pt) but it can be used for more languages (see paper for details). This model has been integrated into the TweetNLP library.

Open weights 514 tokens transformers

Model · Text classification

unbiased-toxic-roberta

Unitary

The huggingface models currently give different results to the detoxify library (see issue here). For the most up to date models we recommend using the models from https://github.com/unitaryai/detoxify Trained models & code to predict toxic comments on 3 Jigsaw challenges: Toxic comment classification, Unintended Bias in Toxic comments, Multilingual toxic comment classification. Built by Laura Hanu at Unitary, where we are working to stop harmful content online by interpreting visual content in context. Score not directly comparable since it is obtained on the validation set provided and not on the test set. To update when the test labels are made available. It is also noteworthy to mention…

Open weights apache-2.0 514 tokens transformers

Model · Text classification

emotion-english-distilroberta-base

Hartmann

With this model, you can classify emotions in English text data. The model was trained on 6 diverse datasets (see Appendix below) and predicts Ekman's 6 basic emotions, plus a neutral class: 1) anger 2) disgust 3) fear 4) joy 5) neutral 6) sadness 7) surprise The model is a fine-tuned checkpoint of DistilRoBERTa-base. For a 'non-distilled' emotion model, please refer to the model card of the RoBERTa-large version. a) Run emotion model with 3 lines of code on single text example using Hugging Face's pipeline command on Google Colab: b) Run emotion model on multiple examples and full datasets (e.g.,.csv files) on Google Colab: Please reach out to [email protected] if you have any…

Open weights 514 tokens transformers