Vinci-Prova-7B-1.0

Repo: simpledirect/Vinci-Prova-7B-1.0

Follow-up — 1 September 2026. The condition this card sets below — "the same frozen recipe on at least three meaningfully different bases" — has since been tested. Vinci Technical Report No. 2 applied the frozen recipe to Qwen3 8B, Ministral 3 8B and OLMo 3 7B with five paired seeds per family. Unsupported assertions declined in all three families under both judges, but no family preserved grounded-answer accuracy well enough to meet the pre-registered bar, and answer coverage also violated its limit under one judge. The Mistral measurements reported below are unchanged; the follow-up narrows how broadly they should be interpreted and does not establish general cross-lineage portability.

Development-tier validation evidence only. The refusal adjustment is Judge-B-only. Capability preservation was not evaluated. No external audit was performed. The primary holdout remains sealed. No model checkpoint is recommended for release.

Read it: Technical Report No. 2 · DOI 10.5281/zenodo.22236690

An experimental post-training transfer study. We applied the Vinci SFT + DPO character recipe to mistralai/Mistral-7B-Instruct-v0.3 to answer one question: does character training developed on a different model lineage transfer to this one? Apache-2.0, 7.25B, drop-in with transformers.

This uses a retired base, and we are saying so first. Mistral lists Mistral 7B Instruct v0.3 as retired as of 30 March 2025 (deprecated 30 November 2024), with Ministral 3 8B as the recommended replacement. "Retired" is Mistral's own lifecycle term. The open weights remain downloadable on Hugging Face under Apache-2.0. We selected this base for continuity with our earlier experiments, not because it is current. If you are choosing a base to build on today, this is not it.

The answer is yes, on the sets we measured. Four internal behavioural evaluations move from FAIL to PASS, and model-judged fabrication falls from 53.8% to 8.6% on our development baits, with 7.5% on a held-out set written after the recipe was frozen.

This is not a Vinci Bozza successor and is not recommended for production. It loses substantially to Bozza on general capability. Vinci Bozza 1.0 remains our recommended small model. We are publishing this because the transfer result is real and because two measurement failures we found along the way are more useful to other people than the checkpoint is.

Scope, stated once and meant throughout. This is evidence of transfer to one base, not evidence of general cross-lineage portability — that would require the same frozen recipe on at least three meaningfully different bases. The evaluation sets are small internal ones (93 fabrication baits plus 11 controls, 40 adversarial prompts, 36 character items, 30 honesty items — 210 unique prompts, verified non-overlapping) that we iterated against while developing the recipe. The post-freeze fabrication suite adds a further 104 prompts (93 baits + 11 controls) which were never used for development. Only the fabrication axis has a post-freeze held-out result; the character, jailbreak and honesty numbers remain development-set findings.

The full write-up is Vinci Technical Report No. 1. The complete study — method, statistics, figures, and limitations — is published as a citable technical report: read it online or download the PDF. George Pu and Ayush Naik, Version 1.0, 13 August 2026, licensed CC BY 4.0. The model weights remain Apache-2.0.

Model at a glance

Weight-file size is not a runtime-memory requirement. Loading, KV cache, context length, batching and the inference runtime all need memory beyond the weights.

Should you use this model?

The prose above already routes most readers away; this is the same judgement made scannable. The left column is what this release was built to test. The right column is where something else will serve you better — and on its own measurements, most general-purpose work is in the right column.

What makes it distinct

What is distinctive about this release is a behaviour change, not capability. Each item is labelled measured or design intent.

• Reduced model-judged fabrication on adversarial baits — measured. 53.8% → 8.6% on the development baits and 46.2% → 7.5% on baits written after the recipe was frozen. Paired McNemar base → this release: 42 items fixed, 0 newly broken, exact p = 4.6 × 10⁻¹³. Judged by openai/gpt-4o with web search, with no human adjudication.
• Character transfer to a different model lineage — measured, development sets only. Four internal behavioural evaluations move from FAIL to PASS against the base, character_pref 19.4% → 94.4%. These sets were iterated against during development and have no post-freeze replication.
• Reticence, not accuracy — measured. On held-out prompts the lower-beta models made fewer specific assertions (17.2 vs 22.0 per 93 baits), and we found no evidence that accuracy conditional on asserting improved. The mechanism is that it answers specifically less often.
• Published item-level evidence — design intent, delivered. Every one of the 15 fabrications, a second AI source-confirmation pass, a false-negative sample, the full beta dose–response and the safety wall that bounds it are in EVAL.md and SOURCE-AUDIT.md, so a reader can disagree with a specific call rather than with a percentage.
• Two measurement failures published at equal prominence — design intent, delivered. The deterministic gate's ranking failure and the development/held-out shrinkage are reported because they are more transferable than the checkpoint.
• Honesty as abstention at a small parameter count — design intent. We are not aware of a small honesty-positioned open model at this scale; that is an observation about a gap, not a priority claim, and the prior work below predates us.
• Not distinctive: general capability — measured. It is beaten on MMLU and GSM8K by smaller models from other vendors and by our own recommended small model. Nothing here changes that.

Results at a glance

Behavioural transfer, on our development sets

Same prompts, same harness, greedy decoding (do_sample=False, max_new_tokens=1024) for the upstream base and this release.

Character results by axis:

The aggregate is strong on this set, but conventional_wisdom remains weak and contains only four items. Four items cannot support a claim in either direction. We do not consider that axis solved.

These are development-set results. We used these sets repeatedly while comparing training arms, so they are evidence of transfer on the measured prompts — not an unbiased estimate of general performance.

Fabrication, model-judged with search

The held-out set was written after the recipe and shipping checkpoint were frozen. It contains 93 adversarial baits and 11 non-adversarial controls, uses different jurisdictions and subject matter, and was screened against the training corpus. It was not used to select this model.

What the held-out column establishes. The upstream base has now been evaluated on the held-out set too, so the base-to-release comparison is reproduced on prompts we never developed against: 46.2% → 7.5%, against 53.8% → 8.6% on the development set. The effect is somewhat smaller on held-out items — the base fabricates less there (46.2% vs 53.8%), so the set is easier for it — but the direction and the rough magnitude both survive.

Two limits worth keeping in view. The held-out set covers fabrication only: the character, jailbreak and honesty results remain development-set findings with no post-freeze replication. And the base's held-out adjudication leaned more heavily on reasoning than search (41 of 69 judged items), which is a weaker evidentiary basis than we would like for the number that anchors the comparison.

Because the same 93 baits are scored at every stage, these are paired data. McNemar's exact test on the discordant items, computed for this release (not for the superseded checkpoint):

Both stages contribute. Note the SFT stage breaks 5 items the base answered acceptably, so "improves fabrication" is not the same as "never makes anything worse" — though the full base→release transition breaks none. Items the screen did not surface are counted as non-fabrications at every stage. That assumption affects both the absolute rates and the measured differences — screening recall was not independently estimated, and misses need not fall equally across checkpoints.

These percentages are rates on prompts deliberately constructed to elicit unsupported specifics. They are not general real-world hallucination rates and should not be quoted as such.

What the DPO beta change did — and what it did not

Across matched beta=0.1 and beta=0.05 training seeds, the lower-beta recipe reduced held-out fabrication by an estimated 2.97 percentage points (95% bootstrap CI +0.89 to +4.87; 14 of 17 paired seeds improved; two-sided exact sign test p = 0.013). Measured on the development set the same contrast looked worth 7.5 points — so the held-out set reduced the estimated effect from 7.5 to 2.97 percentage points.

We checked whether that shrinkage is just the held-out set being easier. Under simple uniform multiplicative compression the ratio between arms would be preserved; it is not (1.47 development, 1.18 held-out). The result is not consistent with simple uniform compression, although differences in item composition may also contribute.

On the held-out prompts, lower-beta models made fewer specific assertions (17.2 vs 22.0 per 93 baits). We found no evidence that accuracy conditional on asserting improved — the observed conditional error rates were 48.8% vs 42.7%, and we did not test that difference for significance. Our supported interpretation:

This training makes the model more reticent when a prompt invites an unsupported answer. We have not shown that it makes the model more accurate once it chooses to answer specifically.

That distinction matters: a model that declines more often can fabricate less without knowing more.

A wider dose–response across beta from 0.0125 to 0.20 is monotone in the same direction. We report it in EVAL.md rather than here, because those checkpoints share seeds, data and training conditions, so treating them as independent observations would overstate the confidence.

Capability trade-offs

This release is not competitive with our mainline small model on general capability. All rows are our own harness at matched protocol.

Bozza leads this release by 18.6 MMLU points while also being larger. A separate comparison: the 4.21B Qwen-derived model beats this release by 15.0 MMLU points while being 42% smaller.

What the training costs, measured against our own base

The most important row in that table is the last one, and until now it was blank. We have now run the untrained base on our own harness at matched protocol:

This training does not improve general capability. It costs 5.6 points of GSM8K against the base (0.516 → 0.460), leaves MMLU effectively unchanged (−0.6 points, within our seed spread), and improves TruthfulQA by 3.0 points. Almost all of the GSM8K loss happens at the SFT stage (0.516 → 0.448); DPO recovers a little of it.

So the honest summary of the trade is: a 53.8% → 8.6% reduction in judged fabrication, bought with 5.6 points of GSM8K. Whether that is a good trade depends entirely on what you are doing. For arithmetic and multi-step reasoning it is a bad one, and you should use a different model.

Against models outside our own lineup

Our table above compares only Vinci models on our own harness. That is the honest protocol, but it also flatters us by omission, so here is the outside view. These figures are from other vendors' published cards, measured on their harnesses, not ours — they are not matched-protocol and should be read as indicative:

A 3.8B MIT-licensed model beats this release on both axes, and so does a 3B Llama. Mistral's own newer small model beats it too. On general capability this release is not competitive at any size, and no framing of ours changes that.

A note on these two benchmarks. MMLU and GSM8K are no longer carried in some major public indices, and several 2026 model cards report neither. We publish them because our historical comparisons use them, not because we think they are the right instruments in 2026.

On base choice. Our implementation of the allied-base constraint incurred a substantial capability cost in these comparisons. We are not claiming that allied bases generally impose such a cost — the age and capability of this particular retired base are major confounders.

One thing DPO clearly does here: TruthfulQA MC2 rises from 0.5397 (SFT parent) to ~0.60 at both DPO betas, about 6.8 points. The stage effect looks real; the difference between the two DPO checkpoints (0.6076 superseded vs 0.6034 here) does not, and moved opposite to fabrication.

Independent paired re-measurement against the base — 21 September 2026

Everything above this subsection was measured by us during development. This subsection adds a separate, later re-measurement on a different harness configuration, comparing this release directly against its own base on per-example records. It adds numbers. It does not revise any figure above, and the two sets are not comparable — see "How this relates to the figures above".

Status: exploratory. One run per arm, not pre-registered, and no negative-control arm was included. These are strong candidates, not confirmed results: a finding selected because it was large is biased upward, and nothing here has been replicated on fresh items. Confirmatory work would pre-register the hypothesis, direction and analysis plan before the run.

Protocol, stated in full so it can be repeated:

• lm-evaluation-harness 0.4.11, HF path, dtype=bfloat16, batch_size=8, seed 0, greedy decoding, no chat template applied to either arm.
• Subject simpledirect/Vinci-Prova-7B-1.0; base mistralai/Mistral-7B-Instruct-v0.3.
• 0-shot on every task except GSM8K, which is 5-shot capped at max_gen_toks=512 on both arms. The cap was fixed before any score was seen.
• Exact McNemar on the retained per-example records, aligned by doc_id, with item identity verified by doc_hash — doc_id alone does not prove both arms saw the same question.
• 90% Clopper-Pearson intervals on the discordant pairs.
• Holm–Bonferroni across the six tasks below. Six tasks on this one model is one family; these results were not pooled with any other model into a larger family.
• Measured 21 September 2026.

b counts items the base answered correctly and this release did not; c counts the reverse. A negative difference means this release scores below its own base.

Five of the six survive Holm correction. All six point the same way: below the base.

How this relates to the figures above. It does not revise them, and no figure above has been changed. The capability tables above came from our own internal lm_eval run whose version was never recorded, using per-task --limit subsets; this is lm-evaluation-harness 0.4.11 on the full task sets with no chat template. Different harness configuration, different item sets, different decoding context — the two sets of numbers are not comparable, and neither corrects the other. Four of the six tasks here (HellaSwag, WinoGrande, ARC-Challenge, PIQA) do not appear anywhere else on this card.

What it does do is put per-example records behind a statement this card already made from aggregates: this training does not improve general capability. On every axis tested, the paired records agree with that statement.

GSM8K is directional, not a budget-independent measurement. The generation cap is part of the protocol and its effect is large — capping or uncapping generation can move a GSM8K score by several percentage points with nothing else changed, because an uncapped model generates past its answer and the strict extractor loses it. That does not cancel in a paired comparison: each model over-generates by a different amount, so a verbose model is penalised more than a terse one and the cap silently reweights the comparison. Both arms here carried the same 512-token cap and it was chosen before the scores were seen, which is the most that protocol can do. Read that row as direction, not as a number independent of the budget. The card's own GSM8K figures above were measured under a different protocol and are unaffected by this row.

MMLU does not survive correction, at p = 0.05063 against a Holm threshold of 0.05 for the last-ranked test in a family of six. That is not a null and should not be read as "no difference". The bound is the part worth citing: on these 14,042 items, any MMLU difference lies between 1.17 points below the base and 0.10 points below it. The interval sits entirely on the negative side, but the effect is not established at this family's correction level, and a single exploratory run is not the place to argue about a 0.00063 margin.

What this subsection does not do. It does not close the gap recorded under "What is not measured yet" — that this card's own capability rows have no per-example logs, no discordant count and no McNemar in either direction. That entry stands as written and describes the figures above, which remain untested aggregates; this is a separate measurement, not a retrofit of those. This subsection also covers only these six tasks against this one base. No fabrication, character, honesty or jailbreak result is re-measured here, no quantised tier is covered, and nothing here is a safety, security or fitness-for-purpose claim.

Known failure modes

It may hedge and then fabricate

The characteristic error is an answer that declines to commit and then asserts a specific anyway — "I cannot pull an exact figure from memory… the relevant section is likely §31 or §32." This reads as careful and is not. It is also why our cheap gate underreports (below).

It sometimes refuses ordinary work

It will decline a fill-in-the-blank or an "answer in exactly two sentences" instruction on the grounds that a clean short answer would be half-right, then answer correctly in its own format. That is a usability cost, and it is the same behaviour as the reticence that lowers its fabrication rate — not a separate flaw.

Training-seed variance is material

Across n = 23 replicates of the beta=0.1 recipe on this base, honest_positive spans 83%–97% and character_pref spans 86%–89%. This release is a beta=0.05 checkpoint and its 94.4% character_pref sits outside that beta=0.1 range; we have not run 23 replicates of the beta=0.05 recipe, so treat its per-gate figures as one draw, not a guarantee. A ~14-point spread on honest_positive exceeds most differences anyone would want to claim between two checkpoints.

Seed discipline. This release uses seed 42, the training script default — not a seed chosen after looking at scores. On the held-out set it ranks 9th of 32 checkpoints we scored; the best (4.3%) is a different seed we are not shipping. We selected this checkpoint on the development set before the held-out set existed, so its 7.5% is confirmation rather than selection.

Evaluation integrity

Provenance of the reported figures

Every number should say what artifact, what harness, what protocol, how many items, and when. This card meets some of that and not all of it, so here is the accounting.

The consequence, stated plainly: the capability and gate figures on this card are not reproducible as stated. Without a harness version and a run date, the same weights measured on a later harness can return a different number, and there is no way to tell which of the two is wrong or whether anything changed at all. That is a defect in our record-keeping, not a hedge about the model. It does not make the figures false; it makes them uncheckable by anyone outside, including by us at a later date.

Two further limits on the tables above. The --limit subsets are internally comparable because every model received the identical limit, and lm_eval itself prints a warning that limited runs must not be treated as real metrics — do not place these numbers in a leaderboard table. And the third-party rows in "Against models outside our own lineup" come from other vendors' published cards on their own harnesses; they are labelled indicative there and are not matched-protocol.

What is recorded lives in two files in this repository, and they are the reason a reader can check most of this at all:

• EVAL.md — the protocol in full, the capability table including the superseded beta=0.1 checkpoint, the four behavioural gates, all 15 item-level fabrication findings with trap type and the judge's basis (search or reasoning), the complete beta dose–response from 0.20 down to 0.0125 with the within-seed analysis, the safety gate failure at beta=0.0125 that is the actual reason 0.05 ships, the seed-variance data, the deterministic gate's rank correlations, an "Open and unverified" list, and the 20-item false-negative sample with its design and its indexing-bug postmortem.
• SOURCE-AUDIT.md — the completed source-confirmation packet dated 10 August 2026, one entry per flagged positive across both sets, each carrying the prompt, the model's assertion, the audit's conclusion, its rationale and its source links, plus the explicit instruction not to describe the result as human-verified.

Development-set reuse

The fabrication, adversarial, character and honesty sets were used repeatedly during recipe development and model comparison. A training-corpus screen found no exact or near-duplicate prompt overlap, but that does not remove evaluation overfitting caused by repeated iteration against the same tests.

The held-out fabrication set was created only after the recipe and checkpoint were frozen. Screening detail, because the two corpus figures in our notes differ and both are correct: the shipping run used 983 preference pairs, selected from a 1,909-pair DPO source pool. The contamination screen ran against 80,752 prompt records — every user-turn prompt extracted from the DPO source pool, the SFT corpus, and the prepared training bundles, counted as records rather than deduplicated unique strings. Zero exact and zero near matches. The near-match metric is Jaccard similarity over word 5-grams, and an item is flagged when similarity ≥ the threshold — so the second pass at ≥0.40 is the more sensitive one (it flags strictly more than ≥0.60). Both returned nothing. A planted positive control was screened first and was caught at 1.000 (exact) and 0.848 (near), confirming the screen can detect a match at all.

Source-based fabrication review — method

This is the foundation of our most important claim, so the method is stated in full.

Rates are counts of baits, not of judged candidates: 8.6% = 8/93 and 7.5% = 7/93.

A source-confirmation pass has now been performed — it is neither blinded nor human verification. After the original adjudication, OpenAI Codex re-checked all 15 flagged positives against public primary or authoritative sources (SOURCE-AUDIT.md, 10 August 2026). Codex saw the original verdicts, so this is a confirmation pass rather than an independent second adjudication — it cannot detect a shared blind spot, only an unsupported call. It did retrieve its own sources. All 15 remained item-level fabrications, so both rates are unchanged: 8.6% development, 7.5% held-out. One development item is partial — the $100,000 PIPEDA maximum is real, but the model attributed it to a non-existent provision — and it still counts as a fabrication under the item-level rubric.

The audit was thorough enough to find errors the original judge missed: the same answer's $18.50 cap is also wrong, the "inflation-indexed" T5 threshold claim is unsupported, and the KM-1227 "successor" framing is not supported by the vendor's own specifications.

We are nonetheless not claiming human verification, because none was performed. The precise status is:

Fabrication findings were initially adjudicated by GPT-4o with web search. All 15 flagged positives were separately source-checked by OpenAI Codex, which saw the original verdicts but retrieved its own supporting sources, against public primary or authoritative sources; no human adjudication was performed. Judge-negative answers were not independently audited by Codex.

Separately, a stratified 20-item sample of judge-negative answers was re-adjudicated by the same judge model (openai/gpt-4o with search), which had not seen the original pass/fail calls for those items. It found no false negatives. The strata were the two ways an answer can count as a non-fabrication — screened then passed by the judge, and never surfaced by the screen at all — sampled 5 per stratum per evaluation set, non-proportionally, with a fixed seed. Method and per-stratum counts are in EVAL.md §9.

This is reassuring but too small to estimate screening recall tightly. The commonly cited rule-of-three bound of ~15% should be treated as heuristic here, because the sample was stratified and non-proportional rather than a simple random draw, and no weighting was applied to combine the strata.

Two model systems agreeing is a stronger evidence trail than one, and it is not the same thing as a person having checked. We describe this throughout as model-judged fabrication. A named human reviewing the completed calls and their linked sources would upgrade that wording; the audit makes that pass much faster, since every call now carries its sources.

Item-level findings for this release — all 8 development and all 7 held-out fabrications, with the judge's reason — are listed in EVAL.md. The original judge's retrieved URLs were not persisted because of a harness defect; the sources independently recovered during the Codex confirmation pass are in SOURCE-AUDIT.md and summarised in EVAL.md. Both sets are dominated by invented legal citations (fake_caselaw, fake_statute).

Publishing the item-level evidence makes this result externally auditable — but it has not been blindly or human-validated. The table is there precisely so a reader does not have to take it on trust.

The deterministic gate cannot rank checkpoints

Our cheap gate marks an answer as acceptable when a hedging/refusal regex matches, and flags fabrication otherwise. That is structurally blind to hedge-then-fabricate: the hedge matches, so the answer is scored as safe while the invented specific inside it goes uncounted.

The consequence, on the exact pair this release replaces:

The gate prefers the checkpoint that fabricates more than twice as often. That is a ranking error, not a calibration error, so no threshold change fixes it. Across 42 models with both scores, its rank correlation with judged fabrication is ρ = +0.105 (p = 0.51) — not distinguishable from zero — and on 16 held-out models it is −0.179. It does estimate the level tolerably, undercounting by a stable ~2×.

If you reproduce our numbers with the regex scorer alone you will get a different ordering than we publish, and ours is the one backed by searched sources. We keep the gate for cheap triage and never use it alone to choose between trained checkpoints.

What is not measured yet

Named specifically, because "limitations" as a word is not useful to anyone. Some of these are restated from the sections above so that the gaps are in one place.

Artifacts we ship but did not evaluate

• No GGUF tier has been evaluated. Vinci-Prova-7B-1.0-GGUF publishes Q4_K_M, Q5_K_M, Q8_0 and f16 builds. Every number on this card was measured on the bf16 model.safetensors here. Quantisation changes behaviour, and with no per-tier measurement we cannot say in which direction or by how much for any tier. Test the tier you intend to use.
• No long-context behaviour was measured. The 32,768 figure is the architecture's position limit, not a measured working length.
• No non-English evaluation. Every evaluation set is English.

Statistical work not done on the capability numbers

• No paired item-level test on capability. MMLU, GSM8K and TruthfulQA are reported as aggregate accuracies with no per-example logs, so there is no discordant count and no McNemar in either direction. The only paired analysis on this card is on fabrication, where the same 93 baits are scored at every stage. Treat the capability gaps as differences between two aggregates, not as tested differences.
• Item counts are absent from the capability rows. The two capability tables and the outside-view table report accuracies without an n. The per-task limits are in EVAL.md §1, and they carry an unresolved disagreement: EVAL.md §1 records GSM8K at limit 250, while the model-index entry in this card's own frontmatter describes GSM8K as the full test set. We have not resolved which is right, so treat neither as authoritative until it is re-run and dated.
• No multiplicity correction is reported for the capability comparisons. Three benchmarks across eight models are compared without stating a family or correcting for it.
• The MMLU seed spread is asserted but not published. The −0.6 MMLU point change is described as "within our seed spread"; the seed-variance data in EVAL.md §6 covers honest_positive and character_pref, not MMLU. The threshold that sentence leans on is not in either file.
• beta=0.0125 was never measured for capability. MMLU was not run on it, so its trade-off is unknown beyond the safety gate it fails.

Evidence gaps this card already concedes, collected

• No human adjudication of any fabrication verdict, no blinded second adjudication, and no inter-rater agreement measurement.
• The judge ran on an unpinned openai/gpt-4o alias with no provider request metadata captured.
• The original judge's retrieved URLs were not persisted, because of a harness defect.
• Judge self-consistency was not measured; the judge was not re-run on the same inputs.
• The 11 control items were never sent to the judge, so there is no judge false-positive rate on answerable items.
• The false-negative check is 20 stratified, non-proportional, unweighted items with 0 events. Its ~15% rule-of-three bound is heuristic, not a properly weighted interval.
• The held-out set covers fabrication only. The character, jailbreak and honesty results have no post-freeze replication and remain development-set findings.
• conventional_wisdom has four items and is not considered solved in either direction.

Evaluations not run at all

• No public honesty benchmark: AA-Omniscience, SimpleQA Verified, AbstentionBench, MASK and Vectara HHEM have not been run on this release.
• No third-party or external audit of any kind, and no safety evaluation beyond the internal gates reported above. A benchmark score here is not a safety claim, a security claim or a fitness-for-purpose claim.
• No measurement of the released artifact under any serving stack other than transformers — vLLM, llama.cpp, TGI and Ollama behaviour is untested here.

Prior and concurrent work

We are not the first to frame honesty as abstention rather than accuracy, and we do not claim the idea.

• Inkling (Thinking Machines, 15 July 2026) shipped open weights trained with "abstention-aware rewards: answering only pays off when the model is likely to be right" — the same thesis as this release, published before it. Its small variant is 276B total parameters.
• AbstentionBench (Kirichenko et al., Meta FAIR) benchmarks abstention directly and reports that reasoning fine-tuning degrades abstention. That result is a large part of why we think this direction is worth working on.
• Abstain-R1 applies verifiable-RL calibrated abstention at 3B.

What we believe is still uncrowded is the small end: we are not aware of a small honesty-positioned open model at this scale. That is a gap in the field, not a claim of priority.

Evaluations we have not run. We measured fabrication on our own adversarial bait sets. We have not run AA-Omniscience, SimpleQA Verified, AbstentionBench, MASK, or Vectara HHEM. A reader entitled to ask why should read that as: our result is on bespoke internal sets, and has not been placed on a public honesty leaderboard. When we run them we will publish the numbers including the ones that go against us, and we will report over-refusal alongside every honesty metric — a model can score well on hallucination purely by answering less, which is precisely the effect we found in ourselves (see above).

Model details

Lineage

mistralai/Mistral-7B-Instruct-v0.3  @ c170c708c41dac9275d15a8fff4eca08d52bab71
  └─ Vinci SFT LoRA, merged
       └─ Vinci DPO LoRA, merged (beta=0.05)  ← this release

DPO configuration

We publish merged weights. The DPO adapter reconstructs this release only when applied to the exact SFT-merged parent in a compatible environment. That parent and the training corpora are not public, so the adapter alone is not an external reproduction path.

We are not publishing the adapter. It reconstructs this release only against a parent nobody outside SimpleDirect has, so releasing it would invite reproduction attempts that cannot succeed and imply a reproducibility we do not offer.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "simpledirect/Vinci-Prova-7B-1.0"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")

messages = [{"role": "user", "content":
             "Explain what a river catchment is, in plain terms."}]
enc = tok.apply_chat_template(messages, add_generation_prompt=True,
                              return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**enc, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][enc["input_ids"].shape[-1]:], skip_special_tokens=True))

Chat template — corrected 21 September 2026. An earlier version of this card stated that the template ships only as a standalone chat_template.jinja and is not embedded in tokenizer_config.json, and warned that older transformers releases would silently fall back to no template. That is wrong for the files this repository serves. At revision 33bae16b, tokenizer_config.json carries a chat_template field whose 3,959 characters are byte-identical to chat_template.jinja, so both loading paths apply the same template and the silent-fallback failure described above does not occur. Verifying the rendered prompt before you rely on it is still worth doing; assuming it was never applied is not.

Do not use this model to produce legal, regulatory or financial citations. Its remaining fabrications are concentrated in exactly that category — invented case names and statute sections — and they arrive wrapped in hedging language that reads as careful.

Provenance and reproducibility

Verify what you downloaded against these hashes. Every evaluation number attributed to this release was produced from the weights hashing to 55f519fa…. Numbers for the base, the SFT parent, other Vinci models and third-party models obviously come from those models.

Note that config.json and tokenizer.json hash identically to the superseded checkpoint — expected, since both derive from the same base and neither DPO run altered them. Only model.safetensors differs.

Correction — the table above was re-measured on 21 September 2026 and three rows no longer match. model.safetensors, tokenizer.json and chat_template.jinja verify exactly against the values published above. The other three files do not:

The original rows are left in place rather than rewritten, so the history is visible. What this does and does not mean: the weight file hashes exactly as published, so the identity of the artifact every evaluation number was produced from is not in doubt, and no figure on this card is affected. But three of the six auxiliary files this card told you to verify would have failed that check with no way to tell which value was wrong, and at least one of them changed in a way that matters — tokenizer_config.json grew because it now carries the embedded chat template (see Usage, above). The claim in the paragraph immediately above that config.json hashes identically to the superseded checkpoint was made against the 689-byte file and has not been re-checked against the 595-byte file now served; treat it as unverified.

Re-measurement command, so you can repeat it:

REV=33bae16b2195f060ccc44c5c6277b99a0e5be9a8
for f in config.json generation_config.json tokenizer_config.json \
         chat_template.jinja tokenizer.json; do
  curl -sL "https://huggingface.co/simpledirect/Vinci-Prova-7B-1.0/resolve/$REV/$f" \
    | sha256sum | sed "s|-|$f|"
done

Status: internally traceable, not externally reproducible. We can identify the exact weights, data and configuration internally, and the base revision and released weights are pinned above. But the SFT parent is not published, the training corpora are not public, and the dependency environment is not locked. Anyone outside SimpleDirect can verify what they downloaded against our hashes once published; nobody outside can rebuild this model from what we have released.

Naming

Vinci models are named Vinci-<Family>-<Size>-<Version>[-<Format>]:

• Family — the model's enduring identity: Piccolo, Bozza, Tela, Prova.
• Size — rounded parameter class, not an exact count.
• Version — a new public weight generation, not every training run.
• Format — separately packaged distributions, e.g. Vinci-Prova-7B-1.0-GGUF.

Base model, training recipe and research hypothesis are metadata, not name components; this card and the base_model field carry them. Internal experiments get run IDs and never public model names — several hundred training runs produced this one release, and branding is not an experiment tracker.

On what comes next. We are running this same frozen recipe on supported, Apache-2.0 bases (OLMo 3 7B and Ministral 3 8B). If the result transfers, it will ship under the appropriate Prova line — a later 7B version or the first 8B version — on a current base, and this release stands as the evidence trail behind it, including the retired-base problem it does not have. This card is not a claim that Mistral-7B-v0.3 is the right substrate; it is a record of what the recipe did on the substrate we had.

Versions are scoped per Family-Size pair: Vinci-Prova-7B-1.1 would be the next generation of this line, while Vinci-Prova-8B-1.0 would be the first of a different one.

Prova is the track for experiments, lineage tests and early public checkpoints. The recommended mainline (Piccolo, Bozza, Tela) is role-based and discloses its substrate in the card.

Citation

@misc{vinci_prova_7b_1_0,
  title  = {Vinci Prova 7B 1.0},
  author = {SimpleDirect},
  year   = {2026},
  note   = {Experimental character-training transfer study on Mistral-7B-Instruct-v0.3},
  url    = {https://huggingface.co/simpledirect/Vinci-Prova-7B-1.0}
}

To cite the study rather than the checkpoint, cite the technical report:

@techreport{pu2026character,
  title       = {Transferring Character Post-Training to Mistral 7B: Reduced model-judged fabrication, increased reticence, and capability trade-offs},
  author      = {Pu, George and Naik, Ayush},
  institution = {SimpleDirect / Vinci Research, Toronto, Canada},
  year        = {2026},
  month       = {8},
  number      = {Vinci Technical Report No. 1},
  note        = {Version 1.0; not peer reviewed},
  url         = {https://www.getsimpledirect.com/research/papers/prova-character-transfer}
}