SAVRN Model Tracker / Methodology
Trace the claim.
Understand the limits.
The tracker records what changed at a source, when we observed it, and what the evidence supports. Authority comes from a record readers can inspect.
← Return to the Model TrackerMethodology updated: .
01 / Scope
A defined watchlist, with visible gaps.
The tracker monitors configured sources and the model records those collections actually observe. It does not claim exhaustive coverage of all models, repositories, providers, or announcements. The source table records expected collection cadence, attempts, successful checks, and coverage limits.
A successful retrieval establishes that a source was checked. It does not independently verify its statements. When a check fails, the last successful observation remains available with its original timestamp. A source with no successful check is shown as awaiting its first check. Unavailable repositories contribute no new update or blog claim until usable evidence is retrieved. Their coverage gaps remain counted while collection retries later; a successful batch does not imply that every enrolled repository is current.
Discovery and continued monitoring are different. Initial observations establish a baseline. A model's first appearance here is not evidence of a new release, and its repository creation date is not automatically its release date.
02 / Classification
Record the difference before interpreting it.
A repository revision means an observed revision identifier changed. That may reflect documentation, configuration, licensing, or model artifacts. It does not by itself prove that weights changed or that a model improved.
A changed official source page is a source-page change. A change in a page's navigation or formatting does not become a model announcement. A release, deprecation, benchmark result, price change, or license change needs evidence supporting that specific classification.
The source publication date and SAVRN observation time are recorded separately. When a source does not provide a publication date, it remains unknown. Observation time is not substituted for release time.
We also record when SAVRN publishes each new observation to the ledger. The tracker’s Content updated date, structured-data dateModified, and sitemap lastmod use the same substantive publication record: a newly established baseline, a changed observation, or a reviewed recommendation, correction, retraction, or scheduled expiry. Sources last checked reports collection activity separately. A retrieval that leaves observations and displayed recommendations unchanged, a failed check, opening this page, and redeploying the site do not advance the content date. A recommendation becomes unavailable at its recorded evidence-freshness deadline or editorial expiry, whichever comes first. Restoring it after a monitoring gap records when it becomes current again; routine checks completed before the deadline do not create a new content date. Older records without a known SAVRN publication time keep that date unknown.
Descriptions and before-and-after values are evidence to inspect. A publisher's claim of improved capability retains its attribution. A benchmark score only supports comparisons when task, dataset version and split, metric, model revision, prompting, tools, inference budget, and evaluation setup are sufficiently matched and disclosed.
03 / Evidence
What each label means.
- Observed at source
- SAVRN recorded a difference at the linked source. The label establishes an observation, not an independently measured improvement.
- Publisher reported
- The model publisher or official source made the statement. Recording it is not independent verification.
- Reported by a third party
- A named outside source reported it. The reporter and source remain attached to the claim.
- SAVRN reviewed
- A documented SAVRN review checked a claim against cited evidence. Reviewing a publisher's assertion does not turn it into a SAVRN measurement.
- SAVRN measured
- SAVRN ran the evaluation and recorded the revision, configuration, method, and results needed to assess it.
- Evidence not established
- The record does not establish the claim. Missing facts are not estimated or promoted into recommendations.
Evidence provenance and editorial review are separate. A reviewed event may still describe a publisher-reported result.
04 / Recommendations
Useful for a task. Supported by evidence.
A recommendation needs an exact model and revision, a defined task, material constraints, supporting evidence, a rationale, a named reviewer, a review date, and an expiry date. Expired recommendations are excluded from the current recommendations list.
For university research, a sound selection also considers institutional policy, data location and retention, license terms, reproducibility, task-specific evaluation, and cost. Model popularity and download counts alone do not establish quality or suitability.
A provider's suggested replacement is a provider recommendation until SAVRN reviews the relevant evidence and use context. Where no current recommendation satisfies the publication requirements, the tracker says so plainly.
05 / Corrections
An inspectable record can be corrected.
If a model identity, date, source attribution, license, or interpretation is wrong, send SAVRN the record and a supporting source. A correction should identify what was wrong, the supporting evidence, and the reason for the change. A later source update is not itself proof that an earlier observation was false.
The Model Hub methodology explains the catalog's publication, version, and comparison rules. The tracker adds a chronological record of observed changes.
Coverage, in the open
The sources behind the tracker
Check times describe successful retrieval, not independent verification of every claim. A repository-source check can cover one batch; model coverage is reported separately on the tracker. Expected cadence is a collection target; the last successful check shows what actually happened. All times are UTC.
| Source | Check status | Last successful check | Last attempt | Expected cadence |
|---|---|---|---|---|
| Anthropic model deprecations ↗ | Checked on schedule | 20 Sep 2026, 03:19 UTC | Hourly | |
| Anthropic platform release notes ↗ | Checked on schedule | 20 Sep 2026, 03:19 UTC | Hourly | |
| Gemini API changelog ↗ | Checked on schedule | 20 Sep 2026, 03:19 UTC | Hourly | |
| Gemini API deprecations ↗ | Checked on schedule | 20 Sep 2026, 03:19 UTC | Hourly | |
| Hugging Face enrolled model repositories ↗ | Partial coverage4 repositories awaiting usable evidence | 20 Sep 2026, 03:46 UTC | Every 15 minutes | |
| Mistral changelog ↗ | Checked on schedule | 20 Sep 2026, 03:19 UTC | Hourly | |
| OpenAI API changelog ↗ | Checked on schedule | 20 Sep 2026, 03:19 UTC | Hourly | |
| OpenAI deprecations ↗ | Checked on schedule | 20 Sep 2026, 03:19 UTC | Hourly |
The model count reflects observed records within this scope. Source failures do not erase existing observations or move their successful-check timestamps forward. Read the full method →