# AI Research Papers Behind Open-Weight Models Source: https://savrn.com/papers Markdown alternate of the page above; the site index is https://savrn.com/llms.txt --- SAVRN Model Hub # Research Papers The research behind the models: full abstracts, authors, publication dates, and every model in the library that cites each paper. Every model came from somewhere, and the paper is where the publisher wrote down what it did. This directory holds 254 papers, filed beside the 2,760 models and 859 datasets the hub tracks across 1,692 publishers. We keep them because a card is a summary and a paper is the record: parameter count, the context length the publisher tested at, what the model was trained on, how the evaluation was run. When we size a rack for a model, those are the pages we go back to. Start from the model you are weighing, not from the paper. The hub spans 32 tasks, with text generation the largest at 338 entries, image and text to text next at 138, and speech recognition at 78, and in every one the paper does the same job: it says what was built and how it was tested. Read it for the training setup and the evaluation conditions, then return to the model page, where GPU prices read Sep 18, 2026 turn that setup into an hourly cost in your own facility. If the paper and the card disagree on a number, write that down before you write a purchase order. 12,579Models 3,583Datasets 291Papers 7,578Publishers 12,850Sourced relationships Updated 2026-10-23 · [How the library is built](https://savrn.com/model-hub/methodology) ## Filter Papers Published - [202636](https://savrn.com/papers?year=2026) - [202536](https://savrn.com/papers?year=2025) - [202438](https://savrn.com/papers?year=2024) - [202339](https://savrn.com/papers?year=2023) - [202229](https://savrn.com/papers?year=2022) - [202142](https://savrn.com/papers?year=2021) - [202021](https://savrn.com/papers?year=2020) - [201929](https://savrn.com/papers?year=2019) - [201812](https://savrn.com/papers?year=2018) - [20176](https://savrn.com/papers?year=2017) - [20161](https://savrn.com/papers?year=2016) - [20151](https://savrn.com/papers?year=2015) - [20121](https://savrn.com/papers?year=2012) [Most recent](https://savrn.com/papers?sort=published)[Name](https://savrn.com/papers?sort=name) 291 papers, sorted by most recent. Paper ### [Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds](https://savrn.com/papers/scaling-in-games-continued-pre-training-for-embodied-agents-in-diverse-virtual-worlds) Scaling data, diversity and model capacity has improved embodied learning in physical environments, where diverse experiences remain grounded in shared physical laws. Whether these gains extend to virtual worlds with different visual styles, dynamics, controls, and objectives remains unclear. We investigate this question through a controlled study of continued pre-training (CPT) across three axes: data volume… Published 2026-10-20 11 authors Code linked [View paper](https://savrn.com/papers/scaling-in-games-continued-pre-training-for-embodied-agents-in-diverse-virtual-worlds) Paper ### [When Does Better Acceptance Pay? A Controlled Study of Speculative Decoding](https://savrn.com/papers/when-does-better-acceptance-pay-a-controlled-study-of-speculative-decoding) The throughput gains of speculative decoding depend on the trade-off between acceptance length and execution latency. We ask when additional acceptance is worth the cost of obtaining it. We establish a comparison protocol spanning training, inference, and evaluation, remeasuring released checkpoints and retraining six drafter recipes on shared target-generated data across ten benchmarks and two target models.… Published 2026-10-20 13 authors [View paper](https://savrn.com/papers/when-does-better-acceptance-pay-a-controlled-study-of-speculative-decoding) Paper ### [Towards Architectural-Level Safety Control: Relocating Hazardous Knowledge into Removable Experts in Mixture-of-Experts Models](https://savrn.com/papers/towards-architectural-level-safety-control-relocating-hazardous-knowledge-into-removable-e) Large language models acquire broad knowledge from web-scale corpora during pretraining, including knowledge that could be hazardous if misused. Removing such knowledge from the model itself without degrading general capability is desirable but difficult, as it is deeply entangled with general capability within the same parameters. Existing safety alignment and unlearning methods typically adjust these shared… Published 2026-10-19 11 authors [View paper](https://savrn.com/papers/towards-architectural-level-safety-control-relocating-hazardous-knowledge-into-removable-e) Paper ### [Memory as Weights: Internalizing Long-Term History for Streaming Videos](https://savrn.com/papers/memory-as-weights-internalizing-long-term-history-for-streaming-videos) Streaming vision-language models (VLMs) must continuously integrate information from the past while remaining responsive to current observations and user queries. Existing memory mechanisms typically compress, retrieve, or cache historical observations and reintroduce them into the input context, forcing historical memory and current observations to compete for limited context. We propose MaLoW, a streaming memory… Published 2026-10-15 5 authors Code linked [View paper](https://savrn.com/papers/memory-as-weights-internalizing-long-term-history-for-streaming-videos) Paper ### [FreeSolo: Linear-time Sparse Attention with Zero-Shot Length Extrapolation](https://savrn.com/papers/freesolo-linear-time-sparse-attention-with-zero-shot-length-extrapolation) Linear-time sequence models typically store long-context information in a recurrent state. A fixed-size state limits how much information can be stored, while hybrid architectures that add full attention for better recall incur quadratic sequence cost. Alternatively, sparse attention with continued pretraining (CPT) can extend the effective context length. Generalization beyond the trained length remains… Published 2026-10-12 2 authors Code linked [View paper](https://savrn.com/papers/freesolo-linear-time-sparse-attention-with-zero-shot-length-extrapolation) Paper ### [YANchor-4B: Effective Long-Horizon Reasoning in O(N) Time with O(1) Memory](https://savrn.com/papers/yanchor-4b-effective-long-horizon-reasoning-in-o-n-time-with-o-1-memory) Long-horizon reasoning demands access to earlier information at a manageable generation cost. Full-history attention incurs growing storage and computation, while recurrent compression can lose precise details. Therefore, we present YANchor-4B, a general-purpose recurrent model that preserves crucial memory as ANchors for retrieval during subsequent reasoning. Beyond O(N)-time generation and O(1) memory, YANchor… Published 2026-10-07 4 authors [View paper](https://savrn.com/papers/yanchor-4b-effective-long-horizon-reasoning-in-o-n-time-with-o-1-memory) Paper ### [IdeaLens: Detecting AI Ideas in Long-form Writing](https://savrn.com/papers/idealens-detecting-ai-ideas-in-long-form-writing) While modern AI detectors identify who wrote the words, emerging policies on AI use increasingly hinge on a different question: who came up with the ideas? We introduce IdeaLens, a detector that identifies whether a document's ideas came from a human or AI (idea provenance), regardless of who wrote its words. To focus IdeaLens on ideas rather than prose, we represent documents as outlines: lists of items that each… Published 2026-10-05 8 authors [View paper](https://savrn.com/papers/idealens-detecting-ai-ideas-in-long-form-writing) Paper ### [Batched Speech Decisions Without Decoding: Single-Token Supervision Lets a Frozen LLM Hear Beyond the Transcript](https://savrn.com/papers/batched-speech-decisions-without-decoding-single-token-supervision-lets-a-frozen-llm-hear) Full-duplex voice agents make many small, closed decisions, which current systems answer by slow autoregressive decoding. We propose DuplexJev, which feeds ASR-encoder hidden states through a small connector into a frozen LLM and reads each question as a single-token distribution over its options. Nothing is decoded, and an 8-GPU node answers 80 decisions about eight utterances in about 0.1 s. With a last-layer… Published 2026-10-02 7 authors [View paper](https://savrn.com/papers/batched-speech-decisions-without-decoding-single-token-supervision-lets-a-frozen-llm-hear) Paper ### [Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation](https://savrn.com/papers/beyond-teacher-assignment-domain-normalized-multi-teacher-on-policy-distillation) Reinforcement learning can turn one language model into several specialists, each excellent at a single skill such as mathematics, coding or following instructions, but users need one model with all of these skills. Multi-teacher on-policy distillation (MOPD) merges them by letting the specialists teach one student: the student answers each prompt, and the specialist for that prompt's domain gives feedback on every… Published 2026-09-28 9 authors Code linked [View paper](https://savrn.com/papers/beyond-teacher-assignment-domain-normalized-multi-teacher-on-policy-distillation) Paper ### [Why Deterministic PRM Guidance Underperforms in Discrete Diffusion Reasoning](https://savrn.com/papers/why-deterministic-prm-guidance-underperforms-in-discrete-diffusion-reasoning) Discrete diffusion language models (dLLMs) expose a denoised solution at every step, which makes process reward model (PRM) guidance look like a way to spend compute at test time. We show that once denoising, PRM scoring, and outcome reward model (ORM) scoring are charged in the same budget of forward passes, its deterministic form loses to a much simpler baseline. Our PRMs score intermediate denoising states and… Published 2026-09-28 3 authors Code linked [View paper](https://savrn.com/papers/why-deterministic-prm-guidance-underperforms-in-discrete-diffusion-reasoning) Paper ### [SMAT: Simple and Efficient Merge-Aware Training](https://savrn.com/papers/smat-simple-and-efficient-merge-aware-training) Model merging integrates the capabilities of multiple experts without joint retraining, but standard expert training optimizes task loss alone and does not guarantee good performance after merging. Merge-aware training (MAT) aims to improve merged performance, but existing methods do not fully account for common merging operations and add training cost. We observe that, from an expert's perspective, common merging… Published 2026-09-27 8 authors Code linked [View paper](https://savrn.com/papers/smat-simple-and-efficient-merge-aware-training) Paper ### [Region-Level Policy Optimization for Fine-grained MLLM Perception](https://savrn.com/papers/region-level-policy-optimization-for-fine-grained-mllm-perception) Fine-grained visual perception in MLLMs is commonly improved by raising the resolution, but the added visual tokens inflate vision-encoding and language-model prefilling costs. We show that the two operations underlying fine-grained perception, localizing the region of interest (RoI) and recognizing its content, have different resolution requirements. In a controlled diagnostic, localization tolerates roughly 3 to 4… Published 2026-09-17 4 authors Code linked [View paper](https://savrn.com/papers/region-level-policy-optimization-for-fine-grained-mllm-perception) Paper ### [dQwen3.5: Hybrid-Attention Diffusion Language Models](https://savrn.com/papers/dqwen3-5-hybrid-attention-diffusion-language-models) Adapting a pretrained autoregressive (AR) model is a cost-efficient route to a diffusion language model (DLM). While nearly all such adaptations start from a full-attention transformer, AR modeling has shifted toward hybrid architectures that interleave attention and RNN layers. This creates an obstacle for adaptation: unlike attention, RNNs are structurally causal and nontrivial to bidirectionalize. Despite this… Published 2026-09-17 6 authors [View paper](https://savrn.com/papers/dqwen3-5-hybrid-attention-diffusion-language-models) Paper ### [UniMate: One Unified Model to Animate Diverse Skeletons](https://savrn.com/papers/unimate-one-unified-model-to-animate-diverse-skeletons) Recent advances in automatic rigging now deliver animation-ready 3D assets at scale, yet generating the motion to drive them remains a bottleneck. Existing learned animators are topology-constrained: they rely on category-specific templates or require per-skeleton fine-tuning and reference motions at inference. We present UniMate, a unified foundation model that synthesizes articulated motion for arbitrary skeletons… Published 2026-09-04 7 authors Code linked [View paper](https://savrn.com/papers/unimate-one-unified-model-to-animate-diverse-skeletons) Paper ### [Mitigating Bias in Locally Constrained Decoding via Tractable Proposals](https://savrn.com/papers/mitigating-bias-in-locally-constrained-decoding-via-tractable-proposals) Generations from large language models often fail to conform to desired constraints such as JSON schema. Existing locally constrained decoding (LCD) approaches enforce constraints by myopically masking out next tokens, resulting in biased sampling and degradation in performance. Recent work uses sequential Monte Carlo (SMC) methods to mitigate such biases, but designing effective proposal distributions or potential… Published 2026-08-16 6 authors Code linked [View paper](https://savrn.com/papers/mitigating-bias-in-locally-constrained-decoding-via-tractable-proposals) Paper ### [Gemma 4 Technical Report](https://savrn.com/papers/gemma-4-technical-report) We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemma 4 model suite features dense and Mixture-of-Experts architectures, ranging from 2.3B to 31B parameters. Alongside improved vision and audio encoders for all model sizes, we propose a unified, encoder-free architecture for our 12B model… Published 2026-07-02 25 authors [View paper](https://savrn.com/papers/gemma-4-technical-report) Paper ### [Disentangling Geometry, Performance, and Training in Language Models](https://savrn.com/papers/disentangling-geometry-performance-and-training-in-language-models) Geometric properties of Transformer weights, particularly the unembedding matrix, have been widely useful in language model interpretability research. Yet, their utility for estimating downstream performance remains unclear. In this work, we systematically investigate the relationship between model performance and the unembedding matrix geometry, particularly its effective rank. Our experiments, involving a suite of… Published 2026-06-21 4 authors [View paper](https://savrn.com/papers/disentangling-geometry-performance-and-training-in-language-models) Paper ### [PP-OCRv6: From 1.5M to 34.5M Parameters, Surpassing Billion-Scale VLMs on OCR Tasks](https://savrn.com/papers/pp-ocrv6-from-1-5m-to-34-5m-parameters-surpassing-billion-scale-vlms-on-ocr-tasks) Vision-Language Models (VLMs) have achieved impressive results on general vision-language tasks, yet they suffer from hallucination, imprecise localization, and prohibitive computational cost when applied to dedicated OCR scenarios. This paper presents PP-OCRv6, a lightweight OCR system that combines architectural innovation with data-centric optimization. PP-OCRv6 redesigns the backbone, detection neck, and… Published 2026-06-11 16 authors [View paper](https://savrn.com/papers/pp-ocrv6-from-1-5m-to-34-5m-parameters-surpassing-billion-scale-vlms-on-ocr-tasks) Paper ### [Toto 2.0: Time Series Forecasting Enters the Scaling Era](https://savrn.com/papers/toto-2-0-time-series-forecasting-enters-the-scaling-era) We show that time series foundation models scale: a single training recipe produces reliable forecast-quality improvements from 4M to 2.5B parameters. We release Toto 2.0, a family of five open-weights forecasting models trained under this recipe. The Toto 2.0 family sets a new state of the art on three forecasting benchmarks: BOOM, our observability benchmark; GIFT-Eval, the standard general-purpose benchmark; and… Published 2026-05-19 13 authors Code linked [View paper](https://savrn.com/papers/toto-2-0-time-series-forecasting-enters-the-scaling-era) Paper ### [When to Think Fast and Slow? AMOR: Adaptive Entropy Gate for Hybrid Models](https://savrn.com/papers/when-to-think-fast-and-slow-amor-adaptive-entropy-gate-for-hybrid-models) Recurrent-attention hybrids aim to combine the efficiency of recurrence with the expressivity of attention, but existing approaches typically apply attention uniformly across all positions, even when the recurrent state alone is sufficient for accurate prediction. We introduce AMOR (Adaptive Metacognitive Output Router), a post-hoc hybrid architecture that selectively invokes attention based on predictive… Published 2026-05-13 2 authors [View paper](https://savrn.com/papers/when-to-think-fast-and-slow-amor-adaptive-entropy-gate-for-hybrid-models) Paper ### [Model Compression with Exact Budget Constraints via Riemannian Manifolds](https://savrn.com/papers/model-compression-with-exact-budget-constraints-via-riemannian-manifolds) Assigning one of K options to each of N groups under a total cost budget is a recurring problem in efficient AI, including mixed-precision quantization, non-uniform pruning, and expert selection. The objective, typically model loss, depends jointly on all assignments and does not decompose across groups, preventing combinatorial solvers from directly optimizing the true objective and forcing reliance on proxy… Published 2026-05-07 2 authors [View paper](https://savrn.com/papers/model-compression-with-exact-budget-constraints-via-riemannian-manifolds) Paper ### [MolmoAct2: Action Reasoning Models for Real-world Deployment](https://savrn.com/papers/molmoact2-action-reasoning-models-for-real-world-deployment) Vision-Language-Action (VLA) models aim to provide a single generalist controller for robots, but today's systems fall short on the criteria that matter for real-world deployment. Frontier models are closed, open-weight alternatives are tied to expensive hardware, reasoning-augmented policies pay prohibitive latency for their grounding, and fine-tuned success rates remain below the threshold for dependable use. We… Published 2026-05-04 25 authors Code linked [View paper](https://savrn.com/papers/molmoact2-action-reasoning-models-for-real-world-deployment) Paper ### [Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence](https://savrn.com/papers/nemotron-3-nano-omni-efficient-and-open-multimodal-intelligence) We introduce Nemotron 3 Nano Omni, the latest model in the Nemotron multimodal series and the first to natively support audio inputs alongside text, images, and video. Nemotron 3 Nano Omni delivers consistent accuracy improvements over its predecessor, Nemotron Nano V2 VL, across all modalities, enabled by advances in architecture, training data and recipes. In particular, Nemotron 3 delivers leading results in… Published 2026-04-27 25 authors [View paper](https://savrn.com/papers/nemotron-3-nano-omni-efficient-and-open-multimodal-intelligence) Paper ### [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://savrn.com/papers/deepseek-v4-towards-highly-efficient-million-token-context-intelligence) We present a preview version of DeepSeek-V4 series, including two strong Mixture-of-Experts (MoE) language models -- DeepSeek-V4-Pro with 1.6T parameters (49B activated) and DeepSeek-V4-Flash with 284B parameters (13B activated) -- both supporting a context length of one million tokens. DeepSeek-V4 series incorporate several key upgrades in architecture and optimization: (1) a hybrid attention architecture that… Published 2026-04-26 25 authors [View paper](https://savrn.com/papers/deepseek-v4-towards-highly-efficient-million-token-context-intelligence) Explore more papers ## Model Collections Hand-picked starting points, each with the reason it exists. Collection · 4 entries ### [Embedding models for retrieval](https://savrn.com/collections/embedding-models-for-retrieval) Sentence and document embedding models used to build retrieval systems. Dimension and sequence length matter more than size here, and both come from the publisher. [View collection](https://savrn.com/collections/embedding-models-for-retrieval) Collection · 4 entries ### [Models that fit on one accelerator](https://savrn.com/collections/models-that-run-on-one-accelerator) Models whose publisher-reported parameter count puts them within reach of a single accelerator at common precisions. Memory needed depends on precision and serving configuration, so treat the parameter count as the starting point, not the answer. [View collection](https://savrn.com/collections/models-that-run-on-one-accelerator) Collection · 6 entries ### [Open-weight text models worth knowing](https://savrn.com/collections/open-weight-text-models-worth-knowing) Widely used open-weight language models, chosen because each one is a distinct family rather than a variant of the one above it. Selection, not a ranking. [View collection](https://savrn.com/collections/open-weight-text-models-worth-knowing) Collection · 3 entries ### [Speech and audio models](https://savrn.com/collections/speech-and-audio) Recognition and synthesis models, grouped so the two directions are easy to compare. [View collection](https://savrn.com/collections/speech-and-audio) ## Related SAVRN Research The hub sits beside SAVRN's market data and infrastructure research: what models cost to run, and what it takes to run them. SAVRN Index ### [What open models cost to run](https://savrn.com/ai-index/pricing/open-models) The same open-weight model priced by every host that serves it, per million tokens. Research Hub ### [Data center trackers and maps](https://savrn.com/data-center-research) Moratoriums, permits, power, water and capital behind the facilities that run these models. Method ### [How the Model Hub is built](https://savrn.com/model-hub/methodology) Sources, evidence labels, refresh behaviour, and the limits of every comparison here.