# cRia-LM-75M by Shreyan Mohanty: Open-Weight Model Source: https://savrn.com/models/cria-lm-75m Markdown alternate of the page above; the site index is https://savrn.com/llms.txt --- ## Runs On What it takes to serve cRia-LM-75M (76M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them. | Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits | | --- | --- | --- | --- | --- | --- | | 16-bit | 0.2 GB | 0.2 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | | 8-bit | 0.1 GB | 0.1 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | | 4-bit | 0.0 GB | 0.0 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the [SAVRN Index](https://savrn.com/ai-index/pricing/gpus), read Oct 24, 2026. [cRia-LM-75M on every accelerator the SAVRN Index prices, at every precision](https://savrn.com/models/cria-lm-75m/gpus) ## Model Card By Shreyan Mohanty, published under apache-2.0, revision a424901d152c. Cria (noun): a baby llama, alpaca, vicuña, or guanaco. Pronounced ˈkrē-ə. ~ Merriam-Webster cRia-LM-75M is a 75.7M-parameter base language model built as a Relaxed Recursive Transformer (RRT). It uses a shared 11-layer recurrent block evaluated twice, with pass-specific LoRA parameters on the second traversal. Training was carried out in three stages. Stage 1 established the 2K base model over 10B tokens. Stage 2 continued training with a 2B-token budget and a capability-focused data curriculum; the released Stage 2 checkpoint is step 10,000, corresponding to about 1.31B continuation tokens. Stage 3 extended the context window from 2,048 to 4,096 tokens with a 50M-token run on codelion/sutra-1B. The released checkpoint continues from that long-context stage with 35 learned residual-gain scalars and one epoch over the HellaSwag, ARC-Easy, ARC-Challenge, PIQA, SciQ and SWAG training splits. [Read the full model card (2,014 words)](https://savrn.com/models/cria-lm-75m/card) ## Configuration Architecture RRTForCausalLM Context length (tokens) 4,096 Layers 13 Hidden size 576 Attention heads 9 Key/value heads 3 Head dimension 64 Vocabulary size 49,152 RoPE base 100000 Model type rrt ## Identity and Version Repository sz14/cRia-LM-75M Publisher Shreyan Mohanty Task Text generation Modality Text Library transformers Parameters 76M parameters Languages en Revision a424901d152c9275eee735101b1e5054b3f22f55 First published 2026-08-20 Last updated 2026-09-19 ## Files and Weights 15 files, 306.7 MB in total. The weights are 1 file totalling 302.9 MB in safetensors. Weights1 file · 302.9 MB Configuration8 files · 83.8 KB Tokenizer2 files · 3.5 MB Documentation2 files · 21.6 KB Other1 file · 122.4 KB Repository1 file · 97 B Every file | File | Type | Size | SHA-256 | | --- | --- | --- | --- | | model.safetensors | Weights | 302.9 MB | 28ef0d085a1f | | benchmark_results.json | Configuration | 1.9 KB | — | | calculate_axiomic_index.py | Configuration | 2.5 KB | — | | config.json | Configuration | 1.5 KB | — | | evaluate_arithmark3.py | Configuration | 6.5 KB | — | | generation_config.json | Configuration | 215 B | — | | modeling_rrt.py | Configuration | 55.0 KB | — | | residual_gain_training_report.json | Configuration | 15.7 KB | — | | run_phase3_lm_eval.py | Configuration | 466 B | — | | README.md | Documentation | 17.1 KB | — | | generation_comparison.md | Documentation | 4.5 KB | — | | banner.png | Other | 122.4 KB | d037c8da3a83 | | .gitattributes | Repository | 97 B | — | | tokenizer.json | Tokenizer | 3.5 MB | — | | tokenizer_config.json | Tokenizer | 752 B | — | ## License and Download License apache-2.0 Access Open weights, no gate Download size 302.9 MB [Download from Shreyan Mohanty](https://huggingface.co/sz14/cRia-LM-75M) Released by Shreyan Mohanty through its official repository on Hugging Face. [Read the license](https://www.apache.org/licenses/LICENSE-2.0). ## Built From - Described by arXiv:2410.20672 - Described by arXiv:2411.09009 - Described by arXiv:2605.21699 - Trained on (disclosed) [HuggingFaceFW/fineweb-edu](https://savrn.com/datasets/fineweb-edu) - Trained on (disclosed) HuggingFaceTB/cosmopedia - Trained on (disclosed) HuggingFaceTB/dclm-edu - Trained on (disclosed) HuggingFaceTB/finemath - Trained on (disclosed) HuggingFaceTB/smollm-corpus - Trained on (disclosed) bigcode/starcoderdata - Trained on (disclosed) codelion/sutra-1B - Trained on (disclosed) [epfml/FineWeb-HQ](https://savrn.com/datasets/fineweb-hq) - Trained on (disclosed) nvidia/Nemotron-CC-Math-v1 - Trained on (disclosed) [wikimedia/wikipedia](https://savrn.com/datasets/wikipedia) ## Memory Requirements | Precision | Weights in memory | | --- | --- | | As published | 302.9 MB | | 16-bit | 0.2 GB | | 8-bit | 0.1 GB | | 4-bit | 0.0 GB | Weights only, from the published parameter count; the key-value cache and runtime add to this. ## Built on This Model - Derived from[cRia-LM-75M-Instruct](https://savrn.com/models/cria-lm-75m-instruct) ## Questions About cRia-LM-75M ### How much GPU memory does cRia-LM-75M need? About 0.2 GB at 16-bit and 0 GB at 4-bit: the weights (76M parameters) plus a working margin. A long context needs more. ### What is the cheapest GPU to run cRia-LM-75M on? At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists. ### Can I use cRia-LM-75M commercially? Yes. cRia-LM-75M is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors. ### What is cRia-LM-75M's context length? 4,096 tokens, from the maximum position embeddings in its published configuration. ## Similar Models Model · Text generation ### [cRia-LM-75M-Instruct](https://savrn.com/models/cria-lm-75m-instruct) [Shreyan Mohanty](https://savrn.com/model-publishers/sz14) cRia-LM-75M-Instruct is a 75.7M-parameter instruction-tuned language model built as a Relaxed Recursive Transformer (RRT). It uses a shared 11-layer recurrent block evaluated twice, with pass-specific LoRA parameters on the second traversal. The model starts from cRia-LM-75M, then adds supervised instruction tuning and preference optimization. It ships with a native chat template using and markers. cRia-LM-75M-Instruct keeps the base model's 13 unique Transformer layers: src="https://hfviewer.com/api/card.svg?source=sz14%2FcRia-LM-75M-Instruct&granularity=auto&v=20260516-title-pills-card" alt="Architecture graph for sz14/cRia-LM-75M-Instruct. Open in hfviewer" width="100%" This gives an… Open weights apache-2.0 76M parameters 4,096 tokens transformers [View model](https://savrn.com/models/cria-lm-75m-instruct) Model · Text generation ### [Inferno-Exp-1.0](https://savrn.com/models/inferno-exp-1-0) [AlgoDriveAI](https://savrn.com/model-publishers/algodriveai) Inferno-Exp-1.0 is a 76.2M-parameter decoder-only base language model trained from scratch by AlgoDriveAI. It is a math-first model: its training data centers on arithmetic and math word problems, and it is geared toward the AxiomicLabs small-language-model benchmarks used in SLM challenges, ArithMark 3.0 and ArithMark 2.0 (results below). Future versions will build on this math foundation with improvements aimed at broader language understanding and logical reasoning. This is a base model (no instruction tuning). The model code ships in the repo, so load it with trustremotecode=True. All scores are zero-shot on the checkpoint in this repo. The ArithMark benchmarks were scored with… Open weights apache-2.0 76M parameters 2,048 tokens transformers [View model](https://savrn.com/models/inferno-exp-1-0) Model · Text generation ### [Discovery](https://savrn.com/models/discovery) [Convergent Intelligence](https://savrn.com/model-publishers/reaperdoesntknow) A 70M parameter causal language model built on the Mixture-of-Attentions (MoA) architecture — distance-based metric attention that respects the triangle inequality by construction, not approximation. Every attention head operates in a proper metric space. The geometry is enforced, not hoped for. Standard transformers compute attention as a dot product: Q·Kᵀ. This has no geometric meaning — it's a bilinear form, not a distance. Two tokens can be "close" by dot product while violating basic metric properties. MoA replaces this with negative squared distance under a learned diagonal Mahalanobis metric, then enforces the triangle inequality through a regularizer over random triples sampled… Open weights cc 71M parameters 2,048 tokens transformers [View model](https://savrn.com/models/discovery) Model · Text generation ### [my_awesome_eli5_clm-model](https://savrn.com/models/my-awesome-eli5-clm-model) [Bornil Phukon](https://savrn.com/model-publishers/bornil20005) This model is a fine-tuned version of distilbert/distilgpt2 on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 8 - evalbatchsize: 8 - lrschedulertype: linear - numepochs: 3.0 - Transformers 5.18.0 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.23.2 Open weights apache-2.0 82M parameters transformers [View model](https://savrn.com/models/my-awesome-eli5-clm-model) Model · Text generation ### [codeparrot-ds](https://savrn.com/models/codeparrot-ds) [Ai Xin](https://savrn.com/model-publishers/aixin2024) This model is a fine-tuned version of distilbert/distilgpt2 on an unknown dataset. The following hyperparameters were used during training: - learningrate: 0.0005 - trainbatchsize: 32 - evalbatchsize: 32 - gradientaccumulationsteps: 8 - totaltrainbatchsize: 256 - lrschedulertype: cosine - lrschedulerwarmupsteps: 1000 - numepochs: 1 - mixedprecisiontraining: Native AMP - Transformers 4.57.1 - Pytorch 2.8.0+cu126 - Datasets 4.0.0 - Tokenizers 0.22.1 Open weights apache-2.0 82M parameters transformers [View model](https://savrn.com/models/codeparrot-ds) Model · Text generation ### [rick-morty-distilgpt2](https://savrn.com/models/rick-morty-distilgpt2) [Zune toka](https://savrn.com/model-publishers/hfzune) This model is a fine-tuned version of distilbert/distilgpt2 on the None dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 4 - evalbatchsize: 4 - gradientaccumulationsteps: 4 - totaltrainbatchsize: 16 - lrschedulertype: linear - numepochs: 1 - mixedprecisiontraining: Native AMP - Transformers 5.17.0 - Pytorch 2.11.0+cu128 - Datasets 5.0.1 - Tokenizers 0.23.1 Open weights apache-2.0 82M parameters transformers [View model](https://savrn.com/models/rick-morty-distilgpt2) ## Shreyan Mohanty [All models and datasets](https://savrn.com/model-publishers/sz14) ## Versions - [a424901d152c](https://savrn.com/models/cria-lm-75m/versions/a424901d152c) · current 2026-09-19 ## Explore More - [All text generation models](https://savrn.com/models/tasks/text-generation) - [All models under apache-2.0](https://savrn.com/models/licenses/apache-2-0) - [Model comparisons](https://savrn.com/models/comparisons) - [The model directory](https://savrn.com/models) - [Open model prices by host](https://savrn.com/ai-index/pricing/open-models) ## Source - Repository metadata, read 2026-10-21. - [Hugging Face record](https://huggingface.co/sz14/cRia-LM-75M) - [How the hub is built](https://savrn.com/model-hub/methodology)