cRia-LM-75M

Model · Text generation

cRia-LM-75M-Instruct

Shreyan Mohanty

cRia-LM-75M-Instruct is a 75.7M-parameter instruction-tuned language model built as a Relaxed Recursive Transformer (RRT). It uses a shared 11-layer recurrent block evaluated twice, with pass-specific LoRA parameters on the second traversal. The model starts from cRia-LM-75M, then adds supervised instruction tuning and preference optimization. It ships with a native chat template using and markers. cRia-LM-75M-Instruct keeps the base model's 13 unique Transformer layers: src="https://hfviewer.com/api/card.svg?source=sz14%2FcRia-LM-75M-Instruct&granularity=auto&v=20260516-title-pills-card" alt="Architecture graph for sz14/cRia-LM-75M-Instruct. Open in hfviewer" width="100%" This gives an…

Model · Text generation

Inferno-Exp-1.0

AlgoDriveAI

Inferno-Exp-1.0 is a 76.2M-parameter decoder-only base language model trained from scratch by AlgoDriveAI. It is a math-first model: its training data centers on arithmetic and math word problems, and it is geared toward the AxiomicLabs small-language-model benchmarks used in SLM challenges, ArithMark 3.0 and ArithMark 2.0 (results below). Future versions will build on this math foundation with improvements aimed at broader language understanding and logical reasoning. This is a base model (no instruction tuning). The model code ships in the repo, so load it with trustremotecode=True. All scores are zero-shot on the checkpoint in this repo. The ArithMark benchmarks were scored with…

Model · Text generation

Discovery

Convergent Intelligence

A 70M parameter causal language model built on the Mixture-of-Attentions (MoA) architecture — distance-based metric attention that respects the triangle inequality by construction, not approximation. Every attention head operates in a proper metric space. The geometry is enforced, not hoped for. Standard transformers compute attention as a dot product: Q·Kᵀ. This has no geometric meaning — it's a bilinear form, not a distance. Two tokens can be "close" by dot product while violating basic metric properties. MoA replaces this with negative squared distance under a learned diagonal Mahalanobis metric, then enforces the triangle inequality through a regularizer over random triples sampled…

This model is a fine-tuned version of distilbert/distilgpt2 on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 8 - evalbatchsize: 8 - lrschedulertype: linear - numepochs: 3.0 - Transformers 5.18.0 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.23.2

This model is a fine-tuned version of distilbert/distilgpt2 on an unknown dataset. The following hyperparameters were used during training: - learningrate: 0.0005 - trainbatchsize: 32 - evalbatchsize: 32 - gradientaccumulationsteps: 8 - totaltrainbatchsize: 256 - lrschedulertype: cosine - lrschedulerwarmupsteps: 1000 - numepochs: 1 - mixedprecisiontraining: Native AMP - Transformers 4.57.1 - Pytorch 2.8.0+cu126 - Datasets 4.0.0 - Tokenizers 0.22.1

This model is a fine-tuned version of distilbert/distilgpt2 on the None dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 4 - evalbatchsize: 4 - gradientaccumulationsteps: 4 - totaltrainbatchsize: 16 - lrschedulertype: linear - numepochs: 1 - mixedprecisiontraining: Native AMP - Transformers 5.17.0 - Pytorch 2.11.0+cu128 - Datasets 5.0.1 - Tokenizers 0.23.1