luna-2.1-base

A 114,114,048-parameter language model trained from scratch on a single 8 GB laptop This is a base model. It continues text; it does not follow instructions or answer questions. At this size it writes fluent, on-topic prose that is often factually wrong. It is published as a baseline and as a record of what one laptop can train in a weekend, not as something to rely on. Validation loss on held-out FineWeb-Edu text, and on held-out code files deduplicated by exact hash against the training code. \ The code loss is flattered by the tokenizer. GPT-2's BPE splits indentation whitespace, against 2.0% for text. Those are easy to predict and pull the average down. The same effect makes greedy code…

DeepNarra AI Ecosystem — Sovereign Frontier Architecture Apex Tri-Hybrid Base Foundation Model DeepNarra Hiraya-1-Base is an official release in the sovereign DeepNarra ecosystem. 1. Two-Tier Hierarchical MoE (H-MoE): 92 Fine-Grained Sub-Experts per layer with active Shared Foundation Expert active on every token for baseline coherence. 2. Coconut Continuous Latent Deliberation: Continuous thought phase space ($\mathbb{R}^{768}$) enabling internal chain-of-thought progression prior to verbal token emissions. 3. Symplectic Störmer-Verlet Physics: Hamiltonian energy-conserving dynamics for stable long-horizon reasoning trajectories. 4. Hardware-Efficient Recurrence: Chunk-wise Gated Linear…

Bayon is a 100M-parameter decoder-only generative language model pretrained from scratch for Khmer (km). The model was designed specifically for Khmer rather than being adapted from an existing multilingual or English-focused pretrained model. Its architecture and tokenizer were developed with Khmer text generation as the primary target. Bayon serves as the base pretrained model for Bayon Instruct. Research into language-specific model and tokenizer design Studying efficient language modeling for low-resource languages Bayon is a base pretrained model, not an instruction-tuned assistant. It may therefore produce continuations rather than direct answers when given natural-language questions…

Bayon Instruct is an instruction-tuned version of Bayon, a 100M-parameter decoder-only language model designed specifically for Khmer. Bayon was pretrained from scratch using a custom 5,000-token Khmer BPE tokenizer and subsequently adapted for instruction following with LoRA. The instruction-tuning data consists of 18,000 Gemini-distilled, Khmer-focused SFT examples. The model is intended primarily for Khmer text generation and instruction-following tasks, particularly where maintaining Khmer-language output is important. The tokenizer retains byte fallback, although byte fallback was not observed in the evaluation described in the associated research. Research on language-specific and…

Model · Text generation

lightning-105m

AobanZ

Lightning is a small, autoregressive transformer which utilizes FlashAttention and MHA. This model is trained on a variety of books from a dataset(300 MB). This is the expanded version of Lightning-60m. Lightning utilizes FlashAttention and AdamW for performance and capability. Lightning is designed to provide quick, coherent outputs, improved with a larger size and weight. Lightning is intended to be used for research, analysis and fine-tuning, stories and other. It is not intended to be used for professional advice, real writing or any kind of heavy work as generated outputs may be incorrect. Lightning can be used directly for text generation, experimentation, and conversational…

This model is a fine-tuned version of GeorgeUwaifo/iviegpt2new01cresults on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 4 - evalbatchsize: 8 - gradientaccumulationsteps: 4 - totaltrainbatchsize: 16 - lrschedulertype: linear - lrschedulerwarmupsteps: 387 - numepochs: 5 - mixedprecisiontraining: Native AMP - Transformers 5.16.1 - Pytorch 2.11.0+cu128 - Datasets 4.8.5 - Tokenizers 0.23.1