https://github.com/jzhang38/TinyLlama The TinyLlama project aims to pretrain a 1.1B Llama model on 3 trillion tokens. With some proper optimization, we can achieve this within a span of "just" 90 days using 16 A100-40G GPUs. The training has started on 2023-09-01. We adopted exactly the same architecture and tokenizer as Llama 2. This means TinyLlama can be plugged and played in many open-source projects built upon Llama. Besides, TinyLlama is compact with only 1.1B parameters. This compactness allows it to cater to a multitude of applications demanding a restricted computation and memory footprint. This is the chat model finetuned on top of…
Open weights
apache-2.0
1.1B parameters
2,048 tokens
transformers
My mathematical formulation to utilize space projections to "measure" the Jump between points of discontinuity found in Non-Differentialable Functions. This Model underwent an additional merge between Qemma-redux and Qwen3-14B, in addition to adding Rope Scaling. Fusion Logic was updated to aid per layer fusion and post fusion embedding alignment. Qemma is a HuggingFace-native hybrid model that merges Gemma-3 (1B) and Qwen-3 (14B) at the weight level (no adapters). This variant uses Yarn based Rope Scaling with 1: Ratio from maxpositionembeddings = 524288 Gemma-3 backbone (26 layers, hidden 1152, MLP 6912) Qwen-style attention regrouped to Gemma’s 4×256 heads. (headdim=128, hidden=5120…
Open weights
osl-3.0
1B parameters
524,288 tokens
transformers
My mathematical formulation to utilize space projections to "measure" the Jump between points of discontinuity found in Non-Differentialable Functions. This Model underwent an additional merge between Qemma-redux and Qwen3-1.7B, in addition to adding Rope Scaling. Fusion Logic was updated to aid per layer fusion and post fusion embedding alignment. Qemma is a HuggingFace-native hybrid model that merges Gemma-3 (1B) and Qwen-3 (1.7B) at the weight level (no adapters). This variant uses Yarn based Rope Scaling with 1: Ratio from maxpositionembeddings = 242144 Gemma-3 backbone (26 layers, hidden 1152, MLP 6912) Qwen-style attention regrouped to Gemma’s 4×256 heads. (headdim=128, hidden=2048…
Open weights
osl-3.0
1B parameters
262,144 tokens
transformers
My mathematical formulation to utilize space projections to "measure" the Jump between points of discontinuity found in Non-Differentialable Functions. This Model underwent an additional merge between Qemma-redux and Qwen3-0.6B, in addition to adding Rope Scaling. Fusion Logic was updated to aid per layer fusion and post fusion embedding alignment. Qemma is a HuggingFace-native hybrid model that merges Gemma-3 (1B) and Qwen-3 (0.6B) at the weight level (no adapters). This variant uses Yarn based Rope Scaling with 1:1 Ratio from maxpositionembeddings Use: research, instruction following, code/help, analysis, further SFT/RLHF. Limits: may hallucinate; not for safety-critical, medical, legal…
Open weights
osl-3.0
1B parameters
131,072 tokens
transformers
Model · Text generation
Google
[Gemma 3 Technical Report][g3-tech-report] [Responsible Generative AI Toolkit][rai-toolkit] [Gemma on Kaggle][kaggle-gemma] [Gemma on Vertex Model Garden][vertex-mg-gemma3] Summary description and brief definition of inputs and outputs. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. Gemma 3 models are multimodal, handling text and image input and generating text output, with open weights for both pre-trained variants and instruction-tuned variants. Gemma 3 has a large, 128K context window, multilingual support in over 140 languages, and is available in more sizes than previous…
Access requested at publisher
gemma
1B parameters
transformers
Model · Text generation
Lamapi
Next-1B is a 1-billion parameter causal language model based on Gemma 3, designed for efficiency, low-resource deployment, and reasoning-focused natural language understanding. Extremely lightweight — can run on consumer GPUs with low VRAM. Optimized for text reasoning, summarization, and creative generation. Supports Turkish natively while remaining multilingual. Open-source and transparent for research and applications. Ideal for developers, students, and organizations needing fast, reliable, and low-resource text-generation. 1. Lightweight Efficiency: Run smoothly on low-resource devices. 2. Reasoning-Focused: Provide logical and coherent text outputs. 3. Accessibility: Fully open-source…
Open weights
mit
1B parameters
32,768 tokens
transformers
