The Llama 3.2 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction-tuned generative models in 1B and 3B sizes (text in/text out). The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks. They outperform many of the available open source and closed chat models on common industry benchmarks. Model Architecture: Llama 3.2 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for…
Access requested at publisher
llama3.2
3.2B parameters
transformers
This is a MarinSkyRL-native Open-MOPD student after 32 optimizer steps. It starts from the authors' mixed-domain SFT checkpoint. Student responses were scored by the authors' math, code, and instruction-following RL teachers, routed by domain. The objective uses the student's selected top-16 token IDs and a clipped policy surrogate. This is an early checkpoint, not the authors' step-200 final model. The checkpoint is an unquantized, six-file Hugging Face export of the durable MarinSkyRL globalstep32 FSDP2 checkpoint. The policy export was used for the independent step-32 evaluation. The export's model.safetensors SHA-256 is bb7326640142069bc2e1fba5f54f15e0cccb1ff861f34f318b372eaab7abaf4b.…
Open weights
apache-2.0
3.3B parameters
65,536 tokens
transformers
PowerMoE-3B is a 3B sparse Mixture-of-Experts (sMoE) language model trained with the Power learning rate scheduler. It sparsely activates 800M parameters for each token. It is trained on a mix of open-source and proprietary datasets. PowerMoE-3B has shown promising results compared to other dense models with 2x activate parameters across various benchmarks, including natural language multi-choices, code generation, and math reasoning. This is a simple example of how to use PowerMoE-3b model.
Open weights
apache-2.0
3.4B parameters
4,096 tokens
transformers
An English LFM2.5 2.6B checkpoint associated with the Trendyol Cybersecurity Instruction Tuning Dataset. This repository contains both Transformers-format model files and local GGUF exports. This is a research release. The repository does not currently publish benchmark, baseline-comparison, or safety-evaluation results. Transformers assets include model.safetensors, configuration files, tokenizer files, and a chat template. Keeping both formats in one repository is convenient, but users should explicitly choose the path that matches their runtime. The checked configuration includes a bitsandbytes quantization block. Pin and test the exact Transformers, Accelerate, bitsandbytes, and device…
Open weights
apache-2.0
2.7B parameters
131,072 tokens
transformers
Bonsai 2 27B by Prism ML, repacked for vLLM. Unofficial; not affiliated with Prism ML. It needs the prismternary vLLM plugin from fraserprice/bonsai-vllm, which has the run command, kernels and throughput numbers: Built and tested for the RTX PRO 6000 Blackwell only; other NVIDIA GPUs are untested. Problems: open an issue. - The ternary weights of the MLX pack, bit for bit: the same 2-bit codes and FP16 group scales (g128), in the same Hadamard-rotated basis. MLX's redundant per-group biases (-scale) are dropped, and the rotation signs move into config.json. - The embedding table is dequantized to BF16. - Norms and the linear-attention state path are Prism ML's own tensors from the MLX…
Open weights
apache-2.0
3.5B parameters
262,144 tokens
vllm
SymbioticLM is a hybrid symbolic–neural language model that integrates a frozen transformer backbone (Qwen2ForCausalLM) with a suite of symbolic cognitive modules for adaptive, interpretable reasoning. The architecture fuses neural token-level generation with symbolic introspection and reasoning: - Dynamic Thought Evolution with Helical Encoding and DNA-Inspired Memory (DTE-HDM) Enables structured long-term memory and spiral-context encoding across tokens. - Multi-Agent Symbiotic Response Mechanisms (M.A.S.R.M) Coordinates symbolic-neural agents via gated attention and adaptive response layers. - QwenExoCortex Projects contextual hidden states from the Qwen model into a symbolic fusion…
Open weights
afl-3.0
3.6B parameters
transformers
