The Llama 3.2 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction-tuned generative models in 1B and 3B sizes (text in/text out). The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks. They outperform many of the available open source and closed chat models on common industry benchmarks. Model Architecture: Llama 3.2 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for…
Access requested at publisher
llama3.2
1.2B parameters
transformers
The Llama 3.2 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction-tuned generative models in 1B and 3B sizes (text in/text out). The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks. They outperform many of the available open source and closed chat models on common industry benchmarks. Model Architecture: Llama 3.2 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for…
Access requested at publisher
llama3.2
1.2B parameters
transformers
This revision publishes the final raw checkpoint of raw6b-normuon-adam2x-20261018: 6,000,082,944 non-padding input tokens, 28,619 optimizer updates. The model was randomly initialized (residual-half, seed42); no pretrained weights or previous training checkpoint initialized this run. Model architecture and The full reserved 50,000-document set has not been evaluated for this checkpoint. PPL is exp(sum held-out NLL / sum valid prediction tokens), not a mean of subset PPLs. Documents are evaluated independently, with stride8191 for long documents. The entire original document split was shuffled with seed42 before tokenization; the last50,000 documents were held out and excluded from all…
Open weights
llama3.2
1.2B parameters
131,072 tokens
transformers
Building and evaluating AI on logistics data. Built with Llama. This checkpoint is a full-parameter fine-tune of meta-llama/Llama-3.2-1B-Instruct, published with the data split, evaluation results, and behavioral tests used to inspect it. The project connects a working training-and-inference pipeline with a retrospective audit of what its score demonstrates. The checkpoint and a depth-2 decision tree both score 100% on the same historical 200-row split. A rule using two supplied fields reproduces every label in the 1,000-row source table. Prompt rewrites reveal additional response failures, including sensitivity to irrelevant text. These findings make the checkpoint useful for studying…
Open weights
llama3.2
1.2B parameters
131,072 tokens
transformers
A FT expert from Table 1 of SMAT: Simple and Efficient Merge-Aware Training (training seed 42). SMAT trains experts that retain task performance while improving model merging. Built with Llama. The included Llama 3.2 license and acceptable-use policy apply.
Open weights
llama3.2
1.2B parameters
131,072 tokens
transformers
A FT expert from Table 1 of SMAT: Simple and Efficient Merge-Aware Training (training seed 42). SMAT trains experts that retain task performance while improving model merging. Built with Llama. The included Llama 3.2 license and acceptable-use policy apply.
Open weights
llama3.2
1.2B parameters
131,072 tokens
transformers
