Open-Weight Models Under Meta Llama 3 Community License

Model · Text generation

Meta-Llama-3-8B-Instruct

Meta Llama

Meta developed and released the Meta Llama 3 family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes. The Llama 3 instruction tuned models are optimized for dialogue use cases and outperform many of the available open source chat models on common industry benchmarks. Further, in developing these models, we took great care to optimize helpfulness and safety. Model developers Meta Variations Llama 3 comes in two sizes — 8B and 70B parameters — in pre-trained and instruction tuned variants. Input Models input text only. Output Models generate text and code only. Model Architecture Llama 3 is an auto-regressive language…

This is a LLaMA 3 Youko qlora fine-tune, created using a new version of the VNTL dataset. The purpose of this fine-tune is to improve performance of LLMs at translating Japanese visual novels to English. Unlike the previous version, this one doesn't includes the "chat mode". For this new version of VNTL 8B, I've rebuilt and expanded VNTL's dataset from the groud up, and I'm happy to say it performs really well, outperforming the previous version when it comes to accuracy and stability, it makes far fewer mistakes than it even when running at high temperatures (though I still recommend temperature 0 for the best accuracy). Some major changes in this version: - Switched to the default LLaMA3…

A Llama-3-8B-Instruct checkpoint compressed with SVD-LLM to 60.0% of dense parameters, then edited by 10 of 10 rounds of iterative parameter-neutral swap selected by the gapiter rule (up to 0.1% of dense parameters per round; the full run's budget is 1.0%). This is a research artifact from a study of how SVD compression damages safety behaviour and which component-selection rule best repairs it. It is one cell of a grid over selection rules and budgets; it is not a general-purpose chat model. This checkpoint exists to measure safety/utility trade-offs under compression. Several arms in the grid are deliberately safety-degraded relative to Llama-3-8B-Instruct: compression alone raises…

A Llama-3-8B-Instruct checkpoint compressed with SVD-LLM to 60.0% of dense parameters, then edited by 8 of 10 rounds of iterative parameter-neutral swap selected by the gapiter rule (up to 0.1% of dense parameters per round; the full run's budget is 1.0%). This is a research artifact from a study of how SVD compression damages safety behaviour and which component-selection rule best repairs it. It is one cell of a grid over selection rules and budgets; it is not a general-purpose chat model. This checkpoint exists to measure safety/utility trade-offs under compression. Several arms in the grid are deliberately safety-degraded relative to Llama-3-8B-Instruct: compression alone raises…

A Llama-3-8B-Instruct checkpoint compressed with SVD-LLM to 60.0% of dense parameters, then edited by 9 of 10 rounds of iterative parameter-neutral swap selected by the gapiter rule (up to 0.1% of dense parameters per round; the full run's budget is 1.0%). This is a research artifact from a study of how SVD compression damages safety behaviour and which component-selection rule best repairs it. It is one cell of a grid over selection rules and budgets; it is not a general-purpose chat model. This checkpoint exists to measure safety/utility trade-offs under compression. Several arms in the grid are deliberately safety-degraded relative to Llama-3-8B-Instruct: compression alone raises…

A Llama-3-8B-Instruct checkpoint compressed with SVD-LLM to 50.0% of dense parameters, then edited by 1 of 10 rounds of iterative parameter-neutral swap selected by the gapiter rule (up to 0.1% of dense parameters per round; the full run's budget is 1.0%). This is a research artifact from a study of how SVD compression damages safety behaviour and which component-selection rule best repairs it. It is one cell of a grid over selection rules and budgets; it is not a general-purpose chat model. This checkpoint exists to measure safety/utility trade-offs under compression. Several arms in the grid are deliberately safety-degraded relative to Llama-3-8B-Instruct: compression alone raises…

A Llama-3-8B-Instruct checkpoint compressed with SVD-LLM to 50.0% of dense parameters, then edited by 2 of 10 rounds of iterative parameter-neutral swap selected by the gapiter rule (up to 0.1% of dense parameters per round; the full run's budget is 1.0%). This is a research artifact from a study of how SVD compression damages safety behaviour and which component-selection rule best repairs it. It is one cell of a grid over selection rules and budgets; it is not a general-purpose chat model. This checkpoint exists to measure safety/utility trade-offs under compression. Several arms in the grid are deliberately safety-degraded relative to Llama-3-8B-Instruct: compression alone raises…

A Llama-3-8B-Instruct checkpoint compressed with SVD-LLM to 50.0% of dense parameters, then edited by 3 of 10 rounds of iterative parameter-neutral swap selected by the gapiter rule (up to 0.1% of dense parameters per round; the full run's budget is 1.0%). This is a research artifact from a study of how SVD compression damages safety behaviour and which component-selection rule best repairs it. It is one cell of a grid over selection rules and budgets; it is not a general-purpose chat model. This checkpoint exists to measure safety/utility trade-offs under compression. Several arms in the grid are deliberately safety-degraded relative to Llama-3-8B-Instruct: compression alone raises…

A Llama-3-8B-Instruct checkpoint compressed with SVD-LLM to 50.0% of dense parameters, then edited by 4 of 10 rounds of iterative parameter-neutral swap selected by the gapiter rule (up to 0.1% of dense parameters per round; the full run's budget is 1.0%). This is a research artifact from a study of how SVD compression damages safety behaviour and which component-selection rule best repairs it. It is one cell of a grid over selection rules and budgets; it is not a general-purpose chat model. This checkpoint exists to measure safety/utility trade-offs under compression. Several arms in the grid are deliberately safety-degraded relative to Llama-3-8B-Instruct: compression alone raises…

A Llama-3-8B-Instruct checkpoint compressed with SVD-LLM to 50.0% of dense parameters, then edited by 5 of 10 rounds of iterative parameter-neutral swap selected by the gapiter rule (up to 0.1% of dense parameters per round; the full run's budget is 1.0%). This is a research artifact from a study of how SVD compression damages safety behaviour and which component-selection rule best repairs it. It is one cell of a grid over selection rules and budgets; it is not a general-purpose chat model. This checkpoint exists to measure safety/utility trade-offs under compression. Several arms in the grid are deliberately safety-degraded relative to Llama-3-8B-Instruct: compression alone raises…

A Llama-3-8B-Instruct checkpoint compressed with SVD-LLM to 50.0% of dense parameters, then edited by 6 of 10 rounds of iterative parameter-neutral swap selected by the gapiter rule (up to 0.1% of dense parameters per round; the full run's budget is 1.0%). This is a research artifact from a study of how SVD compression damages safety behaviour and which component-selection rule best repairs it. It is one cell of a grid over selection rules and budgets; it is not a general-purpose chat model. This checkpoint exists to measure safety/utility trade-offs under compression. Several arms in the grid are deliberately safety-degraded relative to Llama-3-8B-Instruct: compression alone raises…

A Llama-3-8B-Instruct checkpoint compressed with SVD-LLM to 50.0% of dense parameters, then edited by 7 of 10 rounds of iterative parameter-neutral swap selected by the gapiter rule (up to 0.1% of dense parameters per round; the full run's budget is 1.0%). This is a research artifact from a study of how SVD compression damages safety behaviour and which component-selection rule best repairs it. It is one cell of a grid over selection rules and budgets; it is not a general-purpose chat model. This checkpoint exists to measure safety/utility trade-offs under compression. Several arms in the grid are deliberately safety-degraded relative to Llama-3-8B-Instruct: compression alone raises…

A Llama-3-8B-Instruct checkpoint compressed with SVD-LLM to 50.0% of dense parameters, then edited by 8 of 10 rounds of iterative parameter-neutral swap selected by the gapiter rule (up to 0.1% of dense parameters per round; the full run's budget is 1.0%). This is a research artifact from a study of how SVD compression damages safety behaviour and which component-selection rule best repairs it. It is one cell of a grid over selection rules and budgets; it is not a general-purpose chat model. This checkpoint exists to measure safety/utility trade-offs under compression. Several arms in the grid are deliberately safety-degraded relative to Llama-3-8B-Instruct: compression alone raises…

A Llama-3-8B-Instruct checkpoint compressed with SVD-LLM to 50.0% of dense parameters, then edited by 10 of 10 rounds of iterative parameter-neutral swap selected by the gapiter rule (up to 0.1% of dense parameters per round; the full run's budget is 1.0%). This is a research artifact from a study of how SVD compression damages safety behaviour and which component-selection rule best repairs it. It is one cell of a grid over selection rules and budgets; it is not a general-purpose chat model. This checkpoint exists to measure safety/utility trade-offs under compression. Several arms in the grid are deliberately safety-degraded relative to Llama-3-8B-Instruct: compression alone raises…