Open-Weight Models Under llama2

Model · Image and text to text

llava-1.5-7b-hf

Llava Hugging Face

Below is the model card of Llava model 7b, which is copied from the original Llava model card that you can find here. Check out also the Google Colab demo to run Llava on a free-tier Google Colab instance: Or check out our Spaces demo! LLaVA is an open-source chatbot trained by fine-tuning LLaMA/Vicuna on GPT-generated multimodal instruction-following data. It is an auto-regressive language model, based on the transformer architecture. LLaVA-v1.5-7B was trained in September 2023. Paper or resources for more information: https://llava-vl.github.io/ First, make sure to have transformers >= 4.35.3. The model supports multi-image and multi-prompt generation. Meaning that you can pass multiple…

Model · Text generation

Llama-2-7b-hf

Meta Llama

Llama 2 is a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. This is the repository for the 7B pretrained model, converted for the Hugging Face Transformers format. Links to other models can be found in the index at the bottom. Note: Use of this model is governed by the Meta license. In order to download the model weights and tokenizer, please visit the website and accept our License before requesting access here. Meta developed and publicly released the Llama 2 family of large language models (LLMs), a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion…

meta-llama/Llama-2-7b-chat-hf compressed with Basis Sharing using the authors' code then recovered by LoRA on the coefficients alone. (the shared-basis fit) -> LoRA on the coefficients, bases frozen -> merge C' = C + (alpha/r)BA -> fold W = C' @ B to dense 2048 tokens, seed 42 (upstream hard-codes seed 2023; this project calibrates every method at one seed) 2 epochs, lr 0.0001, batch 64, on yahma/alpaca-cleaned Only the coefficients were trained. The shared and per-layer bases are frozen and bit-identical to the compressed model, so every weight keeps rank <= k, each group still shares one basis, and the parameter budget survives recovery exactly. This is not Basis Sharing's own LoRA…