To run the model on GPU, you need to install Flash Attention. You may either install from pypi (which may not work with fused-dense), or from source. To install from source, clone the GitHub repository: The code provided here should work with commit 43950dd.
Model Card
To run the model on GPU, you need to install Flash Attention. You may either install from pypi (which may not work with fused-dense), or from source. To install from source, clone the GitHub repository: The code provided here should work with commit 43950dd. Change to the cloned repo and install: This will compile the flash-attention kernel, which will take some time. If you would like to use fused MLPs (e.g. to use activation checkpointing), you may install fused-dense also from source: The config adds some new parameters: - useflashattn: If True, always use flash attention. If None, use flash attention when GPU is available. If False, never use flash attention (works on CPU).…
Excerpt from the card by Jina AI.
Configuration
- Layers
- 12
- Hidden size
- 768
- Feed-forward size
- 3,072
- Attention heads
- 12
- Vocabulary size
- 30,528
- Stored precision
- float16
- Model type
- bert
Identity and Version
- Repository
- jinaai/jina-bert-flash-implementation
- Publisher
- Jina AI
- Task
- Not stated by the source
- Modality
- Other
- Library
- transformers
- Parameters
- Not stated by the source
- Languages
- Not stated by the source
- Revision
- b78d1595de294f13ffe7b19d6cd63892a6e4e7a4
- First published
- 2024-02-21
- Last updated
- 2024-05-31
Files and Weights
12 files, 144.3 KB in total.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| bert_padding.py | Configuration | 9.8 KB | — |
| block.py | Configuration | 17.4 KB | — |
| config.json | Configuration | 1.2 KB | — |
| configuration_bert.py | Configuration | 5.8 KB | — |
| convert_v2_weights.py | Configuration | 6.1 KB | — |
| embedding.py | Configuration | 2.3 KB | — |
| mha.py | Configuration | 35.4 KB | — |
| mlp.py | Configuration | 8.0 KB | — |
| modeling_bert.py | Configuration | 33.4 KB | — |
| modeling_for_glue.py | Configuration | 10.7 KB | — |
| modeling_lora.py | Configuration | 12.3 KB | — |
| README.md | Documentation | 1.9 KB | — |
License and Download
- License
- Not stated by the source
- Access
- Open weights, no gate
Released by Jina AI through its official repository on Hugging Face.