Question Answering Models

Model · Question answering

roberta-base-squad2

Deepset

This is the roberta-base model, fine-tuned using the SQuAD2.0 dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Extractive Question Answering. We have also released a distilled version of this model called deepset/tinyroberta-squad2. It has a comparable prediction quality and runs at twice the speed of deepset/roberta-base-squad2. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over…

Model · Question answering

deberta-v3-base-squad2

Deepset

This is the deberta-v3-base model, fine-tuned using the SQuAD2.0 dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Question Answering. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. Timo Möller: timo.moeller [at] deepset.ai deepset is the company behind the production-ready open-source AI framework Haystack. We also…

Model · Question answering

bert-large-uncased-whole-word-masking-squad2

Deepset

This is a berta-large model, fine-tuned using the SQuAD2.0 dataset for the task of question answering. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to everyone!

Model · Question answering

distilbert-base-cased-distilled-squad

DistilBERT community

The DistilBERT model was proposed in the blog post Smaller, faster, cheaper, lighter: Introducing DistilBERT, adistilled version of BERT, and the paper DistilBERT, adistilled version of BERT: smaller, faster, cheaper and lighter. DistilBERT is a small, fast, cheap and light Transformer model trained by distilling BERT base. It has 40% less parameters than bert-base-uncased, runs 60% faster while preserving over 95% of BERT's performances as measured on the GLUE language understanding benchmark. This model is a fine-tune checkpoint of DistilBERT-base-cased, fine-tuned using (a second step of) knowledge distillation on SQuAD v1.1. - See this repository for more about Distil\ (a class of…

Model · Question answering

tinyroberta-squad2

Deepset

This is the distilled version of the deepset/roberta-base-squad2 model. This model has a comparable prediction quality and runs at twice the speed of the base model. This model was distilled using the TinyBERT approach described in this paper and implemented in haystack. Firstly, we have performed intermediate layer distillation with roberta-base as the teacher which resulted in deepset/tinyroberta-6l-768d. Secondly, we have performed task-specific distillation with deepset/roberta-base-squad2 as the teacher for further intermediate layer distillation on an augmented version of SQuADv2 and then with deepset/roberta-large-squad2 as the teacher for prediction layer distillation. Haystack is…

Model · Question answering

mdeberta-v3-base-squad2

Tim

It has been finetuned for 3 epochs on SQuAD2.0. DeBERTa improves the BERT and RoBERTa models using disentangled attention and enhanced mask decoder. With those two improvements, DeBERTa out perform RoBERTa on a majority of NLU tasks with 80GB training data. In DeBERTa V3, we further improved the efficiency of DeBERTa using ELECTRA-Style pre-training with Gradient Disentangled Embedding Sharing. Compared to DeBERTa, our V3 version significantly improves the model performance on downstream tasks. You can find more technique details about the new model from our paper. Please check the official repository for more implementation details and updates. mDeBERTa is multilingual version of DeBERTa…

Model · Question answering

splinter-base

Tel Aviv University

Splinter-base is the pretrained model discussed in the paper Few-Shot Question Answering by Pretraining Span Selection (at ACL 2021). Its original repository can be found here. The model is case-sensitive. Note: This model doesn't contain the pretrained weights for the QASS layer (see paper for details), and therefore the QASS layer is randomly initialized upon loading it. For the model with those weights, see tau/splinter-base-qass. Splinter is a model that is pretrained in a self-supervised fashion for few-shot question answering. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an…

Model · Question answering

bert-base-cased-squad2

Deepset

This is a BERT base cased model trained on SQuAD v2 Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to everyone!

Model · Question answering

xlm-roberta-base-squad2

Deepset

Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. Evaluated on the SQuAD 2.0 dev set with the official eval script. "exact": 33.67279167589108 "total": 4517 "exact": 48.739495798319325 "total": 1190 Timo Möller: timo.moeller [at] deepset.ai deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to…

Model · Question answering

bert-large-uncased-whole-word-masking-finetuned-squad

BERT community

Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in this paper and first released in this repository. This model is uncased: it does not make a difference between english and English. Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same. The training is identical -- each masked WordPiece token is predicted independently. After pre-training, this model was fine-tuned on the SQuAD dataset with one of our fine-tuning scripts. See below for more information regarding this…

Model · Question answering

distilbert-base-uncased-distilled-squad

DistilBERT community

The DistilBERT model was proposed in the blog post Smaller, faster, cheaper, lighter: Introducing DistilBERT, adistilled version of BERT, and the paper DistilBERT, adistilled version of BERT: smaller, faster, cheaper and lighter. DistilBERT is a small, fast, cheap and light Transformer model trained by distilling BERT base. It has 40% less parameters than bert-base-uncased, runs 60% faster while preserving over 95% of BERT's performances as measured on the GLUE language understanding benchmark. This model is a fine-tune checkpoint of DistilBERT-base-uncased, fine-tuned using (a second step of) knowledge distillation on SQuAD v1.1. - See this repository for more about Distil\ (a class of…

Model · Question answering

bert-large-cased-whole-word-masking-finetuned-squad

BERT community

Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in this paper and first released in this repository. This model is cased: it makes a difference between english and English. Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same. The training is identical -- each masked WordPiece token is predicted independently. After pre-training, this model was fine-tuned on the SQuAD dataset with one of our fine-tuning scripts. See below for more information regarding this fine-tuning.…

Model · Question answering

bert-medium-squad2-distilled

Deepset

Haystack version 1.x distillation feature was used for training. deepset/bert-large-uncased-whole-word-masking-squad2 was used as the teacher model. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. - Timo Möller: timo.moeller [at] deepset.ai deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to…

Model · Question answering

mobilebert-uncased-squad-v2

Qingqing Cao

MobileBERT is a thin version of BERTLARGE, while equipped with bottleneck structures and a carefully designed balance between self-attentions and feed-forward networks. This model was fine-tuned from the HuggingFace checkpoint google/mobilebert-uncased on SQuAD2.0. CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz Memory: 32 GiB GPUs: 2 GeForce GTX 1070, each with 8GiB memory GPU driver: 418.87.01, CUDA: 10.1 It took about 3.5 hours to finish. Note that the above results didn't involve any hyperparameter search.

Model · Question answering

minilm-uncased-squad2

Deepset

Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. Evaluated on the SQuAD 2.0 dev set with the official eval script. Timo Möller: [email protected] deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to everyone!

Model · Question answering

biobert-large-cased-v1.1-squad

Data Mining and Information Systems Lab

This model can be used for the task of question answering. The model should not be used to intentionally create hostile or alienating environments for people. Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)). Predictions generated by the model may include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. The model creators note in the associated paper: The model creators note in the associated paper: The model…

Model · Question answering

xlm-roberta-large-squad2

Deepset

Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. Evaluated on the SQuAD 2.0 English dev set with the official eval script. Evaluated on German MLQA: test-context-de-question-de.json Evaluated on German XQuAD: xquad.de.json For doing QA at scale (i.e. many docs instead of single paragraph), you can load the model also in haystack: Timo Möller: [email protected]…

Model · Question answering

bert-base-uncased-squad-v1

Qingqing Cao

This model was fine-tuned from the HuggingFace BERT base uncased checkpoint on SQuAD1.1. CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz Memory: 32 GiB GPUs: 2 GeForce GTX 1070, each with 8GiB memory GPU driver: 418.87.01, CUDA: 10.1 It took about 2 hours to finish. Note that the above results didn't involve any hyperparameter search.

Model · Question answering

roberta-large-squad2

Deepset

This is the roberta-large model, fine-tuned using the SQuAD2.0 dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Question Answering. Please note that we have also released a distilled version of this model called deepset/roberta-base-squad2-distilled. The distilled model has a comparable prediction quality and runs at twice the speed of the large model. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering…

Model · Question answering

deberta-v3-large-squad2

Deepset

This is the deberta-v3-large model, fine-tuned using the SQuAD2.0 dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Extractive Question Answering. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. Evaluated on the SQuAD 2.0 dev set with the official eval script. deepset is the company behind the production-ready…

Model · Question answering

distilbert-onnx

Philipp Schmid

This model is a fine-tune checkpoint of DistilBERT-base-cased, fine-tuned using (a second step of) knowledge distillation on SQuAD v1.1. This model reaches a F1 score of 87.1 on the dev set (for comparison, BERT bert-base-cased version reaches a F1 score of 88.7).

Model · Question answering

roberta-base-squad2-distilled

Deepset

haystack's distillation feature was used for training. deepset/roberta-large-squad2 was used as the teacher model. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. Timo Möller: [email protected] deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to everyone!

Model · Question answering

roberta-base-on-cuad

Mohammed Rakib

This model can be used for the task of Question Answering on Legal Documents. Read: An Open Source Contractual Language Understanding Application Using Machine Learning for detailed information on training procedure, dataset preprocessing and evaluation. See CUAD dataset card for more information. See CUAD dataset card for more information. Used V100/P100 from Google Colab Pro Python, Transformers Mohammed Rakib in collaboration with Ezi Ozoani and the Hugging Face team Use the code below to get started with the model.

Model · Question answering

flan-t5-base-squad2

Sebastian Husch Lee

This is the flan-t5-base model, fine-tuned using the SQuAD2.0 dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Extractive Question Answering. UPDATE: With transformers version 4.31.0 the useremotecode=True is no longer necessary. NOTE: The token must be manually added to the beginning of the question for this model to work properly. It uses the token to be able to make "no answer" predictions. The t5 tokenizer does not automatically add this special token which is why it is added manually. The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 16 - evalbatchsize: 8 - gradientaccumulationsteps: 6…

Model · Question answering

vit5-base

VietAI

State-of-the-art pretrained Transformer-based encoder-decoder model for Vietnamese. For more details, do check out our Github repo.

Model · Question answering

mobilebert-uncased-squad-v1

Qingqing Cao

MobileBERT is a thin version of BERTLARGE, while equipped with bottleneck structures and a carefully designed balance between self-attentions and feed-forward networks. This model was fine-tuned from the HuggingFace checkpoint google/mobilebert-uncased on SQuAD1.1. CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz Memory: 32 GiB GPUs: 2 GeForce GTX 1070, each with 8GiB memory GPU driver: 418.87.01, CUDA: 10.1 It took about 3 hours to finish. Note that the above results didn't involve any hyperparameter search.

UlizaLlama is a 7B Parameters language model that builds upon the foundation of Jacaranda/kiswallama-pretrained. Jacaranda/kiswallama-pretrained is a large language model continually-pretrained with 321,530,045 swahili tokens and a customized tokenizer with a swahili vocabulary of 20,000 tokens to extend the capabilities of Meta/Llama2. It offers significant improvements in both encoding and decoding for Swahili text, surpassing the Swahili performance of Meta/Llama2. Moreover, Jacaranda/kiswallama-pretrained excels in providing accurate next-word completions in Swahili, a capability which Meta/Llama2 falls short of. UlizaLlama is optimized for downstream tasks, notably those demanding…

Model · Question answering

electra-base-squad2

Deepset

Evaluated on the SQuAD 2.0 dev set with the official eval script. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. Vaishali Pal vaishali.pal [at] deepset.ai Timo Möller: timo.moeller [at] deepset.ai deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to everyone!

Model · Question answering

xlm-roberta-base-squad2-distilled

Deepset

Haystack's distillation feature was used for training. deepset/xlm-roberta-large-squad2 was used as the teacher model. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. Evaluated on the SQuAD 2.0 dev set Timo Möller: [email protected] deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to…

static quants of https://huggingface.co/Locutusque/OpenCerebrum-2.0-7B weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are Artefact2's thoughts on the matter…

weighted/imatrix quants of https://huggingface.co/nikhilchandak/LlamaForecaster-8B For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/LlamaForecaster-8B-GGUF If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See…

Model · Question answering

bert-base-spanish-wwm-cased-finetuned-spa-squad2-es

Manuel Romero

This model is provided by BETO team and fine-tuned on SQuAD-es-v2.0 for Q&A downstream task. Language model ('dccuchile/bert-base-spanish-wwm-cased'): BETO is a BERT model trained on a big Spanish corpus. BETO is of size similar to a BERT-Base and was trained with the Whole Word Masking technique. Below you find Tensorflow and Pytorch checkpoints for the uncased and cased versions, as well as some results for Spanish benchmarks comparing BETO with Multilingual BERT as well as other (not BERT-based) models. The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: 1. Set the context and ask some questions: 2. Run predictions

Model · Question answering

Gecko-110m-en

LiteRT Community (FKA TFLite)

This model provides a few variants of the embedding model published in the Gecko paper that are ready for deployment on Android or iOS using LiteRT stack or google ai edge RAG SDK. Try out the gecko embedding model in the google ai edge RAG SDK. You can find the SDK on GitHub or follow our android guide to install directly from Maven. We have also published a Use the sentencepiece model as the tokenizer for the Gecko embedding model. Note that all benchmark stats are from a Samsung S23 Ultra. The inference is run on CPU is accelerated via the LiteRT XNNPACK delegate with 4 threads The inference on GPU is accelerated via LiteRT GPU delegate. Benchmark is done assuming XNNPACK cache is…

Model · Question answering

vit5-large

VietAI

State-of-the-art pretrained Transformer-based encoder-decoder model for Vietnamese. For more details, do check out our Github repo.

Model · Question answering

dynamic_tinybert

Intel

Dynamic-TinyBERT has been fine-tuned for the NLP task of question answering, trained on the SQuAD 1.1 dataset. Guskin et al. (2021) note: Here is how to import this model in Python

static quants of https://huggingface.co/behbudiy/Llama-3.1-8B-Instruct-Uz For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality…

This model is a fine-tuned version of distilbert-base-uncased on the naturalquestions dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 64 - evalbatchsize: 64 - lrschedulertype: linear - numepochs: 40 - Transformers 4.17.0 - Pytorch 1.10.0 - Datasets 1.18.4 - Tokenizers 0.11.6

static quants of https://huggingface.co/behbudiy/Llama-3.1-8B-Instruct-Uz For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality…

Model · Question answering

gelectra-base-germanquad

Deepset

We trained a German question answering model with a gelectra-base model as its basis. - The dataset is GermanQuAD, a new, German language dataset, which we hand-annotated and published online. - The training dataset is one-way annotated and contains 11518 questions and 11518 answers, while the test dataset is three-way annotated so that there are 2204 questions and with 2204·3−76 = 6536answers, because we removed 76 wrong answers. See https://deepset.ai/germanquad for more details and dataset download in SQuAD format. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on…

Model · Question answering

gelectra-large-germanquad

Deepset

We trained a German question answering model with a gelectra-large model as its basis. - The dataset is GermanQuAD, a new, German language dataset, which we hand-annotated and published online. - The training dataset is one-way annotated and contains 11518 questions and 11518 answers, while the test dataset is three-way annotated so that there are 2204 questions and with 2204·3−76 = 6536 answers, because we removed 76 wrong answers. See https://deepset.ai/germanquad for more details and dataset download in SQuAD format. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on…

Model · Question answering

bert-multi-cased-finetuned-xquadv1

Manuel Romero

This model was created by Google and fine-tuned on XQuAD like data for multilingual (11 different languages) Q&A downstream task. Deepmind XQuAD As the dataset is based on SQuAD v1.1, there are no unanswerable questions in the data. We chose this setting so that models can focus on cross-lingual transfer. We show the average number of tokens per paragraph, question, and answer for each language in the table below. The statistics were obtained using Jieba for Chinese and the Moses tokenizer for the other languages. As XQuAD is just an evaluation dataset, I used Data augmentation techniques (scraping, neural machine translation, etc) to obtain more samples and split the dataset in order to…

weighted/imatrix quants of https://huggingface.co/erax-ai/EraX-VL-7B-V2.0-Preview For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/EraX-VL-7B-V2.0-Preview-GGUF This is a vision model - mmproj files (if any) will be in the static repository. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are Artefact2's thoughts on…

Model · Question answering

bert-base-uncased-squad2

Deepset

Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. - Timo Möller: timo.moeller [at] deepset.ai deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to everyone!

Model · Question answering

roberta-base-cuad

Taner Akdeniz

This model is the fine-tuned version of "RoBERTa Base" using CUAD dataset https://huggingface.co/datasets/cuad For the use of the model with CUAD: https://github.com/marshmellow77/cuad-demo and https://huggingface.co/spaces/akdeniz27/contract-understanding-atticus-dataset-demo

Model · Question answering

bert-small-cord19qa

NeuML

bert-small-cord19-squad model fine-tuned on the CORD-19 QA dataset. The CORD-19 QA dataset is a SQuAD 2.0 formatted list of question, context, answer combinations covering the CORD-19 dataset.

Model · Question answering

distilbert-base-cased-distilled-squad

Joshua

https://huggingface.co/distilbert-base-cased-distilled-squad with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx).

weighted/imatrix quants of https://huggingface.co/thelamapi/next-12b For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/next-12b-GGUF This is a vision model - mmproj files (if any) will be in the static repository. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are Artefact2's thoughts on the matter…