Open-weight model · Question answering
next-12b-i1-GGUF
by Team Mradermacher mradermacher/next-12b-i1-GGUF
weighted/imatrix quants of https://huggingface.co/thelamapi/next-12b For a convenient overview and download list, visit our model page for this model.
Parameters—
Context—
Weights138.6 GB
Licensemit
AccessOpen weights
Monthly Downloads1.4k
Model Card
By Team Mradermacher, published under mit, revision 59a82aac4e17.
weighted/imatrix quants of https://huggingface.co/thelamapi/next-12b For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/next-12b-GGUF This is a vision model - mmproj files (if any) will be in the static repository. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are Artefact2's thoughts on the matter…
Read Team Mradermacher's full model card
About
weighted/imatrix quants of https://huggingface.co/thelamapi/next-12b
For a convenient overview and download list, visit our model page for this model.
static quants are available at https://huggingface.co/mradermacher/next-12b-GGUF
This is a vision model - mmproj files (if any) will be in the static repository.
Usage
If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files.
Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|---|---|---|---|
| GGUF | imatrix | 0.1 | imatrix file (for creating your own quants) |
| GGUF | i1-IQ1_S | 3.0 | for the desperate |
| GGUF | i1-IQ1_M | 3.3 | mostly desperate |
| GGUF | i1-IQ2_XXS | 3.6 | |
| GGUF | i1-IQ2_XS | 3.9 | |
| GGUF | i1-IQ2_S | 4.1 | |
| GGUF | i1-IQ2_M | 4.4 | |
| GGUF | i1-Q2_K_S | 4.5 | very low quality |
| GGUF | i1-Q2_K | 4.9 | IQ3_XXS probably better |
| GGUF | i1-IQ3_XXS | 4.9 | lower quality |
| GGUF | i1-IQ3_XS | 5.3 | |
| GGUF | i1-IQ3_S | 5.6 | beats Q3_K* |
| GGUF | i1-Q3_K_S | 5.6 | IQ3_XS probably better |
| GGUF | i1-IQ3_M | 5.8 | |
| GGUF | i1-Q3_K_M | 6.1 | IQ3_S probably better |
| GGUF | i1-Q3_K_L | 6.6 | IQ3_M probably better |
| GGUF | i1-IQ4_XS | 6.7 | |
| GGUF | i1-IQ4_NL | 7.0 | prefer IQ4_XS |
| GGUF | i1-Q4_0 | 7.0 | fast, low quality |
| GGUF | i1-Q4_K_S | 7.0 | optimal size/speed/quality |
| GGUF | i1-Q4_K_M | 7.4 | fast, recommended |
| GGUF | i1-Q4_1 | 7.7 | |
| GGUF | i1-Q5_K_S | 8.3 | |
| GGUF | i1-Q5_K_M | 8.5 | |
| GGUF | i1-Q6_K | 9.8 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):
And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to questions you might have and/or if you want some other model quantized.
Thanks
I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
Identity and Version
- Repository
- mradermacher/next-12b-i1-GGUF
- Publisher
- Team Mradermacher
- Task
- Question answering
- Modality
- Text
- Library
- transformers
- Parameters
- Not stated by the source
- Languages
- tr, en, de, ka, el, ku, es, sl
- Revision
- 59a82aac4e17399677c809f090931b096b6576ab
- First published
- 2025-10-29
- Last updated
- 2026-02-19
Files and Weights
27 files, 138.6 GB in total. The weights are 25 files totalling 138.6 GB in gguf.
Weights25 files · 138.6 GB
Documentation1 file · 7.0 KB
Repository1 file · 3.0 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| next-12b.i1-IQ1_M.gguf | Weights | 3.2 GB | aa13574dc62e |
| next-12b.i1-IQ1_S.gguf | Weights | 2.9 GB | 7cc80080e05f |
| next-12b.i1-IQ2_M.gguf | Weights | 4.3 GB | 79d65e4b9449 |
| next-12b.i1-IQ2_S.gguf | Weights | 4.0 GB | 1ccb98160bf4 |
| next-12b.i1-IQ2_XS.gguf | Weights | 3.8 GB | 25ecdd152530 |
| next-12b.i1-IQ2_XXS.gguf | Weights | 3.5 GB | deae57935ee8 |
| next-12b.i1-IQ3_M.gguf | Weights | 5.7 GB | 00abf7f1b2ba |
| next-12b.i1-IQ3_S.gguf | Weights | 5.5 GB | a1751e01dcba |
| next-12b.i1-IQ3_XS.gguf | Weights | 5.2 GB | 61e14e296614 |
| next-12b.i1-IQ3_XXS.gguf | Weights | 4.8 GB | 502f119aad88 |
| next-12b.i1-IQ4_NL.gguf | Weights | 6.9 GB | 439efbce8c6b |
| next-12b.i1-IQ4_XS.gguf | Weights | 6.6 GB | ea3276bd67d9 |
| next-12b.i1-Q2_K.gguf | Weights | 4.8 GB | d7f7561dbfdf |
| next-12b.i1-Q2_K_S.gguf | Weights | 4.4 GB | 2b362725b45c |
| next-12b.i1-Q3_K_L.gguf | Weights | 6.5 GB | 776dda699258 |
| next-12b.i1-Q3_K_M.gguf | Weights | 6.0 GB | b31c4d995a28 |
| next-12b.i1-Q3_K_S.gguf | Weights | 5.5 GB | 361b7caffa96 |
| next-12b.i1-Q4_0.gguf | Weights | 6.9 GB | b19ac3136145 |
| next-12b.i1-Q4_1.gguf | Weights | 7.6 GB | 8d741d33fa16 |
| next-12b.i1-Q4_K_M.gguf | Weights | 7.3 GB | 537aca4069db |
| next-12b.i1-Q4_K_S.gguf | Weights | 6.9 GB | 2863b8d5d50f |
| next-12b.i1-Q5_K_M.gguf | Weights | 8.4 GB | af9fc10003cd |
| next-12b.i1-Q5_K_S.gguf | Weights | 8.2 GB | 1c3ef1766266 |
| next-12b.i1-Q6_K.gguf | Weights | 9.7 GB | a9d91e7cd341 |
| next-12b.imatrix.gguf | Weights | 7.5 MB | df32da3ec295 |
| README.md | Documentation | 7.0 KB | — |
| .gitattributes | Repository | 3.0 KB | — |
License and Download
- License
- mit
- Access
- Open weights, no gate
- Download size
- 138.6 GB
Download from Team Mradermacher
Released by Team Mradermacher through its official repository on Hugging Face. Read the license.
Built From
- Derived from thelamapi/next-12b
- Quantized from thelamapi/next-12b
- Trained on (disclosed) CognitiveKernel/CognitiveKernel-Pro-SFT
- Trained on (disclosed) GreenerPastures/All-Your-Base-Full
- Trained on (disclosed) Gryphe/ChatGPT-4o-Writing-Prompts
- Trained on (disclosed) Gryphe/Opus-WritingPrompts
- Trained on (disclosed) HuggingFaceH4/MATH-500
- Trained on (disclosed) ITCL/FineTomeOs
- Trained on (disclosed) OpenSPG/KAG-Thinker-training-dataset
- Trained on (disclosed) QuixiAI/dolphin-r1
- Trained on (disclosed) SuperbEmphasis/Claude-4.0-DeepSeek-R1-RP-SFWish
- Trained on (disclosed) dongguanting/ARPO-SFT-54K
- Trained on (disclosed) mlabonne/FineTome-100k
- Trained on (disclosed) mlabonne/lmsys-arena-human-sft-55k
- Trained on (disclosed) mlabonne/natural_reasoning-formatted
- Trained on (disclosed) mlabonne/smoltalk-flat
- Trained on (disclosed) uclanlp/Brief-Pro
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 138.6 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About next-12b-i1-GGUF
Can I use next-12b-i1-GGUF commercially?
Yes. next-12b-i1-GGUF is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.
Similar Models
Splinter-base is the pretrained model discussed in the paper Few-Shot Question Answering by Pretraining Span Selection (at ACL 2021). Its original repository can be found here. The model is case-sensitive. Note: This model doesn't contain the pretrained weights for the QASS layer (see paper for details), and therefore the QASS layer is randomly initialized upon loading it. For the model with those weights, see tau/splinter-base-qass. Splinter is a model that is pretrained in a self-supervised fashion for few-shot question answering. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an…
This model can be used for the task of question answering. The model should not be used to intentionally create hostile or alienating environments for people. Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)). Predictions generated by the model may include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. The model creators note in the associated paper: The model creators note in the associated paper: The model…
This model is a fine-tune checkpoint of DistilBERT-base-cased, fine-tuned using (a second step of) knowledge distillation on SQuAD v1.1. This model reaches a F1 score of 87.1 on the dev set (for comparison, BERT bert-base-cased version reaches a F1 score of 88.7).
This model can be used for the task of Question Answering on Legal Documents. Read: An Open Source Contractual Language Understanding Application Using Machine Learning for detailed information on training procedure, dataset preprocessing and evaluation. See CUAD dataset card for more information. See CUAD dataset card for more information. Used V100/P100 from Google Colab Pro Python, Transformers Mohammed Rakib in collaboration with Ezi Ozoani and the Hugging Face team Use the code below to get started with the model.