# specter2_base by Ai2: Open-Weight Model Source: https://savrn.com/models/specter2-base Markdown alternate of the page above; the site index is https://savrn.com/llms.txt --- ## Model Card By Ai2, published under apache-2.0, revision 3447645e1def. ### SPECTER2 SPECTER2 is the successor to [SPECTER](https://huggingface.co/allenai/specter) and is capable of generating task specific embeddings for scientific tasks when paired with [adapters](https://huggingface.co/models?search=allenai/specter-2_). This is the base model to be used along with the adapters. Given the combination of title and abstract of a scientific paper or a short texual query, the model can be used to generate effective embeddings to be used in downstream applications. Note:For general embedding purposes, please use [allenai/specter2](https://huggingface.co/allenai/specter2). To get the best performance on a downstream task type please load the associated adapter with the base model as in the example below. Dec 2023 Update: Model usage updated to be compatible with latest versions of transformers and adapters (newly released update to adapter-transformers) libraries. Aug 2023 Update: 1. The SPECTER2 Base and proximity adapter models have been renamed in Hugging Face based upon usage patterns as follows: | Old Name | New Name | | --- | --- | | allenai/specter2 | [allenai/specter2_base](https://savrn.com/models/specter2-base) | | allenai/specter2_proximity | [allenai/specter2](https://huggingface.co/allenai/specter2) | [Read the full model card (883 words)](https://savrn.com/models/specter2-base/card) ## Configuration Architecture BertModel Context length (tokens) 512 Layers 12 Hidden size 768 Feed-forward size 3,072 Attention heads 12 Vocabulary size 31,090 Stored precision float32 Model type bert ## Identity and Version Repository allenai/specter2_base Publisher Ai2 Task Feature extraction Modality Text Library transformers Parameters Not stated by the source Languages en Revision 3447645e1def9117997203454fa4495937bfbd83 First published 2023-02-16 Last updated 2024-12-04 ## Files and Weights 8 files, 440.7 MB in total. The weights are 1 file totalling 439.7 MB in bin. Weights1 file · 439.7 MB Configuration2 files · 879 B Tokenizer3 files · 945.1 KB Documentation1 file · 9.2 KB Repository1 file · 1.5 KB Every file | File | Type | Size | SHA-256 | | --- | --- | --- | --- | | pytorch_model.bin | Weights | 439.7 MB | 801eda968fad | | config.json | Configuration | 754 B | — | | special_tokens_map.json | Configuration | 125 B | — | | README.md | Documentation | 9.2 KB | — | | .gitattributes | Repository | 1.5 KB | — | | tokenizer.json | Tokenizer | 716.8 KB | — | | tokenizer_config.json | Tokenizer | 453 B | — | | vocab.txt | Tokenizer | 227.8 KB | — | ## License and Download License apache-2.0 Access Open weights, no gate Download size 439.7 MB [Download from Ai2](https://huggingface.co/allenai/specter2_base) Released by Ai2 through its official repository on Hugging Face. [Read the license](https://www.apache.org/licenses/LICENSE-2.0). ## Built From - Trained on (disclosed) allenai/scirepeval ## Memory Requirements | Precision | Weights in memory | | --- | --- | | As published | 439.7 MB | Weights only, from the published parameter count; the key-value cache and runtime add to this. ## Questions About specter2_base ### Can I use specter2_base commercially? Yes. specter2_base is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors. ### What is specter2_base's context length? 512 tokens, from the maximum position embeddings in its published configuration. ## Similar Models Model · Feature extraction ### [all-MiniLM-L6-v2](https://savrn.com/models/all-minilm-l6-v2-2) [Joshua](https://savrn.com/model-publishers/xenova) https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2 with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: You can then use the model to compute embeddings like this: You can convert this Tensor to a nested JavaScript array using.tolist(): Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx). Open weights apache-2.0 512 tokens transformers.js [View model](https://savrn.com/models/all-minilm-l6-v2-2) Model · Feature extraction ### [bge-base-en-v1.5](https://savrn.com/models/bge-base-en-v1-5-2) [Joshua](https://savrn.com/model-publishers/xenova) https://huggingface.co/BAAI/bge-base-en-v1.5 with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: You can then use the model to compute embeddings, as follows: You can also use the model for retrieval. For example: Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx). Open weights mit 512 tokens transformers.js [View model](https://savrn.com/models/bge-base-en-v1-5-2) Model · Feature extraction ### [bge-large-zh-v1.5](https://savrn.com/models/bge-large-zh-v1-5) [Beijing Academy of Artificial Intelligence](https://savrn.com/model-publishers/baai) For more details please refer to our Github: FlagEmbedding. If you are looking for a model that supports more languages, longer texts, and other retrieval methods, you can try using bge-m3. FlagEmbedding focuses on retrieval-augmented LLMs, consisting of the following projects currently: - 1/30/2024: Release BGE-M3, a new member to BGE model series! M3 stands for Multi-linguality (100+ languages), Multi-granularities (input length up to 8192), Multi-Functionality (unification of dense, lexical, multi-vec/colbert retrieval). It is the first embedding model which supports all three retrieval methods, achieving new SOTA on multi-lingual (MIRACL) and cross-lingual (MKQA) benchmarks. Technical… Open weights mit 512 tokens sentence-transformers [View model](https://savrn.com/models/bge-large-zh-v1-5) Model · Feature extraction ### [wavlm-large](https://savrn.com/models/wavlm-large) [Microsoft](https://savrn.com/model-publishers/microsoft) The large model pretrained on 16kHz sampled speech audio. When using the model, make sure that your speech input is also sampled at 16kHz. Note: This model does not have a tokenizer as it was pretrained on audio alone. In order to use this model speech recognition, a tokenizer should be created and the model should be fine-tuned on labeled text data. Check out this blog for more in-detail explanation of how to fine-tune the model. - 60,000 hours of Libri-Light - 10,000 hours of GigaSpeech - 24,000 hours of VoxPopuli Authors: Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin… Open weights transformers [View model](https://savrn.com/models/wavlm-large) Model · Feature extraction ### [1](https://savrn.com/models/1) [Unsloth Backup Account](https://savrn.com/model-publishers/unslothai) Open weights 2,048 tokens transformers [View model](https://savrn.com/models/1) Model · Feature extraction ### [splade-cocondenser-selfdistil](https://savrn.com/models/splade-cocondenser-selfdistil) [NAVER](https://savrn.com/model-publishers/naver) SPLADE model for passage retrieval. For additional details, please visit: This is a SPLADE Sparse Encoder model. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval. First install the Sentence Transformers library: Then you can load this model and run inference. If you use our checkpoint, please cite our work Open weights cc-by-nc-sa-4.0 512 tokens sentence-transformers [View model](https://savrn.com/models/splade-cocondenser-selfdistil) ## Ai2 [All models and datasets](https://savrn.com/model-publishers/allenai) ## Versions - [3447645e1def](https://savrn.com/models/specter2-base/versions/3447645e1def) · current 2026-10-12 ## Explore More - [All feature extraction models](https://savrn.com/models/tasks/feature-extraction) - [All models under apache-2.0](https://savrn.com/models/licenses/apache-2-0) - [Model comparisons](https://savrn.com/models/comparisons) - [The model directory](https://savrn.com/models) - [Open model prices by host](https://savrn.com/ai-index/pricing/open-models) ## Source - Repository metadata, read 2026-10-19. - [Ai2 website](https://allenai.org/) - [Hugging Face record](https://huggingface.co/allenai/specter2_base) - [How the hub is built](https://savrn.com/model-hub/methodology)