webshop_8b_b8_1node_step150

Model · Feature extraction

CLM-v0.1-8B-MLX-8bit

Zhu Lin

This is the 8bit variant. Also available: czl/CLM-v0.1-8B-MLX-4bit, czl/CLM-v0.1-8B-MLX-6bit, and the bf16 reference. The encoder half of Contrastive-LM/CLM-v0.1-8B, quantised for MLX on Apple Silicon. mlxlm is generate-only — it has no embeddings entrypoint, and mlxlm.server routes just /v1/completions, /v1/chat/completions, /v1/models and /health. So clmmlx supplies the pooling pass over mlxlm internals: Qwen3Model.call already returns self.norm(h), the post-final-RMSNorm hidden states, which is what vLLM's pooling runner returns in last-token mode. Agreement against a bf16 MLX reference of the same encoder in the same runtime, over 23,926 scored System One questions, through the real…

Model · Feature extraction

CLM-v0.1-8B-MLX

Zhu Lin

The encoder half of Contrastive-LM/CLM-v0.1-8B, quantised for MLX on Apple Silicon. One repository per bit width, matching mlx-community. Each has the weights at the repo root, so the Hub file browser lists every file with its size and mlxlm.load(" ") works. Agreement against a bf16 MLX reference of the same encoder in the same runtime, over 23,926 scored System One questions, through the real head stack (argmax(scale · cos), scale = 100.0 — cosine error is amplified 100×). Pass line. top-1 >= 1.0000 — the measured bf16-vs-bf16 noise floor of this corpus in this runtime — and top-1 (decisive) >= 0.995, where decisive means the reference's own top-1 led by more than 1 nat. A third condition…

Model · Feature extraction

CLM-v0.1-8B-MLX-6bit

Zhu Lin

This is the 6bit variant. Also available: czl/CLM-v0.1-8B-MLX-4bit, czl/CLM-v0.1-8B-MLX-8bit, and the bf16 reference. The encoder half of Contrastive-LM/CLM-v0.1-8B, quantised for MLX on Apple Silicon. mlxlm is generate-only — it has no embeddings entrypoint, and mlxlm.server routes just /v1/completions, /v1/chat/completions, /v1/models and /health. So clmmlx supplies the pooling pass over mlxlm internals: Qwen3Model.call already returns self.norm(h), the post-final-RMSNorm hidden states, which is what vLLM's pooling runner returns in last-token mode. Agreement against a bf16 MLX reference of the same encoder in the same runtime, over 23,926 scored System One questions, through the real…

Model · Feature extraction

Qwen3-Embedding-8B

Qwen

The Qwen3 Embedding model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks. Building upon the dense foundational models of the Qwen3 series, it provides a comprehensive range of text embeddings and reranking models in various sizes (0.6B, 4B, and 8B). This series inherits the exceptional multilingual capabilities, long-text understanding, and reasoning skills of its foundational model. The Qwen3 Embedding series represents significant advancements in multiple text embedding and ranking tasks, including text retrieval, code retrieval, text classification, text clustering, and bitext mining. Exceptional Versatility: The…

This model is a quantized version of the original model intfloat/e5-mistral-7b-instruct. It's quantized using the BitsAndBytes library to 4-bit using the bnb-my-repo space. - bnb4bitquanttype: nf4 - bnb4bitusedoublequant: True - bnb4bitcomputedtype: bfloat16 - bnb4bitquantstorage: uint8 Improving Text Embeddings with Large Language Models. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei, arXiv 2024 This model has 32 layers and the embedding size is 4096. Below is an example to encode queries and passages from the MS-MARCO passage ranking dataset. Have a look at configsentencetransformers.json for the prompts that are pre-configured, such as websearchquery…

GPTQ Quantized Qwen/Qwen3-Embedding-4B with THUIR/T2Ranking and m-a-p/COIG-CQIA for calibration set. ~0.72% lost in C-MTEB. Evaluation performed with official code. pip install compressed-tensors optimum and auto-gptq / gptqmodel, then goto the official usage guide.