Mou Geren

GPTQ Quantized Qwen/Qwen3-Embedding-4B with THUIR/T2Ranking and m-a-p/COIG-CQIA for calibration set. ~0.72% lost in C-MTEB. Evaluation performed with official code. pip install compressed-tensors optimum and auto-gptq / gptqmodel, then goto the official usage guide.

GPTQ Quantized Qwen/Qwen3-Reranker-4B with Ultrachat, THUIR/T2Ranking and m-a-p/COIG-CQIA for calibration set. VRAM Usage: 17430M -> 11000M (w/o FA2, according to Embedding model's result). I think <5% accuracy, further evaluation on the way... The Embedding one shows ~0.7%. pip install compressed-tensors optimum and auto-gptq / gptqmodel, then goto the official usage guide.