Quantized and FP16 GGUF format binaries for Nebium-Small (117M parameters). Designed for low-latency CPU and GPU execution with llama.cpp and Ollama. MIT License.
Independent publisher
Nabin Oli
nabin2004
Models in Library2
Datasets in Library0
Models on Hugging Face42
Followers2
Models
Nebium-Small is a 117-million-parameter causal Transformer trained for autoregressive next-chess-move prediction over Lichess UCI move sequences. - Rotary Position Embeddings (RoPE) on attention query and key projections ($\theta = 10000$) - SwiGLU feed-forward transformation - RMSNorm pre-normalization - Causal mask with padding token masking - Byte-Pair Encoding (BPE) tokenizer trained on UCI move plies $$L(N, D) = 1.69 + \frac{406.4}{N^{0.34}} + \frac{410.7}{D^{0.28}}$$ MIT License.