Darwin uniform-soup of 3 Spark2.5-1.7B donors (226/226 tensors verified). Agentic/tool-use + uncensored + multilingual blend. Distillation donor for the 4B program. Needs transformers==5.17.0 + trustremotecode=True.
LuminoLex-1.7B is an open-weight model for text generation from LuminoLex AI, released under Apache License 2.0. It has 1.7B parameters and a 1,024-token context. At 16-bit it needs about 4.1 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 564 downloads a month.
LuminoLex-1.7B is Bangladesh’s first decoder-only language model pretrained from scratch. Built on a sparse Mixture-of-Experts (MoE) architecture, it contains 1.7 billion total parameters while activating only ~410 million parameters per token.
Parameters1.7B
Context1,024
Weights3.4 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads564
Runs On
What it takes to serve LuminoLex-1.7B (1.7B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 3.4 GB | 4.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 1.7 GB | 2.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.8 GB | 1.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 15, 2026.
LuminoLex-1.7B on every accelerator the SAVRN Index prices, at every precision
Model Card
By LuminoLex AI, published under apache-2.0, revision 3f89b77a8a79.
LuminoLex-1.7B is Bangladesh’s first decoder-only language model pretrained from scratch. Built on a sparse Mixture-of-Experts (MoE) architecture, it contains 1.7 billion total parameters while activating only ~410 million parameters per token. By combining an always-on shared expert with Grouped-Query Attention (GQA), LuminoLex-1.7B delivers fast, memory-efficient inference without sacrificing model capacity. This model is released under the Apache License 2.0.
Read LuminoLex AI's full model card
LuminoLex-1.7B is Bangladesh’s first decoder-only language model pretrained from scratch. Built on a sparse Mixture-of-Experts (MoE) architecture, it contains 1.7 billion total parameters while activating only ~410 million parameters per token. By combining an always-on shared expert with Grouped-Query Attention (GQA), LuminoLex-1.7B delivers fast, memory-efficient inference without sacrificing model capacity.
Architecture
| Specification | Value |
|---|---|
| Total Parameters | 1.7B (1,699,863,296) |
| Active Parameters | 410M (409,869,056) |
| Architecture | Sparse MoE (64 experts, top-6 active) |
| Layers | 20 |
| Context Length | 1,024 tokens |
| Precision | bfloat16 |
Quickstart
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "LuminoLex/LuminoLex-1.7B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
trust_remote_code=True,
torch_dtype=torch.bfloat16,
device_map="auto",
)
prompt = "User: What is the capital of Japan?\nAssistant:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.inference_mode():
output = model.generate(
**inputs,
max_new_tokens=128,
do_sample=False,
)
response = tokenizer.decode(
output[0, inputs["input_ids"].shape[-1]:],
skip_special_tokens=True,
)
print(response)
License
This model is released under the Apache License 2.0.
Configuration
- Architecture
- HFLuminoLexForCausalLM
- Context length (tokens)
- 1,024
- Layers
- 20
- Hidden size
- 1,536
- Attention heads
- 24
- Key/value heads
- 8
- Head dimension
- 64
- Vocabulary size
- 32,800
- Experts
- 64
- Experts active per token
- 6
- RoPE base
- 10000
- Stored precision
- bfloat16
- Model type
- luminolex
Identity and Version
- Repository
- LuminoLex/LuminoLex-1.7B
- Publisher
- LuminoLex AI
- Task
- Text generation
- Modality
- Text
- Library
- transformers
- Parameters
- 1.7B parameters
- Languages
- en
- Revision
- 3f89b77a8a797917dc30c6583f927a7822599123
- First published
- 2026-09-24
- Last updated
- 2026-09-27
Files and Weights
12 files, 3.4 GB in total. The weights are 1 file totalling 3.4 GB in safetensors.
Weights1 file · 3.4 GB
Configuration5 files · 31.1 KB
Tokenizer2 files · 2.3 MB
Documentation2 files · 13.1 KB
Other1 file · 358 B
Repository1 file · 50 B
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 3.4 GB | 1972827d41b4 |
| config.json | Configuration | 723 B | — |
| configuration_luminolex.py | Configuration | 7.9 KB | — |
| generation_config.json | Configuration | 113 B | — |
| modeling_luminolex.py | Configuration | 22.2 KB | — |
| special_tokens_map.json | Configuration | 99 B | — |
| LICENSE | Documentation | 11.4 KB | — |
| README.md | Documentation | 1.7 KB | — |
| chat_template.jinja | Other | 358 B | — |
| .gitattributes | Repository | 50 B | — |
| tokenizer.json | Tokenizer | 2.3 MB | — |
| tokenizer_config.json | Tokenizer | 232 B | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 3.4 GB
Download from LuminoLex AI
Released by LuminoLex AI through its official repository on Hugging Face. Read the license.
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 3.4 GB |
| 16-bit | 3.4 GB |
| 8-bit | 1.7 GB |
| 4-bit | 0.8 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About LuminoLex-1.7B
How much GPU memory does LuminoLex-1.7B need?
About 4.1 GB at 16-bit and 1 GB at 4-bit: the weights (1.7B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run LuminoLex-1.7B on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use LuminoLex-1.7B commercially?
Yes. LuminoLex-1.7B is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is LuminoLex-1.7B's context length?
1,024 tokens, from the maximum position embeddings in its published configuration.
Similar Models
The first defense AI reasoning model on Hugging Face. Shepherd-Alpha is a tactical reasoning model fine-tuned on dual-perspective military scenario analysis using BiCell Depth Dispersal — a novel training methodology that partitions transformer layers by abstraction depth and trains them asymmetrically to separate representation encoding from task-specific reasoning. Developed by Convergent Intelligence LLC: Research Division Given a tactical scenario, Shepherd-Alpha produces structured dual-perspective analysis: - Attack reasoning — how an adversary would exploit the situation - Defense reasoning — how to counter, mitigate, and survive The model is trained to think like both attacker and…
Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Building upon extensive advancements in training data, model architecture, and optimization techniques, Qwen3 delivers the following key improvements over the previously released Qwen2.5: Qwen3-1.7B-Base has the following features: For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our blog, GitHub, and Documentation. The code of Qwen3 has been in the latest Hugging Face transformers and we advise you to use the latest version of transformers. With transformers<4.51.0, you will…
DualMind TKD Agentic 1.7B is a two-stage derivative of Qwen/Qwen3-1.7B. It combines topology-guided mathematical knowledge distillation with assistant-masked agentic and function-calling specialization. teacher distillation topology, gap-energy diagnostics, and phase-weighted Explore/Examine/Response supervision Stage 1 was designed to transfer mathematical reasoning behavior while placing additional learning pressure on derivation, verification, and high-discrepancy reasoning transitions. - Tool schemas, user messages, and tool-result messages were visible as context but excluded from direct loss - Mathematical replay was mixed into Stage 2 to reduce catastrophic forgetting The files in…
A masked diffusion language model adapted from Qwen3-1.7B. The backbone is full attention, and every layer is made bidirectional. It is the control model in the paper's matched comparison against the hybrid dQwen3.5-2B. This is a base model, with no instruction tuning. Paper: dQwen3.5: Hybrid-Attention Diffusion Language Models. Code: https://github.com/AntonXue/dQwen Needs a CUDA GPU and transformers>=5.13 (tested with torch 2.7.1+cu128, flash-linear-attention 0.5.1). generate decodes the whole canvas at once, committing positions above a confidence threshold (tau=0.9); pass blocklength=32 for left-to-right block decoding, or tau=None, stepsperblock=k for a fixed budget. The 50B-token…
This model is a fine-tuned version of None. It has been trained using TRL. This model was trained with GRPO, a method introduced in DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.