MLX 6-bit weights with group size 64; a 3.732 GB weight package. Fixed memory for long-running generation. This EdgeRock-4B deployment package retains recurrent state, local attention, and Rock Memory Anchors (RMA), keeping persistent history at a fixed capacity as context grows. 固定记忆，长程生成。 本版本保留 EdgeRock-4B 的递推状态、局部注意力和记忆岩锚（RMA），历史内存不随上下文增长。 Rock Matrix Whitepaper / 岩芯矩阵技术白皮书： whitepaper connects EdgeRock's fixed memory, DiffusionRock's parallel generation, and SiliconRock's model-native silicon into the route toward Ultra-Fast Token. 白皮书讲清固定记忆、并行生成、模型固化三条路线，以及它们如何走向更快、更经济的生成。 The instructions and validation below apply to this package. Browse all EdgeRock models. Original-model benchmarks…
Open-weight model · Text generation
EliteReplay-BIRD-4B
by Hanane Nour Moussa hananour/EliteReplay-BIRD-4B
EliteReplay-BIRD-4B is an open-weight model for text generation from Hanane Nour Moussa, released under Apache License 2.0. It has 4.5B parameters and a 262,144-token context. At 16-bit it needs about 10.9 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 447 downloads a month.
This is a Qwen3.5-4B-Base model fine-tuned on BIRD using EliteReplay. EliteReplay combines reinforcement learning with supervised replay from an archive of verified, diverse SQL solutions to preserve solution diversity during training.
Parameters4.5B
Context262,144
Weights9.1 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads447
Runs On
What it takes to serve EliteReplay-BIRD-4B (4.5B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 9.1 GB | 10.9 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 4.5 GB | 5.4 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 2.3 GB | 2.7 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 23, 2026.
EliteReplay-BIRD-4B on every accelerator the SAVRN Index prices, at every precision
Model Card
By Hanane Nour Moussa, published under apache-2.0, revision 4ddaad0b7c55.
This is a Qwen3.5-4B-Base model fine-tuned on BIRD using EliteReplay. EliteReplay combines reinforcement learning with supervised replay from an archive of verified, diverse SQL solutions to preserve solution diversity during training. We train on 5,000 BIRD questions. Archive search uses 1,000 of those questions to select 1,500 verified SQL examples for replay. The model uses GRPO with supervised archive replay at weight μ = 0.05. Hyperparameters and implementation details are available in the training instructions, original training configuration, and checkpoint metadata. Training used 4 NVIDIA A100 80 GB GPUs, with two for policy optimization and two for rollout generation, using veRL…
Read Hanane Nour Moussa's full model card
This is a Qwen3.5-4B-Base model fine-tuned on BIRD using EliteReplay. EliteReplay combines reinforcement learning with supervised replay from an archive of verified, diverse SQL solutions to preserve solution diversity during training.
Model Details
- Task: Text-to-SQL (SQLite)
- Base model: Qwen/Qwen3.5-4B-Base
- Code: skai-research/EliteReplay
- Paper: EliteReplay: Preserving Diverse Solutions in RL Post-Training via Quality-Diversity Search
- License: Apache 2.0
Training Details
Training Data
We train on 5,000 BIRD questions. Archive search uses 1,000 of those questions to select 1,500 verified SQL examples for replay.
Training Procedure
The model uses GRPO with supervised archive replay at weight μ = 0.05. Hyperparameters and implementation details are available in the training instructions, original training configuration, and checkpoint metadata.
Hardware
Training used 4 NVIDIA A100 80 GB GPUs, with two for policy optimization and two for rollout generation, using veRL, Trinity-RFT, and vLLM.
Usage
See example.py for SQL generation with Transformers 5.8.0. Provide the database schema, question, and optional evidence, with thinking disabled. The example was verified on one A100 40 GB GPU.
Saved BIRD and Spider evaluation results are available in evaluation.json.
Configuration
- Architecture
- Qwen3_5ForConditionalGeneration
- Context length (tokens)
- 262,144
- Layers
- 32
- Hidden size
- 2,560
- Feed-forward size
- 9,216
- Attention heads
- 16
- Key/value heads
- 4
- Head dimension
- 256
- Vocabulary size
- 248,320
- Model type
- qwen3_5
Identity and Version
- Repository
- hananour/EliteReplay-BIRD-4B
- Publisher
- Hanane Nour Moussa
- Task
- Text generation
- Modality
- Text
- Library
- transformers
- Parameters
- 4.5B parameters
- Languages
- en
- Revision
- 4ddaad0b7c55c4b9dfd9e88a2b9b8ad9b2829d31
- First published
- 2026-10-16
- Last updated
- 2026-10-19
Files and Weights
13 files, 9.1 GB in total. The weights are 1 file totalling 9.1 GB in safetensors.
Weights1 file · 9.1 GB
Configuration6 files · 10.0 KB
Tokenizer2 files · 20.0 MB
Documentation2 files · 13.4 KB
Other1 file · 7.8 KB
Repository1 file · 1.6 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 9.1 GB | 55382f664967 |
| config.json | Configuration | 2.9 KB | — |
| evaluation.json | Configuration | 2.1 KB | — |
| example.py | Configuration | 2.1 KB | — |
| generation_config.json | Configuration | 204 B | — |
| processor_config.json | Configuration | 1.2 KB | — |
| training_provenance.json | Configuration | 1.5 KB | — |
| LICENSE | Documentation | 11.3 KB | — |
| README.md | Documentation | 2.0 KB | — |
| chat_template.jinja | Other | 7.8 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| tokenizer.json | Tokenizer | 20.0 MB | 06b9509352d2 |
| tokenizer_config.json | Tokenizer | 1.2 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 9.1 GB
Download from Hanane Nour Moussa
Released by Hanane Nour Moussa through its official repository on Hugging Face. Read the license.
Built From
- Derived from Qwen/Qwen3.5-4B-Base
- Described by arXiv:2610.22290
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 9.1 GB |
| 16-bit | 9.1 GB |
| 8-bit | 4.5 GB |
| 4-bit | 2.3 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About EliteReplay-BIRD-4B
How much GPU memory does EliteReplay-BIRD-4B need?
About 10.9 GB at 16-bit and 2.7 GB at 4-bit: the weights (4.5B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run EliteReplay-BIRD-4B on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use EliteReplay-BIRD-4B commercially?
Yes. EliteReplay-BIRD-4B is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is EliteReplay-BIRD-4B's context length?
262,144 tokens, from the maximum position embeddings in its published configuration.
Similar Models
MLX 4-bit weights with group size 32; a 2.875 GB weight package. Fixed memory for long-running generation. This EdgeRock-4B deployment package retains recurrent state, local attention, and Rock Memory Anchors (RMA), keeping persistent history at a fixed capacity as context grows. 固定记忆，长程生成。 本版本保留 EdgeRock-4B 的递推状态、局部注意力和记忆岩锚（RMA），历史内存不随上下文增长。 Rock Matrix Whitepaper / 岩芯矩阵技术白皮书： whitepaper connects EdgeRock's fixed memory, DiffusionRock's parallel generation, and SiliconRock's model-native silicon into the route toward Ultra-Fast Token. 白皮书讲清固定记忆、并行生成、模型固化三条路线，以及它们如何走向更快、更经济的生成。 The instructions and validation below apply to this package. Browse all EdgeRock models. Original-model benchmarks…
MLX 8-bit weights with group size 64; a 4.874 GB weight package. Fixed memory for long-running generation. This EdgeRock-4B deployment package retains recurrent state, local attention, and Rock Memory Anchors (RMA), keeping persistent history at a fixed capacity as context grows. 固定记忆，长程生成。 本版本保留 EdgeRock-4B 的递推状态、局部注意力和记忆岩锚（RMA），历史内存不随上下文增长。 Rock Matrix Whitepaper / 岩芯矩阵技术白皮书： whitepaper connects EdgeRock's fixed memory, DiffusionRock's parallel generation, and SiliconRock's model-native silicon into the route toward Ultra-Fast Token. 白皮书讲清固定记忆、并行生成、模型固化三条路线，以及它们如何走向更快、更经济的生成。 The instructions and validation below apply to this package. Browse all EdgeRock models. Original-model benchmarks…
MLX mixed precision: 4-bit MLPs and 8-bit other eligible matrices, with group size 64; 3.741 GB of weights. Fixed memory for long-running generation. This EdgeRock-4B deployment package retains recurrent state, local attention, and Rock Memory Anchors (RMA), keeping persistent history at a fixed capacity as context grows. 固定记忆，长程生成。 本版本保留 EdgeRock-4B 的递推状态、局部注意力和记忆岩锚（RMA），历史内存不随上下文增长。 Rock Matrix Whitepaper / 岩芯矩阵技术白皮书： whitepaper connects EdgeRock's fixed memory, DiffusionRock's parallel generation, and SiliconRock's model-native silicon into the route toward Ultra-Fast Token. 白皮书讲清固定记忆、并行生成、模型固化三条路线，以及它们如何走向更快、更经济的生成。 The instructions and validation below apply to this package. Browse all…
AWQ W4A16 weights for Rock Matrix's EdgeRock-4B, with a packed INT4 CUDA runtime. Fixed memory for long-running generation. This EdgeRock-4B deployment package retains recurrent state, local attention, and Rock Memory Anchors (RMA), keeping persistent history at a fixed capacity as context grows. 固定记忆，长程生成。 本版本保留 EdgeRock-4B 的递推状态、局部注意力和记忆岩锚（RMA），历史内存不随上下文增长。 Rock Matrix Whitepaper / 岩芯矩阵技术白皮书： whitepaper connects EdgeRock's fixed memory, DiffusionRock's parallel generation, and SiliconRock's model-native silicon into the route toward Ultra-Fast Token. 白皮书讲清固定记忆、并行生成、模型固化三条路线，以及它们如何走向更快、更经济的生成。 EdgeRock combines recurrent state, sliding-window attention and Rock Memory Anchors (RMA) to…
GPTQ W4A16 weights for Rock Matrix's EdgeRock-4B, with a packed INT4 CUDA runtime. Fixed memory for long-running generation. This EdgeRock-4B deployment package retains recurrent state, local attention, and Rock Memory Anchors (RMA), keeping persistent history at a fixed capacity as context grows. 固定记忆，长程生成。 本版本保留 EdgeRock-4B 的递推状态、局部注意力和记忆岩锚（RMA），历史内存不随上下文增长。 Rock Matrix Whitepaper / 岩芯矩阵技术白皮书： whitepaper connects EdgeRock's fixed memory, DiffusionRock's parallel generation, and SiliconRock's model-native silicon into the route toward Ultra-Fast Token. 白皮书讲清固定记忆、并行生成、模型固化三条路线，以及它们如何走向更快、更经济的生成。 EdgeRock combines recurrent state, sliding-window attention and Rock Memory Anchors (RMA) to…