This model is currently still in post-training and is pre-released as a preview checkpoint with gated access only.
Open-weight model · Text generation
PMA-1.3-Rosa-256M
by PatriotMemory-AI patriotmemory-ai/PMA-1.3-Rosa-256M
PMA-1.3-Rosa-256M is an open-weight model for text generation from PatriotMemory-AI, released under Apache License 2.0. It has 262M parameters and a 4,096-token context. At 16-bit it needs about 0.6 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
Rosa is Patriot Memory's 253M-parameter edge assistant for English and Traditional Chinese — built to fit the edge devices you actually ship, with a vocabulary trained natively on Traditional Chinese.
Parameters262M
Context4,096
Weights1.0 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads—
Runs On
What it takes to serve PMA-1.3-Rosa-256M (262M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.5 GB | 0.6 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.3 GB | 0.3 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.1 GB | 0.2 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 7, 2026.
PMA-1.3-Rosa-256M on every accelerator the SAVRN Index prices, at every precision
Model Card
By PatriotMemory-AI, published under apache-2.0, revision be67d7289586.
Rosa is Patriot Memory's 253M-parameter edge assistant for English and Traditional Chinese — built to fit the edge devices you actually ship, with a vocabulary trained natively on Traditional Chinese. It provides accurate information regarding: If you run into multi-GPU tensor device mismatch errors: RuntimeError: Expected all tensors to be on the same device... Run the script with CUDAVISIBLEDEVICES=0 to isolate execution to GPU 0. Good: introducing herself in both languages; answering arithmetic word problems with visible English reasoning steps; clean Traditional Chinese — glyphs, vocabulary and register; running fully offline on edge hardware. Not: her reasoning traces are formatted…
Read PatriotMemory-AI's full model card
Model Overview
Rosa is Patriot Memory's 253M-parameter edge assistant for English and Traditional Chinese — built to fit the edge devices you actually ship, with a vocabulary trained natively on Traditional Chinese. It provides accurate information regarding:
- DDR4 & DDR5 RAM: Specifications, XMP 3.0 / EXPO profile support, dual-channel setups, and overclocking guidance.
- PCIe & SATA SSDs: Gen3/Gen4/Gen5 compatibility, read/write performance specifications, and installation troubleshooting.
- Gaming Peripherals & Storage: USB drives, flash cards, and Viper Gaming gear.
- Tool / Function Calling: Seamless integration with backend APIs (e.g., checking warranty status, looking up technical specs via S/N).
Architecture
| Parameters | 253,283,329 (253M-class; fits 256M edge budget) |
| Memory | ~1.0 GB fp32 master weights (cast to fp16 at load for ~500 MB) |
| Architecture | PMA spine — 19 layers, hidden 1024, GQA 8q/2kv, gated attention output, value residuals, Norm-Head output, tied embeddings |
| Tokenizer | OWN SentencePiece 8k unigram, trained on Traditional-Chinese + English corpus — Traditional-exclusive glyphs encode as real pieces |
| Context | 4096 positions (2048-token training windows) |
| Reasoning | English rationale format: Reasoning: … Answer: … |
| Training | 2.3B-token adapted pretrain on the TW tokenizer (warm-started), SFT with identity/format/rationale pools + targeted repair pass |
Quickstart
pip install -U torch transformers==4.51.0 accelerate sentencepiece protobuf
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
name = "patriotmemory-ai/PMA-1.3-Rosa-256M"
tok = AutoTokenizer.from_pretrained(name,trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
name, torch_dtype=torch.float16, device_map="auto",
trust_remote_code=True)
msgs = [{"role": "user", "content": "博帝的 Viper DDR5 支援 XMP 3.0 嗎?"}]
ids = tok.apply_chat_template(msgs, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=200)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
Decode: temperature 0.8, top_p 0.9, repetition_penalty 1.1
If you run into multi-GPU tensor device mismatch errors: RuntimeError: Expected all tensors to be on the same device... Run the script with CUDA_VISIBLE_DEVICES=0 to isolate execution to GPU 0.
Acceptance gates (measured, 3-sample pooled)
| Gate | Result |
|---|---|
| A — story shape (EN + zh-TW) | 29/30 |
| B — number format | 27/30 |
| D — identity + injection defense | 18/18 |
F — Reasoning: … Answer: rationale format |
27/30 |
| T — Traditional-Chinese purity (all zh generations) | 36/36 |
What she's good at / not good at
Good: introducing herself in both languages; answering arithmetic word problems with visible English reasoning steps; clean Traditional Chinese — glyphs, vocabulary and register; running fully offline on edge hardware.
Not: her reasoning traces are formatted correctly ~90% of the time but at 253M the arithmetic inside them is occasionally wrong-but-confident — treat the steps as display, verify the numbers. Long open-ended creative Chinese can drift into polite deflection, and unusual open-ended generation tasks (draw-me-this, write-me-that in novel domains) can loop in paraphrase instead of producing the artifact — not for long-document streaming.
Research preview: draw
One extra SVG-taught pass — generates valid SVG directly; these are its unretouched outputs, source files included in images/:
Source: images/pelican_bike.svg, images/pelican.svg, images/bicycle.svg, images/cat.svg.
Official Links
Official Website: patriotmemory.com
Viper Gaming: viper.patriotmemory.com
Support & Warranty: patriotmemory.com/support
ACPI Technology: acpitechnology.com
Model Inquiries & Feedback: danton.chu hunter.wang oda.chang [email protected]
License & attribution
Apache-2.0. Built by Patriot Memory (patriotmemory.com).
Configuration
- Architecture
- PMA3ForCausalLM
- Context length (tokens)
- 4,096
- Layers
- 19
- Hidden size
- 1,024
- Feed-forward size
- 3,000
- Attention heads
- 8
- Key/value heads
- 2
- Vocabulary size
- 8,192
- RoPE base
- 1e+06
- Model type
- pma3
Identity and Version
- Repository
- patriotmemory-ai/PMA-1.3-Rosa-256M
- Publisher
- PatriotMemory-AI
- Task
- Text generation
- Modality
- Text
- Library
- transformers
- Parameters
- 262M parameters
- Languages
- zh, en
- Revision
- be67d7289586a857858e09e69626e4d156510b12
- First published
- 2026-09-29
- Last updated
- 2026-09-30
Files and Weights
17 files, 1.0 GB in total. The weights are 1 file totalling 1.0 GB in safetensors.
Weights1 file · 1.0 GB
Configuration4 files · 12.7 KB
Tokenizer2 files · 348.8 KB
Documentation1 file · 4.6 KB
Other8 files · 89.9 KB
Repository1 file · 1.5 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 1.0 GB | a7fd02e43795 |
| config.json | Configuration | 537 B | — |
| generation_config.json | Configuration | 214 B | — |
| model_pma3.py | Configuration | 10.0 KB | — |
| tokenization_pma3.py | Configuration | 1.9 KB | — |
| README.md | Documentation | 4.6 KB | — |
| chat_template.jinja | Other | 157 B | — |
| images/architecture.svg | Other | 9.5 KB | — |
| images/bicycle.svg | Other | 758 B | — |
| images/cat.svg | Other | 481 B | — |
| images/logo.png | Other | 19.2 KB | — |
| images/pelican.svg | Other | 482 B | — |
| images/pelican_bike.svg | Other | 756 B | — |
| images/pelican_gallery.png | Other | 58.6 KB | — |
| .gitattributes | Repository | 1.5 KB | — |
| tokenizer.model | Tokenizer | 348.3 KB | 5fcd308745e7 |
| tokenizer_config.json | Tokenizer | 492 B | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 1.0 GB
Download from PatriotMemory-AI
Released by PatriotMemory-AI through its official repository on Hugging Face. Read the license.
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 1.0 GB |
| 16-bit | 0.5 GB |
| 8-bit | 0.3 GB |
| 4-bit | 0.1 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About PMA-1.3-Rosa-256M
How much GPU memory does PMA-1.3-Rosa-256M need?
About 0.6 GB at 16-bit and 0.2 GB at 4-bit: the weights (262M parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run PMA-1.3-Rosa-256M on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use PMA-1.3-Rosa-256M commercially?
Yes. PMA-1.3-Rosa-256M is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is PMA-1.3-Rosa-256M's context length?
4,096 tokens, from the maximum position embeddings in its published configuration.
Similar Models
This model is a fine-tuned derivative of google/gemma-3-270m, adapted using the Convergent Intelligence sparse fine-tuning setup originally tested on Liquid Foundation Models. The checkpoint was trained on reasoning-style English examples from angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k using a targeted adaptation strategy and the custom CIxOpt optimizer framework. The goal of this model is to test whether a compact Gemma 3 270M backbone can be shaped toward reasoning-style text generation through selective parameter participation rather than broad full-model modification. This is an experimental research checkpoint intended for evaluation, local testing, optimizer research, and…
This is the model card of a transformers model that has been pushed on the Hub. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
Results on demo model (different generation method, one model per language): Keywords generated with vlT5-base-keywords: encoder-decoder architecture, vlT5, keyword generation, scientific articles corpus The biggest advantage is the transferability of the vlT5 model, as it works well on all domains and types of text. The downside is that the text length and the number of keywords are similar to the training data: the text piece of an abstract length generates approximately 3 to 5 keywords. It works both extractive and abstractively. Longer pieces of text must be split into smaller chunks, and then propagated to the model. The model was trained on a POSMAC corpus. Polish Open Science…
The Pythia Scaling Suite is a collection of models developed to facilitate interpretability research (see paper). It contains two sets of eight models of sizes 70M, 160M, 410M, 1B, 1.4B, 2.8B, 6.9B, and 12B. For each size, there are two models: one trained on the Pile, and one trained on the Pile after the dataset has been globally deduplicated. All 8 model sizes are trained on the exact same data, in the exact same order. We also provide 154 intermediate checkpoints per model, hosted on Hugging Face as branches. The Pythia model suite was deliberately designed to promote scientific research on large language models, especially interpretability research. Despite not centering downstream…
Sanskrit-322M-Base is a 322-million-parameter Sanskrit language model trained from scratch with its own 32,000-token SentencePiece Unigram tokenizer. It is intended for Sanskrit text completion, generation experiments, and further task-specific fine-tuning. This is a base pretrained model: it predicts the continuation of text. It has not been instruction-tuned or trained as a conversational assistant. It uses a Llama-compatible decoder architecture; its weights were initialized and trained independently, rather than adapted from a pretrained Llama checkpoint. Published by sraivante under the Apache License 2.0. The tokenizer uses identity normalization, byte fallback, and SentencePiece…