ALEA Institute

Model · Text generation

alea-r4-3B-A0.7B-Base-ckpts

ALEA Institute

Qwen3-MoE architecture; 3.32B parameters in total (3,315,557,632, tied input/output embeddings), 673M active per token. These are intermediate base-model pretraining checkpoints, not instruction-tuned. Safety and behaviour status: pending final checks. They are published for research: to follow what the model learns over training and to load any checkpoint with stock transformers. They are not a release. R4 is ALEA's next model after R3, trained only on lawful data: U.S. legal, regulatory and financial public-record text from the KL3M Data Project, plus synthetic, dictionary, knowledge and task data created by ALEA. main holds only this card. Each checkpoint is its own branch; load one with…

Dataset

kl3m-data-dotgov-www.huduser.gov

ALEA Institute

Version 1.1 replaces the text of the HTML pages in this dataset with a new extraction made with kaos-web and kaos-content. kaos-web picks the main content region of each page (readability), and kaos-content converts that region to a document tree, which is serialized as plain text. Pages are re-extracted from the same crawl captures that version 1 used. What changed - Where 10 or more pages on the same site received identical new text although their version 1 texts differed, those pages keep their version 1 text (52 rows). Blind review. Reviewers compared the new text with the earlier KL3M text, as cleaned for training, on 100 randomly drawn.gov pages from pages that passed the earlier…

Dataset

kl3m-data-dotgov-www.imls.gov

ALEA Institute

Version 1.1 replaces the text of the HTML pages in this dataset with a new extraction made with kaos-web and kaos-content. kaos-web picks the main content region of each page (readability), and kaos-content converts that region to a document tree, which is serialized as plain text. Pages are re-extracted from the same crawl captures that version 1 used. What changed Blind review. Reviewers compared the new text with the earlier KL3M text, as cleaned for training, on 100 randomly drawn.gov pages from pages that passed the earlier training filters, without knowing which text was which. Reviewers preferred the new text 68 times and the earlier text 27 times; 5 were ties. Damage per 1,000…

Dataset

kl3m-data-dotgov-www.ice.gov

ALEA Institute

Version 1.1 replaces the text of the HTML pages in this dataset with a new extraction made with kaos-web and kaos-content. kaos-web picks the main content region of each page (readability), and kaos-content converts that region to a document tree, which is serialized as plain text. Pages are re-extracted from the same crawl captures that version 1 used. What changed Blind review. Reviewers compared the new text with the earlier KL3M text, as cleaned for training, on 100 randomly drawn.gov pages from pages that passed the earlier training filters, without knowing which text was which. Reviewers preferred the new text 68 times and the earlier text 27 times; 5 were ties. Damage per 1,000…

Dataset

kl3m-data-dotgov-www.ignet.gov

ALEA Institute

Version 1.1 replaces the text of the HTML pages in this dataset with a new extraction made with kaos-web and kaos-content. kaos-web picks the main content region of each page (readability), and kaos-content converts that region to a document tree, which is serialized as plain text. Pages are re-extracted from the same crawl captures that version 1 used. What changed - 2 rows were withheld from this version because they contain a personal identification number. Blind review. Reviewers compared the new text with the earlier KL3M text, as cleaned for training, on 100 randomly drawn.gov pages from pages that passed the earlier training filters, without knowing which text was which. Reviewers…