This is the official pretraining dataset for the TinyDTVi-500M causal language model. The corpus is aggressively filtered and deduplicated to maximize the representation of standard Vietnamese text for training resource-constrained models. The initial raw dataset (~135GB) was curated by mixing three high-quality open-source Vietnamese corpora. The approximate mixture ratio is: - 50% CulturaX-Vi (Cleaned web crawl) - 30% VNews (Vietnamese News Corpus) - 20% Vietnamese Wikipedia (Encyclopedia) After strict preprocessing and deduplication, the final dataset size is approximately 85GB (yielding roughly 18 billion unique tokens). To compress the vocabulary down to 50k tokens and focus strictly…
Independent publisher
Hoàng Duy
hoangduy02071997
Machine Learning, Deep Learning
Models in Library0
Datasets in Library1
Models on Hugging Face1
Followers—
Compare the published evidence
This profile connects the 1 model and dataset record shown below to their upstream cards and sources. The library counts describe SAVRN's coverage, rather than this publisher's entire output.
Published tasks: Text generation.
Reported licenses: mit. Open a record to review its specific license and conditions.
Compare model specifications, source dates, and hardware estimates on the individual records. Publisher identity verification is separate from a model's quality, deployment readiness, or suitability for your work.