What is Symato CC? To download all WARC data from Common Crawl then filter out Vietnamese in Markdown and Plaintext format. There is 1% of Vietnamse in CC, extract all of them out should be a lot (~10TB of plaintext).
Dataset Card
What is Symato CC? To download all WARC data from Common Crawl then filter out Vietnamese in Markdown and Plaintext format. There is 1% of Vietnamse in CC, extract all of them out should be a lot (~10TB of plaintext). Main contributors https://huggingface.co/nampdn-ai https://huggingface.co/binhvq https://huggingface.co/th1nhng0 https://huggingface.co/iambestfeed Simple quality filters To make use of raw data from common crawl, you need to do filtering
Excerpt from the card by Symato Team, licensed mit.
Details
- Repository
- Symato/cc
- Publisher
- Symato Team
- Task category
- Not stated by the source
- Tags
- Not stated by the source
- Size category
- 1K<n<10K
- Languages
- vi
- Revision
- c0e1add8a2af3bcdf3ca481d91f6640fb3db7eba
- Last updated
- 2023-07-11
License and Download
- License
- mit
- Access
- No access gate
Released by Symato Team through its official repository on Hugging Face. Read the license.