This is a heavily filtered version of the UltraChat dataset and was used to train Zephyr-7B-β, a state of the art 7b chat model. The original datasets consists of 1.4M dialogues generated by ChatGPT and spanning a wide range of topics.
Dataset Card
By Hugging Face H4, published under mit, revision 8049631c405a.
Dataset Card for UltraChat 200k
Dataset Description
This is a heavily filtered version of the UltraChat dataset and was used to train Zephyr-7B-β, a state of the art 7b chat model.
The original datasets consists of 1.4M dialogues generated by ChatGPT and spanning a wide range of topics. To create UltraChat 200k, we applied the following logic:
- Selection of a subset of data for faster supervised fine tuning.
- Truecasing of the dataset, as we observed around 5% of the data contained grammatical errors like "Hello. how are you?" instead of "Hello. How are you?"
- Removal of dialogues where the assistant replies with phrases like "I do not have emotions" or "I don't have opinions", even for fact-based prompts that don't involve either.
Dataset Structure
The dataset has four splits, suitable for:
- Supervised fine-tuning (
sft). - Generation ranking (
gen) via techniques like rejection sampling or PPO.
The number of examples per split is shown as follows:
Structure
default 515,311 rows
| Split | Rows | Size |
|---|---|---|
| train_sft | 207,865 | 1.4 GB |
| test_sft | 23,110 | 154.2 MB |
| train_gen | 256,032 | 1.3 GB |
| test_gen | 28,304 | 150.3 MB |
Details
- Repository
- HuggingFaceH4/ultrachat_200k
- Publisher
- Hugging Face H4
- Task category
- Text generation
- Tags
- Not stated by the source
- Size category
- 100K<n<1M
- Languages
- en
- Revision
- 8049631c405ae6576f93f445c6b8166f76f5505a
- Last updated
- 2024-10-16
Files
10 files, 1.6 GB in total.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| data/test_gen-00000-of-00001-3d4cd8309148a71f.parquet | Data | 80.4 MB | 937211dbe6e7 |
| data/test_sft-00000-of-00001-f7dfac4afe5b93f4.parquet | Data | 81.2 MB | c18fd6e77395 |
| data/train_gen-00000-of-00003-a6c9fb894be3e50b.parquet | Data | 243.7 MB | 8172417116ff |
| data/train_gen-00001-of-00003-d6a0402e417f35ca.parquet | Data | 243.3 MB | 8b9e9ce202df |
| data/train_gen-00002-of-00003-c0db75b92a2f48fd.parquet | Data | 243.5 MB | 1fe9ba24c1bc |
| data/train_sft-00000-of-00003-a3ecf92756993583.parquet | Data | 244.0 MB | afa8fa742608 |
| data/train_sft-00001-of-00003-0a1804bcb6ae68c6.parquet | Data | 243.9 MB | c8230190bc8b |
| data/train_sft-00002-of-00003-ee46ed25cfae92c6.parquet | Data | 244.1 MB | 6fe7d2a5e95c |
| README.md | Documentation | 3.9 KB | — |
| .gitattributes | Repository | 2.3 KB | — |
License and Download
- License
- mit
- Access
- No access gate
Released by Hugging Face H4 through its official repository on Hugging Face. Read the license.
Models Trained on This Dataset
- Trained on (disclosed)TinyLlama-1.1B-Chat-v1.0
- Trained on (disclosed)Qemma-Q14B
- Trained on (disclosed)Qemma-Q1.7B
- Trained on (disclosed)Qemma-GEI