Further cleaning done. Please look through the dataset and ensure that I didn't miss anything.
Dataset Card
By Z., published under apache-2.0, revision 192ab2185289.
Further cleaning done. Please look through the dataset and ensure that I didn't miss anything.
Update: Confirmed working method for training the model: https://huggingface.co/AlekseyKorshuk/vicuna-7b/discussions/4#64346c08ef6d5abefe42c12c Two choices:
- Removes instances of "I'm sorry, but": https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/blob/main/ShareGPT_V3_unfiltered_cleaned_split_no_imsorry.json
- Has instances of "I'm sorry, but": https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/blob/main/ShareGPT_V3_unfiltered_cleaned_split.json
The choice is yours. The first dataset may go to far and remove valuable data. The second is better for when the AI asks for clarification, but it also may refuse to do stuff like browse the internet, which it actually may be able to do with certain langchain implementations. These are important things to think about before training.
Details
- Repository
- anon8231489123/ShareGPT_Vicuna_unfiltered
- Publisher
- Z.
- Task category
- Not stated by the source
- Tags
- Not stated by the source
- Size category
- Not stated by the source
- Languages
- en
- Revision
- 192ab2185289094fc556ec8ce5ce1e8e587154ca
- Last updated
- 2023-04-12
Files
14 files, 4.3 GB in total.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| HTML_cleaned_raw_dataset/sg_90k_part1.json | Data | 921.6 MB | ed0e3af68ed9 |
| HTML_cleaned_raw_dataset/sg_90k_part1_html_cleaned.json | Data | 551.4 MB | 183c744455c6 |
| HTML_cleaned_raw_dataset/sg_90k_part2.json | Data | 932.3 MB | d74dfddfedc6 |
| HTML_cleaned_raw_dataset/sg_90k_part2_html_cleaned.json | Data | 548.2 MB | 32c3e772d044 |
| ShareGPT_V3_unfiltered_cleaned_split.json | Data | 672.8 MB | 35f0e213ce09 |
| ShareGPT_V3_unfiltered_cleaned_split_no_imsorry.json | Data | 670.5 MB | 014bcc3352fd |
| README.md | Documentation | 4.4 KB | — |
| Vicuna_unfiltered_train.ipynb | Other | 4.5 KB | — |
| clean_sharegpt.py | Other | 4.6 KB | — |
| flash_attn-0.2.8-cp39-cp39-linux_x86_64.whl | Other | 49.9 MB | ea4c106e475c |
| optional_clean.py | Other | 4.5 KB | — |
| pretty_json.py | Other | 475 B | — |
| split_long_conversation.py | Other | 3.4 KB | — |
| .gitattributes | Repository | 3.0 KB | — |
License and Download
- License
- apache-2.0
- Access
- No access gate
Released by Z. through its official repository on Hugging Face. Read the license.