The WikiText language modeling dataset is a collection of over 100 million tokens extracted from the set of verified Good and Featured articles on Wikipedia. The dataset is available under the Creative Commons Attribution-ShareAlike License. Compared to the preprocessed version of Penn Treebank (PTB), WikiText-2 is over 2 times larger and WikiText-103 is over 110 times larger. The WikiText dataset also features a far larger vocabulary and retains the original case, punctuation and numbers - all of which are removed in PTB. As it is composed of full articles, the dataset is well suited for models that can take advantage of long term dependencies. Each subset comes in two different variants…
Publicly accessible
cc-by-sa-3.0
1M<n<10M
Dataset · Text generation
Ai2
A colossal, cleaned version of Common Crawl's web crawl corpus. Based on Common Crawl dataset: "https://commoncrawl.org". This is the processed version of Google's C4 dataset We prepared five variants of the data: en, en.noclean, en.noblocklist, realnewslike, and multilingual (mC4). For reference, these are the sizes of the variants: - en.noclean: 2.3TB - en.noblocklist: 380GB - realnewslike: 15GB - multilingual (mC4): 9.7TB (108 subsets, one per language) The en.noblocklist variant is exactly the same as the en variant, except we turned off the so-called "badwords filter", which removes all documents that contain words from the lists at…
Publicly accessible
odc-by
n<1K
Dataset · Text generation
OpenAI
GSM8K (Grade School Math 8K) is a dataset of 8.5K high quality linguistically diverse grade school math word problems. The dataset was created to support the task of question answering on basic mathematical problems that require multi-step reasoning. - These problems take between 2 and 8 steps to solve. - Solutions primarily involve performing a sequence of elementary calculations using basic arithmetic operations (+ − ×÷) to reach the final answer. - A bright middle school student should be able to solve every problem: from the paper, "Problems require no concepts beyond the level of early Algebra, and the vast majority of problems can be solved without explicitly defining a variable."…
Publicly accessible
mit
1K<n<10K
FineWeb-Edu dataset consists of 1.3T tokens and 5.4T tokens (FineWeb-Edu-score-2) of educational web pages filtered from FineWeb dataset. This is the 1.3 trillion version. To enhance FineWeb's quality, we developed an educational quality classifier using annotations generated by LLama3-70B-Instruct. We then used this classifier to retain only the most educational web pages. FineWeb-Edu outperforms FineWeb on popular benchmarks and shows the power of classifiers trained on synthetic data. The Dataset Curation section details the process for creating the dataset. You can find a deduplicated version of FineWeb-edu in SmolLM-Corpus. We find that the deduplication of this dataset doesn't have…
Publicly accessible
odc-by
n>1T
The FineWeb dataset consists of more than 18.5T tokens (originally 15T tokens) of cleaned and deduplicated english web data from CommonCrawl. The data processing pipeline is optimized for LLM performance and ran on the datatrove library, our large scale data processing library. FineWeb was originally meant to be a fully open replication of RefinedWeb, with a release of the full dataset under the ODC-By 1.0 license. However, by carefully adding additional filtering steps, we managed to push the performance of FineWeb well above that of the original RefinedWeb, and models trained on our dataset also outperform models trained on other commonly used high quality web datasets (like C4…
Publicly accessible
odc-by
n>1T
Dataset · Text generation
Google
This dataset contains the prompts used in the Instruction-Following Eval (IFEval) benchmark for large language models. It contains around 500 "verifiable instructions" such as "write in more than 400 words" and "mention the keyword of AI at least 3 times" which can be verified by heuristics. To load the dataset, run: The IFEval dataset is designed for evaluating chat or instruction fine-tuned language models and is one of the core benchmarks used in the Open LLM Leaderboard. The data in IFEval are in English (BCP-47 en). An example of the train split looks as follows: The data fields are as follows: key: A unique ID for the prompt. prompt: Describes the task the model should perform.…
Publicly accessible
apache-2.0
Synthetic data generated by DataTrove: The finalized run produced 1,354,044,711 (≈1.35B) samples and generated 486,367,076,933 (≈486.4B) completion tokens. Final counts were computed from generated parquet outputs using examples/inference/countcompletiontokens.py and the runs in projects/datatrove/finephrasetokencounts//slurm/stats.json. Each sample includes standard fields such as: - text (source input text from FineWeb-Edu, not the generated output) - rolloutresults (list of generation result objects; one per rollout) - finishreason - text (generated transformed output; for single-rollout runs this is in rolloutresults[0].text) - usage - completiontokens - prompttokens…
Publicly accessible
odc-by
n>1M
This dataset contains the pre-training phase data used to train all Ettin encoder and decoder models. The data is provided in MDS format ready for use with Composer and the ModernBERT training repository. For pre-training, see the ModernBERT repo: https://github.com/AnswerDotAI/ModernBERT Each folder contains one data source in MDS (Mosaic Data Shard) format: - arxiv/ - Academic papers from ArXiv - books/ - Literature and reference books - cchead/ - High-quality Common Crawl documents - ccnews/ - News articles from Common Crawl - dclm/ - DataComp-LM filtered web data - openwebmath/ - Mathematical web content - algebraicstackexchange/ - Math Q&A from StackExchange - pes2o/ - Scientific…
Publicly accessible
mit
FineWeb-HQ is a high-quality, model-filtered pretraining dataset derived as a subset of FineWeb. FineWeb-HQ was created by selecting the top 10% of FineWeb documents based on a deep learning classifier trained to identify structured and knowledge-rich samples. This classifier uses XLM-RoBERTa embeddings to score documents. To validate our approach, we pretrained 1B-parameter LLM models with a Llama-like architecture across multiple languages and scripts. The results showed improvements on standard English benchmarks, with our dataset outperforming its English counterparts DCLM and FineWeb-Edu. For its multilingual version, FineWeb2-HQ, evaluations on CMMLU (Chinese), MMLU (German), and MMLU…
Publicly accessible
odc-by
n>1T
Wikipedia dataset containing cleaned articles of all languages. The dataset is built from the Wikipedia dumps (https://dumps.wikimedia.org/) with one subset per language, each containing a single train split. Each example contains the content of one full Wikipedia article with cleaning to strip markdown and unwanted sections (references, etc.). All language subsets have already been processed for recent dump, and you can load them per date and language this way: Click the Nomic Atlas map below to visualize the 6.4 million samples in the 20231101.en split. The dataset is generally used for Language Modeling. You can find the list of languages here…
Publicly accessible
cc-by-sa-3.0
n<1K
This dataset contains a subset of 500 problems from the MATH benchmark that OpenAI created in their Let's Verify Step by Step paper. See their GitHub repo for the source file: https://github.com/openai/prm800k/tree/main?tab=readme-ov-file#math-splits
Publicly accessible
[12/14/2025] NOTE: We will no longer actively update this dataset. While this dataset is still functional and usable, we recommend you use the SWE-bench/SWE-smith-[lang] datasets. For better maintainability and ease-of-use, we are maintaining language-specific datasets in lieu of this mono-repo. The SWE-smith Dataset is a training dataset of 50137 task instances from 128 GitHub repositories, collected using the SWE-smith toolkit. It is the largest dataset to date for training software engineering agents. All SWE-smith task instances come with an executable environment. To learn more about how to use this dataset to train Language Models for Software Engineering, please refer to the…
Publicly accessible
mit
10K<n<100K
The Stack v3 is the largest, most up-to-date open dataset of source code, crawled directly from GitHub and built to pre-train code LLMs with full-repository context. It is the successor to The Stack v2 and, like its predecessor, is released to make the training of code models more open, reproducible, and transparent. If your code is included in this version and you would like it removed for any reason, please use the "Am I in The Stack?" Space to submit a removal request. Repositories that opted out are removed from the dataset before each patch release. Two things set v3 apart from earlier releases of The Stack: 1. The file contents are included inline. The decoded UTF-8 source text is…
Publicly accessible
odc-by
100M<n<1B
Approximately 860k math problems, where each solution is formatted in a Chain of Thought (CoT) manner. The sources of the dataset range from Chinese high school math exercises to US and international mathematics olympiad competition problems. The data were primarily collected from online exam paper PDFs and mathematics discussion forums. The processing steps include (a) OCR from the original PDFs, (b) segmentation into problem-solution pairs, (c) Translation into English, (d) realignment to produce a CoT reasoning format, and (e) final answer formatting. The dataset is available under the Apache License, Version 2.0.
Publicly accessible
apache-2.0
We introduce MegaMath, an open math pretraining dataset curated from diverse, math-focused sources, with over 300B tokens. MegaMath is curated via the following three efforts: We re-extracted mathematical documents from Common Crawl with math-oriented HTML optimizations, fasttext-based filtering and deduplication, all for acquiring higher-quality data on the Internet. We identified high quality math-related code from large code training corpus, Stack-V2, further enhancing data diversity. We synthesized QA-style text, math-related code, and interleaved text-code blocks from web data or code data. MegaMath is the largest open math pre-training dataset to date, surpassing DeepSeekMath (120B)…
Publicly accessible
odc-by
1B<n<10B
The dataset is a collection of simple math word problems focused on arithmetics. It is derived from. The main addition in this dataset variant is the chain column. It was created by converting the solution to a simple html-like language that can be easily parsed (e.g. by BeautifulSoup). The data contains 3 types of tags: This variant of the dataset is intended for training Chain-of-Thought reasoning models able to use external tools to enhance the factuality of their responses. This dataset presents in-context scenarios where models can outsource the computations in the reasoning chain to a calculator. We provide 2 variants of the dataset. In the first one, the data splits correspond to the…
Publicly accessible
mit
1K<n<10K
Alpaca is a dataset of 52,000 instructions and demonstrations generated by OpenAI's text-davinci-003 engine. This instruction data can be used to conduct instruction-tuning for language models and make the language model follow instruction better. The authors built on the data generation pipeline from Self-Instruct framework and made the following modifications: - The text-davinci-003 engine to generate the instruction data instead of davinci. - A new prompt was written that explicitly gave the requirement of instruction generation to text-davinci-003. - Much more aggressive batch decoding was used, i.e., generating 20 instructions at once, which significantly reduced the cost of data…
Publicly accessible
cc-by-nc-4.0
Dataset · Text generation
Nebius
SWE-rebench-V2 is a curated dataset of software-engineering tasks derived from real GitHub issues and pull requests. The dataset contains 32,079 samples covering Python, Go, TypeScript, JavaScript, Rust, Java, PHP, Kotlin, Julia, Elixir, Scala, Swift, Dart, C, C++, C#, R, Clojure, OCaml, and Lua. For log parser functions, base Dockerfiles, and the prompts used, please see https://github.com/SWE-rebench/SWE-rebench-V2 The detailed technical report is available at “SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale”. The dataset is licensed under the Creative Commons Attribution 4.0 license. However, please respect the license of each specific repository on which a particular…
Publicly accessible
cc-by-4.0
This is a heavily filtered version of the UltraChat dataset and was used to train Zephyr-7B-β, a state of the art 7b chat model. The original datasets consists of 1.4M dialogues generated by ChatGPT and spanning a wide range of topics. To create UltraChat 200k, we applied the following logic: - Selection of a subset of data for faster supervised fine tuning. - Truecasing of the dataset, as we observed around 5% of the data contained grammatical errors like "Hello. how are you?" instead of "Hello. How are you?" - Removal of dialogues where the assistant replies with phrases like "I do not have emotions" or "I don't have opinions", even for fact-based prompts that don't involve either. The…
Publicly accessible
mit
100K<n<1M
This dataset contains problems from the American Invitational Mathematics Examination (AIME) 2024. AIME is a prestigious high school mathematics competition known for its challenging mathematical problems. Each record contains the following fields: - ID: Problem identifier (e.g., "2024-I-1" represents Problem 1 from 2024 Contest I) - Problem: Problem statement - Solution: Detailed solution process - Answer: Final numerical answer This dataset is primarily used for: 1. Evaluating Large Language Models' (LLMs) mathematical reasoning capabilities 2. Testing models' problem-solving abilities on complex mathematical problems 3. Researching AI performance on structured mathematical tasks - Covers…
Publicly accessible
mit
n<1K
English | Ultra-FineWeb is a large-scale, high-quality, and efficiently-filtered dataset. We use the proposed efficient verification-based high-quality filtering pipeline to the FineWeb and Chinese FineWeb datasets (source data from Chinese FineWeb-edu-v2, which includes IndustryCorpus2, MiChao, WuDao, SkyPile, WanJuan, ChineseWebText, TeleChat, and CCI3), resulting in the creation of higher-quality Ultra-FineWeb-en with approximately 1T tokens, and Ultra-FineWeb-zh datasets with approximately 120B tokens, collectively referred to as Ultra-FineWeb. Ultra-FineWeb serves as a core pre-training web dataset for the MiniCPM4 Series and MiniCPM5 Series models. - Ultra-FineWeb-L1: L1 filtered data…
Publicly accessible
apache-2.0
n>1T
Falcon RefinedWeb is a massive English web dataset built by TII and released under an ODC-By 1.0 license. See the paper on arXiv for more details. RefinedWeb is built through stringent filtering and large-scale deduplication of CommonCrawl; we found models trained on RefinedWeb to achieve performance in-line or better than models trained on curated datasets, while only relying on web data. RefinedWeb is also "multimodal-friendly": it contains links and alt texts for images in processed samples. This public extract should contain 500-650GT depending on the tokenizer you use, and can be enhanced with the curated corpora of your choosing. This public extract is about ~500GB to download…
Publicly accessible
odc-by
100B<n<1T
Dataset containing synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary. The models referred to in the paper were trained on TinyStories-train.txt (the file tinystories-valid.txt can be used for validation loss). These models can be found on Huggingface, at roneneldan/TinyStories-1M/3M/8M/28M/33M/1Layer-21M. tinystoriesalldata.tar.gz - contains a superset of the stories together with metadata and the prompt that was used to create each story. TinyStoriesV2-GPT4-train.txt - Is a new version of the dataset that is based on generations by GPT-4 only (the original dataset also has generations by GPT-3.5 which are of lesser quality). It contains all the…
Publicly accessible
cdla-sharing-1.0
N
Dataset · Text generation
Nils
Completely uncurated collection of IRC logs from the Ubuntu IRC channels
Publicly accessible
cc0-1.0
arXiv Papers by Subject A reorganised version of the nick007x/arxiv-papers dataset, partitioned by subject code, year, and month for efficient selective access. This dataset contains metadata for over 2.5 million arXiv papers, organised into a hierarchical directory structure that allows users to download only the specific subjects and time periods they need, rather than the entire dataset. Motivation The original
Publicly accessible
mit
1M<n<10M
1,000 examples from https://huggingface.co/datasets/llamafactory/alpacagpt4en - 1,000 examples from https://huggingface.co/datasets/llamafactory/alpacagpt4zh - 300 examples from https://huggingface.co/datasets/llamafactory/glaivetoolcallen - 300 examples from https://huggingface.co/datasets/llamafactory/glaivetoolcallzh - 91 examples for identity learning - 300 examples from https://huggingface.co/datasets/cognitivecomputations/SystemChat-2.0 - 6 examples for multimodal supervised fine-tuning - 300(en)+300(zh) examples from https://huggingface.co/datasets/hiyouga/DPO-En-Zh-20k - 300 examples from https://huggingface.co/datasets/argilla/kto-mix-15k - 300 examples from…
Publicly accessible
apache-2.0
1K<n<10K
C
Dataset · Text generation
Chi
Publicly accessible
mit
n<1K
FineWeb2 A sparkling update with 1000s of languages What is it? This is the second iteration of the popular FineWeb dataset, bringing high quality pretraining data to over 1000 languages. The FineWeb2 dataset is fully reproducible, available under the permissive ODC-By 1.0 license and extensively validated through hundreds of ablation experiments. In particular, on the set of 9 diverse languages we used to guide our processing decisions
Publicly accessible
odc-by
n>1T
This dataset contains the AI-generated behavioral test suites used to evaluate model solutions in ProgramBench. ProgramBench is a benchmark that evaluates whether language models can rebuild programs from scratch. Given only a compiled binary and its documentation, AI agents must architect and implement a complete codebase that reproduces the original program's behavior. These test suites are used to assess whether a candidate solution is behaviorally equivalent to the original program. The dataset covers 200 task instances spanning open-source repositories in 6 programming languages: Each task instance is a folder named. containing: - tests/ — Compressed archives, each containing a…
Publicly accessible
mit
n<1K
Pustaka adalah korpus teks terbuka berskala besar yang dikembangkan oleh MEA Ecosystem, dimulai dengan fokus penuh pada Bahasa Indonesia. Corpus ini disusun dari beberapa sumber publik berkualitas — web crawl yang telah difilter, ensiklopedia, berita, forum, hingga lexicon bahasa gaul — lalu diproses ulang melalui pipeline pembersihan (deteksi bahasa, filter panjang dokumen, deteksi boilerplate/spam, dan deduplikasi) sebelum dirilis. Pustaka dibangun untuk digunakan siapa saja — baik untuk pretraining model bahasa, penelitian NLP, maupun eksperimen pribadi — dan dirilis secara terbuka di bawah Hugging Face. Tabel di atas mencakup corpus-bahasa-indonesia/ saja. Untuk corpus lain (kode…
Publicly accessible
other
10M<n<100M
The SVG files are the source of truth — the PNGs are rasters of them, generated via rsvg-convert. Regenerate any time the SVG changes: Logo: ultrawhale swims in a 5-ring data loop, breathing HF-yellow samples into the dark. Credits to pocoo.vaked.dev for the visual language. Paste this into any coding agent (Claude / opencode / Cursor / aider / Cline) — works from zero context, from anywhere in the dogfeed-loop. ~280 tokens. Self-bootstrapping. End every reply with the loop-state marker. {id, topic, usermessage, freeresponse, freemodel, deepseekresponse, reference, text, role, format, pov, capabilities, memoryref, spacenode, loopindex, pipeline, sessionid, enrichedat, timestamp…
Access requested at publisher
apache-2.0
1K<n<10K
Y
Dataset · Text generation
Yfield
CoSPlay: Cooperative Self-Play at Test-Time with Self-Generated Code and Unit Test This repository contains the benchmarks, generated data, and evaluation logs for CoSPlay, a training-free framework that jointly improves code generation and unit tests through cooperative self-play at inference time. Paper: CoSPlay: Cooperative Self-Play at Test-Time with Self-Generated Code and Unit Test CoSPlay
Publicly accessible
mit
finetranslations-edu-zhtw 是以 HuggingFaceFW/finetranslations-edu 為來源，將其 translatedchunks（原始多語言教育類網頁內容、先被 pivot 翻譯成英文的版本）進一步翻譯成繁體中文的資料集。 HuggingFaceFW/finetranslations-edu 收錄了原本以英文以外語言（oglanguage，涵蓋約 200 種語言）撰寫、經篩選具教育價值（eduscore）的網頁內容，並將其 pivot 翻譯成英文（translatedtext / translatedchunks）。本資料集在此基礎上，將已經是英文的 translatedchunks 逐段（chunk）翻譯成繁體中文，並依原順序重組回完整段落（translatedtextzhtw），讓來自約 200 種原始語言、已篩選過教育價值的內容，也能以繁體中文呈現。 翻譯由一套 LLM 驅動（模型本身不對外公開資訊，僅說明為 LLM），部署為多個平行推論服務以提升吞吐量。 繁體中文語言模型的持續預訓練（continued pretraining）語料補充，特別是希望涵蓋多元語言/文化來源、且具教育價值的內容。 id / url / oglanguage / oglanguagescore / eduscore / eduscoreraw / translatedtokencount / minhashclustersize：原樣保留自上游 HuggingFaceFW/finetranslations-edu。…
Access requested at publisher
odc-by
10M<n<100M
A FineWeb-style Greek-language text dataset extracted from Common Crawl, following the FineWeb-2 recipe adapted for Greek (ellGrek). Crawl coverage starts at CC-MAIN-2024-22 rather than Common Crawl's earliest snapshots: this project picks up right where the FineWeb-2 dataset's own Greek (ellGrek) subset leaves off (2013 through April 2024), so it extends FineWeb-2's Greek coverage forward instead of re-extracting and re-deduping ground FineWeb-2 already covers. Each processed crawl (e.g. CC-MAIN-2024-22) contributes two kinds of files, exposed as named configs (loaddataset("alexliap/greek-cc", name="CC-MAIN-2024-22"), etc. — see the viewer's dataset-config dropdown, same convention FineWeb…
Publicly accessible
odc-by
10M<n<100M
Synthetic Nepali instruction-tuning data generated with NVIDIA NeMo Data Designer from authoritative Nepali documents (agriculture manuals, legal texts). Answers are grounded strictly in the source; unanswerable questions get an explicit refusal. Records use chat messages format plus metadata and per-record qualityscores (grounding / correctness / naturalness, 1-5, LLM-as-judge). One data/train-.jsonl per source document; shards are overwritten idempotently on re-runs. (grounding>=4, correctness>=3, naturalness>=3). Companion repos hold quality-gate rejects (rejected-) and records whose judge call failed (unjudged-).
Publicly accessible
Synthetic Nepali instruction-tuning data generated with NVIDIA NeMo Data Designer from authoritative Nepali documents (agriculture manuals, legal texts). Answers are grounded strictly in the source; unanswerable questions get an explicit refusal. Records use chat messages format plus metadata and per-record qualityscores (grounding / correctness / naturalness, 1-5, LLM-as-judge). One data/train-.jsonl per source document; shards are overwritten idempotently on re-runs. Records judged BELOW the quality gate; each has rejectreasons. Useful for judge calibration, hard-negative mining, or re-filtering with different thresholds. Do NOT use as-is for instruction tuning.
Publicly accessible
H
Dataset · Text generation
Hub
taskcategories: - text-generation sizecategories
Publicly accessible
apache-2.0
100M<n<1B
Public research release of 102 Singapore legal research questions, model responses from 6 systems, and overlapping grades on five dimensions. Headline metrics are overlapping binary flags, not a ranking and not a partition of 100%. — comparison table, category heatmap, per-question comparison, and every answer with its sources and grades. Hallucination is Incorrectness OR Misgroundedness. Incompleteness and Substantial Correctness are independent of Hallucination. A response may be both Substantially Correct and Hallucinated. Hallucination × Substantial Correctness: H yes / SC yes = 269; H yes / SC no = 178; H no / SC yes = 130; H no / SC no = 35. Each cell is count (percentage; Wilson…
Publicly accessible
cc-by-4.0
n<1K
This preserves the selected C11 writer/criterion-judge examples and their row geometry while discarding all teacher scratch reasoning. Membership follows the complete pinned C11 stopping-objective corpus. The six configurations cross the two candidate variants with the three frozen stopping objectives. Every configuration exposes only its combined training view and preserves the native train, validation, and test splits. - asingh15/amazon-c11-distillation at 3f7302f2eb78cfa8a90110bd370237fd11e1638c (combined; catalog faff332d2bf465d37c7d6e9c0bb068522d80e3ae1e5f777688ec6c607f99f1d2)
Publicly accessible
100K<n<1M
Each retained trajectory contributes one direct rubric-writer target and one full-rubric listwise judge target over the original 40 candidates. Teacher scratch reasoning is discarded. Membership follows the complete pinned C11 stopping-objective corpus. The six configurations cross the two candidate variants with the three frozen stopping objectives. Every configuration exposes only its combined training view and preserves the native train, validation, and test splits. - asingh15/amazon-c11-distillation at 3f7302f2eb78cfa8a90110bd370237fd11e1638c (combined-and-full-judge; catalog faff332d2bf465d37c7d6e9c0bb068522d80e3ae1e5f777688ec6c607f99f1d2)
Publicly accessible
100K<n<1M
Each retained trajectory contributes one direct rubric-writer target and one full-rubric listwise judge target over the original 40 candidates. Teacher scratch reasoning is discarded. Membership follows the pinned C11 quality-filter policy. Signed aggregate filter provenance is under quality/ /; it is not exposed as another dataset configuration. The six configurations cross the two candidate variants with the three frozen stopping objectives. Every configuration exposes only its combined training view and preserves the native train, validation, and test splits. - asingh15/amazon-c11-distillation-filtered at f3cbaf3082ca95ba94db218810b8b3e4fa076e81 (membership-and-combined; catalog…
Publicly accessible
10K<n<100K
L
Dataset · Text generation
Lima
Clean, cross-source-deduplicated training text available no later than 2021-12-31. Target: 15B exact anacoluthe89/chrono-2015 tokens. The build rejects any row that would exceed 15B; 20B is an external safety ceiling, not a collection goal. No lane can exceed 26.7% of the target. News and Stack Exchange additionally use per-domain/site token caps. - availability date must be on or before 2021-12-31; - publication/event date must be on or before 2021-12-31; - chunks mentioning years after 2021 are rejected; - exact normalized text is deduplicated globally across every lane; - token quotas use the submitted model family's fast tokenizer, not chars/4; - low-alpha, repetitive, boilerplate…
Access requested at publisher
other
This dataset has a total of 364,420 pieces of medical QA data, some of which have multiple questions in different ways. We extract medical QA pairs from plain texts (e.g., medical encyclopedias and medical articles). We collected 8,699 encyclopedia entries for diseases and 2,736 encyclopedia entries for medicines on Chinese Wikipedia. Moreover, we crawled 226,432 high-quality medical articles from the Qianwen Health website. https://zh.wikipedia.org/wiki/ https://51zyzy.com/
Publicly accessible
apache-2.0
100K<n<1M
This is a synthetic training corpus generated by the Smolify Foundry. It was used to train the corresponding model smolify/smolified-aiexpense. This dataset is a sovereign asset owned by smolify. Generated via Smolify.ai.
Publicly accessible
apache-2.0
1K<n<10K
