AI Training Datasets

Dataset · Text classification

tiktok-drink-metadata

Ori Yonay

A high-recall subset of datasocial/tiktok-5.6B-videos whose caption, hashtags or on-screen text mention a beverage or drink brand. - Partitioned by source month (data/month=YYYY-MM/) The source dataset is licensed under CC BY-NC 4.0. This subset inherits that license: non-commercial use only, with attribution to the source dataset.

Dataset

tiny-base-300m-data

BJY

This public pilot contains packed uint16 token IDs for pretraining the Tiny Base 300M, 2,048-context language model. It contains 50,000,000 training, 1,000,000 validation, and 1,000,000 test tokens. It does not redistribute raw documents. train.bin, validation.bin, and test.bin are little-endian uint16 streams. Use tokenizer.json with the Hugging Face tokenizers library. manifest.json records exact token counts, split policy, source revisions, mixture, and SHA-256 checksums; settings.json records preparation settings. The deterministic split is SHA-256 of normalized text modulo 1,000: 0–9 test, 10–19 validation, and 20–999 training. The mixture is 70% FineWeb-Edu, 25% English Wikipedia, and…

This is the official pretraining dataset for the TinyDTVi-500M causal language model. The corpus is aggressively filtered and deduplicated to maximize the representation of standard Vietnamese text for training resource-constrained models. The initial raw dataset (~135GB) was curated by mixing three high-quality open-source Vietnamese corpora. The approximate mixture ratio is: - 50% CulturaX-Vi (Cleaned web crawl) - 30% VNews (Vietnamese News Corpus) - 20% Vietnamese Wikipedia (Encyclopedia) After strict preprocessing and deduplication, the final dataset size is approximately 85GB (yielding roughly 18 billion unique tokens). To compress the vocabulary down to 50k tokens and focus strictly…

An Arena-style dataset of anonymized, multi-turn conversations focused on real-world tool use. It is intended for research, evaluation, and training of models that decide when and how to call tools. Data is organized into date-partitioned Parquet files. Each row contains one complete conversation trace with its tool calls and results. Flexible nested payloads are stored losslessly in metadatajson, inputjson, and outputjson; parse these JSON strings before training or analysis. processing.py provides streaming normalize and decode commands plus importable normalizetrace and decodetrace helpers for arbitrary JSON-compatible contexts. The repository is publicly discoverable, but files require…

This dataset was created using LeRobot.

Curated multimodal vision dataset of 1-minute candlestick chart captures with technical indicators and verified trade outcomes (WIN / LOSS). This dataset strictly adheres to the official Hugging Face ImageFolder standard: - train/metadata.jsonl: Single-format JSONL metadata with sample annotations - train/images/.jpg: Candlestick chart captures

Dataset

travel-pointcloud-text-benchmark

Hanna C. Lewandowski

This repository contains a preparation pipeline and a small metadata sample for Travel work with Pointcloud Text inputs. It does not claim to be a complete benchmark release; the loader documents how source data is normalized and validated. - loaddata.py — loading, cleaning, and split preparation code. - datasetinfos.json — schema and split metadata. - metadatasample.jsonl — small, human-readable records for checking the schema. - README.md — data card and usage notes. Before using the prepared data, verify source licenses, duplicates across splits, missing values, label balance, and modality-specific corruption. Record the source version and every filtering rule so a later run can…

Dataset · Image classification

tree_species_classification_slovak_normal_cropped

AgML

This dataset provides real RGB images of tree species collected in natural field environments across Slovakia and the Czech Republic. Images were captured using handheld cameras (Sony Alpha 7 and Canon EOS 4000D) during field surveys in June and August 2022. The dataset contains 1,367 images across 4 classes: European beech, European silver fir, Norway spruce, Sessile oak. This dataset is indexed on https://project-agml.github.io/ as part of the AgML python library. This dataset was reformatted from its original format to match HuggingFace standards.

Trump speak dataset augmented with uwu-style reasoning traces generated by lfm2.5-2.6b

Independent yaw 10/20/30-degree conditions, original agentview, robot-base yaw compensation and action rotation enabled. MuJoCo 3.2.3. Each buffer has 400 source-policy rollouts across 40 tasks and initial states 0-9, retaining both outcomes. Prompt-only adaptation uses 1,000 steps, batch 128, AdamW LR 1e-5. Source SG-LP training used 20,000 steps at LR 1e-5. Dataset NPZs contain normalized observation/state frames and success metadata, with no expert or predicted-action imitation targets. Normalization is the source policy's fixed 8-D state / 7-D action quantile statistics. Models include full source/adapted weights, source normalization, prompt resume states, protocol/configuration, and…

Dataset · Speech recognition

unnamed-ar

Ahmed Wasfy

The data/ar/ partition of espnet/yodas3, repackaged as parquet with the audio inline so it loads without a script: Every row is one long-form recording (median ~5 minutes) with its full transcript. Audio container only. Upstream ships Opus in a WebM/Matroska container, which libsndfile cannot open. Here it is Opus in Ogg, remuxed with ffmpeg -c:a copy — the Opus packets are copied untouched, so the audio is bit-identical to upstream. 48 kHz. Metadata joined. Upstream keeps audio in data/ar/audio/.tar and the 12 metadata columns in data/ar/metadata/.parquet; this version carries both in one row. Dialect columns added (see below). Nothing was filtered, resampled, re-encoded, segmented, or…