osm-polygon-wikidata-and-wikipedia-landuse

A land-use / land-cover relevance label for every sentence of NoeFlandre/osm-polygon-wikidata-and-wikipedia (revision d6fc058). The input is mirrored here unchanged; labels and model outputs are separate tables that join back to it.

In progress: 100 of 761 input files fully labelled, 661 more partially, 74,791,929 rows, 165 unique texts have their full model outputs published so far (outputs ship with the first complete file that contains the text; labels for the other sentences are in labels/).

Failed rows

928,088 rows (1.2% of all rows) have no usable answer. Reasons:

Where the labels are

Each hexagon is an H3 cell (resolution 3) holding the yes/no sentences whose document's first linked polygon (smallest polygon_id with lat/lon) falls in it. Colour is the share of yes in the cell, centred on the dataset-wide share (33.8%); grey cells hold fewer than 10 sentences. 50,401,093 of 50,401,093 yes/no sentences (100.0%) are placed, in 9,667 cells; failed and skipped_unsplit rows are not.

Tables

• viewer/<input path>.parquet (the default view): sentence, label, language and region (the input file), nothing else. It holds only sentences with a model answer (yes, no, failed) or skipped (skipped_unsplit) and with at least 2 letters; debris such as - or 7 stays in labels/.
• labels/<input path>.parquet: one row per sentence. Join keys (sentence_id), text_sha256, decision, parse_mode, failure_reason, generation_id.
• generations/<fingerprint>/*.parquet: one row per unique text: raw output including the reasoning, token counts, finish reason, speculative-decoding statistics, GPU, site, job and code commit. Identical texts are generated once and share a generation_id.
• pending rows belong to partly labelled files and are only in labels/, never in the viewer; their labels are refreshed as the model works, and the generations/ rows of a file are published once the file is complete.

SELECT i.*, l.decision, g.raw_output
FROM read_parquet(['wikipedia/sentences/*.parquet', 'wikivoyage/sentences/*.parquet']) i
JOIN read_parquet(['labels/wikipedia/sentences/*.parquet', 'labels/wikivoyage/sentences/*.parquet']) l USING (sentence_id)
LEFT JOIN 'generations/*/*.parquet' g USING (generation_id);

Method

LiquidAI/LFM2.5-2.6B (revision 654f946) with the speculative-decoding draft LiquidAI/LFM2.5-2.6B-DSpark (revision 458ceda), served with SGLang (BF16), greedy decoding, thinking mode, at most 4,096 new tokens. The prompt is the one of NoeFlandre/benchmark-llms-landuse-relevance.

Quality

Only GPU types that passed a pre-registered non-inferiority gate on the full 25,500-item benchmark generated labels. Differences are against the published reference run (one-sided 95 % lower bounds must stay above -0.02; failed-rate increase under +0.5 pp).

Greedy decoding is not batch- or GPU-invariant: about 12 % of individual decisions would flip between two runs of the reference setup. The gate guarantees aggregate quality, not per-row reproducibility.

License and citation

Labels: ODbL, like the OpenStreetMap-derived input. Wikipedia and Wikivoyage text is CC BY-SA 4.0; attribution columns are kept in the mirrored input.

Code and configuration (fingerprint 71dd8471f52321ab): https://github.com/NoeFlandre/filter-osm-datasets-llms-landuse