osm-polygon-wikidata-only

Blog post: How to describe a place on Earth with Wikidata.

OSM polygons tagged with a wikidata=* reference, enriched with Wikipedia and Wikivoyage text across all available languages. The published tables are:

• polygons/<stem>.parquet — one row per polygon
• wikipedia/documents/<stem>.parquet — one row per unique Wikipedia article
• polygon_articles/<stem>.parquet — unified polygon-to-document many-to-many links for Wikipedia and Wikivoyage; project identifies the source and document_id references its document table
• wikipedia/sections/<stem>.parquet — section-level partitions of Wikipedia document text
• wikivoyage/documents/<stem>.parquet — full Wikivoyage documents associated with places through Wikidata
• wikivoyage/sections/<stem>.parquet — section-level partitions of Wikivoyage document text
• wikidata/facts/<stem>.parquet — structured Wikidata claims for polygon entities

Links are derived from the Wikidata identifiers shared by each OSM polygon and its Wikipedia or Wikivoyage documents.

Generated on 2026-08-02.

Maintained by Noé Flandre.

Source code: GitHub repository.

This Hugging Face dataset is V1, produced by the V1.0.0 GitHub code.

Trackio snapshot

View the dataset presentation for a visual overview of the published tables and geographic coverage.

The finished dataset is recorded in the single public Trackio run final-dataset-snapshot. It contains exactly three plots: a text-coverage funnel, the top ten Wikipedia languages plus Other languages, and dataset composition on a logarithmic scale. The run is a static snapshot with no pipeline timeline.

The funnel's language thresholds use the canonical Wikipedia polygon fields. Wikivoyage is included in the combined document, word, and dataset-composition totals.

Coverage

Polygons with Wikipedia or Wikivoyage text

Each point is a dataset polygon with at least one non-empty Wikipedia document or a non-empty Wikivoyage document sharing its Wikidata entity. A polygon is shown once even when several documents qualify.

All dataset polygons

Each point represents one dataset polygon carrying an OSM wikidata=* tag, whether or not corresponding Wikipedia or Wikivoyage text is available.

Geographic coverage

Wikipedia + Wikivoyage text density

Each H3 cell contains the raw number of polygons with non-empty Wikipedia or Wikivoyage text. A polygon is counted once even when both projects or several documents qualify. Colour uses a logarithmic purple-to-yellow scale; this is an absolute density count, not a proportion of all polygons.

Dataset snapshot

Polygon rows retain regional copies. Identity metrics use one deterministic representative per (osm_type, osm_id); successful text additionally requires fetch_status=ok and trimmed non-empty full_text.

Wikipedia + Wikivoyage document words sums the full Wikipedia and Wikivoyage documents and excludes section rows because sections duplicate document text.

Polygon surface and geometry

Recomputed from every row of the published polygons/*.parquet table before each dataset-card publication: all 1,184,110 rows across 375 regional files, with no sampling and no other input. Areas are the table's own area_m2 values (WGS84 equirectangular approximation, holes subtracted); shape and extent figures are decoded from the geometry and bbox columns. The complete machine-readable report, including the per-source breakdown, is published as stats.json.

Field definitions:

• Total surface: sum of area_m2 over every published polygon row. Overlapping polygons are summed independently, so this is the total polygon surface, not the distinct land area covered.
• Percentiles and medians interpolate linearly between the two closest ranks.
• Degenerate: rows whose recorded area_m2 is zero or negative. Below 1 m2 counts rows with a positive area smaller than one square metre.
• A vertex is one coordinate pair; each ring's repeated closing coordinate is counted once. A hole is an inner ring, and Components is the number of MultiPolygon members, which is 1 for every Polygon row.
• Metre spans convert the degree spans with the same equirectangular rule used for area_m2, evaluated at each bounding box's mean latitude.
• Bbox wider than 180 degrees is the antimeridian signature: the box spans more than half the globe because the polygon crosses longitude 180.
• Area buckets are half-open decade intervals [lower, upper) in square metres; the bucket list is identical in every report so two reports stay comparable.

Language distribution

Top 20 languages across Wikipedia and Wikivoyage documents:

Document counts include canonical rows from both projects. Polygon counts are unique (osm_type, osm_id) polygon identities per language and require successfully extracted (fetch_status=ok) trimmed non-empty full_text. Canonical links use the polygon_articles identity join; legacy Wikivoyage links use the shared Wikidata QID fallback.

Concentration: - Top 1 language: 10.1% of all Wikipedia + Wikivoyage documents - Top 5 languages: 29.9% of all Wikipedia + Wikivoyage documents - Top 10 languages: 45.1% of all Wikipedia + Wikivoyage documents - Top 20 languages: 63.6% of all Wikipedia + Wikivoyage documents

Long-tail: - 351 language(s) total - 5 language(s) appear in fewer than 5 Wikipedia + Wikivoyage documents - 5 language(s) appear in fewer than 10 Wikipedia + Wikivoyage documents - 4 language(s) appear in fewer than 5 polygons with successful non-empty text

Storage accounting

Core tables size and total storage size are additive:

Wikipedia text corpus

Documents

Estimated tokens use a transparent approximation: characters // 4 per non-empty text row, with a minimum of one token per non-empty row.

Sections

Estimated tokens use a transparent approximation: characters // 4 per non-empty text row, with a minimum of one token per non-empty row.

Languages

Wikivoyage text corpus

Documents

Estimated tokens use a transparent approximation: characters // 4 per non-empty text row, with a minimum of one token per non-empty row.

Sections

Estimated tokens use a transparent approximation: characters // 4 per non-empty text row, with a minimum of one token per non-empty row.

Languages

Wikidata facts

Value-type distribution:

Top properties:

English labels are requested where available; multilingual labels are preserved verbatim in the Parquet column property_labels.

Geographic distribution by continent

This table is recomputed from the finalized Parquet tables before each dataset-card publication. Each polygon's WGS84 centroid is spatially matched to the bundled Natural Earth 1:110m Admin-0 country boundaries, then assigned the country's continent. Because this is a coarse global reference, offshore points and centroids outside its country polygons remain Unassigned rather than being guessed.

Metric definitions:

• Polygons: dataset polygons whose centroid is assigned to the continent. Every polygon appears in exactly one continent row, including Unassigned.
• Wikipedia documents: distinct non-empty Wikipedia documents connected to those polygons through polygon_articles. A document is counted once within a continent, but may appear in more than one continent when linked polygons span more than one continent.
• Wikivoyage documents: distinct non-empty Wikivoyage documents whose Wikidata entity is shared by a polygon in the continent. The same cross-continent counting rule applies.
• Polygons with Wikipedia text: polygons linked to at least one non-empty Wikipedia document.
• Polygons with Wikipedia or Wikivoyage text: polygons satisfying the Wikipedia condition or sharing a Wikidata entity with at least one non-empty Wikivoyage document. Each polygon is counted once.
• Text coverage: combined text-covered polygons / all dataset polygons in the continent row.

Schema

polygons

wikipedia/documents

polygon_articles

wikivoyage/documents

wikipedia/sections and wikivoyage/sections

wikidata/facts

Language partitions

The V1 language release is an additive, row-level partition of the published text tables. Each source row is routed by its normalized language value; multilingual rows are not collapsed to a polygon-level preferred language. Missing, blank, malformed, and legacy-unusable values are preserved in the explicit lang-unknown partition.

Validated languages: 348 (including unknown).

The release manifest records the source fingerprint, schema, row counts, and SHA-256 hash for every generated file.

Data sources & licenses

• OpenStreetMap polygons: (c) OpenStreetMap contributors, licensed under ODbL 1.0.
• Wikidata entity data: CC0 1.0.
• Wikipedia article text: licensed under CC BY-SA 4.0 by the respective Wikipedia editors; attributed inline per article.
• Wikivoyage text: licensed under CC BY-SA 4.0 by the respective Wikivoyage editors; attributed inline per document.
• Natural Earth Admin-0 geography: public-domain 1:110m reference data used only to assign centroid-based continent statistics and draw context maps.

How to load

from datasets import load_dataset
ds = load_dataset("parquet", data_files={
    "polygons": "hf://datasets/NoeFlandre/osm-polygon-wikidata-only/polygons/*.parquet",
})

Citation

If you use this dataset, please cite it. Download the dataset citation metadata from CITATION.cff.