TreeUQ

TreeUQ — Geographically-Tiled Bavaria EO Benchmark

TreeUQ is a large-scale Earth observation benchmark for tree species mapping and tree structure estimation (height, count, density, variance) over Bavaria, Germany.

Each 128×128 pixel patch (10 m resolution, EPSG:25832) contains: - Sentinel-2 — 4 seasonal composites (spring/summer/autumn/winter 2025), 10 bands - Sentinel-1 GRD — 4 seasonal composites, VV + VH polarisation, linear gamma-0 - Tree species raster — Bavarian species classification aligned to the 10 m grid - Supervision labels — per-pixel height, count, density from the Bavarian Einzelbäume individual-tree inventory - DOP20 aerial imagery (optional) — 20 cm RGB orthophotos as 6400×6400 chips

Splits (spatial blocks): train 31,806 | validation 7,048 | test 6,565 (45,419 Bavaria patches total).

Croissant 1.0 metadata: croissant.json

Selective Download — Get Only What You Need

This dataset is 5+ TB in total. You do not need to download it all. The layout is organised into 10 km × 10 km geographic tiles with modalities separated:

Minimum download for a 10 km region: ~10–50 MB (core only, no DOP20).

Quick Start

1. Install dependencies

pip install huggingface_hub pandas pyarrow numpy

2. Filter by bounding box and download tiles

from huggingface_hub import hf_hub_download
import pandas as pd

REPO = "iclr2027kiwi/TreeUQ"

# Step 1 — download only the index (~1 MB, instant)
idx_path = hf_hub_download(REPO, "data/index/train.parquet", repo_type="dataset")
idx = pd.read_parquet(idx_path)

# Step 2 — filter by bounding box in EPSG:25832 (metres)
# Example: ~20 km around Munich (approximate)
bbox = (680_000, 5_325_000, 702_000, 5_345_000)  # minx, miny, maxx, maxy
roi = idx[
    (idx["center_x"] >= bbox[0]) & (idx["center_x"] <= bbox[2]) &
    (idx["center_y"] >= bbox[1]) & (idx["center_y"] <= bbox[3])
]
print(f"{len(roi)} patches across {roi['geographic_tile'].nunique()} tiles")

# Step 3 — download only the tiles you need
local_tars = []
for tile in roi["geographic_tile"].unique():
    path = hf_hub_download(
        REPO, f"data/tiles/{tile}/core.tar", repo_type="dataset"
    )
    local_tars.append(path)
    print(f"  {tile}: downloaded")

3. Decode tensors from a core.tar

import tarfile, json
import numpy as np

# Load the schema once
schema_path = hf_hub_download(REPO, "data/schema.json", repo_type="dataset")
with open(schema_path) as f:
    schema = json.load(f)
core_members = schema["core_members"]

patch_ids_wanted = set(roi["patch_id"].tolist())

for tar_path in local_tars:
    with tarfile.open(tar_path, "r:") as tf:
        # Build a {key: {suffix: TarInfo}} lookup
        by_key = {}
        for m in tf.getmembers():
            if m.isfile() and "." in m.name:
                key, suffix = m.name.split(".", 1)
                by_key.setdefault(key, {})[suffix] = m

        for key, members in by_key.items():
            patch_id = int(key)
            if patch_id not in patch_ids_wanted:
                continue

            # Decode each tensor
            arrays = {}
            for suffix, spec in core_members.items():
                if suffix in members:
                    raw = tf.extractfile(members[suffix]).read()
                    arrays[suffix] = np.frombuffer(
                        raw, dtype=np.dtype(spec["dtype"])
                    ).reshape(spec["shape"])

            s2 = arrays["s2_autumn.f32"]   # (128, 128, 10) float32  DN
            s2_reflectance = s2 / 10_000   # normalise to [0, 1]
            labels = arrays["tree_count.f32"]  # (128, 128) float32

            print(f"patch {patch_id}: S2 shape={s2.shape}, labels shape={labels.shape}")
            break

4. Download DOP20 aerial imagery (optional)

# Only download DOP20 for tiles where it is available
for tile in roi[roi["dop20_available"]]["geographic_tile"].unique():
    dop20_path = hf_hub_download(
        REPO, f"data/tiles/{tile}/dop20.tar", repo_type="dataset"
    )
    print(f"DOP20 {tile}: {dop20_path}")

# DOP20 member key inside dop20.tar: {patch_id:06d}.dop20_rgb.u8
# Shape: (6400, 6400, 3)  dtype: uint8

5. Automated helper (CLI)

# Clone the dataset pipeline repo for the helper script
git clone https://huggingface.co/datasets/iclr2027kiwi/TreeUQ

python scripts/download_by_bbox.py \
  --bbox 680000,5325000,702000,5345000 \
  --split train \
  --modality core \
  --out-dir ./treeuq_munich

Repository Layout

data/
  index/
    train.parquet          ← 31,806 patches (with geographic_tile, sample_key columns)
    validation.parquet     ← 7,048 patches
    test.parquet           ← 6,565 patches
  schema.json              ← tensor member definitions (dtype, shape, band names)
  tiles/
    tile_<minx_km>_<miny_km>/    ← one dir per 10 km × 10 km grid cell (EPSG:25832)
      core.tar             ← all S2/S1/label/species members for patches in this tile
      dop20.tar            ← DOP20 RGB chips (only where dop20_available=True)
      manifest.json        ← patch_ids, bbox, split counts
croissant.json

Index Parquet Schema (data/index/*.parquet)

Tensor Schema (data/schema.json)

core_members — in core.tar

dop20_members — in dop20.tar

Normalisation: Divide S2 DN by 10,000 to get approximate [0, 1] surface reflectance. S1 values are linear gamma-0 (not dB); typical range 0–1; log-transform before model input.

Spatial Split

Patches are assigned to splits using a 6×6 geographic block grid (~61×60 km per block). Test and validation blocks are chosen to cover distinct rows and columns so hold-outs are spread across Bavaria (not a random shuffle). Entire blocks are held out to prevent spatial autocorrelation leakage. Minimum train→test centroid distance: 30.1 km. The Parquet indexes are the source of truth for split; JSON sidecars inside existing tar files may still carry the previous label.

Limitations

• Geography: Labels cover inventoried urban and park trees; most Bavarian forest carries no label signal.
• Sparsity: Only ~1.2 % of pixels have tree labels; use masked losses or foreground-biased sampling.
• Boundary effects: The AOI is a rectangle; 43 % of grid patches lie outside Bavaria. All patches in this release are pre-filtered to in_bavaria=True.
• Generalisation: Models trained on this dataset may not transfer outside Bavaria without domain adaptation.