24679-places-water-cnn

Water / no-water photo classifier (compact CNN, Optuna AutoML) — 24-679 Fall 2026

Purpose

Classifies a photograph of a place as water (a river, lake, ocean, waterfall, pond, canal or fountain is visibly identifiable) or no_water. Built for the 24-679 neural-network AutoML homework to practise budgeted architecture / hyperparameter search with early stopping, leakage-safe validation, and model documentation. It is trained on 23 original photographs and is a classroom demonstration, not a production water detector.

Data origin and splits

• Source dataset: ssg1/places-water-binary (commit 38d945896ea893a52d2473388461ff25a0126583), MIT license, prepared by a classmate for Homework 1: 34 original iPhone photos from their camera roll, largest-square-cropped and LANCZOS-resized to 224×224 RGB, labelled by the dataset author (17 water / 17 no-water originals).
• Published splits (kept fixed): train = 23 original photos + 368 offline-augmented copies (391 rows); validation = 5 original photos; test = 6 original photos. Holdouts contain original photos only, and no photo family (parent_id) appears in more than one split (asserted in the notebook).
• Offline augmentation in the dataset (training only): 16 variants per original — horizontal flip, rotation (±8°), brightness/contrast, colour jitter — flagged with is_augmented.

Input, expected preprocessing and augmentation pipeline

• Input resolution: 128×128 RGB, NCHW float tensor.
• Expected preprocessing at inference: take a square 224×224 photo (largest-square crop for other aspect ratios), resize to 128×128 with bilinear antialiased interpolation, scale to [0, 1], normalise with ImageNet mean [0.485, 0.456, 0.406] and std [0.229, 0.224, 0.225]. Output logits → softmax; P(water) ≥ 0.5 → water.
• Training-time (on-the-fly) augmentation selected by the search: none (none = normalise only; light = random horizontal flip + random resized crop scale 0.8–1.0; strong = flip + random resized crop scale 0.5–1.0 + rotation ±10° + colour jitter 0.3/0.3/0.3/0.05), applied on top of the dataset's offline augmentation. No augmentation is applied at evaluation time.

Validation scheme

The published validation split has only 5 photos, so model selection used stratified grouped 3-fold cross-validation over the 23 training originals: each fold trains on whole photo families (original + its augmented copies) and validates on the held-out original photos only, so no augmented copy of a validation photo is ever in that fold's training set. The search objective was the mean over folds of the best validation log-loss. The published validation and test photos were scored once, after the search, with the final model.

Search space, budget and early stopping

• Search: Optuna 5.0.0 TPE sampler (seed 24679, 8 random start-up trials), median pruner after each fold.
• Budget: 1500 s wall-clock or 40 trials, whichever first; per fold ≤ 30 epochs with early stopping on fold-validation loss, patience 6. Actual: 310.1 s, trial states {'COMPLETE': 29, 'PRUNED': 11}.
• Space: architecture family {custom CNN, MobileNetV3-Small}; custom CNN depth n_blocks 2–4, base_width {16, 24, 32, 48} (doubling per block, capped at 256), kernel_size {3, 5}, convs_per_block 1–2, batch-norm on/off, dense_units {0, 64, 128}; MobileNet fine-tuning {head_only, full}; dropout 0–0.5; optimizer {Adam, AdamW, SGD+Nesterov}; lr 1e-4–1e-2 (log); weight_decay 1e-6–1e-2 (log); LR schedule {constant, cosine}; augmentation {none, light, strong}; label smoothing {0, 0.1}; batch size {16, 32}; input resolution {64, 96, 128}.
• Early-stop rationale: validation loss is the monitored quantity because it is continuous (accuracy on 7–8 fold photos moves in large steps); the best trial's folds stopped at epochs [6, 1, 1], so the final model was trained on all 391 training rows for 3 epochs (the rounded mean), with no validation data used for training.

Best architecture and hyperparameters

Best trial #17: MobileNetV3-Small (ImageNet-pretrained), new 2-class head, fine-tuning mode head_only, dropout 0.19; 1,519,906 parameters. Mean CV validation log-loss 0.556, mean CV validation accuracy 0.690 (per fold: loss [0.42, 0.622, 0.626], accuracy [0.75, 0.571, 0.75]).

{
  "family": "mobilenet_v3_small",
  "resolution": 128,
  "batch_size": 16,
  "optimizer": "adam",
  "lr": 0.000272713757722017,
  "weight_decay": 3.899248336041237e-06,
  "lr_schedule": "cosine",
  "augmentation": "none",
  "label_smoothing": 0.1,
  "dropout": 0.1857493936554521,
  "finetune_mode": "head_only"
}

MobileNetV3(
  (features): Sequential(
    (0): Conv2dNormActivation(
      (0): Conv2d(3, 16, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
      (1): BatchNorm2d(16, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
      (2): Hardswish()
    )
    (1): InvertedResidual(
      (block): Sequential(
        (0): Conv2dNormActivation(
          (0): Conv2d(16, 16, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=16, bias=False)
          (1): BatchNorm2d(16, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): ReLU(inplace=True)
        )
        (1): SqueezeExcitation(
          (avgpool): AdaptiveAvgPool2d(output_size=1)
          (fc1): Conv2d(16, 8, kernel_size=(1, 1), stride=(1, 1))
          (fc2): Conv2d(8, 16, kernel_size=(1, 1), stride=(1, 1))
          (activation): ReLU()
          (scale_activation): Hardsigmoid()
        )
        (2): Conv2dNormActivation(
          (0): Conv2d(16, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(16, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        )
      )
    )
    (2): InvertedResidual(
      (block): Sequential(
        (0): Conv2dNormActivation(
          (0): Conv2d(16, 72, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(72, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): ReLU(inplace=True)
        )
        (1): Conv2dNormActivation(
          (0): Conv2d(72, 72, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=72, bias=False)
          (1): BatchNorm2d(72, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): ReLU(inplace=True)
        )
        (2): Conv2dNormActivation(
          (0): Conv2d(72, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(24, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        )
      )
    )
    (3): InvertedResidual(
      (block): Sequential(
        (0): Conv2dNormActivation(
          (0): Conv2d(24, 88, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(88, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): ReLU(inplace=True)
        )
        (1): Conv2dNormActivation(
          (0): Conv2d(88, 88, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=88, bias=False)
          (1): BatchNorm2d(88, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): ReLU(inplace=True)
        )
        (2): Conv2dNormActivation(
          (0): Conv2d(88, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(24, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        )
      )
    )
    (4): InvertedResidual(
      (block): Sequential(
        (0): Conv2dNormActivation(
          (0): Conv2d(24, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(96, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (1): Conv2dNormActivation(
          (0): Conv2d(96, 96, kernel_size=(5, 5), stride=(2, 2), padding=(2, 2), groups=96, bias=False)
          (1): BatchNorm2d(96, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (2): SqueezeExcitation(
          (avgpool): AdaptiveAvgPool2d(output_size=1)
          (fc1): Conv2d(96, 24, kernel_size=(1, 1), stride=(1, 1))
          (fc2): Conv2d(24, 96, kernel_size=(1, 1), stride=(1, 1))
          (activation): ReLU()
          (scale_activation): Hardsigmoid()
        )
        (3): Conv2dNormActivation(
          (0): Conv2d(96, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(40, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        )
      )
    )
    (5): InvertedResidual(
      (block): Sequential(
        (0): Conv2dNormActivation(
          (0): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(240, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (1): Conv2dNormActivation(
          (0): Conv2d(240, 240, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=240, bias=False)
          (1): BatchNorm2d(240, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (2): SqueezeExcitation(
          (avgpool): AdaptiveAvgPool2d(output_size=1)
          (fc1): Conv2d(240, 64, kernel_size=(1, 1), stride=(1, 1))
          (fc2): Conv2d(64, 240, kernel_size=(1, 1), stride=(1, 1))
          (activation): ReLU()
          (scale_activation): Hardsigmoid()
        )
        (3): Conv2dNormActivation(
          (0): Conv2d(240, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(40, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        )
      )
    )
    (6): InvertedResidual(
      (block): Sequential(
        (0): Conv2dNormActivation(
          (0): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(240, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (1): Conv2dNormActivation(
          (0): Conv2d(240, 240, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=240, bias=False)
          (1): BatchNorm2d(240, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (2): SqueezeExcitation(
          (avgpool): AdaptiveAvgPool2d(output_size=1)
          (fc1): Conv2d(240, 64, kernel_size=(1, 1), stride=(1, 1))
          (fc2): Conv2d(64, 240, kernel_size=(1, 1), stride=(1, 1))
          (activation): ReLU()
          (scale_activation): Hardsigmoid()
        )
        (3): Conv2dNormActivation(
          (0): Conv2d(240, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(40, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        )
      )
    )
    (7): InvertedResidual(
      (block): Sequential(
        (0): Conv2dNormActivation(
          (0): Conv2d(40, 120, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(120, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (1): Conv2dNormActivation(
          (0): Conv2d(120, 120, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=120, bias=False)
          (1): BatchNorm2d(120, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (2): SqueezeExcitation(
          (avgpool): AdaptiveAvgPool2d(output_size=1)
          (fc1): Conv2d(120, 32, kernel_size=(1, 1), stride=(1, 1))
          (fc2): Conv2d(32, 120, kernel_size=(1, 1), stride=(1, 1))
          (activation): ReLU()
          (scale_activation): Hardsigmoid()
        )
        (3): Conv2dNormActivation(
          (0): Conv2d(120, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(48, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        )
      )
    )
    (8): InvertedResidual(
      (block): Sequential(
        (0): Conv2dNormActivation(
          (0): Conv2d(48, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(144, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (1): Conv2dNormActivation(
          (0): Conv2d(144, 144, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=144, bias=False)
          (1): BatchNorm2d(144, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (2): SqueezeExcitation(
          (avgpool): AdaptiveAvgPool2d(output_size=1)
          (fc1): Conv2d(144, 40, kernel_size=(1, 1), stride=(1, 1))
          (fc2): Conv2d(40, 144, kernel_size=(1, 1), stride=(1, 1))
          (activation): ReLU()
          (scale_activation): Hardsigmoid()
        )
        (3): Conv2dNormActivation(
          (0): Conv2d(144, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(48, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        )
      )
    )
    (9): InvertedResidual(
      (block): Sequential(
        (0): Conv2dNormActivation(
          (0): Conv2d(48, 288, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(288, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (1): Conv2dNormActivation(
          (0): Conv2d(288, 288, kernel_size=(5, 5), stride=(2, 2), padding=(2, 2), groups=288, bias=False)
          (1): BatchNorm2d(288, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (2): SqueezeExcitation(
          (avgpool): AdaptiveAvgPool2d(output_size=1)
          (fc1): Conv2d(288, 72, kernel_size=(1, 1), stride=(1, 1))
          (fc2): Conv2d(72, 288, kernel_size=(1, 1), stride=(1, 1))
          (activation): ReLU()
          (scale_activation): Hardsigmoid()
        )
        (3): Conv2dNormActivation(
          (0): Conv2d(288, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(96, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        )
      )
    )
    (10): InvertedResidual(
      (block): Sequential(
        (0): Conv2dNormActivation(
          (0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(576, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (1): Conv2dNormActivation(
          (0): Conv2d(576, 576, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=576, bias=False)
          (1): BatchNorm2d(576, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (2): SqueezeExcitation(
          (avgpool): AdaptiveAvgPool2d(output_size=1)
          (fc1): Conv2d(576, 144, kernel_size=(1, 1), stride=(1, 1))
          (fc2): Conv2d(144, 576, kernel_size=(1, 1), stride=(1, 1))
          (activation): ReLU()
          (scale_activation): Hardsigmoid()
        )
        (3): Conv2dNormActivation(
          (0): Conv2d(576, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(96, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        )
      )
    )
    (11): InvertedResidual(
      (block): Sequential(
        (0): Conv2dNormActivation(
          (0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(576, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (1): Conv2dNormActivation(
          (0): Conv2d(576, 576, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=576, bias=False)
          (1): BatchNorm2d(576, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
          (2): Hardswish()
        )
        (2): SqueezeExcitation(
          (avgpool): AdaptiveAvgPool2d(output_size=1)
          (fc1): Conv2d(576, 144, kernel_size=(1, 1), stride=(1, 1))
          (fc2): Conv2d(144, 576, kernel_size=(1, 1), stride=(1, 1))
          (activation): ReLU()
          (scale_activation): Hardsigmoid()
        )
        (3): Conv2dNormActivation(
          (0): Conv2d(576, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(96, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        )
      )
    )
    (12): Conv2dNormActivation(
      (0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (1): BatchNorm2d(576, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
      (2): Hardswish()
    )
  )
  (avgpool): AdaptiveAvgPool2d(output_size=1)
  (classifier): Sequential(
    (0): Linear(in_features=576, out_features=1024, bias=True)
    (1): Hardswish()
    (2): Dropout(p=0.1857493936554521, inplace=False)
    (3): Linear(in_features=1024, out_features=2, bias=True)
  )
)

Metrics

Holdout scores (final CNN vs. baselines; water = positive class)

• Test accuracy bootstrap 95% CI: 0.500–1.000 (2000 resamples of 6 photos — one photo is 16.7 accuracy points).
• Seed sensitivity (final configuration retrained with seeds [0, 1, 2]):

• Grouped-CV baselines (mean over folds): {"majority-class accuracy": 0.524, "colour-logistic accuracy": 0.476, "colour-logistic log-loss": 0.722}.

Per-photo test predictions

Per-photo validation predictions

Known failure modes and limitations

Misclassified holdout photos in this run: - validation photo place_water_009: actual water, predicted no_water (P(water) = 0.39) - test photo place_water_008: actual water, predicted no_water (P(water) = 0.29)

Expected weaknesses given the training data (hypotheses, not measured on held-out data of each kind): - Small, distant or partially hidden water (a fountain in a plaza, a canal between buildings) — the model was trained on 23 originals and probably keys on large blue/reflective regions. - Look-alikes: wet pavement, glass façades, snow or ice, clear skies filling the lower frame, swimming pools, and photos of screens or paintings of water. - Capture conditions absent from the data: night scenes, heavy rain or fog, non-iPhone cameras, non-square crops (the model expects a largest-square crop), indoor scenes. - Statistical: 34 photos in total; augmentation adds variation but no new scenes; single errors change the reported accuracies by 16–20 points; GPU training is not bit-reproducible, so a rerun can select a different trial. - No people are the subject of any photo; the model must not be used for surveillance, flood assessment or any safety-relevant decision.

Hardware and compute budget

Linux-6.6.122+-x86_64-with-glibc2.39, Python 3.13.15, 2 CPU cores, 13.6 GB RAM, device cuda (Tesla T4). Search budget 1500 s / 40 trials (used 310.1 s); final fit 1.3 s; three seed-check refits. PyTorch 2.11.0+cu128, torchvision 0.26.0+cu128, Optuna 5.0.0. Run started 2026-09-21T02:24:17+00:00; seeds: 24679 for the search and final fit.

How to use

import json, torch, huggingface_hub, importlib.util
from PIL import Image
from torchvision.transforms import v2

repo = "shanexf/24679-places-water-cnn"
code_path = huggingface_hub.hf_hub_download(repo, "water_cnn.py")
spec = importlib.util.spec_from_file_location("water_cnn", code_path); water_cnn = importlib.util.module_from_spec(spec); spec.loader.exec_module(water_cnn)
config = json.load(open(huggingface_hub.hf_hub_download(repo, "config.json")))
model = water_cnn.build_model(config["model_config"], pretrained=False)
model.load_state_dict(torch.load(huggingface_hub.hf_hub_download(repo, "model.pt"), map_location="cpu")); model.eval()

res = config["input"]["resolution"]
prep = v2.Compose([v2.PILToTensor(), v2.Resize((res, res), antialias=True), v2.ToDtype(torch.float32, scale=True),
                   v2.Normalize(config["input"]["mean"], config["input"]["std"])])
img = Image.open("photo.jpg").convert("RGB")
side = min(img.size); img = img.crop(((img.width - side) // 2, (img.height - side) // 2, (img.width + side) // 2, (img.height + side) // 2))
p_water = torch.softmax(model(prep(img)[None]), dim=1)[0, 1].item()
print("water" if p_water >= 0.5 else "no_water", round(p_water, 3))

License

MIT, matching the source dataset. Intended for coursework.

AI usage disclosure

Claude (Anthropic) was used to draft this notebook (search space, training loop, evaluation and model-card template) following the 24-679 lecture material. The author ran the notebook end to end, checked the outputs, and edited the text. The photographs were taken and labelled by a classmate; none were generated by AI.

Author

shanexf — 24-679, Carnegie Mellon University, Fall 2026.