← Back to project catalogue
GP-DA-1VQ625KData and AIReady

Earth-observation domain-shift benchmark

A reproducible Earth observation benchmark that measures how land-cover classifiers change across sensor, season, region, and compound domain shifts.

Earth-observation domain-shift benchmark project visual
GP-DA-1VQ625K · Data and AI
  • Python 3.11
  • scikit-learn 1.9
  • NumPy
  • pandas
  • SciPy
  • Matplotlib
  • STAC

Project definition

Problem statement

A land-cover classifier can perform well when training and test observations share the same sensor, season, and region, then lose accuracy when any of those conditions change.

The engineering problem is to isolate these shifts, measure their effect on accuracy and confidence, compare transparent adaptation methods, and retain enough evidence to explain the result.

Project objectives

  • Create controlled source, sensor-shift, season-shift, region-shift, and compound-shift domains.
  • Compare logistic regression, random forest, and histogram gradient boosting under identical scene splits.
  • Test source-only moment alignment and CORAL covariance alignment without using target labels for training.
  • Measure macro F1, balanced accuracy, log loss, Brier score, calibration error, drift, and selective accuracy.
  • Retain every trial, configuration, figure, and summary needed to reproduce the analysis.

Project structure

Project components

01

Domain generator

Creates controlled surface-reflectance observations for four land-cover classes and five documented domains.

02

Feature pipeline

Builds six spectral-band features and the NDVI, NDWI, and NDBI indices.

03

Model laboratory

Trains and evaluates three classifier families with source-only calibration.

04

Alignment laboratory

Applies no alignment, moment alignment, or CORAL under explicit target-information boundaries.

05

Metric engine

Calculates discrimination, calibration, drift, label-distance, coverage, and selective-accuracy measures.

06

Evidence output

Writes trial records, summaries, labelled figures, and an offline results dashboard.

Methodology

Project workflow

  1. 01
    Generate domains

    Create reproducible source and target scenes under fixed sensor, season, and regional transformations.

  2. 02
    Split by scene

    Separate training and calibration scenes so observations from one scene cannot cross the split.

  3. 03
    Train source models

    Fit each model family on source data and learn temperature calibration from source calibration data.

  4. 04
    Apply alignment

    Transform target features with the selected source-only or unsupervised alignment method.

  5. 05
    Evaluate and compare

    Record all metrics across five seeds, domains, models, and alignment choices.

Demonstration scenario

The retained source model is evaluated first on an in-domain control and then on sensor, season, region, and compound shifts. The dashboard shows a 9.18 point raw macro F1 loss under compound shift and compares each model and alignment method, including the best compound result of 0.9721 macro F1.

Engineering

Tools and method

Runtime
Python 3.11 or later with an installable command-line package.
Data model
Controlled semisynthetic multispectral reflectance with scene, class, sensor, season, and region metadata.
Machine learning
scikit-learn classifiers, source-only temperature calibration, and deterministic random seeds.
Domain adaptation
Transparent moment and covariance alignment implemented with NumPy and SciPy.
Earth data discovery
An optional STAC discovery command for locating suitable public imagery without silently changing the retained benchmark.
Quality controls
Nineteen tests, Ruff, dependency audit, delivery validation, and document accessibility audits.

Testing

Evaluation

Evaluation measures

  • Macro F1 and balanced accuracy for every source and target condition
  • Log loss, multiclass Brier score, and expected calibration error
  • Population stability index and label-distribution Jensen-Shannon distance
  • Accuracy at retained prediction coverage levels
  • Mean and variation across five deterministic experiment seeds
  • Automated tests, statement coverage, dependency audit, and delivery checks

Project boundaries

  • The retained experiment is controlled and semisynthetic, not an operational satellite-image accuracy claim.
  • Its four land-cover classes and documented transformations simplify real atmospheric, geographic, and sensor variation.
  • Target labels are used only for evaluation and are not used to train or calibrate the source models.
  • Operational use requires real imagery, field or reference labels, cloud and quality masking, geographic validation, and an application-specific review.

Included

  1. 01Python package, command-line tools, and reproducible experiment runner
  2. 02Five controlled land-cover domains with scene-grouped train and calibration splits
  3. 03225 retained benchmark trials, JSON summary, CSV results, and static dashboard
  4. 04Seven labelled result figures and one referenced literature figure
  5. 0519 automated tests with 97.87 percent statement coverage
  6. 06Complete source code in a private GitHub repository
  7. 0784-page project documentation in PDF and editable Word formats
  8. 0822-page setup and usage guide in PDF and editable Word formats
  9. 0945 annotated references

Project record

No information is collected on this page.

Permanent project ID
GP-DA-1VQ625K
Catalogued
21 Aug 2026
Completed
24 Aug 2026
Verified
24 Aug 2026
Demonstration
Included in repository

Handover

After purchase

  1. 01
    Payment is confirmed

    The project is marked unavailable and cannot be purchased again.

  2. 02
    Repository access is granted

    The buyer's submitted GitHub account receives access to the private repository.

  3. 03
    The purchase record is delivered

    The certification sheet is prepared from the reviewed buyer details and sent privately by email.