Benchmark
Known-truth cubes
driftdrought.synthetic.generate() reproduces the cubes of the DRIFT paper (60 months × 60 × 80 cells):
|
stationary, well-separated events |
|
events migrating 0.3–1.2 cells per month |
|
one event splits into two lobes; two events converge |
|
weaker events on strongly correlated noise |
driftdrought.synthetic.build_library() and generate_irregular() build cubes with
irregular, multi-core events taken from real drought shapes (for example from a DRIFT catalogue of your own
region), so that the method can be tuned to the scale of the target droughts before it is applied. A library of 99
real drought shapes from the Amazon and Orinoco basins is bundled (example_library()).
Protocol
Generate cubes for every scenario, two noise levels (0.3, 0.6) and 20 seeds.
For every method, evaluate a grid of configurations; select the configuration with the highest mean HOTA on half of the seeds and score it on the other half, and vice versa (held-out selection).
Compare methods with paired differences over the evaluation cubes (bootstrap intervals, sign-flip p-values, Holm correction).
import numpy as np
import driftdrought as dd
area = np.ones(dd.synthetic.SHAPE[1:])
rows = []
for sc in dd.synthetic.SCENARIOS:
for seed in range(20):
idx, truth, meta = dd.synthetic.generate(sc, noise=0.6, seed=seed)
lab = dd.detect(dd.smooth(idx, 2.5), area, v_min=50, floor=0.5, footprint="watershed",
ext_floor=0.5, return_info=False)
rows.append(dict(scenario=sc, seed=seed, **dd.metrics.hota(truth, lab, area)))
Mini benchmark
The table and the script that produce it are in Tests and validation.
Results in the paper
With held-out tuning, DRIFT reached HOTA 0.579 (noise 0.3) and 0.459 (noise 0.6), against at most 0.530 and 0.428 for the reference detectors, and was best in seven of eight scenario–noise combinations. On irregular events from observed drought shapes, with every method re-tuned, it remained best on average (+0.03 to +0.04 HOTA over three detectors, on par with 3-D connected components), except under strongly correlated noise. See Citing DRIFT.