Verification¶

The synhydro.verification module evaluates statistical property preservation: whether a synthetic ensemble reproduces the moments, correlations, and distributional shape of the observed record.

The terminology follows Stedinger and Taylor (1982). Verification demonstrates that generated flows reproduce the statistics the generator was designed to reproduce. Validation demonstrates that characteristics not explicitly fit, such as drought behavior, are also consistent with the historical record; see Validation.

How results are reported¶

Every metric is computed once on the observed record and once on each realization. The result compares the observed statistic against the distribution of the statistic across realizations, following the standard reporting convention of the synthetic streamflow literature:

syn_median, syn_q05, ..., syn_q95: quantiles of the statistic across realizations.
obs_percentile: the position of the observed value within the synthetic sample, computed as (n_below + 0.5 * n_equal + 0.5) / (n + 1) where n is the number of realizations. Values near 0 or 1 mean the observed statistic lies in the tail of the ensemble distribution; values near 0.5 mean it is central.
in_90_band: whether the observed value falls between the ensemble's 5th and 95th percentiles.
relative_diff: (syn_median - observed) / |observed|, omitted when the observed value is too close to zero for a ratio to be meaningful.

There is no single cross-category score: metrics with different units are never averaged together. category_summary() provides a unit-free rollup per category.

Metric kinds¶

Each metric has one of four kinds, which determines its signature and how it appears in the tidy frame:

Kind	Signature	Reported as
`scalar`	`f(x: pd.Series) -> float`	one value per site per realization
`curve`	`f(x: pd.Series) -> pd.Series`	one value per component (month, lag, exceedance probability, period band)
`matrix`	`f(frame: pd.DataFrame) -> pd.Series`	one value per site pair
`comparison`	`f(x, reference) -> float or pd.Series`	divergence of one realization from observed; no observed value of its own

Metric selection¶

verify() requires an explicit metric selection: metrics="all", a list of metric names, category names, or callables. There is no default subset, so a report never silently omits categories.

import synhydro

result = synhydro.verify(ensemble, Q_obs, metrics="all")
result = synhydro.verify(ensemble, Q_obs, metrics=["marginal", "acf"])
result.summary()

Metric reference¶

Metric	Category	Kind	Units	Frequencies	Citation
`mean`	marginal	scalar	flow	any	Matalas (1967); Stedinger and Taylor (1982)
`std`	marginal	scalar	flow	any	Matalas (1967); Stedinger and Taylor (1982)
`cv`	marginal	scalar	dimensionless	any	Matalas (1967); Stedinger and Taylor (1982)
`skewness`	marginal	scalar	dimensionless	any	Matalas (1967); Stedinger and Taylor (1982)
`kurtosis`	marginal	scalar	dimensionless	any	Matalas (1967); Stedinger and Taylor (1982)
`minimum`	marginal	scalar	flow	any	Matalas (1967); Stedinger and Taylor (1982)
`maximum`	marginal	scalar	flow	any	Matalas (1967); Stedinger and Taylor (1982)
`flow_q10`	marginal	scalar	flow	any	Matalas (1967); Stedinger and Taylor (1982)
`flow_q50`	marginal	scalar	flow	any	Matalas (1967); Stedinger and Taylor (1982)
`flow_q90`	marginal	scalar	flow	any	Matalas (1967); Stedinger and Taylor (1982)
`ks_statistic`	marginal	comparison	dimensionless	any	Two-sample Kolmogorov-Smirnov distance
`lag1_autocorrelation`	temporal	scalar	dimensionless	any	Matalas (1967)
`lag2_autocorrelation`	temporal	scalar	dimensionless	any	Matalas (1967)
`acf`	temporal	curve	dimensionless	any	Salas et al. (1980); Kirsch et al. (2013)
`hurst`	temporal	scalar	dimensionless	any, 20+ years	Hurst (1951); Koutsoyiannis (2002)
`monthly_mean`	seasonal	curve	flow	sub-annual	Lall and Sharma (1996); Nowak et al. (2010)
`monthly_std`	seasonal	curve	flow	sub-annual	Lall and Sharma (1996); Nowak et al. (2010)
`monthly_skewness`	seasonal	curve	dimensionless	sub-annual	Lall and Sharma (1996); Nowak et al. (2010)
`monthly_maximum`	seasonal	curve	flow	sub-annual	Lall and Sharma (1996); Nowak et al. (2010)
`monthly_minimum`	seasonal	curve	flow	sub-annual	Lall and Sharma (1996); Nowak et al. (2010)
`monthly_lag1_correlation`	seasonal	curve	dimensionless	sub-annual	Kirsch et al. (2013)
`monthly_ranksum_pvalue`	seasonal	comparison	pvalue	sub-annual	Herman et al. (2016)
`monthly_levene_pvalue`	seasonal	comparison	pvalue	sub-annual	Herman et al. (2016)
`annual_mean`	annual	scalar	flow (annual total)	any, 10+ years	Stedinger and Taylor (1982)
`annual_sd`	annual	scalar	flow (annual total)	any, 10+ years	Stedinger and Taylor (1982); Srinivas and Srinivasan (2005)
`annual_cv`	annual	scalar	dimensionless	any, 10+ years	Stedinger and Taylor (1982)
`annual_skewness`	annual	scalar	dimensionless	any, 10+ years	Stedinger and Taylor (1982)
`annual_lag1_autocorrelation`	annual	scalar	dimensionless	any, 10+ years	Stedinger and Taylor (1982)
`annual_minimum`	annual	scalar	flow (annual total)	any, 10+ years	Stedinger and Taylor (1982)
`annual_maximum`	annual	scalar	flow (annual total)	any, 10+ years	Stedinger and Taylor (1982)
`cross_correlation`	spatial	matrix	dimensionless	any	Matalas (1967); Tsoukalas et al. (2018)
`cross_correlation_lag1`	spatial	matrix	dimensionless	any	Matalas (1967); Tsoukalas et al. (2018)
`fdc`	fdc	curve	flow	any	Vogel and Fennessey (1995)
`fdc_log_rmse`	fdc	comparison	dimensionless	any	Vogel and Fennessey (1995)
`l_cv`	lmoments	scalar	dimensionless	any	Hosking (1990)
`l_skewness`	lmoments	scalar	dimensionless	any	Hosking (1990)
`l_kurtosis`	lmoments	scalar	dimensionless	any	Hosking (1990)
`annual_max_mean`	extremes	scalar	flow	any, 10+ years	Stedinger et al. (1993); Zaerpour et al. (2021)
`annual_max_cv`	extremes	scalar	dimensionless	any, 10+ years	Stedinger et al. (1993); Zaerpour et al. (2021)
`gev_rp10`	extremes	scalar	flow	any, 10+ years	Stedinger et al. (1993); Zaerpour et al. (2021)
`gev_rp50`	extremes	scalar	flow	any, 10+ years	Stedinger et al. (1993); Zaerpour et al. (2021)
`gev_rp100`	extremes	scalar	flow	any, 10+ years	Stedinger et al. (1993); Zaerpour et al. (2021)
`annual_min_mean`	extremes	scalar	flow	any, 10+ years	Stedinger et al. (1993); Zaerpour et al. (2021)
`annual_min_cv`	extremes	scalar	dimensionless	any, 10+ years	Stedinger et al. (1993); Zaerpour et al. (2021)
`seven_day_min_mean`	extremes	scalar	flow	daily, 10+ years	Stedinger et al. (1993); Zaerpour et al. (2021)
`seven_day_min_cv`	extremes	scalar	dimensionless	daily, 10+ years	Stedinger et al. (1993); Zaerpour et al. (2021)
`spectral_density`	spectral	curve	dimensionless	any	Nowak et al. (2011)
`low_frequency_variance_fraction`	spectral	scalar	dimensionless	any	Nowak et al. (2011)

Notes:

P-value metrics (monthly_ranksum_pvalue, monthly_levene_pvalue) are computed per realization and summarized as a rejection rate at alpha = 0.05, never as a pseudo relative error. Under a perfect generator the rejection rate is near alpha.
hurst is estimated on annually aggregated flows because sub-annual estimation conflates seasonal persistence with long-range dependence (Stedinger and Taylor, 1982).
Frequency gates are enforced automatically; skipped metrics are recorded in result.skipped with a reason.

Orchestrator and result¶

verify ¶

verify(ensemble: Ensemble, observed: DataFrame, metrics: Union[str, Iterable[Union[str, Callable]]], sites: Optional[list[str]] = None, frequency: Optional[str] = None, hurst_method: str = 'rs', acf_lags: Optional[int] = None) -> VerificationResult

Verify statistical property preservation of a synthetic ensemble.

Each selected metric is computed once on the observed record and once on every realization. The result reports the observed statistic against the distribution of the statistic across realizations, including the observed value's rank position within the ensemble (Stedinger and Taylor, 1982).

Parameters:

Name	Type	Description	Default
`ensemble`	`Ensemble`	Synthetic streamflow ensemble.	required
`observed`	`DataFrame`	Observed streamflow with a DatetimeIndex and sites as columns. Must have the same frequency as the ensemble.	required
`metrics`	`str, or list of str and callables`	Metric selection. `'all'` computes every registered metric. A list may mix metric names, category names (`'marginal'`, `'temporal'`, `'seasonal'`, `'annual'`, `'spatial'`, `'fdc'`, `'lmoments'`, `'extremes'`, `'spectral'`), and callables with signature `f(x: pd.Series) -> float`.	required
`sites`	`list of str`	Subset of sites to verify. Defaults to all sites shared by the ensemble and observed data.	`None`
`frequency`	`str`	Frequency override (pandas alias or `'daily'`, `'weekly'`, `'monthly'`, `'annual'`). By default the frequency is taken from ensemble metadata, checked against the index, and inferred from the index if metadata is missing.	`None`
`hurst_method`	`(rs, dfa)`	Hurst exponent estimation method.	`'rs'`
`acf_lags`	`int`	Maximum lag for the `acf` metric. Defaults to 30 for daily, 26 for weekly, 12 for monthly, and 10 for annual data.	`None`

Returns:

Type	Description
`VerificationResult`	Tidy per-realization values, skipped-metric log, and metadata. Use `.summary()` for the per-metric comparison table and `.category_summary()` for the per-category rollup.

Raises:

Type	Description
`ValueError`	If no metrics are selected, no shared sites exist, or the observed frequency does not match the ensemble frequency.

References

Stedinger, J.R. and Taylor, M.R. (1982). Synthetic streamflow generation: 1. Model verification and validation. Water Resources Research, 18(4), 909-918.

Examples:

>>> result = verify(ensemble, Q_obs, metrics="all")
>>> result.summary()
>>> result = verify(ensemble, Q_obs, metrics=["marginal", "acf"])

VerificationResult `dataclass` ¶

VerificationResult(values: DataFrame, skipped: DataFrame, metadata: dict = dict(), _suite: str = 'verification')

Bases: EvaluationResult

Result of a verification run (statistical property preservation).

Holds the tidy per-realization metric values produced by :func:synhydro.verification.verify. See :meth:to_dataframe for the tidy schema, :meth:summary for the per-metric comparison against observed, and :meth:category_summary for the per-category rollup.

Verification, in the sense of Stedinger and Taylor (1982), demonstrates that generated flows reproduce the statistics the generator was designed to reproduce: moments, correlations, and distributional shape.

References

Stedinger, J.R. and Taylor, M.R. (1982). Synthetic streamflow generation: 1. Model verification and validation. Water Resources Research, 18(4), 909-918.

Bootstrap and comparison tools¶

bootstrap_metric_ci ¶

bootstrap_metric_ci(values: FrameOrResult, n_bootstrap: int = 1000, confidence_level: float = 0.95, statistic: str = 'median', seed: Union[int, None] = None) -> pd.DataFrame

Bootstrap confidence intervals for ensemble metric estimates.

For each (metric, site, component) group, realizations are resampled with replacement n_bootstrap times; the chosen statistic (median or mean across realizations) is recomputed on each resample, and the confidence interval is taken from percentiles of the resampled statistics.

Parameters:

Name	Type	Description	Default
`values`	`VerificationResult, ValidationResult, or pd.DataFrame`	Result object or tidy frame from `verify()` or `validate()`.	required
`n_bootstrap`	`int`	Number of bootstrap resamples.	`1000`
`confidence_level`	`float`	Two-sided confidence level for the interval.	`0.95`
`statistic`	`(median, mean)`	Statistic of the per-realization values to bootstrap.	`'median'`
`seed`	`int`	Seed for reproducible resampling.	`None`

Returns:

Type	Description
`DataFrame`	One row per (category, metric, site, component) with columns `observed, estimate, ci_lower, ci_upper, relative_diff, rd_ci_lower, rd_ci_upper, n_realizations`. Relative columns are NaN for p-value metrics and comparison-kind metrics (which have no observed value).

References

Efron, B. and Tibshirani, R.J. (1993). An Introduction to the Bootstrap. Chapman and Hall.

compare_methods ¶

compare_methods(values_a: FrameOrResult, values_b: FrameOrResult, n_bootstrap: int = 1000, confidence_level: float = 0.95, seed: Union[int, None] = None) -> pd.DataFrame

Compare two generators' metric errors with a bootstrap test.

For each (metric, site, component) group shared by both results, the per-realization absolute error from observed is computed for each method, and the difference in mean absolute error (method A minus method B) is bootstrapped. When both ensembles have the same number of realizations, a paired bootstrap is used: one shared index draw resamples error pairs jointly. Otherwise the two error samples are resampled independently and paired is False.

For comparison-kind metrics (which measure divergence from observed directly), the metric value itself is used as the error.

Parameters:

Name	Type	Description	Default
`values_a`	`VerificationResult, ValidationResult, or pd.DataFrame`	Results from `verify()` or `validate()` on the same observed data. Run the orchestrator on each ensemble first.	required
`values_b`	`VerificationResult, ValidationResult, or pd.DataFrame`	Results from `verify()` or `validate()` on the same observed data. Run the orchestrator on each ensemble first.	required
`n_bootstrap`	`int`	Number of bootstrap resamples.	`1000`
`confidence_level`	`float`	Two-sided confidence level for the difference interval.	`0.95`
`seed`	`int`	Seed for reproducible resampling.	`None`

Returns:

Type	Description
`DataFrame`	One row per shared (category, metric, site, component) with columns `method_a_mae, method_b_mae, diff_estimate, diff_ci_lower, diff_ci_upper, significant, better_method, paired`. `significant` is True when the confidence interval for the difference excludes zero; `better_method` is `'a'` or `'b'` when significant, otherwise `'none'`.

References

Efron, B. and Tibshirani, R.J. (1993). An Introduction to the Bootstrap. Chapman and Hall.

Extending the suite¶

register_metric ¶

register_metric(func: Optional[Callable] = None, *, name: Optional[str] = None, category: str = 'custom', kind: str = 'scalar', needs: Iterable[str] = (), frequencies: Optional[Iterable[str]] = None, min_years: Optional[float] = None, units: str = 'dimensionless', summary_mode: str = 'distribution', citation: str = '', description: str = '') -> Callable

Register a custom verification metric.

Usable as a decorator or a plain call. Registered metrics are selectable by name in :func:verify and appear in :func:list_metrics.

Parameters:

Name	Type	Description	Default
`func`	`Callable`	Metric function `f(x: pd.Series, **opts) -> float` for scalar metrics; see :mod:`synhydro._evaluation._registry` kinds for curve, matrix, and comparison signatures.	`None`
`name`	`str`	Metric name; defaults to the function name.	`None`
`category`	`str`	Category label used for grouping and selection.	`'custom'`
`kind`	`str`	Metric kind: 'scalar', 'curve', 'matrix', or 'comparison'.	`'scalar'`
`needs`	`iterable of str`	Context attributes injected as keyword arguments (e.g. `('steps_per_year',)`).	`()`
`frequencies`	`iterable of str`	Base frequencies the metric supports; None means any.	`None`
`min_years`	`float`	Minimum observed record length in years.	`None`
`units`	`str`	Units label for reporting.	`'dimensionless'`
`summary_mode`	`str`	`'reject_rate'` for p-value metrics.	`'distribution'`
`citation`	`str`	Short citation shown in the metric inventory.	`''`
`description`	`str`	One-line description; defaults to the first docstring line.	`''`

Returns:

Type	Description
`Callable`	The registered function, unchanged.

Examples:

>>> from synhydro.verification import register_metric, verify
>>> @register_metric(category="custom", units="flow")
... def q25(x):
...     return float(x.quantile(0.25))
>>> result = verify(ensemble, Q_obs, metrics=["q25"])

list_metrics ¶

list_metrics() -> pd.DataFrame

List all registered verification metrics.

Returns:

Type	Description
`DataFrame`	One row per metric with columns `name, category, kind, units, frequencies, min_years, citation, description`.

Metric functions¶

All metric functions are directly callable, e.g. synhydro.verification.lag1_autocorrelation(series).

marginal ¶

Marginal (whole-record distributional) verification metrics.

These metrics summarize the marginal distribution of flows at a single site over the full record: moments, extremes of the sample, and flow quantiles. Preservation of the first two moments is the oldest and most universal verification target for synthetic streamflow generators (Matalas, 1967).

References

Matalas, N.C. (1967). Mathematical assessment of synthetic hydrology. Water Resources Research, 3(4), 937-945.

Stedinger, J.R. and Taylor, M.R. (1982). Synthetic streamflow generation: 1. Model verification and validation. Water Resources Research, 18(4), 909-918.

mean ¶

mean(x: Series) -> float

Mean flow over the full record.

std ¶

std(x: Series) -> float

Flow standard deviation (sample, ddof=1).

cv ¶

cv(x: Series) -> float

Coefficient of variation (std / mean).

skewness ¶

skewness(x: Series) -> float

Sample skewness (bias-corrected G1 estimator).

kurtosis ¶

kurtosis(x: Series) -> float

Excess kurtosis (Fisher definition, bias-corrected).

minimum ¶

minimum(x: Series) -> float

Minimum flow over the full record.

maximum ¶

maximum(x: Series) -> float

Maximum flow over the full record.

flow_q10 ¶

flow_q10(x: Series) -> float

10th percentile of the flow distribution.

flow_q50 ¶

flow_q50(x: Series) -> float

Median flow.

flow_q90 ¶

flow_q90(x: Series) -> float

90th percentile of the flow distribution.

ks_statistic ¶

ks_statistic(x: Series, reference: Series) -> float

Two-sample Kolmogorov-Smirnov distance from the observed distribution.

The maximum absolute difference between the empirical CDFs of one realization and the observed record. Zero indicates identical empirical distributions. The p-value is deliberately not reported: with long records the test rejects for trivial differences, and non-rejection is not evidence of equality.

temporal ¶

Temporal dependence verification metrics.

Short-term persistence (lag correlations, autocorrelation function) and long-term persistence (Hurst exponent). Following Stedinger and Taylor (1982), the Hurst exponent is estimated on annually aggregated flows: sub-annual records mix seasonal persistence into the estimate and make it unstable.

References

Matalas, N.C. (1967). Mathematical assessment of synthetic hydrology. Water Resources Research, 3(4), 937-945.

Koutsoyiannis, D. (2002). The Hurst phenomenon and fractional Gaussian noise made easy. Hydrological Sciences Journal, 47(4), 573-595.

Stedinger, J.R. and Taylor, M.R. (1982). Synthetic streamflow generation: 1. Model verification and validation. Water Resources Research, 18(4), 909-918.

lag1_autocorrelation ¶

lag1_autocorrelation(x: Series) -> float

Lag-1 autocorrelation of the full-record series.

lag2_autocorrelation ¶

lag2_autocorrelation(x: Series) -> float

Lag-2 autocorrelation of the full-record series.

acf ¶

acf(x: Series, acf_lags: int = 12) -> pd.Series

Autocorrelation function at lags 1 through acf_lags.

hurst ¶

hurst(x: Series, hurst_method: str = 'rs', steps_per_year: float = 12.0) -> float

Hurst exponent of annually aggregated flows.

Estimated on calendar-year totals because sub-annual estimation conflates seasonal persistence with long-range dependence (Stedinger and Taylor, 1982). Requires at least 20 years of data; estimates from records shorter than about 50 years remain noisy.

seasonal ¶

Seasonal (per-calendar-month) verification metrics.

Monthly-panel statistics are the dominant presentation convention of the nonparametric generation literature: a statistic computed per calendar month on every realization, compared against the observed monthly value (Lall and Sharma, 1996; Nowak et al., 2010).

All statistics here are computed per realization; realizations are never pooled before testing. Pooling inflates the synthetic sample size, so hypothesis-test p-values would shrink toward zero as the ensemble grows, a sample-size artifact rather than a quality signal.

References

Lall, U. and Sharma, A. (1996). A nearest neighbor bootstrap for resampling hydrologic time series. Water Resources Research, 32(3), 679-693.

Nowak, K., Prairie, J., Rajagopalan, B., and Lall, U. (2010). A nonparametric stochastic approach for multisite disaggregation of annual to daily streamflow. Water Resources Research, 46, W08529.

Herman, J.D., Zeff, H.B., Lamontagne, J.R., Reed, P.M., and Characklis, G.W. (2016). Synthetic drought scenario generation to support bottom-up water supply vulnerability assessments. Journal of Water Resources Planning and Management, 142(11), 04016050.

monthly_mean ¶

monthly_mean(x: Series) -> pd.Series

Mean flow per calendar month.

monthly_std ¶

monthly_std(x: Series) -> pd.Series

Flow standard deviation per calendar month (sample, ddof=1).

monthly_skewness ¶

monthly_skewness(x: Series) -> pd.Series

Sample skewness per calendar month (bias-corrected G1).

monthly_maximum ¶

monthly_maximum(x: Series) -> pd.Series

Maximum flow per calendar month.

monthly_minimum ¶

monthly_minimum(x: Series) -> pd.Series

Minimum flow per calendar month.

monthly_lag1_correlation ¶

monthly_lag1_correlation(x: Series) -> pd.Series

Correlation of each month's flow with the previous month's flow.

Computed on monthly mean flows across years: for calendar month m, the correlation between month-m values and month-(m-1) values. This seasonally varying month-to-month correlation is the central temporal check of Kirsch et al. (2013).

monthly_ranksum_pvalue ¶

monthly_ranksum_pvalue(x: Series, reference: Series) -> pd.Series

Wilcoxon rank-sum p-value per calendar month, one realization vs observed.

Tests whether the realization's month-m values and the observed month-m values come from distributions with the same location (Herman et al., 2016). Summarized as the fraction of realizations rejecting at alpha = 0.05; under a perfect generator this rejection rate is near alpha.

monthly_levene_pvalue ¶

monthly_levene_pvalue(x: Series, reference: Series) -> pd.Series

Levene test p-value per calendar month, one realization vs observed.

Tests whether the realization's month-m values and the observed month-m values have equal variance (Herman et al., 2016). Summarized as the fraction of realizations rejecting at alpha = 0.05.

annual ¶

Annual aggregate verification metrics.

Statistics of calendar-year total flows. Annual standard deviation is the classic detector of generators that reproduce sub-annual statistics but fail to carry persistence up to interannual variability (Srinivas and Srinivasan, 2005; Borgomeo et al., 2015).

References

Stedinger, J.R. and Taylor, M.R. (1982). Synthetic streamflow generation: 1. Model verification and validation. Water Resources Research, 18(4), 909-918.

Srinivas, V.V. and Srinivasan, K. (2005). Hybrid moving block bootstrap for stochastic simulation of multi-site multi-season streamflows. Journal of Hydrology, 302(1-4), 307-330.

annual_mean ¶

annual_mean(x: Series, steps_per_year: float = 12.0) -> float

Mean of calendar-year total flows.

annual_sd ¶

annual_sd(x: Series, steps_per_year: float = 12.0) -> float

Standard deviation of calendar-year total flows.

Generators that do not carry sub-annual persistence up to the annual scale understate this statistic (Srinivas and Srinivasan, 2005).

annual_cv ¶

annual_cv(x: Series, steps_per_year: float = 12.0) -> float

Coefficient of variation of calendar-year total flows.

annual_skewness ¶

annual_skewness(x: Series, steps_per_year: float = 12.0) -> float

Sample skewness of calendar-year total flows.

annual_lag1_autocorrelation ¶

annual_lag1_autocorrelation(x: Series, steps_per_year: float = 12.0) -> float

Lag-1 autocorrelation of calendar-year total flows.

annual_minimum ¶

annual_minimum(x: Series, steps_per_year: float = 12.0) -> float

Minimum calendar-year total flow (driest year).

annual_maximum ¶

annual_maximum(x: Series, steps_per_year: float = 12.0) -> float

Maximum calendar-year total flow (wettest year).

spatial ¶

Spatial (cross-site) verification metrics.

Lag-0 cross-correlations between site pairs are the standard check of multisite dependence preservation (Matalas, 1967). Lag-1 cross correlations are directional and are not preserved by all multivariate generation schemes, which makes them a useful additional diagnostic.

References

Matalas, N.C. (1967). Mathematical assessment of synthetic hydrology. Water Resources Research, 3(4), 937-945.

Tsoukalas, I., Efstratiadis, A., and Makropoulos, C. (2018). Stochastic periodic autoregressive to anything (SPARTA): Modeling and simulation of cyclostationary processes with arbitrary marginal distributions. Water Resources Research, 54(1), 161-185.

cross_correlation ¶

cross_correlation(frame: DataFrame) -> pd.Series

Lag-0 Pearson correlation for each unordered site pair.

cross_correlation_lag1 ¶

cross_correlation_lag1(frame: DataFrame) -> pd.Series

Lag-1 cross-correlation for each ordered site pair.

The correlation of flow at the first site with the previous timestep's flow at the second site. Pair labels are directional (A->B correlates A at time t with B at time t-1).

fdc ¶

Flow duration curve verification metrics.

The flow duration curve (FDC) summarizes the full marginal distribution as flow versus exceedance probability and is a standard synthesis verification target (Vogel and Fennessey, 1995).

References

Vogel, R.M. and Fennessey, N.M. (1995). Flow duration curves II: A review of applications in water resources planning. Water Resources Bulletin, 31(6), 1029-1039.

fdc ¶

fdc(x: Series) -> pd.Series

Flow at fixed exceedance probabilities (flow duration curve).

Components are exceedance probabilities: 0.01 is a high flow exceeded 1 percent of the time, 0.99 a low flow exceeded 99 percent of the time.

fdc_log_rmse ¶

fdc_log_rmse(x: Series, reference: Series) -> float

RMSE between log-space flow duration curves, one realization vs observed.

Computed over the fixed exceedance grid with flows floored at 1e-6 before taking logs. Zero indicates identical curves; log space weights low-flow and high-flow errors comparably.

lmoments ¶

L-moment ratio verification metrics.

L-moment ratios are linear combinations of order statistics that characterize distribution shape more robustly than product moments for skewed hydrologic data and small samples (Hosking, 1990).

References

Hosking, J.R.M. (1990). L-moments: Analysis and estimation of distributions using linear combinations of order statistics. Journal of the Royal Statistical Society, Series B, 52(1), 105-124.

Hosking, J.R.M. and Wallis, J.R. (1997). Regional Frequency Analysis: An Approach Based on L-Moments. Cambridge University Press.

l_cv ¶

l_cv(x: Series) -> float

L-coefficient of variation (tau-2 = L-scale / L-mean).

l_skewness ¶

l_skewness(x: Series) -> float

L-skewness ratio (tau-3).

l_kurtosis ¶

l_kurtosis(x: Series) -> float

L-kurtosis ratio (tau-4).

extremes ¶

Extreme flow verification metrics.

Annual maxima and minima statistics, GEV return-period quantiles fit to annual maxima, and 7-day low flows for daily records. Comparing GEV quantile estimates across the ensemble against the observed-record fit follows Zaerpour et al. (2021).

References

Stedinger, J.R., Vogel, R.M., and Foufoula-Georgiou, E. (1993). Frequency analysis of extreme events. In Handbook of Hydrology, edited by D.R. Maidment, McGraw-Hill, Chapter 18.

Zaerpour, M., Papalexiou, S.M., and Nazemi, A. (2021). Informing stochastic streamflow generation by large-scale climate indices at single and multiple sites. Advances in Water Resources, 156, 104037.

annual_max_mean ¶

annual_max_mean(x: Series, steps_per_year: float = 12.0) -> float

Mean of annual maximum flows.

annual_max_cv ¶

annual_max_cv(x: Series, steps_per_year: float = 12.0) -> float

Coefficient of variation of annual maximum flows.

gev_rp10 ¶

gev_rp10(x: Series, steps_per_year: float = 12.0) -> float

GEV 10-year return level of annual maximum flows (L-moment fit).

gev_rp50 ¶

gev_rp50(x: Series, steps_per_year: float = 12.0) -> float

GEV 50-year return level of annual maximum flows (L-moment fit).

gev_rp100 ¶

gev_rp100(x: Series, steps_per_year: float = 12.0) -> float

GEV 100-year return level of annual maximum flows (L-moment fit).

annual_min_mean ¶

annual_min_mean(x: Series, steps_per_year: float = 12.0) -> float

Mean of annual minimum flows.

annual_min_cv ¶

annual_min_cv(x: Series, steps_per_year: float = 12.0) -> float

Coefficient of variation of annual minimum flows.

seven_day_min_mean ¶

seven_day_min_mean(x: Series, steps_per_year: float = 365.25) -> float

Mean of annual 7-day minimum flows (daily records only).

seven_day_min_cv ¶

seven_day_min_cv(x: Series, steps_per_year: float = 365.25) -> float

Coefficient of variation of annual 7-day minimum flows (daily only).

spectral ¶

Spectral verification metrics.

The distribution of variance across period bands summarizes how much of a series' variability lives at seasonal, interannual, and longer time scales. Comparing synthetic and observed spectra follows the wavelet autoregressive modeling lineage (Nowak et al., 2011).

References

Nowak, K.C., Rajagopalan, B., and Zagona, E. (2011). Wavelet Auto- Regressive Method (WARM) for multi-site streamflow simulation of data with non-stationary spectra. Journal of Hydrology, 410(1-2), 1-12.

spectral_density ¶

spectral_density(x: Series, steps_per_year: float = 12.0) -> pd.Series

Fraction of spectral variance in fixed period bands.

The periodogram of the standardized series is integrated over period bands (in years) and normalized to sum to one. Components are band labels ordered from the longest periods to the shortest.

low_frequency_variance_fraction ¶

low_frequency_variance_fraction(x: Series, steps_per_year: float = 12.0) -> float

Fraction of spectral variance at periods longer than 2 years.

Low values indicate a series dominated by seasonal and shorter variability; generators that ignore interannual persistence understate this fraction.

Verification¶

How results are reported¶

Metric kinds¶

Metric selection¶

Metric reference¶

Orchestrator and result¶

verify ¶

VerificationResult dataclass ¶

Bootstrap and comparison tools¶

bootstrap_metric_ci ¶

compare_methods ¶

Extending the suite¶

register_metric ¶

list_metrics ¶

Metric functions¶

marginal ¶

mean ¶

std ¶

cv ¶

skewness ¶

kurtosis ¶

minimum ¶

maximum ¶

flow_q10 ¶

flow_q50 ¶

flow_q90 ¶

ks_statistic ¶

temporal ¶

lag1_autocorrelation ¶

lag2_autocorrelation ¶

acf ¶

hurst ¶

seasonal ¶

monthly_mean ¶

monthly_std ¶

monthly_skewness ¶

monthly_maximum ¶

monthly_minimum ¶

monthly_lag1_correlation ¶

monthly_ranksum_pvalue ¶

monthly_levene_pvalue ¶

annual ¶

annual_mean ¶

annual_sd ¶

annual_cv ¶

annual_skewness ¶

annual_lag1_autocorrelation ¶

annual_minimum ¶

annual_maximum ¶

spatial ¶

cross_correlation ¶

cross_correlation_lag1 ¶

fdc ¶

fdc ¶

fdc_log_rmse ¶

lmoments ¶

l_cv ¶

l_skewness ¶

l_kurtosis ¶

extremes ¶

annual_max_mean ¶

annual_max_cv ¶

gev_rp10 ¶

gev_rp50 ¶

gev_rp100 ¶

annual_min_mean ¶

annual_min_cv ¶

seven_day_min_mean ¶

seven_day_min_cv ¶

spectral ¶

spectral_density ¶

low_frequency_variance_fraction ¶

VerificationResult `dataclass` ¶