Validation¶

The synhydro.validation module evaluates fit for purpose: whether a synthetic ensemble reproduces characteristics that the generator was not explicitly fit to, currently drought duration, severity, and frequency.

The terminology follows Stedinger and Taylor (1982). Validation demonstrates that characteristics not explicitly reproduced by parameter estimation, such as drought and storage behavior, are consistent with the historical record. For statistical property preservation, see Verification.

Results are reported in the same form as the verification suite: each statistic is computed once on the observed record and once per realization, and the observed value is compared against the distribution across realizations. See Verification for the summary columns.

Storage-yield and further run-theory statistics (sequent-peak storage, reliability) are planned extensions of this suite.

Metric selection¶

import synhydro

result = synhydro.validate(ensemble, Q_obs, metrics="all")
result = synhydro.validate(ensemble, Q_obs, metrics=["threshold_drought"])
result.summary()

Metric reference¶

Metric	Category	Units	Description	Citation
`mean_drought_duration`	threshold_drought	timesteps	Mean below-threshold run length	Yevjevich (1967); Salas et al. (1980)
`mean_drought_severity`	threshold_drought	flow (cumulative deficit)	Mean cumulative deficit per event	Yevjevich (1967); Salas et al. (1980)
`max_drought_duration`	threshold_drought	timesteps	Longest below-threshold run	Yevjevich (1967); Salas et al. (1980)
`max_drought_severity`	threshold_drought	flow (cumulative deficit)	Largest cumulative deficit	Yevjevich (1967); Salas et al. (1980)
`drought_frequency`	threshold_drought	events per timestep	Drought events per timestep	Yevjevich (1967); Salas et al. (1980)
`ssi_mean_drought_duration`	ssi_drought	timesteps	Mean SSI drought event length	McKee et al. (1993)
`ssi_max_drought_duration`	ssi_drought	timesteps	Longest SSI drought event	McKee et al. (1993)
`ssi_mean_drought_severity`	ssi_drought	dimensionless	Mean cumulative absolute SSI per event	McKee et al. (1993)
`ssi_max_drought_severity`	ssi_drought	dimensionless	Largest cumulative absolute SSI	McKee et al. (1993)
`ssi_drought_frequency`	ssi_drought	events per year	SSI drought events per year	McKee et al. (1993)

Notes:

Threshold drought events are below-threshold runs: an event is a maximal sequence of consecutive timesteps with flow below the truncation level (by default the 20th percentile of observed flows at each site). Severity is the cumulative deficit, the sum of (threshold minus flow) over the event.
SSI drought events start when the Standardized Streamflow Index drops to -1 or below and end when it returns to zero or above. The SSI is fit on the observed record and the same fitted distributions transform every realization, so synthetic droughts are measured on the observed climatology.

Orchestrator and result¶

validate ¶

validate(ensemble: Ensemble, Q_obs: DataFrame, metrics: Union[str, Iterable[str]], sites: Optional[list[str]] = None, frequency: Optional[str] = None, drought_threshold: Optional[float] = None, ssi_timescale: int = 12, ssi_dist: str = 'gamma') -> ValidationResult

Validate fit-for-purpose drought behavior of a synthetic ensemble.

Each drought statistic is computed once on the observed record and once per realization; the result reports the observed value against the distribution across realizations, as in :func:synhydro.verification.verify.

Parameters:

Name	Type	Description	Default
`ensemble`	`Ensemble`	Synthetic streamflow ensemble.	required
`Q_obs`	`DataFrame`	Observed streamflow with a DatetimeIndex and sites as columns. Must have the same frequency as the ensemble.	required
`metrics`	`str or list of str`	`'all'` or a list of categories: `'threshold_drought'` (run-theory events below a flow threshold) and `'ssi_drought'` (events on the Standardized Streamflow Index).	required
`sites`	`list of str`	Subset of sites. Defaults to all sites shared by the ensemble and observed data.	`None`
`frequency`	`str`	Frequency override; see :func:`synhydro.verification.verify`.	`None`
`drought_threshold`	`float`	Flow threshold for threshold drought identification, applied to every site. If None, each site uses the 20th percentile of its observed flows.	`None`
`ssi_timescale`	`int`	SSI accumulation timescale in months.	`12`
`ssi_dist`	`str`	Distribution used for the SSI fit.	`'gamma'`

Returns:

Type	Description
`ValidationResult`	Tidy per-realization values, skipped-metric log, and metadata.

Raises:

Type	Description
`ValueError`	If no metrics are selected, no shared sites exist, or the observed frequency does not match the ensemble frequency.

References

Stedinger, J.R. and Taylor, M.R. (1982). Synthetic streamflow generation: 1. Model verification and validation. Water Resources Research, 18(4), 909-918.

Examples:

>>> result = validate(ensemble, Q_obs, metrics="all")
>>> result.summary()

ValidationResult `dataclass` ¶

ValidationResult(values: DataFrame, skipped: DataFrame, metadata: dict = dict(), _suite: str = 'validation')

Bases: EvaluationResult

Result of a validation run (fit-for-purpose evaluation).

Holds the tidy per-realization metric values produced by :func:synhydro.validation.validate. See :meth:to_dataframe for the tidy schema, :meth:summary for the per-metric comparison against observed, and :meth:category_summary for the per-category rollup.

Validation, in the sense of Stedinger and Taylor (1982), demonstrates that generated flows reproduce characteristics not explicitly fit by the generator, such as drought duration and severity.