Signature Kernel Scoring Rule: A Spatio-Temporal Diagnostic for Probabilistic Weather Forecasting

arXiv stat.ML / 5/1/2026

💬 OpinionDeveloper Stack & InfrastructureIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper argues that common evaluation/training metrics like MSE are ill-suited for probabilistic weather forecasting because they treat each time point independently and miss spatio-temporal correlations inherent in weather dynamics.
It introduces the “signature kernel scoring rule,” which represents weather variables as continuous spatio-temporal paths and uses iterated integrals to encode temporal and spatial dependencies.
The proposed scoring rule is shown to be strictly proper (a desirable property for probabilistic forecast verification) using path augmentations to ensure uniqueness.
Experiments on WeatherBench 2 weather scorecards indicate strong discriminative power and improved ability to capture path-dependent interactions.
The authors also train sliding-window generative neural networks on ERA5 using a predictive-sequential scoring rule and find that signature-kernel-based training beats climatology for forecast paths up to 15 timesteps.

Abstract

Modern weather forecasting has increasingly transitioned from numerical weather prediction (NWP) to data-driven machine learning forecasting techniques. While these new models produce probabilistic forecasts to quantify uncertainty, their training and evaluation may remain hindered by conventional scoring rules, primarily MSE, which are designed for single time point predictions and ignore the highly correlated data structures present in weather behaviour. This work introduces the signature kernel scoring rule to the domain of weather forecasting, which reframes weather variables as continuous paths to encode temporal and spatial dependencies through iterated integrals. Validated as strictly proper through the use of path augmentations to guarantee uniqueness, the signature kernel provides a theoretically robust metric for forecast verification and model training. Empirical evaluations through weather scorecards on WeatherBench 2 models demonstrate the signature kernel scoring rule's high discriminative power and unique capacity to capture path-dependent interactions. Following previous demonstration of successful adversarial-free probabilistic training, we train sliding window generative neural networks using a predictive-sequential scoring rule on ERA5 reanalysis weather data. Using a lightweight model, we demonstrate that signature kernel based training outperforms climatology for forecast paths of up to fifteen timesteps.