Did the ECMWF seasonal forecast model outperform a statistical model over the last 15 years?

Since 1997 the European Centre for Medium-Range Weather Forecasts (ECMWF) has made seasonal forecasts with ensembles of a coupled ocean-atmosphere model (S1). In January 2002, a new version (S2) was introduced. For the calibration of these models, hindcasts have been performed starting in 1987, so that 15 years of hindcasts and
forecasts are now available for verification.

Seasonal predictability is to a large extent due to the El Niño - Southern Oscillation (ENSO) climate oscillations. ENSO predictions of the ECMWF models are compared with those of a few statistical models, mainly based on past sea surface temperature observations. The relative skill depends strongly on the season. The dynamical models are much better at forecasting the onset of El Niño or La Niña in
boreal spring to summer. The statistical models are comparable at predicting the evolution of an event in boreal fall and winter.

El Niño and La Niña perturb the average weather in many regions and seasons throughout the world. A set of statistical models (STAT) is constructed based on persistence and a lagged regression with an
ENSO index over 1901-1986 wherever the correlations are significant. As the number of verification data points is very low (15), the simplest measure of skill is used: the anomaly correlation coefficient of the ensemble mean. To further reduce the sampling uncertainties we restrict ourselves to areas and seasons of known ENSO teleconnections.

The dynamical ECMWF models show better skill in 2-meter temperature forecasts over sea and the tropical land areas than STAT. The modeled ENSO teleconnection pattern to North America is shifted
relative to observations, leading to little pointwise skill. Precipitation forecasts of the ECMWF models are very good, better than those of the statistical model, in South-East Asia, the equatorial Pacific and in the Americas in Dec-Feb but in Mar-May the skill is lower there. Overall, S1(S2) show better skill than STAT at lead time 2 months in 29(32) out of 40 regions and seasons of known ENSO teleconnections.