News

A comparative verification of raw and bias-corrected ECMWF seasonal ensemble precipitation reforecasts in Java (Indonesia)

17 July 2019

Seasonal forecasts of precipitation are becoming an increasingly important element in decision making systems in Indonesia, especially in the agriculture and hydrological sectors. In these two sectors, stakeholders and decision makers need seasonal forecasts to assist them in their planning strategy. However, seasonal forecasts from global circulation models are afflicted with biases to a degree that precludes their direct use, including seasonal ensemble forecasts of precipitation from the new ECMWF Seasonal Forecast System 5 (ECMWF-SEAS5). Lead author Dian Nur Ratri says "Our study shows that biases can be corrected using Empirical Quantile Mapping (EQM) bias correction (BC)". The results were published in the Journal of Applied Meteorology and Climatology.

BACKGROUND
Predicting when above or below average rainfall might occur can be valuable for agriculture because such information can help farmers to decide the type of crop that they will plant during that season. For example, farmers may plant a crop that requires less water if they have been informed in advance that the forecast rainfall will be below average. The seasonal forecast is also important in an irrigated system regarding the availability of irrigation water as well as the timing of high/low river stream flows. Currently, demands on seasonal forecasting are getting higher: users require seasonal rainfall forecasts that are skillful, statistically reliable, and free of bias (e.g. Schepen et al. 2014).  

DATA AND METHOD
EQM is used in this study. This is a popular BC method and works with empirical probability density functions (PDFs) or cumulative distribution functions (CDFs) for both the forecasts and the observations. We bias correct SEAS5 24-hour rainfall accumulations at seven monthly lead-times, over the period 1981–2010 in Java, Indonesia. For the observations we have used a new high-resolution (0.25o) land-only gridded rainfall dataset (SA-OBS). A comparative verification of both raw and bias-corrected reforecasts is performed using several verification metrics. In this verification, the daily rainfall data was aggregated to monthly accumulated rainfall. We focus on July, August and September because these are agriculturally important months.
 

A figure showing that the bias-corrected forecasts are more skilful according to the CRPS
Figure 1. CRPSS for raw (red) and bias-corrected (blue) forecasts valid in July, August and September. The solid lines indicate the median and the shaded areas show the CRPSS spread of the grid cells between the 0.025 and 0.975 quantiles (dotted lines).
A figure showing that the bias-corrected forecasts are more skilful according to the BSS
Figure 2. BSS for raw (red) and bias-corrected (blue) forecasts as a function of threshold for 1-month lead time
A figure showing the higher potential economic value of the bias-corrected forecasts
Figure 3. Potential Economic Value (PEV) of raw (red) and bias-corrected (blue) forecasts as a function of cost-loss ratio for 1-month lead times, 100-mm threshold, in July, August, and September

RESULTS
The results in this study show the skill and potential economic value of SEAS5 raw and bias-corrected precipitation reforecasts for July – September in Java. For the dry season (May to November), the bias-corrected SEAS5 forecasts show positive CRPSS values (with climatology as a reference) for most grid cells for 1-month lead time and for July – October  also 2-month lead times. In July and August (September) the median CRPSS values of lead times ≤ 2 (3) months are positive after BC, but they are still negative for longer lead times (lead times 5 to 7 months for July, 3 to 7 for August and 4 to 7 for September) [Figure 1]. 
According to the Brier skill score (BSS) the BC reforecasts improve upon the raw reforecasts for the lower precipitation thresholds, at  the 1-month lead time. The BSS values of the bias-corrected SEAS5 forecasts are lower in August compared to July and September, maybe because August is mostly the peak of the dry season. In July and August at the threshold of 50 mm, the median BSS reaches it maximum value (nearly 0.5 in July) and the range of BSS values is almost entirely positive. The bias-corrected forecasts still show negative BSS for some grid cells and especially the higher thresholds, but the BSS is generally higher than for the raw forecasts [Figure 2]. 
BC by EQM is not only able to improve forecast skill (as defined by the BSS and CRPSS), but also enhances the potential economic value of the SEAS5 forecasts, especially in the dry season, which is an agriculturally important season. The corrected SEAS5 forecasts have a higher PEV than the raw forecasts for all ranges of users, or in other words the bias-corrected SEAS5 forecasts are more valuable than the raw forecasts, even for the longest lead time of 7 months [Figure 3].

 

Ratri, D.N., K. Whan, and M. Schmeits, 2019: A comparative verification of raw and bias-corrected ECMWF seasonal ensemble precipitation reforecasts in Java (Indonesia). Journal of Applied Meteorology and Climatology.