Meteorological parameters
For the 2-week period examined in this study, the skill of the basic
meteorological model performance is presented in Fig. in terms of
bias and unbiased forecast error for relative humidity, wind speed and
temperature for the 42 h forecasts. The statistics are based on all forecasts
at all 10 observation sites for each forecast length. The unbiased forecast
error averaged over the forecast window amounts to about 7 %,
2.3 m s-1 and 0.9 ∘C for relative humidity, wind speed and
temperature, respectively. The forecast error is expected to increase with
forecast length, which is the case for temperature (Fig. c). The
unbiased forecast error of the wind speed and relative humidity
(Fig. a and b), however, displays a flat shape suggesting that the
error is saturated during the forecast window. This behaviour points to a
difficulty in the analysis, i.e. the initialization of the wind field in the
model. One aspect in this regard is the lack of wind observations in the
planetary boundary layer to initialize the meso-scale wind field.
The temperature bias increases from -0.3 ∘C to close to
-1 ∘C after 42 h (Fig. c). This behaviour might be
caused by a spin-up problem in the model, since it is changing with forecast
length. Additionally, a negative temperature bias that is not changing with
forecast length can be attributed to the warm turbine affecting the
measurements. A difference in the temperature bias between mast (Mast) and turbine
(WT) measurements amounts on average to -0.4 ∘C for mast and
-1.1 ∘C for turbine.
The bias of the wind speed (Fig. b)
decreases slightly with forecast length from 1 to 0.5 m s-1 after
42 h. The bias in the relative humidity (Fig. a) displays only
small changes with forecast length and remains generally small with absolute
values less than 1 %.
In general, the meteorological model displays good performance for the basic
meteorological parameters. The lack of forecast error growth in wind speed
and relative humidity suggests that the model forecast could be improved by
assimilating more observations, especially for wind and humidity. A better
initial state of the forecast might even improve the bias behaviour.
CM: unbiased forecast error SD and bias for (a) relative
humidity in percent, (b) wind speed in m s-1 and
(c) temperature in ∘C for increasing forecast length. The
statistical measures are calculated for the 2-week period and averaged over
all available stations. The meteorological observations at 60 to 150 m
height above ground from 10 sites (9 for wind speed) are used.
![]()
The meteorological performance of the different uncertainty quantification
approaches in terms of spread and skill is shown in Fig. . The
skill as defined as the unbiased forecast error SD, and spread for the
temperature forecasts is displayed for each approach. The baseline approach
of a deterministic forecast (CM) shows the largest forecast error. For the
examined period, the benefit of using the neighbourhood and/or ensemble methods
can already be seen in the first hours of the forecasts, and increases with
forecast length as expected. The CMngb has the smallest improvement of
forecast error, but still suggesting that the neighbourhood method is
valuable if an ensemble forecast is not available. Even higher improvement
is achieved by the approaches ENS and EM that provide by construction the
same values here. The largest reduction of the forecast error is achieved
using both the ensemble forecast method and the neighbourhood method
(ENSngb), with an average reduction of 9 % for the temperature, 7 %
for the wind speed and 12 % for the relative humidity forecast error,
averaged for all sites and all forecasts averaged for all sites and all
forecasts.
Unbiased forecast error SD for all uncertainty quantification
approaches for the temperature in ∘C and the spread, SPRE, for the
three approaches consisting of multiple forecasts as a function of increasing
forecast length. The SD is calculated for the average forecast in case of
multiple forecasts, i.e. approaches CMngb, ENS and ENSngb.
![]()
Figure also displays the forecast spread of the approaches ENS,
ENSngb and CMngb. The spread is always clearly lower than the unbiased
forecast error, which means that the forecast uncertainty is underestimated
in all approaches. This behaviour might result from neglected uncertainty
sources in the modelling chain (Fig. ) The employed ensemble
represents uncertainties in the initial and boundary conditions, but it does
not take into account uncertainties in the model physics or numerical
formulations. There are methods to account for these uncertainties, such as
stochastic physics, which increase the spread , but they
are not in the scope of this study. It is also important to consider the
observational error when validating the spread–skill relationship of the
forecast, as discussed in Sect. . Since the forecast error estimate
is not corrected for the observational error, the under-dispersiveness of the
approaches is not as large as it appears in Fig. .
On its own, the ensemble forecasting method (ENS) has better skill, spread
and spread–skill relationship than the neighbourhood method (CMngb) for this
period, around 0.6 and 0.3, respectively, for all meteorological
parameters. Specifically, the spread–skill relationship improves from around
0.3 for CMngb to around 0.6 for ENS for temperature, wind speed and
relative humidity. The approach ENSngb results in the best spread–skill
relationship around 0.7. This implies that the neighbourhood method is
adding additional information about the forecast uncertainty, which is
valuable for both the forecast skill and the forecast spread. The spread
resulting from the neighbourhood method is constant with forecast length
(CMngb in Fig. ), since it represents the internal variations in
the weather over the scale of the neighbourhood domain, but it does not take
into account uncertainties due to initial boundary conditions, or model
formulations that increase with increasing forecast length.
Site A. (a) Forecasted ice load in kg m-2. Blue lines
are ensemble members, grey line is the CM and black line is the ENS mean. The
ice load observations are in red. (b) The related spread of the
ensemble members in ENS. (c) Forecasted ENS ensemble mean liquid
water content in g kg-1 for the period.
![]()
Forecasted icing and production losses
The forecasted icing is, as mentioned before, more difficult to validate
since the observations of icing are unreliable. Alternatively, the forecasted
production loss for the three sites with production data is also considered
as a measure of the forecast skill of the icing. Results from two of the
three sites, here A and B, with consistent production observations will be
presented in more detail in the following to point out some interesting
features when using the probabilistic forecasts. Site C is not shown, but had
icing during about 3 days of the 2 weeks. In Sect. , the
average skill and spread of the different approaches for the three sites are
discussed. Sites A and C were affected by a frontal passage during the 2 weeks,
while site A experienced general cloudiness and was less influenced by
this front.
Site A
Two icing events were forecasted at site A during the 2 weeks. During the
first icing event, around the 30 December, only
a small amount of ice is forecasted, while during the second icing event,
starting around 1 January, both the modelled and observed ice load amounted to
several kg m-2. In Fig. a the forecasted icing using the
ENS approach and the CM approach is presented together with the ensemble mean
of the ENS approach and observed ice load. The spread of the ensemble members
from the ENS, which displays the forecast uncertainty in the icing, are
presented in Fig. b. During the icing events, the spread signals
uncertainty in the ice amount. The largest forecast uncertainty is found at the
end of the second icing event with around 1.5 kg m-2. The magnitude of
the overall forecast uncertainty, or spread, amounts to about 50 % of the
ice load here.
Site A. (a) Forecasted production loss in percent. Black
line is the ensemble mean using ENS, grey line is the CM and red line is the
observed production loss. (b) The related spread of the ensemble
members in ENS.
![]()
In this period, first the build up and then the loss of ice happen almost
simultaneously for all the ensemble members. This is especially visible in
the spread for the second icing event with values close to zero at the start
and end of the event. Forecasted and observed production loss for the site
allow for studying the timing during the event in more detail
(Fig. a). The forecasted production loss starts about 12 h later
than the observed production loss, pointing to a problem with the agreement
between the ensemble members. A closer examination reveals low forecasted
liquid and solid water contents in the beginning of the icing period
(Fig. c), resulting in no accumulation of ice by the icing model.
Instead the ice starts to accumulate in the model when solid water components
are forecasted, in addition to the liquid water content. Interestingly, all
ensemble members behave very similarly with this timing (not shown). This
could be related to an error in the modelled cloud characteristics or in the
icing model causing a too slow ice buildup.
Additionally, the ensemble fails to describe the end of the icing period
(6–9 January) where the modelled production loss drops to zero for the
remaining period, while the observed production loss is high around 8 January
(Fig. a). The end of the 2 weeks also lacks forecasted liquid
and solid water contents, and thus no further buildup by the icing model is
possible. Again, the ensemble was overconfident with all members displaying a
similar behaviour. By including the neighbourhood method to the ensemble,
there is some improvement in the end of the 2 weeks adding some forecasted
liquid water content at the end of the simulation (Fig. ) in some
members. However, the amount of forecasted water content was still too low to
generate any notable buildup by the icing model.
A closer look at the weather situation shows that site A is affected by a
frontal passage during the second week. The ensemble members all have the
front passage at the same time, resulting in similar values for cloud cover
as well as liquid and solid water contents. This problem results from the
insufficient representation of the uncertainties in the boundaries. The
boundary data from different members of the ECMWF EPS prescribes very similar
surface pressure patterns for this event resulting in an overconfidence in
the arrival time of the cloud front. Furthermore, small scales are
initialized for all ensemble members in the same way using the control member
analysis. During the icing event, however, the ensemble members have a spread
of around 25 % of the ENS mean value for liquid and solid water
components, leading to a spread in the amount of built-up ice
(Fig. c). In it was shown using a similar model
that the resulting ice load was highly sensitive to a variation in the
ingoing median volume droplet size of the water droplets, suggesting that
this is an uncertain part of the icing model. Here, the droplet size is
calculated from the liquid/solid water content before being used in the icing
model. This is discussed further in the section below (Sect. ).
Site A. Forecasted liquid water content in g kg-1 using the
ENSngb approach.
![]()
Site B
At site B only one icing event was forecasted and observed, starting already
on the first days of the period. This site was not as strongly affected by
frontal passages as site A, and had more general cloudiness during the
period. Figure a–c shows the forecasted and observed ice load,
spread of forecasted ice load and liquid water content. In
Fig. a–b the forecasted and observed production loss as well as
the related forecast spread is presented.
Same as Fig. , but for Site B.
![]()
In the beginning of the 2 weeks, there is 1 day with a large amount of
liquid and solid water content; however, no ice started to build up during
this day, mainly because of too low wind speeds. At the start of the actual
icing event, the ice growth is first small, but with increasing pace after
1 January 2012, reaching the largest load of about 2.5 kg m-2 on
4 January (Fig. a). The observed production loss shows a similar
behaviour (Fig. ). In Fig. a the ENS mean median volume diameter (MVD) for the
solid water components can be seen for this site and the corresponding mean
spread is shown in Fig. b. The liquid water components ENS mean MVD
is not shown, as it follows the liquid water content amount in this case (Fig. c).
Large values of MVD (Fig. a) coincide with large icing rates
(Fig. a) stressing the role of MVD for the icing intensity. The
large spread of the MVD around 4 January (Fig. b) can be connected
to the ice load differences of the different ensemble members
(Fig. a) and also to the simultaneous spread in the production
loss forecast (Fig. b). This behaviour agrees with the effect of
different MVDs discussed in .
Generally, the forecasted ice and production loss displays a reasonable
agreement with observations. However, there are two interesting deviations.
Firstly, in the first half of the time series the forecast is generally
underestimating the observed production loss (Fig. ), while the
forecasted ice load agrees well with the observations (Fig. a).
The reason of the disagreement seems to be related to deficiencies in our
production model. Secondly, the observed drop in both ice load and production
loss at the end of the period is delayed in all forecasts, opposite to the
behaviour at site A.
The high forecast spread, or forecasted uncertainty in the icing forecast
(Fig. b), shows the value of having a probabilistic forecast. This
variation of the ensemble members is probably due to a large variation in the
liquid and solid water contents (between 25 % and 50 % of the mean amount)
resulting in a variation of the calculated MVD. The benefit of an ensemble in
this case can also be seen in Fig. a where the ensemble mean
(black line) of the production loss forecast starts to decrease, getting
closer to observations, while the CM is remaining nearly at 100 %. The
forecast spread of the production loss is also increasing during the last day
of the period, suggesting an enlarged uncertainty in the forecast. Generally,
forecasting the end of an icing period has been shown to be challenging due
to the difficulties in modelling ice loss (e.g. ). On the
other hand the start of the stronger icing period from the 3 January is well
timed in the forecast data; and furthermore, the spread of the production
loss forecast increases from approximately 5 % to 30 %
simultaneously. This increased spread is a useful indicator for the
uncertainty in the start of the ice period and provides additional
information to the actual forecast.
Same as Fig. , but for Site B.
![]()
Site B. (a) ENS mean MVD for the solid water components.
(b) Mean spread of MVD for the solid water components.
![]()
Forecast performance of the different approaches
In Fig. a the forecasted next-day production using the four
different uncertainty quantification approaches are compared with the single
forecast of CM and observed production for site A. For the approaches that
generate multiple forecasts, the average of the forecast members is
presented.
The different approaches are following the observed production most of the
time. As the figure suggests visually, and the RMSE calculation below
confirms, the ENSngb mean is the most skillful approach for this site. A
closer inspection of Fig. a reveals some typical behaviour
for the approaches. All approaches overestimate the production during the
start of the second icing period around 1 January, which agrees with the
underestimated production loss in Fig. a. The different model
forecasts tend to overestimate the production occasionally. This probably
happens due to an overestimation of the potential production, since the wind
speed has a positive bias (Fig. b). A bias correction of the wind
speed would be useful to reduce this error.
In Fig. a the single forecast of CM has stronger variations
than the other approaches. These variations are partly a sign of uncertainty
in the forecast and belong to unpredictable phenomena. They are filtered out
when averaging over the members of the probabilistic approaches. However, it
is important to realize that this filtering creates a so-called unoccupied
average, i.e. the smoothed state is unrealistic, since part of the variance
is now included in the spread of the forecast members describing the forecast
uncertainty. This spread for the uncertainty quantification approaches can be
seen in Fig. b. The ensemble (ENS) provides a larger spread than
the neighbourhood method applied to a single forecast (CMngb). Similar to the
meteorological parameters, the largest spread, and thus forecasted
uncertainty, is generally seen for the combination of ensemble and
neighbourhood method (ENSngb). The average spread to skill ratio of the
ENSngb production forecast amounts to around 0.7, compared to 0.5 for
ENS. The increased spread for the ENSngb should therefore provide the best
estimate for the actual forecast uncertainty, even if the model is still
overconfident.
(a) Production forecast in MW at site A for the different
approaches (Fig. ). For CMngb, ENSngb and ENS the lines present
the average of the multiple forecasts. Observed production is given with the
dash-dotted line. The production is calculated for a 2 MW turbine.
(b) The spread of the multiple members in CMngb, ENSngb and ENS.
![]()
As a summary of the forecast performance, Table yields the mean
error as RMSE of the 06:00 UTC +18-42 h forecasts for the production loss
and for the production, averaged over the three observation sites where
production data were available. The forecast quality of the production
forecast using the different approaches generally follows the order in the
first step of the modelling chain, i.e. for the meteorological parameters
(Fig. ). The benefit of the neighbourhood method for the
production loss forecast can be seen comparing the RMSE for CM and the CMngb.
However, the production loss RMSE is the same for ENS and ENSngb, suggesting
that the neighbourhood method neither contributes to nor reduces the forecast
skill when added to the ENS approach. Using the ENS or ENSngb approach
compared to CM results in a reduction of the production loss forecast error
from 26 % to 21 %. It should be noted that the usage of the ensemble
mean as the input to the icing and production loss model (approach EM)
deteriorates the forecast quality to 29 % compared to the ensemble-based
approach ENS with 21 %, where the output from each member is calculated
through the entire chain. This results from the non-linearity of the icing
and production loss model. The increased forecast quality for the forecasted
meteorological parameters by the ensemble mean is lost by the usage of the
unoccupied average as input into the icing and production model.
For the production forecast, the best forecast is provided by the ENSngb
approach, while the worst comes from the single forecast of CM. Here, adding
the neighbourhood method to the ENS approach reduces the RMSE
(Table ). Using the ENSngb approach compared to CM results in a
reduction of the production forecast error from 0.49 to 0.41 MW or by
16 % relative to the CM forecast.
The two different versions of neighbourhood methods, i.e. terrain-following or
constant-height version described in Sect. , displays some differences for
the production forecast. Assuming that wind power is installed at higher
elevation than the surroundings, the terrain-following version provides
neighbours from lower absolute heights with higher moisture content, and thus
more atmospheric icing, and more surface-affected wind fields compared to the
constant-height version. The quality of the resulting production loss
forecast is very similar for the two versions, resulting in the same RMSE.
For the final production forecast, however, an improvement is seen from the
constant-height version compared to the terrain-following one which can be
traced back to better wind forecasts from the constant-height neighbourhood
method.
It should be pointed out once more that the statistical significance of
the results could not be assessed since this is a case study with a limited sample size,
but the results are consistent in the different analyses and supports theoretical
expectations. The improved forecast skill of the production loss and of the production, using
the four probabilistic forecasting approaches instead of the single forecast of CM,
also suggests that the icing forecast is improved, even though it is not possible
to validate with the available ice load observations.
RMSE of the different approaches for production and for production
loss forecasts averaged over the three sites with production data. The
production is calculated for a 2 MW turbine.
Approach
Production (MW)
Production loss (%)
CM
0.49
26
CMngb
0.44
23
EM
0.47
29
ENS
0.44
21
ENSngb
0.41
21