Storm Ciara was an extratropical cyclone that passed over the Belgian North Sea on 10 February 2020. Storm Ciara originated in the Atlantic Ocean, moving from the North American continent (starting 3 February 2020) to the European continent (16 February 2020). Storm Ciara swept across the majority of western Europe including the United Kingdom and Norway, bringing in heavy precipitation and strong winds with a maximum recorded wind gust of 219 km h-1 at Cap Corse, Corsica, France (https://www.meteo-paris.com/actualites/retro-meteo-2020-les-evenements-climatiques-marquants, last access: 21 April 2022). Over Belgium, the RMI-B (https://www.meteo.be/nl/info/nieuwsoverzicht/storm-ciara, last access: 21 April 2022) reported wind gusts of up to 115 km h-1 in Ostend, located at the Belgian coast, with heavy precipitation accompanied by strong winds and thunderstorms.

During the early hours of Storm Ciara on 10 February 2020, an offshore wind farm recorded fast changes in wind direction accompanied by severe yaw misalignment and concentrated rainfall. An RMI-B radar snapshot at 04:40 UTC is presented in Fig. a, illustrating the presence of a bow echo moving from the British Isles to Belgium, an indication of a possible micro-burst phenomenon . Synoptic maps by the Royal Netherlands Meteorological Institute (RNMI; https://www.knmi.nl/nederland-nu/klimatologie/daggegevens/weerkaarten, last access: 21 April 2022) presented in Fig. a indicate a trough passage during this period. Further, precipitation data from a wind profiler located within the wind farm highlights fast changes in wind direction accompanied by sudden precipitation during the period of interest at 04:40 UTC, presented in Fig. .

On 24 December 2020, the Belgian offshore wind farms observed heavy precipitation accompanied by fast changes in wind direction. Synoptic maps presented in Fig. b indicate the presence of a low-pressure system over the English Channel and Normandy. Radar observations from the RMI-B indicate large precipitation cells over the Belgian North Sea, presented in Fig. b. SCADA data record fast changes in wind direction of ∼100∘ at 02:00 UTC accompanied by severe yaw misalignment and precipitation.

On 27 June 2020, the Belgian offshore wind farms experienced fast changes in wind direction and sudden precipitation around 15:30 UTC. The synoptic maps provided by the RNMI indicate the presence of a low-pressure system over the British Isles, with a trough passage across the Belgian North Sea, presented in Fig. b. Radar observations provided by the RMI-B indicate the presence of precipitation cells over the offshore wind farms, presented in Fig. b. The SCADA data record fast changes in wind direction of ∼60∘ during this hour accompanied by severe yaw misalignment and precipitation.

The common model parameters considered for all WRF simulations are summarized in Table . The baseline horizontal grid spacing of the parent domain d01 is 27 km, while the one-way nested domains are sequentially refined by a factor of 3, resulting in horizontal grid spacings of 9, 3, and 1 km for d02, d03, and d04, respectively. In the vertical direction, 57 terrain-following model levels are considered with a model top pressure at 1000 Pa. The vertical velocity damping option based on the Courant–Friedrichs–Lewy condition as implemented in WRF is also turned on. A time step of 20 s is considered for parent domain d01, while the time step of nested domains is sequentially refined by a factor of 3. The initial conditions and LBCs are derived from ERA5 reanalysis (downloaded from 10.24381/cds.adbb2d47, and 10.24381/cds.bd0915c6, ). WRF simulations were initialized with a spin-up period of 24 h for all case studies. In addition, an evaluation period of 21 h from 00:00 to 21:00 UTC on 10 February 2020 is considered for Storm Ciara. For the low-pressure system and trough passage, evaluation periods of 6 h from 00:00 to 06:00 UTC on 24 December 2020 and 6 h from 12:00 to 18:00 UTC on 27 June 2020 are considered, respectively. The simulations have been performed as a continuous run including spin-up and evaluation periods. Therein, the selected evaluation periods adequately capture the time periods of interest for respective case studies as described in Sect. . The one-way nested domain configuration common to all simulations in this study is presented in Fig. .

The Rapid Radiative Transfer Model (RRTMG) for longwave and shortwave radiation physics is used by all simulations. Similarly, the land–surface interactions are defined by the unified Noah land surface model . The PBL, cumulus, and microphysics schemes are varied amongst the mentioned options as described in Table .

To categorize and distinguish the key features of different WRF physics parameterizations and options available, a combination of different simulation pairs as described in Table  is considered. A total of 12 WRF simulations are categorized into different simulation pairs (A–J) assigned to variations of the update interval of the LBCs and the PBL, cumulus, and microphysics schemes. For each of the varied parameters, at least two different simulation pairs are considered. For example, simulation pairs A and B are assigned to the variation in update interval of the LBCs. More specifically, the simulation pairs considered are as follows. The sensitivity to hourly and 3-hourly update intervals of LBCs is assessed in simulation pairs A and B. Further, the sensitivity to scale-aware (SH and Zhang) and non-scale-aware (MYNN) PBL schemes is evaluated in pairs C, D, and E. The sensitivity to scale-aware (msKF and GF) and non-scale-aware (KF and GD-3D) cumulus parameterizations is evaluated in pairs F, G, and H. Given the convection-permitting resolutions of 3 and 1 km for d03 and d04, respectively, the non-scale-aware KF model is explicitly turned off in these domains in simulation runs 2, 3, 5, and 6

It was verified that this approach resulted in better reproduction of precipitation cells and lower error metrics.

The simulated wind direction and horizontal wind speed are evaluated against 10 min averaged SCADA data from the southwestern front row of an offshore wind farm located in the Belgian North Sea. Model accuracy is assessed using a standard mean absolute error (MAE) for wind direction and wind speed. To recover a single performance metric, a so-called normalized Euclidean distance (NED) is defined by NED=MAEWDn2+MAEWSn2, with MAEWDn the normalized MAE of wind direction and MAEWSn the normalized MAE of horizontal wind speed. Normalization is performed with the mean over all simulations. Precipitation fields are qualitatively compared between WRF simulations. The simulated radar reflectivity is converted to precipitation rate using the Marshall and Palmer relation .

This study also evaluates the performance of an ensemble average compared to single deterministic simulation runs. The ensemble average is defined as the mean of all simulation runs considered for a given case study. In this study, ensemble members are initialized with identical initial conditions from ERA5 reanalysis. Subsequently, variability in the ensemble average is only caused by the variation in update interval of LBCs and physics parameterizations. Therein, the current definition of ensemble average differs from traditional ensemble forecasts, where variations in initial conditions are also considered (e.g., ).

The MAE of wind direction, horizontal wind speed, and the NED for the Storm Ciara runs is presented in Table . Overall, for simulation runs 2 through 12, relatively lower MAE values for wind direction and wind speed are observed, with a maximum of 9.26∘ and 2.72 m s-1, respectively. Considering NED, the best-case setup is determined to be simulation run 7, with a NED value very similar to the ensemble average (<1 % difference). Run 7 uses the scale-aware SH PBL scheme coupled with the scale-aware msKF cumulus scheme, single-moment five-class WSM5 microphysics scheme, and hourly ERA5 lateral boundary condition updates. In a general sense, simulation runs 7 through 10 observe the lowest overall NED. These simulation runs consider scale-aware PBL and cumulus parameterizations coupled with hourly update intervals of LBCs. A qualitative analysis of wind direction and wind speed time series for all simulation runs highlighting the ensemble average and the best-case setup is presented in Fig. . Compared to the SCADA reference data, the changes in wind direction are captured reasonably well by all runs, with the ensemble average capturing the general transience of wind direction better than the best-case setup. However, accurately capturing the variability on wind speed is found to be more challenging, as shown by the large spread among different modeling setups in the afternoon and evening hours.

The summary of the performance evaluation metrics for the low-pressure system is presented in Table . The best-case setup is found to be simulation run 2, comprising scale-aware SH PBL coupled with non-scale-aware KF cumulus, WSM5 microphysics, and hourly ERA5 reanalysis data as LBCs. Unlike the case of Storm Ciara, no trend in better results for simulation runs combining scale-aware PBL, scale-aware cumulus, and higher-order microphysics is found (i.e., simulation runs 6 through 10 in Table ). The overall trend in wind speed MAE results shows that simulations using the MYNN PBL scheme, i.e., runs 1, 3, and 4, perform poorly. Similar to the case of Storm Ciara, the best-case results are found to be very similar to the ensemble average, with a relative difference in NED of 3.2 %. However, it must be noted that the ensemble average tends to dampen the fast changes in wind direction, as plotted in Fig.  along with the best-case setup. A qualitative analysis of the time series indicates that all simulation runs capture the fast change in wind direction for the period of interest. However, these often exhibit a time lag compared to SCADA data.

The performance evaluation metrics for the trough passage are presented in Table . Overall, the considered WRF setups are found to be highly sensitive to the combinations and type of physics parameterizations. Overall, MAE values are found to be higher than the other cases, indicating that accurately predicting wind direction and wind speed is more challenging for this particular event. No clear trend in any combinations of the sequence of simulation runs is found, in contrast to Storm Ciara and the low-pressure system. Interestingly, the best-case setup for the low-pressure-system case performed the worst for the trough passage case. Considering NED as the evaluation metric, the best-case setup is observed to be run 12 by a significant margin. Run 12 uses the scale-aware Zhang 3D PBL scheme and the scale-aware GF cumulus scheme coupled with Morrison microphysics and hourly ERA5 LBC updates. Simulation time series including the ensemble average and best-case setup are presented in Fig. . Qualitatively, simulation runs 1 through 11 underpredict the fast changes in wind direction for the evaluation period, whereas wind speeds are underpredicted by all runs. Due to the joint poor performance and persistent offsets compared to SCADA data for all simulations except run 12, the ensemble average does not yield any better match to the data.

The effect of varying the update interval of ERA5 LBCs between hourly and 3-hourly is investigated in this section. Simulation pairs A and B represent four WRF setups for each case study. Figure  shows the results for simulation pair A. Error bars indicate 1 standard error of the sample mean. Starting with wind direction (Fig. a), hourly update intervals of ERA5 LBCs are observed to perform better for Storm Ciara and the trough passage. For the low-pressure system, lower MAE is observed for hourly reanalysis data; however, these values lie well within the standard error bars, therefore leading to inconclusive overall results. For wind speed (Fig. b), a significant improvement in MAE is observed when hourly reanalysis data are used for Storm Ciara, with a 34 % reduction compared to 3-hourly data. In contrast, for the low-pressure system and the trough passage, 3-hourly reanalysis data produce lower MAE; however these values lie within the standard error. Overall, for simulation pair A, a distinction in better performance for hourly reanalysis is observed for Storm Ciara; however no significant benefit is observed for the other two cases.

Similarly, Fig.  shows the results for simulation pair B. The MAE comparison for wind direction is presented in Fig. a, indicating statistically inconclusive results for all cases. Similarly, MAE results for wind speeds are presented in Fig. b, indicating inconclusive results for Storm Ciara and the low-pressure system, yet a better performance with hourly reanalysis data in the case of the trough passage.

To summarize the overall inferences from both simulation pairs, hourly updates of LBCs do not systematically lead to higher accuracy, although improvements are observed for certain combinations of events and wind variables. Therefore, more frequent updates of LBCs may prove advantageous when trying to capture certain fast transient weather events.

In this section, the influence of using classical non-scale-aware PBL schemes versus scale-aware PBL schemes is elaborated in simulation pairs C, D, and E. More specifically, the standard MYNN scheme is compared to the scale-aware 1D SH and scale-aware 3D Zhang PBL schemes. Simulation pairs C and D compare the influence of MYNN and SH PBL schemes, and simulation pair E compares SH and Zhang PBL schemes.

Figure  presents the consolidated results for simulation pair C. First, considering wind direction MAE (Fig. a), the SH PBL scheme performs better for the case of Storm Ciara. In contrast, for the low-pressure system and the trough passage, the MYNN PBL scheme shows better performance. Considering wind speed (Fig. b), no conclusive set of inferences is drawn for the case of Storm Ciara, as the lower MAE by SH lies within the range of the standard error. For the low-pressure system, the SH PBL scheme outperforms the MYNN scheme in terms of wind speed, reversing the trend found for wind direction. For the trough passage, MYNN wind speeds outperform SH. Overall, for the simulation pair C, no clear conclusions can be drawn for Storm Ciara and the low-pressure-system cases. In contrast, better performance by the MYNN PBL scheme is observed for the trough passage.

Figure  illustrates MAEs for simulation pair D. Starting with wind direction (Fig. a), a clear advantage in utilizing SH PBL is observed for the case of Storm Ciara. However, this distinction is not observed for the low-pressure system. The MYNN PBL scheme performs better than the SH PBL scheme for the trough passage. This trend is found to be similar to simulation pair C. Considering wind speeds (Fig. b), SH outperforms MYNN by a significant margin for the case of Storm Ciara. A similar distinction is observed for the low-pressure system. However, similar to pair C, for the trough passage this trend is reversed.

Overall, for the simulation pairs C and D, a distinctly better performance by the SH PBL scheme is observed for the case of Storm Ciara. However, few to no conclusions can be drawn for the low-pressure system. For the trough passage, better performance is observed for the MYNN PBL scheme.

To further distinguish between scale-aware PBL schemes of different complexity, Fig.  presents results for simulation pair E, comparing the SH and the Zhang PBL schemes. The wind direction MAE results (Fig. a) show an advantage in using SH for the case of Storm Ciara. However, this distinction is not statistically significant for the low-pressure system. The Zhang PBL scheme outperforms the SH PBL scheme by a significant margin for the trough passage. Considering wind speed (Fig. b), the Zhang scheme outperforms the SH scheme for Storm Ciara and the trough passage. However, this trend is not statistically significant for the low-pressure-system case. The trough passage observes better performance by the Zhang PBL scheme. Overall, for simulation pair E, no clear distinction in better performance for both wind direction and wind speed combined is observed for SH or Zhang PBL schemes for Storm Ciara and the low-pressure-system cases. However, for the case of the trough passage, a clear advantage in using the Zhang PBL scheme is observed.

A qualitative comparison of time series for the low-pressure-system and trough passage cases is presented in Fig. b, indicating better performance by the Zhang PBL scheme to capture the transience in wind direction. However, this qualitative advantage is not observed in the case of Storm Ciara (not shown).

The current results indicate a high sensitivity in wind direction and wind speed relative to the choice of PBL scheme. A single PBL scheme is not found to outperform the others for all three case studies. The simulation results indicate a possible dependency of PBL schemes to cumulus and microphysics parameterizations, as has also been reported in the literature by , , and .

This section presents and discusses the results for cumulus simulation pairs F, G, and H, as well as microphysics simulation pairs I and J. As these two types of physics schemes both relate to the modeling of precipitation in WRF, they are considered jointly in this section. A set of eight WRF simulations is considered, covering a combination of four cumulus schemes (KF, GD-3D, msKF, and GF, the latter two being scale-aware) and three microphysics schemes (WSM5, Thompson, and Morrison, in order of increasing modeling complexity).

Figure  depicts simulation results for pair F, which uses the SH PBL scheme coupled with the WSM5 microphysics parameterization while varying the cumulus scheme between KF and msKF. Figure  presents the results for MAE of wind direction and wind speed for all three test cases. Starting with wind direction (Fig. a), the msKF simulation produces better results for the case of Storm Ciara. However, for the low-pressure-system case, KF produces lower MAE, which lies within the standard error bars, disallowing statistically significant conclusions. Similarly, for the trough passage, the msKF scheme results in a negligible reduction in MAE. Therefore, conclusions can only be drawn in the case of Storm Ciara, indicating better performance with the msKF scheme on wind direction. Focusing on wind speed (Fig. b), again better results for the msKF cumulus scheme are observed for the case of Storm Ciara. For the low-pressure-system case both schemes produce similar MAE values. For the trough passage, KF performs better than msKF, reversing the trend found for wind direction. Summarizing, for simulation pair F, the case of Storm Ciara is more accurately predicted by the msKF cumulus scheme in comparison to the KF scheme. For the low-pressure-system and trough passage cases, comparative results are inconclusive.

The results for simulation pair G are presented in Fig. . Pair G applies the SH PBL scheme coupled with the Thompson microphysics parameterization while varying the cumulus scheme between KF and msKF. Overall, the current combination of physics schemes produces similar MAE on wind direction and wind speed compared to simulation pair F. For Storm Ciara, msKF produces lower MAE in wind direction and wind speed. For the low-pressure system, no clear trend in performance is observed. For the trough passage, lower wind direction MAE is observed for msKF, and lower wind speed MAE is observed for KF. Consolidating results for simulation pairs F and G, for the case of Storm Ciara, a clear improvement is observed when using the scale-aware msKF cumulus scheme; however no conclusive statements can be made for the low-pressure-system and trough passage cases.

Figure  shows results for simulation pair H, which applies the SH PBL scheme coupled with the Morrison microphysics while varying the cumulus scheme between msKF, GD-3D, and GF. Starting with wind direction (Fig. a), Storm Ciara is much better captured by the GD-3D cumulus scheme. For the low-pressure system, no such trend in better performance is observed. For the trough passage, msKF outperforms both GD-3D and GF schemes. Considering wind speed (Fig. b), msKF performs better than GD-3D and GF for Storm Ciara. For the low-pressure system, GD-3D is found to be the best performer. For the trough passage, msKF and GF both perform better than GD-3D. However, no distinction is found between msKF and GF schemes. Summarizing, the overall inferences for the simulation pair H are found to be statistically inconclusive and highly sensitive; thus one cannot conclude that a specific cumulus parameterization systematically outperforms the others.

Simulation pairs I and J focus on the impact of microphysics schemes. Starting with pair I, results are presented in Fig. . Pair I uses scale-aware SH PBL coupled with non-scale-aware KF cumulus schemes and varies the microphysics scheme between WSM5 and Thompson. Overall, no distinction in better performance by either microphysics scheme considering MAE of wind direction and wind speed is observed (Fig. a and b). For Storm Ciara and the trough passage EWEs, wind speed and wind direction MAE are found to be insensitive to the variation in microphysics schemes. However, this trend is not clear for the case of the low-pressure system, where the Thompson scheme is found to produce different MAE of wind direction, with significantly larger error margins (Fig. a and b). However, when comparing the combination of cumulus schemes to microphysics setups, more specifically, the combination of WSM5 + KF to Thompson + KF/msKF (Figs. a and a), lower MAEs of wind direction are observed for Thompson microphysics when combined with the scale-aware msKF cumulus scheme. However, this trend is reversed for WSM5 + msKF cumulus schemes. This appears to highlight case-to-case variability in skill and the importance of the combination of cumulus and microphysics schemes.

Results for simulation pair J, which applies SH PBL with msKF cumulus schemes and varies WSM5, Thompson, and Morrison microphysics schemes, are presented in Fig. . Considering MAE of wind direction (Fig. a), for the case of Storm Ciara and the low-pressure system, a clear distinction in the performance of different microphysics schemes is not observed. For the case of trough passage, Morrison microphysics perform better by comparison. For wind speed MAE (Fig. b), Storm Ciara shows similar results as for wind direction. For the low-pressure system, WSM5 and Thompson produce better wind speed than Morrison. For the trough passage, Morrison outperforms WSM5 and Thompson microphysics. Overall, for simulation pair J, Storm Ciara shows an insensitivity to the variation in microphysics schemes. For the low-pressure system, no clear trend in better performance is observed, whereas a clear advantage in using the more complex Morrison scheme is observed in the trough passage case.

The previous sections investigated the individual influence of varying a single physics parameterization in the modeling chain on the accuracy of the match between WRF simulation results and observational data when subject to three EWE case studies. A clear trend in improved performance with higher model complexity common to all case studies is not found.

When looking at the update interval of LBCs, the qualitative differences in hourly and 3-hourly update intervals are found to be marginal. The quantitative indicators show a unanimous improvement for the case of Storm Ciara (see Figs.  and ). However, for the low-pressure system and the trough passage this distinction is not evident.

The variation in PBL scheme results in highly sensitive metrics for all three case studies. For the case of Storm Ciara, a clear advantage in using scale-aware SH PBL compared to non-scale-aware MYNN PBL is observed (see Figs.  and ). However, this trend is not evident for the case of the low-pressure system and the trough passage. Comparing scale-aware schemes of different fidelity, more specifically Zhang PBL and SH PBL, a promising trend is observed in which the higher-complexity Zhang PBL leads to a better match with SCADA data for the trough passage case (Fig. ). However, this trend is not found for Storm Ciara and the low-pressure system. Furthermore, considering wind speeds, the Zhang PBL run is either the best setup (cold front) or results in MAE very close to the best setup (Storm Ciara and low-pressure system). In contrast, the Zhang PBL setup results in higher wind direction errors for Storm Ciara and the low-pressure-system cases.

Regarding the cumulus and microphysics simulation pairs, the combination of cumulus and microphysics is observed to have more impact on MAE of wind direction in comparison to variation in stand-alone cumulus or microphysics schemes. This is highlighted by Storm Ciara and the low-pressure system, where the change in microphysics schemes in combination with msKF cumulus results in marginal changes in MAE (see Fig. ). However, when comparing the combinations of lower-order microphysics with scale-aware and non-scale-aware cumulus schemes for wind direction, i.e., WSM5/Thompson + KF/msKF, results indicate an overall reduction in MAE for Thompson + msKF (see Figs. a and a). These results potentially indicate scale-aware cumulus schemes to be more compatible with higher-order microphysics schemes. The performance of cumulus and microphysics schemes is found to be strongly dependent on the type of weather phenomenon.

Precipitation results were qualitatively compared to each other and to the radar images from RMI-B (Fig. ). It was found that all precipitation results are highly sensitive to the combination of physics schemes and type of EWE, yet no conclusions on modeling fidelity could be drawn from this analysis, and hence further discussion is omitted. As an example, a sample of results for the case of Storm Ciara is presented in Fig. , illustrating the completely different precipitation fields produced by different model setups. A direct quantitative comparison of simulated reflectivity and observed raw radar fields using, for example, tools for comparing gridded observations such as MODE is impeded by the lack of filtering and post-processing information on the raw radar observation data. Therefore, a quantitative assessment of precipitation modeling is out of the scope of the current paper and left for future work.

Finally, the ensemble average (as defined in Sect. ) is observed to rank very similar to the best-case model setup (see Tables  and ) for the cases of Storm Ciara and the low-pressure system. However, the fast changes in wind direction are dampened by the ensemble averaging (see Figs.  and ). For the trough passage, the ensemble averaging performs poorly compared to the best-case setup by a significant margin (Table ), caused by a persistent offset by all but the best-case setup.