Please see the attached report.

Review in connection with the manuscript

Evaluating mesoscale model predictions of diurnal speedup events in the Altamont Pass Wind Resource Area of California

By Arthur et al.

WES-24-137

General considerations

In this contribution, the authors present three-month long simulations of wind data in the vicinity of a wind park in central California. They use WRF with two different boundary layer parameterizations, the well-known (1-dimensional) MYNN scheme and a recently introduced 3D PBL parameterization. To estimate wind power -related parameters they employ a ‘wind farm parameterization’ (WFP). Atmospheric data for verification stem from 2 wind lidars at some distance from the wind farm and a number of meso-net station distributed in the domain.

The goal of the study is (1) to ‘evaluate the 3D PBL scheme in complex terrain’ and (2) ‘to test the WFP coupled to the 3D PBL scheme in a realistic configuration with terrain’ (l. 68ff).

The results to support (1) are presented as average daily time-height cross sections (Figs. 3&6) or average profiles for different times (Figs. 4&5) - and in concert with the chosen error metrics do not strongly support the goal of model evaluation. The resulting fractional biases (Tab. 2) for wind speed, for example suggest an almost perfect simulation (a fractional bias of a few permille (!), what simply suggests that biases are approximately normally distributed (in space and time).

The results to support goal (2) are again presented as some average statistics and figures – which might be more informative for the wind power community (and hence the audience of the present journal).

I have got a number of major comments, which I feel need to be addressed before the paper can be recommended for publication. In addition, a number of minor comments are given at the end.

Major comments

1. The ‘3D’ simulation is used in the BL approximation, BLA (the 1D MYNN simulation has the BLA as an intrinsic restriction). With this, it does not take into account what is considered by some authors (e.g.,Zhong and Chow, 2013, Muñoz-Esparza et al., 2015, Goger et al. 2018) to be the most relevant missing process in BLA schemes in complex terrain, i.e. TKE production due to horizontal shear. Indeed, the 3D PBL (BLA) scheme accounts for horizontal mixing (as the authors claim), but if the (horizontal fraction of) TKE is not adequately produced, the effect of this mixing must be minimal – or even detrimental. I suspect that the almost identical results for the MYNN and 3D PBL (BLA) schemes is to a substantial fraction due to this BLA choice. I think the paper would largely gain, if at least a ‘sample day’ (as some sort of case study) would be presented and discussed (could be in an appendix or supplemental material).
2. Wind data. I am not familiar with the lidar type used in this study (ZephIR300) – but I trust that the authors use the instruments according to its specifications – with an amazingly high accuracy for a very short averaging time (15 s), and high vertical resolution at the same time. It is mentioned (l. 116) that in an earlier study (Wharton et al. 2015) data was ‘corrected’ according to some ‘Dynamics software’ provided by the manufacturer. It is not stated, however, whether this correction was also applied in the present study. Is it? Also, the magnitude of the correction factors are used to estimate ‘uncertainty’ of the data. I am not sure whether this is a valid approach. Corrections are usually applied to measured data in order to correct for a known deficiency or violation of an assumption. If the correction is well based (and documented), the data is better (more reliable) after correction – irrespective of the (relative) magnitude of the correction. However, in a model evaluation study it must strictly be distinguished between model errors (what is investigated) and observational errors. If the data is not accurate enough, it cannot be used for model verification (or evaluation).
3. Observed TKE: if I understand correctly (l.297) the authors determine the velocity variances from only 8 ‘instantaneous’ velocity estimates (every 15 s) over a 2-min period. This of course corresponds to only a small fraction of the total power spectrum and likely means that actual magnitude of TKE is (largely?) underestimated. Possibly, one of the cited observational studies has tested these TKE estimates against true turbulence observations (e.g., from a sonic anemometer)? In any case, data from a sonic anemometer (not necessarily at the same site) could be used to ‘model’ the chosen approach (i.e., sampling a wind component every 15 s, and calculating the variance according to the chosen approach) and comparing it to the ‘full TKE’.
4. The overall statistics (Tab. 2) for TKE are not overly informative (see general considerations). But it is interesting to compare Fig. 6b and 6d. For a given time and height, the 3D PBL TKE scheme produces less TKE than the 1D MYNN scheme (difficult to judge, though, from the colour bar for hights>50 m and nighttime conditions). During the night, both parameterizations underpredict TKE, while during the day the MYNN scheme overpredicts and the 3D scheme still (dominantly) underpredicts. This is at odds with previous experience with 1D turbulence schemes in complex terrain – where usually underprediction is claimed due to neglecting horizontal shear production. As both schemes are employing the BLA (and the TKE observations are not particularly trustworthy, see major comment 3), it is more the relative performance of the two schemes that is interesting. Apparently, the additional (horizontal) mixing in the 3D scheme – and at the same time neglection of the relevant production terms in the TKE equation (BLA) - has an overall detrimental effect on the TKE levels. In this context it is interesting to note that in the original publication of the 3D PBL scheme (Kosović et al., 2020), TKE (i.e., the three velocity variances) were largely underestimated during the day in a complex terrain verification study (their Fig. 5). In the BLA, additional mixing in the 3D PBL parameterization may lead to an unwanted overcompensation. I think this should at least be discussed.
5. Comparison between the MYNN and the 3D-PBL(BLA) schemes. I think it is fair to state that there is no statistically significant difference between the two schemes – at least not when taking the statics as presented into account. If indeed the advantages of the 3D PBL scheme in complex terrain should be evaluated, the statistical information should definitively be extended – and it would probably be advisable to use the full (i.e., non-BLA) 3D PBL scheme.

Detailed comments

l.25     ‘…referred to more generally as numerical weather prediction (NWP) models’: I don’t think this can be said (global NWP models with a grid spacing of some 10 km – and more -  will not qualify as ‘meso-scale model’. I suggest to simply delete this part of the sentence….

l.26      ‘Historically, NWP models….’: again, this is a little short history. Historically (in the fifties of the last century – to give history a date), NWP models have started with several hundreds of km as grid-spacing. The present reviewer remembers  the introduction of first so-called ‘limited area models’ (downscaled from the global models – but only on a limited area) with a grid-spacing of some 20-30 km (and this was thought to be revolutionizing at the time…). In this sentence, only ‘or larger’ is approximately correct…..

Tab 1   ‘Mfr-P_R’ (in the title row) is not explained. Similarly, the two variables ‘H’ and ‘D’ have not been introduced (even they might be guessed from the context). Finally, ‘NREL-2.3’ / ‘Bonus’ etc. need to be explained.

l. 215 (eq. 1): I don’t think this eq. defines what usually is called a bias. This is just a difference between a modelled and an observed value of variable ‘VAR’ at some time and location. Upon averaging over time and/or location (height) this may eventually lead to a bias estimate, i.e. a systematic model deficiency.

l. 226 ‘..error metrics are presented in Tab.2’: First of all, the caption of Tab 2 must clearly specify that average bias is shown. Not in the sense of the previous comment (because bias is always associated with an average (systematic) behavior).Rather, it must become clear that this is a temporal and spatial average (this is at least what I must assume when I compare Fig. 3b to the first row of Tab.2). For the  spatial (i.e., vertical) average it is essential over which height range the spatial averaging is applied (and why). Having said that, the resulting numbers (close to zero through heavy averaging) are quite useless – and might [heavily] change if the height range over which averaging is applied – or the time - were changed. Also, the error metrics must be explained in the caption or at least a reference must be given where their definition can be found).

l.238    ‘…has only several model levels….’: this is rather unspecific (i.e., more than one but less than ‘many’?) – and thus not very helpful.

Fig. 4, caption: it is stated that ‘The shaded regions show ± 1 standard deviation, as well as potential ± 10% error in the observations following’. How is this information combined? The 10% added to the standard deviation? The larger of the two? Another approach? Can the authors be more specific?

l.273ff: Following the presentation of results, the authors emphasize the errors in the observations (which is, of course, a little ‘bad style’ in a model evaluation study: to attribute an important source for the differences to the errors of the observations). It is clear that the authors cannot be made responsible for the observational errors (or uncertainties) -  but when having uncertain data to compare with, the analysis procedure should take this into account (and there are various approaches in the literature how to do this). If the data quality is not good enough, then the data cannot be used for model evaluation.

l.297    calculation of velocity variances: if I understand correctly, there are only 8 values going into the estimation of the variance – and this in a frequency range that only covers a small range in the power spectrum. If the authors would use full-resolution turbulence data (from a sonic anemometer, say) it could be tested (in some sort of model propagator) how large the variance loss actually is under different conditions (will be much smaller under stable conditions than during the day).

l.363    ‘…reducing the negative bias by as much as 50%’. Looking at Fig. 8 or 9 it is probably fair to add  that after 12 PST it can also [more than] double it.

l. 458 ‘….larger horizontal gradients will be resolved…..’:

l. 610 please correct the reference….

References

Goger B, Rotach MW, Gohm A, Fuhrer O, Stiperski I, Holtslag AAM: 2018, The Impact of 3D Effects on the Simulation of Turbulence Kinetic Energy Structure in a Major Alpine Valley, Boundary-Layer Meteorol, 168 (1), 1-27.

Juliano TW, Kosović B, Jiménez P A, Eghdami M, Haupt SE, and Martilli A: 2022, “Gray zone” simulations using a three-dimensional planetary boundary layer parameterization in the Weather Research and Forecasting model, Mon. Wea. Rev., 150, 1585–1619

Kosović B, PA Jiménez, TW Juliano, A Martilli, M Eghdami, A P Barros, and S E Haupt, 2020: Three-dimensional planetary boundary layer parameterization for high-resolution mesoscale simulations. J. Phys.: Conf. Ser., 1452, 012080, https://doi.org/10.1088/1742-6596/1452/1/012080.

Muñoz-Esparza D, Sauer JA, Linn RR, Kosović B (2015) Limitations of one-dimensional mesoscale PBL parameterizations in reproducing mountain-wave flows. J Atmos Sci 73(7):2603–2614

Zhong S, Chow FK: 2013 Meso- and fine-scale modeling over complex terrain: parameterizations and applications. In: Chow FK, De Wekker SFJ, Snyder BJ (eds) Mountain weather research and forecasting, Springer atmospheric sciences. Springer, Berlin, pp 591–653