General comments

The paper addresses two topics, namely participation in the reserve market, and layout optimisation for the same. The introduction mentions a third contribution, but since that refers to a case study of the previous two topics, I consider that to be inherent to those two contributions.

As the title of the paper implies, the layout optimisation is the main topic, and the contribution to participation in the reserve market is subservient to that. However, in the introduction and the conclusions that part of the study is given an almost equal status. The authors could consider repositioning that part of the study as a means for the layout-optimisation study, rather than as a research subject in itself. This would change the evaluation of the participation in the reserve market (in section 4.1) into a validation of the suitability of the formulation for the purpose of layout optimisation. The exploratory nature of these results could then be a spin-off. However, as a dedicated study into the potential of participation in the reserve market, the formulations may be insufficiently accurate and the validation may be too limited.

Having said that, the modelling of participation in the reserve market is convincing for the purpose of layout optimisation. The layout optimisation itself is also convincing, although I have some doubt about the use of expected values for the yearly profit and energy supplied (which may be either an issue of clarification or of method).

The authors have chosen useful experiments for comparisons. The comparisons of the experiments could be arranged in a more logical structure, and the interpretation of results seems to be somewhat biased by a presumption of the authors about the benevolence of considering participation in the reserve market in layout optimisation. This is also reflected in one of the main conclusions (“layouts optimized for profit maximization with reserve markets lead to better yearly profits than when considering day-ahead market only in the objective function”), which in my opinion is not supported by the results.

In my opinion the four issues that require some more consideration are the role of the study on participation in the reserve market, the use of expected values for profit and supply, the order of comparison of the experiments and the interpretation of results that leads to the main conclusions. Besides these four issues, I think the paper should spend a bit more attention on the size of the reserve capacity market, and the consequential relevance of this study. Having said that, most of my detailed comments are merely suggestions and corrections for readability of the paper. Apart from the potential methodical flaw in the use of expected value for profit and energy supplied, I agree with the entire setup and execution of the body of the research. Although I think that the considerations that I give below can have a major impact on the presentation and conclusions of the research, I think that the actual changes that need to be made are not as major, thanks to the high quality of the modelling, optimisation and experiments.

Specific comments

Significance of the reserve capacity market

The reserve capacity market is small compared to the day-ahead market. The maximum required capacity of 117 MW at any instant during 2023 is indicative of this. The authors justly argue that this market will grow. However, there is insufficient understanding of how much wind farms will be able to contribute to this market in the future, considering the correlation between their causing the problem (in the day-ahead market) and their ability to provide the solution (in the reserve market). Either way, the conclusion “Future wind farms should therefore be designed for that purpose” (line 460) does certainly not apply to all wind farms, due to the limited size of the market. Although the quoted statement is given in an introductory part of the conclusion, the authors should be careful with such a statement in the conclusions chapter, since this is not sufficiently supported by the outcomes of this research. I think the issue of the size of the current and anticipated future market needs some attention in the introduction and careful formulation of the conclusions.

Role of study on participation in reserve market

To perform layout optimisation that considers participation in the reserve market, the performance of this participation under a reasonable bidding strategy has been modelled. Several simplifications are made for this. Forecast errors of all forecasted parameters are modelled with independent, gaussian distributions. The paper already mentions several aspects that might be violated by this assumption. E.g., the authors observe that “Reserve and regulation prices are characterized by higher volatility, lower mean, more frequent price spikes and a more skewed distribution compared to electric energy prices. Thus modelling their behavior is potentially more challenging” (line 184-185). However, what concerns me most is the neglect of correlations. For instance, “a small prediction error in wind speed can lead to a tremendous need of reserve” (line 291). This indicates a potentially significant correlation between wind speed forecast errors around cut-out wind speed with reserve activation. It is also noted that the optimiser could base bidding on the ratio between day-ahead prices and imbalance fees (line 315-317). However, imbalance fees may be correlated with forecast errors, especially if multiple bidders of wind energy have a systematic error in said forecast, since they use the same or similar weather forecasts.

Another simplification is the replacement of the penalty regulations by a penalty price. The current formulation allows the optimiser to exploit the penalty system, if it just comes at a (monetary) price. The result for the reserve penalty in table 2 shows that this is substantial, compared to the reserve profits. This might indicate that in reality the TSO might already have imposed restrictions on participation to this BSP. (Whether the conditions of the actual regulation are met could be checked from the results a posteriori.) The reserve penalty is implied to be much more substantial for the cases with higher bid limits, thus expectedly having a higher number of bid periods with failure to deliver. One can expect that this will only increase in future, weather-dominated systems. As argued above for the potential correlation between imbalance fees and power forecast errors, it can be argued that reserve activation becomes more and more correlated with times of over-forecasting wind power. Since these will also be times where the ability to meet reserve capacity activation is at risk, even if it is prioritised, the risk of failing to meet the capacity test may be underestimated in this study. Furthermore, consistent failure to deliver by wind farms operating in the reserve market may lead to changes in the penalty regulations.

A third simplification is that reserve capacity and reserve activation are aggregated over the bid period. Although there is some inconsistency in the use of power and energy in the paper (see the later technical corrections), the formulations that are used are effectively energy based, aggregating power variations during the bid period to a single value. Although the secondary reserve market allows delayed response, variations of available power of the wind farm and timing of reserve activation within the bid period do matter. An episode of lower power availability could be compensated for the day-ahead market by an episode of higher power in the same bid period. However, it could negatively affect the ability to deliver reserve capacity during this episode of low power in a timely manner. This increases the potential reduced ability to deliver reserve capacity that was discussed above.

The report also refers to the relevance of the study to future, weather-dominated systems, where reserve markets will play a bigger role. I concur with the latter, but the modelling of the reserve market might require significant modifications for such a future system. As argued above, the role and performance of wind farms in the reserve market might change significantly when wind farms become a more important factor in the problems that need to be solved by this market. In the very least, the prices will change (dramatically) for such future systems, possibly accounting for the drawback of decreased reliability of the reserve bids.

Because of the doubts that this raises on the accuracy and (untested) validity of the simplified model, I suggest that this aspect of the study is not presented as an inherent contribution (in the introduction) with separate conclusions. Presenting it as a means for the research on layout optimisation, with the associated lower burden of accuracy, seems more appropriate and matches better with the expectations set by the title of the paper. Otherwise, more validation would be required, especially to support the conclusion “results show that yearly profits are expected to increase in a significant way when accounting for participation to reserve markets, while exhibiting a lower supplied energy. This profit augmentation is amplified when the maximum value for reserve bids is increased” (line 4655-467).

Use of expected values for yearly profit and energy supplied

On p.4 the use of forecast uncertainty and Monte-Carlo sampling of a set of S forecast errors is explained. Eq. (11) on p.8 articulates how these forecasts are used to optimise bidding. It is clear that this is a probabilistic formulation to deal with the forecast uncertainty. However, it is unclear why and how uncertainty comes into the picture in the yearly profit and energy supplied. The data of realised wind speeds, wind directions, prices, fees, etc. is available, so one would expect the profit and energy supplied to be deterministic, once the bidding is known. In other words, in e.g. Eq. (10) on p.7, the profit can be calculated directly from the realised conditions and operation, rather than determining an expected value for a set of forecast conditions. It is not clear why and how the authors use this stochastic formulation of profit (and energy supplied), leading to values for mean and standard deviation in the tables. Associated places that added to my confusion about this are:

p.5, line 134: Perfect forecasts are used to get revenue from the day-ahead market with Eq. (4). For the optimisation of bidding the equivalent of this equation is used with imperfect forecasting, while for the actual revenue one would expect to use the realised situation, rather than a ‘perfect forecast’.

P.7, Eq. (6): The supplied activated reserve uses the wind power forecast, where one would expect the use of the realised wind power, based on the realised wind conditions.

p.9, Eq(14): Here the objective function for layout optimisation is expressed as an expected value, using the samples of forecast errors that where generated for the optimisation of the bidding strategy. Nevertheless, on line 222 it is stated that the objective is to maximise profit, and not the expected value of profit.

p.9, line 239: This mentions that results are computed as expected yearly profits and supplied energy, instead of deterministic yearly profits and supplied energy.

p.14, caption of table 1 (and later tabulated results): This table shows the expected values and explains in the caption how the mean and certainty interval are associated with the sample set S of forecast errors. Again, this association is understandable for the profits and available power used in the optimisation of bidding (related to p.8, Eq. (11)), but is not what one expects to use for the realised profits and energy supplied.

The use of these expected values should be better argued and explained, or (more likely) these parameters should be treated as deterministic parameters.

Order of comparisons of experiments

In section 4.2 the performance of a layout optimised for JERM is compared to that of the base layout. The risk of this approach is that improved performance of the JERM-optimised layout is assigned disproportionately to JERM-optimisation, whereas a majority of the improvement could have been reached with other optimisations as well. Indeed, the discussion of the results immediately zooms in on the 3.10% improvement on JERM, while the significance of the 3.58% increase for the simplest participation on DAEM only is ignored. That increase indicates that much of the improved performance could be assigned to AEP improvement of the farm, rather than to JERM optimisation specifically. The AEP improvement is mentioned and analysed, but without consideration of the meaning of that for the association of JERM-optimisation with the total improvement of 3.10%.

The authors later compare JERM optimisation with AEP and DAEM optimisation. Indeed, the expected logical order of performance on JERM is recognised to be: base layout, AEP-optimised, DAEM-optimised, JERM-optimised. However, by first making the jump from base to JERM-optimised and then do a backward analysis to the intermediate optimisation options, the perception and interpretation of the results is biased by the first indications. In my opinion, the contribution to JERM (and DAEM) performance of different types of optimisation would have been much clearer when the experiments were done and shown in the order of expected optimality, comparing the improvements one step at the time: base layout to AEP-optimised, AEP-optimised to DAEM-optimised, DAEM-optimised to JERM-optimised.

As an extension of the chosen approach, the performance to the unseen 2024 data in section 4.5 focused on a comparison of the base layout with the JERM-optimised layout. Also here the robustness of JERM optimisation is obscured by the large improvement in AEP (visible in the large improvement of performance on simple DAEM-only operation). I think the specific merits of JERM optimisation would become clearer in a comparison of DAEM- and JERM-optimised layouts operating in 2023 and 2024 markets. Comparison of AEP- and DAEM-optimised layouts for the same could serve as a baseline, to assess which differences can be attributed to robustness to market conditions (DAEM- versus JERM-optimised) and which to robustness to wind conditions (AEP- versus DAEM-optimised).

Conclusions and interpretation of results

I will first reflect on some interpretations of the results and then give my own interpretation. Subsequently, I’ll address how this might affect the conclusions.

I already addressed the bias caused by the interpretation of results in section 4.2. Therefore, I continue with the analysis of figure 8 on p.19. I will continue to use ‘DAEM-optimised’ for what is called ‘Optimized without reserve’ in the figure, and ‘JERM-optimised’ for what is called ‘Optimized with reserve’. Each point in the graph is an optimised layout. The scatter indicates the stochastic nature of the optimisation. The width of the scatter, when compared to the difference between the DAEM-optimised and JERM-optimised points indicates that no direct comparison can be made between any best performing layouts. That would put more emphasis on ‘luck’ of drawing a good sample from the layouts, rather than on the difference between the two optimisation types. Somewhat in line with the discussion of the authors of figure 9 on p.20, the fairest comparison in performance on JERM seems to be to draw a diagonal fit through all DAEM-optimised layouts and a fit through all JERM-optimised layouts, and to compare those. These two fits would be almost the same. If JERM-optimised layouts would consistently perform better on JERM than DAEM-optimised layouts, one would expect the fit to JERM-optimised layouts to lie higher than that of DAEM-optimised layouts: Any layout that achieves a certain performance on DAEM, irrespective of how they were optimised, should achieve a better performance on JERM if it was JERM-optimised. A similar argument is applied by the authors in the discussion of figure 9 on p.20, based on the observation that the scatter of circles lies upward of the scatter of triangles. On p.20, the authors associate the downward shift of the AEP-optimised layouts compared to the JERM-optimised layouts with the performance characteristics of AEP optimisation. It seems inconsistent to then not associate the lack of a downward/upward shift with equal performance for DAEM and JERM optimisation. In my opinion the only conclusion that is supported by the results in figure 8 is therefore that JERM-optimised layouts do not perform better than DAEM-optimised layouts, neither on DAEM, nor on JERM.

Although the authors discuss the general shift between the points in figure 9, the magnitude and significance of this shift is not addressed. Two diagonal fits between the two sets of points would have a vertical shift of about 0.1 MEuro, which corresponds with less than 0.15%. Even between the highest JERM-optimised and highest AEP-optimised points the difference is only about 0.2 MEuro: less than 0.3%. Figure 12 indicates an upward shift of about 0.08 MEuro for unseen data of 2024, corresponding to about 0.3%. These percentages are most indicative of the performance of DAEM optimisation over AEP optimisation.

In the discussion of figure 8 the authors zoom in on the performance of the two two right-most points for JERM-optimised farms. Liekwise, on p.20 they focus on the right-most point in figure 9, addressing that it performs better on AEP than the (best) AEP optimisation. As argued above about the scatter of the results, these results seem inconclusive as to the inherent superiority of the two right-most points in figure 8 and the one right-most point in figure 9, as opposed to their ‘lucky sampling’. The authors argue that JERM optimisation might be slightly more likely to find solutions with high optimality (for AEP, DAEM, as well as JERM performance), due to better gradients (line 396-398 and line 412-413). However, this doesn’t confirm the significance of using JERM as an objective over using AEP or DAEM, but rather the significance of how either objective is formalised and how the problem is solved. In other words, it doesn’t mean that these layouts are better optimised for JERM per se.

Considering the above, I propose a re-interpretation of the results along the lines of the proposed reordering of the experiments. Since I don’t have all (intermediate) results, my interpretation will be rough and based on ball-park figures:

- From base layout to AEP-optimised: Improvement of performance on DAEM and JERM of about 3%. This is based on the results in Table 4, and the subsequent minimal contributions that I identify for the other optimisation improvements.

- From AEP-optimised to DAEM-optimised: Improvement on DAEM and JERM of about 0.15-0.3%. For the improvement on JERM, I base this directly on figures 9 and 12, as discussed above. The improvement on DAEM would be similar, due to the absence of improvement between JERM and DAEM optimisation.

- From DAEM-optimised to JERM-optimised: No observable improvement. This is based on my discussion of figure 8 above.

This interpretation of the results identifies the improvement attained by layout optimisation for AEP as the main cause of the improvements seen in JERM optimisation, with a small contribution of optimisation for DAEM. It identifies no improvement of JERM optimisation over DAEM optimisation. I fully agree with the underlying mechanism that the authors discuss for AEP optimisation, as well as for the mechanism of changing weights for wind-direction sectors in case of DAEM optimisation. However, I don’t think the results support any conclusion about the significance of and need for JERM optimisation, such as “this highlights the importance of accounting for participation of wind farms to reserve markets in the layout optimization process” (line 393-394) and “layouts optimized for profit maximization with reserve markets lead to better yearly profits than when considering day-ahead market only in the objective function” (line 467-468). I think the authors should reflect on such conclusions and suggestions of mechanisms to support them considering the previous discussion.

Technical and textual suggestions and corrections

• Please be more precise with the distinction between power and energy. Where needed, add ‘*delta_t’ for the duration of the time step, to get from power to energy. As an example, revenues in equation 4 are derived from power times price, where prices are given e.g. in figure 4 in Euro/MWh. Figure 4 also gives price for reserve capacity in Euro/MWh (instead of Euro/MW), for which it is not clear whether or not that is consistent. Please also take this into account for reserve activation R*k_a, which should be in terms of energy and not power (see also figure 1). The true implementation of power and energy in the model seems correct, since the revenues for DAEM only correspond with an estimate of them with a reasonable capacity factor for the wind farm.
• Please be more precise with the use of the term profit. In many places, such as in table 1, ‘profit’ is used, where ‘(net) revenue’ is meant, since costs for the wind farm are not accounted for. For convenience, I copied the use of ‘profit’ in my comments above, also where this is not correct.
• It could be helpful if chapter 2 already stated for which parameters data is used. That closes the set of equations. (So, for wind speed and direction, day-ahead prices, reserve market capacity and activation prices, imbalance fees, reserve market activation level, other bids in the reserve market, …?) As specified in a few comments below, for some of these parameters the text causes confusion as to whether these parameters are modelled, or whether (only) their forecast errors are modelled.
• 2, line 23: Please rephrase for readability.
• 3, line 85: analysis > analyses.
• 4, line 115-116: I propose to scratch ‘(normally not known by the wind farm operator)’. This is never known, by anyone.
• Forecast error modelling:
  • 5, Eq. (3): The modelling error associated with the wind farm model is not explained, nor are the values for its mean and standard deviation given.
  • 5, line 138: Can this sentence please be corrected or clarified. It seems that the distribution of forecast error is meant, but it states ‘of day-ahead electricity prices’ (for which a zero mean makes no sense). Can be articulated of what the percentage is taken?
  • 7, line 184-186: It is unclear whether these sentences are about modelling of prices, or (as would be expected) modelling of their forecast errors. Could that be clarified? Does ‘regulation prices’ mean ‘imbalances fees’, for which the (forecast) model has not been addressed elsewhere? Can the mean and standard deviations used for forecast errors of reserve capacity prices, reserve activation prices and imbalance fees be given? The current last sentence of this paragraph is open-ended.
• 5, line 131: similar patterns than > patterns similar to those of.
• 6, line 149: Could be specified to which assumption is referred here? The previous text only describes certain mechanisms and the choice of only providing upward reserve regulation; it gives no assumption.
• 6, line 154: farm > farms.
• 6, line 158-160: Could you clarify how the distribution of reserve activation amongst multiple bidders is modelled? Is data about bids in 2023 available and used? Furthermore, could be clarified how is determined if or how much of the reserve capacity bid is won, in case of multiple players and/or limited reserve need? The current formulation (Eq. (14)) implies that any bid is always won in full.
• 7, line 171-172: It is unclear how the availability tests for reserve capacity is implemented. In Eq. (10) and (11) the penalty of this test (= Eq. (8)) appears at every time step. Does this mean that the test is effectively done every time step, or is delta_R set equal to zero at all but 12 (random) time steps? Does the algorithm for optimising operation (Eq. (11)) in any way account for a probability that this test is performed in that time step, or is it assumed that the test will be performed in that particular time step with 100% certainty?
• 7, Eq. (9), (10), and several other related equations: Consider not to replace gamma_a by 1.3, since other values are also not substituted. The superscript c has been incidentally dropped from gamma in Eq. (10) and several other equations.
• 8, line 200-201: Please formulate more clearly what is meant by the 117 MW. Is that the highest needed reserve capacity in that year?
• 8, line 207: Can you clarify to which approach ‘this approach’ refers? This could be the approach of Soares et al., or it could be the approach that is presented in the current manuscript. If it is the latter, then the last sentence about submission steps of 1 MW doesn't seem to correspond with the given formulation.
• 8, line 211: do > does.
• 9, line 217: smaller than or equal to > larger than or equal to.
• 9, line 218-221: Eq. (15) keeps the turbines within the boundaries of a square wind farm area. This is not what is used in the case study. Could this be made consistent?
• 9, line 232: bee > be.
• 10, line 244: dependant > dependent.
• 11, line 254-255: Consider making explicit that the data is nevertheless used to optimise bidding for that year.
• 11, line 264: Can be clarified what is meant by this sentence? This seems more like an explanation of a multi-start optimisation to improve optimality than an improvement of statistical significance. That also aligns with later statements about results being given for the best performing layout (p.16, line 354-355). In case statistically significant is indeed meant: of which stochastic output?
• 14, line 317: This confirmed in > This is confirmed in.
• 14, line 319: I propose to scratch ‘losses’.
• 14, line 322: Please correct ‘turbines lifetime’.
• 15, line 327: day-prices > day-ahead prices
• 16, Table 3: Much of this table is a repetition of table 2. Consider whether table 2 can be removed, by using table 3 differently.
• 21, line 429: 4.1 > 4.4.
• 21, line 439: than > as.
• 23, line 451: than > as.
• 23, line 452: lead > leads.
• 23, line 453: than > as.
• 23, line 454: year > years.
• 23, line 455: than > as.
• 24, line 472: Please rephrase ‘periods of electricity shortage’. Belgium rarely (if ever) has electricity shortage.