the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 09 Oct 2023
Machine Learning-Based Estimate of The Wind Speed Over Complex Terrain Using the LSTM Recurrent Neural Network
Abstract. Accurate estimate of the wind speed profile is crucial for a range of activities such as wind energy and aviation. The power law and the logarithmic-based profiles have been widely used as universal formulas to extrapolate the wind speed profile. However, these traditional methods have limitations in capturing the complexity of the wind flow, mainly over complex terrain. In recent years, the machine learning techniques have emerged as a promising tool for estimating the wind speed profiles. In this study, we used the Long Short-Term Memory (LSTM) Recurrent Neural Network and observational lidar datasets from three different sites over complex terrain to estimate the wind profile until 230 m. Our results showed that the LSTM outperformed the Power Law as the distance from the surface increased. The coefficient of determination (R2) was greater than 90 % until 100 m when the input dataset included only variables of 40 m height. However, the performance of the model improved when the 60 m wind speed was added to the input dataset. Furthermore, we found that the LSTM model trained on one site with 40 and 60 m observational data and applied to others sites also outperformed the Power Law. Our results show that the machine learning techniques, particularly LSTM, is a promising tool for accurately estimating the wind speed profiles over complex terrain, even for short observational campaigns.
- Preprint
(1974 KB) - Metadata XML
- BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on wes-2023-104', Anonymous Referee #1, 07 Dec 2023
General Comments
This article works to further the use and understanding of wind forecasting using machine learning algorithms. This work uses wind observations at lower levels to make timeseries predictions of winds higher above the surface and compares them to the established method, the Power Law. The authors chose locations with complex terrain to experiment with predictive methods to research where they have the greatest forecast skill. They also tested the viability of swapping models to different environments to look at their fit under different conditions.
The authors break down the model output as a function of the local environments at the different sites, taking the time to delve into the meteorological reasons why the outputs look the way that they do. Despite this, I have concerns about the application of the Power Law, with α being kept constant. As alpha is explained to be a proxy for atmospheric stability, the different terrain scenarios will greatly affect this (i.e. stability is rapidly affected by the onset and movement of the sea breeze and land breeze), especially because the replication of the diurnal cycle was important to the results. The period of record in the training datasets will likely not capture the environmental response to the changing of seasons. The article opens with a lot of connections to previous works but fails to discuss any connections between the research done here and those previous studies.
Most of the paper requires grammatical revisions. There are also portions of the manuscript that do not pertain to the subject matter at hand. This work tackles research that can benefit the scientific community and the furthering of wind forecasting and its application with machine learning methods and their portability, thus I would recommend significant revisions.
Specific Questions/Comments
Page 1, Lines 14-24: The first paragraph of the introduction lacks substance that is specific to the work being done here. It discusses the use of Machine learning and its applications, but the scope is too broad. This may be done more concisely by collapsing this paragraph into a sentence or two discussing the variety of machine learning applications with several examples, while transitioning quickly into your second paragraph, which is far more relevant.
Page 4, Lines 88-94: Along with the governing equations of the LSTM cell, I suggest adding a diagram to help the reader. You can also connect to the different elements within the cell to strength your further explanation on lines 95-97.
Page 5, Line 122: The explanation for Experiment 3 and 4 needs more clarity. After reading into the results, I understood that your goal was to apply each trained model to a site that it was not originally trained for, however this was not clear when it was first proposed.
Page 5&6, Section 2.4: You lead off with the statement of “we chose the metrics that have typically been used in similar works”. This statement is rather vague and could use some support. At the end of the paragraph, you say “Zhou et al. (2022) and Baquero et al. (2022) provide detailed explanations for those metrics”. I suggest moving these references to the beginning of the paragraph and combining your statements… example: “We chose verification metrics like those used in Zhou et al. and Baquero et al., because…. For further information on these metrics, see Zhou and Baquero.”
Page 6, Line 150: [This is a stylistic critique] You use the wording “a Metropolitan Region (São Paulo city)”. Would it be better to say “the Metropolitan Region of São Paulo”? As the reader passes over parenthetical statements, they tend to cause a pause. This style of adding parenthetical statements for further description is used frequently in the paper. In my opinion, the number of parenthetical statements hurts the reading fluency and detracts from the reading experience. Please apply the stylistic switch where you deem necessary.
Page 6, Lines 157-166: The history of research on pollution in and around the Cubatão area is not greatly relevant to the background of this paper, nor its focus on wind forecasting. If the authors are trying to tie the importance of windspeed forecasting to the distribution of pollutants, this does not come across as a motivating factor clearly. If the authors did not intend for this to be a point of motivation for this research, I recommend the removal of this block of text.
Page 7, Lines 170-171: “Gasparoto et al. (2014) summarized some characteristics of the Ipanema National Forest.” If there is further information that the reader should be aware of regarding the characteristics of the Ipanema National Forest that are relevant to the research being conducted at site 3, I recommend paraphrasing from the source provided or removing the sentence if the information is not overly relevant.
Page 7, Line 179: “Both of them present a diurnal cycle” Your figures do not show a time series showing this result. Although I would caution adding more figures, you may put some thought into showing a time series here.
Page 8, Line 181: Could you elaborate on impact of the LLJ at the heights above ground you are using in this research?
Page 8, Lines 184 and 185: [Klockow and Targa (1998) figure 2] The figure was extremely simplified, to the point that I was left wondering how it provided any insight into windspeed prediction or anything to do with the sea breeze. Could you please add further insight from the use of this referenced figure?
Page 10, Lines 198-202: There are several references made in these lines, but they are not referencing the papers which foundationally developed the models. I would recommend adding a reference to the foundational works in addition to the references that guided the research done here thru the application and review of these models.
Page 10, Line 206: “Thus, using the entire dataset for training and testing the model takes a while.” Wording could be more professional here. As a technical comment, this is a good point to reference your computing resources. The use of Google Colab would obviously be different than if someone was training on new GPUs and a dedicated machine. The simplicity of this setup can be a benefit to further implementation on a wider scale by more users, as discussed on lines 294-297.
Page 11, Line 207: “At each step” At what steps? For each experiment?
Page 11, Line 211: “a short observational campaign can produce reliable results” Based on Table 1, your datasets only spanned certain times of the year sometimes missing seasons, and for site 3… entirely in the southern hemisphere winter. Is there any concern for the model’s usability for prediction outside of these timeframes?
Page 12, Lines 219-224: This paragraph covers a lot of different results. Due to the number of figures and tables, many of them are referenced in this paragraph in quick succession. This made things very difficult to keep track of. I suggest spacing out references and/or adding more elaboration on the results from each figure/table to increase the value of their inclusion.
Page 17, Line 261: “The forecast for the Site 1 highly improves when the 60 m wind speed is included on the input dataset for training the model at Site 3”. Where is the point of origin of the 60 m wind dataset? This has been left a bit unclear. See the silly equation below that popped into my head as I read this sentence…
[Prediction for Site 3] = [Model trained at Site 1] + ([60m data from Site 3]? Or [60m data from Site 1]?)
Additionally, have you considered adding 10m/2m data to any of these tests? These heights are the generally the most common in modelling and observed networks. When talking about applying a pretrained model from another region [maybe with similar terrain], it may be worthwhile to test this concept with data that would be more readily available at a wider range of locations.
Page 27, Lines 276-277: “Ensemble and hybrid methods are strategies that also contribute to the model performances.” Were these concepts tested in this study? If they were not, maybe this should be moved to an area discussing future work.
Page 27, Lines 279-281: “After testing some commonly used algorithms for the wind speed forecast (Random Forest Trees, Support Vector Regression and Multi-layer Perceptron), we found out the LSTM outperformed all of them. The LSTM outperformed even the decomposition methods.” These models were briefly discussed on page 9 and 10 starting at the end of line 196, but no results were ever shown to the reader, nor was it said that those models had poorer performance, only that “From this point, we refer to results with the LSTM RNN”. This needs to be revised in the results section before this statement is made in the conclusion.
Page 27, Line 283: “90%” How much of the remaining 10% of the data went towards validation and testing?
Page 27, Line 288: “However, the improvement was better to the Sites 1 and 2 than for the Site 3” Is there any speculation or further explanation of why this was the outcome?
Page 27, Lines 292-293: “The Site 2 is strongly influenced by the sea and land breezes and the LSTM model captured the abrupt changes of the wind profile better than the Power Law” Please see my major comment in the ‘general comments' section about my concerns with keeping alpha constant when implementing the Power Law. This is one of the cases where there would be a rapid change in stability. You incorporated 60m data into the training of your models, so can you use the change in windspeed between 40-60m over a short timeseries to better estimate alpha ~windshear coefficient?
Page 27, Lines 294-297: Reword these statements for better clarity.
Technical Corrections
Quick preface, if something is corrected here, it may have occurred more than once, however I will only be pointing out the first instance.
Page 1, Line 6: The use of the word “until” in this spot is grammatically incorrect. “Up to” may be a valid correction.
Page 1, Line 8: “variables of 40 m” changed to “variables at 40 m”.
Page 2, Lines 39-40: “Due the random feature of the wind speed” recommended rewording “Due to the random nature of windspeed when looking at data over short time scales”.
Page 3, Lines 81-82: “state-of-art” changed to “state-of-the-art”.
Page 4, Table 1: “31/Dec/2-16” changed to “31/Dec/2016”
Page 4, Lines 106-107: “retrieve information each 10 minutes” changed to “retrieve information every 10 minutes”.
Page 5, Lines 110-111: “the standard deviation of the horizontal (σu + σv) e vertical (σw) wind speed to forecast the wind speed at higher heights” Please revise, I am unsure what ‘e vertical’ is referring to.
Page 6, Line 151: “the strong social difference” I am unsure what is meant here; maybe ‘strong social disparity’?
Page 6, Line 157: “looks like a big wall.” Please revise with more scientific language or remove this statement.
Page 8, Figure 2: There are numbers along the concentric circles of the wind rose that lack units. Based on other examples in literature, these are “%” percentages based on the normalized time period, i.e. relative frequency. See the pdf file for a screenshot that depicts the content of this comment.
Page 9, Figure 3: Values discussed in my comment above are overlapped by data, making the relative frequency values impossible to read. See pdf for the illustration of this comment.
Page 9, Line 190: “along the day” change to “throughout the day”
Page 9, Line 196: “until to reach the results that will be presented in this section” Please revise wording. Maybe? “the model configurations that had the best performance will be presented in this section”
Page 10, Line 205: “50 thousand data” 50 thousand timesteps? Datapoints?
Page 11, Lines 206-207: “Surprisingly, we found that the model only improves until a limited dataset size and was unnecessary to take the entire dataset.” Suggested edit for clarity “Surprisingly, we found that model improvement plateaued without using all of the datapoints of record.”
Page 13, Line 238: The references used here should be prefaced as examples. ~ (e.g. Vassallo et al., 2020; Mohandes and Rehman, 2018)
Page 16, Figure 6: The line representing the Power Law stops being visible around 170m along the x-axis. This would be more representative if we could see the behavior of all three models tested.
Page 27, Lines 273-275: Grammar issues in first two sentences of the conclusion. Please revise for clarity.
-
AC1: 'Reply on RC1', Cassia Beu, 16 Jan 2024
Dear referee, we acknowledge your comments. All the comments will surely
enrich our work. We tried, on the best way, to attend all the contributions of both
referees. All the comments are answered on the attached file. Concerning the grammar, if the
draft is accepted, before publication, it will undergo a professional revision.
-
AC1: 'Reply on RC1', Cassia Beu, 16 Jan 2024
-
RC2: 'Comment on wes-2023-104', Anonymous Referee #2, 08 Dec 2023
This is a relevant study for the Wind Energy community, presenting a machine learning approach to estimate wind profiles over complex terrain, using valuable input data from field campaigns in the global South (where observations are typically more sparse).
My comments and suggestions are intended to improve the clarity of the paper.
Minor Comments - language and typos:
Lines 6-7: Replace 'until' with 'up to'.
Line 8: Replace ' when the input dataset included only variables of 40 m height' with 'for input variables up to 40 m high only'.
Line 10: Replace 'others' with 'other'.
Line 14: Replace 'is' with 'are'; and 'adopted as a power tool' with 'adopted as power tools'.
Lines 38-40: These two sentence could be improved. Here's a suggestion: 'Wind forecasts underpin wind power prediction, which is essential to support wind energy production in the short-term. Although winds have been traditionally forecasted with Numerical Weather Prediction models, the use of Machine Learning has become more widespread, not only to correct the biases derived from the highly variable nature of the winds, but also as stand-alone prediction models. Wang et al. (2021) ...'.
Line 43: Replace 'some works' with ' a few studies'.
Lines 44-45': Replace sentence as: 'They proposed the use of Long-Short Term Memory (LSTM) to improve wind speed forecasting for power prediction.'
Lines 48-49: Replace 'and still recommend that the adaptiveness of the hybrid models needs to be further researched' with 'whilst recommending further investigation on the capabilities of hybrid model approaches'.
Lines 53-54: Delete all 'the'.
Line 62: Replace 'others' with 'other'.
Line 64: Replace 'Following the same tendency' with 'Similarly, '.
Line 66: Replace as 'and observations at four different heights as input.
Lines 67-69: Replace sentence as: 'Bodini and Optis (2020a) and Bodini and Optis (2020b) found that random forests outperform standard wind extrapolation approaches, using a "round-robin" validation method. They highlighted the benefits of including observational data capturing the diurnal variability of the atmospheric boundary layer, namely the Obukhov length (...) at 4 m high.'.
Lines 72- 73: Replace as: ', advising to carefully select the input data and emphasizing the importance of normalization'.
Line 74: Replace 'over almost plain terrains' with 'over low complex terrain'.
Line 75: Replace 'to' with 'in'; 'the most of the studies' with 'most studies'.
Line 76: Replace 'for plain terrains' with 'low complexity orographies'; 'that analyzed different surfaces' with 'who analyzed different types of terrain complexity'.
Line 79: Replace as: Recurrent Neural Networks (RNNs) are a type of artificial neural networks where the output of one time step is used as an input ...'.
Line 109,120: Where you say 'levels' do you mean 'heights'?
Line 111: Replace 'e' with 'and'.
Lines 128-129: Replace start of the sentence: 'The authors state that:'.
Lines 146: Replace 'occurred along three years' with 'took place over a three-year period'.
Lines 146-147: Replace sentences as: 'All three observational sites are within x km from the coast, and clearly marked on the map.' - it'd be useful to know of far exactly they are from the coast, as proximity to the sea has a great influence over sea breezes and consequently the wind profile'.
Line 148-149: Replace 'among each other' with 'between sites'; 'the surfaces' with 'types of terrain'; 'starting by the altitude ...' with 'namely the height and surface roughness'. - My understanding is that here you mean height (above ground) and not altitude (above mean sea level), although altitude is also a relevant factor to consider.
Line 150-152: Replace 'by a chaotic constructive pattern that mixes high buildings common to a big city and simple residences' with 'by a densely mixed urban matrix'. (As is Sao Paulo, with a mixture of densely packed high-rise buildings and lower residential dwellings). I'd also remove the last sentence.
Lines 153-154: Delete the first 'The'; '... which is characterized by a large number of industries from the largest harbor complex in Latin America.'.
Line 155: Replace 'mountain chain ...' with 'mountain range, where Serra do Mar rises sharply, up to more than 700 m high, across 5 km wide.'.
Line 167: Delete the first 'The'.; replace as ', as shown in Fig. 1.'
Line 169: Replace as ' is the Aracoiaba Hill to the southeast, rising up more than 300 m high up to 900 m altitude.'
Lines 195-222: Replace simply as: '... testing different machine learning models with multiple configurations, namely: Random Forests (...). Here we only present results for the best performing model LSTM RNN ...'.
Lines 204-205: Replace as: 'Data from Site 3 was first used to train the model; starting with wind speeds at 40 m to predict speeds at higher heights'.
Line 209-210: Replace as: 'As the time series is comprised of 10-min temporal averages, that corresponds to roughly two months worth of observational data.'.
Line 212: Delete 'The' before tables.
Line 215: Replace 'As we see, for the three sites' with 'For all three sites'.
Lines 294-297: Replace as: 'The results found from this observational campaign, albeit short, show the benefits of Doppler Lidar in improving model results to estimate winds at height. This is particularly relevant to help support the Energy transition and Net Zero targets. Despite the costs associated with Doppler Lidars, the authors would encourage further strategic collaborations to drive observational data improvements leading to advances in model prediction.'
Major comments - literature review:
Lines 15-37: Although the literature review on Machine Learning in the environmental sciences is interesting, I am struggling to see how this is relevant to the main focus of the paper (apart from the Cubatao context given in page 6). I'd recommend replacing these paragraphs with a more significant state-of-the-art summary, for example about Machine Learning applied to wind profile prediction, wind resource assessment, and wind power forecasting. Here are a few suggestions: https://doi.org/10.5194/acp-23-3181-2023; https://doi.org/10.1088/1742-6596/1343/1/012047; https://doi.org/10.1016/j.renene.2021.10.034; https://doi.org/10.1063/1.4942120; https://doi.org/10.1002/9781394186570.ch11; https://doi.org/10.1016/j.renene.2020.03.148; https://doi.org/10.1016/B978-0-08-100504-0.00003-2; https://doi.org/10.2172/968212.
Lines 75-76: Regarding the statement made here about vertical wind extrapolation studies, I'd refer the authors to these works looking at a range of terrain complexities, as well as Machine Learning methods: https://doi.org/10.1002/we.2013; https://doi.org/10.1016/j.egyai.2022.100209. I'd suggest a read through these papers and references to complement the Introduction.
Lines 156-168: This is really interesting for socio-economic context and from a geographic point of view, but again I am struggling to see how it is relevant to the overarching theme of the paper. I think the authors could simply refer the reader to the Vieira and Gramani (2015) paper for further reading. Instead, I'd suggest to add a few notes on the climatic characteristics for that Sao Paulo region (where the 3 sites are located), as well as orographic complexity indices at each site (e.g. roughness length or silhouette area per unit area, if available). How are seasons typically classified for those areas and terrains - this would help complement the interpretation stated in page 9.
Major comments - graphics:
Lines 88-97: The mathematical explanation of the LSTM could benefit from a diagram of the network, to help readers understand visually the in/out/update mechanisms.
Figures 2,3,4: Would it be possible to round the edges, so that they match the circle lines? Also, it'd be useful to see seasonal results for wind roses per month or season - these could be added to the appendix.
Figures 5,6,7: Please add errors bars. Could you also add subplots for RMSE, MAE and F-score?
Major comments - requiring additional input:
Lines 99-100: Can you please expand how the interpolation was performed and how the data was normalized? For example, was it a simple linear interpolation, or normalized by the maximum wind speed value? This is important to understand how the input data was processed (e.g. were diurnal cycle or spatial corrections accounted for) and how that could be affecting the representativeness of the outputs. It would also be useful to know the percentage of missing data that was processed in this way.
Section 2.4 and Table 4: Can I suggest adding results for Root Mean Square Error (RMSE), so that error values are in the same scale as MAE and MAPE? Also, F-score is widely used in ML model evaluation and comparison - please consider computing it.
Lines 206-208; 219-220; 225-226: Regarding the statement "Thus, using the entire dataset for training and testing the model takes a while. (...) was unnecessary to take the entire dataset" - how long is 'a while', and did you consider compute optimization options such as parallelisation? This could help understand why a shorter input dataset was better - was it just due to compute constraints, or was it the quality of the data? I'd second these questions for Sites 1 and 2.
Line 209: Can you please clarify about the train-test-validation split ratios for each site experiment (i.e. was it just train-test or did you reserve some for validation)? Also could you highlight in the manuscript whether the LSTM model was trained and optimised on a site-basis, rather than for all 3 sites simultaneously?
Sections 3.1-3.4: A list (e.g. adding to Table A1) of all input variables for each experiment, as well as any relevant LSTM hyperparameters and network functions, would be useful to share.
Lines 210-211: Regarding the statement: "This means that a short observational campaign can produce reliable results", I think it's a bit premature to reach that conclusion, without seeing results for at least a whole annual cycle. My recommendation is to delete this.
Lines 298-300: For your final remarks, I'd encourage emphasizing the outcomes in two parts: obs campaings and ML model improvements. Both of which are valuable and complementary, although your ML modelling could be further improved if considering other sources of input data (e.g. reanalysis, NWP, etc.) and additional parameters (e.g. humidity, air density, pressure, temperature, orography and land-use).
Citation: https://doi.org/10.5194/wes-2023-104-RC2 - AC2: 'Reply on RC2', Cassia Beu, 16 Jan 2024
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 326 | 116 | 20 | 462 | 12 | 9 |
- HTML: 326
- PDF: 116
- XML: 20
- Total: 462
- BibTeX: 12
- EndNote: 9
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1