Thermal stratification plays an important role in wind farm flows and must therefore be included in simulations of such flows. At the same time, wind farms are covering larger areas, requiring very large domains and leading to exceptional computational costs for large-eddy simulations (LESs). The lattice Boltzmann method (LBM) is a novel approach to LES of wind farm flows that is particularly efficient and suitable for massively parallel hardware, such as graphics processing units (GPUs). In this work, we present a novel model for LES-LBM of stratified atmospheric boundary layers, using a so-called double-distribution function approach. We develop a novel boundary condition to apply Monin–Obukhov similarity theory and implement a number of other components required for simulations of stratified boundary layers in the GPU-resident version of the open-source LBM solver VirtualFluids. The model is validated for conventionally neutral and stably stratified boundary layers. Results agree closely with numerical references. The model is able to simulate conventionally neutral boundary layers with parameters typical for wind energy applications of the order of real time on a single GPU. Future work will include development of a precursor–successor method for wind farm flow simulations and improvements to the collision operator of the temperature model.
Wake recovery , the persistence of wind farm wakes , and the occurrence of gravity waves all depend on the thermal stratification of the atmospheric boundary layer (ABL). Large-eddy simulation (LES) enables the most accurate examination of these phenomena that is feasible with currently available hardware. Wind farms or clusters of wind farms cover large areas, yet simulations need to be well resolved to fully capture the behavior of the boundary and inversion layer . The resulting simulations thus feature extremely large numbers of degrees of freedom. Current LES models for the ABL, typically CPU-resident finite-volume or pseudospectral solvers, require days or months of computing time on very large compute clusters to perform such simulations.
A new generation of solvers seeks to alleviate the immense computational cost by utilizing graphics processing units (GPUs), such as MircoHH, AMR-Wind, and FastEddy. report that 32 CPU cores are necessary to achieve the same computational performance as one NVIDIA Quadro K6000. have already reported that 1 GPU equals the performance of 256 CPU cores, highlighting the rapidly increasing speed of GPUs.
However, a different approach, based on the lattice Boltzmann method (LBM), has also been introduced to wind energy and boundary layer research over the last decade. A recent review can be found in . The LBM's mathematical structure is well suited for the use of massively parallel hardware, such as GPUs. In the LBM, fluid is described as a set of populations on the nodes of a Cartesian grid. The LBM then follows a two-step algorithm. First, the populations at a node collide, then they are advected to neighboring nodes . The collision step is potentially non-linear but is a local operation, while the advection step simply consists of moving memory on a computer. While initial formulations of the LBM were unstable at high Reynolds numbers, recent advances in the formulation of the collision step have rendered it a suitable method for high-Reynolds flows and LES .
LES based on the LBM has been used now for over a decade to simulate large-scale boundary layer flows. present a simulation of 10 km× 10 km of Tokyo's urban area at a 1m resolution on up to 1000 GPUs, demonstrating the method's suitability for very large problems. Further examples of simulations utilizing GPUs include and . Both report near-real-time computational performance, demonstrating the high computational efficiency of the LBM on GPUs. None of the aforementioned simulations include wall models. introduced a wall-modeling approach suitable for atmospheric boundary layers and showed very good agreement with reference results.
All of the aforementioned models only consider isothermal boundary layers, and very few models considering thermal stratification have previously been presented in the literature. The temperature equation can be discretized either via “traditional” approaches, such as finite difference or finite volume, or via a modified LBM. The former is named a hybrid approach, while the latter is referred to as a double-distribution function approach (DDF). A hybrid approach was applied in one of the earliest studies of stratified boundary layer flows with the LBM, the TheLMA project, described in a series of publications . Another solver designed for atmospheric boundary layers implementing a hybrid approach is presented in . ProLB employs the hybrid recursive regularized collision model and the wall-modeling approach by . However, it is not mentioned that it utilizes GPUs, and no remarks on its computational performance are given. An overview of advection–diffusion LBM models can be found in , where the authors also compare a number of more advanced LBM models. They find that models based on the cascaded LBM yield the most accurate results. A similar model is applied in to simulate the dissolution in porous media. present a method using the DDF approach for simulation of the stratified flow over a ridge. Both LBM models (for momentum and temperature) employ a multiple-relaxation time method .
In this paper, we propose a novel model to simulate the stratified atmospheric boundary layer via a DDF LBM model based on the cumulant LBM for momentum and a factorized cascaded model for the advection–diffusion equation. We describe the methodology in Sect. . We compare results obtained with our method to reference data for both a neutral and a stably stratified test case in Sect. and give our concluding remarks in Sect. . As with any model development, we tried many different variants until we converged on the model we present here. We document some of those approaches in Appendix .
Methodology
We will be begin by describing the fundamentals of the cumulant lattice Boltzmann method and its modifications to simulate atmospheric boundary layers. Subsequently, we present the method used to simulate the advection–diffusion of the potential temperature. Finally, we discuss novel formulations for boundary conditions and other aspects specific to modeling thermally stratified boundary layers.
Governing equations
Our aim is to simulate the filtered incompressible Navier–Stokes equations with Coriolis forces coupled to an advection–diffusion equation of potential temperature via the Boussinesq approximation seeand references therein:
1∂ui∂xi=0,2∂ui∂t+uj∂ui∂xj=-1ρ0∂p∂xi+∂∂xjνe∂ui∂xj+FiC+FiB,3∂θ∂t+uj∂θ∂xj=∂∂xjDe∂θ∂xj.
Here, the coordinate system is denoted as x=[x,y,z]T; t is time; ρ0 is the density; u is the filtered velocity; p is the pressure deviation from the background pressure; θ is the filtered potential temperature; the Coriolis and buoyancy forces are FC and FB, respectively; and subgrid stresses and heat flux are parameterized via an effective viscosity and diffusivity, νe and De, respectively. We use Einstein's summation convention. The Coriolis force is given by
FiC=-ϵij3(Gj-uj)fc,
where ϵijk is the Levi–Civita symbol, G=G[cosα,sinα,0]T is the geostrophic wind with the geostrophic wind speed G and direction α, and fc=2Ωsinφ is the Coriolis parameter that depends on the rotational speed of the Earth Ω and latitude φ. The Boussinesq approximation assumes Fib=gθ-θrθrδi3, where g is the gravitational acceleration, θr is a reference temperature, and δij is the Kronecker delta. However, to remove the hydrostatic pressure, we take the horizontal average, denoted by 〈⋅〉, of Eq. () with i=3, following, e.g., . We find that 1ρ0∂〈p〉∂z=〈θ〉-θ0θ0, since the continuity equation dictates 〈u3〉=0. Splitting p into p′+〈p〉 and inserting the previous result in Eq. (), we can write
FiB=gθ-〈θ〉(z)θrδi3
if we replace p by p′. Contrary to “classical” computational fluid dynamics, we do not discretize these equations but instead solve them via the lattice Boltzmann method implemented in the GPU-resident version of the open-source solver VirtualFluids.
The cumulant lattice Boltzmann method
The fundamental variable of the lattice Boltzmann method is the particle distribution function (PDF) f. The PDF describes the probability of encountering a particle with velocity ξ at time t and location x. The discretization of velocity space to a lattice of discrete velocities cικλ=(ιδi1+κδi2+λδi3)c, with c=ΔxΔt as lattice velocity, leads to the discrete populations fικλ(x,t):=f(cικλ,x,t). Following , we denote lattice directions with triplets of Greek indices corresponding to their directions in space and define ι‾:=-ι. Note that Greek indices are not subject to the summation convention, and triplet indices in parentheses represent all possible permutations of that triplet. We employ a D3Q27 lattice, i.e., the set of 27 lattice directions of all permutations with ι,κ,λ∈{-1,0,1}. The lattice speed of sound is cs=c3, and each cικλ has an associated weight wικλ. Macroscopic quantities, that is quantities on the scale of continuum mechanics, are obtained by taking different order moments of fικλ, for example density (zeroth order) and velocity (first order):
6ρ=∑fικλ,7ρu=∑cικλfικλ+F2.
F is the total force density. The evolution of the PDF is described by the Boltzmann equation. Replacing the continuous PDFs with the discrete populations and integration along the characteristic x=cικλt from t to t+Δt yields the lattice Boltzmann equation:
fικλ(x+cικλΔt,t+Δt)=fικλ(x,t)+ΔtΩικλ(x,t),
where Ω is the collision operator, which is discussed later on. Asymptotic analysis shows that the moments of the lattice Boltzmann equation yield the weakly compressible Navier–Stokes equations, which approximate the incompressible Navier–Stokes equations with an error proportional to O(Ma3), with the Mach number Ma=V0cs and V0 being a reference velocity. The size of this error is effectively controlled by the size of the time step and the grid spacing since
Ma=3V0ΔtΔx.
By limiting Ma<0.1, we ensure that the error is small. The collision operator is of great importance for the accuracy and stability of the method. In this work we employ the cumulant collision operator ; here we present a short overview. Generally, collision operators relax the populations towards an equilibrium. The cumulant collision operator performs this relaxation in cumulant space, eliminating many shortcomings of traditional multi-relaxation time methods, mainly due to the fact that cumulants are statistically independent and can therefore be relaxed at independent rates. First, the populations in continuous form undergo a Laplace transformation to wave number space:
F(Ξ)=L∑ικλfικλδ(ιc-ξ)δ(κc-υ)δ(λc-ζ),
where δ is the Dirac delta function. Thereafter, cumulants cαβγ are obtained from the cumulant generating function
cαβγ=c-α-β-γ∂α∂β∂γ∂Ξα∂Υβ∂ZγlnFΞ|Ξ=Υ=Z=0.
Note that transformation from populations to cumulants is implemented via the chimera transform, which greatly reduces the computational cost while significantly improving the numerical precision of the computation Appendix I. After transformation, the cumulants are relaxed towards their respective equilibrium cαβγeq:
cαβγ∗=ωαβγcαβγeq+(1-ωαβγ)cαβγ,
where cαβγ∗ denotes the post-collision cumulant. Finally, the post-collision cumulants are transformed back to populations. The relaxation rates ωαβγ are computed according to . The relaxation rate of the second-order cumulants ω(110) is related to the kinematic viscosity by
1ω(110)=νcs2Δt+12.
In this study, we employ an eddy-viscosity model to explicitly model the subgrid scales of the LES. Some studies, e.g., and , suggest using the cumulant operator alone to conduct implicit LES. However, we have found this method unsuitable for performing LES of the atmospheric boundary layer due to the very high Reynolds number, as discussed in .
The lattice Boltzmann method for advection–diffusion
To solve the advection–diffusion equation, one can either use a so-called hybrid solver that solves the Navier–Stokes equations via the LBM and the advection–diffusion equation via finite differences or use the finite-volume method. The other possibility is to use another LBM solver to solve the advection–diffusion equation with a double-distribution function (DDF) approach. The hybrid method has the advantage that it requires significantly less memory, since for every node in the grid we only have to save one quantity, as compared to the DDF, which needs to save 27 (if one uses a D3Q27 lattice) quantities per node. Therefore, we also implemented a hybrid approach first, but, despite much effort, it was not successful. We discuss the details of the approaches we tried in Appendix . Instead, we pivoted to a DDF approach: We can describe a scalar, such as the potential temperature θ, with a second set of populations, gικλ, and define
θ=∑gικλ.
During collision, only the zeroth-order moment, i.e., θ, is conserved. The lattice Boltzmann equation for the advection–diffusion problem is analogous to the formulation for momentum:
gικλ(x+cικλΔt,t+Δt)=gικλ(x,t)+ΔtΩικλAD(x,t).
In this work, we employ the factorized central moment-based collision operator described in . Similar to the cumulant method, the populations gικλ first undergo a Laplace transform,
G(Ξ)=L∑ικλgικλδ(ιc-ξ)δ(κc-υ)δ(λc-ζ).
Central moments κ̃ are then obtained from the moment-generating function,
κ̃αβγ=c-α-β-γ∂α∂β∂γ∂Ξα∂Υβ∂Zγe-u⋅ΞG(Ξ)|Ξ=Υ=Z=0.
Again, the computation is performed via the chimera transform. To obtain the factorized central moments καβγ, the following orthogonalization is applied:
κ000=κ̃000κ(100)=κ̃(100)κ(110)=κ̃(110)κ111=κ̃111κ(200)=κ̃(200)-13κ000κ(210)=κ̃(210)-13κ(010)κ(211)=κ̃(211)-13κ(011)κ(220)=κ̃(220)-13κ000κ(221)=κ̃(221)-19κ(001)κ222=κ̃222-127κ000.
The factorized central moments are then relaxed towards their equilibria:
καβγ∗=ωαβγκαβγeq+(1-ωαβγ)καβγ.
All equilibria are zero, except
19κ(200)eq=cs2κ000,20κ(220)eq=cs4κ000,21κ(222)eq=cs6κ000.
The first-order relaxations are related to the diffusivity by
1ω(100)=Dcs2Δt+12,
while we set all other relaxation rates to one. After the relaxation, the factorized central moments have to be transformed back to central moments and then populations.
Boundary conditions
We require two boundary conditions for both momentum and potential temperature fields. At the top of the domain we set a slip condition for the fluid flow and either a Neumann- or a Dirichlet-type boundary condition for potential temperature. At the bottom we set a combined stress and flux boundary condition computed from a wall model that is discussed in Sect. . Boundary conditions in the LBM have to be specified for the populations; therefore a variety of methods can be found resulting in the same macroscopic boundary condition. For the sake of completeness, we describe the boundary conditions for the fluid in more detail in Appendix . A Dirichlet-type boundary condition for the scalar can be implemented via the anti-bounce-back rule as described in p. 318. A Neumann boundary condition can be implemented via this approach as well. However, in preliminary studies, we found this approach to cause spurious oscillations at the top of the domain.
Instead, we present here a formulation for a flux boundary condition, which will also be used as a Neumann boundary condition at the top of the domain. We first recall a few basic relations. The total flux j is the sum of the diffusive and advective fluxes jD and jA:
j=jD+jA=-D∇θ+uθ.
Our goal is now to set a specified wall flux qw, which will be computed either from a wall model or, in the case of a Neumann boundary condition, from qw=-D∂θ∂n, where ∂θ∂n is the specified gradient in the wall normal direction n, with n pointing to the fluid domain.
We compute the diffusive flux at the node from the first-order moment of g:
jiD=∑gικλcικλ,i-uiθ.
We then prescribe the flux at the wall jw to be equal to the diffusive flux in the tangential direction and equal to qw in the wall normal direction:
jiw=jiD-jjDnjni+qwni.
Finally, we employ the bounce-back rule to compute the missing distributions gικλ‾:
gικλ‾=gικλ-2wικλcικλ,ijiwcs2.
Wall model
At the bottom boundary, we make use of the standard Monin–Obukhov similarity theory (MOST) to determine the wall shear stress τw and heat flux qw:
27ζ(z)=zL=-zκgqwu∗3θr,28u(z)=u∗κlnzz0-ψM(ζ),29θ(z)-θ0=-qwu∗κlnzz0,H-ψH(ζ),
where ζ is a stability parameter based on the Obukhov length L; u∗=τwρ is the friction velocity; κ is the von Kármán constant; z0 and z0,H are roughness lengths for momentum and temperature, respectively; and θ0 is the surface temperature . The similarity functions for momentum (ψM(ζ)) and heat (ψH(ζ)) have to be determined experimentally. We use the classical Businger–Dyer relations :
30ϕM(ζ)=(1-γMζ)-1/4,31ϕH(ζ)=(1-γHζ)-1/2,32ψM(ζ)=ln1+ϕM22ϕM21+ϕM2ϕM2-2tan-11ϕM+π2,ζ<0,-βMζ,ζ≥033ψH(ζ)=2ln1+ϕH2ϕH,ζ<0-βHζ,ζ≥0.
To facilitate the comparison with reference data from in Sect. , we set βM=4.8 and βH=7.8. However, no values for unstable stratification are reported in . We therefore set γM=γH=15, as suggested by . Based on a user-specified distance zEL, we sample an exchange-location temperature θEL and velocity uEL. Additionally, the exchange-location quantities are exponentially averaged over time, as is recommended by . We do not apply any spatial averaging. The user can choose to prescribe either the surface heat flux or the surface temperature. We employ a variation of algorithm(1) or algorithm(2) from , depending on the prescribed quantity, displayed in Algorithm . In the following, the subscript t denotes vectors tangential to the wall, overbars denote temporal averages, and quantities at the first node in the fluid domain are denoted with the subscript 1. Superscripts indicate the time step.
Algorithm for computing wall shear stress and kinematic heat flux.
u∗t←u∗t-1, qwt←qwt-1
repeat
u∗Old←u∗t
compute ζ via Eq. ()
compute ψH and ψM via Eqs. () and ()
u∗t=κuEL,tt‾lnzELz0-ψM-1
if surface temperature given then
qwt=-u∗tκ(θ‾EL-θ0)lnzELz0,H-ψH-1
endif
untilu∗t-u∗Oldu∗Old<10-4
τw=ρ(u∗t)2u1,t|u1,t|‾
The combined boundary condition, which we refer to as the surface layer boundary condition, is executed in the following steps:
Load fικλ∗ at the boundary node, and compute ρ1 and u1 via Eqs. () and ().
Load gικλ∗ at the boundary node, and compute θ1 via Eq. ().
Compute τw and qw according to Algorithm using uEL, θEL, z0, z0,H, zEL, and θ0.
Apply the inverse momentum exchange method to determine uw and fικλ‾.
Apply the flux boundary condition to determine gικλ‾.
Thus, the boundary condition is entirely local, with the exception of uEL and θEL. Furthermore, the populations gικλ‾ can be computed independently, making the boundary condition easily adaptable to curved boundaries. However, the populations fικλ‾ cannot be computed independently.
Further models
A number of further modifications to VirtualFluids had to be implemented in order for it to be fully equipped to conduct simulations of atmospheric boundary layers. Namely, Coriolis and buoyancy forces have to be computed, and a Rayleigh damping layer has to be implemented.
Coriolis force
The Coriolis force can be computed directly from Eq. () based on a user-prescribed geostrophic wind and Coriolis parameter. The Coriolis force is simply added to the body force field in our implementation. Further potential for optimization by combining the computation with the collision kernel was left for future work.
Buoyancy force
As described in the beginning, we model the effect of buoyancy via the Boussinesq approximation. After the collision kernel for g, we add a number of models to compute the buoyancy. For simplicity's sake, we allocate an array with the size of the grid for a local reference temperature. The constant-buoyancy provider only computes a buoyancy force according to Eq. () and adds it to the body force field. Thus we can implement a constant-reference-temperature profile simply by changing the way that the reference temperature is initialized. The second variant computes buoyancy relative to the horizontally averaged temperature, as described by Eq. ().
Damping layer
Since the free atmosphere is essentially an undamped oscillator, spurious oscillations that can arise at the top of the capping inversion propagate throughout the domain. To mitigate these waves, it is common practice to use Rayleigh damping layers . The force of in the damping layer FD is computed by
FiD=-fz-zsze-zswδi3.
The function f(z̃) of height normalized between start and end heights zs and ze of the damping layer can be chosen freely in our implementation but is normally set to f(z̃)=fRsin2π2z̃, with a damping factor roughly fR= 1 × 10−41s-1.
Turbulence models
As mentioned in Sect. , we parameterize subgrid-scale fluxes with effective viscosity and diffusivity models. A few turbulent viscosity models have been implemented in previous studies, namely the Smagorinsky–Lilly , QR , and anisotropic minimum dissipation (AMD) models. Note that due to the relation between second-order cumulants and the stress tensor, the Smagorinsky and QR models can be computed very efficiently during the collision step. In addition, we have implemented a number of turbulent diffusivity models for this study, two standalone diffusivity models, namely a constant turbulent Prandtl number model and the model suggested by . Finally, we have also implemented the stratified AMD model proposed by , which augments the original AMD model for turbulence viscosity with a term modeling the effect of buoyancy on turbulence and includes a turbulence diffusivity. As this model requires the full velocity gradient tensor, which is only available in between collisions, we compute the effective turbulence viscosity and diffusivity for computing the collision at time step t from the velocity and temperature field at time t-1. In preliminary studies, we found the stratified AMD model to yield the best results; therefore we use only this model in the remainder of this study.
Results
We validate our model against reference data in two stability conditions, namely conventionally neutral and stable conditions. We provide a convergence study of the advection–diffusion model in Appendix and find the order of convergence to be slightly above 2 for both advection and diffusion.
Conventionally neutral boundary layer
We begin validation of our model for thermally stratified boundary layers by comparing them to the conventionally neutral boundary layer (CNBL) simulation described in . This case has also been used to validate the LES solver AMR-Wind, and we therefore have two references to compare them to. The geostrophic wind speed G is 5 ms-1, the Coriolis parameter is 10−41s-1, and the lapse rate of the free atmosphere is ∂θ/∂z= 3 Kkm-1. The reference temperature is θr= 290 K, and gravitational acceleration is 9.81 ms-1. The domain has an extent of 2560 m× 2560 m× 896 m. We conduct simulations at two resolutions, Δx= 7 m and Δx= 3.5 m, corresponding to grids B and C of the original publication, respectively. The boundaries in the streamwise and lateral directions are periodic. At the top we employ a slip condition for momentum and the Neumann condition, as described in Sect. , for the potential temperature, with a temperature gradient equal to the free atmosphere lapse rate. The bottom boundary is a surface layer boundary condition with a prescribed heat flux of 0 Kms-1 and a roughness length of z0= 0.05 m. The domain is initialized with constant geostrophic wind speed and a constant temperature gradient equal to the lapse rate of the free atmosphere. Further details of the reference case can be found in . We assume an eddy turnover time TE= 1700 s and average from 55 to 65 TE. We use the stratified AMD model with a model constant set to 1/3. Since we use an explicit subgrid-scale model, we “turn off” the limiter of the cumulants by setting it to 105. No damping is activated.
The original study employs a pseudospectral code developed at the National Center for Atmospheric Research (NCAR) over the last 40 years, with pseudospectral spatial discretization in horizontal directions and second-order finite differences in the vertical direction. Time stepping is performed with a third-order Runge–Kutta scheme. In addition to the results from the original publication, we also compare our results to the results published in the Exawind benchmark database obtained with AMR-Wind. AMR-Wind utilizes a combination of finite-volume and finite-element methods for spatial discretization and second-order accurate time stepping; details on AMR-Wind can be found in . We compare our results to results obtained on grids C and D, with a resolution of Δx= 3.5 m and Δx= 1.75 m, respectively, as these are the resolutions available in the Exawind benchmark database.
Instantaneous velocity fields of the CNBL case for grids B (left) and C (right). Horizontal wind speed (top) and vertical velocity (bottom) in the plane at z= 37 m are shown. The black arrow in the top row indicates the average wind direction in the plane.
Instantaneous velocity fields of the CNBL case for grids B (left) and C (right). Horizontal wind speed (top) and vertical velocity (bottom) in the plane at z= 333 m are shown. The black arrow indicates the average wind direction in the plane.
We provide a qualitative impression of the simulation in Figs. and , where we show instantaneous horizontal wind speed and vertical velocity at z= 35 m and z= 333 m, equivalent to 10 % and 90 % of the boundary layer height, respectively. Similar plots are shown in . At the lower height we see the dominance of small-scale turbulent structures in both horizontal and vertical directions, as expected due to the proximity of the wall. At higher resolution we can observe that smaller scales are resolved. Close to the inversion height, we find much fewer small scales; instead, larger structures dominate. Comparing the direction of the mean horizontal velocity indicated by the black arrow, we observe the expected veer.
Vertical profiles of averaged velocity (left), wind veer (center), and temperature (right) of the CNBL reference case.
Moving to a more quantitative analysis, we show the horizontal averages of wind speed S, wind direction ϕ, and potential temperature of our model, referred to as VF, alongside respective results from references in Fig. . Overall, we find very good agreement between our results and the references. Within the boundary layer in particular we observe very close agreement in all quantities, indicating that the boundary condition and other models behave correctly. However, we observe that the B grid is not able to properly resolve the upper edge of the capping inversion, resulting in a weaker gradient. This is in line with observations in and a range of other literature, for example . Furthermore, we observe that the wind direction in the free atmosphere is not exactly aligned with the geostrophic wind for the higher-resolution case. This inaccuracy is due to the forcing of the geostrophic wind being very small compared to the streamwise velocity in the free atmosphere. See Appendix for more details. Nevertheless, the error is small and seems to have a negligible effect on the wind direction below the inversion.
From the averaged results we can compute an inversion height zi as the height with the maximum temperature gradient. We list our results next to the results reported by and AMR-Wind in Table . Furthermore, we list the friction velocity computed from the wall model. All results agree closely. The inversion height decreases with higher resolution for all solvers. AMR-Wind reports the lowest inversion heights, while our results for grid C are in the middle. We report the lowest friction velocity, while report the highest. Nevertheless, the results agree within 10 % of each other.
Friction velocity and inversion height of the CNBL case compared to references.
QuantityVirtualFluidsBerg et al. (2020) AMR-WindBCCDCDu∗/ms-10.2070.2050.2250.2210.2080.203zi/m381.5362.3372350352.1337.0
Vertical profiles of resolved vertical momentum flux in the streamwise (left) and lateral (center) directions and resolved turbulence intensity (right) from the CNBL reference case.
We show second-order statistics in Fig. , where we present the vertical momentum flux in the streamwise (u′w′) and lateral directions (v′w′) and turbulence intensity based on the horizontally averaged wind speed, computed as 2TKE/3/〈S〉‾, where TKE=12(〈u′u′〉‾+〈v′v′〉‾+〈w′w′〉‾) is the resolved turbulence kinetic energy. Again, we find very good agreement with both references. The streamwise flux even shows excellent agreement. We can see the influence of the mismatch in wind direction in the lateral vertical momentum flux, which is slightly too high in the upper region of the boundary layer. The turbulence intensity is slightly higher near the ground than the results from AMR-Wind, while the agreement with the results from Berg et al. (2020) is very close. The prominent increase in turbulence intensity at the inversion height observed in AMR-Wind is significantly less prominent in our results, even at the same resolution.
Vertical profiles of total vertical momentum flux in the streamwise (left) and lateral (center) directions and total turbulence intensity (right).
To evaluate the performance of the SGS model in more detail, we present the total flux as the sum of resolved and subgrid-scale fluxes τij=νe(∂ui∂xj+∂uj∂xi) and the total turbulence intensity TKEtotal=TKE+12〈τii〉‾ in Fig. . Note that total fluxes are only available from . Again, we find very close agreement with the reference data. Only the lateral flux deviates somewhat from the reference results in the upper half of the boundary layer.
Horizontal (top) and vertical (bottom) spectra at z= 37 m (left), z= 186 m (center), and z= 333 m (right).
Finally, we compare spatial velocity spectra obtained at three different heights (z= 37, 186, and 333 m) in Fig. . We compute spectra from horizontal planes following the procedure described in . First, we compute the spectral tensors Φ11, Φ22, and Φ33 from the covariance function Rij(rx,ry,z)=〈ui′(x,y,z)uj′(x+rx,y+ry,z)〉:
Φij(kx,ky,z)=1(2π)2∬Rije1^(rxkx+ryky)drxdry,
where 1^ is the imaginary unit.
Then, we compute the ring-averaged energy in horizontal and vertical spectra Eh(kh) and Ev(kh) with kh=kx2+ky2:
36Eh(kh)=12∫02πΦ11(kh,ϕ)+Φ22(kh,ϕ)dϕ,37Ev(kh)=∫02πΦ33(kh,ϕ)dϕ.
Results are binned into 50 equally sized bins. Wavelengths are normalized with the computed inversion height, and spectra are normalized by 2πziu∗ah and 2πziu∗av, with ah=0.54(55/18)a, av=0.61(55/18)a, and a=0.5, in accordance with . As a final step, we average the spectra computed from 10 time steps to reduce noise.
Our results match the results from AMR-Wind very closely at all heights despite the lower resolution. This demonstrates the lower dissipativity of the cumulant LBM compared to finite-volume solvers. At lower wavenumbers there is also very good agreement with the results from . At high wavenumbers, the LBM is more dissipative than the pseudospectral solver. Close to the inversion height, the small wavenumbers are lower due to the capping inversion, and a characteristic hump is visible in the vertical spectra, which we could accurately reproduce with our solver.
Overall we find very good agreement in all examined quantities with the reference data, even at lower resolutions. In general, the accuracy of the model is positioned between the two reference models. The newly developed surface boundary condition is able to model the wall region accurately in neutral conditions.
Our simulations were carried out using a single NVidia RTX A6000 GPU on a workstation computer. Simulating 1.225 × 105s with grid B required 2.7 × 104s of wall time, while the simulation of grid C ran for 4.3 × 105s, which is approximately a 16-fold increase as expected. Hence, we are able to run full boundary layer simulations using a workstation computer of the order of real time. Further details of the computational efficiency when scaled to multiple GPUs can be found in Appendix . Compared to isothermal simulations on the same hardware, the computational speed is reduced by around 26 % due to the double-distribution approach, which essentially doubles the memory accessed per node as well as the additional models for Coriolis and buoyancy forces. Future work will combine the different forces and collision kernels to minimize memory access and increase computational efficiency.
Stably stratified boundary layer
To examine the performance of our model in stable boundary layer simulations, we compare it with the well-known GABLS1 benchmark . The domain has an extent of 400 m× 400 m× 400 m, with periodic boundaries in the streamwise and lateral directions. The geostrophic wind is set to 8 ms-1 and the Coriolis parameter to 1.39 × 10−41s-1. The domain is initialized with a constant velocity equal to the geostrophic wind and a two-layered temperature profile. The lowest 100 m is initialized with a constant temperature θ0= 265 K, above which sits an inversion layer with a temperature gradient of 0.01 Km-1. In the lowest 50 m, the temperature is superimposed with random fluctuations with an amplitude of 0.1 K. The surface temperature is also initialized with θ0, and a constant cooling rate of 0.25 Kh-1 is applied. The roughness length is set to z0= 0.1 m. The reference temperature is set to 263.5 K, density to 1.3223 kgm-3, and gravity to 9.81 ms-2. A Rayleigh damping layer is used at the top, with the damping factor set to 1.6 × 10−31s-1. We apply the same boundary conditions as in the previous case, but now the surface temperature is prescribed in the surface layer boundary condition. The total simulated time is 9 h, and averages are computed over the last hour. We simulate two grids, one with Δx= 2 m and a finer resolution of Δx= 1 m.
We compare our results to data from the original benchmark and a later publication by . In the original paper, a number of different models are compared, with a large variety of formulations. employ a pseudospectral discretization in horizontal directions and second-order finite differences in the vertical direction and Adam–Bashforth time stepping. They compare three different turbulence models, the Smagorinsky model, the AMD model, and the Lagrangian-averaged scale-dependent model (LASD) . We compare our results only to the results obtained with the Smagorinsky and LASD models since the results differ only marginally between LASD and AMD. conduct all simulations at an isotropic resolution of 2.08 m.
Instantaneous velocity and temperature fields at x= 200 m of simulations of the GABLS1 reference case with both resolutions at t= 9 h.
As with the previous case, we first show an instantaneous view of the simulation in Fig. . The capping inversion is clearly visible in the velocity field, exhibiting a super geostrophic wind and high veer. Above the inversion, a clear reduction in turbulence can be observed. At the higher resolution the transition from boundary layer to free atmosphere is sharper, which is discussed in more detail later on. The temperature field shows a stable stratification and the presence of a strong capping inversion.
Planar-averaged first- and second-order statistics of the GABLS1 reference case. From left to right: wind speed and lateral velocity, temperature, total momentum fluxes, and total vertical temperature flux. The area covered by all results at 2 m resolution from is shown as a blue-shaded area.
We show profiles of horizontally averaged quantities of simulating the GABLS1 reference case in Fig. . The wind speed below the inversion agrees well with the reference data in both cases. The case with lower resolution exhibits a significantly lower velocity gradient than the higher-resolution case. This is in line with findings from , where simulations at a lower resolution also exhibited this behavior. At the higher resolution, our results match those of closely. We find higher negative veer in the inversion layer compared to , particularly at higher resolution, which also results in higher wind speed in that region. However, there seems to be only a very limited effect on the flow in the boundary layer. The temperature profile at the lower resolution agrees well with the reference data within the boundary layer. In the inversion layer the gradient is slightly lower than that of but still well within the range of results from . At the higher resolution we find very good agreement. The vertical momentum fluxes agree very well with the reference results. Only results from are available. The vertical temperature fluxes give a similar picture, although they are slightly smaller than the reference data. Furthermore, there exists a small hump in the results from the case with lower resolution. We believe this is connected to the velocity profile, where the top of the inversion had a significantly lower gradient.
Friction velocity, boundary layer height, and buoyancy flux of the GABLS1 case.
QuantityVirtualFluidsBeare et al. (2006) Gadde et al. (2021) Δx= 2 mΔx= 1 mΔx= 2 mΔx= 1 mSmag.LASDu∗/ms-10.270.260.24–0.28–0.2650.253h/m179162162–197149–164166166-g〈wθ〉θ0⋅104/m2/s34.053.663.5–4.7–4.13.8
A comparison of some quantities of interest is shown in Table . Note that the friction velocity and buoyancy flux are computed from the total fluxes at the second node since this is the node we use as the exchange location for the wall model. We compute the boundary layer height h with the same method used in the references. We first find the height h0.05 at which the shear stress (u′w′)2+(v′w′)2 is less than 5 % of the wall shear stress and extrapolate by h=h0.050.95.
The friction velocity and buoyancy flux are well within the range of the reference results. This indicates that our wall-modeling approach and boundary condition yield accurate results. The boundary layer height decreases with increasing resolution, as was also observed in .
To examine the performance of the new boundary condition in more detail, we show the time evolution of friction velocity and buoyancy flux in Fig. . In the initial seconds of the simulation, both quantities exhibit large spikes. The friction velocity first decreases quickly as a boundary layer develops, decreasing shear in the lowest part of the domain. The surface heat flux increases in magnitude as the surface cools, and thus the temperature gradient near the surface increases. Both quantities stabilize towards the end of the simulation, indicating that the simulation has reached a state of equilibrium. This behavior is qualitatively similar to the results reported in and . Despite varying formulations, different approaches yield similar results near the equilibrium state of the boundary layer.
Time evolution of friction velocity and buoyancy flux of the GABLS1 reference case.
It is worth noting that the reference results also yield a large variation in all predicted quantities, as noted by other studies comparing to this case, e.g., and . Generally, our results within the boundary layer fall well within the range of the results obtained by other solvers, despite the differences in approach, subgrid-scale models, etc. At the top of the boundary layer, our model requires higher resolution than most other solvers to accurately represent the capping inversion. We believe this is caused by the model not representing the temperature gradient accurately enough. One way to improve the model is to further refine the collision operator of the advection–diffusion LBM. In a comparison of different advection–diffusion collision operators, found a two-relaxation time central moment operator to be more accurate than the central moment operator most similar to the one employed here. By introducing more relaxation times, leading-order error terms can be canceled out, and accuracy of the collision operator can be improved, as was done, for example, in .
Conclusions
This paper presents a novel method for conducting large-eddy simulation of thermally stratified atmospheric boundary layers using the double-distribution function (DDF) lattice Boltzmann method (LBM) in a GPU-resident solver. Very few applications of the LBM to stratified atmospheric boundary layers have been presented in the literature so far, and this work comprises the first application of a DDF approach to such flows in conjunction with employing GPUs. We give a thorough description of our methodology for the simulation of the bulk flow, present a novel boundary condition to use a combined wall model for wall shear stress and heat flux prescribed by Monin–Obukhov similarity theory, and present other models implemented in the GPU-resident LBM solver VirtualFluids in order to simulate stratified atmospheric boundary layers, including horizontally averaged buoyancy, Coriolis force, and Rayleigh damping layer.
We test our model in simulations of conventionally neutral and stably stratified boundary layers. Simulations of the conventionally neutral boundary layer agree very well with reference data obtained with both pseudospectral and finite-volume methods. At a coarse resolution of 7 m, the inversion layer can not be represented accurately. At a finer resolution of 3.5 m, the results match closely with the reference data at twice the resolution obtained with pseudospectral and finite-volume methods. Second-order statistics also agree very well. The spectra obtained at three heights show that the LBM exhibits excellent spectral properties and has lower diffusion than the finite-volume solver at higher resolution. Damping of vertical motions near the inversion layer is also clearly present.
Simulations of the stably stratified GABLS1 reference case also yield satisfactory results. The proposed boundary condition is able to properly reproduce the friction velocity and buoyancy flux at the wall. The boundary layer height agrees with results obtained at the same resolution.
A general shortcoming of the model is its inability to correctly reproduce the direction of the geostrophic wind, and this discrepancy grows with increasing resolution. However, we find that this does not affect the results in the boundary layer, and the misalignment is small. Overall, we achieve satisfactory accuracy for both cases. The method exhibits the expected behavior, generally being more accurate than a second-order finite-volume method, but not as accurate as a pseudospectral approach.
The present model exhibits excellent computational efficiency. All simulations, even the highly resolved neutral boundary layer with ≈ 140 million nodes, are carried out on a single graphics card. At a coarse resolution of Δx= 7 m, the simulation is carried out 4.5 times faster than real time. Increasing resolution to Δx= 3.5 m results in a simulation at 0.28 real time.
This article comprises a model for an empty boundary layer with flat terrain. Future work will focus on implementing a precursor–successor setup to simulate wind farms. One of the limitations of our model is that the surface layer boundary condition is only formulated for straight walls. As noted by , an extension of the inverse momentum exchange method is possible but not available as of yet. Nevertheless, this work represents an important step for the lattice Boltzmann method and CFD in general towards simulations of stratified boundary layers while fully leveraging the computational efficiency of GPUs. It represents one of the most cost-effective and fastest methods available to conduct LES of such complexity. The reduction in computational cost benefits researchers in many ways, from reducing development time to enabling larger parameter studies. Furthermore, a reduction in computational cost is crucial for enabling industrial application of LES, for example in the wind energy sector. That way, manufacturers, developers, and operators can take into account the complex behavior of the atmospheric boundary layer and reduce model uncertainties, ultimately reducing the cost of electricity.
Unsuccessful preliminary studies
The development of this model was rich with paths that led us nowhere, as is often the case when developing a new model. We want to record some of those paths in the hope that others might learn to either avoid those paths or will see where we went wrong and will be able to tell us what we should have done instead.
Hybrid finite-difference scheme
We first tried to implement a hybrid solver by using finite differences for the advection–diffusion problem. The hybrid method has a number of benefits. The finite-difference approach is much simpler and much more well known; hence there is also more literature on the topic. Furthermore, it requires significantly less memory while also yielding higher computational performance. Hence it is used in a number of other thermal LBM models, for example and . Results for the canonical test cases looked very promising, so we decided to pursue this direction further. However, when we simulated the atmospheric boundary layer, we could never avoid spurious oscillations that ultimately degraded the simulation, particularly at the top of the inversion layer. We implemented a variety of approaches, beginning with central differences and Euler forward time stepping. We refined our approach, using a variety of different finite-difference schemes, such as the second-order upwind scheme, the MUSCL scheme , the QUICK and QUICKEST schemes , and the mixed fourth-order central difference and QUICK schemes. We also tried a second-order Adam–Bashforth time integration but all to no avail. At this point we pivoted to a DDF approach that was already implemented in VirtualFluids since adding even higher-order approaches did not seem promising. As of yet it is unclear what the differences were in our approach compared to, for example, the model implemented in ProLB. On the one hand, we use a much less diffusive collision operator; thus oscillations are not damped as much. On the other hand, no other study actually simulates a capping inversion, where the oscillations originated from our simulations. Furthermore, there is very little literature concerning this issue. We speculate that the instabilities occur due to the differences in stencil/lattice. The lattice Boltzmann method only accesses the direct neighbors, while all the higher-order methods require information from the second neighbor as well; thus information can travel at different speeds. As this was not the aim of this work, we did not explore this direction further.
Reference temperature
To improve the numerical precision, the populations gικλ are set so that θ-θm=∑gικλ. The choice of the reference temperature θm is crucial to improve the accuracy of the simulation. A naïve choice would be to either set it to the reference temperature θr or the surface temperature θ0; however we found that oscillations tended to originate from areas where the temperature is far away from θm. We found that the best choice was usually to set θm to a value of the temperature in the inversion layer.
Boundary condition fluid
We set a slip boundary condition using a similar method to the one used to set the flux boundary condition. At the uppermost fluid node, we compute the velocity from Eq. () and then compute the tangential velocity with
uit=ui-(ujnj)ni.
Then we apply the bounce-back rule:
fικλ‾=fικλ-2ρwικλuicικλ,ics2.
At the bottom boundary we employ the iMEM approach from . We want to give a few clarifications and correct some misprints in the original publication. Recall that the momentum transferred from the fluid to the wall is
Δpικλ,i=(fικλ+fικλ‾)cικλ,i.
Hence, the total force exerted onto the wall by the fluid is
Fi=Δx3Δt∑ικλ∈ΓΔpικλ,i,
where Γ is the set of all links cutting the wall. From the wall model we compute the force acting on the wall from the wall shear stress
Fi=τiwΔx2.
We now seek a wall velocity uw such that the bounce-back rule in Eq. () results in the correct force. To that end, we split the force into two components, the force due to the population and due to the wall velocity, Ff and Fuw:
B6Fif=Δx3Δt∑ικλ∈Γcικλ,i(fικλ+fικλ‾),B7Fiuw=-Δx3Δt∑ικλ∈Γ2ρwικλcικλ,iujcικλ,jcs2.
Thus, Fiuw=Fi-Fif, resulting in a system of equations that needs to be solved for uw. For the case of a straight wall at the bottom, the solution is
uw=-3cs2ρΔt2Δx43F1uw,3F2uw,F3uwT.
Convergence study
To determine the order of convergence of the advection–diffusion equation numerically, we simulate the advection–diffusion of a Gaussian hill of concentration. Note that in this case, θ is a passive scalar. The initial field of concentration θ is described by p. 322:
θ(x,t=0)=θ0exp-x-x022σ0.
Analytical solution for the concentration in the test case of a Gaussian hill of concentration, projected onto the x–y plane. In the white region concentration θ≤ 1 × 10−10.
Maximum error Δθ of the numerical results of simulating the advection–diffusion of a Gaussian hill of concentration.
Absolute difference between analytical and numerical solution of the Gaussian hill of concentration at t=Tend. The top row shows the case with Pe=1 and the bottom row the case with Pe=104. Resolution increases from left to right.
Under a constant advection velocity u,
θ(x,t)=σ02σ02+σD2θ0exp-x-x0-ut22(σ02+σD2)
is a solution for the advection–diffusion equation
∂θ∂t+u⋅∂θ∂x=1Pe∂2θ∂x2,
with σD=2Dt, the Peclét number Pe=σ0UD, and u=[1,1,1]TU. We conduct a convergence study for a diffusion-dominated problem (Pe=1) and an advection-dominated problem (Pe=105). We vary N=σ0/Δx=2,4,8,16, while we keep σ0=1 and Δt=1 constant. In the case of Pe=1, we set the diffusivity in lattice units D̃=DΔtΔx2=0.01, and we simulate until Tend=0.3σ0U is reached. In the case of Pe=105, D̃= 1 × 10−5 and Tend=3σ0U. In both cases the domain has a size of 18σ0×18σ0×18σ0. To optimally utilize the simulation domain, we set x0=-TendU/2. The analytical solutions for both Peclét numbers at t=0 and t=Tend are shown in Fig. . The results of the convergence test for both Peclét numbers can be found in Fig. . The results clearly show a convergence rate slightly above second order in both cases, as expected. More precisely, we compute a convergence rate of 2.5 and 2.2 for Pe=1 and Pe=105, respectively. A more detailed view of the error can be found in Fig. , where we show the absolute difference between the analytical and numerical solution. We see that in both cases, errors decrease with higher resolution and that the error is symmetric in the diffusion-dominated case, while the advection case exhibits errors aligned with the direction of advection. We also see that errors become negligibly small towards the boundaries of the domain, so the domain was chosen to be large enough to not affect the results.
Scaling study
TwallTsim for all cases of the scaling study.
Number of GPUs1248Weak scaling1.812.032.122.12Strong scaling1.811.090.630.55Strong scaling reference1.810.920.450.23
We conduct both a weak and a strong scaling study of the model based on the neutrally stratified test case with grid C (Δx= 3.5 m). However, we want to emphasize that the model in its current state has not yet been optimized for performance and not at all for multi-GPU performance. These improvements will be a topic of future studies.
The study is conducted on the Dardel supercomputer from 1 to 8 GPUs. Each node has 4 Nvidia GH200 Grace Hopper superchips with 120 GB memory each. The domain is partitioned into equally sized subdomains along the streamwise direction. This was found to be the optimal partitioning strategy for this case. Each case is run for 100 s of simulation time, and the wall time Twall of the longest-running process is recorded. On a single GPU, the domain has a size of 3060 m× 3600 m to fully use the available memory of the card.
For the weak scaling study we simply extended the domain to 7200 m× 3600 m, 7200 m× 7200 m, and 14 400 m× 7200 m for 2, 4, and 8 GPUs, respectively. The reference wall time Twall,ref for all simulations is the wall time of the single-GPU case.
In the strong scaling study, we partition the same domain into smaller and smaller subdomains. However, if the subdomain becomes too small, the GPU is not saturated any more, and the parallel efficiency decreases. We therefore ran a reference case for each case of the scaling study where the domain had the same size as one subdomain and use the wall time of that case as Twall,ref. By normalizing in this manner, ideal scaling corresponds to a constant ratio of Twall,ref/Twall.
The wall time for each case can be found in Table , and the results of both scaling studies are shown in Fig. .
Results of the strong and weak scaling study from 1 to 8 GPUs.
The weak scaling efficiency of the model is already good. Even at 8 GPUs the efficiency is still at 85 %, indicating that the model is suitable for large-scale computation. We also find that the efficiency does not noticeably decrease from 4 to 8 GPUs. We observe that the model does not scale as well in the strong scaling study. On the one hand, this can be explained simply by the square-cube law; i.e., the volume of the subdomains shrinks faster than the surface. The communication hiding implemented in VirtualFluids becomes less effective for smaller subdomains; see . This problem is exacerbated due to the planar averaging in Eq. (), which also requires inter-GPU communication and becomes less efficient when partitioned along the streamwise direction. Nevertheless, the efficiency is still around 35 %. Comparing these results with , we find that our model has comparable scaling efficiency in the weak scaling study, although we use significantly fewer GPUs.
Unit conversion in LBM and its effect on small forces
The LBM relies on non-dimensionalizing all quantities with grid size Δx, time step size Δt, and reference density ρ0. Furthermore, recall that Δx3Δt=cs=V0Ma, which we have kept constant. Thus, velocities in SI units are scaled as ũ=uCu, where Cu=ΔxΔt=const, and ũ is the velocity in LB units. Forces, however, are scaled as f̃=fCf, with Cf=ρ0ΔxΔt2=ρ03cs2ΔxChap. 7. Therefore, non-dimensionalized forces decrease with grid spacing. In the cumulant LBM, forces are only applied via Eq. () . Therefore, the result of the addition becomes less accurate with increasing resolution, since the contribution of the force can not be represented in finite-precision floating point numbers. This becomes particularly important for small forces, such as the Coriolis force, and when using single-precision floating point numbers. We have tried the Kahan summation algorithm as a remedy but that did not improve the results.
Code and data availability
VirtualFluids is available as open-source code at https://git.rz.tu-bs.de/irmb/VirtualFluids (last access: 15 July 2026). The model described in this paper is published in version 0.3.0 (, 10.5281/zenodo.20681486). The data for creating the plots in Sect. and the corresponding postprocessing scripts are available at https://source.coderefinery.org/wind_energy_uu/stratificationinlbm (last access: 15 July 2026). A permanent record is kept at 10.5281/zenodo.20512678.
Author contributions
HK, HA, and SI conceptualized the project. HK, HA, MG, and MS developed the methodology and implemented the code. HK conducted the simulations and the postprocessing under the guidance of HA and SI. HK prepared the original draft, and all authors reviewed and edited the paper. SI acquired the resources and supervised the project.
Competing interests
The contact author has declared that none of the authors has any competing interests.
Disclaimer
Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. The authors bear the ultimate responsibility for providing appropriate place names. Views expressed in the text are those of the authors and do not necessarily reflect the views of the publisher.
Acknowledgements
HK would like to thank Antonio Salvini, Luca Lanzilao, and Hugo Olivares-Espinosa for their helpful discussions and Lawrence Cheung and Gopal Yalla for their help with the results from the Exawind benchmark. Some of the computations were enabled by resources provided by the National Academic Infrastructure for Supercomputing in Sweden (NAISS), partially funded by the Swedish Research Council through grant agreement no. 2022-06725.
Financial support
The publication of this article was funded by the Swedish Research Council, Forte, Formas, and Vinnova.
Review statement
This paper was edited by Sukanta Basu and reviewed by three anonymous referees.
ReferencesAbkar, M. and Porté-Agel, F.: Influence of Atmospheric Stability on Wind-Turbine Wakes: A Large-Eddy Simulation Study, Phys. Fluids, 27, 035104, 10.1063/1.4913695, 2015.Abkar, M., Bae, H. J., and Moin, P.: Minimum-Dissipation Scalar Transport Model for Large-Eddy Simulation of Turbulent Flows, Phys. Rev. Fluids, 1, 041701, 10.1103/PhysRevFluids.1.041701, 2016.Alihussein, H., Geier, M., and Krafczyk, M.: A Parallel Coupled Lattice Boltzmann-Volume of Fluid Framework for Modeling Porous Media Evolution, Materials, 14, 2510, 10.3390/ma14102510, 2021.Allaerts, D. and Meyers, J.: Boundary-Layer Development and Gravity Waves in Conventionally Neutral Wind Farms, J. Fluid Mech., 814, 95–130, 10.1017/jfm.2017.11, 2017. Arya, S. P.: Introduction to Micrometeorology, 2nd edn., vol. 79 of International Geophysics Series, Academic Press, San Diego, ISBN 0-12-059354-8, 2001.Asmuth, H., Janßen, C. F., Olivares-Espinosa, H., and Ivanell, S.: Wall-Modeled Lattice Boltzmann Large-Eddy Simulation of Neutral Atmospheric Boundary Layers, Phys. Fluids, 33, 105111, 10.1063/5.0065701, 2021.Basu, S., Holtslag, A. A. M., Van De Wiel, B. J. H., Moene, A. F., and Steeneveld, G.-J.: An Inconvenient “Truth” about Using Sensible Heat Flux as a Surface Boundary Condition in Models under Stably Stratified Regimes, Acta Geophys., 56, 88–99, 10.2478/s11600-007-0038-y, 2008.Beare, R. J., Macvean, M. K., Holtslag, A. A. M., Cuxart, J., Esau, I., Golaz, J.-C., Jimenez, M. A., Khairoutdinov, M., Kosovic, B., Lewellen, D., Lund, T. S., Lundquist, J. K., Mccabe, A., Moene, A. F., Noh, Y., Raasch, S., and Sullivan, P.: An Intercomparison of Large-Eddy Simulations of the Stable Boundary Layer, Bound.-Lay. Meteorol., 118, 247–272, 10.1007/s10546-004-2820-6, 2006.Berg, J., Patton, E. G., and Sullivan, P. P.: Large-Eddy Simulation of Conditionally Neutral Boundary Layers: A Mesh Resolution Sensitivity Study, J. Atmos. Sci., 77, 1969–1991, 10.1175/JAS-D-19-0252.1, 2020.Bou-Zeid, E., Meneveau, C., and Parlange, M.: A Scale-Dependent Lagrangian Dynamic Model for Large Eddy Simulation of Complex Turbulent Flows, Phys. Fluids, 17, 025105, 10.1063/1.1839152, 2005.Businger, J. A., Wyngaard, J. C., Izumi, Y., and Bradley, E. F.: Flux-Profile Relationships in the Atmospheric Surface Layer, J. Atmos. Sci., 28, 181–189, 10.1175/1520-0469(1971)028<0181:FPRITA>2.0.CO;2, 1971.Deardorff, J.: A Three-dimensional Numerical Investigation of the Idealized Planetary Boundary Layer, Geophysical Fluid Dynamics, 1, 377–410, 10.1080/03091927009365780, 1970.d'Humières, D.: Generalized Lattice-Boltzmann Equations, in: Rarefied Gas Dynamics: Theory and Simulations, Progress in Astronautics and Aeronautics, American Institute of Aeronautics and Astronautics, 450–458, https://arc.aiaa.org/doi/10.2514/5.9781600866319.0450.0458 (last access: 15 July 2026), 1994.Feng, Y., Miranda-Fuentes, J., Guo, S., Jacob, J., and Sagaut, P.: ProLB: A Lattice Boltzmann Solver of Large-Eddy Simulation for Atmospheric Boundary Layer Flows, J. Adv. Model. Earth Sy., 13, e2020MS002107, 10.1029/2020MS002107, 2021.Gadde, S. N., Stieren, A., and Stevens, R. J. A. M.: Large-Eddy Simulations of Stratified Atmospheric Boundary Layers: Comparison of Different Subgrid Models, Bound.-Lay. Meteorol., 178, 363–382, 10.1007/s10546-020-00570-5, 2021.Gehrke, M. and Rung, T.: Periodic Hill Flow Simulations with a Parameterized Cumulant Lattice Boltzmann Method, Int. J. Numer. Meth. Fl., 94, 1111–1154, 10.1002/fld.5085, 2022.Geier, M., Schönherr, M., Pasquali, A., and Krafczyk, M.: The Cumulant Lattice Boltzmann Equation in Three Dimensions: Theory and Validation, Comput. Math. Appl., 70, 507–547, 10.1016/j.camwa.2015.05.001, 2015.Geier, M., Pasquali, A., and Schönherr, M.: Parametrization of the Cumulant Lattice Boltzmann Method for Fourth Order Accurate Diffusion Part II: Application to Flow around a Sphere at Drag Crisis, J. Comput. Phys., 348, 889–898, 10.1016/j.jcp.2017.07.004, 2017.Geier, M., Lenz, S., Schönherr, M., and Krafczyk, M.: Under-Resolved and Large Eddy Simulations of a Decaying Taylor–Green Vortex with the Cumulant Lattice Boltzmann Method, Theor. Comp. Fluid Dyn., 10.1007/s00162-020-00555-7, 2020.Geier, M., Kutscher, K., Schönherr, M., Wellmann, A., Peters, S., Alihussein, H., Linxweiler, J., and Krafczyk, M.: VirtualFluids – Open Source Parallel LBM Solver, Comput. Phys. Commun., 109810, 10.1016/j.cpc.2025.109810, 2025.Gruszczyński, G. and Łaniewski-Wołłk, Ł.: A Comparative Study of 3D Cumulant and Central Moments Lattice Boltzmann Schemes with Interpolated Boundary Conditions for the Simulation of Thermal Flows in High Prandtl Number Regime, I. J. Heat Mass Tran., 197, 123259, 10.1016/j.ijheatmasstransfer.2022.123259, 2022.Jacob, J., Malaspinas, O., and Sagaut, P.: A New Hybrid Recursive Regularised Bhatnagar–Gross–Krook Collision Model for Lattice Boltzmann Method-Based Large Eddy Simulation, J. Turbul., 19, 1051–1076, 10.1080/14685248.2018.1540879, 2018.Kahan, W.: Pracniques: Further Remarks on Reducing Truncation Errors, Commun. ACM, 8, 40, 10.1145/363707.363723, 1965.Khan, M. A., Allaerts, D., Watson, S. J., and Churchfield, M. J.: Investigating the relationship between simulation parameters and flow variables in simulating atmospheric gravity waves for wind energy applications, Wind Energ. Sci., 10, 1167–1185, 10.5194/wes-10-1167-2025, 2025.King, M.-F., Khan, A., Delbosc, N., Gough, H. L., Halios, C., Barlow, J. F., and Noakes, C. J.: Modelling Urban Airflow and Natural Ventilation Using a GPU-based Lattice-Boltzmann Method, Build. Environ., 125, 273–284, 10.1016/j.buildenv.2017.08.048, 2017.Korb, H.: Plot data for “Large Eddy Simulation of Thermally Stratified Atmospheric Boundary Layers with a Lattice Boltzmann Method”, Zenodo [data set], 10.5281/zenodo.20512678, 2026.Korb, H., Bastin, J., Asmuth, H., and Ivanell, S.: The lattice Boltzmann method for wind farm simulations: a review, Wind Energ. Sci., 11, 983–1012, 10.5194/wes-11-983-2026, 2026.Krüger, T., Kusumaatmaja, H., Kuzmin, A., Shardt, O., Silva, G., and Viggen, E. M.: The Lattice Boltzmann Method: Principles and Practice, Graduate Texts in Physics, 1 edn., Springer International Publishing, http://link.springer.com/10.1007/978-3-319-44649-3 (last access: 15 July 2026), 2017.Kuhn, M. B., Cheung, L., Yalla, G., Henry de Frahan, M. T., and Mohan, P.: Exawind Benchmarks, GitHub, https://github.com/Exawind/exawind-benchmarks/tree/main/amr-wind/atmospheric_boundary_layer/neutral (last access: 15 July 2026), 2025a.Kuhn, M. B., Henry de Frahan, M. T., Mohan, P., Deskos, G., Churchfield, M., Cheung, L., Sharma, A., Almgren, A., Ananthan, S., Brazell, M. J., Martínez-Tossas, L. A., Thedin, R., Rood, J., Sakievich, P., Vijayakumar, G., Zhang, W., and Sprague, M.: AMR-Wind: A Performance-Portable, High-Fidelity Flow Solver for Wind Farm Simulations, Wind Energy, 28, e70010, 10.1002/we.70010, 2025b.Kutscher, K., Schönherr, M., Geier, M., Alihussein, H., Wellmann, A., Korb, H., Sukhman, D., Horneff, N., Yescas Frangos, P., Bastin, J., Lopez Hermida, D., Mohammadi, M. M., and Qin, Z.: VirtualFluids, Zenodo [code], 10.5281/zenodo.20681486, 2026.Lanzilao, L. and Meyers, J.: A Parametric Large-Eddy Simulation Study of Wind-Farm Blockage and Gravity Waves in Conventionally Neutral Boundary Layers, J. Fluid Mech., 979, A54, 10.1017/jfm.2023.1088, 2024.Lenz, S., Schönherr, M., Geier, M., Krafczyk, M., Pasquali, A., Christen, A., and Giometto, M.: Towards Real-Time Simulation of Turbulent Air Flow over a Resolved Urban Canopy Using the Cumulant Lattice Boltzmann Method on a GPGPU, J. Wind Eng. Ind. Aerod., 189, 151–162, 10.1016/j.jweia.2019.03.012, 2019.Leonard, B. P.: A Stable and Accurate Convective Modelling Procedure Based on Quadratic Upstream Interpolation, Comput. Method. Appl. M., 19, 59–98, 10.1016/0045-7825(79)90034-3, 1979.Lilly, D.: The Representation of Small-Scale Turbulence in Numerical Simulation Experiments, University Corporation for Atmospheric Research, 10.5065/D62R3PMM, 1966.Malaspinas, O. and Sagaut, P.: Wall Model for Large-Eddy Simulation Based on the Lattice Boltzmann Method, J. Comput. Phys., 275, 25–40, 10.1016/j.jcp.2014.06.020, 2014.Min, M., Brazell, M., Tomboulides, A., Churchfield, M., Fischer, P., and Sprague, M.: Towards Exascale for Wind Energy Simulations, Int. J. High Perform. C., 38, 337–355, 10.1177/10943420241252511, 2024.Moeng, C.-H.: A Large-Eddy-Simulation Model for the Study of Planetary Boundary-Layer Turbulence, J. Atmos. Sci., 41, 2052–2062, 10.1175/1520-0469(1984)041<2052:ALESMF>2.0.CO;2, 1984.Obrecht, C., Kuznik, F., Tourancheau, B., and Roux, J.-J.: The TheLMA Project: A Thermal Lattice Boltzmann Solver for the GPU, Comput. Fluids, 54, 118–126, 10.1016/j.compfluid.2011.10.011, 2012.Obrecht, C., Kuznik, F., Tourancheau, B., and Roux, J.-J.: Multi-GPU Implementation of the Lattice Boltzmann Method, Comput. Math. Appl., 65, 252–261, 10.1016/j.camwa.2011.02.020, 2013.Obrecht, C., Kuznik, F., Merlier, L., Roux, J.-J., and Tourancheau, B.: Towards Aeraulic Simulations at Urban Scale Using the Lattice Boltzmann Method, Environ. Fluid Mech., 15, 753–770, 10.1007/s10652-014-9381-0, 2015.Onodera, N., Aoki, T., Shimokawabe, T., and Kobayashi, H.: Large-Scale LES Wind Simulation Using Lattice Boltzmann Method for a 10 Km× 10 Km Area in Metropolitan Tokyo, TSUBAME ESJ, 9, 2–8, 2013.Onodera, N., Idomura, Y., Hasegawa, Y., Nakayama, H., Shimokawabe, T., and Aoki, T.: Real-Time Tracer Dispersion Simulations in Oklahoma City Using the Locally Mesh-Refined Lattice Boltzmann Method, Bound.-Lay. Meteorol., 10.1007/s10546-020-00594-x, 2021.Platis, A., Hundhausen, M., Lampert, A., Emeis, S., and Bange, J.: The Role of Atmospheric Stability and Turbulence in Offshore Wind-Farm Wakes in the German Bight, Bound.-Lay. Meteorol., 182, 441–469, 10.1007/s10546-021-00668-4, 2022.Rozema, W., Bae, H. J., Moin, P., and Verstappen, R.: Minimum-Dissipation Models for Large-Eddy Simulation, Phys. Fluids, 27, 085107, 10.1063/1.4928700, 2015.Sauer, J. A. and Muñoz-Esparza, D.: The FastEddy® Resident-GPU Accelerated Large-Eddy Simulation Framework: Model Formulation, Dynamical-Core Validation and Performance Benchmarks, J. Adv. Model. Earth Sy., 12, e2020MS002100, 10.1029/2020MS002100, 2020.Smagorinsky, J.: GENERAL CIRCULATION EXPERIMENTS WITH THE PRIMITIVE EQUATIONS: I. THE BASIC EXPERIMENT, Mon. Weather Rev., 91, 99–164, 10.1175/1520-0493(1963)091<0099:GCEWTP>2.3.CO;2, 1963.Stoll, R., Gibbs, J. A., Salesky, S. T., Anderson, W., and Calaf, M.: Large-Eddy Simulation of the Atmospheric Boundary Layer, Bound.-Lay. Meteorol., 177, 541–581, 10.1007/s10546-020-00556-3, 2020.van Heerwaarden, C. C., van Stratum, B. J. H., Heus, T., Gibbs, J. A., Fedorovich, E., and Mellado, J. P.: MicroHH 1.0: a computational fluid dynamics code for direct numerical simulation and large-eddy simulation of atmospheric boundary layer flows, Geosci. Model Dev., 10, 3145–3165, 10.5194/gmd-10-3145-2017, 2017. Van Leer, B.: Towards the Ultimate Conservative Difference Scheme. IV. A New Approach to Numerical Convection, J. Comput. Phys., 23, 276–299, 10.1016/0021-9991(77)90095-X, 1977.Verstappen, R.: When Does Eddy Viscosity Damp Subfilter Scales Sufficiently?, J. Sci. Comput., 49, 94, 10.1007/s10915-011-9504-4, 2011.Wang, Y., MacCall, B. T., Hocut, C. M., Zeng, X., and Fernando, H. J. S.: Simulation of Stratified Flows over a Ridge Using a Lattice Boltzmann Model, Environ. Fluid Mech., 20, 1333–1355, 10.1007/s10652-018-9599-3, 2020.Yang, X., Mehmani, Y., Perkins, W. A., Pasquali, A., Schönherr, M., Kim, K., Perego, M., Parks, M. L., Trask, N., Balhoff, M. T., Richmond, M. C., Geier, M., Krafczyk, M., Luo, L.-S., Tartakovsky, A. M., and Scheibe, T. D.: Intercomparison of 3D Pore-Scale Flow and Solute Transport Simulation Methods, Adv. Water Resour., 95, 176–189, 10.1016/j.advwatres.2015.09.015, 2016.Yang, X. I. A., Park, G. I., and Moin, P.: Log-Layer Mismatch and Modeling of the Fluctuating Wall Stress in Wall-Modeled Large-Eddy Simulations, Phys. Rev. Fluids, 2, 104601, 10.1103/PhysRevFluids.2.104601, 2017.