Reinforcement learning for wall modeling in large-eddy simulation

This work advances the field of large-eddy simulation (LES) for wall-bounded turbulence through four key contributions. We begin by conducting direct numerical simulations (DNS) of rotating turbulent pipe flow at friction Reynolds numbers up to $\Rey_\tau \approx 3000$, investigating the physical mechanisms underlying drag reduction induced by steady axial rotation. Next, we present CaLES, a graphics processing unit (GPU) accelerated LES solver featuring advanced subgrid-scale (SGS) models and numerical schemes, with validation demonstrated on decaying isotropic turbulence, turbulent channel flow, and turbulent duct flow. We then develop SmartFlow, a nearly solver-agnostic deep reinforcement learning (DRL) framework for computational fluid dynamics (CFD) deployable on high-performance computing (HPC) systems, showcasing its versatility through single-agent synthetic-jet control for cylinder flows, multi-agent wake control, and multi-task multi-agent wall-model training. Finally, we employ multi-task multi-agent DRL to construct wall models for LES of wall-bounded turbulence, focusing on model robustness and generalizability across flow conditions. DNS of rotating pipe flows up to $\Rey_\tau \approx 3000$ are carried out to investigate drag reduction effects associated with axial rotation, extending previous studies carried out at a modest Reynolds number~\citep{Orlandi1997a, Orlandi2000}. The results show that the drag reduction, which we theoretically show to be equivalent to net power saving assuming no mechanical losses, monotonically increases as either the Reynolds number or the rotation number increases, proportionally to the inner-scaled rotational speed. Net drag reduction up to about $70\%$ is observed, while being far from flow relaminarization. Scaling laws for the mean axial and azimuthal velocity are proposed, from which a predictive formula for the friction factor is derived. The formula can correctly represent the dependency of the friction factor on the Reynolds and rotation numbers, maintaining good accuracy for low-to-moderate rotation numbers. Examination of the turbulent structures highlights the role of rotation in widening and elongating the small-scale streaks, with subsequent suppression of sweeps and ejections. In the core part of the flow, clear weakening of large-scale turbulent motions is observed at high Reynolds numbers, with subsequent suppression of the outer-layer peak in the pre-multiplied spectra. The Fukagata-Iwamoto-Kasagi decomposition indicates that, consistent with a theoretically derived formula, the outer layer yields the largest contribution to drag reduction at increasingly high Reynolds numbers. In contrast, both the inner and the outer layers contribute to drag reduction as the rotation number increases. We introduce CaLES, a GPU-accelerated finite-difference solver designed for LES of incompressible wall-bounded flows in massively parallel environments. Built upon the existing DNS solver CaNS, CaLES relies on low-storage, third-order Runge-Kutta schemes for temporal discretization, with the option to treat viscous terms via an implicit Crank-Nicolson scheme in one or three directions. A fast direct solver, based on eigenfunction expansions, is used to solve the discretized Poisson/Helmholtz equations. For turbulence modeling, the classical Smagorinsky model with van Driest near-wall damping and the dynamic Smagorinsky model are implemented, along with a logarithmic law wall model. GPU acceleration is achieved through OpenACC directives, following CaNS-2.3.0. Performance assessments were conducted on the Leonardo cluster at CINECA, Italy. Each node is equipped with one Intel Xeon Platinum 8358 central processing unit (CPU) (2.60 GHz, 32 cores) and four NVIDIA A100 GPUs (64 GB HBM2e), interconnected via NVLink 3.0 (200 GB/s). The inter-node communication bandwidth is 25 GB/s, supported by a DragonFly+ network architecture with NVIDIA Mellanox InfiniBand HDR. Results indicate that the computational speed on a single GPU is equivalent to approximately 15 CPU nodes, depending on the treatment of viscous terms and the SGS model, and that the solver efficiently scales across multiple GPUs. The predictive capability of CaLES has been tested using multiple flow cases, including decaying isotropic turbulence, turbulent channel flow, and turbulent duct flow. The high computational efficiency of the solver enables grid convergence studies on extremely fine grids, pinpointing non-monotonic grid convergence for wall-modeled LES. DRL is emerging as a powerful tool for fluid-dynamics research, encompassing active flow control, autonomous navigation, turbulence modeling and discovery of novel numerical schemes. We introduce SmartFlow, a CFD-solver-agnostic framework for both single- and multi-agent DRL algorithms that can easily integrate with message passing interface (MPI) parallel CPU and GPU-accelerated solvers. Built on Relexi and SmartSOD2D, SmartFlow uses the SmartSim infrastructure library and our newly developed SmartRedis-MPI library to enable asynchronous, low-latency, in-memory communication between CFD solvers and Python-based DRL algorithms. SmartFlow leverages PyTorch's Stable-Baselines3 for training, which provides a modular, Gym-like environment API. We demonstrate its versatility via three case studies: single-agent synthetic-jet control for drag reduction in a cylinder flow simulated by the high-order FLEXI solver, multi-agent cylinder wake control using the GPU-accelerated spectral-element code SOD2D, and multi-agent wall-model learning for LES with the finite-difference solver CaLES. SmartFlow's CFD-solver-agnostic design and seamless HPC integration are promising to accelerate reinforcement learning (RL) driven fluid-mechanics studies. Finally, we develop wall models for LES of wall-bounded turbulence using multi-task multi-agent DRL. The wall models are trained within the SmartFlow framework coupled with the GPU-accelerated CaLES solver, utilizing turbulent channel flows across multiple Reynolds numbers to enhance generalizability. Model robustness is evaluated through comprehensive testing at various Reynolds numbers and wall-model matching heights, demonstrating the potential of RL approaches for developing robust wall models.