Neural Networks for PDEs: Neural Control of Finite Element Methods
In recent years there has been tremendous interest in the merging of machine-learning algorithms with traditional methods in computational science and mathematics. In this talk, I will focus on the use of neural networks for partial differential equations (PDEs). I will start by highlighting the three general paradigms: Learning the PDE solution, learning the PDE model, and learning the discretization method. All three aim to minimize a certain PDE-related cost functional over sets of neural-network functions. For their numerical analysis, I will present an abstract theory [1] that establishes so-called quasi-optimally convergent quasi-minimizing neural networks.
Next, I will consider the learning of PDE discretizations. In particular, I will present a framework for the neural optimization of discrete weak formulations, suitable for finite element methods [1,2,3]. The main idea of the framework is to include a neural-network function acting as a control variable (a weight function) within the weak form. Finding the neural control that (quasi-) minimizes a suitable cost functional then yields a numerical approximation with desirable attributes. The framework allows in a natural way the incorporation of known data of the exact solution, or the incorporation of stabilization mechanisms (e.g., to remove spurious oscillations). Using the abstract theory, we establish that, under certain conditions, the neural-weighted discrete weak forms are stable, and quasi-minimizing neural controls exist, which converge quasi-optimally. Elementary numerical experiments are presented for neural-weighted Least Squares FEM and neural-weighted MinRes FEM [4], which support our findings and demonstrate the potential of the framework.
[1] Brevis, Muga, Van der Zee, Neural Control of Discrete Weak Formulations: Galerkin, Least-Squares and Minimal-Residual Methods with Quasi-Optimal Weights, CMAME (2022) [2] Brevis, Muga, Pardo, Rodriguez, Van der Zee, Learning quantities of interest from parametric PDEs: An efficient neural-weighted Minimal Residual approach, CAMWA (2023) [3] Brevis, Muga, Van der Zee, A machine-learning minimal-residual (ML-MRes) framework for goal-oriented finite element discretizations, CAMWA (2021) [4] Muga, Van der Zee, Discretization of Linear Problems in Banach Spaces: Residual Minimization, Nonlinear Petrov- Galerkin, and Monotone Mixed Methods, SINUM (2020)