Special Session 126: Machine Learning and New Framework for Solving Partial Differential Equations

Learning coarse-grained models and quantifying transitions between metastable states in molecules and clusters of interacting particles

Maria K Cameron University of Maryland, College Park USA Co-Author(s):    Maria Cameron, Shashank Sule, Jiaxin Yuan

Abstract: Many processes in nature such as conformal changes in biomolecules and clusters of interacting particles are modeled using stochastic differential equations with small noise. The study of rare transitions between metastable states in such systems is of great interest and importance. The direct simulation of rare transitions is difficult due to long waiting times and high dimensionality. Transition Path Theory (E and Vanden-Eijnden, 2006) is a mathematical framework for describing transition processes. The key component of its implementation is the numerical solution of the committor problem, a certain boundary value problem for the stationary Backward Kolmogorov equation. In this talk, I will discuss how one can learn coarse-grained models, solve the committor problem accurately in moderately high dimensions, and use optimal stochastic control to quantify transition processes between the metastable states.

A novel shape optimization approach for source identification in elliptic equations

Wei Gong Academy of Mathematics and Systems Science, Chinese Academy of Sciences Peoples Rep of China Co-Author(s):    Wei Gong and Ziyi Zhang

Abstract: In this talk, we propose a novel shape optimization approach for the source identification of elliptic equations. This identification problem arises from two application backgrounds: actuator placement in PDE-constrained optimal controls and the regularized least-squares formulation of source identifications. The optimization problem seeks both the source strength and its support. By eliminating the variable associated with the source strength, we reduce the problem to a shape optimization problem for a coupled elliptic system, known as the first-order optimality system. As a model problem, we derive the shape derivative for the regularized least-squares formulation of the inverse source problem and propose a gradient descent shape optimization algorithm, implemented using the level-set method. Several numerical experiments are presented to demonstrate the efficiency of our proposed algorithms.

A new method using C0IPG for the biharmonic eigenvalue problem

Xia Ji Beijing institute of technology Peoples Rep of China Co-Author(s):    yongxiang xi, jiguang sun

Abstract: The talk presents a new proof of the $C^0$IPG method ($C^0$ interior penalty Galerkin method) for the biharmonic eigenvalue problem. Instead of using the proof following the structure of discontinuous Galerkin method, we rewrite the problem as the eigenvalue problem of a holomorphic Fredholm operator function of index zero. The convergence for $C^0$IPG is proved using the abstract approximation theory for holomorphic operator functions. We employ the spectral indicator method which is easy in coding to compute the eigenvalues. Numerical examples are presented to validate the theory.

Solving Reaction Diffusion Equation Using Transformer-based Koopman Autoencoder

Nitu Kumari Indian Institute of Technology Mandi India Co-Author(s):

Abstract: A Transformer based Koopman autoencoder is proposed for linearizing reaction diffusion equation. The primary focus of this study is on using deep learning techniques to find complex spatiotemporal patterns in the reaction diffusion system. The emphasis is not just solving the equation but also transforming the system dynamics into a more comprehensible, linear form. Global coordinate transformations are achieved through the autoencoder, which learns to capture the underlying dynamics by training on a dataset with 60,000 initial conditions. Extensive testing on multiple datasets was used to assess the efficacy of the proposed model, demonstrating its ability to accurately predict the system evolution as well as to generalize. We provide a thorough comparison study, comparing our suggested design to a few other comparable methods using experiments on various PDEs. Results show improved accuracy, highlighting the capabilities of the Transformer based Koopman autoencoder. The proposed architecture is significantly ahead of other architectures, in terms of solving different types of PDEs using a single architecture. Our method relies entirely on the data, without requiring any knowledge of the underlying equations. This makes it applicable to even the datasets where the governing equations are not known.

Ball Mass-preserving Parameterizations with Applications on Brain Tumor Segmentations

Tiexiang Li Southeast University Peoples Rep of China Co-Author(s):

Abstract: A parameterization of a given manifold refers to a bijective map which transforms the manifold to a canonical domain. In this talk, we introduce an optimal mass transport (OMT) algorithm for achieving mass-preserving parameterizations, which transforms a 3-manifold into a unit ball. The OMT theory inherently guarantees the mass-preservation of the map. The accuracy and efficiency of the proposed OMT algorithm are shown in the numerical experiments. We leverage the OMT algorithm in the context of brain tumor segmentations. The integrated UNet combined with OMT demonstrates notable performance in the Brain Tumor Segmentation (BraTS) Challenge.

Reduced Krylov Basis Methods

Yuwen Li Zhejiang University Peoples Rep of China Co-Author(s):    Yuwen Li

Abstract: The reduced basis method is popular for numerically solving a family of parametrized PDEs. In this talk, I will present our new reduced basis algorithm based on preconditioned Krylov subspace methods. The proposed methods use a preconditioned Krylov subspace method for a high-fidelity discretization of one parameter instance to generate orthogonal basis vectors of the reduced basis subspace. Then the family of large-scale discrete parameter-dependent problems are approximately solved in the low-dimensional Krylov subspace. The material in my talk is based on joint works with Ludmil Zikatanov and Cheng Zuo.

Bayesian Compressive Sensing and Medical Imaging Using Generative Models

Jiulong Liu Academy of Mathematics and Systems Science,CAS Peoples Rep of China Co-Author(s):

Abstract: Bayesian statistical inversion and sparsity are successful mathematical tools for reducing the sampling rate in compressive sensing reconstruction, extending their applications to many underdetermined imaging systems, such as MRI and CT. However, with the advent of deep learning, numerous methods have emerged that learn data representations, proving more efficient in signal and image processing. To efficiently and stably solve under-determined and ill-conditioned inverse problems with fewer measurements, we developed compressive sensing and Bayesian compressive sensing reconstruction methods using generative priors. These approaches have demonstrated superior efficiency compared to traditional regularizations or other data-driven regularization techniques. Furthermore, we provide theoretical guarantees for the recovery performance of our proposed methods. In this talk, I will introduce some of these methods and present recent results in MRI reconstruction, phase retrieval, and other nonlinear inverse problems.

Structure-preserving parametric finite element methods for curve diffusion

Chunmei Su Tsinghua University Peoples Rep of China Co-Author(s):    Harald Garcke, Wei Jiang, Ganghui Zhang

Abstract: We propose a novel formulation for parametric finite element methods to simulate surface diffusion of closed curves, which is also called as the curve diffusion. Several high-order temporal discretizations are proposed based on this new formulation. To ensure that the numerical methods preserve geometric structures of curve diffusion (i.e., the perimeter-decreasing and area-preserving properties), our formulation incorporates two scalar Lagrange multipliers and two evolution equations involving the perimeter and area, respectively. By discretizing the spatial variable using piecewise linear finite elements and the temporal variable using either the Crank-Nicolson method or the backward differentiation formulae method, we develop high-order temporal schemes that efectively preserve the structure at a fully discrete level. These new schemes are implicit and can be efficiently solved using Newtons method. Extensive numerical experiments demonstrate that our methods achieve the desired temporal accuracy, as measured by the manifold distance, while simultaneously preserving the geometric structure of the curve diffusion.

Dual-robust iterative analysis of divergence-conforming IPDG FEM for thermally coupled inductionless MHD system

Haiyan Su Xinjiang University Peoples Rep of China Co-Author(s):    Guodong Zhang

Abstract: This talk presents dual-robust iterative algorithms for the 2D/3D steady thermally coupled inductionless magnetohydrodynamics system in a general Lipschitz domain. Both velocity and current density are discretized by the divergence-conforming elements. Furthermore, we utilize an interior penalty discontinuous Galerkin (IPDG) approach to guarantee the H^1-continuity of velocity. With the system strong nonlinearity, we propose three iterative algorithms (Stokes, Newton and Oseen iterations) and provide analytical proofs for their stability and convergence. And the feature of these methods is that simultaneously ensures the complete divergence-free of discrete velocity and discrete current density. Finally, the numerical simulations verify theoretical analysis and the effectiveness of proposed algorithms.

Deep Neural Networks with Rectified Power Units: Efficient Training and Applications in Partial Differential Equations

Haijun Yu Academy of Mathematics and Systems Science, Chinese Academy of Science Peoples Rep of China Co-Author(s):    Haijun Yu

Abstract: Deep neural networks (DNN) equipped with rectified power units (RePU) have demonstrated superior approximation capabilities compared to those utilizing rectified linear units (ReLU), as highlighted by B. Li, S. Tang, and H. Yu [Commun. Comput. Phys., 2020, 27(2): 379-411]. These units are particularly advantageous for machine learning tasks requiring high-order continuity in the functions represented by neural networks. Despite their theoretical benefits, however, the practical application of RePU DNNs has been limited due to challenges such as gradient vanishing and explosion during training. In this talk, we explore various strategies aimed at facilitating the training of RePU DNNs. Our primary focus lies on the numerical solutions of partial differential equations. We demonstrate that, with appropriate training techniques, RePU DNNs can achieve better results than standard DNNs employing other commonly used activation functions, and do so with a faster training rate.

A deep learning enabled massive parallel simulator for porous media flow

Chensong Zhang Academy of Mathematics and Systems Science Peoples Rep of China Co-Author(s):

Abstract: Due to the complex composition of oil and gas resources, reservoir engineers usually switch between different mathematical models when describing the properties of petroleum reservoirs. In addition to the commonly used black oil model, various compositional models have been proposed. Some EOR techniques, such as polymer flooding, must be simulated based on the framework of compositional models. Some other applications of porous media flow, such as CO2 sequestration, groundwater contamination, and geothermal resource development, can also be simulated using compositional models. But the compositional models tend to be associated with more complex PDEs, more variables, and higher computational costs. In this talk, we will discuss a general-purpose compositional framework and our efforts in developing its solution methods, including discretizations, nonlinear solvers, linear solvers, parallelization and AI capabilities. Furthermore, we will introduce an open-source software project for simulating multi-component multi-phase porous media flow.

Complex dualities and new solution frameworks

Shuo Zhang Academy of Mathematics and Systems Science, Chinese Academy of Sciences Peoples Rep of China Co-Author(s):

Abstract: Once a series of operators with domain spaces formulate a complex, their respective adjoint operators formulate a complex simultaneously; such pairs of complexes are called complex dualities. New solution framework can be stimulated, with respect to both classical numerical methodologies and machine learning type methodologies, by revealing the structure of complex dualities at continuous or discrete levels. Novel neural network methods and finite element methods for certain model problems are presented for example.

Weak Generative Sampler to Solve High - Dimensional PDEs for Stochastic Models: Efficiency and Adaptivity

Xiang ZHOU City University of Hong Kong Hong Kong Co-Author(s):    Zhiqiang CAI, Yu CAO, Yuanfei Huang

Abstract: The solution of many typical high-dimensional PDEs (such as the Fokker-Planck, and McKean-Vlasov equations) is associated with a probability distribution. To solve such PDEs by deep learning techniques is usually to simply find a neural network for the density function itself, subject to certain positivity and normalization conditions. The further utilization of the solution requires random sampling again. We introduce a framework of Weak Generative Sampler (WGS) to both solve the PDE and generate samples more efficiently than the PINN and the Ritz method. Our proposed loss function is based on the weak form and the generic probability interpretation of the loss function. The details of this talk will explain why the efficiency and adaptivity are so easy to achieve in this WGS for high-dimensional PDEs.