Special Session 107: Recent Advances in Data Assimilation with Machine Learning

An Efficient Online Smoother and Sampling Algorithm for Partially Observed Nonlinear Dynamical Systems

Marios Andreou University of Wisconsin-Madison USA Co-Author(s):    Nan Chen, Yingda Li

Abstract: State estimation is a critical task in practice as it is the prerequisite for effective parameter estimation (PE), skillful prediction, optimal control, and the development of robust and complete datasets. Of essence is the probabilistic state estimation of the unobserved variables in high-dimensional complex nonlinear turbulent dynamical systems with intermittent instability, given a partially observed time series. Data assimilation (DA) through Bayesian inference combines the partial observations and the model-induced likelihood, potentially plagued by a small error in model structure or initial condition, to form the posterior distribution for optimal state estimation. In this work, an efficient algorithm with closed-form explicit expressions is developed for the optimal (in the mean-squared sense) online smoother and sampling of a rich class of nonlinear turbulent dynamical systems widely appearing in modeling natural phenomena like physics-constrained stochastic models (noisy Lorenz models, low-order models of Charney-DeVore flows, paradigm models for topographic mean flow interaction), stochastically coupled reaction-diffusion models in neuroscience and ecology (stochastically coupled FitzHugh-Nagumo models, stochastically coupled SIR epidemic models), multiscale models for geophysical flows (noisy Boussinesq equations, stochastically forced rotating shallow water equation), and to develop realistic systems for the Madden-Julian oscillation and Arctic sea ice. The conditional Gaussian nonlinear system framework is highly flexible since it encompasses the linear Kalman filter and the Rauch-Tung-Striebel smoother and facilitates studying extreme events, stochastic parameterisation, DA, and online PE. The method particularly handles systems with hidden intermittency and extreme events along with the associated highly non-Gaussian features, such as heavy tails in the probability density functions, by using a computationally effective and systematic method to determine the adaptive lag for the online procedure. A rigorous analysis of the performance of the framework will be illustrated through numerical simulations of high-dimensional Lagrangian DA and online model identification of highly intermittent time series via an expectation-maximization procedure.

A mechanism learning based method for data filling of physical fields

Yu Chen Shanghai University of Finance and Economics Peoples Rep of China Co-Author(s):    Yu Chen, Jin Cheng, Xinyue Luo

Abstract: This talk is concerned with an data filling method based on mechanism learning, from the perspective of inverse problems. The underlying data mechanism, characterized by temporal-spatial or cross-sectional linear differential equations, is identified from observations on the known area and then exploited to infer that on the missing part, which improves interpretability and generalizability. Attention is paid to incorporation of prior information as higher order mechanism. Method analysis and numerical examples demonstrate effectiveness, robustness and flexibility of the method and it performs well over mechanism/scientific data, such as oceanic and atmospheric fields.

Data-driven model selections of interacting particle dynamics via Gaussian processes with uncertainty quantification

Jinchao Feng Great Bay University Peoples Rep of China Co-Author(s):    Charles Kulick, Yunxiang Ren, and Sui Tang

Abstract: In this talk, I will introduce a data-driven method to identify a comprehensive second-order particle-based model, which integrates numerous advanced models for simulating aggregation and collective behaviors of agents with comparable sizes and shapes. The model is represented as a high-dimensional set of ordinary differential equations, parameterized by dual interaction kernels that evaluate the coordination of positions and velocities. We propose a Gaussian Process (GP)-based methodology for estimating the model parameters, employing two separate GP priors for the latent interaction kernels, which are calibrated against both dynamical and observational data. This approach yields a nonparametric model for interacting dynamical systems, incorporating uncertainty quantification. Additionally, we provide a theoretical analysis to elucidate our method and assess conditions necessary for kernel recovery. The efficacy of our approach is validated through applications to various prototype systems, emphasizing system order and interaction type selection.

A unified Bayesian inversion approach for a class of tumor growth models with different pressure laws

Yu Feng Great Bay University Peoples Rep of China Co-Author(s):    Liu Liu, Zhennan Zhou

Abstract: We use the Bayesian inversion approach to study the data assimilation problem for a family of tumor growth models described by porous-medium type equations. The models contain uncertain parameters and are indexed by a physical parameter $m$, which characterizes the constitutive relation between density and pressure. Based on these models, we employ the Bayesian inversion framework to infer parametric and nonparametric unknowns that affect tumor growth from noisy observations of tumor cell density. We establish the well-posedness and the stability theories for the Bayesian inversion problem and further prove the convergence of the posterior distribution in the so-called incompressible limit, $m \rightarrow \infty$. Since the posterior distribution across the index regime $m\in[2,\infty)$ can thus be treated in a unified manner, such theoretical results also guide the design of the numerical inference for the unknown. We propose a generic computational framework for such inverse problems, which consists of a typical sampling algorithm and an asymptotic preserving solver for the forward problem. With extensive numerical tests, we demonstrate that the proposed method achieves satisfactory accuracy in the Bayesian inference of the tumor growth models, which is uniform with respect to the constitutive relation.

Efficient Derivative-Free Bayesian Inference for Large-Scale Inverse Problems

Daniel Zhengyu Huang Peking University Peoples Rep of China Co-Author(s):

Abstract: We consider Bayesian inference for large-scale inverse problems, where computational challenges arise from the need for repeated evaluations of an expensive forward model, which is often given as a black box or is impractical to differentiate. We propose a framework, which is built on Kalman methodology and Fisher-Rao Gradient flow, to efficiently calibrate and provide uncertainty estimations of such models with noisy observation data.

A random reconstruction method in optical tomography

Yiwen Lin Shanghai Jiao Tong University Peoples Rep of China Co-Author(s):    Min Tang, Li Wang

Abstract: In this work, we address the inverse problem of radiative transfer equation (RTE) using the Random Ordinate Method (ROM) for the forward problem and the Stochastic Gradient Descent (SGD) based method for the inverse problem, aimed at efficiently reconstructing the absorption and scattering coefficients by minimizing the mismatch between computed and measured outgoing data. Since the Discrete Ordinates Method (DOM) for solving RTE is computationally intensive and suffers from the ray effect, we utilize ROM to mitigate the ray effect due to the low regularity of the solution in the velocity direction. ROM offers several advantages over DOM, including comparable computational costs, minimal changes to existing code, and ease of parallelization. The SGD algorithm requires low memory and computation, and advances fast. Numerical examples demonstrate the accuracy and efficiency of the proposed method

An Asymptotic-Preserving Neural Network approach for the Boltzmann equation with uncertainties

Liu Liu Chinese University of Hong Kong Peoples Rep of China Co-Author(s):    Zhenyi Zhu, Xueyu Zhu

Abstract: In this talk, we develop the Asymptotic-Preserving Neural Networks (APNNs) approach to study the forward and inverse problem for the semiconductor Boltzmann equation. The goal of the neural network is to resolve the computational challenges of conventional numerical methods and multiple scales of the model. In a micro-macro decomposition framework, we design such an AP formulation of loss function. The convergence analysis of both the loss function and its neural network is shown, based on the hypocoercivity theory of the model equation. Our analysis also suits for the general collisional kinetic equation including the full Boltzmann. We will show a series of numerical tests for forward and inverse problems, also extend to uncertainty quantification problems, to demonstrate the efficiency and robustness of our approach.

Inverse Transfer and Coherence in Rotating Stratified Turbulence with Clouds and Phase Transitions

Yingshuo Peng Shanghai University of Finance and Economics Peoples Rep of China Co-Author(s):    Yeyu Zhang, Leslie M. Smith

Abstract: Inverse energy transfer to large-scale coherent structures in idealized geophysical flow models has been a key research focus for over four decades. Extensive knowledge exists regarding inverse transfer in rotating and stratified dry dynamics, characterized by the Rossby number and a single dry Froude number. This study includes water, phase changes, and rainfall in the dynamics, characterized by the Rossby number, two Froude numbers for unsaturated and saturated environments, and a rainfall speed. Using numerical computations with random forcing, inverse transfer of energy is examined in Boussinesq model, incorporating water vapor and liquid water in the limit of asymptotically-fast cloud microphysics. Total energy includes kinetic energy, buoyant potential energies from each phase, and latent moist energy at phase boundaries. The rotation and stratification terms are comparable, such that the evolution of evolution dry equations is dominated by inverse transfer of pseudo potential vorticity. Phase changes and latent heat exchange raise the potential-to-kinetic energy ratio and reduce overall energy transfer to large scales. However, upscale transfer persists, influenced by nonlinear waves near phase interfaces, resulting in coherent updrafts and downdrafts aligned with large-scale phase boundaries. The relationship between coherent updrafts/downdrafts and moist pseudo potential vorticity is also examined.

Reduced-Order Models for Data Assimilation of Multiscale Turbulent Systems

Di Qi Purdue University USA Co-Author(s):    Jian-Guo Liu

Abstract: The capability of using imperfect stochastic and statistical reduced-order models to capture key statistical features in multiscale nonlinear dynamical systems is investigated. A new efficient ensemble forecast algorithm is developed dealing with the nonlinear multiscale coupling mechanism as a characteristic feature in high-dimensional turbulent systems. To address challenges associated with closely coupled spatio-temporal scales in turbulent states and expensive large ensemble simulation for high-dimensional complex systems, we introduce efficient computational strategies using the so-called random batch method. It is demonstrated that crucial principal statistical quantities in the most important large scales can be captured efficiently with accuracy using the new reduced-order model in various dynamical regimes of the flow field with distinct statistical structures. Finally, the proposed model is applied for a wide range of problems in uncertainty quantification, data assimilation, and control.

Error estimates of two invariant-preserving difference schemes for the rotation-two-component Camassa--Holm system

Qifeng Zhang Zhejiang Sci-Tech University Peoples Rep of China Co-Author(s):    Zhimin Zhang, Zhi-Zhong Sun, Dinghua Xu

Abstract: In this talk, we develop, analyze and numerically test two classes of invariant-preserving difference schemes for a rotation-two-component Camassa-Holm system (R2CH), which contains strongly nonlinear terms and high-order derivative terms. One of them is linearized one and another one is fully nonlinear. We prove that both the numerical schemes are uniquely solvable and second-order convergent for the spatial and temporal discretizations. Optimal error estimates for the velocity in the infinite norm and for the surface elevation in the L2-norm are obtained. Extensive numerical experiments verify the convergence results as well as conservation.

An Efficient Multiscale Stochastic Reduced-Order Model and Nonlinear Filtering Scheme for Two-Dimensional Stratified Turbulence

Yeyu Zhang Shanghai University of Finance and Economics Peoples Rep of China Co-Author(s):    Nan Chen, Changhong Mou

Abstract: This work aims to design a systematic multiscale stochastic Reduced-Order Model (ROM) framework for complex systems such as Boussinesq systems, which exhibit chaotic or turbulent characteristics. Based on this framework, an efficient general multiscale stochastic data assimilation scheme is developed to provide accurate tools for inversion and prediction. The primary focus of ROM design is to restore large-scale dynamics as accurately as possible. A unique feature of the generated ROM is that it facilitates the construction of an efficient and accurate nonlinear data assimilation scheme, which provides solutions through closed-form analytical expressions. This analytically solvable data assimilation scheme significantly accelerates the computation of recovering unobserved states from partial observations (e.g., efficiently updating the posterior distribution of unobserved variables, such as temperature, based on measurable data like wind fields and vice versa) through closed-form solutions, thereby avoiding many potential numerical sampling issues. This work also presents the performance of various ROMs with different model errors and different data assimilation schemes in recovering unobserved variables in complex systems. While understanding model errors, it also analyzes how to balance the filter`s reliance on the model versus observations in different physical dynamical regimes, which exhibit distinct flow characteristics (i.e., more stable layered flow or more turbulent flow).