Special Session 49: Stochastic Control, Filtering and Related Fields

Partially observed mean-field game and related mean-field forward-backward stochastic differential equation

Kai Du Shandong University Peoples Rep of China Co-Author(s):

Abstract: In this talk, we study a linear-convex mean-field game with input constraints for partially observed forward-backward system, where both types of mean-field terms, asynchronous style (state-averages) and synchronous style (state expectations), are considered. The observation is a controlled process, whose drift term is linear with respect to state and control variable. For the general case, by using the mean-field method and the backward separation approach, we obtain the decentralized optimal strategies through a Hamiltonian system and related Consistency Condition (CC), which are given by two types of mean-field forward-backward stochastic differential equations with filtering. In virtue of continuation method and discounting method, the well-posedness of such kind of equations is proved under two different conditions. For the linear-quadratic case under linear subspace constraints, we give the feedback representation of the decentralized optimal strategies, and the Riccati type CC system is also given. As one application, an asset-liability management problem is solved.

Policy Iteration Reinforcement Learning Method for Continuous-time Linear-Quadratic Mean-Field Control Problem

Na Li School of Statistics and Mathematics, Shandong University of Finance and Economics Peoples Rep of China Co-Author(s):

Abstract: This paper employs a policy iteration reinforcement learning (RL) method to study continuous-time linear quadratic mean-field control problems in the infinite horizon. The drift and diffusion terms in the dynamics involve the state as well as the control. We investigate the stability and convergence of the RL algorithm using a Lyapunov Recursion. Instead of solving a pair of coupled Riccati equations, the RL technique focuses on strengthening an auxiliary function and the cost functional as the objective functions and updating the new policy to compute the optimal control via state trajectories. A numerical example sheds light on the established theoretical results.

Indefinite linear-quadratic large population problem with partial observation

Tianyang Nie Shandong University Peoples Rep of China Co-Author(s):

Abstract: We investigate an indefinite linear-quadratic partially observed large population system with common noise, where both the state-average and control-average are considered. All weighting matrices in the cost functional can be indefinite. We obtain the decentralized optimal strategies by the Hamiltonian approach and demonstrate the well-posedness of Hamiltonian system by virtue of relaxed compensator. The related Consistency Condition and the feedback form of decentralized optimal strategies are derived. Moreover, we prove that the decentralized optimal strategies are $\varepsilon$-Nash equilibrium by using the relaxed compensator. The talk is based on the joint work with Dr, Tian Chen and Prof. Zhen Wu.

Sequential Markov Chain Monte Carlo for Filtering

Hamza Ruzayqat King Abdullah University of Science and Technology Saudi Arabia Co-Author(s):    Alexandros Beskos, Dan Crisan, Ajay Jasra, Nikolas Kantas

Abstract: We consider the problem of high-dimensional filtering/data assimilation of continuous and discrete state-space models at discrete times. This problem is particularly challenging as analytical solutions are usually not available and many numerical approximation methods can have a cost that scales exponentially with the dimension of the hidden state. We utilize a sequential Markov chain Monte Carlo method to obtain samples from an approximation of the filtering distribution. For certain state-space models, this method is proven to converge to the true filter as the number of samples, N, tends to infinity. We benchmark our algorithms on linear Gaussian state-space models against competing ensemble methods and demonstrate a significant improvement in both execution speed and accuracy (the algorithm cost can range from O(Nd) to O(Nd[d+1]/2) based on the model noise covariance matrix structure, where d is the dimension of the hidden state. We then consider a state-space model with Lagrangian observations such that the spatial locations of these observations are unknown and driven by the partially observed hidden signal. This problem is exceptionally challenging as not only is high-dimensional, but the model for the signal yields longer-range time dependencies through the observation locations. Finally, the algorithm is tested on the high-dimensional rotating shallow water model with real data obtained from drifters in the ocean.

A Risk-Sensitive Global Maximum Principle for Controlled Fully Coupled FBSDEs with Applications

Jingtao Shi Shandong University Peoples Rep of China Co-Author(s):    Jingtao Lin

Abstract: This paper is concerned with a kind of risk-sensitive optimal control problem for fully coupled forward-backward stochastic systems. The control variable enters the diffusion term of the state equation and the control domain is not necessarily convex. A new global maximum principle is obtained without assuming that the value function is smooth. The maximum condition, the first- and second-order adjoint equations heavily depend on the risk-sensitive parameter. An optimal control problem with a fully coupled linear forward-backward stochastic system and an exponential-quadratic cost functional is discussed. The optimal feedback control and optimal cost are obtained by using Girsanov theorem and completion-of-squares approach via risk-sensitive Riccati equations. A local solvability result of coupled risksensitive Riccati equations is given by Picard-Lindelof Theorem.

Robust optimal control of Bi-objective LQ system with noisy observation

Guangchen Wang Shandong University Peoples Rep of China Co-Author(s):    Zhuangzhuang Xing

Abstract: This talk is concerned with a kind of partially observable LQ control problem, where the coefficients of cost functional are uncertain representing different market conditions. By virtue of backward separation technique, stochastic maximum principle, as well as filtering method, a feedback form of candidate optimal control is designed. Moreover, through some delicate analysis, the existence of maximal reference probability is certified. Finally, a numerical simulation is presented to authenticate the theoretical results.

Mean-field stochastic linear quadratic control problem with random coefficients

Xu Wen Southern University of Science and Technology Peoples Rep of China Co-Author(s):    Jie Xiong

Abstract: In this talk, we will present our recent studies on mean-field stochastic linear quadratic (MFSLQ) control problems with random coefficients. We discovered that, despite the presence of terms like $\mathbb{E}[A(\cdot)X(\cdot)]$ in the adjoint equation preventing us from decoupling the optimal system, the MFSLQ problem can still be solved explicitly using an extended Lagrange multiplier method. This method decomposes the MFSLQ control problem into two constrained SLQ control problems without mean-field terms. This talk is based on joint work with Professor Jie Xiong.

Extrapolation Methods for Solving Backward Stochastic Differential Equations

Weidong Zhao Shandong University Peoples Rep of China Co-Author(s):    Yafei Xu

Abstract: For the $\theta$-scheme and the Crank-Nicolson scheme for solving backward stochastic differential equations (BSDEs), by using the Adomian decomposition for the nonlinear generator of BSDEs and by introducing a system of new BSDEs, we theoretically obtain their asymptotic expansions. Based on the expansions, we propose extrapolation methods of the two schemes for solving FBSDEs. Our numerical tests verify our theoretical conclusions, and show that the extrapolation algorithms are very efficient and have the capacity of solving complicated physical problems.

On Mean-field super-Brownian motions

Jiayu Zheng Shenzhen MSU-BIT University Peoples Rep of China Co-Author(s):    Yaozhong Hu, Michael A. Kouritzin and Panqiu Xia

Abstract: The mean-field stochastic partial differential equation (SPDE) corresponding to a mean-field super-Brownian motion (sBm) is obtained and studied. In this mean-field sBm, the branching-particle lifetime is allowed to depend upon the probability distribution of the sBm itself, producing an SPDE whose space-time white noise coefficient has, in addition to the typical sBm square root, an extra factor that is a function of the probability law of the density of the mean-field sBm. This novel mean-field SPDE is thus motivated by population models where things like overcrowding and isolation can affect growth. A two step approximation method is employed to show existence for this SPDE under general conditions. Then, mild moment conditions are imposed to get uniqueness. Finally, smoothness of the SPDE solution is established under a further simplifying condition.