| Abstract: |
| We introduce the Mean-Field Actor-Critic (MFAC) flow, a continuous-time learning dynamics for solving mean-field games (MFGs), drawing on ideas from reinforcement learning, generative modeling, and optimal transport. The MFAC framework jointly evolves the actor, critic, and distribution through gradient-based updates, with the distribution governed by a novel Optimal Transport Geodesic Picard (OTGP) flow. The OTGP flow drives the distribution toward equilibrium along Wasserstein-2 geodesics. We rigorously analyze the MFAC flow using Lyapunov functionals and establish global exponential convergence under suitable time scales. The analysis highlights the coupled structure of the algorithm and offers practical guidelines for choosing learning rates. Numerical results further support the theory and demonstrate the effectiveness of the proposed approach. This is joint work with Mo Zhou (UCLA) and Haosheng Zhou (UCSB).We develop a stochastic game-theoretic model for intraday dispatch of grid-scale battery energy storage systems (BESS). We assume that each BESS operator competitively manages her state-of-charge to maximize energy arbitrage revenues, driven by the endogenized electricity price that depends on the sum of the charging rates. We characterize the Nash equilibrium of the resulting finite-player linear-quadratic stochastic differential game with common noise, obtaining closed-form representations of equilibrium feedback controls and equilibrium price both in the general heterogeneous and the simplified homogeneous BESS setting via a system of Riccati equations. We then analyze competitive effects, including the marginal externality of additional BESS entering the market, the benefit of coordination and the corresponding market power of large operators, and supply effects from hybrid-type BESS. We further study the asymptotic regime as the number of agents grows large. Our model provides a quantitative testbed to study the impact of decentralized BESS deployment on the grid and the resulting reduction in daily price spreads. This is joint work with Mike Ludkovski and Hezhong Zhang (UCSB). |
|