| Abstract: |
| Deep neural networks (DNN) equipped with rectified power units (RePU) have demonstrated superior approximation capabilities compared to those utilizing rectified linear units (ReLU), as highlighted by B. Li, S. Tang, and H. Yu [Commun. Comput. Phys., 2020, 27(2): 379-411]. These units are particularly advantageous for machine learning tasks requiring high-order continuity in the functions represented by neural networks. Despite their theoretical benefits, however, the practical application of RePU DNNs has been limited due to challenges such as gradient vanishing and explosion during training. In this talk, we explore various strategies aimed at facilitating the training of RePU DNNs. Our primary focus lies on the numerical solutions of partial differential equations. We demonstrate that, with appropriate training techniques, RePU DNNs can achieve better results than standard DNNs employing other commonly used activation functions, and do so with a faster training rate. |
|