| Abstract: |
| Neural networks are generally treated as black boxes. In an effort to uncover the mathematical structure underlying them, we explain how ReLU nets can be interpreted as zero-sum
turn-based, stopping games.
The game runs in the opposite direction to the net and running the net is the same thing as running the Shapley- Bellman backwards recursion for the value of the game.
Net weights are used to define state transition probabilities and the biases define rewards.
The input to the net is the terminal reward of the game, the output of any neuron is the value of the game at a corresponding game state.
We will explain applications of this construction, among which is the calculation of interval bounds for the output of every neuron given interval bounds for the input to the net.
Another is the interpretation of a ReLU net classifier as a game between 2 players, one of which is trying to prove the input is in a given class and the other trying to prove the opposite.
Adding an entropic regularization to the ReLU net game allows us to interpret Softplus nets as games analogously.
This is joint work with St\`ephane Gaubert. |
|