2020-02-11: Invariant Risk Minimization Games https://arxiv.org/abs/2002.04692v1The set of solutions to this game is exactly the same as the set of invariant predictors across training environments

‹›
The standard risk minimization paradigm of machine learning is brittle when
operating in environments whose test distributions are different from the
training distribution due to spurious correlations. Training on data from many
environments and finding invariant predictors reduces the effect of spurious
features by concentrating models on features that have a causal relationship
with the outcome. In this work, we pose such invariant risk minimization as
finding the Nash equilibrium of an ensemble game among several environments. By
doing so, we develop a simple training algorithm that uses best response
dynamics and, in our experiments, yields similar or better empirical accuracy
with much lower variance than the challenging bi-level optimization problem of
Arjovsky et.al. (2019). One key theoretical contribution is showing that the
set of Nash equilibria for the proposed game are equivalent to the set of
invariant predictors for any finite number of environments, even with nonlinear
classifiers and transformations. As a result, our method also retains the
generalization guarantees to a large set of environments shown in Arjovsky
et.al. (2019). The proposed algorithm adds to the collection of successful
game-theoretic machine learning algorithms such as generative adversarial
networks.