In a multi-agent environment, the goal of an agent is to maximize its total reward against its opponent. Game-theoretic solutions, such as the Nash equilibrium, can achieve robust performance in certain environments, but they fail to leverage historical and observed data obtained through repeated interactions. Adversarial modeling algorithms incorporate machine learning techniques to exploit available data to exploit non-optimal opponents, but the effectiveness of these approaches in games with incomplete information has been limited to date. This paper shows that existing adversarial modeling approaches fail to satisfy a simple desirable property, even for static opponents drawn from a known prior distribution. That is, they fail to guarantee that the model approximates the opponent's true strategy as the number of game iterations approaches infinity. In this paper, we develop a novel algorithm that achieves this property and efficiently solves a convex minimization problem based on a sequence-form game representation using projected gradient descent. This algorithm is guaranteed to converge efficiently to the opponent's true strategy, utilizing observations from game play and, when available, additional historical data.
