This paper addresses the problem of automatic synchronization of spintronic oscillators (STOs) using reinforcement learning (RL). We simulate STOs using numerical solutions of the macroscopic spin Landau-Lipschitz-Gilbert-Slonczewski equations and train two types of RL agents to synchronize to a target frequency within a fixed step. We explore modifications to the underlying task and demonstrate that convergence and energy efficiency improvements in synchronization can be readily achieved in a simulation environment.