This paper presents the implicit safety set algorithm, a model-free safety control algorithm that guarantees the safety of deep reinforcement learning (DRL) agents. While existing DRL methods struggle to guarantee safety, our algorithm generates safety indices (barrier certificates) and safety control laws using only black-box dynamic functions (e.g., digital twin simulators). We theoretically prove convergence to a safe set within finite time and forward invariance for both continuous and discrete-time systems. Furthermore, we demonstrate our performance on the Safety Gym benchmark, achieving a cumulative reward of 95% ± 9%, outperforming the existing state-of-the-art safe DRL method, without safety violations. Furthermore, we demonstrate scalability to high-dimensional systems via parallel computing.
