To address the challenge of tightening constraints on neural network outputs in safety-critical control applications, this paper proposes a neural network training method that derives accurate images of nonconvex input sets for neural networks with ReLU nonlinearities to avoid nonconvex unsafe regions. This is achieved through reachability analysis using an extended hybrid zonotope set representation that enables differentiable collision detection via mixed-integer linear programming (MILP). The method is proven to be efficient and fast for networks with up to 240 neurons, and its computational complexity is dominated by inverse operations on matrices that scale linearly with the number of neurons and the complexity of the input and unsafe sets. We demonstrate its practicality in training forward-invariant neural network controllers for affine dynamic systems with nonconvex input sets and in generating safe reach-avoidance plans for black-box dynamic systems.
