To address the problem of real-time motion generation, which is essential for achieving responsive and adaptive motion under motion constraints in high-dimensional systems, we present a two-step approach. First, we learn a low-dimensional trajectory manifold offline that satisfies task-related constraints, and then we perform a fast online search within this manifold. Extending the existing discrete-time motion manifold primitive (MMP) framework, we propose a novel neural network architecture, the differentiable motion manifold primitive (DMMP), which is trained on trajectory optimization data collected offline with a strategy that guarantees constraint satisfaction. Through dynamic throwing experiments using a 7-DOF robotic arm, we demonstrate that DMMP outperforms existing methods in terms of planning speed, task success rate, and constraint satisfaction.