This paper presents an opportunity to discover large-scale causal graphs representing regulatory interactions among genes, leveraging modern high-throughput biological datasets containing thousands of perturbations. Existing discriminable causal graph models have been used to infer gene regulatory networks (GRNs) from large-scale perturbation datasets and capture causal gene regulatory relationships from genetic perturbations. However, they suffer from limited expressiveness and scalability and fail to address the dynamic nature of biological processes such as cell differentiation. In this paper, we propose PerturbODE, a novel framework that integrates biologically informative neural ordinary differential equations (neural ODEs) to model cell state trajectories and derive causal GRNs from the parameters of the neural ODEs. We demonstrate the efficacy of trajectory prediction and GRN inference on simulated and real-world overexpression datasets.