In this paper, we propose DesiGNN, a knowledge-centric framework, to address the challenges of automatically designing graph neural networks (GNNs) using large-scale language models (LLMs). DesiGNN transforms existing model design experiences into a structured knowledge dictionary, which is utilized for meta-learning of LLMs, and performs empirical feature filtering and adaptive information gathering through literature analysis using benchmarks and LLMs. Through this, it builds meta-knowledge between understanding unknown graph data and effective architectural patterns, proposes a top-level GNN model in a short time, and achieves superior performance at a much lower cost than existing methods.