This paper introduces Recursive Thematic Partitioning (RTP), a novel framework that leverages large-scale language models (LLMs) to interactively construct binary trees to address the challenges of unsupervised text corpus analysis. RTP constructs an interpretable taxonomy by structuring each node with a natural language question for semantic segmentation of the data. We demonstrate that RTP offers higher interpretability than keyword-based clustering in conventional topic models and can be leveraged as a powerful feature in downstream classification tasks. Furthermore, we demonstrate that the topic paths generated through RTP can serve as structured and controllable prompts for a generative model, enabling a powerful synthesis tool that consistently mimics specific features discovered in the source corpus.