In this paper, we propose LLMFP, a general-purpose framework that leverages large-scale language models (LLMs) to solve various planning problems. To overcome the limitations of existing LLM-based planning methods that require prior preparation for complex problems or specific tasks, LLMFP formulates and solves the planning problem as a constrained optimization problem. It solves problems without task-specific examples by leveraging the common sense, reasoning, and programming capabilities of LLMs, and achieves an average optimal solution rate of 83.7% and 86.8% in experiments on nine planning problems using GPT-4 and Claude 3.5, showing significant performance improvement over existing methods. We analyze the components of LLMFP and the causes of success/failure through experimental results and ablation studies.