This paper presents a study on the development of an agent framework that can autonomously perform complex tasks by integrating multiple large-scale language models (LLMs), and its application to the reverse engineering of photonic metamaterials. Given a desired optical spectrum, the framework autonomously proposes and develops a forward-looking deep learning model, accesses external tools via APIs for tasks such as simulation and optimization, utilizes memory, and generates the final design via deep inverse methods. It demonstrates effectiveness through automation, reasoning, planning, and adaptive capabilities, and can generate diverse and potentially novel outcomes through internal reflection and decision flexibility.