Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Agentic Large Language Models for Conceptual Systems Engineering and Design

Created by
  • Haebom

Author

Soheyl Massoudi, Mark Fuge

Outline

This paper evaluates whether a structured multi-agent system (MAS) can more effectively manage requirements elicitation, functional decomposition, and simulator code generation than a simple two-agent system (2AS) to overcome the limitations of existing large-scale language model (LLM) workflows in early-stage engineering design that requires complex iterative reasoning. Targeting solar water treatment system design, we introduce a design state graph (DSG), a JSON-serializable representation that bundles requirements, physical implementations, and Python-based physical models as graph nodes, and compare a nine-role MAS with a 2AS consisting of a generator-reflector loop. We conduct a total of 60 experiments using Llama 3.3 70B and a reasoning-distilled DeepSeek R1 70B model and measure JSON validity, requirements fulfillment rate, implementation presence, code compatibility, workflow completion rate, execution time, and graph size. As a result, both systems maintain perfect JSON integrity and implementation tagging, but the requirements fulfillment rate is low, less than 20%. Code compatibility reached 100% in a specific 2AS setup, but MAS was below 50%. DeepSeek R1 70B-based MAS produced more fine-grained DSGs (average 5-6 nodes), and the reasoning-distilled model improved the workflow completion rate, but the problems of low requirements fulfillment rate and lack of coding fidelity persisted.

Takeaways, Limitations

Takeaways: A structured multi-agent system (MAS) can generate more detailed design information in early-stage engineering design than a simple two-agent system (2AS). Reasoning-distilled LLM improves workflow completion rates. Structured representations such as DSG are useful for LLM-based design process management.
Limitations: Very low requirement fulfillment rate (less than 20%). Low code compatibility (less than 50% for MAS), low coding fidelity of LLM. Perfect JSON integrity was maintained in all experiments, but it does not guarantee the completeness of the actual design. Results may vary significantly depending on the performance of the LLM used.
👍