Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

3D Software Synthesis Guided by Constraint-Expressive Intermediate Representation

Created by
  • Haebom

Author

Shuqing Li, Anson Y. Lam, Yun Peng, Wenxuan Wang, and Michael R. Lyu

Outline

In this paper, we present a novel approach called Scenethesis, which focuses on automatic generation of 3D spatial environment-based GUI software, unlike the existing 2D-based GUI software generation studies. Scenethesis maintains formal traceability between user requirements and generated 3D software, and transforms natural language requirements into executable 3D software using a domain-specific language, ScenethesisLang. ScenethesisLang functions as a comprehensive scene description language that enables fine-grained modification of 3D software elements, and a formal constraint representation specification language that can express complex spatial constraints. Scenethesis decomposes 3D software synthesis into several steps based on ScenethesisLang, enabling independent verification, goal-oriented modification, and systematic constraint satisfaction. Experimental results show that Scenethesis accurately captures more than 80% of user requirements, satisfies more than 90% of hard constraints while simultaneously processing more than 100 constraints. In addition, it improves BLIP-2 visual evaluation scores by 42.8% compared to state-of-the-art methods.

Takeaways, Limitations

Takeaways:
Presenting new possibilities in the field of automatic generation of 3D GUI software
Ensuring formal traceability between user requirements and generated software.
Fine-grained control and handling of complex spatial constraints through ScenethesisLang
Achieve high accuracy and efficiency (satisfies more than 80% of requirements, more than 90% of constraints, and improves BLIP-2 score by 42.8%)
Limitations:
Further research is needed on the generalization and extensibility of ScenethesisLang.
Need to verify applicability to various types of 3D software and user interfaces
Need to evaluate performance and stability in real user environments
To date, only a limited range of experimental results have been presented, and more extensive experiments are needed.
👍