Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

SMART-Editor: A Multi-Agent Framework for Human-Like Design Editing with Structural Integrity

Created by
  • Haebom

Author

Ishani Mondal, Meera Bharadwaj, Ayush Roy, Aparna Garimella, Jordan Lee Boyd-Graber

Outline

SMART-Editor is a framework for constructive layout and content editing in both structured (posters, websites) and unstructured (natural image) domains. Unlike existing models that perform local editing, SMART-Editor maintains global consistency through two strategies: Reward-Refine, an inference-time reward-guided refinement method, and RewardDPO, a training-time preference optimization approach that uses reward-aligned layout pairs. To evaluate model performance, we introduce SMARTEdit-Bench, a benchmark that encompasses multi-domain, cascading editing scenarios. SMART-Editor outperforms strong baseline models such as InstructPix2Pix and HIVE, with RewardDPO achieving up to 15% performance gains in structured settings, and Reward-Refine demonstrating its advantage on natural images. Automatic and human evaluations confirm the value of reward-based schemes in generating semantically consistent and visually aligned edits.

Takeaways, Limitations

Takeaways:
It presents an effective framework for constructive layout and content editing in both structured and unstructured areas.
It maintains global consistency and produces high-quality editing results through two strategies: Reward-Refine and RewardDPO.
We provide a new benchmark, SMARTEdit-Bench, which includes multi-domain, cascading editing scenarios.
It outperforms existing models, and RewardDPO in particular shows significant performance improvements in structured settings.
We experimentally demonstrate the importance of reward-based planning.
Limitations:
Further research may be needed on the scale and diversity of SMARTEdit-Bench.
Performance may be limited for certain types of edits or domains.
Further analysis may be required to determine the interaction and optimization of Reward-Refine and RewardDPO.
Further research is needed on generalization performance and robustness to different editing types.
👍