Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Multi-objective Optimization in CPU Design Space Exploration: Attention is All You Need

Created by
  • Haebom

Author

Runzhen Xue, Hao Wu, Mingyu Yan, Ziheng Xiao, Guangyu Sun, Xiaochun Ye, Dongrui Fan

Outline

This paper presents AttentionDSE, a novel design space exploration (DSE) framework for CPU design in high-dimensional design spaces. Existing DSE frameworks suffer from issues such as poor accuracy and scalability of surrogate models, inefficient exploration relying on manual heuristics or exhaustive search, and difficulties in interpretation. AttentionDSE addresses these issues by integrating performance predictions and design guidance through an attention-based neural network architecture. Attention weights perform the dual role of accurately estimating performance while simultaneously exposing performance bottlenecks. Key innovations include a perception-driven attention mechanism that leverages hierarchy and locality (reducing complexity from $\mathcal{O}(n^2)$ to $\mathcal{O}(n)$) and attention-aware bottleneck analysis, which automatically suggests critical parameters for goal-directed optimization. In a high-dimensional CPU design space evaluation using the SPEC CPU2017 benchmark, AttentionDSE achieves up to 3.9% higher Pareto Hypervolume and over 80% faster search time compared to state-of-the-art baseline models.

Takeaways, Limitations

Takeaways:
We present a novel, efficient and interpretable framework for solving DSE problems in high-dimensional design spaces.
Leveraging the attention mechanism to integrate performance prediction and design guidance, building an efficient optimization loop.
Reduced computational complexity and removal of domain-specific heuristic dependence through perception-driven attention and attention-aware bottleneck analysis.
Experimental results verify the effectiveness of improving Pareto Hypervolume and reducing search time.
Limitations:
Currently, the framework is specialized for CPU design, and further research is needed to determine its generalizability to other design spaces.
Optimization research is needed on the sliding window size and parameter settings of the perception-driven attention mechanism.
The experiment is limited to the SPEC CPU2017 benchmark and requires validation on other benchmarks or real hardware.
👍