Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Zero-Shot Neural Architecture Search with Weighted Response Correlation

Created by
  • Haebom

Author

Kun Jing, Luoyu Chen, Jungang Xu, Jianwei Tai, Yiyu Wang, Shuaimin Li

Outline

This paper presents a novel zero-shot neural architecture search (NAS) method for automatic neural network architecture design. To overcome the limitations of existing zero-shot NAS methods in efficiency, stability, and generality, we propose a new training-free proxy called Weighted Response Correlation (WRCor). WRCor utilizes the correlation coefficient matrix of responses to different input samples to compute a proxy score for the estimated structure, thereby measuring its expressiveness and generalization performance. Experimental results demonstrate that WRCor and the voting-based proxy are more efficient estimation strategies than existing proxies, and when combined with various search strategies, they outperform existing NAS algorithms. On the ImageNet-1k dataset, we discovered a structure that achieves a test error rate of 22.1% within 4 GPU hours. The source code is publicly available.

Takeaways, Limitations

Takeaways:
A new proxy (WRCor) proposed to overcome the limitations of existing zero-shot NAS methods.
Zero-shot NAS algorithm using WRCor outperforms existing NAS algorithms.
High-performance structure discovery in a short time on ImageNet-1k using efficient exploration.
Ensuring reproducibility and scalability through source code disclosure
Limitations:
Further validation of the generalization performance of the WRCor proxy is needed.
Additional experiments are needed on various datasets and tasks.
Need to assess dependency on specific search strategies
👍