[공지사항]을 빙자한 안부와 근황 
Show more

Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Neural Architecture Search with Mixed Bio-inspired Learning Rules

Created by
  • Haebom

Author

Imane Hamzaoui, Riyadh Baghdadi

Outline

In this paper, we propose a method to improve the accuracy and scalability of bio-inspired neural networks by applying different biological learning rules to each layer using Neural Architecture Search (NAS). We extend the search space of existing NAS-based models to include various biological learning rules, and automatically find the optimal architecture and learning rules for each layer through NAS. Experimental results show that neural networks using different biological learning rules for each layer achieve higher accuracy than those using a single rule. On CIFAR-10, CIFAR-100, ImageNet16-120, and ImageNet datasets, we break the best performance of existing biologically inspired models, and in some cases, outperform backpropagation-based networks. This suggests that the diversity of learning rules per layer contributes to improved scalability and accuracy.

Takeaways, Limitations

•
Takeaways:
◦
We demonstrate that automatically assigning optimal biological learning rules to each layer via neural architecture search (NAS) is effective in improving performance.
◦
Overcoming the accuracy limitations of existing biologically inspired models through a combination of different biological learning rules, and in some cases achieving performance that outperforms backpropagation-based models.
◦
We suggest that the diversity of learning rules per layer plays an important role in improving the scalability and accuracy of neural networks.
◦
A new direction in the study of biologically inspired neural networks (study of combinations of various learning rules).
•
Limitations:
◦
The proposed method may be computationally expensive (due to the nature of NAS).
◦
The type and scope of biological learning rules used may be limited.
◦
Further research is needed on generalization performance across diverse datasets and tasks.
👍
Made with Slashpage