This page organizes papers related to artificial intelligence published around the world. This page is summarized using Google Gemini and is operated on a non-profit basis. The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.
Dendritic Resonate-and-Fire Neuron for Effective and Efficient Long Sequence Modeling
Created by
Haebom
Author
Dehao Zhang, Malu Zhang, Shuai Wang, Jingya Wang, Wenjie Wei, Zeyu Ma, Guoqing Wang, Yang Yang, Haizhou Li
Outline
Resonate-and-Fire (RF) neurons are well-suited for modeling long sequences, but they suffer from limited memory capacity and trade-offs between energy efficiency and learning speed. Inspired by the dendritic structure of biological neurons, this paper proposes a Dendritic Resonate-and-Fire (D-RF) model that explicitly integrates multiple dendrites and a body structure. Each dendritic branch encodes a specific frequency band by exploiting the unique oscillatory dynamics of RF neurons, achieving a comprehensive frequency representation. Furthermore, an adaptive thresholding mechanism is introduced into the body structure, which adjusts the threshold based on historical spiking activity, reducing redundant spikes while maintaining training efficiency in long sequence tasks.
Takeaways, Limitations
•
Takeaways:
◦
The D-RF model achieves spike sparsity while maintaining competitive accuracy.
◦
It can be an effective and efficient solution for modeling long sequences on edge platforms without compromising computational efficiency during training.
◦
Introduction of multi-dendritic and body structures, and adaptive threshold mechanism.
•
Limitations:
◦
Limitations stated in the paper is not directly presented.