Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Kolmogorov-Arnold Fourier Networks

Created by
  • Haebom

Author

Jusheng Zhang, Yijia Fan, Kaitong Cai, Keze Wang

Outline

While Kolmogorov-Arnold-based interpretable networks (KANs) possess powerful theoretical expressiveness, they face parameter explosion and high-frequency feature capture issues in high-dimensional tasks. To address these challenges, this paper proposes a Kolmogorov-Arnold-Fourier network (KAF) that effectively integrates learnable random Fourier features (RFFs) and a novel hybrid GELU-Fourier activation mechanism to balance parameter efficiency and spectral representational capabilities. Our key contributions include: (1) significantly reducing parameters by merging the dual matrix structure of KANs with the matrix-coupling property; (2) introducing a learnable RFF initialization strategy to eliminate spectral distortion in high-dimensional approximation tasks; and (3) implementing an adaptive hybrid activation function that progressively improves frequency representation during training. Comprehensive experiments demonstrate the superiority of KAF across a variety of domains, including vision, NLP, audio processing, and differential equation solving tasks, effectively combining theoretical interpretability with practicality and computational efficiency.

Takeaways, Limitations

•
Takeaways:
◦
We present a novel network architecture (KAF) that effectively addresses the parameter explosion and high-frequency feature capture problems of KAN in high-dimensional tasks.
◦
Enhanced spectral representation and parameter efficiency through learnable RFF initialization strategies and adaptive hybrid activation functions.
◦
Demonstrated excellent performance in various fields such as vision, NLP, audio processing, and differential equation solving.
◦
Simultaneously achieving theoretical interpretability, practicality, and computational efficiency.
•
Limitations:
◦
Further research is needed on the generalization performance of the proposed KAF.
◦
More extensive experiments on diverse high-dimensional datasets are needed.
◦
Further research is needed on specific hyperparameter optimization.
👍
Made with Slashpage