Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

ZetA: A Riemann Zeta-Scaled Extension of Adam for Deep Learning

Created by
  • Haebom

Author

Samiksha BC

Outline

ZetA is a novel deep learning optimization algorithm that integrates dynamic scaling based on the Riemann zeta function into Adam. It improves generalization performance and robustness through a hybrid update mechanism that incorporates adaptive decay, cosine-similarity-based momentum boosting, entropy-regularized loss, and Sharpness-Aware Minimization (SAM)-style perturbations. It demonstrates improved test accuracy compared to Adam on the SVHN, CIFAR10, CIFAR100, STL10, and noisy CIFAR10 datasets, trained for 5 epochs using a lightweight fully-connected network with mixed-precision settings. It demonstrates a computationally efficient and robust alternative to Adam, especially for noisy or high-dimensional classification tasks.

Takeaways, Limitations

•
Takeaways:
◦
A novel approach to optimization is presented by applying dynamic scaling based on the Riemann zeta function.
◦
Experimentally demonstrated improved generalization performance and robustness over Adam on various datasets.
◦
It has been proven to be computationally efficient and effective on noisy datasets.
◦
Combines the advantages of various optimization techniques through a hybrid update mechanism.
•
Limitations:
◦
Limited 5 epoch training, lack of long-term performance evaluation.
◦
Only experimental results for lightweight fully connected networks are presented; generalizability to other network structures needs to be verified.
◦
Lack of analysis of various hyperparameter settings.
◦
A more comprehensive comparative analysis with other optimization algorithms is needed.
👍
Made with Slashpage