Daily Arxiv

This page organizes papers related to artificial intelligence published around the world.
This page is summarized using Google Gemini and is operated on a non-profit basis.
The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.

DeNOTS: Stable Deep Neural ODEs for Time Series

Created by
  • Haebom

Author

Ilya Kuleshov, Evgenia Romanenkova, Vladislav Zhuzhel, Galina Boeva, Evgeni Vorsin, Alexey Zaytsev

Outline

This paper presents a novel method for enhancing the expressive power of Neural Coefficients Derived (CDEs), a natural way to process irregular time series. Existing Neural CDEs adjust the depth (the number of function evaluations, or NFE) of the model by lowering the solver error tolerance to increase numerical accuracy. However, this approach is insufficient for enhancing expressive power. In this paper, we present a simple yet effective alternative: expanding the integration time horizon to increase NFE and deepen the model. To address the uncontrolled growth of the original vector field, we propose the Negative Feedback (NF) technique, which guarantees stability without restricting flexibility. We provide theoretical bounds on the risk of Neural ODEs using Gaussian process theory, and experiments on four public datasets demonstrate that the proposed method outperforms existing methods, including Neural RDEs and state-space models, by up to 20%. The proposed method, called DeNOTS, combines expressive power, stability, and robustness to enable reliable modeling in the continuous-time domain.

Takeaways, Limitations

Takeaways:
We present a novel method (integrated time horizon extension and negative feedback) to effectively improve the expressive power of neural CDEs.
Ensure model stability through negative feedback and secure robustness by presenting theoretical boundaries.
Demonstrated superior performance over existing methods on various datasets.
Presenting the possibility of reliable modeling in the continuous time domain.
Limitations:
Further verification of the generalization performance of the proposed method is needed.
Applicability evaluation for various types of irregular time series is required.
Further research is needed on the Negative Feedback parameter optimization strategy.
👍