Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Quantifying Robustness: A Benchmarking Framework for Deep Learning Forecasting in Cyber-Physical Systems

Created by
  • Haebom

Author

Alexander Windmann, Henrik Steude, Daniel Boschmann, Oliver Niggemann

Outline

This paper presents a robustness evaluation framework for deep learning (DL)-based predictive models for predictive modeling and health management (PHM) that utilize complex time-series data generated in cyber-physical systems (CPS) such as manufacturing and energy distribution. Existing robustness evaluations have limitations in that they focus on formal verification or adversarial interference and fail to sufficiently reflect the complexity of real CPS environments. In this paper, we propose a practical robustness definition based on distributional robustness and present a systematic framework to thoroughly analyze the robustness of predictive models on real CPS datasets by simulating realistic disturbances such as sensor drift, noise, and irregular sampling. We demonstrate the applicability and effectiveness of the proposed approach through empirical studies on various DL architectures (including recurrent, convolutional, attention-based, modular, and structured state-space models), and disclose robustness benchmarks to encourage further research and reproducibility.

Takeaways, Limitations

Takeaways:
Presenting a practical robustness evaluation framework that considers the complexity of real industrial CPS environments
Provides standardized robustness scores based on distributional robustness, facilitating model comparison and selection
Support for optimal model design through robustness comparison analysis of various DL architectures
Future research and reproducibility enhancement through published benchmarks
Limitations:
Further research is needed on the generalizability of the proposed framework.
Comprehensive consideration of various types and intensities of failures in real industrial environments is needed.
The need to develop robustness assessment criteria specific to specific industries
👍