Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Addressing The Devastating Effects Of Single-Task Data Poisoning In Exemplar-Free Continual Learning

Created by
  • Haebom

Author

Stanis{\l}aw Pawlak (Warsaw University of Technology, Poland), Bart{\l}omiej Twardowski (IDEAS Research Institute, Poland, Computer Vision Center, Universitat Autonoma de Barcelona, Spain), Tomasz Trzci nski (Warsaw University of Technology, Poland, IDEAS Research Institute, Poland), Joost van de Weijer (Computer Vision Center, Universitat Autonoma de Barcelona, Spain)

Outline

This study addresses the security issue of data poisoning, which has been overlooked in Continuous Learning (CL). While previous research has focused on scenario-dependent attacks, this study focuses on the simpler and more realistic threat of single-task poisoning (STP). In an STP attack, the adversary has no access to the model, previous task data, or future task data. The adversary only has access to the current task data within the data stream, and we demonstrate that it can degrade model performance by exploiting standard image corruption. STP attacks disrupt the entire continuous learning process, reducing both performance on past tasks (stability) and the ability to adapt to new tasks (plasticity). Finally, we propose a high-level defense framework for CL, along with a task vector-based poisoned task detection method.

Takeaways, Limitations

Takeaways:
We reveal the severity of single-task poisoning (STP) attacks in continuous learning.
Presentation of an effective defense framework against STP attacks and a method for detecting poisoning operations.
We demonstrate that data poisoning attacks are possible even under conditions of limited information access.
Limitations:
Further research is needed to evaluate the generalization performance and applicability of the proposed defense framework to real-world environments.
Analysis of more sophisticated and diverse attack types is needed.
Computational cost and performance overhead analysis of defense mechanisms against STP attacks is needed.
👍