Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Enhancing Plasticity for First Session Adaptation Continual Learning

Created by
  • Haebom

Author

Imad Eddine Marouf, Subhankar Roy, St ephane Lathuili ere, Enzo Tartaglione

Outline

To overcome the limitations of first-session adaptation (FSA), an efficient strategy to integrate large pre-trained models (PTMs) into class incremental learning (CIL), this paper presents a plasticity-enhanced test-time adaptation (PLASTIC) method. PLASTIC leverages test-time adaptation (TTA) to dynamically fine-tune layer regularization (LayerNorm) parameters on unlabeled test data, thereby enhancing adaptability to evolving tasks and improving robustness to data corruption. A teacher-student distillation framework prevents model variance caused by TTA and maintains stable learning. Experimental results on various benchmarks show that PLASTIC outperforms existing and state-of-the-art PTM-based CIL approaches, while also demonstrating robustness to data corruption.

Takeaways, Limitations

Takeaways:
It effectively solves the problem of heterogeneity in task distribution, which is a limitation of existing PTM-based CIL methods.
Test Time Adaptation (TTA) improves model adaptability and robustness to data corruption.
Ensures stable learning and generalization performance through teacher-student distillation framework.
It outperforms existing and state-of-the-art methods on various benchmarks.
Limitations:
Further research is needed to determine how effectively fine-tuning only the layer regularization parameters restores the overall plasticity of the model.
Robustness against certain types of data corruption may be limited.
The computational cost of the teacher-student distillation framework may increase.
👍