Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Boost Self-Supervised Dataset Distillation via Parameterization, Predefined Augmentation, and Approximation

작성자
  • Haebom

Author

Sheng-Feng Yu, Jia-Jiun Yao, Wei-Chen Chiu

Outline

This paper proposes a self-supervised dataset distillation (DSD) technique to address the high costs associated with the massive datasets required for training large-scale deep learning models. Unlike conventional supervised dataset distillation, we present a method for compressing images and representations obtained through self-supervised learning into a distilled dataset. To achieve this, we propose a novel method for parameterizing images and representations using low-dimensional bases, a pre-determined augmentation technique to address data augmentation instability, and a lightweight network for compressing distillation pairs. Experiments on various datasets demonstrate the effectiveness of the proposed method, its generalization performance across various architectures, and its superior transfer learning performance.

Takeaways, Limitations

•
Takeaways:
◦
We present a novel method for efficiently compressing large datasets using self-supervised learning.
◦
We improve distillation efficiency and generalization performance through techniques such as parameterization using low-dimensional basis, predetermined augmentation techniques, and distillation pair compression using lightweight networks.
◦
It demonstrates excellent transfer learning performance across various architectures.
•
Limitations:
◦
The performance of the proposed method may be affected by the selected low-dimensional basis. Further research is needed to determine the optimal basis.
◦
Pre-determined augmentation techniques can limit the diversity of data. Further research is needed to determine how to effectively utilize various augmentation techniques.
◦
The experimental results presented in this paper may be limited to a specific dataset and architecture. Further experiments on a broader range of datasets and architectures are needed.
👍
Slashpage로 제작됨