Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

NegMerge: Sign-Consensual Weight Merging for Machine Unlearning

Created by
  • Haebom

Author

Hyo Seo Kim, Dongyoon Han, Junsuk Choe

Outline

This paper proposes a novel method to address the hyperparameter sensitivity issue of existing machine unlearning methods. While existing methods utilize the weight vector of a single model fine-tuned with specific hyperparameters, this paper utilizes the weight vectors of multiple models trained with various hyperparameters. Machine unlearning is performed by collecting only elements with consistent signs among multiple vectors to generate a new vector and subtracting it from the weight vector of the original model. Experimental results using 12 datasets and 4 backbone architectures show that the proposed method outperforms existing state-of-the-art methods while consuming similar or less computational resources.

Takeaways, Limitations

Takeaways:
It effectively solves the hyperparameter sensitivity problem of existing machine unlearning methods.
Enables more robust and accurate machine learning by leveraging various hyperparameters.
It outperforms existing state-of-the-art methods and is also computationally resource efficient.
Reproducibility and usability are improved through open code.
Limitations:
It is possible that the performance improvements of the proposed method may be limited to specific datasets or architectures.
The computational cost of exploring different hyperparameter combinations may increase (though the paper claims it is similar or less than existing methods).
Using only elements with consistent signs can lead to information loss.
👍