Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

POPri: Private Federated Learning using Preference-Optimized Synthetic Data

Created by
  • Haebom

Author

Charlie Hou, Mei-Yu Wang, Yige Zhu, Daniel Lazar, Giulia Fanti

Outline

This paper presents a method for using differentially privacy-preserving synthetic data in privacy-conscious federated learning (DP-FL). Existing DP synthetic data generation algorithms require careful prompt engineering based on public information or iterative private client feedback. In this paper, we propose the POPri algorithm, which treats private client feedback collected from existing methods as reinforcement learning (RL) rewards and fine-tunes LLMs using a policy optimization algorithm (e.g., DPO) to generate high-quality DP synthetic data. We evaluate POPri using LargeFedBench, a novel federated text benchmark, and find that it significantly improves the usability of DP synthetic data compared to existing methods, reducing the gap between fully private and non-private settings by up to 58%. The code and data are available on GitHub.

Takeaways, Limitations

•
Takeaways:
◦
Improvement of existing DP synthetic data generation methods: We improved the quality of DP synthetic data through reinforcement learning-based policy optimization.
◦
DP-FL Performance Improvement: We demonstrated performance improvements over existing DP-FL methods and existing synthetic data methods (up to 58% performance improvement).
◦
Introducing a New Benchmark: We've released LargeFedBench, a new benchmark for LLM assessment in federated learning environments.
◦
Ensuring reproducibility: We increased the reproducibility of our research by making our code and data open.
•
Limitations:
◦
Generality of LargeFedBench: Further research may be needed to determine the generality and scalability of the presented LargeFedBench benchmark to other datasets.
◦
Algorithmic complexity: The complexity of reinforcement learning-based algorithms can lead to high computational costs.
◦
Privacy Guarantee Level: Further analysis and verification of the algorithm's privacy guarantee level may be required.
👍
Made with Slashpage