Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Secure Reinforcement Learning via Shuffle Privacy Model

Created by
  • Haebom

Author

Shaojie Bai, Mohammad Sadegh Talebi, Chengcheng Zhao, Peng Cheng, Jiming Chen

Outline

This paper focuses on privacy issues in reinforcement learning (RL), particularly the risk of privacy inference attacks in cyber-physical systems (CPS). Existing centralized differential privacy (DP) models rely on trusted servers, while distributed local models suffer from severe performance degradation. Therefore, this paper proposes a novel algorithm for episodic RL, Shuffled Differential Privacy Policy Elimination (SDP-PE), utilizing the shuffle model, an intermediate trust model. SDP-PE balances privacy and learning performance by introducing an exponential batch schedule and a "forget" mechanism, achieving a near-optimal regret upper bound and offering a superior privacy-regret trade-off than local models. This demonstrates the applicability of the shuffle model for secure data-driven CPS control.

Takeaways, Limitations

•
Takeaways:
◦
We propose a new solution to the privacy-preserving reinforcement learning problem in CPS environments by presenting a reinforcement learning algorithm, SDP-PE, that utilizes the shuffle model.
◦
SDP-PE overcomes the limitations of existing centralized and local models and effectively achieves a balance between privacy and performance.
◦
We present a method to effectively control the balance between privacy and learning performance through an exponential batch schedule and a 'forget' mechanism.
◦
Demonstrating the practicality of the shuffle model for secure data-driven control of CPS.
•
Limitations:
◦
Dependence on the assumptions of the shuffle model: Since the shuffle model does not guarantee perfect anonymity, further research may be needed to investigate the attack potential during the shuffling process.
◦
Complexity of the algorithm: The high complexity of the SDP-PE algorithm may make it difficult to implement and apply in practice.
◦
Limitations for episodic RL: The results of this paper are limited to episodic RL, and their applicability to continuous RL problems requires further study.
👍
Made with Slashpage