In this paper, we present a P4 (Personalized, Private, Peer-to-Peer) methodology that enables efficient and personalized learning in a distributed environment among resource-constrained heterogeneous IoT devices. P4 is designed to provide personalized models for resource-constrained IoT devices while guaranteeing differential privacy and robustness against malicious attacks. It detects client similarity while preserving privacy using a lightweight, fully distributed algorithm and forms collaborative groups, and within each group, clients jointly learn models using differential privacy knowledge distillation. We evaluate P4 on popular benchmark datasets using linear and CNN-based architectures in various heterogeneous settings and attack scenarios, and show that it achieves 5% to 30% higher accuracy than existing differential privacy peer-to-peer approaches and maintains robustness even in the presence of up to 30% malicious clients. We also demonstrate its practicality by showing that collaborative learning between two clients adds only about 7 seconds of overhead on resource-constrained devices.
