This paper studies privacy-preserving federated learning (PPFL), which trains a global model using data from multiple clients while preserving privacy. To overcome the limitations of existing PPFL protocols, such as poor accuracy, the need for key sharing, and the need for cooperation during key generation or decryption, we propose a novel PPFL protocol utilizing neural networks. This protocol incorporates homomorphic adversarial networks (HANs) that integrate an aggregatable hybrid encryption scheme tailored to the requirements of PPFL. It performs tasks similar to multi-key homomorphic encryption (MK-HE) while solving key distribution and collaborative decryption problems. Experimental results demonstrate that HANs are robust against privacy attacks, exhibit minimal accuracy loss (up to 1.35%) compared to non-privacy-preserving federated learning, and achieve a 6,075x increase in encryption aggregation speed compared to existing MK-HE schemes, but with a 29.2x increase in communication overhead.