Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

ACMP: Allen-Cahn Message Passing with Attractive and Repulsive Forces for Graph Neural Networks

Created by
  • Haebom

Author

Yuelin Wang, Kai Yi, Xinliang Liu, Yu Guang Wang, Shi Jin

Outline

In this paper, we model neural message passing, the fundamental feature extraction unit for graph-structured data, as an interacting particle system with attractive and repulsive forces and Allen-Cahn forces arising from phase transition modeling. The dynamics of the system is a non-explosive reaction-diffusion process that separates particles. This leads to Allen-Cahn message passing (ACMP) for graph neural networks, where numerical iterations of the particle system solution constitute message passing propagation. ACMP, which can be implemented simply using a neural ODE solver, can increase the network depth up to 100 layers with a theoretically proven strictly positive lower bound on the Dirichlet energy. Thus, we provide a deep model of GNN that solves the common GNN problem of over-smoothing. GNNs using ACMP achieve state-of-the-art performance on real-world node classification tasks on both homogeneous and heterogeneous datasets. The code can be found at https://github.com/ykiiiiii/ACMP .

Takeaways, Limitations

Takeaways:
We show that Allen-Cahn message passing (ACMP) can dramatically increase the depth of GNNs, thereby solving the over-smoothing problem.
Achieving state-of-the-art performance on real-world node classification tasks based on simple implementation and theoretical justification.
Theoretical proof of a strictly positive lower bound for the Dirichlet energy.
Limitations:
Further analysis of the computational cost and scalability of the proposed method is needed.
Additional experiments are needed to determine generalization performance for different graph structures and tasks.
There is a possibility of bias towards certain types of graph data.
👍