This paper presents a novel ensemble forecasting model for the El Niño Southern Oscillation (ENSO) phenomenon, a challenging long-term forecasting scenario. Existing multi-modal ENSO forecasting (MEF) models utilize 80 ensemble forecasts from two deep learning modules: a 3D CNN and a time series module. However, the weighting and evaluation of individual ensemble members are limited. This study directly models the similarity between the 80 ensemble members using graph-based analysis, identifying and clustering similar and accurate forecasts. A community detection method is used to obtain an optimized subset of 20 members, which are then averaged to produce the final forecast. This method improves forecast performance by removing noise and emphasizing ensemble consistency, resulting in more stable and consistent results, especially in long-term forecasting situations. Furthermore, because it is model-independent, it can be applied to a variety of forecasting models.