In this paper, we propose a novel framework to address the generalizability problem of device identification (DI) for enhancing the security of Internet of Things (IoT) devices. Existing machine learning-based DI approaches overlook the difficulty of model generalizability in diverse network environments. In this study, we improve the feature and model selection method using genetic algorithms and external feedback, and evaluate the model generalizability by utilizing datasets collected from different network environments. We experimentally demonstrate the limitations of existing techniques such as sliding window and flow statistics, and the lack of reliability of statistical methods that depend on network characteristics, contributing to the advancement of IoT security and device identification research.