In this paper, we propose a lightweight end-to-end model, Neuro-MSBG, to solve the high computational complexity and latency issues of hearing loss simulation models essential for the deployment of hearing aids. Neuro-MSBG performs effective time-frequency modeling using a personalized audiogram encoder and supports parallel inference to reduce the simulation execution time by 46 times (from 0.970 s to 0.021 s based on 1 s input) while maintaining the intelligibility and perceptual quality of the existing MSBG (SRCC 0.9247 based on STOI and SRCC 0.8671 based on PESQ).