This paper aims to address the challenges of multimodal data mixing, activity heterogeneity, and complex model deployment in sensor-based human activity recognition (HAR). To this end, we propose a spatial-temporal attention modal decomposition alignment fusion strategy to address the mixed distribution of sensor data, capture key discriminative features of activities through multimodal spatial-temporal separate representations, and combine gradient modulation to mitigate data heterogeneity. In addition, we build a wearable deployment simulation system and conduct experiments on a number of public datasets to demonstrate the effectiveness of our model.
