This paper addresses the problem of identifying user inaccurate actions from egocentric video data. To handle subtle and rare mistakes, we propose a Dual-Stage Reweighted Mixture-of-Experts (DR-MoE) framework. In the first stage, features are extracted using a fixed ViViT model and a LoRA-tuned ViViT model, which are then combined through a feature-level expert module. In the second stage, three classifiers are trained using reweighted cross-entropy to mitigate the imbalanced class problem, AUC loss to improve ranking in skewed distributions, and label-aware loss and sharpness-aware minimization to enhance calibration and generalization. Their predictions are fused using a class-level expert module. The proposed method demonstrates particularly robust performance in identifying rare and ambiguous mistakes.
