This paper proposes a mobile-based image analysis framework for the early diagnosis of arsenicosis, a serious public health problem in South and Southeast Asia. We constructed a dataset of arsenic-induced skin diseases and other skin diseases containing over 11,000 images and compared and evaluated CNN- and Transformer-based models. The Swin Transformer model achieved the highest performance, achieving 86% accuracy. The model's interpretability was enhanced using LIME and Grad-CAM. The practical applicability was also demonstrated through a web-based diagnostic tool. This study demonstrates the potential of non-invasive, accessible, and explainable arsenicosis diagnosis using mobile images.