As understanding of autism and ableism grows, so does the understanding of ableist language associated with autism. This language presents significant challenges for NLP research due to its nuanced and context-dependent nature. However, detecting anti-autistic ableist language remains an unexplored area, and existing NLP tools often fail to capture its subtle expressions. In this paper, we address this critical gap by presenting AUTALIC, the first benchmark dataset dedicated to detecting anti-autistic ableist language in context. This dataset consists of 2,400 autism-related sentences and their surrounding context collected from Reddit, annotated by experienced experts with a background in neurodiversity. Comprehensive evaluations demonstrate that current language models, including state-of-the-art LLMs, struggle to reliably identify anti-autistic ableism and match human judgment, highlighting limitations in this area. By publicly releasing AUTALIC, along with its individual annotations, we provide a valuable resource for researchers studying ableism, neurodiversity, and the discrepancy in annotation efforts. This dataset is an important step toward developing more comprehensive and context-aware NLP systems that better reflect diverse perspectives.
