This paper addresses the problem of automatically classifying texts’ political slant and politicalness using Transformer models. By comprehensively reviewing existing datasets and models, we find that current approaches produce disjoint solutions and underperform on out-of-distribution texts. To address these limitations, we combine twelve datasets for political slant classification to create a diverse dataset, and extend eighteen existing datasets with appropriate labels to create a new dataset for politicalness. We evaluate the performance of existing models through extensive benchmarking using leave-one-in and leave-one-out methodologies, and train new models with improved generalization ability.
