This paper questions why state-of-the-art Vision Transformers (ViTs) are not designed to exploit natural geometric symmetries such as 90-degree rotations and reflections, and argues that the lack of efficient implementations is the cause. To address this, we propose Octic Vision Transformers (octic ViTs), which leverage octic group isomorphism. Octic linear layers reduce FLOPs by 5.33x and memory by up to 8x compared to conventional linear layers. We study two families of ViTs composed of octic blocks and train them on ImageNet-1K using supervised (DeiT-III) and unsupervised (DINOv2) learning methods. We demonstrate significant efficiency improvements while maintaining baseline accuracy.