This paper presents the development of an accurate and scalable machine learning-based interatomic potential (MLIP), essential for molecular simulation. Unlike existing models that explicitly enforce rotation-translation symmetry, this study proposes a novel training paradigm, TransIP. TransIP induces anisotropic Transformer-based models to learn SO(3)-isotropy by optimizing their representations in the embedding space. Trained on the Open Molecules (OMol25) dataset, TransIP effectively learns symmetries in the latent space and outperforms data augmentation-based models by 40% to 60%.