DiMA, a latent diffusion framework that uses protein language model representations, presents a robust methodology that generalizes across a variety of protein encoders (8M to 3B parameters). Compared to existing autoregressive, discrete diffusion, and flow-consistent language models, it consistently performs well across extensive experiments using multiple protein representations (ESM-2, ESMc, CHEAP, SaProt) and various evaluation metrics (quality, diversity, novelty, and distribution congruence), generating novel, high-quality, and diverse protein sequences. It also supports conditional generative tasks, such as protein family generation, motif scaffolding and filling, and fold-specific sequence design.