RandAR is a decoder-only visual autoregressive (AR) model that can generate images with arbitrary token order. While existing decoder-only AR models rely on predefined generation order, RandAR removes this inductive bias and provides novel capabilities in decoder-only generation. The core design enables arbitrary order by inserting a “position-indicating token” that indicates the spatial location before the next image token to be predicted. RandAR trained with randomly permuted token sequences (a more difficult task than fixed-order generation) achieves comparable performance to existing raster-order models. More importantly, decoder-only transformers trained with random order acquire novel capabilities. To address the efficiency bottleneck of AR models, RandAR adopts parallel decoding with KV-Cache at inference time, achieving a 2.5x speedup without compromising generation quality. RandAR also supports inpainting, outpainting, and resolution extrapolation in a zero-shot manner.
