This paper presents a novel generative model, ByteGen, to address the challenging problem of generative modeling of high-frequency order book (LOB) dynamics. Existing approaches suffer from limitations due to their reliance on simplified probabilistic assumptions or, in the case of modern deep learning models like Transformer, tokenization techniques that affect the high-precision numerical properties of the data. ByteGen overcomes these limitations by directly processing the raw byte stream of LOB events. To represent market messages without information loss, we design a 32-byte compressed binary format and address the problem with an autoregressive next-byte prediction task. By completely eliminating feature engineering and tokenization, we learn market dynamics from a basic representation. By applying the H-Net architecture, we utilize a dynamic chunking mechanism to discover the inherent structure of market messages without predefined rules. By training on over 34 million events from CME Bitcoin futures, we successfully reproduce key features of financial markets, including realistic price distributions, heavy-tail returns, and burst event timing.
