This paper addresses a fundamental question about the phenomenon of memorization of training data in large-scale language models (LLMs): how can we characterize the difficulty of memorizing training data? Through experiments using the OLMo family of open models, we propose the entropy-memorization law, which suggests that data entropy is linearly correlated with memorization scores. Furthermore, through a case study of memorizing highly randomized strings (gibberish), we observe that these strings, despite their apparent randomness, exhibit unexpectedly low empirical entropy compared to the extensive training corpus. Adopting the same strategy used to discover the entropy-memorization law, we derive Dataset Inference (DI), a simple yet effective approach for distinguishing training and test data.
