This paper presents a geometric framework for analyzing memorization phenomena in diffusion models using the sharpness of the log probability density. We mathematically justify the effectiveness of previously proposed score difference-based memorization metrics and propose a novel memorization metric that captures sharpness in the early stages of image generation in latent diffusion models, providing early insights into potential memorization phenomena. Leveraging this metric, we develop a mitigation strategy that optimizes early noise in the generation process using a sharpness-aware regularization term.
