We develop a framework for the duality of the Kolmogorov structure function h x (α) to enable computationally feasible complexity proxies. We establish a mathematical analogy between information-theoretic constructions and statistical mechanics, and introduce appropriate partition functions and free energy functions. We explicitly prove the Legendre-Fenchel duality between structure functions and free energies, show a detailed balance of Metropolis kernels, and interpret the acceptance probability in terms of information-theoretic scattering amplitudes. It is shown that variance, such as the susceptibility of model complexity, peaks precisely at the loss-complexity trade-off, which is interpreted as a phase transition. Practical experiments with linear and tree-based regression models verify these theoretical predictions, and we explicitly show the interplay between model complexity, generalization, and overfitting thresholds.
