This paper discusses diffusion bridge, a deep learning methodology for sampling from non-normalized distributions. Recent research has shown that log-variance (LV) loss consistently outperforms rKL loss when computing the inverse Kullback-Leibler (rKL) gradient using reparameterization techniques. However, in diffusion bridges with learnable diffusion coefficients, we show that LV loss does not produce the same gradient as rKL loss. Therefore, this paper argues that LV loss in diffusion bridges does not represent an optimization objective that can be justified by data processing inequalities, as rKL loss does. Our analysis demonstrates that rKL loss using the log-derivative trick (rKL-LD) not only avoids these conceptual issues but also consistently outperforms LV loss. Experimental results on various types of diffusion bridges demonstrate that samplers trained with rKL-LD loss achieve better performance. Furthermore, rKL-LD requires less hyperparameter optimization and exhibits more stable learning behavior.
