This paper systematically evaluates state abstraction approaches in Deep Reinforcement Learning (DRL) that approximate action metrics (specifically, similarity metrics) and apply them to representation spaces. While previous research has demonstrated robustness to task-irrelevant noise, the source of improved metric estimation accuracy and performance remains unclear. This study benchmarks five recent approaches, conceptually unified as isometric embeddings with various design options, using various noise settings across 20 state-based and 14 pixel-based tasks (370 task configurations in total). In addition to the final return, we evaluate the denoising factor to quantify the encoder's ability to filter out interference. To further elucidate the effectiveness of metric learning, we propose and evaluate an independent metric estimation setting where the encoder is affected only by metric loss. Finally, we release a modular open-source codebase to enhance reproducibility and support future metric learning research.
