This paper proposes "ambiguity-aware ordinal emotion representations," a novel framework for emotion recognition that simultaneously considers the ambiguity and dynamic nature of emotions. Unlike previous studies that ignore emotional ambiguity or treat it as a static variable, this study presents an approach that models emotional ambiguity as a temporal rate of change. Using two emotion datasets, RECOLA and GameVibe, we evaluate the proposed method for constrained (arousal, valence) and unconstrained (immersion) continuous emotion tracking. Results show that ordinal emotion representations outperform existing ambiguity-aware models in unconstrained labels and achieve the highest performance in Concordance Correlation Coefficient (CCC) and Signed Differential Agreement (SDA) scores, demonstrating their effectiveness in modeling the dynamics of emotion tracking. For constrained labels, ordinal emotion representations outperform SDA, demonstrating their superior ability to capture relative changes in annotated emotion tracks.