This paper highlights that speculative decoding, which accelerates large-scale language model inference, relies on a fixed speculation length, which is not optimal in large-scale batch service environments with diverse requests. Therefore, this paper explores new directions for dynamic adaptation by investigating a new type of post-test diagnostic signal. To this end, we propose the Dynamic Speculative Decoding Engine (DSDE), a training-free framework based on two main components: first, a prediction signal based on the variance of the Kullback-Leibler (KLD) divergence, which diagnoses the local stability of the generation; and second, an adaptive speculation length upper bound to mitigate delay issues at each sequence decoding. Experimental results demonstrate the potential of using KLD-based stability signals for dynamic adaptation. Algorithms guided by these signals achieve end-to-end latency competitive with best-in-class benchmarks and exhibit excellent robustness across a variety of workloads. This robustness is particularly valuable in low-capacity regimes, where maintaining diagnostic utility is challenging for the proposed signal. In conclusion, these findings validate that posterior signals are a crucial component for building more robust and intelligent LLM inference systems, and highlight promising directions for future research on dynamic speculation length adaptation.
