This paper explores the parallel performance of speculative decoding (SD), a promising technique for accelerating large-scale language model (LLM) inference. To address the bottleneck caused by serial execution of existing SD methods, we propose SpecBranch , a novel framework inspired by branch prediction in modern processors . SpecBranch introduces parallel speculative branches to mitigate expected rejections and enhances parallelism through adaptive draft length and a combination of implicit and explicit model confidence levels. Experimental results on various models and benchmarks demonstrate that SpecBranch achieves 1.8x to 4.5x speedup compared to autoregressive decoding, while maintaining the same sampling distribution and reducing rollback tokens by 50% even under poorly aligned models.
