In this paper, we propose a novel Stochastic Optimal Control (SOC)-based diffusion sampler, the Non-equilibrium Annealed Adjoint Sampler (NAAS), which leverages annealed reference dynamics without relying on importance sampling, building on recent advances in learning-based diffusion samplers that aim to sample from non-normalized probability densities. NAAS enables efficient and scalable training using a concise adjoint system inspired by adjoint matching. We demonstrate the effectiveness of our approach on various tasks, including sampling from classical energy landscapes and molecular Boltzmann distributions. Unlike existing methods that rely on importance sampling, which suffer from high variance and limited scalability, NAAS addresses these issues.