This study presents BenchRL-QAS, a reinforcement learning (RL)-based quantum architecture search (QAS) benchmarking framework for variational quantum algorithm tasks on 2-8 qubit systems. We systematically evaluate nine different RL agents, including value-based and policy-gradient methods, on quantum problems such as variational eigenvalue computation, quantum state diagonalization, variational quantum classification (VQC), and state preparation, under both noise-free and noisy execution settings. To ensure fair comparisons, we propose a weighted ranking metric that integrates accuracy, circuit depth, gate count, and training time. The results demonstrate that no single RL method universally outperforms, and that performance varies depending on the task type, number of qubits, and noise conditions. This strongly supports the principle of no free lunch in RL-QAS. Furthermore, we observe that carefully selected RL algorithms outperform baseline VQC in RL-based VQC. BenchRL-QAS establishes the most comprehensive benchmark for RL-based QAS to date, and the code and experiments are publicly available for reproducibility and future development.
