Reranking using large-scale language models (LLMs) demonstrates powerful performance, but its high computational requirements hinder its practical deployment. This paper proposes new metrics, such as Ranking Metric per PetaFLOP (RPP) and Query Per PetaFLOP (QPP), to evaluate the efficiency of LLM-based reranking. Furthermore, we develop an interpretable FLOPs estimator that can estimate FLOPs of LLM-based reranking without requiring experiments. Using the proposed metrics, we evaluate various LLM-based reranking methods and explore their efficiency-effectiveness tradeoffs.
