This paper addresses the machine unlearning problem in neural information retrieval (IR) and proposes a new task called Neural Machine UnRanking (NuMuR). This problem has emerged due to the increasing demand for data privacy compliance and selective information removal in neural IR systems. Existing task- or model-agnostic unlearning methods are mainly designed for classification tasks, and therefore are not optimal for NuMuR. This is due to two key challenges. First, neural rankers output non-normalized relevance scores instead of probability distributions, which limits the effectiveness of existing teacher-student distillation frameworks. Second, entangled data scenarios where queries and documents appear simultaneously in the to-be-forgotten and to-be-maintained datasets can degrade the retention performance of existing methods. To address these issues, we propose a dual-objective framework called Contrastive and Consistent Loss (CoCoL). CoCoL consists of (1) a contrastive loss that maintains the performance of entangled samples while reducing the relevance score of the dataset to be forgotten, and (2) a consistent loss that preserves the accuracy of the maintenance dataset. Through extensive experiments on four neural IR models on the MS MARCO and TREC CAR datasets, we show that CoCoL achieves significant forgetting with minimal maintenance and generalization performance loss. Our method enables more effective and controllable data removal than existing techniques.
