This paper proposes the ActiveKD framework, which integrates knowledge distillation (KD) into active learning (AL). It aims to minimize annotation costs by leveraging the zero-shot and few-shot capabilities of large-scale vision-language models (VLMs). The key is to leverage the VLM's structural prediction bias to capture generalizable output patterns beneficial to student learning. To achieve this, we propose a sample selection strategy called Probabilistic CoreSet (PCoreSet), which maximizes coverage in the probability space rather than the feature space. PCoreSet probabilistically selects diverse unlabeled samples, facilitating efficient transfer of teacher knowledge under a limited annotation budget.
