In self-supervised reinforcement learning (RL), a key challenge is for agents to learn diverse skills to prepare for unknown future tasks. Scalability and evaluation remain challenges. Identifying meaningful skills can be obscured by high-dimensional feature spaces, and assessing skill diversity requires a fixed notion of what "diversity" means, making comparisons difficult and potentially leaving diverse forms of diversity unexplored. To address these challenges, this paper uses the Vendi score, a sample diversity measure that applies ecological ideas to machine learning, allowing users to specify and evaluate desired forms of diversity. VendiRL facilitates skill evaluation using this metric and presents a unified framework for learning diverse skill sets. VendiRL utilizes different similarity functions to induce different forms of diversity, supporting skill diversity pre-learning in novel, richly interactive environments where optimization for diverse forms of diversity may be desirable.
