Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Do Vendi Scores Converge with Finite Samples? Truncated Vendi Score for Finite-Sample Convergence Guarantees

Created by
  • Haebom

Author

Azim Ospanov, Farzan Farnia

Outline

This paper analyzes the statistical convergence of Vendi score and RKE score, which are existing methods for evaluating the diversity of generative models without reference data, and presents a new method to solve the computational cost problem of Vendi score. The existing Vendi score requires eigenvalue decomposition of a high-dimensional matrix, which has limitations in applying it to large-scale datasets. In this paper, we propose $t$-truncated Vendi score that truncates the eigenspectrum, improving it to converge stably even with a limited number of samples. In addition, we show that Nystr om and FKEA approximation methods converge to the asymptotic limit of $t$-truncated Vendi score. On the other hand, we prove that RKE score guarantees universal convergence for all kernel functions. Through experiments, we show that Vendi score using Nystr om and FKEA converges closely to $t$-truncated Vendi score, and analyze the correlation with the diversity of image and text data.

Takeaways, Limitations

•
Takeaways:
◦
We present a $t$-truncated Vendi score that converges reliably even with limited sample sizes, enabling the evaluation of generative model diversity on large-scale datasets.
◦
We show that the computational cost can be efficiently reduced by using the Nystr om and FKEA approximation methods.
◦
We prove the universal convergence of the RKE score.
◦
We validate the validity of the proposed method through experimental results on image and text data.
•
Limitations:
◦
There may be a lack of clear guidance on setting the $t$ value for the $t$-truncated Vendi score.
◦
Additional experiments with different types of generative models and data may be required.
◦
The performance of the proposed method may depend on the kernel function used.
👍
Made with Slashpage