Daily Arxiv

This page organizes papers related to artificial intelligence published around the world.
This page is summarized using Google Gemini and is operated on a non-profit basis.
The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.

SingMOS-Pro: An Comprehensive Benchmark for Singing Quality Assessment

Created by
  • Haebom

Author

Yuxun Tang, Lan Liu, Wenhao Feng, Yiwen Zhao, Jionghao Han, Yifeng Yu, Jiatong Shi, Qin Jin

SingMOS-Pro: A Dataset for Automatic Singing Quality Assessment

Outline

This study highlights the importance of song quality assessment in light of the advancement of singing voice generation technology. We introduce SingMOS-Pro, a dataset for automatic singing quality assessment. SingMOS-Pro is an extended version of the original SingMOS, offering a wider range and diversity of annotations, including lyrics, melody, and overall quality in addition to overall quality. It contains 7,981 song clips generated from 41 models and 12 datasets, each of which received at least five ratings from professional annotators. Furthermore, we explore how to effectively utilize MOS data annotated with various standards and benchmark widely used evaluation methods in SingMOS-Pro, providing a robust baseline and practical reference for future research. The dataset is available at https://huggingface.co/datasets/TangRain/SingMOS-Pro .

Takeaways, Limitations

Takeaways:
We provide a new dataset, SingMOS-Pro, for automatic song quality assessment research.
It provides a variety of annotations including lyrics, melody, and overall quality, allowing for a comprehensive evaluation.
It contains a huge amount of song clips generated from various models and datasets.
Reliability and consistency were ensured through ratings from professional commentators.
We explore various ways to utilize MOS data and conduct benchmarking to provide standards.
Limitations:
There is no specific mention of Limitations in the paper itself.
👍