Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

NVS-SQA: Exploring Self-Supervised Quality Representation Learning for Neurally Synthesized Scenes without References

Created by
  • Haebom

Author

Qiang Qu, Yiran Shen, Xiaoming Chen, Yuk Ying Chung, Weidong Cai, Tongliang Liu

Outline

In this paper, we propose a novel method for quality assessment of neural network-based view synthesis (NVS), called NVS-SQA. Existing NVS quality assessment methods rely on full-reference methods such as PSNR, SSIM, and LPIPS, which have limitations due to the absence of dense reference views and the difficulty of obtaining human perceptual labels. To solve these problems, NVS-SQA proposes a method to learn reference-free quality representations through self-supervised learning without human labels. Instead of following the assumptions of existing self-supervised learning, we use heuristic cues and quality scores as learning objectives, and improve the learning efficiency through a special contrastive pair preparation process. Experimental results show that NVS-SQA significantly outperforms 17 existing reference-free methods and 16 full-reference methods.

Takeaways, Limitations

Takeaways:
It effectively solves the lack of dense reference views and the difficulty of securing human labels in the existing NVS quality assessment Limitations.
We present a self-supervised learning-based, no-reference quality assessment method that can reduce data collection and labeling costs.
It showed excellent performance, significantly outperforming existing no-reference and full-domain reference methods.
Limitations:
Further research may be needed to explore the generalizability of the proposed heuristic cues and quality scores.
It may be optimized only for certain types of NVS models, and its generalization performance on a wider range of NVS models needs to be further validated.
Due to the nature of self-directed learning, the interpretability of the learning process may be somewhat limited.
👍