Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Uncertainty-Guided Face Matting for Occlusion-Aware Face Transformation

Created by
  • Haebom

Author

Hyebin Cho, Jaehyup Lee

Outline

This paper proposes a novel face matting framework, FaceMat, that estimates a high-resolution alpha matte that separates occluding elements from the face to address the occlusion problem (when hands, hair, accessories, etc. obscure the face) that degrades the performance of face filters. To achieve this, we present FaceMat, a face matting framework that accounts for uncertainty without trimaps. FaceMat is trained using a teacher-student model learning pipeline. The teacher model predicts both alpha matte and pixel-wise uncertainty, and this uncertainty information is then used to spatially adaptively guide the student model. Unlike existing methods, FaceMat operates without auxiliary inputs (trimaps or segmentation masks) and improves the synthesis strategy by clearly distinguishing skin into foreground and background occlusions. Furthermore, we conduct experiments using a newly constructed, large-scale synthetic dataset, CelebAMat, and demonstrate that our approach outperforms existing state-of-the-art methods across various benchmarks. The source code and the CelebAMat dataset are publicly available.

Takeaways, Limitations

Takeaways:
A novel face matting method that addresses the occlusion problem of face filters.
Proposing an efficient FaceMat framework that does not require a trimap.
Improved mating performance and generalization performance by leveraging uncertainty information.
Release of the large-scale synthetic dataset CelebAMat
Efficiency suitable for real-time applications
Skin/occlusion differentiation for clear synthesis strategies
Limitations:
Currently, we have conducted experiments using the synthetic dataset CelebAMat, so further performance evaluation on real-world data is required.
Due to limitations of synthetic datasets, further validation of generalization performance for various occlusion types in the real world is needed.
There is a lack of detailed analysis of FaceMat's computational cost and real-time processing performance.
👍