Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

CCL-LGS: Contrastive Codebook Learning for 3D Language Gaussian Splatting

Created by
  • Haebom

Author

Lei Tian, Xiaomin Li, Liqian Ma, Hao Yin, Zirui Zheng, Hefei Huang, Taiqing Li, Huchuan Lu, Xu Jia

Outline

This paper proposes CCL-LGS, a novel framework for 3D semantic understanding. Addressing the challenges faced by existing 2D prior-based methods, which suffer from cross-view semantic inconsistencies due to occlusion, image blur, and view-dependent variations, we propose a method that enhances view-consistent semantic supervision by incorporating multi-view semantic cues. Specifically, we align SAM-generated 2D masks using a zero-shot tracker, extract robust semantic encodings using CLIP, and extract discriminative semantic features by enhancing intra-class compactness and inter-class distinctiveness through the Contrastive Codebook Learning (CCL) module. Unlike existing methods, CCL-LGS explicitly resolves semantic conflicts while maintaining category discriminability, rather than directly applying CLIP to incomplete masks. Experimental results demonstrate that CCL-LGS outperforms existing state-of-the-art methods.

Takeaways, Limitations

Takeaways:
We present a novel framework that can improve the accuracy of 3D semantic understanding by integrating multi-view semantic cues.
Effectively solve cross-view semantic inconsistencies problem by utilizing zero-shot tracker and CLIP and CCL modules.
Experimentally verified superior performance over existing methods.
Limitations:
Lack of analysis of the computational cost and complexity of the proposed method.
Further experiments are needed to evaluate generalization performance across different environments and datasets.
There are parts that depend on the performance of other models such as SAM and CLIP.
👍