Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Blending 3D Geometry and Machine Learning for Multi-View Stereopsis

Created by
  • Haebom

Author

Vibhas Vats, Md. Alimoor Reza, David Crandall, Soon-heung Jung

Outline

While existing multi-view stereoscopic (MVS) methods primarily rely on photometric and geometric consistency constraints, modern learning-based algorithms often rely on planar sweep algorithms to infer 3D geometry and apply explicit geometric consistency (GC) checks only in a postprocessing step, leaving the learning process itself unaffected. In this study, we present GC MVSNet plus plus, a novel method that actively enforces geometric consistency of reference view depth maps across multiple source views (multi-view) and scales (multi-scale) during the learning process (see Figure 1). This integrated GC check directly penalizes geometrically inconsistent pixels, significantly accelerating the learning process and reducing the number of training iterations by half compared to other MVS methods. Furthermore, we present a densely connected cost regularization network with two unique block designs (simple and feature-dense) optimized to leverage dense feature connections for improved regularization. Extensive experiments demonstrate that the proposed method achieves state-of-the-art performance on the DTU and BlendedMVS datasets and ranks second on the Tanks and Temples benchmark. GC MVSNet plus plus is the first method to enforce multi-view, multi-scale supervised geometric consistency during training. The code is open source.

Takeaways, Limitations

Takeaways:
MVS performance improvement and learning speed improvement are achieved by strengthening the geometric consistency of multi-view and multi-scale during the learning phase.
DTU achieves state-of-the-art performance on the BlendedMVS dataset and achieves excellent performance on the Tanks and Temples benchmark.
Enhanced regulatory effectiveness through densely connected cost regulation networks.
Ensure reproducibility and extensibility through open code.
Limitations:
Lack of specific discussion on the performance limitations and potential for improvement of the proposed method.
Further validation of generalization performance across diverse datasets and scenarios is needed.
Lack of clarity regarding dependencies on specific hardware or software environments.
👍