Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Exploring Non-Local Spatial-Angular Correlations with a Hybrid Mamba-Transformer Framework for Light Field Super-Resolution

Created by
  • Haebom

Author

Haosong Liu, Xiancheng Zhu, Huanqiang Zeng, Jianqing Zhu, Jiuwen Cao, Junhui Hou

Outline

This paper presents an improvement on the Mamba-based method, which boasts long-range information modeling and linear complexity, to optimize computational cost and performance in Lightweight Image Super-Resolution (LFSR). To address the inefficient and redundant feature extraction problems of conventional multidirectional scanning strategies applied to complex LF data, this paper designs a Subspace Simple Mamba Block (SSMB) based on the Subspace Simple Scanning (Sub-SS) strategy, achieving more efficient and accurate feature extraction. Furthermore, to address the limitations of state space in preserving spatial-angle and disparity information, a two-stage modeling strategy is proposed to more comprehensively explore nonlocal spatial-angle correlations. In the first stage, the Spatial-Angular Residual Subspace Mamba Block (SA-RSMB) is used to extract shallow spatial-angle features. In the second stage, a dual-branch parallel architecture combining the Epipolar Plane Mamba Block (EPMB) and the Epipolar Plane Transformer Block (EPTB) is used to enhance deep epipolar features. Based on these modules and strategies, we propose LFMT, a hybrid Mamba-Transformer framework that integrates the strengths of the Mamba and Transformer models. LFMT enables comprehensive information exploration across spatial, angular, and epipolar domains. Experimental results demonstrate that LFMT significantly outperforms existing state-of-the-art LFSR methods while maintaining low computational complexity on real and synthetic LF datasets.

Takeaways, Limitations

Takeaways:
By improving the efficiency of the Mamba-based method, we reduce the computational cost of LFSR and improve its performance.
Sub-SS strategy and SSMB enable more efficient and accurate feature extraction.
The two-step modeling strategy improves the preservation of spatial-angular and parallax information.
We propose an LFMT framework that combines the strengths of Mamba and Transformer, resulting in improved performance.
We achieve performance that outperforms existing state-of-the-art techniques on real and synthetic datasets.
Limitations:
Further validation of the generalization performance of the proposed method may be required.
Optimization may have been done for a specific dataset, and performance evaluation on other types of LF data is needed.
Further analysis is needed to determine the extent of the reduction in computational complexity and its effectiveness in practical applications.
There may be a lack of detailed explanation of the parameter settings of the Sub-SS strategy and discussion of optimization methods.
👍