[공지사항]을 빙자한 안부와 근황 
Show more

Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Grounding Degradations in Natural Language for All-In-One Video Restoration

Created by
  • Haebom

Author

Muhammad Kamran Janjua, Amirhosein Ghasemabadi, Kunlin Zhang, Mohammad Salameh, Chao Gao, Di Niu

Outline

In this paper, we propose an all-in-one video restoration framework that concatenates corruption-aware semantic contexts of video frames into natural language via a foundation model. Unlike previous works, we do not presuppose knowledge of corruptions at training or test time, and instead safely separate the foundation model during inference to learn an approximation of the foundation knowledge without additional cost. We also call for benchmark standardization in the field of all-in-one video restoration, and propose three-task (3D) and four-task (4D) benchmarks in multi-corruption settings, as well as two time-varying composite corruption benchmarks, one of which is a proposed dataset with various snow intensities that naturally simulates the impact of weather deterioration on videos. We compare our proposed method with previous works and report state-of-the-art performance on all benchmarks.

Takeaways, Limitations

•
Takeaways:
◦
We present an interpretable and flexible video restoration framework leveraging a foundational model.
◦
Effective video restoration without any damage knowledge.
◦
Base model separation is possible without additional cost during inference.
◦
Promoting standardization in the field of video restoration by presenting various benchmarks.
◦
The proposed method achieves state-of-the-art performance on various benchmarks.
•
Limitations:
◦
Further validation of the generalization performance of the proposed dataset and benchmarks is needed.
◦
Further research is needed on robustness to different types of damage.
◦
Further analysis is needed to determine the interpretability of the underlying model.
👍
Made with Slashpage