Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

SLTNet: Efficient Event-based Semantic Segmentation with Spike-driven Lightweight Transformer-based Networks

Created by
  • Haebom

Author

Xianlei Long, Xiaxin Zhu, Fangming Guo, Wanyi Zhang, Qingyi Gu, Chao Chen, Fuqiang Gu

Outline

This paper highlights the potential of event-based semantic segmentation in autonomous driving and robotics, leveraging the advantages of event-based cameras (high dynamic range, low latency, and low power consumption). Existing ANN-based segmentation methods suffer from high computational requirements, image frame requirements, and high energy consumption, limiting their efficiency and applicability on resource-constrained edge/mobile platforms. To address these issues, we present SLTNet, a lightweight spike-based Transformer network designed for event-based semantic segmentation. SLTNet extracts rich semantic features while reducing model parameters based on efficient spike-based convolutional blocks (SCBs), and enhances long-range contextual feature interactions through spike-based Transformer blocks (STBs) and binary mask operations. Extensive experiments on the DDD17 and DSEC-Semantic datasets demonstrate that SLTNet achieves up to 9.06% and 9.39% mIoU improvements over state-of-the-art SNN-based methods, while consuming 4.58x less energy and achieving an inference speed of 114 FPS. The source code is publicly available.

Takeaways, Limitations

Takeaways:
An efficient semantic segmentation method using an event-based camera is presented.
Improving energy efficiency and inference speed with lightweight spike-based networks (SLTNet).
Performance improvement over state-of-the-art SNN-based methods (up to 9.06% and 9.39% improvement in mIoU)
Suggesting the possibility of expanding research through open-source code disclosure
Limitations:
Further validation of the generalization performance of the proposed method is needed.
Additional experimental results for various event camera sensors and datasets are needed.
Need for application and performance evaluation for actual autonomous driving and robotic systems
👍