Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

StreetViewAI: Making Street View Accessible Using Context-Aware Multimodal AI

Created by
  • Haebom

Author

Jon E. Froehlich, Alexander Fiannaca, Nimer Jaber, Victor Tsaran, Shaun Kane

Outline

StreetViewAI is the first accessible street view tool for people with visual impairments. Interactive streetscape mapping tools like Google Street View (GSV) and Meta Mapillary allow users to virtually explore and experience real-world environments through immersive 360-degree imagery, but they are fundamentally inaccessible to people with visual impairments. StreetViewAI solves this problem by combining context-aware multimodal AI, accessible navigation controls, and interactive voice. With StreetViewAI, people with visual impairments can virtually review destinations, explore the open world, and virtually travel across GSV's distributed collection of over 220 billion images and over 100 countries. Through an iterative design process with a mixed-vision team and evaluations with 11 visually impaired users, we demonstrated the value of accessible street view in supporting point-of-interest (POI) surveys and remote route planning. Finally, we list key guidelines for future research.

Takeaways, Limitations

Takeaways: Demonstrates the potential of an accessible Street View tool for the visually impaired. Demonstrates its effectiveness in supporting POI surveys and remote route planning. Effectively integrates multimodal AI, accessible navigation controls, and conversational voice.
Limitations: The number of users evaluated was limited (11). Further research is needed with users of varying visual impairments and skill levels. Continued usability and effectiveness evaluations are needed over time. Further research is needed to determine generalizability across diverse environments and situations.
👍