Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

StreetViewAI: Making Street View Accessible Using Context-Aware Multimodal AI

Created by
  • Haebom

Author

Jon E. Froehlich, Alexander Fiannaca, Nimer Jaber, Victor Tsara, Shaun Kane

Outline

StreetViewAI is the first accessible Street View tool for people with visual impairments. Existing interactive streetscape mapping tools like Google Street View (GSV) and Meta Mapillary allow users to virtually explore and experience real-world environments through immersive 360-degree imagery, but are fundamentally inaccessible to people with visual impairments. StreetViewAI solves this problem by combining context-aware multimodal AI, accessible navigation controls, and interactive voice. Blind users can virtually explore GSV's distributed collection of over 220 billion images and over 100 countries, explore destinations in detail, and plan remote routes. StreetViewAI's usability was proven through an iterative design process with a mixed-vision team and evaluations with 11 blind users.

Takeaways, Limitations

Takeaways: Provides Street View accessibility for the visually impaired, offering new possibilities for POI investigation and remote route planning. It presents a design approach that provides effective accessibility through a combination of multimodal AI, accessible navigation controls, and interactive voice. It demonstrates the importance of an iterative design process with user participation.
Limitations: Currently, only 11 visually impaired users are evaluated, and further research with a broader user base is needed. Further research is needed to identify potential issues and areas for improvement during long-term use. Further accessibility testing is also needed for users with various types of visual impairments.
👍