Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Vital Insight: Assisting Experts' Context-Driven Sensemaking of Multi-modal Personal Tracking Data Using Visualization and Human-In-The-Loop LLM

Created by
  • Haebom

Author

Jiachen Li, Xiwen Li, Justin Steinberg, Akshat Choube, Bingsheng Yao, Xuhai Xu, Dakuo Wang, Elizabeth Mynatt, Varun Mishra

Outline

This paper points out that in the study of human behavior monitoring using smartphones and wearable sensors, there is a difficulty in deriving high-level insights that are context-aware beyond the existing simple behavior recognition (e.g., physical activity recognition). The research team explored solutions to this problem through three user studies targeting 21 experts. To this end, we developed Vital Insight (VI), an LLM-based prototype system, and presented a method to support and visualize the insight derivation process (sensemaking) through human-computer interaction of multimodal passive sensing data. Through VI, we observed the interactions of experts and developed an expert sensemaking model that explains the transition process between data representation and AI-assisted inference. Finally, we present design implications for designing an AI-augmented visualization system that better supports the sensemaking process of experts in multimodal health sensing data.

Takeaways, Limitations

•
Takeaways:
◦
We present a novel approach to derive high-dimensional insights from multi-modal sensing data by leveraging LLM-based systems.
◦
In-depth understanding of the sensemaking process of experts and suggestions for designing AI-augmented visualization systems based on this.
◦
Presenting a strategy to complement the limitations of AI systems and increase reliability through a human-in-the-loop approach.
◦
Presenting the possibility of developing a system applicable to various application fields (e.g. health care).
•
Limitations:
◦
Currently, it is a prototype system and further research is needed on scalability and generalizability in real environments.
◦
Because the number of participating experts was limited, further validation of the generalizability of the results is needed.
◦
Because the results of this study are limited to a specific type of sensing data and expert group, further research is needed on other types of data or expert groups.
◦
Need to review LLM's bias and reliability of interpretation.
👍
Made with Slashpage