Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

VIDEE: Visual and Interactive Decomposition, Execution, and Evaluation of Text Analytics with Intelligent Agents

Created by
  • Haebom

Author

Sam Yu-Te Lee, Chenyang Ji, Shicheng Wen, Lifu Huang, Dongyu Liu, Kwan-Liu Ma

Outline

This paper introduces VIDEE, a system that enables even novice data analysts to perform advanced text analytics using intelligent agents. Based on a human-agent collaborative workflow, VIDEE consists of (1) a decomposition phase utilizing a Monte-Carlo Tree Search algorithm that incorporates human feedback; (2) an execution phase that generates an executable text analytics pipeline; and (3) an evaluation phase that integrates LLM-based evaluation and visualization to support user validation of the execution results. We evaluate the effectiveness of VIDEE through two quantitative experiments and analyze common agent errors. Furthermore, we demonstrate the system's usability and analyze user behavior patterns through a user study with participants with varying levels of NLP and text analytics experience. The findings suggest a design Takeaways for human-agent collaboration, validate the practicality of VIDEE for non-expert users, and inform future improvements to intelligent text analytics systems.

Takeaways, Limitations

Takeaways:
This paper presents the possibility of developing a system that enables even non-experts to easily perform advanced text analysis by utilizing large-scale language models (LLMs).
We demonstrate that human-agent collaborative workflows can increase the efficiency and improve the accuracy of text analysis processes.
Provides important Takeaways for designing human-agent collaborative systems through analysis of user experiences at various levels.
The practicality and usability of the VIDEE system were empirically verified.
Limitations:
This paper lacks a detailed description of the specific performance indicators and limitations of the VIDEE system.
More detailed information is needed about the scale of user studies and the diversity of participants.
There is a need for discussion on the reliability and limitations of LLM-based assessments.
Further research is needed on the generalization performance of VIDEE to various types of text data.
👍