[공지사항]을 빙자한 안부와 근황 
Show more

Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

VIDEE: Visual and Interactive Decomposition, Execution, and Evaluation of Text Analytics with Intelligent Agents

Created by
  • Haebom

Author

Sam Yu-Te Lee, Chengyang Ji, Shicheng Wen, Lifu Huang, Dongyu Liu, Kwan-Liu Ma

Outline

In this paper, we introduce VIDEE, a system that enables advanced text analytics without requiring natural language processing (NLP) expertise. VIDEE is based on a human-agent collaborative workflow and consists of (1) a decomposition phase that uses a Monte Carlo tree search algorithm that integrates human feedback, (2) an execution phase that generates executable text analytics pipelines, and (3) an evaluation phase that integrates LLM-based evaluation and visualization to support users’ validation of the execution results. Through two quantitative experiments and a user study with participants with varying levels of NLP and text analytics experience, we evaluate the effectiveness and usability of VIDEE and present design implications for human-agent collaboration.

Takeaways, Limitations

Takeaways:
Leveraging the LLM to enable advanced text analysis for users without NLP expertise.
Supporting efficient and accurate text analysis through human-agent collaborative workflows.
Verify the usability of the system and suggest future improvement directions through user research.
Presenting the possibility of a practical text analysis system for non-expert users.
Limitations:
This paper lacks a detailed description of specific agent error types and solutions.
Further research is needed on the generalization performance of VIDEE to different types of text data.
There is a need to evaluate the scalability and performance of VIDEE on large datasets.
Additional information is needed on the number and diversity of participants in user studies.
👍