Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Assistance or Disruption? Exploring and Evaluating the Design and Trade-offs of Proactive AI Programming Support

Created by
  • Haebom

Author

Kevin Pu, Daniel Lazaro, Ian Arawjo, Haijun Xia, Ziang Xiao, Tovi Grossman, Yan Chen

Outline

This paper develops and experiments with a design-probing LLM agent called Codellaborator to evaluate the impact of proactive, predictive assistance from AI agents on programming efficiency during the programming process. Codellaborator initiates programming assistance based on editor activity and task context. We compare and analyze the pros and cons of AI assistance in three interface variations: prompt-only, proactive, and proactive agents with presence and context. Experiments with 18 participants revealed that proactive agents improved efficiency compared to prompt-only approaches, but also introduced workflow disruption. However, presence indicators and interaction context support mitigated these disruptions and enhanced users' awareness of the AI process. In conclusion, our study contributes to the design exploration and evaluation of proactive AI systems and suggests design implications for AI-integrated programming workflows. We also highlight tradeoffs between user control, ownership, and code comprehension, suggesting that proactive assistance should be tailored to the programming process.

Takeaways, Limitations

Takeaways:
Predictive AI agents demonstrate promise for increased programming efficiency.
Presence indicators and interactive context support mitigate workflow disruptions caused by AI agents and enhance users' understanding of AI processes.
Provides Takeaways for AI integrated programming workflow design.
Provides critical insights for design exploration and evaluation of predictive AI systems.
Limitations:
The negative impact of predictive AI assistance on user control, ownership, and code comprehension must be considered.
The number of participants was limited (N=18), limiting generalizability.
The results may be specific to a specific programming environment and task.
Lack of specific methodologies for pre-emptive adjustments to the programming process.
👍