Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Response and Prompt Evaluation to Prevent Parasocial Relationships with Chatbots

Created by
  • Haebom

Author

Emma Rath, Stuart Armstrong, Rebecca Gorman

Outline

This paper raises the issue that forming parasocial relationships with AI agents can have serious, sometimes tragic, consequences for human well-being. Preventing these dynamics is challenging, as parasocial cues emerge gradually in private conversations and not all forms of emotional engagement are detrimental. To address this, we present a simple response evaluation framework that leverages state-of-the-art language models to assess parasocial cues in real time. We test the feasibility of this approach using a small synthetic dataset of 30 conversations (parasocial, flattering, and neutral conversations). Iterative evaluations using five rounds of testing demonstrate that, under a lenient matching rule, all parasocial conversations can be identified while avoiding false positives, typically within the first few exchanges. These results provide preliminary evidence that the evaluation agent can provide a viable solution for preventing parasocial relationships.

Takeaways, Limitations

Takeaways: We demonstrate the feasibility of a simple framework for detecting parasocial relationships in real time by reusing state-of-the-art language models. We demonstrate that parasocial cues can be effectively detected in the early stages of a conversation. We present a novel approach to preventing unhealthy relationships with AI agents.
Limitations: Validated using a small synthetic dataset. Further research is needed to determine generalizability to real-world applications. Detection performance for various types of parasocial relationships and cues needs to be verified. The specific criteria and limitations of the "lenient matching rule" need to be clearly defined. There is a lack of in-depth discussion of the ethical implications.
👍