Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Conversational Education at Scale: A Multi-LLM Agent Workflow for Procedural Learning and Pedagogic Quality Assessment

Created by
  • Haebom

Author

Jiahuan Pei, Fanghua Ye, Xin Sun, Wentao Deng, Koen Hindriks, Junxiao Wang

Outline

This paper proposes a multi-agent workflow, WikiHowAgent, that leverages large-scale language models (LLMs) to simulate interactive teaching-learning conversations. WikiHowAgent integrates teacher and learner agents, an interaction manager, and an evaluator to facilitate procedural learning and assess educational quality. We present a dataset of 114,296 teacher-learner conversations spanning 17 domains and 727 topics, based on 14,287 tutorials. We use an evaluation protocol that combines computational and criteria-based metrics with human judgment alignment. We demonstrate the effectiveness of the workflow in various settings and provide insights into the capabilities of LLMs across domains. The dataset and implementation are fully open-source.

Takeaways, Limitations

Takeaways:
A new approach to developing interactive teaching-learning systems using LLM.
Providing large-scale datasets covering a variety of domains and topics
Presenting an objective evaluation protocol based on computation and human judgment.
Providing insight into the educational potential and limitations of LLMs.
Increasing reproducibility and scalability of research through open source
Limitations:
Further validation is needed on the generalizability of learning scenarios based on WikiHow data.
Limited applicability to different types of learning and assessment methods
Lack of consideration of LLM's bias and ethical issues
Lack of comparative analysis with human teachers
Further research is needed on application to real-world educational environments.
👍