The proliferation of large-scale language model (LLM) applications since 2022 has raised both expectations and concerns about the use of LLM in education. This study aims to teach students how to effectively use LLM to improve their learning. To this end, we propose a new concept, 'pedagogical prompting', to elicit learning-centered responses from LLM, and designed a learning intervention that goes from conceptual design to empirical research in a real-world educational environment targeting early-stage undergraduate-level computer science education (CS1/CS2). We collected information necessary for the instructional design through a survey of instructors (N=36), and designed and developed the learning intervention through an interactive system including scenario-based instruction. The effectiveness of the learning intervention was evaluated through pre- and post-tests targeting CS novice students (N=22), and the results confirmed that learners' LLM-based pedagogical help-seeking skills were improved, their positive attitudes toward the system were improved, and their willingness to use pedagogical prompting in the future was increased. The contributions of this study include (1) a theoretical framework for instructional prompting, (2) empirical insights into current instructors’ attitudes toward instructional prompting, and (3) promising results for the design of a learning intervention that includes interactive learning tools and scenario-based instruction and LLM-based help-seeking training. The approach of this study has the potential to be implemented more broadly in classrooms and integrated into tools such as ChatGPT to promote learning-centered use of generative AI.
