Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Constructing a Norm for Children's Scientific Drawing: Distribution Features Based on Semantic Similarity of Large Language Models

Created by
  • Haebom

Author

Yi Zhang, Fan Wei, Jingyi Li, Yan Wang, Yanyan Yu, Jianli Chen, Zipo Cai, Xinyu Liu, Wei Wang, Sensen Yao, Peng Wang, Zhong Wang

Outline

This study utilized a large-scale language model (LLM) and the word2vec algorithm to overcome the limitations of previous studies assessing scientific concept understanding through children's drawings (task-dependent picture content and subjective interpretation by researchers). We analyzed 1,420 children's drawings on nine scientific topics to explore the consistency of their picture representations across topics and to propose a standard for children's scientific drawings. The results confirmed the presence of consistency in most of the drawings, demonstrating high semantic similarity (mostly >0.8). However, we also found a consistency bias, which was independent of LLM accuracy. We also analyzed the correlation between factors such as sample size, abstraction level, and focus and the picture consistency and LLM recognition accuracy, and examined whether these factors reflected the course content. The results confirmed that LLM recognition accuracy was the most sensitive indicator, and was also related to sample size and semantic similarity. Furthermore, we found that consistency between the instructional experiment and the educational objectives was an important factor, with many students tending to focus on the experiment itself rather than the explanation.

Takeaways, Limitations

•
Takeaways:
◦
A novel method for quantitatively analyzing the meaning of children's drawings using large-scale language models is presented.
◦
By revealing the consistency and bias of children's scientific picture representations, we provide a new perspective on the study of assessing children's understanding of scientific concepts.
◦
Identify factors influencing image analysis (sample size, level of abstraction, focus, reflection of class content, etc.).
◦
Contributed to the establishment of standards and criteria for children's science drawing research.
•
Limitations:
◦
As this is an analysis method dependent on the performance of LLM, there is a possibility that the limitations of LLM may affect the research results.
◦
Difficulty in generalizing due to limitations in the graphic data used in the analysis (specific topics, age groups, etc.).
◦
Further analysis is needed to determine the cause of consistency bias.
◦
There is still a possibility that subjective judgment may be involved in the interpretation of the picture.
👍
Made with Slashpage