Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Divergent Creativity in Humans and Large Language Models

Created by
  • Haebom

Author

Antoine Bellemare-Pepin (CoCo Lab, Psychology department, Universit e de Montréal , Montreal, QC, Canada, Music department, Concordia University, Montreal, QC, Canada), Fran\c{c}ois Lespinasse (Sociology and Anthropology department, Concordia University, Montreal, QC, Canada), Philipp Th olke (CoCo Lab, Psychology department, Universit e de Montréal , Montreal, QC, Canada), Yann Harel (CoCo Lab, Psychology department, Universit e de Montréal , Montreal, QC, Canada), Kory Mathewson (Mila), Jay A. Olson (Department of Psychology, University of Toronto Mississauga, Mississauga, ON, Canada), Yoshua Bengio (Mila, Department of Computer Science and Operations Research, Universit e de Montréal , Montreal, QC, Canada), Karim Jerbi (CoCo Lab, Psychology department, Universite de Montréal , Montreal, QC, Canada, UNIQUE Center)

Outline

This paper systematically evaluates semantic diversity, which has been an important overlooked factor in discussions on creativity of large-scale language models (LLMs), which have recently been rapidly developed, and compares it with human divergent thinking. Using the state-of-the-art LLM and a dataset of 100,000 humans, we evaluate the performance of LLMs in divergent association tasks and find that LLMs surpass the average human level and approach creative writing ability. However, it falls short of the level of highly creative humans, and we confirm that there are limitations that LLMs currently cannot overcome. Through a human-machine benchmarking framework, we provide an objective measure for the debate on replacing human creative labor by AI, and suggest techniques for improving the semantic diversity of LLMs, such as prompt design and hyperparameter tuning.

Takeaways, Limitations

•
Takeaways:
◦
Empirically demonstrate that LLM can exhibit semantic diversity beyond the average human level.
◦
Presenting a framework to objectively measure and compare the performance differences between highly creative humans and LLMs.
◦
Present specific techniques (prompt design, hyperparameter tuning, etc.) to improve the semantic diversity of LLM.
◦
Provides objective, data-driven analysis of the debate over AI replacing human creative labor.
◦
Raises the need for in-depth exploration of the differences between creative thinking between humans and AI.
•
Limitations:
◦
Current LLMs fall short of the standards of highly creative humans.
◦
The representativeness and homogeneity of the 100,000-person human dataset needs to be reviewed.
◦
Further research is needed on the generalizability and effectiveness of the proposed semantic diversity enhancement technique.
👍
Made with Slashpage