Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Rethinking the Illusion of Thinking

Created by
  • Haebom

Author

I naki Dellibarda Varela, Pablo Romero-Sorozabal, Eduardo Rocon, Manuel Cebrian

Outline

This paper revisits the argument that the large-scale reasoning model (LRM) lacks reasoning ability, raised in Apple's paper "The Illusion of Thinking". Apple's paper argues that LRM is simply a probabilistic parrot, and presents Towers of Hanoi and River Crossing problems as examples. This paper reproduces and improves the experiments on these two problems, introducing step-by-step prompts and interactive dialogues to show that the conclusions of previous studies are exaggerated. We show that LRM's failure in Towers of Hanoi is due to cognitive limitations as well as output constraints, and its failure in River Crossing is due to an unsolvable problem setting. When limited to solvable problems, LRM easily solves large-scale problems with more than 100 agent pairs. Therefore, LRM is a probabilistic, reinforcement learning-tuned explorer in a discrete state space, and suggests that further detailed analysis is needed for the development of symbolic and long-term reasoning.

Takeaways, Limitations

Takeaways:
Shows that the conclusions of Apple's "The Illusion of Thinking" paper were exaggerated.
Refuting the conventional, simplistic interpretation of LRM's inferential ability.
The causes of LRM failure are analyzed by dividing them into output constraints and cognitive limitations.
We found that step-by-step prompts and interactive dialogues contributed to improving the performance of LRM.
Emphasizes the importance of understanding the discrete state space of LRM.
Provides detailed analysis methods for future research.
Limitations:
Limited to analysis of specific problems (Towers of Hanoi, River Crossing).
Limitations on generalizability to other types of inference problems.
Lack of detailed mechanisms underlying LRM's cognitive limitations.
👍