Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Self-Questioning Language Models

Created by
  • Haebom

Author

Lili Chen, Mihir Prabhudesai, Katerina Fragkiadaki, Hao Liu, Deepak Pathak

Outline

This paper presents research on whether large-scale language models can improve their performance by generating questions and answers independently, without external data. To achieve this, we propose an asymmetric self-learning framework called the Self-Questioning Language Model (SQLM). SQLM consists of a proposer that generates questions and a solver that generates answers, trained through reinforcement learning. The proposer aims to generate problems of appropriate difficulty, and the solver is judged correct through majority voting. For coding problems, the proposer generates unit tests, which the solver verifies. We conducted experiments on three benchmarks: three-digit multiplication, algebraic problems from the OMEGA benchmark, and programming problems from Codeforces, demonstrating performance improvements without external data.

Takeaways, Limitations

Takeaways:
We demonstrate that it is possible to improve the inference ability of large-scale language models without external data.
Presenting a data-efficient learning method through a self-learning framework.
Applicability to various problem types (mathematics, coding) has been verified.
Limitations:
Using majority voting as the criterion for determining correct answers is not a perfect way to determine correct answers.
The performance of the proposed framework may vary across benchmarks.
The need for verification of the quality of self-generated data.
👍