This is a page that curates AI-related papers published worldwide. All content here is summarized using Google Gemini and operated on a non-profit basis. Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.
Position: Machine Learning Conferences Should Establish a “Refutations and Critiques” Track
Created by
Haebom
Author
Rylan Schaeffer, Joshua Kazdan, Yegor Denisov-Blanch, Brando Miranda, Matthias Gerstgrasser, Susan Zhang, Andreas Haupt, Isha Gupta, Elyas Obbad, Jesse Dodge, Jessica Zosa Forde, Koustuv Sinha, Francesco Orabona, Sanmi Koyejo, David Donoho
Outline
This paper points out the problem of erroneous or flawed research being published in academic journals due to the rapid development of the machine learning (ML) field, and proposes the establishment of a “Refutations and Critiques (R&C)” track within the academic journal as a solution to this problem. The R&C track aims to strengthen the self-correcting function of the research ecosystem through critical review of existing research. The paper discusses the track design, review principles, potential problems, etc., and presents a case of refuting a paper presented at ICLR 2025 oral presentation.
Takeaways, Limitations
•
Takeaways:
◦
It can contribute to improving research reliability and strengthening self-correction capabilities in the field of machine learning.
◦
The R&C track provides a systematic process to identify and correct faulty research early.
◦
It can increase the transparency and accountability of research.
•
Limitations:
◦
There is a lack of specific plans for the design and operation of R&C tracks.
◦
Ensuring the quality control and fairness of review of rebuttal and critical research is an important task.
◦
The R&C track has the potential to lead to excessive debate or unproductive activity.
◦
Researcher participation and acceptance of the R&C track may be low.
