This is a page that curates AI-related papers published worldwide. All content here is summarized using Google Gemini and operated on a non-profit basis. Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.
Giuliano Martinelli, Tommaso Bonomo, Pere-Llu is Huguet Cabot, Roberto Navigli
Outline
This paper addresses the limitations of existing coreference resolution systems that are evaluated on short or medium-length documents, and presents a new benchmark, BOOKCOREF, for book-scale documents (over 200,000 tokens on average). It is built by developing and leveraging an automatic pipeline that generates high-quality coreference resolution annotations. With BOOKCOREF, we demonstrate that it significantly improves the performance of existing long-length document processing systems (up to +20 CoNLL-F1 points) and presents new challenges arising from book-scale documents. The data and code are publicly available ( https://github.com/sapienzanlp/bookcoref ).
Takeaways, Limitations
•
Takeaways:
◦
We first present BOOKCOREF, a benchmark for resolving book-sized co-references.
◦
Demonstrates potential for performance improvement over existing systems (up to +20 CoNLL-F1 points).
◦
Presenting a new research direction for the development of a long document processing system.
◦
Enabling research through open data and code.
•
Limitations:
◦
Further validation of the accuracy of the automatic annotation generation pipeline is needed.
◦
Lack of specific analysis and solution proposals for new challenges revealed by BOOKCOREF.
◦
A detailed analysis is needed to understand why current models fail to achieve the performance they achieve on small documents on large documents.
