This is a page that curates AI-related papers published worldwide. All content here is summarized using Google Gemini and operated on a non-profit basis. Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.
This paper presents LLM-QL, a novel methodology for improving dense retrieval performance using large-scale language models (LLMs). It leverages the generative power of LLMs to introduce an auxiliary task that maximizes query likelihood (QL). Furthermore, it enhances LLM's global information modeling capabilities through two components: Attention Block (AB) and Document Corruption (DC). AB blocks attention to tokens preceding the last token in a document, while DC corrupts the document by masking some tokens in the prediction. Experimental results using the MS MARCO and BEIR datasets demonstrate that LLM-QL outperforms other LLM-based retrieval models.
Takeaways, Limitations
•
Takeaways:
◦
A novel method for improving dense search performance by leveraging the generative power of LLM is presented.
◦
Improving Global Information Modeling in LLM with QL Maximization Auxiliary Tasks
◦
Validation of the effectiveness of AB and DC components
◦
Demonstrated superior performance compared to other LLM-based retrieval models on MS MARCO and BEIR datasets.
•
Limitations:
◦
Further research is needed on the generalization performance of the methodology presented in this paper.
◦
Experimental results on other types of LLM or datasets are limited.
◦
Further analysis is needed to determine optimal hyperparameter settings for AB and DC.
