Daily Arxiv

This page organizes papers related to artificial intelligence published around the world.
This page is summarized using Google Gemini and is operated on a non-profit basis.
The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.

Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generation

Created by
  • Haebom

Author

Ruizhe Li, Chen Chen, Yuchen Hu, Yanjun Gao, Xi Wang, Emine Yilmaz

Outline

To address the challenge of context attribution, which involves attributing responses from Retrieval-Augmented Generation (RAG) models to specific context segments, this study proposes Attribute Response to Context (ARC-JSD), a novel methodology leveraging Jensen-Shannon Divergence. ARC-JSD efficiently and accurately identifies essential context sentences without additional fine-tuning, gradient calculations, or surrogate modeling. It demonstrates superior accuracy and computational efficiency over existing surrogate-based methods on various RAG benchmarks, and analyzes specific attention heads and MLP layers to provide insight into the model's internal workings.

Takeaways, Limitations

Takeaways:
An efficient methodology is presented to solve the context attribution problem without additional fine-tuning.
Demonstrated high accuracy and computational efficiency compared to existing methodologies in various RAG benchmarks.
Provides insight into the inner workings of the RAG model (analyzing the attention head and MLP layers).
Limitations:
Specific Limitations is not stated in the paper.
👍