Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Reference-Aligned Retrieval-Augmented Question Answering over Heterogeneous Proprietary Documents

Created by
  • Haebom

Author

Nayoung Choi, Grace Byun, Andrew Chung, Ellie S. Paek, Shinsun Lee, Jinho D. Choi

Outline

This paper proposes a Retrieval-Augmented Generation (RAG)-based question-answering (QA) system to address the challenges of information access due to the vast volume and unstructured nature of internal corporate documents. Using crash test documents from the automotive industry as an example, we focus on processing diverse data types, maintaining data confidentiality, and ensuring traceability between generated answers and the original documents. To achieve this, we present the RAG-QA framework, which includes a data pipeline that transforms various document types into a structured corpus and QA pairs, an in-house privacy-preserving architecture, and a lightweight reference matcher that links answers to supporting content. Our experimental results demonstrate improvements in factual accuracy, informativeness, and usability compared to existing systems when applied to the automotive industry.

Takeaways, Limitations

Takeaways:
Presenting an effective RAG-QA framework for utilizing internal corporate documents.
Implementing functions for processing various types of data (multimodal) and maintaining confidentiality
Improved reliability by ensuring traceability of generated answers
Suggesting the potential to improve information accessibility and decision-making efficiency in various industries, including the automotive industry.
Improved reliability by measuring performance through human and LLM evaluators
Limitations:
The proposed framework is specialized for the automotive industry, requiring further validation when applied to other industries.
Consideration needs to be given to the costs and resource consumption of system construction and maintenance.
Scalability and potential performance degradation for large-scale data processing
Further review of the reliability of LLM evaluators is needed.
Further research is needed on generalization performance for various types of multimodal data.
👍