This is a page that curates AI-related papers published worldwide. All content here is summarized using Google Gemini and operated on a non-profit basis. Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.
This paper presents Technical-Embeddings, a novel framework for optimizing semantic retrieval of technical documents in hardware and software development. It focuses on solving the challenges of understanding and retrieving complex technical content by leveraging large-scale language models (LLMs). It expands user queries to better capture user intent and enhances dataset diversity to enrich the fine-tuning process of the embedding model. Furthermore, it applies summary extraction techniques to encode key information in technical documents and improve their representation. Soft prompting is used to fine-tune a dual-encoder BERT model, and separate learning parameters for query and document context are used to capture subtle semantic differences. Evaluation results on two public datasets, RAG-EDA and Rust-Docs-QA, demonstrate that Technical-Embeddings significantly outperforms baseline models in both precision and recall. This demonstrates the effectiveness of integrating query expansion and contextual summarization to improve information access and comprehension in technical fields. This study advances the Retrieval-Augmented Generation (RAG) system and presents a novel method for efficient and accurate technical document retrieval in engineering and product development workflows.
Takeaways, Limitations
•
Takeaways:
◦
A framework for optimizing technical document retrieval using LLM is presented.
◦
Improve search performance by incorporating query expansion and context summarization.
◦
Capturing subtle semantic differences through fine-tuning using soft prompting techniques.
◦
Contribute to the advancement of RAG systems and suggest potential improvements to engineering and product development workflows.
◦
Experimentally verified performance improvement over existing models on RAG-EDA and Rust-Docs-QA datasets.
•
Limitations:
◦
The datasets used may be limited (only two datasets are used: RAG-EDA and Rust-Docs-QA).
◦
Generalization performance for other types of technical documents or more complex queries requires further study.
◦
Further research may be needed to determine optimal parameter settings for soft prompting.
◦
Additional experiments and verification are required for application in actual industrial environments.
