This is a page that curates AI-related papers published worldwide. All content here is summarized using Google Gemini and operated on a non-profit basis. Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.
This paper addresses how Instruction Fine-Tuning (IFT) can increase the informativeness of large-scale language models (LLMs) at the expense of their accuracy. IFT induces LLMs to generate responses that incorporate long-tail knowledge that was not sufficiently addressed during the pretraining process, resulting in increased informativeness but at the expense of poor generalization to new tasks. This paper experimentally demonstrates how unfamiliar knowledge in the IFT dataset negatively impacts the accuracy of LLMs, and proposes two novel IFT paradigms to address this issue: $UNIT_{cut}$ and $UNIT_{ref}$. $UNIT_{cut}$ identifies and removes unfamiliar knowledge from the IFT dataset, mitigating its impact on model accuracy, whereas $UNIT_{ref}$ trains LLMs to be aware of their uncertainty and explicitly mark it at the end of their responses. Experimental results show that $UNIT_{cut}$ significantly improves the accuracy of LLM, while $UNIT_{ref}$ reduces illusions by maintaining high informativeness and distinguishing between confident and uncertain statements.
Takeaways, Limitations
•
Takeaways:
◦
We empirically demonstrate that unfamiliar knowledge in the IFT dataset has a negative impact on the accuracy of LLM.
◦
We address the trade-off between accuracy and informativeness of LLM by proposing two new IFT paradigms, $UNIT_{cut}$ and $UNIT_{ref}$.
◦
$UNIT_{cut}$ improves the accuracy of LLM, and $UNIT_{ref}$ reduces hallucinations while maintaining informativeness.
•
Limitations:
◦
Further research is needed to determine the generalization performance of the proposed methodology.
◦
Additional experiments on various LLM and IFT datasets are needed.
◦
The accuracy of the process of identifying and removing unfamiliar knowledge in $UNIT_{cut}$ needs to be evaluated.
◦
A more in-depth analysis of the effectiveness of the uncertainty representation method of $UNIT_{ref}$ is needed.
