This page organizes papers related to artificial intelligence published around the world. This page is summarized using Google Gemini and is operated on a non-profit basis. The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.
Linguistic Neuron Overlap Patterns to Facilitate Cross-lingual Transfer on Low-resource Languages
Created by
Haebom
Author
Yuemei Xu, Kexin Xu, Jian Zhou, Ling Hu, Lin Gui
Outline
This paper presents a data-efficient method for improving the performance of large-scale language models (LLMs) for low-resource languages. To improve zero-shot cross-linguistic contextual learning (X-ICL) for low-resource languages without expensive fine-tuning, we propose a simple yet effective method, BridgeX-ICL, from a language-bridge perspective. Unlike previous studies that focus on language-specific neurons, BridgeX-ICL explores whether neuron sharing can improve the cross-linguistic performance of LLMs. We construct neuron probe data using the correct answer data from the MUSE bilingual dictionary and define a subset of language-redundant neurons to ensure full activation of these fixed neurons. We then propose an HSIC-based metric to quantify the internal linguistic spectrum of LLMs based on the overlapping neurons, guiding the selection of optimal bridges. We validate the effectiveness of BridgeX-ICL through experiments on four cross-lingual tasks across seven diverse language families and 15 language pairs (including high- and low-resource language pairs and medium- and low-resource language pairs) and provide empirical insights into the underlying multilingual mechanisms of LLM. The code is publicly available at https://github.com/xuyuemei/BridgeX-ICL .
