Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

EEG2TEXT-CN: An Exploratory Study of Open-Vocabulary Chinese Text-EEG Alignment via Large Language Model and Contrastive Learning on ChineseEEG

Created by
  • Haebom

Author

Jacky Tai-Yu Lu, Jung Chiang, Chi-Sheng Chen, Anna Nai-Yun Tung, Hsiang Wei Hu, Yuan Chiao Cheng

Outline

EEG2TEXT-CN is one of the first open-vocabulary EEG-to-text generation frameworks for Chinese. Based on a biologically based EEG encoder (NICE-EEG) and a small pre-trained language model (MiniLM), it aligns multi-channel brain signals with natural language representations through mask pre-training and contrastive learning. Using a subset of the Chinese EEG dataset (each sentence is aligned with approximately 10 Chinese characters and 128-channel EEG recorded at 256 Hz), it segments EEG into character-wise embeddings and predicts whole sentences in zero-shot settings. The decoder is trained with teacher-forced and padded masks to handle variable-length sequences. Evaluation results on over 1,500 training-validation sentences and 300 separate test samples show promising lexical alignment with a maximum BLEU-1 score of 6.38%. Although syntactic fluency remains a challenge, this study demonstrates the feasibility of non-phonetic, cross-modal language decoding from EEG. This opens new directions for multilingual brain-text research and lays the foundation for future Chinese-based cognitive-language interfaces.

Takeaways, Limitations

Takeaways:
We present an open-vocabulary EEG-to-text generation framework for Chinese.
Demonstrating the feasibility of non-phonetic, cross-modal language decoding.
Presenting new directions in multilingual brain-text research and laying the foundation for Chinese-based cognitive-linguistic interfaces.
Effective alignment of EEG signals and natural language expressions via mask pretraining and contrastive learning.
Limitations:
Syntactic fluency remains a challenge.
There is room for performance improvement with a BLEU-1 score of 6.38%.
The size of the dataset used may be relatively small.
👍