Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

CTourLLM: Enhancing LLMs with Chinese Tourism Knowledge

Created by
  • Haebom

Author

Qikai Wei, Mingzhi Yang, Jinqiang Wang, Wenwei Mao, Jiabo Xu, Huansheng Ning

Outline

This paper proposes CTourLLM, a large-scale language model (LLM) specialized in Chinese cultural tourism. To address the lack of tourism knowledge in existing LLMs, we build a new dataset called Cultour, which consists of a tourism knowledge database, travelogue data, and tourism QA data. Using this dataset, we fine-tune a Qwen-based model using supervised learning. To evaluate the performance of CTourLLM, we propose a new evaluation metric called Relevance, Readability, and Availability (RRA), and perform both automated and human evaluations. Experimental results show that CTourLLM outperforms ChatGPT by 1.21 on the BLEU-1 scale and 1.54 on the Rouge-L scale. The Cultour dataset is publicly available.

Takeaways, Limitations

Takeaways:
Contributing to improving tourism-related services through the development and publication of high-quality LLMs specializing in Chinese cultural tourism.
Building a new dataset, Cultour, to provide a resource for future research.
Presenting new indicators for LLM performance evaluation through the presentation of RRA evaluation criteria.
Validation of the effectiveness of the proposed model through improved performance compared to ChatGPT.
Limitations:
Currently focused solely on Chinese cultural tourism, it is necessary to review expansion into other fields.
Further research is needed to determine the objectivity and generalizability of the RRA evaluation criteria.
Further review is needed of the size and diversity of the dataset used for evaluation.
Limitations of evaluation methods that simply focus on improving BLEU and ROUGE scores.
👍