Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

FEVO: Financial Knowledge Expansion and Reasoning Evolution for Large Language Models

Created by
  • Haebom

Author

Bo Pang, Yalu Ouyang, Hangfei Xu, Ziqi Jia, Panpan Li, Shengzhao Wen, Lu Wang, Shiyong Li, Yanpeng Wang

Outline

In this paper, we propose Financial Evolution (FEVO), a multi-stage augmentation framework for improving the performance of large-scale language models (LLMs) in finance. FEVO extends the financial domain knowledge of LLMs, injects structured inference patterns, and integrates the learned inference ability with domain knowledge through three stages: continuous pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning (RL). For each stage, we construct a high-quality dataset FEVO-Train, and train C32B, S32B, and R32B models based on Qwen2.5-32B. Through the evaluation of seven benchmarks, FEVO-R32B achieves the previous state-of-the-art performance on five financial benchmarks, and significantly outperforms FEVO-R32B-0, which only uses RL, demonstrating the effectiveness of financial domain knowledge augmentation and structured inference.

Takeaways, Limitations

Takeaways:
Presenting an effective multi-level framework (FEVO) to improve LLM performance in finance
Proof of the utility of an approach combining CPT, SFT, and RL
Emphasize the importance of high-quality dataset FEVO-Train
FEVO-R32B model achieves SOTA performance on multiple financial benchmarks
Demonstrating the importance of domain knowledge expansion and structural reasoning
Limitations:
Lack of detailed description of the composition and quality of the FEVO-Train dataset.
Lack of clear discussion of the types and limitations of benchmarks used.
More in-depth comparative analysis with other large-scale language models is needed.
Lack of verification of application and stability in real financial markets
👍