QZhou-Embedding is a general-purpose contextual text embedding model developed using the Qwen2.5-7B-Instruct model. It features an integrated multi-task framework that incorporates data transformation methods that integrate diverse text datasets and task-specific learning strategies to enhance model training efficiency. It enhances semantic richness and sample difficulty through a data synthesis pipeline utilizing the LLM API, and employs a two-stage learning strategy of retrieval-focused pre-training and global task fine-tuning. It achieves state-of-the-art performance on the MTEB and CMTEB benchmarks, and also demonstrates top performance on tasks such as reranking and clustering. This demonstrates that high-quality, diverse data is crucial for improving retrieval model performance, and leveraging the generative capabilities of LLM can contribute to improved embedding model performance. The model weights are open sourced from HuggingFace under the Apache 2.0 license, and evaluation code and instructions are available on GitHub for reproducibility.
