Transformer-based time series prediction methods have achieved excellent results, but existing Transformers have limited sequence modeling due to their excessive emphasis on temporal dependencies. This problem incurs additional computational costs but does not lead to improved performance. This paper finds that the performance of the Transformer is highly dependent on the embedding method used for effective representation learning. To address this issue, we extract multivariate features to amplify the effective information captured in the embedding layer. This results in a multidimensional embedding that delivers richer and more meaningful sequence representations. Specifically, we introduce Hybrid Temporal and Multivariate Embedding (HTME). The HTME extractor integrates a lightweight temporal feature extraction module with a carefully designed multivariate feature extraction module, providing complementary features to achieve a balance between model complexity and performance. By combining HTME with the Transformer architecture, we propose HTMformer, and leverage the enhanced feature extraction capabilities of the HTME extractor to build a lightweight predictor. Experiments on eight real-world datasets demonstrate that the proposed method outperforms existing baselines in both accuracy and efficiency.
