This paper re-evaluates the prevailing belief that pre-trained (PT) models and fine-tuning outperform meta-learning algorithms in small-sample learning. Using diverse datasets, we compare PT with model-agnostic meta-learning (MAML) under the same architecture, optimizer, and training conditions until convergence. We rigorously verify statistical significance using the effect size (Cohen's d) and analyze the data by calculating the coefficient of formal diversity. The results show that PT outperforms MAML when the dataset has low formal diversity, and MAML outperforms PT when the dataset has high formal diversity. However, the effect size is less than 0.2, indicating a statistically insignificant difference. We also conducted experiments on a large dataset, including 21 small-sample learning benchmarks and the Meta-Dataset, and found no significant difference in experiments using GPT-2 on the Openwebtext dataset. Therefore, we conclude that pre-trained models do not always outperform meta-learning models and that the formal diversity of the dataset is an important factor.
