LEMUR is an open-source dataset and framework that provides a large collection of PyTorch-based neural networks across a variety of tasks (classification, segmentation, detection, natural language processing, etc.). All models follow a unified template, and their configurations and results are stored in a structured database, ensuring consistency and reproducibility. It integrates automatic hyperparameter optimization via Optuna, statistical analysis, and visualization tools, and provides an API for seamless access to performance data. It is designed to be extensible, allowing the addition of new models, datasets, or metrics while maintaining compatibility. By standardizing implementations and unifying evaluations, it aims to accelerate AutoML research, enable fair benchmarking, and reduce barriers to large-scale neural network experimentation.
