Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

"I see models being a whole other thing": An Empirical Study of Pre-Trained Model Naming Conventions and A Tool for Enhancing Naming Consistency

Created by
  • Haebom

Author

Wenxin Jiang, Mingyu Kim, Chingwo Cheung, Heesoo Kim, George K. Thiruvathukal, James C. Davis

Outline

This paper presents the first empirical investigation into the naming practices of pretrained models (PTMs) in the Hugging Face PTM registry. We present the results of a survey of 108 Hugging Face users, analyzing their PTM naming practices and differences from existing software package naming practices. The survey results reveal a discrepancy between engineers' preferences and current PTM naming practices. We then introduce DARA, the first automated DNN architecture assessment technique designed to detect PTM naming inconsistencies. DARA identifies model types with 94% accuracy based solely on architectural information, and achieves over 70% performance with other architectural metadata. We also highlight potential use cases for automated naming tools, such as model validation, PTM metadata generation and validation, and plagiarism detection. This study provides a foundation for automating naming inconsistency detection and suggests future research on automated tools for standardizing package naming, improving model selection and reuse, and enhancing PTM supply chain security.

Takeaways, Limitations

Takeaways:
We conduct the first empirical study of PTM naming practices, revealing discrepancies between engineers' preferences and the current status.
We achieved high accuracy by developing a technology called DARA that automatically detects PTM naming inconsistencies.
We present the potential of automated naming tools in various fields, including model validation, metadata generation and verification, and plagiarism detection.
Laying the foundation for strengthening PTM supply chain security.
Limitations:
The study was limited to the Hugging Face PTM registry. Generalizability to other PTM registries or repositories requires further research.
DARA's performance relies on architectural information, so if architectural information is insufficient or inaccurate, performance may degrade.
The development and implementation of automated naming standardization tools requires further research.
👍