[공지사항]을 빙자한 안부와 근황 
Show more

Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Beyond Model Base Selection: Weaving Knowledge to Master Fine-grained Neural Network Design

Created by
  • Haebom

Author

Jialiang Wang, Hanmo Liu, Shimin Di, Zhili Wang, Jiachuan Wang, Lei Chen, Xiaofang Zhou

Outline

This paper is about integrating machine learning capabilities into database systems to solve the high computational cost incurred by existing automatic neural network model selection. Existing studies suggest a method to query the model records with the highest performance indicators for a specific task by organizing the benchmark repository as a model base (MB), but it has limitations in that it overlooks the fine and evolving relational dependencies between various task queries and model architecture variations, which fails to achieve optimal matching and effectively improves the model. In this paper, we propose M-DESIGN, a curated model knowledge base (MKB) pipeline for neural network improvement, to overcome these limitations. M-DESIGN uses a knowledge weaving engine that adaptively weaves prior insights on model architecture modifications into an adaptive query problem to reframe model improvement. Given a user’s task query, it rapidly matches and iteratively improves candidate models by leveraging a graph relational knowledge schema that explicitly encodes data properties, architecture variations, and pairwise performance differences as joinable relations. This schema supports fine-grained relational analysis of architecture adjustments and drives a predictive query planner that can detect and adapt to out-of-distribution (OOD) tasks. In this paper, we implement M-DESIGN for graph analysis tasks, enriching existing benchmarks with structured metadata that includes three graph tasks and 22 graph datasets, providing 67,760 graph model data records. Experimental results show that M-DESIGN provides optimal models for 26 out of 33 data-task pairs within a limited budget.

Takeaways, Limitations

Takeaways:
We present a novel method to integrate machine learning capabilities into database systems to improve the efficiency of neural network model selection.
We propose the M-DESIGN framework to select and improve the optimal model by considering relational dependencies between model architecture variations and task queries.
Verification of the effectiveness of M-DESIGN through experimental results on graph analysis tasks.
We present a method for building a model knowledge base by adding structured metadata to existing benchmarks.
Limitations:
Since M-DESIGN has been applied only to graph analysis tasks, further research is needed on its generalizability to other types of tasks.
Although they said they were looking for the optimal model within a limited budget, there was a lack of clear explanation of the definition and limits of the budget.
Lack of discussion about the scalability and maintainability of MKB.
Adaptability to OOD (out-of-distribution) tasks has been proven, but there is a lack of specific analysis on the level and limitations of its performance.
👍