This paper proposes Network Automatic Relevance Determination (NARD), an extension of Automatic Relevance Determination (ARD) for linear probabilistic models. NARD models the sparse relationship between inputs X and outputs Y while simultaneously capturing the correlation structure between Y. It increases the sparsity of the model by identifying and removing irrelevant features using a matrix normal prior distribution containing sparsity-inducing parameters. Algorithmically, iteratively updates the precision matrix and the relationship between Y and refined inputs. To improve computational efficiency, we propose Sequential NARD, which sequentially evaluates features, and the Surrogate Function Method, which leverages efficient approximations of marginal likelihoods and simplifies the computation of the determinant and inverse of intermediate matrices. Combining the Sequential Update and Surrogate Function methods further reduces computational costs. The computational complexity per iteration of the three methods is reduced to O(m³+p³), O(m³+d²), and O(m³+p²), respectively, where p is the number of features ultimately used in the model and p << d. We demonstrate significant improvements in computational efficiency and comparable performance on synthetic and real-world datasets.
