This paper deals with Sparse Autoencoders (SAEs), which aim to decompose the activation space of a large-scale language model (LLM) into human-interpretable potential directions or features. Increasing the number of features in an SAE leads to feature splitting, which is a phenomenon in which hierarchical features are split into more fine-grained features (e.g., “mathematics” is split into “algebra”, “geometry”, etc.). However, this paper shows that sparse decomposition and splitting of hierarchical features are not robust. In particular, features with a seemingly single meaning are not properly activated and are “absorbed” into child features, which is called feature absorption. This phenomenon is revealed to occur in the process of optimizing sparsity in SAEs when the underlying features form a hierarchical structure. In this paper, we present a metric for detecting absorption in SAEs and conduct experimental validation on hundreds of LLM SAEs. We suggest that simply changing the size or sparsity of SAEs is not enough to solve this problem. Finally, we discuss fundamental theoretical issues that need to be addressed before LLM can be robustly and large-scalely interpreted using SAE, as well as potential solutions to these issues.
