This paper presents a method for interpreting the internal workings of large-scale language models trained on code, focusing on applications requiring trustworthiness, transparency, and semantic robustness. We propose a global posterior interpretability framework, Code Concept Analysis (CoCoA), which clusters contextualized token embeddings into human-interpretable concept groups, thereby uncovering the lexical, syntactic, and semantic structures emerging in the representational space of the code language model. We propose a hybrid annotation pipeline that combines static analysis-based phrase alignment with a prompt-engineered large-scale language model (LLM) to scalably label latent concepts across levels of abstraction. Experimental evaluations across multiple models and tasks demonstrate that CoCoA remains stable under semantically preserving perturbations (average cluster sensitivity index, CSI = 0.288) and discovers concepts that evolve predictably with fine-tuning. A user study on a programming language classification task demonstrates that concept-enhanced explanations clarify token roles and improve human-centered explainability by 37% compared to token-level attribution using unified gradients.
