This paper addresses the unique challenges posed by human-AI interactions in safety-critical systems. Existing frameworks only partially address these challenges. The complex interplay between requirements for transparency, trustworthiness, and explainability and the need for robust and secure decision-making is the root of these challenges. Therefore, a framework that holistically integrates human and AI capabilities and addresses these concerns is needed, contributing to bridging the critical gap in the design, deployment, and maintenance of safe and effective systems. This paper proposes a holistic conceptual framework for critical infrastructure, adopting an interdisciplinary approach. It integrates traditionally distinct fields such as mathematics, decision theory, computer science, philosophy, psychology, and cognitive engineering, drawing on specialized engineering disciplines, particularly in energy, mobility, and aviation. Its flexibility is further demonstrated through a case study of power grid management.