This page organizes papers related to artificial intelligence published around the world. This page is summarized using Google Gemini and is operated on a non-profit basis. The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.
Conformal Safety Shielding for Imperfect-Perception Agents
Created by
Haebom
Author
William Scarbro, Calum Imrie, Sinem Getir Yaman, Kavan Fatehi, Corina S. Pasareanu, Radu Calinescu, Ravi Mangal
Outline
This paper addresses the problem of safe control of discrete autonomous agents using components learned for imperfect perception (or, more generally, state estimation) from high-dimensional observations. We propose a shield architecture that provides runtime safety guarantees under perceptual errors by restricting the actions available to the agent, modeled as a Markov decision process, as a function of state estimates. This architecture uses conformal predictions for perceptual components to ensure that the set of predicted estimates for each observation contains the true state with a user-specified probability. The shield provides local safety by allowing actions only if they are permissible for all estimates in the predicted set. We also clarify and prove the global safety property of existing shield architectures for perfectly perceptual agents and bound the probability of reaching an unsafe state if the agent always selects the shield-specified action. We illustrate this approach through a case study of an experimental autonomous system that guides an aircraft on a runway using a high-dimensional perceptual DNN.
Takeaways, Limitations
•
Takeaways: A novel solution to the safety control problem of autonomous agents using high-dimensional observations is presented. Runtime safety is guaranteed through a shield structure utilizing convergence prediction. Applicability to a real-world autonomous system (aircraft runway guidance) is demonstrated. Local and global safety properties are proven.
•
Limitations: The performance of the proposed shield structure depends on the accuracy of the consensus prediction. The computational cost of processing high-dimensional observations may be high. Case studies of experimental autonomous systems may be limited to specific domains. The global safety property is guaranteed only if the agent always selects the action specified by the shield.
