Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Stopping Criteria for Value Iteration on Concurrent Stochastic Reachability and Safety Games

Created by
  • Haebom

Author

Marta Grobelna, Jan K\v{r}et insk y, Maximilian Weininger

Outline

This paper considers two-player zero-sum simultaneous stochastic games (CSGs) with reachability and safety objectives on graphs. While degenerate classes such as Markov decision processes and turn-based stochastic games can be solved using linear or quadratic programming, in practice, Value Iteration (VI) outperforms other approaches and is the most commonly implemented method. This practical performance makes VIs an attractive alternative to standard theoretical solutions using existential real number theory for CSGs. Existing VIs start with an approximation of the target value for each state and iteratively update it, traditionally terminating when two successive approximations approach ε-approximation. However, these termination criteria lack guarantees regarding the accuracy of the approximation. In this paper, we present a bounded (interval) VI for CSGs that compensates for over-approximation sequences that converge to the standard VI and terminates when both over- and under-approximation approaches ε-approximation.

Takeaways, Limitations

Takeaways: We present a boundary value iteration algorithm for CSGs, providing guarantees on the accuracy of approximations. This overcomes the limitations of existing value iteration algorithms and yields more reliable results.
Limitations: The computational complexity and practical performance of the boundary value iteration algorithm presented in this paper are insufficient. Experimental evaluation on CSGs of various sizes and complexities is needed. Furthermore, further research may be needed to determine the algorithm's effectiveness for specific types of CSGs.
👍