Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Unveiling the Landscape of LLM Deployment in the Wild: An Empirical Study

Created by
  • Haebom

Author

Xinyi Hou, Jiahao Han, Yanjie Zhao, Haoyu Wang

Outline

This paper presents the results of a large-scale empirical study of the security vulnerabilities of large-scale language models (LLMs) deployed via open-source and commercial frameworks. Through internet-wide measurements, we identified 320,102 publicly available LLM services across 15 frameworks and extracted 158 unique API endpoints, categorized into 12 functional groups. Our analysis revealed that over 40% of endpoints used plain HTTP, and over 210,000 lacked valid TLS metadata. Some frameworks exhibited highly inconsistent API exposures, responding to over 35% of unauthenticated API requests, potentially leading to model or system information leaks. We observed widespread use of insecure protocols, improper TLS configurations, and unauthorized access to critical operations. These security vulnerabilities can lead to serious consequences, including model leaks, system compromise, and unauthorized access.

Takeaways, Limitations

Takeaways:
The security vulnerabilities in LLM services are serious and demonstrate the need for secure default settings and enhanced deployment practices.
We identify security level inconsistencies between various LLM frameworks and suggest ways to improve them.
It highlights the importance of strengthening security by specifically highlighting risks such as model leaks, system corruption, and unauthorized access.
Limitations:
Because this study is based on a snapshot of a specific time period, it may not reflect changes over time.
Since the analysis is limited to services publicly available on the Internet, it does not reflect the security status of LLMs that are distributed privately.
There may be a lack of quantitative assessment of the severity and potential impact of discovered vulnerabilities.
👍