Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

RefPentester: A Knowledge-Informed Self-Reflective Penetration Testing Framework Based on Large Language Models

Created by
  • Haebom

Author

Hanzheng Dai, Yuanliang Li, Jun Yan, Zhibo Zhang

Outline

This paper proposes RefPentester, a knowledge-based self-reflective framework, to solve the __T212271_____ of large-scale language model (LLM)-based automated penetration testing (AutoPT). Existing LLM-based AutoPTs underperform human experts due to knowledge imbalance, short-term planning, and hallucinations. RefPentester models the penetration testing stages as a seven-stage state machine, selects appropriate tactics and techniques for each stage, and learns from previous failures to address these issues. In the evaluation on Hack The Box's Sau machine, RefPentester demonstrates 16.7% better performance than the baseline GPT-4o, and also has higher success rates for each stage.

Takeaways, Limitations

Takeaways:
We present a novel framework to improve the performance of LLM-based AutoPT.
We sought to overcome the limitations of the LLM through a knowledge-based, self-reflective approach.
Increased efficiency by improving the success rate of each penetration test step.
Practicality was proven through performance verification in a real environment (Hack The Box).
Limitations:
Further research is needed to explore the generalizability of the proposed framework.
Additional testing is needed for different types of systems.
A clear explanation of the scope and limitations of the seven-step state machine model is needed.
There is a lack of discussion about safety and ethical considerations in a real-world penetration testing environment.
👍