Daily Arxiv

This page organizes papers related to artificial intelligence published around the world.
This page is summarized using Google Gemini and is operated on a non-profit basis.
The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.

JALMBench: Benchmarking Jailbreak Vulnerabilities in Audio Language Models

Created by
  • Haebom

Author

Zifan Peng, Yule Liu, Zhen Sun, Mingchen Li, Zeren Luo, Jingyi Zheng, Wenhan Dong, Xinlei He, Xuechao Wang, Yingjie Xue, Shengmin Xu, Xinyi Huang

Outline

We present JALMBench, a comprehensive benchmark for evaluating the security of audio language models (ALMs). JALMBench comprises a dataset containing 11,316 text samples and 245,355 audio samples (over 1,000 hours), supporting 12 major ALMs, four text-based attack methods, four audio-based attack methods, and five defense methods. Using JALMBench, we provide in-depth analysis of attack effectiveness, topic sensitivity, voice diversity, and architecture, and explore attack mitigation strategies at the prompt and response levels.

Takeaways, Limitations

Takeaways:
Contribute to identifying potential security vulnerabilities in ALM.
Provides an integrated framework for comparing and evaluating various attack methods and defense strategies.
Suggesting research directions to improve the safety of ALM.
Limitations:
Unknown (not mentioned in the abstract of the paper Limitations).
👍