In this paper, we present the OpenMiniSafety safety dataset, consisting of 1,067 difficult questions, to evaluate the safety and reliability impact of quantization techniques for improving the efficiency of large-scale language models (LLMs). Using this dataset, we release 4,268 annotated question-answer pairs for four LLMs (both quantized and precision versions) and evaluate 66 quantized model variants using four post-training quantization (PTQ) and two quantization-aware training (QAT) methods on four safety benchmarks (including human-centered evaluation). Our results show that both PTQ and QAT can degrade safety alignment, while QAT techniques such as QLORA or STE are less safe. We highlight that no single method consistently outperforms the others across benchmarks, precision settings, or models, demonstrating the need for safety-conscious compression strategies. Also, precision-specific methods such as QUIK and AWQ (4-bit), AQLM and Q-PET (2-bit) outperform their target precision, which means that these methods are not better at compression, but rather different approaches.
