In this paper, we propose a novel mixed-precision post-learning quantization technique, Task-Circuit Quantization (TaCQ), to address the performance degradation problem in low-bit (2-3 bit) quantization. TaCQ works by directly conditioning the quantization process on the weight circuit, which is a set of weights related to the performance of a specific task. The weights that are important for the performance of a specific task are kept as 16 bits, and the remaining weights are quantized, thereby efficiently reducing memory usage while minimizing performance degradation. We utilize gradient information to predict the weight changes due to quantization and their impact on task performance, and experimentally demonstrate that it outperforms existing methods on a variety of tasks (QA, mathematical reasoning, text-to-SQL) and models (Llama-3, Qwen2.5) using both general-purpose and task-specific data. In particular, it achieves significant performance improvements over existing state-of-the-art methods in 2-bit and 3-bit quantization environments.
