MoQE proposes a quantization inference framework based on the Mixed Expert (MoE) architecture to improve model efficiency and reduce deployment costs. MoQE combines multiple quantization variants into specialized "quantization experts" and dynamically routes input data to the most appropriate expert based on its characteristics. Experiments on the ImageNet, WikiText, C4, and OpenWebText datasets using ResNet, LLaMA, and Qwen models demonstrate that MoQE achieves performance comparable to state-of-the-art quantization models without significantly increasing inference latency.
