POP: Prefill-Only Pruning for Efficient Large Model Inference