This paper addresses the optimization problem of port container handling by integrating quayside crane double cycling (QCDC) and minimizing yard rehandling. Recognizing the interdependence between QCDC loading sequences and yard planning, we propose QCDC-DR-GA, a hybrid genetic algorithm (GA) that optimizes both maximizing the number of double cycles (DCs) and minimizing the number of yard rehandlings. QCDC-DR-GA employs specialized crossover and mutation strategies. Extensive experiments on various vessel sizes demonstrate that QCDC-DR-GA reduces total operation time for large vessels by 15-20% compared to existing methods. Statistical validation using a two-tailed t-test confirms significant improvements at the 5% significance level. This study highlights the inefficiencies of separate optimization approaches and demonstrates the need for integrated algorithms in port operations.