Timely and accurate detection of colon polyps plays a crucial role in the diagnosis and prevention of colon cancer. In this study, we propose a novel, lightweight, and efficient polyp detection framework that combines the Local Outlier Factor (LOF) algorithm to filter out noisy data and the YOLO-v11n deep learning model. In an experimental study using a public dataset, we applied 5-fold cross-validation, removed outliers using LOF, and trained the model using YOLO-v11n to improve polyp detection performance.