This paper proposes, for the first time, a context-based learning (ICL) theory using an LLM-based transformer to address the low throughput performance issue in dynamic channel environments with the binary exponential backoff scheme used in WiFi 7. To address the problem of high throughput loss due to the fixed node density assumptions of existing model-based approaches (e.g., non-persistent and p-persistent CSMA), we design a transformer-based ICL optimizer that generates predicted collision window thresholds (CWTs) using collision threshold data and query collision instances as input prompts for the transformer. We develop an efficient algorithm that guarantees near-optimal CWT predictions within a limited training time. Considering the difficulty of obtaining perfect data in real-world environments, we present an extended model that allows for erroneous data inputs. NS-3 simulation results demonstrate that our approach achieves faster convergence and near-optimal throughput compared to existing model-based and DRL-based approaches.