To overcome the limitations of in-context learning (ICL) in reinforcement learning (RL) environments, this paper proposes the Retrieval-Augmented Decision Transformer (RA-DT), which uses a memory mechanism to retrieve only partial paths relevant to the current context from past experience. RA-DT employs a domain-independent search component that requires no training and outperforms existing methods in grid-world environments, robot simulations, and procedurally generated video games. Notably, it achieves high performance even with short context lengths. This paper identifies the limitations of existing ICL methods in complex environments, suggests future research directions, and presents datasets for the four environments in which it was used.
