This paper explores how to instruct robots using natural language. While existing methods require a large amount of language-annotated demonstration data for each robot, this paper presents an approach that separates "what to achieve" from "how to achieve it." "What to achieve" can leverage external observation data, while "how to achieve it" depends on the specific robot's configuration. To achieve this, we propose a reward model, Video-Language Critic, which can be trained on diverse robot data using contrastive learning and temporal ranking. The reward model, trained on Open X-Embodiment data, demonstrates twice the sample efficiency compared to using only sparse rewards on Meta-World tasks, demonstrating its effectiveness even under large domain gaps. Furthermore, it achieves higher sample efficiency than existing language-conditional reward models in the challenging task generalization setting of Meta-World. Unlike existing models, our model does not use binary classification, static images, or utilize temporal information from video data.
