MolLangBench is a comprehensive benchmark designed to evaluate molecular-language interface tasks, such as molecular structure recognition, editing, and generation using language prompts. To ensure accurate, clear, and deterministic output, recognition tasks were constructed using automated cheminformatics tools, and editing and generation tasks were curated through rigorous expert annotation and validation. MolLangBench supports a variety of molecular representations and language interfaces, including linear strings, molecular images, and molecular graphs. The state-of-the-art model (GPT-5) achieved 86.2% and 85.5% accuracy for the recognition and editing tasks, respectively, but only 43.0% accuracy for the generation task, limiting its performance.
