This paper introduces FGBench, a novel dataset containing 625,000 molecular feature inference problems, aimed at improving the performance of large-scale language models (LLMs) leveraging functional group (FG) information in chemistry. FGBench accurately annotates and localizes functional groups within molecules, strengthening the connection between molecular structures and textual descriptions and facilitating the development of more interpretable and structure-aware LLMs. It encompasses regression and classification tasks for 245 different functional groups across three categories (single-functional group influence, multi-functional group interactions, and direct molecular comparisons). Benchmark results from state-of-the-art LLMs demonstrate that current LLMs struggle with feature inference at the functional group level. The FGBench methodology is expected to serve as a foundation for generating novel question-answer pairs with functional group-level information, enabling LLMs to better understand fine-grained molecular structure-property relationships. The dataset and evaluation code are publicly available on GitHub.
