This paper presents a novel framework for automating cardiovascular event adjudication in cardiovascular disease clinical trials using large-scale language models (LLMs). To address the time-consuming, resource-intensive, and inter-adjudicator variability of traditional manual adjudication methods, we develop a two-step approach: an LLM-based pipeline for event information extraction from unstructured clinical data, and an LLM-based adjudication process guided by the Tree of Thoughts approach and Clinical Endpoints Committee (CEC) guidelines. Using cardiovascular event-specific clinical trial data, we achieve an F1 score of 0.82 for event extraction and an accuracy of 0.68 for adjudication. We also present a novel automated metric, the CLEART score, specifically designed to assess the quality of AI-generated clinical inference in cardiovascular event adjudication. This approach demonstrates the potential to significantly reduce adjudication time and cost while maintaining high-quality, consistent, and auditable outcomes in clinical trials. By reducing variability and improving standardization, we can more quickly identify and mitigate risks associated with cardiovascular therapies.
