CoT-RAG is a novel inference framework that improves the performance of complex tasks for large-scale language models (LLMs). To address the limitations of existing Chain-of-Thought (CoT) inference, such as the unreliability of the inference and lower inference performance under code prompts than under natural language prompts, we present three key design features. First, we construct a CoT based on a knowledge graph, which enhances inference reliability by leveraging the knowledge graph to regulate the LLM's inference chain generation. Second, we integrate a learnable knowledge case-aware RAG into the knowledge graph, which retrieves relevant subcases and subexplanations to provide learnable information to the LLM. Third, we implement pseudoprogram prompting, which encourages the LLM to execute inference tasks as pseudoprograms, thereby enhancing logical rigor. Evaluation results on nine public datasets across three inference tasks demonstrate significant accuracy improvements of 4.0% to 44.3% over state-of-the-art methods. Testing on four domain-specific datasets also demonstrates superior accuracy and efficient execution, highlighting its practicality and scalability.
