Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

An Auditable Pipeline for Fuzzy Full-Text Screening in Systematic Reviews: Integrating Contrastive Semantic Highlighting and LLM Judgment

Created by
  • Haebom

Author

Pouria Mortezaagha, Arya Rahgozar

Outline

This paper presents a scalable and auditable pipeline based on fuzzy logic to address expert screening, a major bottleneck in systematic reviews (SRs). Instead of using traditional dichotomous inclusion/exclusion criteria, inclusion/exclusion is reframed as a fuzzy decision problem, dividing the literature into overlapping chunks and embedding them with a domain-adaptive model. For each criterion (Population, Intervention, Outcome, Study Approach), contrastive similarity (inclusion-exclusion cosine) and ambiguity margin are computed, and a Mamdani fuzzy controller is used to map these to graded inclusion scores with dynamic thresholds in a multi-label setting. A large-scale language model (LLM) assesses highlighted intervals with tertiary labels, confidence scores, and criterion-referenced evidence, reducing fuzzy membership instead of excluding them when evidence is insufficient. In a pilot study targeting the POPCORN (Population Health Modeling Consensus Reporting Network for Noncommunicable Diseases) positive gold set (16 full-text articles; 3,208 chunks), the proposed fuzzy system achieved high recall (Population 81.3%, Intervention 87.5%, Outcome 87.5%, Study Approach 75.0%), surpassing the statistically significant baseline. The inclusion rate for papers meeting all criteria was 50.0%, higher than the baselines (25.0% and 12.5%). The screening time was reduced from approximately 20 minutes to less than 1 minute, significantly reducing costs. In conclusion, the system combining fuzzy logic, contrastive highlighting, and LLM judgment provides high recall, robust evidence, and end-to-end traceability.

Takeaways, Limitations

Takeaways:
A novel method to streamline and improve the accuracy of the specialized screening process for systematic literature reviews is presented.
Effectively handling uncertainty and achieving high recall using fuzzy logic and LLM.
Dramatically reduce screening time and costs.
Achieved high inter-model agreement and human-machine agreement.
Improved auditability through end-to-end traceability.
Limitations:
The sample size of the pilot study was small (16 subjects).
Limited evaluation using only all positive gold sets.
There is a need to verify the generalizability of the results to various types of literature.
Further research is needed on the generalization performance and limitations of domain adaptation models.
Further research is needed on fuzzy logic parameter optimization.
👍