ModaRoute is an LLM-based intelligent routing system that dynamically selects the optimal modality for multi-modal video retrieval. Existing dense text captioning methods achieve 75.9% in Recall@5, but require expensive offline processing and miss important visual information in 34% of clips where scene text is not captured by ASR. ModaRoute analyzes query intent and predicts information needs to achieve 60.9% in Recall@5 while reducing computational overhead by 41%. It uses GPT-4.1 to route queries to ASR (speech), OCR (text), and visual indexes, and improves efficiency with an average of 1.78 modalities per query compared to full search (3.0 modalities). Evaluation results on 1.8 million video clips show that intelligent routing provides a practical solution for scaling multi-modal retrieval systems, reducing infrastructure costs while maintaining competitive effectiveness for real-world deployments.
