Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

LyS at SemEval 2025 Task 8: Zero-Shot Code Generation for Tabular QA

Created by
  • Haebom

Author

Adri an Gude, Roi Santos-R ios, Francisco Prado-Vali no, Ana Ezquerro, Jes us Vilares

Outline

This paper describes our participation in SemEval 2025 Task 8 (Tabular Question Answering). We developed a zero-shot pipeline that leverages large-scale language models to generate functional code that extracts relevant information from tabular data based on input questions. The approach is a modular pipeline centered around a core code generation module, with additional components that improve extraction accuracy by identifying relevant columns and analyzing data types. If the generated code fails, an iterative improvement process is implemented, incorporating error feedback into new generation prompts to enhance robustness. We demonstrate that zero-shot code generation is a viable approach for tabular question answering without task-specific fine-tuning, ranking 33rd out of 53 participating teams during the testing phase.

Takeaways, Limitations

Takeaways: We present zero-shot code generation as an effective approach for tabular question-answering. We demonstrate the potential for performance enhancements through a modular pipeline and an iterative improvement process based on error feedback.
Limitations: Ranked 33rd out of 53 teams without any task-specific tuning, indicating room for improvement. More sophisticated error analysis and handling, as well as better handling of diverse data types, may be needed. This result clearly demonstrates the limitations of the zero-shot approach.
👍