Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

AFABench: A Generic Framework for Benchmarking Active Feature Acquisition

Created by
  • Haebom

Author

Valter Schutz, Han Wu, Reza Rezvan, Linus Aronsson, Morteza Haghir Chehreghani

Outline

This paper introduces AFABench, the first benchmark framework for systematically evaluating Active Feature Acquisition (AFA) methods. AFA selectively acquires features from a subset of data instances, attempting to trade off predictive performance and acquisition cost. AFABench encompasses a variety of synthetic and real-world datasets, supports a variety of acquisition policies, and offers a modular design for easy integration of new methods and tasks. We implement and evaluate representative algorithms, including static, greedy, and reinforcement learning-based approaches, and introduce AFAContext, a novel synthetic dataset designed to expose the limitations of greedy selection. The results reveal key trade-offs between various AFA strategies and provide actionable insights for future research. The benchmark code is available on GitHub.

Takeaways, Limitations

•
Takeaways:
◦
Providing a standardized benchmark framework (AFABench) for fair and systematic comparative evaluation of AFA methods.
◦
Wide range of evaluations possible through support for various synthetic and real-world datasets and acquisition policies.
◦
A new synthetic dataset (AFAContext) that exposes the limitations of greedy selection is presented.
◦
Present experimental results demonstrating trade-offs between various AFA strategies and suggesting future research directions.
◦
Expanding research and ensuring reproducibility through open source code disclosure
•
Limitations:
◦
AFABench may not cover all possible AFA methods and datasets.
◦
Further validation of the generalizability of the AFAContext dataset is needed.
◦
Further application and performance evaluation in real-world applications are needed.
👍
Made with Slashpage