Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Impoola: The Power of Average Pooling for Image-Based Deep Reinforcement Learning

Created by
  • Haebom

Author

Raphael Trumpp, Ansgar Schafftlein , Mirco Theile, Marco Caccamo

Outline

This paper highlights that increasing model size is a critical factor in improving performance in image-based deep reinforcement learning, and presents research to improve the existing Impala-CNN (a 15-layer ResNet-based image encoder). Instead of flattening the output feature map of Impala-CNN, we propose Impoola-CNN, which utilizes global average pooling. We experimentally demonstrate that Impoola-CNN outperforms existing models, particularly in generalization, on the Procgen benchmark. This performance improvement is particularly pronounced in games without agent-centric observation, and we speculate that it is related to the network's reduced sensitivity to transformations. In conclusion, we emphasize the importance of efficient network design, not just increasing model size.

Takeaways, Limitations

•
Takeaways:
◦
We demonstrate that global average pooling is effective in improving the performance of deep reinforcement learning image encoders through improvements to Impala-CNN.
◦
Impoola-CNN generalizes better on the Procgen benchmark than larger models.
◦
Suggesting that performance improvements can be achieved without increasing model size through improved network design.
◦
By analyzing the impact of agent-centric observation, we suggest that reducing the network's transformation sensitivity can contribute to improved performance.
•
Limitations:
◦
The performance improvements of the proposed Impoola-CNN are limited to the Procgen benchmark. Further research is needed to determine generalization performance in other environments.
◦
There is a lack of a clear explanation for the underlying cause of the performance improvement using global average pooling. Factors other than reduced transformation sensitivity should be considered.
◦
Since this is an improvement based on Impala-CNN, its applicability to other architectures requires further research.
👍
Made with Slashpage