Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Reshaping the Forward-Forward Algorithm with a Similarity-Based Objective

Created by
  • Haebom

Author

James Gong, Raymond Luo, Emma Wang, Leon Ge, Bruce Li, Felix Marattukalam, Waleed Abdulla

Outline

This paper focuses on improving the performance of the forward-forward algorithm, which was proposed to overcome the limitations of the backpropagation algorithm, such as its biological impracticality and global error propagation. Existing forward-forward algorithms have been significantly inferior to the backpropagation algorithm in terms of accuracy and inference efficiency. In this study, we propose the FAUST (Forward-Forward Algorithm Unified with Similarity-based Tuplet loss) algorithm, which integrates a similarity learning framework into the forward-forward algorithm, eliminating the need for multiple forward propagation passes during the inference process. Experimental results using the MNIST, Fashion-MNIST, and CIFAR-10 datasets demonstrate that FAUST significantly improves accuracy compared to existing forward-forward algorithms, narrowing the performance gap with the backpropagation algorithm. Specifically, on the CIFAR-10 dataset, using a simple multilayer perceptron architecture, we achieve an accuracy of 56.22%, approaching the 57.63% accuracy of the backpropagation algorithm.

Takeaways, Limitations

•
Takeaways:
◦
We show that the accuracy of the forward-forward algorithm can be improved by utilizing similarity learning techniques.
◦
Provides a more biologically plausible alternative to the backpropagation algorithm.
◦
Improve the efficiency of the inference process to increase practical applicability.
•
Limitations:
◦
There is still an accuracy gap compared to the backpropagation algorithm on the CIFAR-10 dataset.
◦
Performance evaluation for more complex network structures or large datasets is required.
◦
Further research is needed on the generalization performance of the FAUST algorithm.
👍
Made with Slashpage