Daily Arxiv

This page organizes papers related to artificial intelligence published around the world.
This page is summarized using Google Gemini and is operated on a non-profit basis.
The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.

APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation

Created by
  • Haebom

Author

Ravin Kumar

Outline

In this paper, we propose APTx Neuron, a novel unified neural computational unit that integrates nonlinear activation and linear transformation into a single trainable expression. APTx Neuron is derived from the APTx activation function, eliminating the need for separate activation layers and improving computational efficiency and architectural simplicity. The proposed neuron has the form $y = \sum_{i=1}^{n} ((\alpha_i + \tanh(\beta_i x_i)) \cdot \gamma_i x_i) + \delta$, where all parameters $\alpha_i$, $\beta_i$, $\gamma_i$, and $\delta$ are trainable. We validate the APTx Neuron-based architecture on the MNIST dataset, achieving a test accuracy of up to $96.69$ within 11 epochs using approximately 332K trainable parameters. These results highlight the superior expressive power and computational efficiency of APTx Neurons compared to conventional neurons, and suggest a new paradigm for integrated neuron design and architectures built on top of them.

Takeaways, Limitations

Takeaways:
Computational efficiency and architectural simplicity are improved by integrating nonlinear activation and linear transformation.
We demonstrate the performance of APTx Neuron by achieving high accuracy on the MNIST dataset.
We present new architectural design possibilities through integrated neuron design.
We increased the reproducibility of our research by making the source code public.
Limitations:
Validation was only performed on the MNIST dataset, and further verification of generalizability to other datasets and complex problems is required.
Further analysis and performance evaluation are required for comparison with existing neurons.
There is a lack of detailed analysis of parameter tuning of APTx Neuron.
👍