Comparing the performance of Hebbian against backpropagation learning using conv nns
# Comparing the performance of Hebbian against backpropagation learning using conv nns
#Omnivore #hebbian
# Highlights
The results on various datasets are compared with those obtained by unsupervised VAE, and the potentials and limitations of the methods are highlighted; we also deemed interesting to report the results of supervised backprop training in our discussion; ⤴️ ^02caac6f
Pros of Hebbian learning:
- Effective for training low-level feature extractors;
- Produces better features than VAE for the classification task;
- Effective for re-training higher network layers in fewer epochs than other approaches;
- Some hybrid combinations of Hebbian and backprop help improving performance in some cases, as can be observed in Appendix 1;
Cons of Hebbian learning:
- Not effective for training intermediate layers;
- Even though HPCA provides a reduction in the gap between unsupervised and supervised methods, the latter are still preferable for end-to-end network training;
- Finding the best combination of Hebbian and backprop layers is not immediate and requires exploring various network configurations. ⤴️ ^2fb11229
our results suggest that the Hebbian approach is suitable for training early feature extraction layers or to re-train the final layers of a pre-trained deep neural network, requiring fewer training epochs than other methods. ⤴️ ^4b1ee85a
Hebbian approaches outperform VAE training ⤴️ ^e0d3b015
github.com/GabrieleLagani/HebbianPCA/tree/hebbpca ⤴️ ^396bc53b