Parameter-efficient image classification with convolutional spiking neural network via fast sigmoid surrogate gradient descent

Abstract

Spiking Neural Networks (SNNs) offer several advantages over traditional Artificial Neural Networks (ANNs), especially in terms of biologically inspired event-driven mechanisms and energy efficiency. This paper aims to demonstrate parameter efficiency of Convolutional Spiking Neural Networks (CSNNs) by training extremely small CSNNs and comparing those with conventional Convolutional Neural Networks (CNNs) of same parameter count and depth, as well as against state-of-the-art SNN implementations. Through experiments conducted on multiple small-scale CSNN models, we observed that CSNNs are able to achieve exceptional parameter efficiency and performance with the help of a fast sigmoid surrogate gradient descent, proving an astonishing balance between accuracy and compactness. The findings fulfill the promise that CSNNs hold as a feasible solution for resource-constrained and low-power applications at the same time, as it demonstrates the biological plausibility of learning in such complex machine learning tasks.