Sanja Karilanova: Bridging Spiking Neural Networks and Deep State-Space Models

Abstract

Modern artificial intelligence achieves impressive results, but at a high energy cost. Neuromorphic computing seeks to reduce this cost by using brain-inspired principles. Central to this approach are spiking neural networks (SNNs), in which neurons communicate through sparse binary events called spikes. Separately, in signal processing and control theory, state-space models (SSMs) have long offered a systematic description of dynamical systems. More recently, SSMs have re-emerged as powerful deep learning models for processing long sequences, called deep SSMs. Spiking neurons are themselves dynamical systems, yet there has been limited interaction between SNNs and deep SSMs. This thesis brings these two together in a common framework and uses this common framework to explore two research directions.

The first direction asks how neuron models should be designed to capture richer temporal dynamics while remaining computationally efficient. Drawing on tools from SSMs, we propose enrichments along four axes: (i) We generalise neuron models with a single input and a single output channel to multiple input and multiple output channels, widening the bandwidth constrained by binary spikes. (ii) We generalise the reset mechanism, and show that it can stabilise learning even when the underlying linear dynamics are unstable. (iii) We equip neurons with structured, learnable delays that provide direct access to past inputs. (iv) We allow neurons' dynamics to vary over time, enabling them to track signals that switch between operating modes.

The second direction focuses on the mismatch that arises when the temporal resolution of the training data differs from that of the test data. We propose principled, zero-shot adaptation methods that recover the performance lost to this mismatch by adjusting only the neurons' dynamics parameters. We test the methods both in the single-device setting and in a multi-device federated learning framework aggregating models from devices operating at heterogeneous temporal resolutions.