# To Spike or Not To Spike: A Digital Hardware Perspective on Deep Learning
Acceleration

Fabrizio Ottati
Politecnico di Torino
Torino, Italy
<EMAIL_ADDRESS>
Delft University of Technology
Delft, The Netherlands
<EMAIL_ADDRESS>
& Chang Gao
Delft University of Technology
Delft, The Netherlands
<EMAIL_ADDRESS>
& Qinyu Chen
University of Zurich, ETH Zurich
Zurich, Switzerland
<EMAIL_ADDRESS>
& Giovanni Brignone
Politecnico di Torino
Torino, Italy
<EMAIL_ADDRESS>
& Mario R. Casu
Politecnico di Torino
Torino, Italy
<EMAIL_ADDRESS>
& Jason K. Eshraghian
University of California Santa Cruz
Santa Cruz, California, USA
<EMAIL_ADDRESS>
& Luciano Lavagno
Politecnico di Torino
Torino, Italy
<EMAIL_ADDRESS>
Corresponding author<EMAIL_ADDRESS>
###### Abstract

As deep learning models scale, they become increasingly competitive from
domains spanning computer vision to natural language processing; however, this
happens at the expense of efficiency since they require increasingly more
memory and computing power. The power efficiency of the biological brain
outperforms the one of any large-scale deep learning (DL) model; thus,
neuromorphic computing tries to mimic the brain operations, such as spike-
based information processing, to improve the efficiency of DL models.

Despite the benefits of the brain, such as efficient information transmission,
dense neuronal interconnects, and the co-location of computation and memory,
the available biological substrate has severely constrained the evolution of
biological brains. Electronic hardware does not have the same constraints;
therefore, while modeling spiking neural networks might uncover one piece of
the puzzle, the design of efficient hardware backends for SNNs needs further
investigation, potentially taking inspiration from the available work done on
the artificial neural networks side. As such, when is it wise to look at the
brain while designing new hardware, and when should it be ignored?

To answer this question, we quantitatively compare the digital hardware
acceleration techniques and platforms of ANNs and SNNs. As a result, we
provide the following insights:  (i) ANNscurrently process static data more
efficiently,  (ii) SNNefficiency heavily depends on the network’s firing
activity and the hardware sparsity-aware operation,  (iii) SNNsoften
outperform ANNs when processing data produced by neuromorphic sensors, such as
event-based cameras and silicon cochleas,  (iv) online and on-chip training
techniques for SNNs should be investigated further, as these represent major
challenges for ANN models, given their size and complexity, and  (v) hybrid
approaches combining SNNs and ANNs can result in the best solutions,
accounting for both efficiency and loss optimization.

_Keywords_ Spiking Neural Networks $\cdot$ Artificial Neural Networks $\cdot$
Digital Hardware $\cdot$ Deep Learning $\cdot$ Neuromorphic Computing

##  1 Introduction

From smartphones to televisions and cars, DL has become pervasive in our daily
lives. Many modern DL models, especially large language models [5],
recommender systems [38], and vision transformers [14] require huge amounts of
energy for both training and inference. For instance, Vit-G/14 [54], a top
performing model in object recognition, requires
$2.86\text{\,}\mathrm{GFLOP}\text{/}$ for a single inference on ImageNet [12].
The model contains 184.3 billions of parameters, and optimizing these
parameters has also a non negligible environmental impact [20]. In fact,
Vit-G/14 has a training time of $3\cdot 10^{4}$ TPUv3 days [54]; given that a
TPUv3 consumes an average of $220\text{\,}\mathrm{W}\text{/}$[9], the energy
consumption for training can be estimated to be
$159\text{\,}\mathrm{MW}\text{\,}\mathrm{h}\text{/}$. In stark contrast, the
brain runs within a $20\text{\,}\mathrm{W}\text{/}$ power budget.

Tackling these challenges requires more efficient neural network models and
hardware. Many neuromorphic engineers are looking at how the brain might offer
us a blueprint for making DL more efficient [46]. This is because the brain is
capable of causal reasoning, integrating various sensory modalities in
executing long-term planning, and providing sensorimotor feedback to enable us
to engage with our environments actively. Moreover, it does so far more
efficiently than any combination of large-scale DL models currently available.
The pursuit of brain-inspired computing has triggered a surge in the
popularity of SNNs [16] with the ultimate goal of realizing efficient
artificial intelligence (AI). For instance, SpikeGPT [55], the first spiking
LLM, is estimated to use $22\text{\,}\times$ fewer operations in its execution
when compared to its conventional non-spiking counterpart. Thus, model
optimizations may ultimately outweigh the low-level hardware issues discussed
in this paper.

At the inception of neuromorphic computing, analog-domain computation was
shown to be a promising substrate for the deployment of brain-inspired
hardware. The metal-oxide-semiconductor transistor in the sub-threshold regime
can emulate the diffusion-based dynamics of neuronal ion channels. At the same
time, memristive technologies reproduce key features of biological neurons
[15, 43]. However, analog designs suffer from scalability issues due to
transistor mismatch and noise, and long design time due to reduced electronic
design automation (EDA) tool support [17]. In contrast, in deep sub-micron
technologies, digital designs benefit from strong EDA support and reliable
operation. While research on analog computation and in-memory processing has
flourished over the past decade, digital accelerators are easier to design and
deploy in the immediate short term.

This paper quantitatively reviews the broad landscape of digital accelerators
for both SNN and conventional ANN accelerators. To extract trends and use-case
driven guidelines for the use of spike-based representations, we segment the
analysis into static and temporal (e.g., sequence learning) workloads as
follows:

  * •

Section 2 analyzes the low-level neuron models employed in SNNs and ANNs.

  * •

Section 3 introduces the metrics and the methods used to compare the hardware
accelerators from the ANN and SNN domains.

  * •

Section 4 analyzes and compares the hardware acceleration of convolutions for
static vision workloads (in particular, object recognition), and then compares
state-of-the-art (SOTA) digital ANN and SNN accelerators.

  * •

Section 5 analyzes and compares the hardware acceleration of temporal
workloads, i.e., tasks that involve sequences of inputs evolving in time.

We derive that spikes are efficient only when exploiting their sparsity, even
if the current dominant SNN training methods result in dense rate-based coding
[16] networks with very high spiking activity during inference. We make the
following conclusions:

  * •

On static data classification tasks, ANNs perform better than SNNs in both
processing efficiency and classification accuracy. To reduce this gap, a
possible way is to leverage sparse coding techniques, that potentially lead to
an improvement in the model execution efficiency on hardware.

  * •

On temporal tasks, SNNs show energy efficiency comparable to ANN chips;
furthermore, only in the SNN domain there is an example of on-chip training
with competitive task performance; hence, further investigation in this
direction is needed, together with model training techniques to improve
performance (e.g., classification accuracy) and the system energy consumption.

  * •

Classification-based workloads employing bio-inspired sensors data, such as
event cameras [21] and silicon cochleas [52], can harness the stateful nature
of spiking neurons with competitive performance and high efficiency.

  * •

The optimal solution for a given classification task might be a heterogeneous
model, made of ANN and SNN parts that operate synergically together. ANN are
effective where floating point operations per seconds are plentiful, and SNNs
are effective where bandwidth is limited. A mixed approach takes advantage of
both.

In the end, we ask ourselves the fateful question: “To spike, or not to
spike?”.

##  2 Neuron Models

A basic introduction to the SNN and ANN models employed in this analysis is
provided. Using these models, a quantitative estimation of the energy
consumption of these neurons on vision and sequence learning tasks is made.

###  2.1 The spiking neuron

The discrete leaky integrate and fire (LIF) neuron model is among the most
commonly used models in the literature [16], and is described by:

$v_{l,i}[t]=\beta\cdot v_{l,i}[t-1]+u_{l,i}[t]-\vartheta\cdot S_{l,i}[t-1]$
(1)

$v_{i}[t]$ is the neuron state, $\beta$ is the decay coefficient associated
with the leakage, and $S_{i}[t]$ is the output spike. The $l$ suffix denotes
the $l$-th layer in the network. The input current $u_{l,i}[t]$ is given by:

$u_{l,i}[t]\overset{\Delta}{=}\sum_{j}{w_{l,ij}\cdot S_{l-1,j}[t]}$ (2)

The output spike is modeled by:

$S_{i}[t]=\begin{cases}1&\text{if }v_{i}[t]\geq\vartheta\\\
0&\text{otherwise}\end{cases}$

$S_{i}[t]$ is equal to 1 at spike time (i.e., if at timestamp $t$ the state
$v_{i}[t]$ is larger than the threshold $\vartheta$) and 0 elsewhere.

Note that since $S_{l-1,j}[t]$ is either 0 or 1, the input current
$u_{l,i}[t]$ is the sum of the synaptic weights of the $l-1$-th layer neurons
spiking at timestamp $t$.

Equation 1 requires three inputs to update the state $v[t]$: the weights
$w_{ij}$, the previous value $v[t-1]$ and the input spikes $S_{j}[t]$. This
means that SNN SNN hardware processing element (PE) needs access to 3 memory
structures to retrieve the inputs, the state, and the weights.

###  2.2 The artificial neuron

The most common artificial neuron model [28] is static in time, i.e., it does
not preserve any state between successive inputs. A particular class of ANNs,
namely recurrent neural networks, employ special gates that introduce a state
in the neuron [28]. Here, the stateless ANN neuron is analyzed while in
Section 5, recurrent cells are discussed.

The artificial neuron model is:

$z_{l,i}=\varphi(\sum_{j}z_{l-1,j}\cdot w_{l,ij}+b_{l,j})$ (3)

The weights are multiplied by the inputs from the previous layer, $z_{l-1,j}$,
and accumulated; an optional bias term $b_{l,j}$ is added in Eq. 3. An
activation function, $\varphi(x)$, is applied to the accumulated value. The
specific choice of activation function often depends on the application,
layer, and the type of neural network [28].

Distinct from Eq. 1, only two inputs are needed in Eq. 3: the previous layer
activations $z_{l-1,j}$ and the current layer weights $w_{l,ij}$. This implies
that only two memory structures are needed for the artificial PE being
implemented in hardware: this represents an advantage over SNN implementations
in both memory (and area) occupation of the circuit, and energy consumption,
since memory accesses are the most energy-intensive operations in modern
digital hardware architectures[36].

##  3 Methodology

In ANN architectures, the most common operation is the multiply and accumulate
(MAC) [48], which includes the input activation multiplication by the
corresponding weight and the subsequent accumulation. We treat each MAC as a
pair of operations, since a multiplication and an addition are computed, even
if in hardware they might be merged. This is to isolate the contribution of
each computation to total power consumption, and is the same approach adopted
by the ANN hardware research considered in this analysis [30, 37, 49, 42]. In
SNN hardware, no multiplication is performed [16]: a weight is accumulated if
there is an input spike, otherwise it is not. Hence, this is considered a
single operation in our analysis.

In the SNN literature, there are numerous references to the synaptic operation
(SynOP) metric [3, 19]. However, there is little consensus on its formal
definition. Given the ambiguous definition of this metric, it is not used to
measure the efficiency of SNN SNN accelerator in this work. In addition to
ambiguity, the SynOP metric does not tend to account for stateful operations
(state access and updates).

To obtain an estimation of the energy cost of SNN and ANN architectures, two
simple mathematical models are provided in Sections 4 and 5. These allow us to
approximately compare ANN and SNN hardware accelerators a priori. While
evaluating the actual hardware performance, considering efficiencies in terms
of operations per second per watt (OPS/W) is not enough: on static data, SNN
SNN would need multiple time steps to perform a classification, while ANN ANN
accelerator performs an inference in a single forward-pass. To ensure
fairness, this paper adopts the energy consumption per inference as the most
balanced metric for comparison; latency and accuracy-related metrics are
accounted for separately.

Sparsity is ignored in the energy consumption analysis performed in Sections 4
and 5, since modern ANN accelerators have complex and very efficient sparse
dataflows [30, 25, 24]. It is, however, included in the energy consumption
results shown in Tables 2 and 3, and our model for factoring it into energy
estimations is provided in Section 4.

##  4 Static tasks

Equation 2 embeds one of the major advantages of SNN processing: the removal
of multiplication between the input feature map (i.e., spikes) and the
synaptic weights. In theory, this leads to both hardware and energy savings.
However, consider the data in Table 1:

Table 1: Energy consumption comparison between integer add, multiply and memory operation on on-chip SRAM caches [36], targeting a $45\text{\,}\mathrm{nm}\text{/}$ CMOS process. | Energy [$\mathrm{pJ}\text{/}$] | Energy density [$\mathrm{pJ}\text{/}\mathrm{B}$]
---|---|---
Add $8\text{\,}\mathrm{bit}\text{/}$ | 0.03 | 0.03
Multiply $8\text{\,}\mathrm{bit}\text{/}$ | 0.20 | 0.20
Read $64\text{\,}\mathrm{bit}\text{/}$ ($8\text{\,}\mathrm{kB}\text{/}$ capacity) | 10.00 | 2.50

the energy consumption for reading a byte from the on-chip buffer, implemented
as SRAM, exceeds the cost of an $8\text{\,}\mathrm{bit}\text{/}$ integer
multiplication. Hence, a multiply-free SNN digital hardware accelerator is not
necessarily more efficient than ANN ANN accelerator, given that SNNs require
additional memory accesses to update the neuron states.

To evaluate these mathematical models quantitatively on static tasks, the
convolution operations shown in Algorithms 1 and 2

Algorithm 1 SNN convolution of a single window and timestamp.

1:$S,\beta,\vartheta$ $\triangleright$ Stride, leakage, threshold.

2:W $\triangleright$ Convolution kernel weights.

3:M $\triangleright$ Neurons states.

4:ifmap, ofmap $\triangleright$ Input and output feature maps.

5:${(c_{o},~{}h_{o},~{}w_{o})}$ $\triangleright$ Output value coordinates.

6:$I\leftarrow 0$ $\triangleright$ Input synaptic current.

7:for $c_{i}\leftarrow 0,~{}CI-1$ do $\triangleright$ Input channels.

8: for $h_{k}\leftarrow 0,~{}HK-1$ do $\triangleright$ Kernel height.

9: for $w_{k}\leftarrow 0,~{}WK-1$ do $\triangleright$ Kernel width.

10: $h_{i}\leftarrow h_{o}*S+h_{k}$

11: $w_{i}\leftarrow w_{o}*S+w_{k}$

12: if $ifmap[c_{i}][h_{i}][w_{i}]\neq 0$ then

13: $I\leftarrow I~{}+~{}W[c_{o}][c_{i}][h_{k}][w_{k}]$

14: end if

15: end for

16: end for

17:end for

18:$m\leftarrow M[c_{o}][h_{o}][w_{o}]*\beta+I$ $\triangleright$ State update.

19:if $m\geq\vartheta$ then

20: $m\leftarrow m-\vartheta$

21: $ofmap[c_{o}][h_{o}][w_{o}]=1$

22:else

23: $ofmap[c_{o}][h_{o}][w_{o}]=0$

24:end if

25:$M[c_{o}][h_{o}][w_{o}]\leftarrow m$

are used as benchmarks, since convolution is the fundamental operation
employed in the large majority of current hardware accelerators for object
recognition, detection and so on [42, 49, 34, 33]. In more recent approaches,
transformer architectures are emerging as better-performing vision models
[31]. As such, a transformer accelerator [30] is considered in the final
results section.

With respect to Algorithms 1 and 2, all activations, weights, and states are
quantized to $8\text{\,}\mathrm{bit}\text{/}$, while spikes are treated as
unary quantities. Consider Algorithm 1:

  * •

$\mathit{CI}\cdot\mathit{HK}\cdot\mathit{WK}$ weights and spikes are read from
memory, where $\mathit{CI}$ is the number of channels in the input feature
map, and $\mathit{HK}$ and $\mathit{WK}$ represent the shape of the
convolution kernel. For the sake of clarity, this quantity is denoted with
$N_{\textit{rd}}$.

$N_{\textit{rd}}\overset{\Delta}{=}\mathit{CI}\cdot\mathit{HK}\cdot\mathit{WK}$

Assuming that 8 spikes are encoded to an $8\text{\,}\mathrm{bit}\text{/}$
memory word in the spike scratchpad (i.e., the memory structure used to host
the input and output data near the PE), the energy associated with a spike
memory operation (either read or write) can be approximated to
$E_{\textit{rd}}/8$, where $E_{\textit{rd}}$ ($E_{\textit{wr}}$) is the energy
consumption of a memory read (write) considering the
$8\text{\,}\mathrm{kB}\text{/}$ cache field in Table 1. The energy consumption
associated with the memory accesses of weights and spikes is denoted with
$E_{\textit{rd}_{\textit{tot}}}$:

$E_{\textit{rd${}_{\textit{tot}}$}}=N_{\textit{rd}}\cdot(E_{\textit{rd}}+E_{\textit{rd}}/8)$

  * •

$\mathit{CI}\cdot\mathit{HI}\cdot\mathit{WI}$ additions are performed on the
synaptic current. The energy associated to checking if the spike
($ifmap[c_{i}][h_{i}][w_{i}]$) is equal to 1 is negligible since its cost is
included in the memory access performed to retrieve the spikes. Hence, this
energy is denoted with $E_{\textit{acc}}$.

$E_{\textit{acc}}=N_{\textit{rd}}\cdot E_{\textit{add}}$

  * •

Next, one state has to be retrieved from memory, multiplied by the leakage,
$\beta$, added to the synaptic current, thresholded (compared against the
threshold, $\vartheta$, and, if higher, reduced by $\vartheta$ via
subtraction) and written back. In the worst case, this energy denoted with
$E_{\textit{state}}$, is given by:

$E_{\textit{state}}=E_{\textit{rd}}+E_{\textit{mult}}+E_{\textit{add}}+E_{\textit{comp}}+E_{\textit{sub}}+E_{\textit{wr}}$

  * •

The output spike must then be written to the scratchpad. This energy is
denoted with $E_{\textit{ofmap}}$.

$E_{\textit{ofmap}}=E_{\textit{wr}}/8$

Hence, the total energy involved in SNN SNN convolution, defined as
$E_{\textit{SNN}}$, is:

$E_{\textit{SNN}}=E_{\textit{rd${}_{\textit{tot}}$}}+E_{\textit{acc}}+E_{\textit{state}}+E_{\textit{ofmap}}$

The following values are employed for numerical estimation: the shape of the
convolution, i.e., of the input feature map window to be evaluated in order to
compute a single output feature map value, is
$(\mathit{CI},\mathit{HI},\mathit{WI})=(512,3,3)$, which means the number of
memory reads is $N_{\textit{rd}}=4608$. For memory operations,
additions/subtractions, and multiplications, the data from Table 1 are used.
It has to be remarked thas, in this analysis, we do not consider any memory
optimization (e.g., buffering on registers) since both ANNs and SNNs access
the memories with the same window patterns; thus, these would bring similar
advantages to both of them, without impacting the comparison. The energy
consumed by a threshold comparison, $E_{\textit{comp}}$, is assumed to be the
same as that of an $8\text{\,}\mathrm{bit}\text{/}$ addition. This is a
reasonable assumption, since comparison commonly employs a subtraction. This
leads to:

$E_{\textit{SNN}}=$13.1\text{\,}\mathrm{nJ}\text{/}$$

The energy consumption obtained above assumes a dense input feature map, i.e.,
all input neurons are firing. To take into account sparse firing activity, a
coefficient $\gamma_{\textit{SNN}}$ can be introduced that reflects the
proportion of neurons in the feature map that are firing at a given time, and
the hardware overhead due to the sparse data structures employed. Hence:

$\displaystyle
E_{\textit{SNN}}=$13.1\text{\,}\mathrm{nJ}\text{/}$\cdot\gamma_{\textit{SNN}}$
$\displaystyle 0<\gamma_{\textit{SNN}}\leq 1$

Consider now the convolution operation performed in ANN ANN, which is reported
in Algorithm 2.

Algorithm 2 ANN convolution of a single window.

1:$a\leftarrow 0$ $\triangleright$ Activation.

2:for $c_{i}\leftarrow 0,~{}\mathit{CI}-1$ do

3: for $h_{k}\leftarrow 0,~{}\mathit{HK}-1$ do

4: for $w_{k}\leftarrow 0,~{}\mathit{WK}-1$ do

5: $h_{i}\leftarrow h_{o}*S+h_{k}$

6: $w_{i}\leftarrow w_{o}*S+w_{k}$

7: $a\leftarrow
a~{}+~{}W[c_{o}][c_{i}][h_{k}][w_{k}]~{}*~{}ifmap[c_{i}][h_{i}][w_{i}]$

8: end for

9: end for

10:end for

11:$z=\varphi(a)$ $\triangleright$ Non-linear activation.

12:$ofmap[c_{o}][h_{o}][w_{o}]=\psi(z)$ $\triangleright$ Quantisation.

Following the same approach adopted for the SNN convolution:

  * •

$N_{rd}$ weights and inputs are read from memory:

$E_{\textit{rd${}_{\textit{tot}}$}}=2N_{\textit{rd}}\cdot E_{\textit{rd}}$

  * •

The same number of additions and multiplications are performed, and this
energy is denoted with $E_{\textit{MAC}}$.

$E_{\textit{MAC}}=N_{\textit{rd}}\cdot(E_{\textit{add}}+E_{\textit{mult}})$

  * •

The obtained value is then processed by the nonlinear activation function
$\varphi(z)$. This energy is denoted with $E_{\textit{act}}$.

  * •

The result is quantized in order to be processed by the next layer in the
network. The quantization step is modeled through a function $\psi(z)$, with
an associated energy cost $E_{\textit{quant}}$, which is estimated as an
$8\text{\,}\mathrm{bit}\text{/}$ addition (for instance, the rectified linear
unit (ReLU) activation is a 0-thresholding).

  * •

Finally, the result is written to the scratchpad memory:

$E_{\textit{ofmap}}=E_{\textit{wr}}$

The total energy consumption of ANN ANN convolution is:

$E_{\textit{ANN}}=$24.1\text{\,}\mathrm{nJ}\text{/}$\cdot\gamma_{\textit{ANN}}$

A similar sparsity coefficient can be introduced, denoted with
$\gamma_{\textit{ANN}}$. The approximations made for $E_{quant}$ and $E_{act}$
are acceptable since their contribution to the total energy is negligible,
given that they are performed only on the final result of the convolution.

Hence, despite the additional memory accesses and state operations involved in
SNNs, ANN convolutions consume $1.84\text{\,}\times$ more energy. Of course,
this result depends heavily on the convolution filter depth and size, but it
gives a reasonable approximation of the different costs between ANN and SNN
processing, as highlighted in Section 4.1. In Fig. 1,

$90$$92$$94$$96$$98$$0$$100$$200$$300$Sparsity [%]Energy
[$\mathrm{\SIUnitSymbolMicro J}\text{/}$]Total
energyANNSNN$90$$92$$94$$96$$98$Sparsity [%]Memory accesses
energy$90$$92$$94$$96$$98$Sparsity [%]Computations energyEnergy consumption of
convolution v.s. input data sparsity Figure 1: The energy model proposed in
Section 4 for the convolution is evaluated for different values of input data
sparsity, defined as the percentage of zero-value inputs. The estimations are
performed on an input feature map of shape 512×14×14, with 3×3 convolution
kernels, stride of 1 and no padding. The code used to generate these data can
be found at https://github.com/fabrizio-ottati/to-spike-or-not.

the energetic model is run for different values of input data sparsity, and
the range $80\text{\,}\mathrm{\char 37\relax}\text{/}99\text{\,}\mathrm{\char
37\relax}\text{/}$ is highlighted. One can notice that memory accesses
dominate the energy consumption in both ANNs and SNNs, and the SNNs are
characterized by a lower consumption since the input data is
$1\text{\,}\mathrm{bit}\text{/}$, while it is $8\text{\,}\mathrm{bit}\text{/}$
for ANNs. Weights are quantized to $8\text{\,}\mathrm{bit}\text{/}$ for both
models. It has to be remarked that this analysis is ideal, i.e., no overheads
are considered and the null activations are completely skipped during the
computations by associating null energetic cost to the zero-checking.

However, the energy estimation obtained for the SNN corresponds to a single
time step! When dealing with static data, such as images, SNN SNN needs
multiple time steps ($T$, for instance), since the input image pixels are
encoded to multiple spikes in time [16]. Hence, the actual energy consumption
associated with a convolution operation in SNN SNN must be multiplied by the
number of time steps needed to perform an inference:

$E_{\textit{SNN}}=$13.06\text{\,}\mathrm{nJ}\text{/}$\cdot
T\cdot\gamma_{\textit{SNN}}$

Since $T>1$, SNNs are less efficient on static vision data than ANNs, if the
same degree of sparsity-awareness
($\gamma_{\textit{SNN}}=\gamma_{\textit{ANN}}$) is taken into account, as
stated in Section 3. This conclusion is validated by the data shown in Table
2, which includes SOTA accelerators at the time of writing. The given results
account for how the hardware handles sparse feature maps, as stated by the
author of the papers considered [30, 37, 49, 42, 34, 33].

###  4.1 Discussion

Table 2: Deep learning accelerators evaluated on ImageNet [12]. The best efficiency is considered for each design, and the associated task accuracy is evaluated under the same conditions. By mixed (  ), an accelerator running both SNN and ANN processing elements is indicated. |  ANN accelerators |  SNN accelerators |  Mixed
---|---|---|---
Work | Keller’23 [30] | Park’22 [42] | Mo’21 [37] | SNPU’23 [34] | C-DNN’23 [33]
Process [$\mathrm{nm}\text{/}$] | 5 | 7 | 28 | 28 | 28
Area [$\mathrm{m}\mathrm{m}^{2}$] | 0.154 | 4.74 | 1.9 | 6.3 | 20.25
V${}_{\textbf{{dd}}}$ [$\mathrm{V}\text{/}$] | 0.46 - 1.05 | 0.55 - 1 | 0.6 - 0.9 | 0.63 - 1.1 | 0.7 - 1.1
f${}_{\textbf{{ck}}}$ [$\mathrm{MHz}\text{/}$] | 152 - 1760 | 332 - 1196 | 100 - 470 | 2 - 200 | 50 - 200
Data format | INT4-VSQ [30] | INT8 | INT8 | INT8, INT4 | INT1-16, INT4/8
Task accuracy [$\mathrm{\char 37\relax}\text{/}$] | 80.5 | 71.68 | 76.92 | 66.8 | 77.1
Network model | DeiT-Base | MobileNetTPU | ResNet50 | ResNet18 | ResNet50
Parameters [$\mathrm{M}$] | 768 | 3.4 | 25 | 11.7 | 25
Power [$\mathrm{mW}\text{/}$] | 55.62 | 5114 | 125.8 | 76.63 | 34.4
Throughput [FPS] | 56.2 | 3433 | 40.4 | 245 | NA
Energy/inference [$\mathrm{mJ}\text{/}$] | 0.99 | 1.49 | 1.2 | 1.95 | 0.28

In the literature, many overviews of hardware accelerators for SNNs are
provided, ranging from digital hardware to in-memory computing (IMC) and
mixed-signal architectures [3, 4, 1]. An up-to-date list of SNN hardware
accelerators and processors can be found in [40]. However, most SNN hardware
reviews are missing an important feature: an objective comparison with SOTA
ANN hardware accelerators. Most SNN accelerators are benchmarked on static
datasets; such datasets are inappropriate for proper system characterization,
because  (i) they are often considered trivial or “solved” datasets (e.g.,
CIFAR-10, MNIST) [3], and  (ii) ANNstend to be more efficient on static data.

Table 2 reports SOTA ANN vision accelerators and the best performing SNN
accelerator [34] (SNPU’23). All accelerators run the same task: object
classification on ImageNet [12]. SNPU’23 [34] is chosen as the SNN reference
since it is the only accelerator targeting complex vision workloads such as
ImageNet. In addition to SNPU’23, C-DNN’23 [33] is analyzed. This chip employs
both ANN and SNN PEs to maximize inference efficiency by inferring part of the
neural network layers on the SNN hardware and part of these on the ANN
hardware. Moreover, the design also provides on-chip training capabilities,
which most ANN accelerators lack.

Table 2 considers the most efficient SNN digital hardware accelerator, SNPU’23
[34], and a mixed-topology design, C-DNN’23 [33]. These are compared to
various ANN accelerators for object recognition, which target both
transformer-based models [31] and convolutional neural networks [29].
Different observations can be made:

  * •

The model employed by SNPU’23 [34], ResNet18 [29] SNN, is the lowest
performing model in terms of classification accuracy:
$66.8\text{\,}\mathrm{\char 37\relax}\text{/}$, against
$80.5\text{\,}\mathrm{\char 37\relax}\text{/}$ (Keller’23 [30]),
$71.68\text{\,}\mathrm{\char 37\relax}\text{/}$ (Park’22 [42]),
$76.92\text{\,}\mathrm{\char 37\relax}\text{/}$ (Mo’21 [37]) and
$77.1\text{\,}\mathrm{\char 37\relax}\text{/}$ (C-DNN’23 [49]).

  * •

The energy per inference of SNPU’23 [34] is the highest among all the designs
($1.95\text{\,}\mathrm{mJ}\text{/}\mathrm{inf}$), even considering an ANN
accelerator implemented on the same $28\text{\,}\mathrm{nm}\text{/}$
complementary metal-oxide-semiconductor (CMOS) node, i.e., Mo’21 [37]
($0.46\text{\,}\mathrm{mJ}\text{/}\mathrm{inf}$).

Figure 2

$0$$10$$20$$30$$40$$50$$0$$0.5$$1$$1.5$$2$BetterKeller’23 @ 5 nmPark’22 @ 7
nmMo’21 @ 28 nmSNPU’23 @ 28 nmC-DNN’23 @ 28 nmClassification error [%]Energy
[mJ]Energy per inference v.s. classification error on ImageNet [12]ANNSNNMixed
Figure 2: DL accelerators evaluated on ImageNet, by measuring classification
error and energy consumption per inference performed in hardware.

shows the energy consumption per inference vs. the top-1% classification error
on ImageNet, for each accelerator reported in Table 2. While SNPU’23 is
considered the SOTA SNN accelerator at the time of writing, it is nonetheless
Pareto-dominated by all ANN accelerators.

For what concerns C-DNN’23 [33], the mixed architecture leads to a very low
energy consumption per inference ($281\text{\,}\mathrm{\SIUnitSymbolMicro
J}\text{/}$), the best among all the chips despite being implemented on CMOS
CMOS node older than Keller’23 [30] and Park’22 [42]
($28\text{\,}\mathrm{nm}\text{/}$ vs. $5\text{\,}\mathrm{nm}\text{/}$ and
$7\text{\,}\mathrm{nm}\text{/}$, respectively). However, the model being run
on C-DNN’23, the ResNet50, is much smaller than the vision transformer of
Keller’23, which achieves a higher accuracy (80.5% v.s. 77.1%).

The design presented in SNPU’23 is among the best-performant in the SNN
domain. Accelerating ResNet family models on-chip is an impressive feat, given
that many SNN accelerators are limited to smaller-scale architectures. But our
analysis of SNPU’23 against ANN accelerators highlights that static data is
not the optimal way to demonstrate processing efficiency. A possibility is to
focus on tasks that take advantage of the event-based nature of SNNs, such as
dynamic vision sensor (DVS) data [21]. These sensors capture scenes in the
form of ‘events’ that can be treated naturally as spikes, without any
artificial encoding. Some alternative SNN accelerators target dynamic
workloads, though only those that handle large-scale (at least on the scale of
ResNets) models with static data are considered in the above analysis [13,
18]. These accelerators represent the minority [3], since most benchmarks are
limited to the MNIST and CIFAR-10 static datasets.

Beyond classification, more complex event-based vision tasks are less explored
by the neuromorphic community, though tend to dominate the modern computer
vision ANN field [45, 10, 26, 2]. Conversely, on-chip learning solutions in
the SNN domain [18, 19] are more advanced than for the ANN community. This
advantage should be exploited to reduce training costs and allow for adaptive
intelligence at the edge. In order to reduce the memory access burden for
computation of neuron states, low-rate training techniques should be explored
for efficient hardware inference [16, 19].

##  5 Temporal tasks

SNNs are inherently time-aware neural networks due to their statefulness (see
Section 2). As such, they are a natural fit for sequential data processing. In
video processing tasks, such as video segmentation, both non-spiking and
spiking neural networks often employ convolution structures to extract
features [51, 7, 8]. Given that the computational costs of spiking and non-
spiking convolutional operators are addressed in Section 4, this section
primarily concentrates on audio processing, another prominent subset of
temporal tasks.

In previous work [6, 44], a fully connected feed-forward RNN targeting keyword
prediction is used to compare ANNs, rate-based SNNs, and latency-based SNNs
[16]. The rate-based SNN is only $9\text{\,}\mathrm{\char 37\relax}\text{/}$
more efficient than the ANN due to the high input firing rate
($2.5\text{\,}\mathrm{kHz}\text{/}$) necessary to match the ANN performance.
The limited efficiency advantage makes the effort of migrating to a new type
of neural network hard to justify. Conversely, the latency-based SNN is
$84\text{\,}\mathrm{\char 37\relax}\text{/}$ more efficient than the ANN,
primarily thanks to the significantly lower firing rate that leads to a
reduction in the number of the timesteps evaluated by the SNN. However, this
methodology sets the target time window to $75\text{\,}\mathrm{ms}\text{/}$ to
incorporate temporal information, which is not suitable for real-time
processing.

To evaluate the performance and efficiency of ANN and SNN accelerators, in the
following audio processing benchmarks are considered, such as keyword spotting
(KWS), voice activity detection (VAD) and automatic speech recognition (ASR)
[11, 50, 41]. In particular, we present an energy analysis similar to the one
in Section 4. Finally, we compare SOTA digital hardware accelerators from the
spiking and artificial domains.

###  5.1 RNN versus SNN

RNN are designed for discerning patterns in sequential data, uniquely
characterized by their capacity to retain memory of previous inputs within
their hidden state. Variants that aim to enhance the “memory” of such neurons
have also emerged, such as lost short-term memories and gated recurrent units
[28], and have been adopted in audio processing tasks [23, 35]. The
formulation of a vanilla RNN layer is:

$\displaystyle h_{\textit{t}}$
$\displaystyle=\sigma{{}_{\textit{h}}}(U_{\textit{h}}\cdot
x_{\textit{t}}+V_{\textit{h}}\cdot h_{\textit{t-i}}+b_{\textit{h}})$ (4)
$\displaystyle o_{\textit{t}}$
$\displaystyle=\sigma{{}_{\textit{o}}}(W_{\textit{o}}\cdot
h_{\textit{t}}+b_{\textit{o}})$

where $x$ is the input, $h$ the hidden layer, and $o$ the output.
$U_{\textit{h}}$, $W_{\textit{o}}$, and $V_{\textit{h}}$ are the weight
matrices, $b$ is the bias vector and $\sigma$ is the activation function.
Differently from the SNN model defined by Eq. 1, the next state is computed
considering a bias $b_{\textit{h}}$ and by multiplying the previous state by a
matrix $V_{\textit{h}}$; in SNNs, $V_{\textit{h}}$ reduces to a scalar
$\beta$, employed for all the neurons [16]. The equivalent synaptic current is
represented by $U_{\textit{h}}\cdot x_{\textit{t}}$.

We compare the computational cost of these operations in the context of a
speech recognition task. We consider the cost of a vanilla RNN and SNN SNN,
which share the same mechanism of implicit recurrence through the neuron state
[17, 28]. As in Section 4, activations, weights, and states are quantized to 8
bits, while spikes are single-bit quantities. As a use case, we consider a
layer with $N$ inputs and calculate the energy consumption of a neuron in the
layer, as in Section 4.

With the SNN model:

  1. 1.

$N$ weights and input spikes are loaded from memory:

$E_{\textit{rd}_{\textit{tot}}}=N\cdot(E_{\textit{rd}}+E_{\textit{rd}}/8)$

  2. 2.

These values are then accumulated depending on the spike value to obtain the
activation to be fed to the state:

$E_{\textit{acc}}=N\cdot E_{\textit{add}}$

  3. 3.

The state is loaded from memory and decayed (i.e., multiplied) by a factor
$\beta$; then, it is accumulated with the activation computed in the previous
step, compared against the threshold $\vartheta$, reset if needed and stored
to memory:

$E_{\textit{state}}=E_{\textit{rd}}+E_{\textit{mult}}+E_{\textit{add}}+E_{\textit{comp}}+E_{\textit{sub}}+E_{\textit{wr}}$

  4. 4.

The output spike, if generated, is then stored to the scratchpad:

$E_{\textit{ofmap}}=E_{\textit{wr}}/8$

Assuming $N=1024$ inputs and considering the data reported in Table 1, the
total energy, $E_{\textit{SNN}}$, is given by:

$E_{\textit{SNN}}=$2.92\text{\,}\mathrm{nJ}\text{/}$\cdot\gamma_{\textit{SNN}}$

Consider now the vanilla artificial RNN layer:

  1. 1.

$N$ weights and $N$ inputs are read from memory, together with the hidden
state and its recurrent weight:

$E_{\textit{rd}_{\textit{tot}}}=(2N+2)\cdot E_{\textit{rd}}$

  2. 2.

the inputs and the state are multiplied by the weights and accumulated:

$E_{\textit{MAC}}=(N+1)\cdot(E_{\textit{add}}+E_{\textit{mult}})$

  3. 3.

the state is written back to memory. As in Section 4, the quantization and
activation energies are neglected:

$E_{\textit{state}}=E_{\textit{wr}}$

  4. 4.

the obtained value is processed by the nonlinear activation function,
quantized and written back to memory:

$E_{\textit{ofmap}}=E_{\textit{wr}}$

Hence, the total energy consumption is:

$E_{\textit{ANN}}=$5.37\text{\,}\mathrm{nJ}\text{/}$\cdot\gamma_{\textit{ANN}}$

The energy analysis shows that in both vanilla RNN and spiking layers, the
memory accesses for weights and states consume the majority of the energy, as
in the convolution case. Notice that $E_{\textit{SNN}}$ is
$1.84\text{\,}\times$ smaller than $E_{\textit{ANN}}$: this is due to the fact
that while in the ANN the inputs are on $8\text{\,}\mathrm{bit}\text{/}$, in
the SNN these are single-bit quantities. Since the memory access energy
dominates the other figures, this results in a major overhead of ANN models
with respect to SNN ones. Moreover, the ANN model involves a multiplication
when processing inputs, which is more energy-hungry than the addition (Table
1).

Differently from the convolution case, the number of timesteps needed to
process the inputs is larger than 1 for both ANNs and SNN, since the data is
now evolving through time.

$90$$92$$94$$96$$98$$0$$100$$200$$300$Sparsity [%]Energy
[$\mathrm{\SIUnitSymbolMicro J}\text{/}$]Total
energyANNSNN$90$$92$$94$$96$$98$Sparsity [%]Memory accesses
energy$90$$92$$94$$96$$98$Sparsity [%]Computations energyEnergy consumption of
fully-connected recurrent layer v.s. input data sparsity Figure 3: The energy
model proposed in Section 5 for the SNN and RNN layers is evaluated for
different values of input data sparsity, defined as the percentage of zero-
value inputs. The estimations are performed on a layer of shape 1024×512. The
code used to generate these data can be found at https://github.com/fabrizio-
ottati/to-spike-or-not.

Both these energies can be multiplied by the input sparsity factors presented
in Section 4, $\gamma_{\textit{SNN}}$ and $\gamma_{\textit{ANN}}$. In Fig. 3,
an evaluation of the energy consumption for different value of input sparsity,
using a layer of shape 1024×512, is reported. One can notice that, as in the
convolution case, memory accesses dominate the energy consumption and that
ANNs almost close the efficiency gap with SNNs for very high value of input
data sparsity.

It has to be taken into account that very simple RNN and SNN models are
considered. In Section 5.2, digital hardware accelerator targeting more
sophisticated neural network architectures are considered.

Table 3: SNN and ANN accelerators evaluated on temporal tasks including VAD, KWS and ASR. | VAD | KWS
---|---|---
|  SNN |  ANN |  SNN |  ANN
Work | Yang’19 [53] | Oh’19 [39] | Frenkel’22 [18] | Kim’22 [32] | Giraldo’22 [27] | Shan’23 [47]
Process [$\mathrm{nm}\text{/}$] | 180 | 180 | 28 | 65 | 65 | 28
Area [$\mathrm{m}\mathrm{m}^{2}$] | 1.6 | - | 0.45 | 2.03 | 2.56 | 3.6
V${}_{\textrm{dd}}$ [$\mathrm{V}\text{/}$] | 0.55 | 0.6 | 0.5 | 0.75 | 0.6 | 0.4
f${}_{\textrm{ck}}$ [$\mathrm{MHz}\text{/}$] | 0.5 | 0.7 | 13 | 0.25 | 0.25 | 0.2
Feature Extractor | Analog | Analog | No FEx | Analog | Digital | Digital
Data Format | INT1 | INT4 | INT8 | INT8 | INT8 | INT1
Dataset |  | Aurora4 w/
---
DEMAND
| LibriSpeech w/
---
NOISEX-92
| Spiking
---
Heidelberg Digits
GSCD
Task accuracy [$\mathrm{\char 37\relax}\text{/}$] | 85 | 90 |  | 90.7
---
(1-word)
| 86
---
(10-word)
| 90.9
---
(10-word)
| 97.8
---
(2-word)
Network model | FCN | FCN | Spiking RNN | GRU | LSTM | DSCNN
Parameters [$\mathrm{k}$] | 4.6 | 1.6 | 132 | 24 | 21.5 | 4.7
Power [$\mathrm{\SIUnitSymbolMicro W}\text{/}$] | 1 | 0.142 | 79 | 23 | 10.6 | 0.8
Energy/inference [$\mathrm{nJ}\text{/}$] | 10 | 2.3 | 42 | 285.2 | 169.6 | 23.6

###  5.2 Discussion

Regarding audio processing tasks, most accelerators are tested on or designed
specifically for VAD [53, 39] and KWS [18, 32, 27, 47]. This is due to the
fact that VAD and KWS represent less challenging problems to tackle; hence,
simple neural network architectures are employed, which allow achieving higher
efficiency on hardware inference.

In Table 3 we chose the two most efficient SNN chips [18, 53], respectively in
VAD and KWS, which are compared against SOTA ANN counterparts. For the VAD
task, Oh’19 [39] achieves the best efficiency measured in energy per
inference, despite being implemented on an old
$180\text{\,}\mathrm{nm}\text{/}$ CMOS technology node. It results to be more
efficient and to perform better, in terms of classification accuracy, than
Yang’19 [53], which runs an SNN model and uses the same CMOS node.

As for the KWS task, different CMOS technologies are employed in the
accelerators. These values are not normalized with Dennard scaling since all
the chips have a power consumption in the order of
$10\text{\,}\mathrm{\SIUnitSymbolMicro W}\text{/}$; in these conditions, most
of the power consumption is static, while Dennard scaling gives an
approximation of how dynamic power scales across technology nodes.

Also for KWS, the highest energy efficiency is achieved by an ANN chip,
Shan’23 [47], which also has a lower average power consumption and
significantly higher accuracy than all the other chips targeting the same
task. The SNN accelerator, Frenkel’22 [18], achieves the lowest latency and is
the only chip that supports online learning and on-chip training: in fact, the
whole training process is performed on-chip, that is validated also on vision
and autonomous agent tasks. It should be noted that the number of parameters
of the spiking RNN in [18] is $28\text{\,}\times$ larger than that of the ANN
chip in [47], while achieving lower accuracy. This might suggest that SNNs are
still lacking in terms of classification accuracy when compared to their ANN
counterparts; however, Shan’23 [47] is an inference only chip, hence the model
is fine-tuned for the task, while Frenkel’22 supports arbitrary network
topologies and on-chip training and can be repurposed. This is a significant
advantage for chips deployed on edge devices, which model might need to be re-
adapted to the environment in which the system is deployed.

Figure 4

$0$$5$$10$$15$$20$$10^{0}$$10^{1}$$10^{2}$$10^{3}$$10^{4}$$10^{5}$BetterY1.0e1]ptOh’19
@ 180 nmY1.4e1]ptKim’22 @ 65 nmY9.1e0]ptGiraldo’22 @ 65 nmY2.2e0]ptShan’23 @
28 nmY1.3e0]ptGao’20 (FPGA)Y1.57e0]ptGao’18 (FPGA)Y1.5e1]ptYang’19 @ 180
nmY9.3e0]ptFrenkel’22 @ 28 nmClassification error [%]Energy [nJ]Energy per
inference v.s. classification error on VAD and KWS.ANNSparse-ANNSNN Figure 4:
Energy consumption per inference v.s. classification error of recent digital
SNN and ANN accelerators measured on the VAD and KWS tasks.

shows the distribution of these accelerators in terms of energy efficiency and
accuracy along with FPGA accelerators [22, 24], that achieve the highest
accuracy in KWS. One can notice that Shan’23 [47] Pareto-dominates all the
other chips, thanks to a highly efficient sparsity-aware chip architecture and
a performant neural network model. Frenkel’22 [18] presents a very competitive
efficiency, taking into account that it is the only chip with in-hardware
training capabilities: in fact, the network benchmarked on KWS is trained
directly on it; nonetheless, the resulting classification accuracy on the task
results to be competitive even when considering most of the ANN chips
analyzed.

##  6 Conclusions

This paper analyzes SOTA digital hardware accelerators implementing ANN and
SNN models performing two types of tasks: static and temporal datasets. The
architectures are compared on classification accuracy and energy consumption
per inference. The results suggest that:

  * •

On static images and object recognition tasks, current SNN models and hardware
perform worse than their ANN counterparts. This requires improvements in both
the neural network models and the hardware by targeting higher sparsity in the
network firing activity and improved hardware capabilities in taking advantage
of it.

  * •

On temporal tasks, SNNs show a really competitive energy efficiency, and
represent the only case in which full on-chip training and learning is
performed [18]. This advantage should be exploited, together with new training
strategies that promote both improvement in classification accuracy, which is
still lower than the ANN counterpart, and sparse firing activity, that would
lower further the energy consumption of the accelerator.

  * •

Further investigation of efficient model and hardware solutions targeting bio-
inspired sensors data, such as event cameras [21] and silicon cochleas [52],
are needed to take advantage of the time-related functioning of spiking
neurons and architectures.

  * •

Hybrid SNN and ANN solutions might be the key to maximize task performance and
efficiency. C-DNN’23 [33] shows the second-best efficiency among the
accelerators analyzed, despite being implemented in a
$28\text{\,}\mathrm{nm}\text{/}$ CMOS process, while the best-performing chip
takes advantage of a $5\text{\,}\mathrm{nm}\text{/}$ CMOS technological node.

In conclusion, the answer to our original question is “Let it spike, but at
the right rate, on the right task and, maybe, not alone.”.

## Acknowledgments

This work was partially supported by the Key Digital Technologies Joint
Undertaking under the REBECCA Project with grant agreement number 101097224,
receiving support from the European Union, Greece, Germany, Netherlands,
Spain, Italy, Sweden, Turkey, Lithuania, and Switzerland.

The authors would like to thank: Kim Sangyeob, who is with the Korea Advanced
Institute of Science and Technology (KAIST), Daejeon, South Korea, for
providing the energy per inference measurements for C-DNN’23 [33]; Alexander
Henkes, for reviewing the manuscript draft.

## References

  * Azghadi et al. [2020] M. R. Azghadi, C. Lammie, J. K. Eshraghian, M. Payvand, E. Donati, B. Linares-Barranco, and G. Indiveri.  Hardware implementation of deep network accelerators towards healthcare and biomedical applications.  _IEEE Transactions on Biomedical Circuits and Systems_ , 14(6):1138–1159, 2020.
  * Barchid et al. [2022] S. Barchid, J. Mennesson, J. Eshraghian, C. Djéraba, and M. Bennamoun.  Spiking neural networks for frame-based and event-based single object localization.  _arXiv preprint arXiv:2206.06506_ , 2022.
  * Basu et al. [2022] A. Basu, L. Deng, C. Frenkel, and X. Zhang.  Spiking neural network integrated circuits: A review of trends and future directions.  In _2022 IEEE Custom Integrated Circuits Conference (CICC)_ , pages 1–8. IEEE, 2022.
  * Bouvier et al. [2019] M. Bouvier, A. Valentian, T. Mesquida, F. Rummens, M. Reyboz, E. Vianello, and E. Beigne.  Spiking neural networks hardware implementations and challenges: A survey.  _ACM Journal on Emerging Technologies in Computing Systems (JETC)_ , 15(2):1–35, 2019.
  * Brown et al. [2020] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901, 2020.
  * Chen et al. [2014] G. Chen, C. Parada, and G. Heigold.  Small-footprint keyword spotting using deep neural networks.  In _2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 4087–4091. IEEE, 2014.
  * Chen et al. [2021] Q. Chen, B. Rueckauer, L. Li, T. Delbruck, and S.-C. Liu.  Reducing latency in a converted spiking video segmentation network.  In _2021 IEEE International Symposium on Circuits and Systems (ISCAS)_ , pages 1–5. IEEE, 2021.
  * Chen et al. [2022] Q. Chen, C. Gao, X. Fang, and H. Luan.  Skydiver: A spiking neural network accelerator exploiting spatio-temporal workload balance.  _IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems_ , 41(12):5732–5736, 2022.
  * Cloud [2023] G. Cloud.  System architecture of tpu vm.  https://cloud.google.com/tpu/docs/system-architecture-tpu-vm?hl=en#tpu_v3, 2023\.  Accessed on June 27, 2023.
  * Cordone et al. [2022] L. Cordone, B. Miramond, and P. Thierion.  Object detection with spiking neural networks on automotive event data.  In _2022 International Joint Conference on Neural Networks (IJCNN)_ , pages 1–8. IEEE, 2022.
  * Cramer et al. [2020] B. Cramer, Y. Stradmann, J. Schemmel, and F. Zenke.  The heidelberg spiking data sets for the systematic evaluation of spiking neural networks.  _IEEE Transactions on Neural Networks and Learning Systems_ , 33(7):2744–2757, 2020.
  * Deng et al. [2009] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pages 248–255. Ieee, 2009.
  * Di Mauro et al. [2022] A. Di Mauro, A. S. Prasad, Z. Huang, M. Spallanzani, F. Conti, and L. Benini.  Sne: an energy-proportional digital accelerator for sparse event-based convolutions.  In _2022 Design, Automation & Test in Europe Conference & Exhibition (DATE)_, pages 825–830. IEEE, 2022.
  * Dosovitskiy et al. [2020] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_ , 2020.
  * Douglas et al. [1995] R. Douglas, M. Mahowald, and C. Mead.  Neuromorphic analogue vlsi.  _Annual review of neuroscience_ , 18(1):255–281, 1995.
  * Eshraghian et al. [2021] J. K. Eshraghian, M. Ward, E. Neftci, X. Wang, G. Lenz, G. Dwivedi, M. Bennamoun, D. S. Jeong, and W. D. Lu.  Training spiking neural networks using lessons from deep learning.  _arXiv preprint arXiv:2109.12894_ , 2021.
  * Eshraghian et al. [2022] J. K. Eshraghian, X. Wang, and W. D. Lu.  Memristor-based binarized spiking neural networks: Challenges and applications.  _IEEE Nanotechnology Magazine_ , 16(2):14–23, 2022.
  * Frenkel and Indiveri [2022] C. Frenkel and G. Indiveri.  Reckon: A 28nm sub-mm2 task-agnostic spiking recurrent neural network processor enabling on-chip learning over second-long timescales.  In _2022 IEEE International Solid-State Circuits Conference (ISSCC)_ , volume 65, pages 1–3. IEEE, 2022.
  * Frenkel et al. [2023] C. Frenkel, D. Bol, and G. Indiveri.  Bottom-up and top-down approaches for the design of neuromorphic processing systems: Tradeoffs and synergies between natural and artificial intelligence.  _Proceedings of the IEEE_ , 2023.
  * Fu et al. [2021] A. Fu, M. S. Hosseini, and K. N. Plataniotis.  Reconsidering co2 emissions from computer vision.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 2311–2317, 2021.
  * Gallego et al. [2020] G. Gallego, T. Delbrück, G. Orchard, C. Bartolozzi, B. Taba, A. Censi, S. Leutenegger, A. J. Davison, J. Conradt, K. Daniilidis, et al.  Event-based vision: A survey.  _IEEE transactions on pattern analysis and machine intelligence_ , 44(1):154–180, 2020.
  * Gao et al. [2018] C. Gao, D. Neil, E. Ceolini, S.-C. Liu, and T. Delbruck.  Deltarnn: A power-efficient recurrent neural network accelerator.  In _Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays_ , FPGA ’18, page 21–30, New York, NY, USA, 2018\. Association for Computing Machinery.  ISBN 9781450356145.  doi:10.1145/3174243.3174261.  URL https://doi.org/10.1145/3174243.3174261.
  * Gao et al. [2019] C. Gao, S. Braun, I. Kiselev, J. Anumula, T. Delbruck, and S.-C. Liu.  Real-time speech recognition for iot purpose using a delta recurrent neural network accelerator.  In _2019 IEEE International Symposium on Circuits and Systems (ISCAS)_ , pages 1–5, 2019.  doi:10.1109/ISCAS.2019.8702290.
  * Gao et al. [2020] C. Gao, A. Rios-Navarro, X. Chen, S.-C. Liu, and T. Delbruck.  Edgedrnn: Recurrent neural network accelerator for edge inference.  _IEEE Journal on Emerging and Selected Topics in Circuits and Systems_ , 10(4):419–432, 2020.  doi:10.1109/JETCAS.2020.3040300.
  * Gao et al. [2022] C. Gao, T. Delbruck, and S.-C. Liu.  Spartus: A 9.4 top/s fpga-based lstm accelerator exploiting spatio-temporal sparsity.  _IEEE Transactions on Neural Networks and Learning Systems_ , pages 1–15, 2022.  doi:10.1109/TNNLS.2022.3180209.
  * Gehrig and Scaramuzza [2023] M. Gehrig and D. Scaramuzza.  Recurrent vision transformers for object detection with event cameras.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 13884–13893, 2023.
  * Giraldo et al. [2020] J. S. P. Giraldo, S. Lauwereins, K. Badami, and M. Verhelst.  Vocell: A 65-nm speech-triggered wake-up soc for 10- $\mu$ w keyword spotting and speaker verification.  _IEEE Journal of Solid-State Circuits_ , 55(4):868–878, 2020.  doi:10.1109/JSSC.2020.2968800.
  * Goodfellow et al. [2016] I. Goodfellow, Y. Bengio, and A. Courville.  _Deep learning_.  MIT press, 2016.
  * He et al. [2016] K. He, X. Zhang, S. Ren, and J. Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * Keller et al. [2023] B. Keller, R. Venkatesan, S. Dai, S. G. Tell, B. Zimmer, C. Sakr, W. J. Dally, C. T. Gray, and B. Khailany.  A 95.6-tops/w deep learning inference accelerator with per-vector scaled 4-bit quantization in 5 nm.  _IEEE Journal of Solid-State Circuits_ , 2023.
  * Khan et al. [2022] S. Khan, M. Naseer, M. Hayat, S. W. Zamir, F. S. Khan, and M. Shah.  Transformers in vision: A survey.  _ACM computing surveys (CSUR)_ , 54(10s):1–41, 2022.
  * Kim et al. [2022] K. Kim, C. Gao, R. Graça, I. Kiselev, H.-J. Yoo, T. Delbruck, and S.-C. Liu.  A 23-uw keyword spotting ic with ring-oscillator-based time-domain feature extraction.  _IEEE Journal of Solid-State Circuits_ , 57(11):3298–3311, 2022.  doi:10.1109/JSSC.2022.3195610.
  * Kim et al. [2023a] S. Kim, S. Kim, S. Hong, S. Kim, D. Han, and H.-J. Yoo.  C-dnn: A 24.5-85.8 tops/w complementary-deep-neural-network processor with heterogeneous cnn/snn core architecture and forward-gradient-based sparsity generation.  In _2023 IEEE International Solid-State Circuits Conference (ISSCC)_ , pages 334–336. IEEE, 2023a.
  * Kim et al. [2023b] S. Kim, S. Kim, S. Um, S. Kim, J. Lee, and H.-J. Yoo.  Snpu: An energy-efficient spike domain deep-neural-network processor with two-step spike encoding and shift-and-accumulation unit.  _IEEE Journal of Solid-State Circuits_ , 2023b.
  * Li et al. [2020] J. Li, R. Zhao, E. Sun, J. H. Wong, A. Das, Z. Meng, and Y. Gong.  High-accuracy and low-latency speech recognition with two-head contextual layer trajectory lstm model.  In _ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 7699–7703. IEEE, 2020.
  * Mark [2014] H. Mark.  Computing’s energy problem (and what we can do about it).  In _Proceedings of the IEEE International Solid-State Circuits Conference, San Francisco, CA, USA_ , pages 9–13, 2014.
  * Mo et al. [2021] H. Mo, W. Zhu, W. Hu, G. Wang, Q. Li, A. Li, S. Yin, S. Wei, and L. Liu.  9.2 a 28nm 12.1 tops/w dual-mode cnn processor using effective-weight-based convolution and error-compensation-based prediction.  In _2021 IEEE International Solid-State Circuits Conference (ISSCC)_ , volume 64, pages 146–148. IEEE, 2021.
  * Naumov et al. [2019] M. Naumov, D. Mudigere, H.-J. M. Shi, J. Huang, N. Sundaraman, J. Park, X. Wang, U. Gupta, C.-J. Wu, A. G. Azzolini, et al.  Deep learning recommendation model for personalization and recommendation systems.  _arXiv preprint arXiv:1906.00091_ , 2019.
  * Oh et al. [2019] S. Oh, M. Cho, Z. Shi, J. Lim, Y. Kim, S. Jeong, Y. Chen, R. Rothe, D. Blaauw, H.-S. Kim, and D. Sylvester.  An acoustic signal processing chip with 142-nw voice activity detection using mixer-based sequential frequency scanning and neural network classification.  _IEEE Journal of Solid-State Circuits_ , 54(11):3005–3016, 2019.  doi:10.1109/JSSC.2019.2936756.
  * Ottati [2023] F. Ottati.  Awesome neuromorphic hardware.  https://github.com/fabrizio-ottati/awesome-neuromorphic-hw, 2023\.
  * Panayotov et al. [2015] V. Panayotov, G. Chen, D. Povey, and S. Khudanpur.  Librispeech: an asr corpus based on public domain audio books.  In _2015 IEEE international conference on acoustics, speech and signal processing (ICASSP)_ , pages 5206–5210. IEEE, 2015.
  * Park et al. [2022] J.-S. Park, C. Park, S. Kwon, T. Jeon, Y. Kang, H. Lee, D. Lee, J. Kim, H.-S. Kim, Y. Lee, et al.  A multi-mode 8k-mac hw-utilization-aware neural processing unit with a unified multi-precision datapath in 4-nm flagship mobile soc.  _IEEE Journal of Solid-State Circuits_ , 58(1):189–202, 2022.
  * Payvand et al. [2023] M. Payvand, S. D’Agostino, F. Moro, Y. Demirag, G. Indiveri, and E. Vianello.  Dendritic computation through exploiting resistive memory as both delays and weights.  _arXiv preprint arXiv:2305.06941_ , 2023.
  * Pedroni et al. [2018] B. U. Pedroni, S. Sheik, H. Mostafa, S. Paul, C. Augustine, and G. Cauwenberghs.  Small-footprint spiking neural networks for power-efficient keyword spotting.  In _2018 IEEE Biomedical Circuits and Systems Conference (BioCAS)_ , pages 1–4, 2018.  doi:10.1109/BIOCAS.2018.8584832.
  * Perot et al. [2020] E. Perot, P. De Tournemire, D. Nitti, J. Masci, and A. Sironi.  Learning to detect objects with a 1 megapixel event camera.  _Advances in Neural Information Processing Systems_ , 33:16639–16652, 2020.
  * Schmidgall et al. [2023] S. Schmidgall, J. Achterberg, T. Miconi, L. Kirsch, R. Ziaei, S. Hajiseyedrazi, and J. Eshraghian.  Brain-inspired learning in artificial neural networks: a review.  _arXiv preprint arXiv:2305.11252_ , 2023.
  * Shan et al. [2023] W. Shan, J. Qian, L. Zhu, J. Yang, C. Huang, and H. Cai.  Aad-kws: A sub-u w keyword spotting chip with an acoustic activity detector embedded in mfcc and a tunable detection window in 28-nm cmos.  _IEEE Journal of Solid-State Circuits_ , 58(3):867–876, 2023.  doi:10.1109/JSSC.2022.3197838.
  * Sze et al. [2017] V. Sze, Y.-H. Chen, T.-J. Yang, and J. S. Emer.  Efficient processing of deep neural networks: A tutorial and survey.  _Proceedings of the IEEE_ , 105(12):2295–2329, 2017.
  * Wang et al. [2022] Y. Wang, Y. Qin, D. Deng, J. Wei, Y. Zhou, Y. Fan, T. Chen, H. Sun, L. Liu, S. Wei, et al.  A 28nm 27.5 tops/w approximate-computing-based transformer processor with asymptotic sparsity speculating and out-of-order computing.  In _2022 IEEE International Solid-State Circuits Conference (ISSCC)_ , volume 65, pages 1–3. IEEE, 2022.
  * Warden [2018] P. Warden.  Speech commands: A dataset for limited-vocabulary speech recognition.  _arXiv preprint arXiv:1804.03209_ , 2018.
  * Xu et al. [2019] K. Xu, L. Wen, G. Li, L. Bo, and Q. Huang.  Spatiotemporal cnn for video object segmentation.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 1379–1388, 2019.
  * Yang et al. [2016] M. Yang, C.-H. Chien, T. Delbruck, and S.-C. Liu.  A 0.5 v 55$\mu$w 6x 2 channel binaural silicon cochlea for event-driven stereo-audio sensing.  _IEEE Journal of Solid-State Circuits_ , 51(11):2554–2569, 2016.
  * Yang et al. [2019] M. Yang, C.-H. Yeh, Y. Zhou, J. P. Cerqueira, A. A. Lazar, and M. Seok.  Design of an always-on deep neural network-based 1- $\mu$ w voice activity detector aided with a customized software model for analog feature extraction.  _IEEE Journal of Solid-State Circuits_ , 54(6):1764–1777, 2019.  doi:10.1109/JSSC.2019.2894360.
  * Zhai et al. [2022] X. Zhai, A. Kolesnikov, N. Houlsby, and L. Beyer.  Scaling vision transformers.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 12104–12113, 2022.
  * Zhu et al. [2023] R.-J. Zhu, Q. Zhao, and J. K. Eshraghian.  Spikegpt: Generative pre-trained language model with spiking neural networks.  _arXiv preprint arXiv:2302.13939_ , 2023.

  *[DL]: deep learning
  *[SNNs]: spiking neural network
  *[ANNs]: artificial neural network
  *[SNN]: spiking neural network
  *[ANN]: artificial neural network
  *[AI]: artificial intelligence
  *[LLM]: large language model
  *[EDA]: electronic design automation
  *[SOTA]: state-of-the-art
  *[LIF]: leaky integrate and fire
  *[PE]: processing element
  *[MAC]: multiply and accumulate
  *[SynOP]: synaptic operation
  *[SRAM]: static random access memory
  *[CMOS]: complementary metal-oxide-semiconductor
  *[ReLU]: rectified linear unit
  *[IMC]: in-memory computing
  *[PEs]: processing element
  *[DVS]: dynamic vision sensor
  *[RNN]: recurrent neural network
  *[KWS]: keyword spotting
  *[VAD]: voice activity detection
  *[ASR]: automatic speech recognition


argument, and this fact seems not to have been noticed so far.

###### Lemma 9.1

Let $k\in]0,2]$. Then the functional

$f\mapsto\Psi(f)=\int_{\mathbb{R}^{3}}\int_{\mathbb{R}^{3}}e^{\lambda}\,\chi(f)\,dx\,dv$

is convex.

Proof : Let $\varphi\in C_{0}^{\infty}(\mathbb{R}^{3}\times\mathbb{R}^{3})$.
To begin with,

$e^{\lambda_{f}}=\frac{1}{\sqrt{1-\frac{2m}{r}}}.$

Since $m=m_{f}$ is linear in $f$, we obtain

$D(e^{\lambda_{f}})(\varphi)=\frac{1}{r}\,\Big{(}1-\frac{2m_{f}}{r}\Big{)}^{-3/2}m_{\varphi}=\frac{1}{r}\,e^{3\lambda_{f}}\,m_{\varphi}$

for the derivative. Thus

$D\Psi(f)(\varphi)=\int_{\mathbb{R}^{3}}\int_{\mathbb{R}^{3}}\bigg{(}\frac{1}{r}\,e^{3\lambda_{f}}\,m_{\varphi}\,\chi(f)+e^{\lambda_{f}}\,\chi^{\prime}(f)\varphi\bigg{)}\,dx\,dv.$

In addition,
$D(e^{3\lambda_{f}})(\varphi)=\frac{3}{r}\,e^{5\lambda_{f}}\,m_{\varphi}$.
Hence, for the second derivative,

$D^{2}\Psi(f)(\varphi,\varphi)=\int_{\mathbb{R}^{3}}\int_{\mathbb{R}^{3}}\bigg{[}\frac{3}{r^{2}}\,e^{5\lambda_{f}}\,m_{\varphi}^{2}\,\chi(f)+\frac{2}{r}\,e^{3\lambda_{f}}\,m_{\varphi}\,\chi^{\prime}(f)\varphi+e^{\lambda_{f}}\,\chi^{\prime\prime}(f)\varphi^{2}\bigg{]}\,dx\,dv.$
(9.1)

Using $\chi(f)=\frac{k}{k+1}\,f^{1+1/k}$, $\chi^{\prime}(f)=f^{1/k}$ and
$\chi^{\prime\prime}(f)=\frac{1}{k}\,f^{1/k-1}$, the relation (9.1) can be
rewritten as

$\displaystyle D^{2}\Psi(f)(\varphi,\varphi)$ $\displaystyle=$
$\displaystyle\frac{k}{k+1}\int_{\mathbb{R}^{3}}\int_{\mathbb{R}^{3}}e^{\lambda_{f}}f^{1/k-1}\bigg{[}\frac{3}{r^{2}}\,e^{4\lambda_{f}}\,m_{\varphi}^{2}\,f^{2}+\frac{2}{r}\,e^{2\lambda_{f}}\,m_{\varphi}\,\Big{(}1+\frac{1}{k}\Big{)}f\varphi+\frac{1}{k}\Big{(}1+\frac{1}{k}\Big{)}\varphi^{2}\bigg{]}\,dx\,dv$
$\displaystyle=$
$\displaystyle\frac{k}{k+1}\int_{\mathbb{R}^{3}}\int_{\mathbb{R}^{3}}e^{\lambda_{f}}f^{1/k-1}\bigg{[}\bigg{(}\frac{1}{\sqrt{3}}\Big{(}1+\frac{1}{k}\Big{)}\varphi+\sqrt{3}\,\frac{1}{r}\,e^{2\lambda_{f}}m_{\varphi}f\bigg{)}^{2}+\frac{(2-k)}{3k}\Big{(}1+\frac{1}{k}\Big{)}\varphi^{2}\bigg{]}\,dx\,dv,$

which is positive, owing to $2-k\geq 0$. Thus $\Psi$ is convex. $\Box$

###  9.2 A lemma on $2m/r$

###### Lemma 9.2

Let $M=\int_{\mathbb{R}^{3}}\int_{\mathbb{R}^{3}}\sqrt{1+|v|^{2}}\,f\,dx\,dv$
be the mass and

$\sigma_{M}=\frac{3}{32\pi M^{2}}.$

If $\rho(r)\leq\sigma\leq\sigma_{M}$ for all $r$, then

$\frac{2m(r)}{r}\leq\bigg{(}\frac{\sigma}{\sigma_{M}}\bigg{)}^{1/3}\quad\mbox{for
all}\quad r.$

In particular, if $\sigma<\sigma_{M}$, then

$\frac{2m(r)}{r}\leq\bigg{(}\frac{\sigma}{\sigma_{M}}\bigg{)}^{1/3}<1\quad\mbox{for
all}\quad r.$

Furthermore, if $\rho(r)\leq\sigma_{M}$ for all $r$ and

$\frac{2m(r)}{r}=1\quad\mbox{for some}\quad r=r_{\ast},$ (9.2)

then $r_{\ast}=2M$.

Proof : To establish the first claim, let
$r_{1}=(\frac{3M}{4\pi\sigma})^{1/3}$. From the assumption we have

$m(r)=4\pi\int_{0}^{r}\eta^{2}\,\rho(\eta)\,d\eta\leq\frac{4\pi}{3}\,\sigma\,r^{3}$

for all $r$. Hence, if $r\leq r_{1}$, then we get

$\frac{2m(r)}{r}\leq\frac{8\pi}{3}\,\sigma\,r^{2}\leq\frac{8\pi}{3}\,\sigma\,r_{1}^{2}=\frac{8\pi}{3}\,\sigma\,\bigg{(}\frac{3M}{4\pi\sigma}\bigg{)}^{2/3}=\bigg{(}\frac{\sigma}{\sigma_{M}}\bigg{)}^{1/3}.$
(9.3)

On the other hand, if $r\geq r_{1}$, then also

$\frac{2m(r)}{r}\leq\frac{2M}{r}\leq\frac{2M}{r_{1}}=2M\bigg{(}\frac{4\pi\sigma}{3M}\bigg{)}^{1/3}=\bigg{(}\frac{\sigma}{\sigma_{M}}\bigg{)}^{1/3}.$
(9.4)

For the second assertion, suppose that $\rho(r)\leq\sigma_{M}$ for all $r$ and
that (9.2) holds. We are going to take $\sigma=\sigma_{M}$ in the first part
of the argument. If we had $r_{\ast}<r_{1}$, then repeating (9.3) with the
strict inequality would imply

$1=\frac{2m(r_{\ast})}{r_{\ast}}<1,$

which is impossible. Accordingly, we must have $r_{\ast}\geq r_{1}$. But then
(9.4) leads to

$1=\frac{2m(r_{\ast})}{r_{\ast}}\leq\frac{2M}{r_{\ast}}\leq 1,$

which gives $r_{\ast}=2M$ as desired. $\Box$

## References

  * [1] Andréasson H.: On the Buchdahl inequality for spherically symmetric static shells, Comm. Math. Phys. 274, 399-408 (2007)
  * [2] Andréasson H.: The Einstein-Vlasov system/kinetic theory, Living Rev. Relativ. 14 (2011)
  * [3] Andréasson H.: Sharp bounds on $2m/r$ of general spherically symmetric static objects, J. Differential Equations 245, 2243-2266 (2008)
  * [4] Andréasson H. & Kunze M.: Comments on the paper ‘Static solutions of the Vlasov-Einstein system’ by G. Wolansky, Arch. Ration. Mech. Anal. 235, 783-791 (2020)
  * [5] Andréasson H. & Kunze M.: Static solutions to the spherically symmetric Einstein-Vlasov system: A particle-number-Casimir approach, SIAM J. Math. Anal.  55, 4843-4879 (2023)
  * [6] Andréasson H. & Rein G.: A numerical investigation of the stability of steady states and critical phenomena for the spherically symmetric Einstein-Vlasov system, Classical Quantum Gravity 23, 3659-3677 (2006)
  * [7] Dacorogna B.: Direct Methods in the Calculus of Variations, Springer, Berlin-New York 1989
  * [8] Günther S.; Körner, J.; Lebeda, T.; Pötzl, B.; Rein, G.; Straub, C. & Weber J.: A numerical stability analysis for the Einstein-Vlasov system, Classical Quantum Gravity 38, 035003 (2021)
  * [9] Günther S., Rein G. & Straub C.: Collisionless equilibria in general relativity: stable configurations beyond the first binding energy maximum, Astrophys. J. 918:48 (12pp) (2021)
  * [10] Guo Y. & Rein G.: Stable steady states in stellar dynamics, Arch. Ration. Mech. Anal. 147, 225-243 (1999)
  * [11] Lemou M., Méhats F. & Raphaël P.: Orbital stability of spherical galactic models, Invent. Math. 187, 145-194 (2012)
  * [12] Rein G.: Collisionless kinetic equations from astrophysics–the Vlasov-Poisson system, in Handbook of Differential Equations: Evolutionary Equations. Vol. III, Elsevier/North-Holland, Amsterdam 2007, pp. 383-476
  * [13] Wolansky G.: Static solutions of the Vlasov-Einstein system, Arch. Ration. Mech. Anal. 156, 205-230 (2001)
  * [14] Zel’dovich Y.B. & Podurets M.A.: The evolution of a system of gravitationally interacting point masses, Soviet Astronomy 9, 742-749 (1966)