random error in the final estimation of the parameters. The maximum-likelihood
cost function reads

$\displaystyle\mathsf{C}^{\rm
pd}_{\mathrm{TL}}(\bm{\theta})=-\sum_{i,m_{x}}N_{i}\tilde{f}_{i,m_{x}}\log
p_{i}(m_{x}|\bm{\theta}),$ (69)

where $\tilde{f}_{i,m_{x}}$ denotes
$\tilde{f}_{i}(m_{x}\,|\bm{\theta}_{\star})$ and $p_{i}(m_{x}|\bm{\theta})$
denotes $p^{\rm TL}_{i}(m_{x}|\bm{\theta})$. The minimum of the cost function
satisfies

$\partial_{\bm{\theta}}C_{\rm TL}^{\rm
pd}=-\sum_{i,m_{x}}N_{i}\tilde{f}_{i,m_{x}}\frac{\partial_{\bm{\theta}}p_{i}(m_{x}|\bm{\theta})}{p_{i}(m_{x}|\bm{\theta})}=0.$
(70)

Since
$\tilde{f}_{i,m_{x}}=p_{i}(m_{x}|\bm{\theta}_{\star})+\Delta\tilde{f}_{i,m_{x}}$,
with $\Delta\tilde{f}_{i,m_{x}}$ small, the minimum of the cost function is
slightly displaced from the true minimum $\bm{\theta}_{\star}$ to
$\bm{\theta}_{\star}+\Delta\bm{\theta}$. Taylor expanding Eq. (70) around
$\bm{\theta}_{\star}$ to first order we have

$\displaystyle\partial_{\theta_{j}}C_{\rm TL}^{\rm pd}\approx$
$\displaystyle-\sum_{i,m_{x}}N_{i}\left[p_{i}(m_{x}|\bm{\theta})+\Delta\tilde{f}_{i,m_{x}}\right]\left[\left.\frac{\partial_{\theta_{j}}p_{i}(m_{x}|\bm{\theta})}{p_{i}(m_{x}|\bm{\theta})}\right|_{\bm{\theta}=\bm{\theta}_{\star}}\right.$
$\displaystyle\left.+\sum_{k}\left.\partial_{\theta_{k}}\frac{\partial_{\theta_{j}}p_{i}(m_{x}|\bm{\theta})}{p_{i}(m_{x}|\bm{\theta})}\right|_{\bm{\theta}=\bm{\theta}_{\star}}\Delta\theta_{k}\right]=0.$
(71)

Keeping only first-order terms in $\Delta\bm{\theta}$ and
$\Delta\tilde{f}_{i,m_{x}}$ and simplifying we obtain

$\sum_{i,j}\Delta\bm{\theta}_{j}N_{i}[I_{i}(\bm{\theta}_{\star})]_{jk}=\sum_{i,m_{x}}N_{i}\Delta\tilde{f}_{i,m_{x}}\left.\frac{\partial_{\theta_{k}}p_{i}(m_{x}|\bm{\theta})}{p_{i}(m_{x}|\bm{\theta})}\right|_{\bm{\theta}=\bm{\theta}_{\star}},$
(72)

where we have defined the matrix

$\displaystyle[I_{i}$ $\displaystyle(\bm{\theta}_{\star})]_{jk}=$ (73)
$\displaystyle\sum_{m_{x}}p_{i}(m_{x}|\bm{\theta}_{\star})\partial_{\theta_{j}}\log
p_{i}(m_{x}|\bm{\theta})\partial_{\theta_{k}}\log
p_{i}(m_{x}|\bm{\theta})|_{\bm{\theta}=\bm{\theta}_{\star}}.$

Note that this is the Fisher information given in Eq. (30). Taking into
account that $\Delta\tilde{f}_{i,1}=-\Delta\tilde{f}_{i,0}$ and
$\tilde{f}_{i,1}=1-\tilde{f}_{i,0}$, and defining the matrices

$J_{jk}=\sum_{i}N_{i}[I_{i}(\bm{\theta}_{\star})]_{jk},\quad\
F_{ji}=N_{i}\frac{\left.\partial_{\theta_{j}}p_{i}(0|\bm{\theta})\right|_{\bm{\theta}=\bm{\theta}_{\star}}}{p_{i}(0|\bm{\theta}_{\star})(1-p_{i}(0|\bm{\theta}_{\star}))},$
(74)

we arrive at the expression

$\Delta\theta_{k}=\sum_{i,j}J^{-1}_{jk}F_{ji}\Delta\tilde{f}_{i,0},$ (75)

which relates differences between the expected and measured values
$\Delta\tilde{f}_{i,m_{x}}=\tilde{f}_{i,m_{x}}-p_{i}(m_{x}|\bm{\theta}_{\star})$
to differences between the estimated and the true parameters
$\Delta\theta_{k}$. When $N_{i}$ is sufficiently large, $\tilde{f}_{i,0}$
behaves as the normal distribution
$N\left[\mu=p_{i}(0|\bm{\theta}_{\star}),\sigma_{f}^{2}=p_{i}(0|\bm{\theta}_{\star})p_{i}(1|\bm{\theta}_{\star})/N_{i}\right]$.
Thus, we have

$\displaystyle\Delta\tilde{f}_{i,0}\sim
N\left[\mu=0,\sigma^{2}_{f_{i}}=p_{i}(0|\bm{\theta}_{\star})p_{i}(1|\bm{\theta}_{\star})/N_{i}\right].$
(76)

After the linear transformation (75), we obtain that $\Delta\bm{\theta}$
behaves as

$\Delta\bm{\theta}\sim N\left[\mu=0,\
\Sigma_{\hat{\bm{\theta}}}=J^{-1}\right],$ (77)

where we have used that
$J^{-1}F\mathrm{diag}(\sigma^{2}_{f})F^{T}(J^{-1})^{T}=J^{-1}$. Thus, the shot
noise produces a normal random error with covariance matrix
$\Sigma_{\bm{\theta}}=J^{-1}$ which scales as $1/N_{\rm shot}$, with $N_{\rm
shot}=\sum_{i}N_{i}$, which is the result presented in Eq. (31).

## References

  * Nielsen and Chuang [2010] M. A. Nielsen and I. L. Chuang,  _Quantum Computation and Quantum Information: 10th Anniversary Edition_ (Cambridge University Press, 2010).
  * Bruzewicz  _et al._ [2019] C. D. Bruzewicz, J. Chiaverini, R. McConnell, and J. M. Sage, Trapped-ion quantum computing: Progress and challenges, Applied Physics Reviews 6, 021314 (2019).
  * Morgado and Whitlock [2021] M. Morgado and S. Whitlock, Quantum simulation and computing with Rydberg-interacting qubits, AVS Quantum Science 3, 023501 (2021).
  * Wendin [2017] G. Wendin, Quantum information processing with superconducting circuits: a review, Reports on Progress in Physics 80, 106001 (2017).
  * Arute  _et al._ [2019] F. Arute, K. Arya, R. Babbush, D. Bacon, J. C. Bardin, R. Barends, R. Biswas, S. Boixo, F. G. S. L. Brandao, D. A. Buell, B. Burkett, Y. Chen, Z. Chen, B. Chiaro, R. Collins, W. Courtney, A. Dunsworth, E. Farhi, B. Foxen, A. Fowler, C. Gidney, M. Giustina, R. Graff, K. Guerin, S. Habegger, M. P. Harrigan, M. J. Hartmann, A. Ho, M. Hoffmann, T. Huang, T. S. Humble, S. V. Isakov, E. Jeffrey, Z. Jiang, D. Kafri, K. Kechedzhi, J. Kelly, P. V. Klimov, S. Knysh, A. Korotkov, F. Kostritsa, D. Landhuis, M. Lindmark, E. Lucero, D. Lyakh, S. Mandrà, J. R. McClean, M. McEwen, A. Megrant, X. Mi, K. Michielsen, M. Mohseni, J. Mutus, O. Naaman, M. Neeley, C. Neill, M. Y. Niu, E. Ostby, A. Petukhov, J. C. Platt, C. Quintana, E. G. Rieffel, P. Roushan, N. C. Rubin, D. Sank, K. J. Satzinger, V. Smelyanskiy, K. J. Sung, M. D. Trevithick, A. Vainsencher, B. Villalonga, T. White, Z. J. Yao, P. Yeh, A. Zalcman, H. Neven, and J. M. Martinis, Quantum supremacy using a programmable superconducting processor, Nature 574, 505 (2019).
  * Zhong  _et al._ [2020] H.-S. Zhong, H. Wang, Y.-H. Deng, M.-C. Chen, L.-C. Peng, Y.-H. Luo, J. Qin, D. Wu, X. Ding, Y. Hu, P. Hu, X.-Y. Yang, W.-J. Zhang, H. Li, Y. Li, X. Jiang, L. Gan, G. Yang, L. You, Z. Wang, L. Li, N.-L. Liu, C.-Y. Lu, and J.-W. Pan, Quantum computational advantage using photons, Science 370, 1460 (2020).
  * Pino  _et al._ [2021] J. M. Pino, J. M. Dreiling, C. Figgatt, J. P. Gaebler, S. A. Moses, M. S. Allman, C. H. Baldwin, M. Foss-Feig, D. Hayes, K. Mayer, C. Ryan-Anderson, and B. Neyenhuis, Demonstration of the trapped-ion quantum CCD computer architecture, Nature 592, 209 (2021).
  * Egan  _et al._ [2021] L. Egan, D. M. Debroy, C. Noel, A. Risinger, D. Zhu, D. Biswas, M. Newman, M. Li, K. R. Brown, M. Cetina, and C. Monroe, Fault-tolerant control of an error-corrected qubit, Nature 598, 281 (2021).
  * Postler  _et al._ [2022] L. Postler, S. Heu$\beta$en, I. Pogorelov, M. Rispler, T. Feldker, M. Meth, C. D. Marciniak, R. Stricker, M. Ringbauer, R. Blatt, P. Schindler, M. Müller, and T. Monz, Demonstration of fault-tolerant universal quantum gate operations, Nature 605, 675 (2022).
  * Krinner  _et al._ [2022] S. Krinner, N. Lacroix, A. Remm, A. Di Paolo, E. Genois, C. Leroux, C. Hellings, S. Lazar, F. Swiadek, J. Herrmann, G. J. Norris, C. K. Andersen, M. Müller, A. Blais, C. Eichler, and A. Wallraff, Realizing repeated quantum error correction in a distance-three surface code, Nature 605, 669 (2022).
  * Zhao  _et al._ [2022] Y. Zhao, Y. Ye, H.-L. Huang, Y. Zhang, D. Wu, H. Guan, Q. Zhu, Z. Wei, T. He, S. Cao, F. Chen, T.-H. Chung, H. Deng, D. Fan, M. Gong, C. Guo, S. Guo, L. Han, N. Li, S. Li, Y. Li, F. Liang, J. Lin, H. Qian, H. Rong, H. Su, L. Sun, S. Wang, Y. Wu, Y. Xu, C. Ying, J. Yu, C. Zha, K. Zhang, Y.-H. Huo, C.-Y. Lu, C.-Z. Peng, X. Zhu, and J.-W. Pan, Realization of an error-correcting surface code with superconducting qubits, Phys. Rev. Lett. 129, 030501 (2022).
  * Acharya  _et al._ [2023] R. Acharya, I. Aleiner, R. Allen, T. I. Andersen, M. Ansmann, F. Arute, K. Arya, A. Asfaw, J. Atalaya, R. Babbush, D. Bacon, J. C. Bardin, J. Basso, A. Bengtsson, S. Boixo, G. Bortoli, A. Bourassa, J. Bovaird, L. Brill, M. Broughton, B. B. Buckley, D. A. Buell, T. Burger, B. Burkett, N. Bushnell, Y. Chen, Z. Chen, B. Chiaro, J. Cogan, R. Collins, P. Conner, W. Courtney, A. L. Crook, B. Curtin, D. M. Debroy, A. Del Toro Barba, S. Demura, A. Dunsworth, D. Eppens, C. Erickson, L. Faoro, E. Farhi, R. Fatemi, L. Flores Burgos, E. Forati, A. G. Fowler, B. Foxen, W. Giang, C. Gidney, D. Gilboa, M. Giustina, A. Grajales Dau, J. A. Gross, S. Habegger, M. C. Hamilton, M. P. Harrigan, S. D. Harrington, O. Higgott, J. Hilton, M. Hoffmann, S. Hong, T. Huang, A. Huff, W. J. Huggins, L. B. Ioffe, S. V. Isakov, J. Iveland, E. Jeffrey, Z. Jiang, C. Jones, P. Juhas, D. Kafri, K. Kechedzhi, J. Kelly, T. Khattar, M. Khezri, M. Kieferová, S. Kim, A. Kitaev, P. V. Klimov, A. R. Klots, A. N. Korotkov, F. Kostritsa, J. M. Kreikebaum, D. Landhuis, P. Laptev, K.-M. Lau, L. Laws, J. Lee, K. Lee, B. J. Lester, A. Lill, W. Liu, A. Locharla, E. Lucero, F. D. Malone, J. Marshall, O. Martin, J. R. McClean, T. McCourt, M. McEwen, A. Megrant, B. Meurer Costa, X. Mi, K. C. Miao, M. Mohseni, S. Montazeri, A. Morvan, E. Mount, W. Mruczkiewicz, O. Naaman, M. Neeley, C. Neill, A. Nersisyan, H. Neven, M. Newman, J. H. Ng, A. Nguyen, M. Nguyen, M. Y. Niu, T. E. O’Brien, A. Opremcak, J. Platt, A. Petukhov, R. Potter, L. P. Pryadko, C. Quintana, P. Roushan, N. C. Rubin, N. Saei, D. Sank, K. Sankaragomathi, K. J. Satzinger, H. F. Schurkus, C. Schuster, M. J. Shearn, A. Shorter, V. Shvarts, J. Skruzny, V. Smelyanskiy, W. C. Smith, G. Sterling, D. Strain, M. Szalay, A. Torres, G. Vidal, B. Villalonga, C. Vollgraff Heidweiller, T. White, C. Xing, Z. J. Yao, P. Yeh, J. Yoo, G. Young, A. Zalcman, Y. Zhang, N. Zhu, and G. Q. AI, Suppressing quantum errors by scaling a surface code logical qubit, Nature 614, 676 (2023).
  * Kim  _et al._ [2023] Y. Kim, A. Eddins, S. Anand, K. X. Wei, E. van den Berg, S. Rosenblatt, H. Nayfeh, Y. Wu, M. Zaletel, K. Temme, and A. Kandala, Evidence for the utility of quantum computing before fault tolerance, Nature 618, 500 (2023).
  * Bluvstein  _et al._ [2024] D. Bluvstein, S. J. Evered, A. A. Geim, S. H. Li, H. Zhou, T. Manovitz, S. Ebadi, M. Cain, M. Kalinowski, D. Hangleiter, J. P. Bonilla Ataides, N. Maskara, I. Cong, X. Gao, P. Sales Rodriguez, T. Karolyshyn, G. Semeghini, M. J. Gullans, M. Greiner, V. Vuletić, and M. D. Lukin, Logical quantum processor based on reconfigurable atom arrays, Nature 626, 58 (2024).
  * Gupta  _et al._ [2024] R. S. Gupta, N. Sundaresan, T. Alexander, C. J. Wood, S. T. Merkel, M. B. Healy, M. Hillenbrand, T. Jochym-O’Connor, J. R. Wootton, T. J. Yoder, A. W. Cross, M. Takita, and B. J. Brown, Encoding a magic state with beyond break-even fidelity, Nature 625, 259 (2024).
  * Wang  _et al._ [2023] Y. Wang, S. Simsek, T. M. Gatterman, J. A. Gerber, K. Gilmore, D. Gresh, N. Hewitt, C. V. Horst, M. Matheny, T. Mengle, B. Neyenhuis, and B. Criger, Fault-tolerant one-bit addition with the smallest interesting colour code (2023), arXiv:2309.09893 [quant-ph] .
  * Yamamoto  _et al._ [2024] K. Yamamoto, S. Duffield, Y. Kikuchi, and D. Muñoz Ramo, Demonstrating Bayesian quantum phase estimation with quantum error detection, Physical Review Research 6, 10.1103/physrevresearch.6.013221 (2024).
  * Ryan-Anderson  _et al._ [2022] C. Ryan-Anderson, N. C. Brown, M. S. Allman, B. Arkin, G. Asa-Attuah, C. Baldwin, J. Berg, J. G. Bohnet, S. Braxton, N. Burdick, J. P. Campora, A. Chernoguzov, J. Esposito, B. Evans, D. Francois, J. P. Gaebler, T. M. Gatterman, J. Gerber, K. Gilmore, D. Gresh, A. Hall, A. Hankin, J. Hostetter, D. Lucchetti, K. Mayer, J. Myers, B. Neyenhuis, J. Santiago, J. Sedlacek, T. Skripka, A. Slattery, R. P. Stutz, J. Tait, R. Tobey, G. Vittorini, J. Walker, and D. Hayes, Implementing fault-tolerant entangling gates on the five-qubit code and the color code (2022).
  * Dalzell  _et al._ [2023] A. M. Dalzell, S. McArdle, M. Berta, P. Bienias, C.-F. Chen, A. Gilyén, C. T. Hann, M. J. Kastoryano, E. T. Khabiboulline, A. Kubica, G. Salton, S. Wang, and F. G. S. L. Brandão, Quantum algorithms: A survey of applications and end-to-end complexities (2023), arXiv:2310.03011 [quant-ph] .
  * Viola  _et al._ [1999] L. Viola, E. Knill, and S. Lloyd, Dynamical decoupling of open quantum systems, Phys. Rev. Lett. 82, 2417 (1999).
  * Khodjasteh and Lidar [2005] K. Khodjasteh and D. A. Lidar, Fault-tolerant quantum dynamical decoupling, Phys. Rev. Lett. 95, 180501 (2005).
  * Khodjasteh and Viola [2009] K. Khodjasteh and L. Viola, Dynamically error-corrected gates for universal quantum computation, Phys. Rev. Lett. 102, 080501 (2009).
  * Lidar [2014] D. A. Lidar, Review of decoherence-free subspaces, noiseless subsystems, and dynamical decoupling, in  _Quantum Information and Computation for Chemistry_ (John Wiley & Sons, Ltd, 2014) pp. 295–354.
  * Temme  _et al._ [2017] K. Temme, S. Bravyi, and J. M. Gambetta, Error mitigation for short-depth quantum circuits, Phys. Rev. Lett. 119, 180509 (2017).
  * Nation  _et al._ [2021] P. D. Nation, H. Kang, N. Sundaresan, and J. M. Gambetta, Scalable mitigation of measurement errors on quantum computers, PRX Quantum 2, 040326 (2021).
  * van den Berg  _et al._ [2023] E. van den Berg, Z. K. Minev, A. Kandala, and K. Temme, Probabilistic error cancellation with sparse Pauli–Lindblad models on noisy quantum processors, Nature Physics 19, 1116 (2023).
  * Cai  _et al._ [2023] Z. Cai, R. Babbush, S. C. Benjamin, S. Endo, W. J. Huggins, Y. Li, J. R. McClean, and T. E. O’Brien, Quantum error mitigation, Rev. Mod. Phys. 95, 045005 (2023).
  * Calderbank and Shor [1996] A. R. Calderbank and P. W. Shor, Good quantum error-correcting codes exist, Phys. Rev. A 54, 1098 (1996).
  * Steane [1996] A. M. Steane, Error correcting codes in quantum theory, Phys. Rev. Lett. 77, 793 (1996).
  * Terhal [2015] B. M. Terhal, Quantum error correction for quantum memories, Rev. Mod. Phys. 87, 307 (2015).
  * Eisert  _et al._ [2020] J. Eisert, D. Hangleiter, N. Walk, I. Roth, D. Markham, R. Parekh, U. Chabaud, and E. Kashefi, Quantum certification and benchmarking, Nature Reviews Physics 2, 382 (2020).
  * Kliesch and Roth [2021] M. Kliesch and I. Roth, Theory of quantum system certification, PRX Quantum 2, 010201 (2021).
  * Gebhart  _et al._ [2023] V. Gebhart, R. Santagati, A. A. Gentile, E. M. Gauger, D. Craig, N. Ares, L. Banchi, F. Marquardt, L. Pezzè, and C. Bonato, Learning quantum systems, Nature Reviews Physics 5, 141 (2023).
  * Zanardi and Rasetti [1997] P. Zanardi and M. Rasetti, Noiseless quantum codes, Phys. Rev. Lett. 79, 3306 (1997).
  * Lidar  _et al._ [1998] D. A. Lidar, I. L. Chuang, and K. B. Whaley, Decoherence-free subspaces for quantum computation, Phys. Rev. Lett. 81, 2594 (1998).
  * Knill  _et al._ [2000] E. Knill, R. Laflamme, and L. Viola, Theory of quantum error correction for general noise, Phys. Rev. Lett. 84, 2525 (2000).
  * Kempe  _et al._ [2001] J. Kempe, D. Bacon, D. A. Lidar, and K. B. Whaley, Theory of decoherence-free fault-tolerant universal quantum computation, Phys. Rev. A 63, 042307 (2001).
  * Lidar and Birgitta Whaley [2003] D. A. Lidar and K. Birgitta Whaley, Decoherence-free subspaces and subsystems, in  _Lecture Notes in Physics_ (Springer Berlin Heidelberg, 2003) p. 83–120.
  * Viola and Lloyd [1998] L. Viola and S. Lloyd, Dynamical suppression of decoherence in two-state quantum systems, Phys. Rev. A 58, 2733 (1998).
  * Kofman and Kurizki [2004] A. G. Kofman and G. Kurizki, Unified theory of dynamically suppressed qubit decoherence in thermal baths, Phys. Rev. Lett. 93, 130406 (2004).
  * Gordon  _et al._ [2007] G. Gordon, N. Erez, and G. Kurizki, Universal dynamical decoherence control of noisy single- and multi-qubit systems, Journal of Physics B: Atomic, Molecular and Optical Physics 40, S75 (2007).
  * Uhrig [2007] G. S. Uhrig, Keeping a quantum bit alive by optimized $\pi$-pulse sequences, Phys. Rev. Lett. 98, 100504 (2007).
  * Cywiński  _et al._ [2008] L. Cywiński, R. M. Lutchyn, C. P. Nave, and S. Das Sarma, How to enhance dephasing time in superconducting qubits, Phys. Rev. B 77, 174509 (2008).
  * Biercuk  _et al._ [2011] M. J. Biercuk, A. C. Doherty, and H. Uys, Dynamical decoupling sequence construction as a filter-design problem, Journal of Physics B: Atomic, Molecular and Optical Physics 44, 154002 (2011).
  * Clemens  _et al._ [2004] J. P. Clemens, S. Siddiqui, and J. Gea-Banacloche, Quantum error correction against correlated noise, Phys. Rev. A 69, 062313 (2004).
  * Klesse and Frank [2005] R. Klesse and S. Frank, Quantum error correction in spatially correlated quantum noise, Phys. Rev. Lett. 95, 230503 (2005).
  * Aharonov  _et al._ [2006] D. Aharonov, A. Kitaev, and J. Preskill, Fault-tolerant quantum computation with long-range correlated noise, Phys. Rev. Lett. 96, 050504 (2006).
  * Terhal and Burkard [2005] B. M. Terhal and G. Burkard, Fault-tolerant quantum computation for local non-Markovian noise, Phys. Rev. A 71, 012336 (2005).
  * Aliferis  _et al._ [2006] P. Aliferis, D. Gottesman, and J. Preskill, Quantum accuracy threshold for concatenated distance-3 codes, Quantum Info. Comput. 6, 97–165 (2006).
  * Ng and Preskill [2009] H. K. Ng and J. Preskill, Fault-tolerant quantum computation versus Gaussian noise, Phys. Rev. A 79, 032318 (2009).
  * Ángel Rivas  _et al._ [2014] Ángel Rivas, S. F. Huelga, and M. B. Plenio, Quantum non-Markovianity: characterization, quantification and detection, Reports on Progress in Physics 77, 094001 (2014).
  * Breuer  _et al._ [2016] H.-P. Breuer, E.-M. Laine, J. Piilo, and B. Vacchini, Colloquium: Non-Markovian dynamics in open quantum systems, Rev. Mod. Phys. 88, 021002 (2016).
  * Li  _et al._ [2018] L. Li, M. J. Hall, and H. M. Wiseman, Concepts of quantum non-Markovianity: A hierarchy, Physics Reports 759, 1–51 (2018).
  * Addis  _et al._ [2015] C. Addis, F. Ciccarello, M. Cascio, G. M. Palma, and S. Maniscalco, Dynamical decoupling efficiency versus quantum non-Markovianity, New Journal of Physics 17, 123004 (2015).
  * Berk  _et al._ [2023] G. D. Berk, S. Milz, F. A. Pollock, and K. Modi, Extracting quantum dynamical resources: consumption of non-Markovianity for noise reduction, npj Quantum Information 9, 104 (2023).
  * Hakoshima  _et al._ [2021] H. Hakoshima, Y. Matsuzaki, and S. Endo, Relationship between costs for quantum error mitigation and non-Markovian measures, Phys. Rev. A 103, 012611 (2021).
  * Donvil  _et al._ [2023] B. I. C. Donvil, R. Lechler, J. Ankerhold, and P. Muratore-Ginanneschi, Quantum trajectory approach to error mitigation (2023), arXiv:2305.19874 [quant-ph] .
  * Chuang and Nielsen [1997] I. L. Chuang and M. A. Nielsen, Prescription for experimental determination of the dynamics of a quantum black box, Journal of Modern Optics 44, 2455 (1997).
  * Poyatos  _et al._ [1997] J. F. Poyatos, J. I. Cirac, and P. Zoller, Complete characterization of a quantum process: The two-bit quantum gate, Phys. Rev. Lett. 78, 390 (1997).
  * Fiurášek and Hradil [2001] J. Fiurášek and Z. c. v. Hradil, Maximum-likelihood estimation of quantum processes, Phys. Rev. A 63, 020101 (2001).
  * Sacchi [2001] M. F. Sacchi, Maximum-likelihood reconstruction of completely positive maps, Phys. Rev. A 63, 054104 (2001).
  * Ježek  _et al._ [2003] M. Ježek, J. Fiurášek, and Z. c. v. Hradil, Quantum inference of states and processes, Phys. Rev. A 68, 012305 (2003).
  * Nielsen and Chuang [2000] M. A. Nielsen and I. L. Chuang,  _Quantum Computation and Quantum Information_ (Cambridge University Press, 2000).
  * Watrous [2018] J. Watrous,  _The Theory of Quantum Information_ (Cambridge University Press, 2018).
  * Childs  _et al._ [2001] A. M. Childs, I. L. Chuang, and D. W. Leung, Realization of quantum process tomography in NMR, Phys. Rev. A 64, 012314 (2001).
  * Weinstein  _et al._ [2004] Y. S. Weinstein, T. F. Havel, J. Emerson, N. Boulant, M. Saraceno, S. Lloyd, and D. G. Cory, Quantum process tomography of the quantum Fourier transform, The Journal of Chemical Physics 121, 6117 (2004).
  * Mitchell  _et al._ [2003] M. W. Mitchell, C. W. Ellenor, S. Schneider, and A. M. Steinberg, Diagnosis, prescription, and prognosis of a Bell-state filter by quantum process tomography, Phys. Rev. Lett. 91, 120402 (2003).
  * O’Brien  _et al._ [2004] J. L. O’Brien, G. J. Pryde, A. Gilchrist, D. F. V. James, N. K. Langford, T. C. Ralph, and A. G. White, Quantum process tomography of a controlled-not gate, Phys. Rev. Lett. 93, 080502 (2004).
  * Riebe  _et al._ [2006] M. Riebe, K. Kim, P. Schindler, T. Monz, P. O. Schmidt, T. K. Körber, W. Hänsel, H. Häffner, C. F. Roos, and R. Blatt, Process tomography of ion trap quantum gates, Phys. Rev. Lett. 97, 220407 (2006).
  * Home  _et al._ [2009] J. P. Home, D. Hanneke, J. D. Jost, J. M. Amini, D. Leibfried, and D. J. Wineland, Complete methods set for scalable ion trap quantum information processing, Science 325, 1227 (2009).
  * Monz  _et al._ [2009] T. Monz, K. Kim, W. Hänsel, M. Riebe, A. S. Villar, P. Schindler, M. Chwalla, M. Hennrich, and R. Blatt, Realization of the quantum Toffoli gate with trapped ions, Phys. Rev. Lett. 102, 040501 (2009).
  * Bialczak  _et al._ [2010] R. C. Bialczak, M. Ansmann, M. Hofheinz, E. Lucero, M. Neeley, A. D. O’Connell, D. Sank, H. Wang, J. Wenner, M. Steffen, A. N. Cleland, and J. M. Martinis, Quantum process tomography of a universal entangling gate implemented with Josephson phase qubits, Nature Physics 6, 409 (2010).
  * Poletto  _et al._ [2012] S. Poletto, J. M. Gambetta, S. T. Merkel, J. A. Smolin, J. M. Chow, A. D. Córcoles, G. A. Keefe, M. B. Rothwell, J. R. Rozen, D. W. Abraham, C. Rigetti, and M. Steffen, Entanglement of two superconducting qubits in a waveguide cavity via monochromatic two-photon excitation, Phys. Rev. Lett. 109, 240505 (2012).
  * Breuer and Petruccione [2002a] H. P. Breuer and F. Petruccione,  _The theory of open quantum systems_ (Oxford University Press, Great Clarendon Street, 2002).
  * de Vega and Alonso [2017] I. de Vega and D. Alonso, Dynamics of non-Markovian open quantum systems, Rev. Mod. Phys. 89, 015001 (2017).
  * Chruściński [2022] D. Chruściński, Dynamical maps beyond Markovian regime (2022), arXiv:2209.14902 [quant-ph] .
  * Velázquez  _et al._ [2024] J. M. S. Velázquez, A. Steiner, R. Freund, M. Guevara-Bertsch, C. D. Marciniak, T. Monz, and A. Bermudez, Dynamical quantum maps for single-qubit gates under non-Markovian phase noise (2024), arXiv:2402.14530 [quant-ph] .
  * Lindblad [1976] G. Lindblad, On the generators of quantum dynamical semigroups, Communications in Mathematical Physics 48, 119 (1976).
  * Gorini  _et al._ [1976] V. Gorini, A. Kossakowski, and E. C. G. Sudarshan, Completely positive dynamical semigroups of n‐level systems, Journal of Mathematical Physics 17, 821 (1976).
  * Wolf  _et al._ [2008] M. M. Wolf, J. Eisert, T. S. Cubitt, and J. I. Cirac, Assessing non-Markovian quantum dynamics, Physical Review Letters 101, 10.1103/physrevlett.101.150402 (2008).
  * Boulant  _et al._ [2003] N. Boulant, T. F. Havel, M. A. Pravia, and D. G. Cory, Robust method for estimating the Lindblad operators of a dissipative quantum process from measurements of the density operator at multiple time points, Phys. Rev. A 67, 042322 (2003).
  * Howard  _et al._ [2006] M. Howard, J. Twamley, C. Wittmann, T. Gaebel, F. Jelezko, and J. Wrachtrup, Quantum process tomography and Linblad estimation of a solid-state qubit, New Journal of Physics 8, 33 (2006).
  * Onorati  _et al._ [2021] E. Onorati, T. Kohler, and T. Cubitt, Fitting quantum noise models to tomography data (2021), arXiv:2103.17243 [quant-ph] .
  * Bairey  _et al._ [2020] E. Bairey, C. Guo, D. Poletti, N. H. Lindner, and I. Arad, Learning the dynamics of open quantum systems from their steady states, New Journal of Physics 22, 032001 (2020).
  * Pastori  _et al._ [2022] L. Pastori, T. Olsacher, C. Kokail, and P. Zoller, Characterization and verification of Trotterized digital quantum simulation via Hamiltonian and Liouvillian learning, PRX Quantum 3, 030324 (2022).
  * França  _et al._ [2022] D. S. França, L. A. Markovich, V. V. Dobrovitski, A. H. Werner, and J. Borregaard, Efficient and robust estimation of many-qubit Hamiltonians (2022), arXiv:2205.09567 [quant-ph] .
  * Samach  _et al._ [2022] G. O. Samach, A. Greene, J. Borregaard, M. Christandl, J. Barreto, D. K. Kim, C. M. McNally, A. Melville, B. M. Niedzielski, Y. Sung, D. Rosenberg, M. E. Schwartz, J. L. Yoder, T. P. Orlando, J. I.-J. Wang, S. Gustavsson, M. Kjaergaard, and W. D. Oliver, Lindblad tomography of a superconducting quantum processor, Phys. Rev. Appl. 18, 064056 (2022).
  * Zhang  _et al._ [2022] H. Zhang, B. Pokharel, E. Levenson-Falk, and D. Lidar, Predicting non-Markovian superconducting-qubit dynamics from tomographic reconstruction, Phys. Rev. Appl. 17, 054018 (2022).
  * Ben Av  _et al._ [2020] E. Ben Av, Y. Shapira, N. Akerman, and R. Ozeri, Direct reconstruction of the quantum-master-equation dynamics of a trapped-ion qubit, Phys. Rev. A 101, 062305 (2020).
  * Dobrynin  _et al._ [2024] D. Dobrynin, L. Cardarelli, M. Müller, and A. Bermudez, Compressed-sensing Lindbladian quantum tomography with trapped ions (2024), arXiv:2403.07462 [quant-ph] .
  * Chaturvedi and Shibata [1979] S. Chaturvedi and F. Shibata, Time-convolutionless projection operator formalism for elimination of fast variables. Applications to Brownian motion, Zeitschrift für Physik B Condensed Matter 35, 297 (1979).
  * Nakajima [1958] S. Nakajima, On Quantum Theory of Transport Phenomena: Steady Diffusion, Progress of Theoretical Physics 20, 948 (1958).
  * Zwanzig [2004] R. Zwanzig, Ensemble Method in the Theory of Irreversibility, The Journal of Chemical Physics 33, 1338 (2004).
  * Hall  _et al._ [2014] M. J. W. Hall, J. D. Cresser, L. Li, and E. Andersson, Canonical form of master equations and characterization of non-Markovianity, Phys. Rev. A 89, 042120 (2014).
  * Bužek [1998] V. Bužek, Reconstruction of Liouvillian superoperators, Phys. Rev. A 58, 1723 (1998).
  * Rossi [2018] R. J. Rossi,  _Mathematical statistics an introduction to likelihood based inference_ , 1st ed. (John Wiley & Sons, Hoboken, NJ, 2018).
  * Rivas  _et al._ [2010] A. Rivas, S. F. Huelga, and M. B. Plenio, Entanglement and non-Markovianity of quantum evolutions, Phys. Rev. Lett. 105, 050403 (2010).
  * Kofman and Kurizki [2000] A. G. Kofman and G. Kurizki, Acceleration of quantum decay processes by frequent observations, Nature 405, 546 (2000).
  * Kofman and Kurizki [2001] A. G. Kofman and G. Kurizki, Universal dynamical control of quantum mechanical decay: Modulation of the coupling to the continuum, Phys. Rev. Lett. 87, 270405 (2001).
  * Uhrig [2008] G. S. Uhrig, Exact results on dynamical decoupling by $\pi$ pulses in quantum information processes, New Journal of Physics 10, 083024 (2008).
  * Almog  _et al._ [2011] I. Almog, Y. Sagi, G. Gordon, G. Bensky, G. Kurizki, and N. Davidson, Direct measurement of the system–environment coupling as a tool for understanding decoherence and dynamical decoupling, Journal of Physics B: Atomic, Molecular and Optical Physics 44, 154006 (2011).
  * Rivas  _et al._ [2014] Á. Rivas, S. F. Huelga, and M. B. Plenio, Quantum non-Markovianity: Characterization, quantification and detection, Reports on Progress in Physics 77, 094001 (2014).
  * Kullback and Leibler [1951] S. Kullback and R. A. Leibler, On information and sufficiency, The Annals of Mathematical Statistics 22, 79 (1951).
  * Mogilevtsev  _et al._ [2012] D. Mogilevtsev, J. Řeháček, and Z. Hradil, Self-calibration for self-consistent tomography, New Journal of Physics 14, 095001 (2012).
  * James [2006] F. James,  _Statistical Methods in Experimental Physics_, 2nd ed. (WORLD SCIENTIFIC, 2006) https://www.worldscientific.com/doi/pdf/10.1142/6096 .
  * Hoadley [1971] B. Hoadley, Asymptotic Properties of Maximum Likelihood Estimators for the Independent Not Identically Distributed Case, The Annals of Mathematical Statistics 42, 1977 (1971).
  * Virtanen  _et al._ [2020] P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, İ. Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henriksen, E. A. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. van Mulbregt, and SciPy 1.0 Contributors, SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python, Nature Methods 17, 261 (2020).
  * Blume-Kohout [2012] R. Blume-Kohout, Robust error bars for quantum tomography (2012), arXiv:1202.5270 [quant-ph] .
  * Thinh  _et al._ [2019] L. P. Thinh, P. Faist, J. Helsen, D. Elkouss, and S. Wehner, Practical and reliable error bars for quantum process tomography, Phys. Rev. A 99, 052311 (2019).
  * Granade  _et al._ [2017] C. Granade, C. Ferrie, I. Hincks, S. Casagrande, T. Alexander, J. Gross, M. Kononenko, and Y. Sanders, QInfer: Statistical inference software for quantum applications, Quantum 1, 5 (2017).
  * Gardiner [2004] C. W. Gardiner,  _Handbook of stochastic methods for physics, chemistry and the natural sciences_ , 3rd ed., Springer Series in Synergetics, Vol. 13 (Springer-Verlag, Berlin, 2004) pp. xviii+415.
  * Gillespie [1996] D. T. Gillespie, The mathematics of Brownian motion and Johnson noise, American Journal of Physics 64, 225 (1996).
  * Clerk  _et al._ [2010] A. A. Clerk, M. H. Devoret, S. M. Girvin, F. Marquardt, and R. J. Schoelkopf, Introduction to quantum noise, measurement, and amplification, Rev. Mod. Phys. 82, 1155 (2010).
  * von Lüpke  _et al._ [2020] U. von Lüpke, F. Beaudoin, L. M. Norris, Y. Sung, R. Winik, J. Y. Qiu, M. Kjaergaard, D. Kim, J. Yoder, S. Gustavsson, L. Viola, and W. D. Oliver, Two-qubit spectroscopy of spatiotemporally correlated quantum noise in superconducting qubits, PRX Quantum 1, 010305 (2020).
  * Bermudez  _et al._ [2013] A. Bermudez, T. Schaetz, and M. B. Plenio, Dissipation-assisted quantum information processing with trapped ions, Phys. Rev. Lett. 110, 110502 (2013).
  * Cirac  _et al._ [1992] J. I. Cirac, R. Blatt, P. Zoller, and W. D. Phillips, Laser cooling of trapped ions in a standing wave, Phys. Rev. A 46, 2668 (1992).
  * Cormick  _et al._ [2013] C. Cormick, A. Bermudez, S. F. Huelga, and M. B. Plenio, Dissipative ground-state preparation of a spin chain by a structured environment, New Journal of Physics 15, 073027 (2013).
  * Bermudez and Schaetz [2016] A. Bermudez and T. Schaetz, Quantum transport of energy in controlled synthetic quantum magnets, New Journal of Physics 18, 083006 (2016).
  * Carmichael [1993] H. J. Carmichael,  _An open systems approach to quantum optics_ (Springer-Verlag, Berlin Heidelberg, 1993).
  * Kampen [2007] N. V. Kampen,  _Stochastic processes in physics and chemistry_ (North Holland, 2007).
  * Kraus [1971] K. Kraus, General state changes in quantum theory, Annals of Physics 64, 311 (1971).
  * Choi [1975] M.-D. Choi, Completely positive linear maps on complex matrices, Linear Algebra and its Applications 10, 285 (1975).
  * Terwiel [1974] R. Terwiel, Projection operator method applied to stochastic linear differential equations, Physica 74, 248 (1974).
  * Zoller [2018] P. Zoller, Theoretical Quantum Optics: A practical guide to stochastic processes & calculus (Master and PhD Course, University of Innsbruck) (2018).
  * Kubo [1962] R. Kubo, Generalized cumulant expansion method, Journal of the Physical Society of Japan 17, 1100 (1962).
  * Van Kampen [1974] N. Van Kampen, A cumulant expansion for stochastic linear differential equations. i, Physica 74, 215 (1974).
  * Breuer and Petruccione [2002b] H. P. Breuer and F. Petruccione,  _The theory of open quantum systems_ (Oxford University Press, Great Clarendon Street, 2002).
  * Fox [2008] R. F. Fox, Critique of the generalized cumulant expansion method, Journal of Mathematical Physics 17, 1148 (2008).
  * Itano  _et al._ [1993] W. M. Itano, J. C. Bergquist, J. J. Bollinger, J. M. Gilligan, D. J. Heinzen, F. L. Moore, M. G. Raizen, and D. J. Wineland, Quantum projection noise: Population fluctuations in two-level systems, Phys. Rev. A 47, 3554 (1993).
  * Haase  _et al._ [2016] J. F. Haase, A. Smirne, S. F. Huelga, J. Kołodynski, and R. Demkowicz-Dobrzanski, Precision limits in quantum metrology with open quantum systems, Quantum Measurements and Quantum Metrology 5, 13 (2016).
  * Degen  _et al._ [2017] C. L. Degen, F. Reinhard, and P. Cappellaro, Quantum sensing, Rev. Mod. Phys. 89, 035002 (2017).
  * Sidhu and Kok [2020] J. S. Sidhu and P. Kok, Geometric perspective on quantum parameter estimation, AVS Quantum Science 2, 014701 (2020), https://pubs.aip.org/avs/aqs/article-pdf/doi/10.1116/1.5119961/16700179/014701_1_online.pdf .
  * Martínez-García  _et al._ [2019] F. Martínez-García, D. Vodola, and M. Müller, Adaptive bayesian phase estimation for quantum error correcting codes, New Journal of Physics 21, 123027 (2019).


11institutetext: Politecnico di Torino, Italy
11email: {name.surname<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
# Natively neuromorphic LMU architecture for encoding-free SNN-based HAR on
commercial edge devices

Vittorio Fra  0000-0001-9175-2838 Benedetto Leto  0009-0007-8891-5051 Andrea
Pignata  0009-0001-4278-8348 Enrico Macii  0000-0001-9046-5618 Gianvito Urgese
0000-0003-2672-7593

###### Abstract

Neuromorphic models take inspiration from the human brain by adopting bio-
plausible neuron models to build alternatives to traditional Machine Learning
(ML) and Deep Learning (DL) solutions. The scarce availability of dedicated
hardware able to actualize the emulation of brain-inspired computation, which
is otherwise only simulated, yet still hinders the wide adoption of
neuromorphic computing for edge devices and embedded systems. With this
premise, we adopt the perspective of neuromorphic computing for conventional
hardware and we present the L2MU, a natively neuromorphic Legendre Memory Unit
(LMU) which entirely relies on Leaky Integrate-and-Fire (LIF) neurons.
Specifically, the original recurrent architecture of LMU has been redesigned
by modelling every constituent element with neural populations made of LIF or
Current-Based (CuBa) LIF neurons. To couple neuromorphic computing and off-
the-shelf edge devices, we equipped the L2MU with an input module for the
conversion of real values into spikes, which makes it an encoding-free
implementation of a Recurrent Spiking Neural Network (RSNN) able to directly
work with raw sensor signals on non-dedicated hardware. As a use case to
validate our network, we selected the task of Human Activity Recognition
(HAR). We benchmarked our L2MU on smartwatch signals from hand-oriented
activities, deploying it on three different commercial edge devices in
compressed versions too. The reported results remark the possibility of
considering neuromorphic models not only in an exclusive relationship with
dedicated hardware but also as a suitable choice to work with common sensors
and devices.

###### Keywords:

Neuromorphic computing Edge devices IoT Encoding-free LIF neurons Legendre
memory unit

##  1 Introduction

Smart devices, wearable sensors and edge computing can be looked at as a fire
triangle for efficient implementations of miniaturized, intelligent Body
Sensory Networks (BSNs) in domains like healthcare, sport or working
environment monitoring [8, 6, 1, 7, 14, 20]. The resulting variety of human-
centered tasks is enclosed in the general definition of Human Activity
Recognition (HAR). Typically characterized by data collected with wearable
devices through Inertial Measurement Units (IMUs) [22, 12], HAR is in practice
also a possible prototyping framework for Machine Learning (ML) techniques,
including Deep Learning (DL) models, intended for time-varying signals [29,
36, 10, 19]. In the DL domain, Spiking Neural Networks (SNNs) [24] offer an
appealing approach to HAR thanks to their unique capability of offering a
twofold perspective: on the one hand, they are alternative to traditional
methods in adopting the neuromorphic paradigm of driving inspiration from the
human brain through bio-plausible neuron models; on the other hand, they can
be deployed, besides the conventional hardware, on dedicated chips [9, 25, 30,
28, 5, 31] designed to perform asynchronous and sparse computation. Such dual
character of SNNs translates into the need for diverse evaluation criteria.
When we observe them through the lens of being alternative to non-spiking DL
solutions for synchronous digital devices, task-specific Key Performance
Indicators (KPIs) have to be primarily accounted for; when neuromorphic
hardware is targeted, energy and power are fundamental metrics to be included.
In both cases, however, the inherent hurdle they introduce to interface with
data from the analog external world is the encoding step, needed to produce
the spikes to be used as input for the network.

In this work, we propose a solution to employ SNNs on non-dedicated edge
devices to solve the HAR task through an alternative approach to more
traditional DL methods. With our LIF-based LMU (L2MU), a natively neuromorphic
implementation of the Legendre Memory Unit (LMU) architecture [37] where each
block of the network is made of Leaky Integrate-and-Fire (LIF) neuron
populations, we show a neuro-inspired model working with raw sensor data on
conventional hardware. We use the Wireless Sensor Data Mining (WISDM)
smartphone and smartwatch activity and biometrics dataset [39, 40] as specific
HAR use case, and, thanks to the results achieved after model compression, we
shine light on possible applications of the L2MU in the domain of real-time
Internet of Things (IoT) not only through still rare neuromorphic hardware but
also with non-dedicated edge devices.

##  2 Background

SNNs and the HAR task inherently have in common the fundamental role played by
time, which carries significant information in both cases [35, 26]. As a
result, neuromorphic computing can be successfully adopted by applying the
former to solve the latter with both feedforward and recurrent architectures
[13].

###  2.1 Human activity recognition

HAR is the classification task involving signals from human actions. Depending
on the employed sensor, or sensor network, different classes of HAR can be
defined [16]. Among them, the ones relying on IMU data turn out to be
attractive and promising thanks to the increasing diffusion of wearable
devices suitable for easy and non-invasive monitoring of motion [34].

The WISDM smartphone and smartwatch activity and biometrics dataset [39, 40],
selected for this work, includes recordings derived from sensors worn by 51
subjects instructed to perform 18 different tasks lasting 3 minutes each. The
signals are acquired by capturing accelerometer and gyroscope readings with a
sampling frequency of 20 Hz, and the activities are grouped into three
categories: i) non-hand-oriented activities, ii) hand-oriented activities
related to general tasks, and iii) hand-oriented activities tied to eating
actions. Compared to its earlier version [21], the dataset in this release
provides a more balanced distribution of samples across the 18 activities.

###  2.2 Legendre Memory Unit (LMU)

The LMU is a relevant Recurrent Neural Network (RNN) architecture for
classification tasks of time-based signals [38, 17, 3] originally implemented
in the Nengo framework [4]. We here summarize its main properties by reviewing
the key definitions directly from [37] for ease of reference.

A central attribute of the LMU is the representation of a signal with temporal
delay $u(t-\theta^{{}^{\prime}})$ within a sliding time window of duration
$\theta$ with $0\leq\theta^{{}^{\prime}}\leq\theta$. This process is achieved
by means of a high-dimensional projection of the input signal $u(t)$ through
the shifted Legendre polynomials. The element of degree $i$ in such polynomial
basis is defined as in Eq. 1:

$P_{i}(x)=(-1)^{i}\sum_{j=0}^{i}\binom{i}{j}\binom{i+j}{j}(-x)^{i}$ (1)

and it contributes to the input signal delaying as by Eq. 2:

$u(t-\theta^{{}^{\prime}})\approx\sum_{i=0}^{d-1}P_{i}\left(\frac{\theta^{{}^{\prime}}}{\theta}\right)m_{i}(t)$
(2)

where the highest order ($d-1$) in the series expansion is related to the
dimension $d$ of the state vector $\textbf{m}(t)$, defined by Eq. 3 as

$\theta\dot{\textbf{m}}(t)=\textbf{Am}(t)+\textbf{B}u(t)$ (3)

with A and B representing the ideal state-space matrices determined using the
Padé approximants by means of Eq. 4 and Eq. 5 with $i,j\in[0,d-1]$:

A $\displaystyle=[a]_{ij}\in\textbf{R}^{d\times d}$ $\displaystyle a_{ij}$
$\displaystyle=(2i+1)\begin{cases}-1&i<j\\\ (-1)^{i-j+1}&i\geq j\end{cases}$
(4) B $\displaystyle=[b]_{i}\in\textbf{R}^{d}$ $\displaystyle b_{i}$
$\displaystyle=(2i+1)(-1)^{i}$ (5)

###  2.3 Neuron models

SNNs, also referred to as the third-generation neural networks, aim at
emulating the asynchronous and sparse computation typical of the human brain
[24]. To achieve such a result, their elemental building block, i.e. the
fundamental computational unit, is directly inspired by biological neurons,
with typical definitions given by LIF-based models which offer a practically
acceptable sacrifice of biological plausibility in favour of a low
computational complexity [18]. For the L2MU, working within the snnTorch
framework111https://github.com/jeshraghian/snntorch/tree/master, we selected
two versions [11]:

####  2.3.1 Leaky model

The neuronal dynamics is described through the evolution in time of the
membrane potential (Eq. 6) and a spiking condition (Eq. 7):

$\centering U_{t}=\beta\cdot U_{t-1}+W\cdot
X_{t}-S_{t-1}\cdot\Theta\@add@centering$ (6)
$S_{t}=\bigg{\\{}\begin{array}[]{l}1,\quad if\quad U_{t}>\Theta\\\ 0,\quad
otherwise\\\ \end{array}$ (7)

where $\beta$ is the decay factor of the membrane potential, $t$ identifies
the simulation time step, $W$ is the synaptic weights matrix, $X$ represents
the input to the neuron and $\Theta$ represents the threshold voltage.

####  2.3.2 Synaptic model

This Current-Based LIF (CuBa-LIF) model includes an additional internal state
$I_{syn,t}=\alpha\cdot I_{syn,t-1}+W\cdot X_{t}$ to model a synaptic
conductance, with the resulting neuronal dynamics described by Eq. 8 as:

$\centering U_{t}=\beta\cdot
U_{t-1}+I_{syn,t}-S_{t-1}\cdot\Theta\@add@centering$ (8)

Similarly to $\beta$, the decay factor $\alpha$ refers to an exponential
decay. The spiking condition is defined by Eq. 7 as for the Leaky model.

##  3 Methodology

Building a neuromorphic model means taking inspiration from discrete and
sparse computation. Nonetheless, just as the human brain must account for
real-world signals which can be considered continuous, so has to do every SNN
to interface with traditional sensors. In this Section, we describe how we
implemented our L2MU to work with raw data and how we employed it to solve the
HAR task on commercial edge devices.

###  3.1 Activities selection and segmentation

Looking at final applications in real-time regimes, we applied a windowing
procedure to the signals from both accelerometer and gyroscope in order to
reduce the temporal length of individual samples. Specifically, we adopted a
timescale of 2 s, consistent with the perception of continuous human-machine
interaction [27, 33], on the hand-oriented general activities. As a result, we
obtained a subset of 7 classes with 36,201 non-overlapping samples made of 40
timestamps from the three axes of the accelerometer and the three axes of the
gyroscope. We then performed a 60:20:20 partition to define training,
validation, and test set respectively.

By selecting smartwatch signals from hand-oriented general activities, we
focused on cross-domains actions and movements. We indeed aimed to prove, at
least in a prototyping phase, the possibility to adopt our model in different
contexts and for various applications dealing with wearable devices.

Figure 1: Schematic of the whole model. The 6-axis input is shown as six
different channels in the leftmost part. Connected to them is the 3-layer,
fully-connected, encoding module. It transmits spikes to the L2MU which in
turn feeds the output layer with the spikes from the hidden state (h)
population.

###  3.2 Encoding module

An encoding module serves the purpose of converting the raw input signals into
spike trains. For our model, we adopted a three-layer, fully connected
structure. At the input side, signal expansion is performed by one LIF-based
neural population specific for each input channel, which increases data
dimensionality through the spiking activity. Such spikes are then propagated
to the subsequent layer, where LIF-based neurons fuse the information coming
from the channel-specific ones. As final layer, LIF-based neurons are used to
further harmonize the information contained in the spiking activity propagated
across the encoding module. The dimensions of the second and third layer are
considered as hyperparameters of the whole model, and they undergo
optimization.

###  3.3 LIF-based LMU (L2MU)

Differently from other works in the SNN domain inspired by LMU [2, 15, 23],
the natively neuromorphic redesign of LMU we propose with the L2MU relies on
the conversion of every constituent element of the original architecture into
neuron populations. In Fig. 1, the architecture is depicted, with the memory
state, the hidden state and the projection on the Legendre polynomial basis
represented by the m population, the h population and the u population
respectively. With this approach, we translated all the equations governing
the interactions among the different components of the LMU into neural
activity. The distinguishing inherent neuromorphic feature of the L2MU hence
is the neuro-inspired communication between the different building blocks,
mediated by spikes generated as response to incoming currents. Such conversion
in terms of spiking activity is performed with two alternatives, relying on
populations of neurons with Leaky or Synaptic model.

As explained in the original paper [37], the LMU takes a value x and produces
a hidden state h and a memory state m to define the new representation u of
the input according to Eq. 9:

$\textbf{u}_{t}=\textbf{e}_{x}^{\mathsf{T}}\cdot\textbf{x}_{t}+\textbf{e}_{y}^{\mathsf{T}}\cdot\textbf{h}_{t-1}+\textbf{e}_{m}^{\mathsf{T}}\cdot\textbf{m}_{t-1}$
(9)

where $\textbf{e}_{x}$, $\textbf{e}_{y}$ and $\textbf{e}_{m}$ are named as
encoding vectors [37]. In the L2MU, these steps are formulated with Eq. 10
relying on the spiking activity of the different populations of neurons:

$\textbf{u}_{t,curr}=\textbf{e}_{x}^{\mathsf{T}}\cdot\textbf{x}_{t,spk}+\textbf{e}_{y}^{\mathsf{T}}\cdot\textbf{h}_{t-1,spk}+\textbf{e}_{m}^{\mathsf{T}}\cdot\textbf{m}_{t-1,spk}$
(10)

with the corresponding spiking activity $\textbf{u}_{t,spk}$ produced through
$\textbf{u}_{t,curr}$ as the neuronal input and
$\textbf{u}_{t,mem}>\Theta^{u}$ as spiking condition coherently with Eq. 7.

The new input representation is then written into the memory state, whose
evolution in time can be discretized, starting from the original definition of
Eq. 3, as in Eq. 11:

$\textbf{m}_{t}=\overline{\textbf{A}}\cdot\textbf{m}_{t-1}+\overline{\textbf{B}}\cdot\textbf{u}_{t}$
(11)

where $\overline{\textbf{A}}=(\Delta t/\theta)\cdot\textbf{A}+\textbf{I}$ and
$\overline{\textbf{B}}=(\Delta t/\theta)\cdot\textbf{B}$, with I being the
identity matrix and $\Delta t$ representing a time step within the window of
length $\theta$. With similar arguments as for Eq. 10, the spiking version of
Eq. 11 is defined by Eq. 12 as:

$\textbf{m}_{t,curr}=\overline{\textbf{A}}\cdot\textbf{m}_{t-1,spk}+\overline{\textbf{B}}\cdot\textbf{u}_{t,spk}$
(12)

with the spiking activity $\textbf{m}_{t,spk}$ determined by such neuronal
input $\textbf{m}_{t,curr}$ depending on the condition
$\textbf{m}_{t,mem}>\Theta^{m}$ coherently with Eq. 7.

Finally, the hidden state originally defined by Eq. 13 as

$\textbf{h}_{t}=f(\textbf{W}_{x}\cdot\textbf{x}_{t}+\textbf{W}_{h}\cdot\textbf{h}_{t-1}+\textbf{W}_{m}\cdot\textbf{m}_{t})$
(13)

is translated into Eq. 14:

$\textbf{h}_{t,curr}=\textbf{W}_{x}\cdot\textbf{x}_{t,spk}+\textbf{W}_{h}\cdot\textbf{h}_{t-1,spk}+\textbf{W}_{m}\cdot\textbf{m}_{t,spk}$
(14)

with the corresponding spiking activity $\textbf{h}_{t,spk}$ determined by
$\textbf{h}_{t,curr}$ as neuronal input and the spiking condition
$\textbf{h}_{t,mem}>\Theta^{h}$ coherently with Eq. 7.

###  3.4 Model optimization

In order to identify the best configuration for our model to achieve as high
as possible classification accuracy for the selected task, we carried out
hyperparameter optimization (HPO) experiments. Particularly, we employed the
Neural Network Intelligence (NNI) toolkit222https://github.com/microsoft/nni
by designing a specific procedure and hyperparameter search space for each of
the two neuron models accounted for. We explored both architecture-related and
neuron-related hyperparameters, thus performing a combined optimization aimed
at identifying the best possible combination of neuron parameters ($\Theta$,
$\alpha$ and $\beta$) and model structure.

All the optimization trials were performed by training the model for 300
epochs, and the validation accuracy was used as objective metrics. At the end
of each trial, we evaluated the test accuracy, and the model achieving the
highest result was selected as the best model of each HPO experiment. The
following optimal dimensions were identified for the Leaky-based and the
Synaptic-based L2MU respectively: 30 and 30 channel-specific neurons per
population, 170 and 180 fusion neurons, 10 and 10 harmonization neurons, 150
and 230 u population neurons, 60 and 180 h population neurons, 1050 and 1840 m
population neurons.

Following the HPO, we retrained with different seed values the best models
found, in order to acquire deeper information about the classification
performance through a statistical analysis.

###  3.5 Model compression

To further prepare the L2MU-based models for real-time IoT applications on
conventional edge devices, pruning of the optimized models, with subsequent
fine-tuning, has been investigated. We compressed both the Leaky-based and the
Synaptic-based L2MU by means of the sconce Python
library333https://github.com/satabios/sconce. Specifically, starting from the
pre-trained optimal model, we performed Gradual Magnitude Pruning (GMP)
followed by fine-tuning for 300 epochs. We adopted the same statistical
approach as for the non-compressed models, using ten different seed values to
perform the two steps. As for the HPO experiments, we selected as the best
model for each neuron type the one providing the highest test accuracy.

###  3.6 Deployment on hardware

For the deployment of our encoding-free neuromorphic models on conventional
edge devices, we identified three commercial boards embedding different ARM-
based microprocessor units (MPUs) oriented to edge applications. The first one
is a STM32MP157F-DK2 by ST Microelectronics, which is based on the STM32MP157F
MPU with two ARM-A7 cores running at 800 MHz and a 32-bit architecture, and
includes 500 MB of DDR3 RAM. The second one is a Raspberry Pi 3B+, with a
Broadcom BCM2837B0 MPU (four ARM-A53 cores, 1.4 GHz, 64-bit) and 1 GB of DDR2
RAM. The last one is a Raspberry Pi 4B, embedding a Broadcom BCM2711 (four
ARM-A72 cores, 1.8 GHz, 64-bit) and 4 GB of DDR4 RAM. All the boards have a
full Linux distribution: OpenSTLinux, based on the OpenEmbedded project, for
the one from ST Microelectronics, and Raspberry Pi OS, based on Debian, for
the Raspberry Pi ones.

We deployed the compressed models through ONNX, performing on-edge inference
with the ONNX Runtime Python library. The export of the model to ONNX was
possible through torch.onnx.export thanks to the PyTorch-based foundation of
snnTorch.

The boards ran the last versions of the operating system recommended by the
manufacturers, with only the software needed for inference installed; and they
were connected to a local network via WiFi. We monitored energy over USB-C
connection while running the entire hardware for inference, and we measured
RAM usage through the psutil Python library.

##  4 Results

The proposed encoding-free neuromorphic approach for HAR has been evaluated
from different perspectives. First, we statistically analysed the results from
the training of the optimal models and their compressed versions; then, we
carried out a more detailed comparison of the non-compressed models focusing
on test accuracy. Finally, we investigated the inference phase for the
deployed models.

###  4.1 Leaky vs Synaptic

Figure 2: Learning curves from retraining of the best model with Leaky
neurons. The accuracy for the optimal model identified through HPO (a) is
reported together with the results for the compressed model (b). Mean values
and standard deviations are shown by solid line and shaded area respectively.

For each type of LIF neuron considered, we trained for 300 epochs, with
validation at each of them, the optimal model obtained from HPO, using ten
different seed values. For each seed, and for both the Leaky-based and the
Synaptic-based model, we evaluated the test accuracy at the end of the
training stage. Fig. 2a and Fig. 3a show the training and validation results,
while test results are inspected in Fig. 4 through confusion matrices.

Overall, Leaky neurons turned out to provide higher and more robust
classification accuracy: the median value (93.29%) is larger than the one
achieved with Synaptic neurons (92.72%), and the standard deviation is smaller
(0.49% instead of 0.61%). Compared to previous results on the same dataset
[13], the L2MU outperformed the standard LMU with either neuron model.

By analysing the disaggregated results through the classification accuracy per
class reported in Fig. 4, it can be further highlighted that both the Leaky
and the Synaptic neuron model provided high-level performance, overcoming
chance level by more than 6 times and with only two classes below 90%.

Figure 3: Learning curves from retraining of the best model with Synaptic
neurons. The accuracy for the optimal model identified through HPO (a) is
reported together with the results for the compressed model (b). Mean values
and standard deviations are shown by solid line and shaded area respectively.
Figure 4: Illustrative confusion matrices for a detailed breakdown of the
classification performance achieved by the optimized models after retraining
with ten different seed values. With Leaky neurons (a), a maximum test
accuracy of 94.14% is achieved; while 93.48% is the highest result obtained on
test data with Synaptic neurons (b).

####  4.1.1 Compressed models

Figure 5: Comparison of the different models. Results from the median values
of the statistical analyses are shown. With both Leaky and Synaptic neurons,
an increase in test accuracy is achieved performing pruning and fine-tuning.
On the contrary, the memory footprint and the number of synaptic operations
per sample are reduced, for both neuron types, by the compressed models.

The learning curves resulting from pruning and subsequent fine-tuning for 300
epochs with different seed values are shown in Fig. 2b and Fig. 3b for the
Leaky-based and the Synaptic-based model respectively. In Fig. 5, a summary is
instead shown to compare different metrics for all the trained models, either
with or without compression.

With respect to the classification performance before compression, a slight
improvement was achieved, with test accuracy equal to (93.56 $\pm$ 0.21)% for
Leaky neurons and (93.32 $\pm$ 0.23)% for Synaptic neurons.

A remarkably beneficial impact of pruning was observed for the memory
footprint. By evaluating the non-zero parameters after compression, the model
size turned out to be reduced by 52.12% and 79.52% for the Leaky-based model
and the Synaptic-based model respectively.

By leveraging the NeuroBench framework [41], we also evaluated an additional
neuro-inspired metrics: the number of effective synaptic operations per
sample. Defined as the number of operations per sample performed on non-zero
parameters, such quantity can be related to the computational effort in the
neuromorphic perspective. It indeed provides a measure of how sparse and
asynchronous operations could benefit energy saving. Compared to the
corresponding non-compressed counterpart, a reduction of such operations by
4.99% was achieved with Leaky neurons and by 40.25% with Synaptic neurons.

###  4.2 Deployed models

Table 1: Results achieved deploying the compressed models on commercial edge
devices to solve the HAR task on raw data

Neuron model |  Device |  Used RAM |  Mean inference time |  Mean energy per inference |  Accuracy (∗)
---|---|---|---|---|---
Leaky |  STM32MP1 |  65.7 MB |  0.13 s |  215.1 mJ |  93.91%
|  Raspberry Pi 3B+ |  77.8 MB |  0.06 s |  268.8 mJ |
|  Raspberry Pi 4B |  77.4 MB |  0.03 s |  153.9 mJ |
Synaptic |  STM32MP1 |  167.9 MB |  0.22 s |  383.4 mJ |  93.84%
|  Raspberry Pi 3B+ |  187.5 MB |  0.15 s |  727.5 mJ |
|  Raspberry Pi 4B |  187.4 MB |  0.07 s |  348.9 mJ |

(∗) Evaluated on the whole test set

For the deployment on hardware, we considered the best compressed models after
fine-tuning. Table 1 summarizes the results obtained on the three selected
devices with single-sample inferences. For both Leaky and Synaptic neurons,
RAM usage, mean inference time and mean energy per inference are reported,
together with accuracy on the whole test set. We also further investigated the
latency of the deployed L2MUs by running the models for 2 hours on the
devices. Fig. 6 show the inference time probability distributions for all the
deployed models, providing a comparative analysis suitable for latency-based
hardware selection.

The reported results show that both the types of LIF neuron can be deployed on
traditional hardware with good results. Specifically, the L2MU with the Leaky
neuron model provided 93.91% test accuracy with mean inference time of 0.13 s
at most, and the L2MU with Synaptic neurons reached a classification accuracy
on test data of 93.84% with inference time slightly higher: the mean values
ranged from 0.07 s to 0.22 s.

Our L2MU is hence capable, with both neuron types, of providing classification
responses within a delay time consistent with an anthropocentric definition of
real-time [33, 32], with values of Energy-Delay Product (EDP) ranging, on
average, from 4.6 mJ$\cdot$s with Leaky neurons on the Raspberry Pi 4B to
109.1 mJ$\cdot$s with Synaptic neurons on the Raspberry Pi 3B+.

Figure 6: Inference time probability distribution for the deployed models with
Leaky (a) and Synaptic (b) neurons. All the histograms have a bin width of 5
ms.

##  5 Conclusion

In this work we have shown a successful application of neuromorphic models
with off-the-shelf hardware through our L2MU, a natively neuromorphic
redefinition of the LMU entirely based on populations of LIF neurons. We built
it in the snnTorch framework by transforming all the components of the LMU
into neural populations, and we equipped it with an encoding module to allow
direct interface with traditional sensors collecting real-valued data. By
means of pruning and subsequent fine-tuning, we produced compressed versions
of the proposed model, evaluating them on the HAR task by adopting an
encoding-free approach to perform classification on raw data. Through
deployment on three different commercial edge devices, we reported various
metrics collected during on-edge inference. By analyzing all of them, Leaky
neurons turned out to offer the most attractive solution, providing the best
results in classification accuracy, memory footprint and EDP.

With inference times always lower than 300 ms and classification accuracy
above 93%, our L2MU demonstrated capability to operate in real-time regimes
suitable for IoT applications; and the possibility of deploying it on
commercial edge devices to work with an encoding-free strategy establishes a
promising link between neuromorphic models and non-dedicated hardware.

####  5.0.1 Acknowledgements

This research is funded by the European Union - NextGenerationEU Project
3A-ITALY MICS (PE0000004, CUP E13C22001900001, Spoke 6) and the Fluently
project with Grant Agreement No. 101058680. We acknowledge a contribution from
the Italian National Recovery and Resilience Plan (NRRP), M4C2, funded by the
European Union – NextGenerationEU (Project IR0000011, CUP B51E22000150006,
“EBRAINS-Italy”).

## References

  * [1] Allahbakhshi, H., Conrow, L., Naimi, B., Weibel, R.: Using accelerometer and GPS data for real-life physical activity type detection. Sensors (Switzerland) 20 (2020). https://doi.org/10.3390/s20030588
  * [2] Arsalan, M., Santra, A., Issakov, V.: Low power radar-based air-writing system using genetic algorithm-assisted spiking legendre memory unit. In: 20th European Radar Conference (EuRAD) (2023)
  * [3] Bartlett, M.E., Stewart, T.C., Thill, S.: Estimating levels of engagement for social human-robot interaction using legendre memory units. In: ACM/IEEE International Conference on Human-Robot Interaction (2021)
  * [4] Bekolay, T., Bergstra, J., Hunsberger, E., DeWolf, T., Stewart, T.C., Rasmussen, D., Choo, X., Voelker, A.R., Eliasmith, C.: Nengo: A Python tool for building large-scale functional brain models. Frontiers in Neuroinformatics 7 (2014). https://doi.org/10.3389/fninf.2013.00048
  * [5] Bos, H., Muir, D.: Sub-mw neuromorphic snn audio processing applications with rockpool and xylo. In: Embedded Artificial Intelligence. River Publishers (2023)
  * [6] Capela, N.A., Lemaire, E.D., Baddour, N.: Feature Selection for Wearable Smartphone-Based Human Activity Recognition with Able bodied, Elderly, and Stroke Patients. PLOS ONE 10 (2015). https://doi.org/10.1371/journal.pone.0124414
  * [7] Ceolini, E., Frenkel, C., Shrestha, S.B., Taverni, G., Khacef, L., Payvand, M., Donati, E.: Hand-Gesture Recognition Based on EMG and Event-Based Camera Sensor Fusion: A Benchmark in Neuromorphic Computing. Frontiers in Neuroscience 14 (2020). https://doi.org/10.3389/fnins.2020.00637
  * [8] Dami, S., Yahaghizadeh, M.: Predicting cardiovascular events with deep learning approach in the context of the internet of things. Neural Computing and Applications 33 (2021). https://doi.org/10.1007/s00521-020-05542-x
  * [9] Davies, M., Srinivasa, N., Lin, T.H., Chinya, G., Cao, Y., Choday, S.H., Dimou, G., Joshi, P., Imam, N., Jain, S., Liao, Y., Lin, C.K., Lines, A., Liu, R., Mathaikutty, D., McCoy, S., Paul, A., Tse, J., Venkataramanan, G., Weng, Y.H., Wild, A., Yang, Y., Wang, H.: Loihi: A Neuromorphic Manycore Processor with On-Chip Learning. IEEE Micro 38 (2018). https://doi.org/10.1109/MM.2018.112130359
  * [10] Demrozi, F., Pravadelli, G., Bihorac, A., Rashidi, P.: Human Activity Recognition Using Inertial, Physiological and Environmental Sensors: A Comprehensive Survey. IEEE Access 8 (2020). https://doi.org/10.1109/ACCESS.2020.3037715
  * [11] Eshraghian, J.K., Ward, M., Neftci, E., Wang, X., Lenz, G., Dwivedi, G., Bennamoun, M., Jeong, D.S., Lu, W.D.: Training spiking neural networks using lessons from deep learning. arXiv preprint arXiv:2109.12894 (2024)
  * [12] Ferrari, A., Micucci, D., Mobilio, M., Napoletano, P.: Trends in human activity recognition using smartphones. Journal of Reliable Intelligent Environments 7 (2021). https://doi.org/10.1007/s40860-021-00147-0
  * [13] Fra, V., Forno, E., Pignari, R., Stewart, T.C., Macii, E., Urgese, G.: Human activity recognition: suitability of a neuromorphic approach for on-edge AIoT applications. Neuromorphic Computing and Engineering 2 (2022). https://doi.org/10.1088/2634-4386/ac4c38
  * [14] Frank, A.E., Kubota, A., Riek, L.D.: Wearable activity recognition for robust human-robot teaming in safety-critical environments via hybrid neural networks. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2019). https://doi.org/10.1109/IROS40897.2019.8968615
  * [15] Gaurav, R., Stewart, T.C., Yi, Y.: Reservoir based spiking models for univariate time series classification. Frontiers in Computational Neuroscience (2023)
  * [16] Gomaa, W., Khamis, M.A.: A perspective on human activity recognition from inertial motion data. Neural Computing and Applications (2023)
  * [17] Gupta, G., Kshirsagar, M., Zhong, M., Gholami, S., Ferres, J.L.: Comparing recurrent convolutional neural networks for large scale bird species classification. Scientific reports (2021)
  * [18] Izhikevich, E.M.: Dynamical systems in neuroscience. MIT press (2007)
  * [19] Khan, N.S., Ghani, M.S.: A Survey of Deep Learning Based Models for Human Activity Recognition. Wireless Personal Communications (2021). https://doi.org/10.1007/s11277-021-08525-w
  * [20] Kulsoom, F., Narejo, S., Mehmood, Z., Chaudhry, H.N., Butt, A., Bashir, A.K.: A review of machine learning-based human activity recognition for diverse applications. Neural Computing and Applications 34 (2022)
  * [21] Kwapisz, J.R., Weiss, G.M., Moore, S.A.: Activity recognition using cell phone accelerometers. ACM SIGKDD Explorations Newsletter 12 (2011). https://doi.org/10.1145/1964897.1964918
  * [22] Lara, O.D., Labrador, M.A.: A Survey on Human Activity Recognition using Wearable Sensors. IEEE Communications Surveys & Tutorials 15 (2013). https://doi.org/10.1109/SURV.2012.110112.00192
  * [23] Liu, Z., Datta, G., Li, A., Beerel, P.A.: Lmuformer: Low complexity yet powerful spiking model with legendre memory units. arXiv preprint arXiv:2402.04882 (2024)
  * [24] Maass, W.: Networks of spiking neurons: The third generation of neural network models. Neural Networks 10 (1997). https://doi.org/10.1016/S0893-6080(97)00011-7
  * [25] Mayr, C., Hoeppner, S., Furber, S.: Spinnaker 2: A 10 million core processor system for brain simulation and machine learning. arXiv preprint arXiv:1911.02385 (2019)
  * [26] Mekruksavanich, S., Jitpattanakul, A.: Deep Convolutional Neural Network with RNNs for Complex Activity Recognition Using Wrist-Worn Wearable Sensor Data. Electronics 10 (2021). https://doi.org/10.3390/electronics10141685
  * [27] Miller, R.B.: Response time in man-computer conversational transactions. In: Proceedings of the December 9-11, 1968, fall joint computer conference, part I (1968)
  * [28] Müller-Cleve, S.F., Fra, V., Khacef, L., Pequeño-Zurro, A., Klepatsch, D., Forno, E., Ivanovich, D.G., Rastogi, S., Urgese, G., Zenke, F., Bartolozzi, C.: Braille letter reading: A benchmark for spatio-temporal pattern recognition on neuromorphic hardware. Frontiers in Neuroscience 16 (2022). https://doi.org/10.3389/fnins.2022.951164
  * [29] Nweke, H.F., Teh, Y.W., Al-garadi, M.A., Alo, U.R.: Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges. Expert Systems with Applications 105 (2018). https://doi.org/10.1016/j.eswa.2018.03.056
  * [30] Orchard, G., Frady, E.P., Rubin, D.B.D., Sanborn, S., Shrestha, S.B., Sommer, F.T., Davies, M.: Efficient Neuromorphic Signal Processing with Loihi 2. In: IEEE Workshop on Signal Processing Systems (SiPS). vol. 2021-Octob (2021). https://doi.org/10.1109/SiPS52927.2021.00053
  * [31] Pedersen, J.E., Abreu, S., Jobst, M., Lenz, G., Fra, V., Bauer, F.C., Muir, D.R., Zhou, P., Vogginger, B., Heckel, K., et al.: Neuromorphic intermediate representation: A unified instruction set for interoperable brain-inspired computing. arXiv preprint arXiv:2311.14641 (2023)
  * [32] Pelikan, H., Hofstetter, E.: Managing delays in human-robot interaction. ACM Transactions on Computer-Human Interaction (2023)
  * [33] Popovski, P., Chiariotti, F., Huang, K., Kalør, A.E., Kountouris, M., Pappas, N., Soret, B.: A perspective on time toward wireless 6g. Proceedings of the IEEE (2022)
  * [34] Ramanujam, E., Perumal, T., Padmavathi, S.: Human Activity Recognition with Smartphone and Wearable Sensors using Deep Learning Techniques: A Review. IEEE Sensors Journal 21 (2021). https://doi.org/10.1109/JSEN.2021.3069927
  * [35] Roy, K., Jaiswal, A., Panda, P.: Towards spike-based machine intelligence with neuromorphic computing. Nature 575 (2019). https://doi.org/10.1038/s41586-019-1677-2
  * [36] Slim, S.O., Atia, A., M.A., M., M.Mostafa, M.S.: Survey on Human Activity Recognition based on Acceleration Data. International Journal of Advanced Computer Science and Applications 10 (2019). https://doi.org/10.14569/IJACSA.2019.0100311
  * [37] Voelker, A., Kajić, I., Eliasmith, C.: Legendre memory units: Continuous-time representation in recurrent neural networks. Advances in neural information processing systems 32 (2019)
  * [38] Voelker, A.R., Eliasmith, C.: Programming neuromorphics using the neural engineering framework. Handbook of Neuroengineering (2020)
  * [39] Weiss, G.M.: WISDM Smartphone and Smartwatch Activity and Biometrics Dataset. UCI Machine Learning Repository: WISDM Smartphone and Smartwatch Activity and Biometrics Dataset Data Set 7 (2019)
  * [40] Weiss, G.M., Yoneda, K., Hayajneh, T.: Smartphone and Smartwatch-Based Biometrics Using Activities of Daily Living. IEEE Access 7 (2019). https://doi.org/10.1109/ACCESS.2019.2940729
  * [41] Yik, J., den Berghe, K.V., den Blanken, D., Bouhadjar, Y., Fabre, M., Hueber, P., Kleyko, D., Pacik-Nelson, N., Sun, P.S.V., Tang, G., Wang, S., Zhou, B., Ahmed, S.H., Joseph, G.V., Leto, B., Micheli, A., Mishra, A.K., Lenz, G., Sun, T., Ahmed, Z., Akl, M., Anderson, B., Andreou, A.G., Bartolozzi, C., Basu, A., Bogdan, P., Bohte, S., Buckley, S., Cauwenberghs, G., Chicca, E., Corradi, F., de Croon, G., Danielescu, A., Daram, A., Davies, M., Demirag, Y., Eshraghian, J., Fischer, T., Forest, J., Fra, V., Furber, S., Furlong, P.M., Gilpin, W., Gilra, A., Gonzalez, H.A., Indiveri, G., Joshi, S., Karia, V., Khacef, L., Knight, J.C., Kriener, L., Kubendran, R., Kudithipudi, D., Liu, Y.H., Liu, S.C., Ma, H., Manohar, R., Margarit-Taulé, J.M., Mayr, C., Michmizos, K., Muir, D., Neftci, E., Nowotny, T., Ottati, F., Ozcelikkale, A., Panda, P., Park, J., Payvand, M., Pehle, C., Petrovici, M.A., Pierro, A., Posch, C., Renner, A., Sandamirskaya, Y., Schaefer, C.J., van Schaik, A., Schemmel, J., Schmidgall, S., Schuman, C., sun Seo, J., Sheik, S., Shrestha, S.B., Sifalakis, M., Sironi, A., Stewart, M., Stewart, K., Stewart, T.C., Stratmann, P., Timcheck, J., Tömen, N., Urgese, G., Verhelst, M., Vineyard, C.M., Vogginger, B., Yousefzadeh, A., Zohora, F.T., Frenkel, C., Reddi, V.J.: Neurobench: A framework for benchmarking neuromorphic computing algorithms and systems. arXiv preprint arXiv:2304.04640 (2024)