Neural Synchrony-Based State Representation in Liquid State Machines, an Exploratory Study
by Nicolas Pajot, Mounir Boukadoum* 
Department of Computer Science, University of Quebec at Montreal, Quebec, Canada
* Author to whom correspondence should be addressed.
Journal of Engineering Research and Sciences, Volume 2, Issue 11, Page # 1-14, 2023; DOI: 10.55708/js0211001
Keywords: Liquid state machine, state representation, temporal decoding, separation property, classification
Received: 06 September 2023, Revised: 25 October 2023, Accepted: 26 November 2023, Published Online: 30 November 2023
APA Style
Pajot, N., & Boukadoum, M. (2023). Neural Synchrony-Based State Representation in Liquid State Machines, an Exploratory Study. Journal of Engineering Research and Sciences, 2(11), 1–14. https://doi.org/10.55708/js0211001
Chicago/Turabian Style
Pajot, Nicolas, and Mounir Boukadoum. “Neural Synchrony-Based State Representation in Liquid State Machines, an Exploratory Study.” Journal of Engineering Research and Sciences 2, no. 11 (November 1, 2023): 1–14. https://doi.org/10.55708/js0211001.
IEEE Style
N. Pajot and M. Boukadoum, “Neural Synchrony-Based State Representation in Liquid State Machines, an Exploratory Study,” Journal of Engineering Research and Sciences, vol. 2, no. 11, pp. 1–14, Nov. 2023, doi: 10.55708/js0211001.
122 Downloads
Abstract
Full Text
References
Cited By
Metrics
Abstract
Solving classification problems by Liquid State Machines (LSM) usually ignores the influence of the liquid state representation on performance, leaving the role to the reader circuit. In most studies, the decoding of the internally generated neural states is performed on spike rate-based vector representations. This approach occults the interspike timing, a central aspect of biological neural coding, with potentially detrimental consequences on the LSM performance. In this work, we propose a model of liquid state representation that builds the feature vectors from temporal information extracted from the spike trains, hence using spike synchrony instead of rate. Using pairs of Poisson-distributed spike trains in noisy conditions, we show that such model outperforms a rate-only model in distinguishing two spike trains regardless of the sampling frequency of the liquid states or the noise level. In the same vein, we suggest a synchrony-based measure of the separation property (SP), a core feature of LSMs regarding classification performance, for a more robust and biologically plausible interpretation.
Full Text
References
- W. Maass, “Liquid State Machines: Motivation, Theory, and Applications,” World Scientific Review, Vol. March 25/2010, doi: 10.1142/9781848162778_0008
- D. Verstraeten, B. Schrauwen, B., D. Stroobandt, “Isolated word recognition with the liquid state machine: a case study,” Proc. 13th European Symposium on Artificial Neural Nets (ESANN), vol. 95(6), pp. 435–440, 2005.
- E. Goodman, E., Ventura, D. “ Spatiotemporal pattern recognition via liquid state machines,” International Joint Conference on Neural Networks (IJCNN), pp. 3848–3853, 2006.
- W. Maass, T. Natschlager, H. Markram, “Fading memory and kernel properties of generic cortical microcircuit models,”Journal of Physiology (Paris), vol. 98, pp. 315–330, 2004.
- J. De Gruijl, M. Wiering, “Musical Instrument Classification using Democratic Liquid State,” Proc. 15th Belgian-Dutch Conference on Machine Learning, 2006.
- L. Pape, R. Jornt, J. De Gruijl, M. Wiering, “Democratic Liquid State Machines for Music Recognition”, Speech, Audio, Image and Biomedical Signal Processing using Neural Networks. Volume 83b, Berlin, 2008.
- J. Hertzberg, H. Jager, F. Schonherr. F., “Learning to ground fact symbols in behavior-based robot,” Proc. 15th European Conference on Artificial Intelligence., pp. 708–712, Amsterdam, 2002.
- H. Burgsteiner, M. Kroll, A. Leopold, G., Steinbauer, “Movement prediction from real-world images using a liquid state machine,” Appl. Intell. vol. 27 (2), pp. 99–109, 2007.
- H. Ju, J. Xu, A. VanDongen, A. “Classification of Musical Styles Using Liquid State Machines,” Proc. International Joint Conference on Neural Networks s (IJCNN), 2010, doi: 10.1109/IJCNN.2010.5596470.
- F. Rhéaume, D. Grenier, E. Bosse, “Multistate combination approaches for liquid state machine in supervised spatiotemporal pattern classification,” Neurocomputing, vol. 74, pp. 2842–2851, 2011.
- M. Aoun, M. Boukadoum, “Chaotic Liquid State Machine,” International Journal of Cognitive Informatics and Natural Intelligence, vol. 9(4), 1-20, October-December 2015
- J. Huang, H. Fang, Y. Wang, “ Directional classification of cortical signals using a liquid state machine,” Proc. 17th World Congress of The International Federation of Automatic Control, Seoul, Korea, July 6-11, 2008.
- E. Hourdakis, P. Trahanias , “Use of the separation property to derive Liquid State Machines with enhanced classification performance,” Neurocomputing, vol. 107, pp40–48, 2013.
- R. Legenstein, W. Maass, “Edge of chaos and prediction of computational performance for neural circuit models, Neural Networks, vol 20(3), pp. 323–334, 2007.
- D. Norton, D. Ventura, “Preparing More Effective Liquid State Machines Using Hebbian Learning,” Proc. International Joint Conference on Neural Networks (IJCNN), 2006.
- J. Matser, “ Structured liquids in liquid state machines,” (Master Thesis, Utrecht University, 2010).
- S. Kok, “Liquid State Machine Optimization,” (Master Thesis, Utrecht University, 2007).
- D. Norton, D. Ventura, “Improving the performance of liquid state machines through iterative refinement of the reservoir,” Neurocomputing, vol. 73, pp. 2893–2904, 2010.
- J. Huang, Y. Wang, J. Huang, “The Separation Property Enhancement of Liquid State Machine by Particle Swarm Optimization,” International Symposium on Neural Networks (ISNN 2009: Advances in Neural Networks), pp 67-76, 2009.
- A. Kasinski, F. Ponulak, “Comparison of supervised learning methods for spike time coding in spiking neural networks,”.Int. J. Appl. Math. Comput. Sci., vol. 16(1), pp. 101–113, 2006.
- T. Kreuz, T. (2001). Measures of spike train synchrony, Scholarpedia, vol. 6(10):11934, 2001.
- http://www.scholarpedia.org/article/Measures_of_spike_train_synchrony
- W. Maass, T. Natschlager, H. Markram, H. “Real-time computing without stable states: A new framework for neural computation based on perturbations,” Neural Computation, vol. 14(11), pp. 2531–2560, 2002.
- D. Buonomano, W. Maass, “State-dependent computations: Spatiotemporal processing in cortical networks,” Nature Reviews in Neuroscience, vol. 10, no. 2, pp. 113–125, 2009.
- J. Mayor, W. Gerstner, “ Transient information flow in a network of excitatory and inhibitory model neurons: role of noise and signal autocorrelation,” Journal of Physiology (Paris), vol. 98, pp. 417–428, 2004.
- CSIM user manual: http://www.lsm.tugraz.at/download/csim-1.1-usermanual.pdf
- R. Legenstein, H. Markram, W. Maass, “Input prediction and autonomous movement analysis in recurrent circuits of spiking neurons,” Reviews in the Neurosciences (Special Issue on Neuroinformatics of Neural and Artificial Computation), vol. 14, no. 1-2, pp. 5–19, 2003
- S. Hausler, H. Markram, W. Maass, “Perspectives of the high-dimensional dynamics of neural microcircuits from the point of view of low-dimensional readouts”. Complexity, vol. 8(4), pp. 39–50, 2003.
- A. Oliveri, R. Rizzo, A. Chella, “An application of spike-timing-dependent plasticity to readout circuit for liquid state machine,” IEEE International Joint Conference on Neural Networks, pp. 1441–1445, 2007.
- H. Ju et al., “Effects of synaptic connectivity on liquid state machine performance,” Neural Networks, vol. 38, pp. 39–51, 2013.
- R. Brette, “Philosophy of the Spike: Rate-Based vs. Spike-Based Theories of the Brain,”. Front. Syst. Neurosci. 2015, doi: 10.3389/fnsys.2015.00151
- B. Meftah et al., “Image Processing with Spiking Neuron Networks,” Artif. Intell., Evol. Comput. and Metaheuristics, SCI 427, 525–544, 2012.
- J. D. Victor, “Spike train metrics,” Current Opinion in Neurobiology, vol. 15, pp. 585–592, 2005.
- T. Kreuz, “Measures of spike train synchrony,“ Scholarpedia, vol. 6(10), pp. 11934, 2001.
- W. Gerstner et al., Neuronal Dynamics: From single neurons to networks and models of cognition, Cambridge University Press, 2014, http://neuronaldynamics.epfl.ch/online/Ch7.S6.html.
- G. B. Stanley, “Reading and writing the neural code,” Nature Neuroscience, vol. 16, pp. 259–263, 2013, doi:10.1038/nn.3330
- K. P. Dockendorf et al., “Liquid state machines and cultured cortical networks: the separation property,” Biosystems, vol. 95(2), pp. 90–97, 2009.
- M. C. W. Van Rossum, “A novel spike distance,” Neural Comput., vol. 13, pp. 751–763, 2001.
- J. D. Victor, K. P. Purpura K.P., “Spike Metrics,” In: Analysis of Parallel Spike Trains. Ed. Stefan Rotter and Sonja Gruen. Springer, pp. 129-156, 2010. http://www-users.med.cornell.edu/~jdvicto/pdfs/vipu10.pdf.
- E. Satuvurori et al., “ Measures of spike train synchrony for data with multiple time scales,” J Neurosci. Meth, vol. 287, pp. 25-38, 2017
- T. Kreuz et al., “Measuring spike train synchrony,” J Neurosci Meth., vol. 165, pp. 151–161, 2007.
- T. Kreuz et al., “Measuring synchronization in coupled model systems: a comparison of different approaches,” Phys D., vol. 225, pp. 29–42, 2007.
- T. Kreuz et al., “Measuring multiple spike train synchrony,” J. Neurosci Meth., vol. 183, pp. 287–299, 2009.
- T. Kreuz, D. Chicharro, M. Greschner, R. G. Andrzejak, “Time-resolved and time-scale adaptive measures of spike train synchrony,” J. Neurosci Meth., vol. 195, pp. 92–106, 2011.
- T. Kreuz et al., “Monitoring spike train synchrony,” J. Neurophysiol., vol. 109, pp. 1457-72, 2013.
- http://www.scholarpedia.org/article/SPIKE-distance
- J. D. Victor, K. P. Purpura, ”Nature and precision of temporal coding in visual cortex: a metric-space analysis,” J. Neurophysiol., vol. 76, pp. 1310–1326, 1996.
- J. D. Victor, K. P. Purpura, ”Metric-space analysis of spike trains: theory, algorithms and application, Network, vol. 8, pp. 127–164, 1997.
- S. Schreiber et al., “A new correlation-based measure of spike timing reliability,” Neurocomputing, vol. 52, pp. 925–931, 2003.
- J. D. Hunter, G. Milton, G., “Amplitude and frequency dependence of spike timing: implications for dynamic regulation,” J. Neurophysiol., vol. 90, pp. 387‐394, 2003.
- R. Quian Quiroga, T. Kreuz, P. Grassberger, “Event synchronization: a simple and fast method to measure synchronicity and time delay patterns,” Phys Rev E Stat Nonlin Soft Matter Phys, vol. 66:041904, 2002, doi: 10.1103/PhysRevE.66.041904.
- T. Kreuz, M. Mulansky, N. Bozanic, “SPIKY: A graphical user interface for monitoring spike train synchrony,” J Neurophysiol., vol 113, pp. 3432, 2015.
- R. Jolivet et al., “A benchmark test for a quantitative assessment of simple neuron models,” J Neurosci Meth., vol 169, pp. 417–424, 2008.
- http://www.lsm.tugraz.at/download/circuit-tool-1.1-manual.pdf
- T. Natschlager, W. Maass, H. Markram. Chapter 9, Computer Models and Analysis Tools for Neural Microcircuits, http://www.lsm.tugraz.at/papers/lsm-koetter-chapter-144.pdf
- http://www.lsm.tugraz.at/download/csim-1.1-usermanual.pdf
- H. Markram, Y. Wang, M. Tsodyks, “ Differential Signaling via the Same Axon of Neocortical Pyramidal Neurons,” Proc Natl Acad Sci U S A., 1998, doi: 10.1073/pnas.95.9.5323..
- CSIM: A neural Circuit SIMulator, http://www.lsm.tugraz.at/csim/
- R. Brette et al., “Simulation of networks of spiking neurons: A review of tools and strategies,” J. Comput Neurosci., vol. 23(3), pp. 349-98, 2007, doi 10.1007/s10827-007-0038-6
- B. Gardner, I. Sporea, A. Gruning, “Encoding Spike Patterns in Multilayer Spiking Neural Networks,” arxiv.org/abs/1503.09129v1, 2015.
- S. Shinomoto, S. Grun, S. Rotter, S. (2010). Chapter 2, Estimating the Firing Rate. Analysis of Parallel Spike Trains, in S. Grün, S. Rotter (eds.), Analysis of Parallel Spike Trains, Springer Series in Computational Neuroscience 7, 2010, doi 10.1007/978-1-4419-5675-0_2
- G. M. Wojcik, M. Wazny, “ Bray-Curtis Metrics as Measure of Liquid State Machine Separation Ability in Function of Connections Density,” Procedia Computer Science, Vol. 51, pp. 2938–2942, Proc. International Conference on Computational Science (ICCS), 2015.
- M. Loginov, “Beyond decoding: representational similarity, analysis on fMRI data,” (Master’s Thesis, University of Tartu, 2015).
- S. Roy, A., Banerjee, A. Basu, “Liquid State Machine with Dendritically Enhanced Readout for Low-power, Neuromorphic VLSI Implementations,” IEEE Trans. Biomedical Circuits and Systems, vol. 8(5), pp. 681-695, 2014. doi: 10.1109/TBCAS.2014.2362969
- H. Hazan, L. M. Manevitz “Topological constraints and robustness in liquid state machines,” Expert Systems with Applications, vol. 39, pp. 1597–1606, 2012.
- B. J. Grzyb, E. Chinellato, G. M. Wojcik, W. A. Kaminski, W.A., “Which model to use for the Liquid State Machine?” Proc. International Joint Conference on Neural Networks (IJCNN), pp. 1018-1024, 2009, doi: 10.1109/IJCNN.2009.5178822.
Cited By
Metrics
