24F Final Project: Neuromorphic Computing

By Chris Mecane

Neuromorphic computing, also known as neuromorphic engineering, is a fairly recent development in the realm of computational neuroscience. The goal of neuromorphic computing is to construct computers whose hardware and software mimic the anatomy and physiology of the human brain via replication of neural structure and emulation of synaptic communication. Though still in its nascent years, the field is a source of hope for the future of computing and artificial intelligence.

Background

Though neuromorphic computing presently remains no more than a promising concept, it is agreed upon that the foundations of the field were laid down by Caltech's Carver Mead in the late 1980s. Using human neurobiology as the model, Mead contrasted the brain's hierarchical encoding of information, such as its ability to instantly integrate sensory perception as an event with an associated emotion, with a computer's glaring lack of an ability to encode in this way; he further noted the brain alone's capacity to combine signal processing with gain control. Mead nonetheless recognized the glaring need to better understand the brain before an accurately-designed neuromorphic computer could be realized; he emphasizes the fact that it is barely understood how the brain, for some given amount of energy output, is able to perform many times more computations than even the most advanced computer. Mead himself, along with his PhD student Misha Mahowald, provided the first practical example of the potential of neuromorphic computing through their development of a silicon retina in 1991, which successfully imitated output signals seen in true human retinas most notably in response to moving images.

How Neuromorphic Computing Works

Neuromorphic computers are partially defined by the fact that they are different from the widespread von Neumann computers, which have distinct CPUs and memory units and store data as binary. In contrast, neuromorphic computers, in their efforts to process information analogously to the human brain, integrate memory and processing into a single mechanism

regulated by its neurons and synapses; neurons receive "spikes" of information, with the timing, magnitude, and shape of the spike all being meaningful attributes in the encoding of numerical information. As such, neuromorphic computers are said to be modeled using spiking neural networks (SSNs); spiking neurons behave similarly to biological neurons, in that they factor in characteristics such as threshold values for neuronal activation and synaptic weights which can change over time. The aforementioned features can all be found in existing, standard neural networks that are capable of continual learning, such as perceptrons. Neuromorphic computers, however, would surpass these traditional networks in their ability to incorporate neuronal and synaptic delay; as information flows in, "charge" accumulates in the neurons until the surpassing of some charge threshold produces a "spike," or action potential. If the charge does not exceed the threshold over some given time period, it "leaks." The neuronal and synaptic makeup of neuromorphic computers additionally allows for, unlike traditional computers, for many parallel operations to be running in different neurons at a given time; van Neumann computers utilize sequential processing of information. Through their parallel processing, along with their integration of processing and memory, neuromorphic computers provide a glimpse into a future full of vastly more energy-efficient computation.

Algorithms for neuromorphic computation differ based on their specific applications. These algorithms are broadly associated with one of two categories: machine-learning algorithms and manually-constructed non-machine learning algorithms. In the case of machine-learning algorithms, the spiking neural network optimizes synaptic weights

Various training methods for SSNs, including a.) backpropagation b.) mapping c.) reservoir computing d.) evolutionary machine learning and e.) spike-timing-dependent plasticity

relevant to achieving a specific goal; training of the network can occur, for example, through backpropagation, mapping of a trained traditional artificial neural network onto the spiked artificial neural network, or reservoir computing. While these three methods of training are currently utilized by artificial neural networks, neuromorphic computers would uniquely look to utilize an evolutionary approach, in which the structure, connections, number of neurons, and weights of the network, like a brain, change over time without relaying on any particular network architecture, as well as an approach referred to as "spike-timing-dependent plasticity; since, unlike artificial neural networks, neuromorphic computers would be capable of incorporating delay, the weights synapses would be able to change as a function of the timings of spikes from both pre- and post-synaptic neurons. Machine-learning algorithms are the most common method by which spiking neural networks train.