The detection of any of these patterns causes an NMDA spike and subsequent depolarization at the soma.
It might seem that eight to twenty synapses could not reliably recognize a pattern of activity in a large population of cells. However, robust recognition is possible if the patterns to be recognized are sparse; i.e. few neurons are active relative to the population (Olshausen and Field, 2004). For example, consider a population of 200K cells where 1% (2,000) of the cells are active at any point in time. We want a neuron to detect when a particular pattern occurs in the 200K cells. If a section of the neuron' s dendrite forms new synapses to just 10 of the 2,000 active cells, and the threshold for generating an NMDA spike is 10, then the dendrite will detect the target pattern when all 10 synapses receive activation at the same time Note that the dendrite could falsely detect many other patterns that share the same 10 active cells. However, if the patterns are sparse, the chance that the 10 synapses would become active for a different random pattern is small. In this example it is only 9.8 x 10^-21.
The probability of a false match can be calculated precisely as follows. Let n represent the size of the cell population and A the number of active cells in that population at a given point in time, for sparse patterns A ≪ n . Let s be the number of synapses on a dendritic segment and θ be the NMDA spike threshold. We say the segment recognizes a pattern if at least θ synapses become active, i.e. at least θ of the s synapses match the currently active cells.
-- If a section of the neuron' s dendrite forms new synapses to just 10 of the 2,000 active cells,and the threshold for generating an NMDA spike is 10, then the dendrite will detect the target pattern when all 10 synapses receive activation at the same time.
Assuming a random distribution of patterns, the exact probability of a false match is given by:
! !!! ×
(1)
The denominator is simply the total number of possible patterns containing A active cells in a population of N total cells. The numerator counts the number of patterns that would connect to θ or more of the s synapses on one dendritic segment. A more detailed description of this equation can be found in (Ahmad and Hawkins, 2015).
The equation shows that a non-linear dendritic segment can robustly classify a pattern by sub-sampling (forming synapses to only a small number of the cells in the pattern to be classified). Table A in S1 Text lists representative error probabilities calculated from Eq.(1).
By forming more synapses than necessary to generate an NMDA spike, recognition becomes robust to noise and variation. For example, if a dendrite has an NMDA spike threshold of 10, but forms 20 synapses to the pattern it wants to recognize, twice as many as needed, it allows the dendrite to recognize the target pattern even if 50% of the cells are changed or inactive. The extra synapses also increase the likelihood of a false positive error. Although the chance of error has increased, Eq.(1) shows that it is still tiny when the patterns are sparse. In the above example, doubling the number of synapses and hence introducing a 50% noise tolerance, increases the chance of error to only 1.6 x 10^-18 . Table 1B in S1 Text lists representative error rates when the number of synapses exceeds the threshold.
The synapses recognizing a given pattern have to be co-located on a dendritic segment. If they lie within 40μm of each other then as few as eight synapses are sufficient to create an NMDA spike (Major et al., 2008). If the synapses are spread out along the dendritic segment, then up to twenty synapses are needed (Major et al., 2013). A dendritic segment can contain several hundred synapses; therefore each segment can detect multiple patterns. If synapses that recognize different patterns are mixed together on the dendritic segment, it introduces an additional possibility of error by co-activating synapses from different patterns. The probability of this type of error depends on how many sets of synapses share the dendritic segment and the sparsity of the patterns to be recognized. For a wide range of values the chance for this type of error is still low (Table C in S1 Text). Thus the placement of synapses to recognize a particular pattern is somewhat precise (they must be on the same dendritic segment and ideally within 40μm of each other), but also somewhat imprecise (mixing with other synapses is unlikely to cause errors).
If we assume an average of 20 synapses are allocated to recognize each pattern, and that a neuron has 6,000 synapses, then a cell would have the ability to recognize approximately 300 different patterns. This is a rough approximation, but makes evident that a neuron with active dendrites can learn to reliably recognize hundreds of patterns within a large population of cells. The recognition of any one of these patterns will depolarize the cell. Since all excitatory neurons in the neocortex have thousands of synapses, and, as far as we know, they all have active dendrites, then each and every excitatory neocortical neuron recognizes hundreds of patterns of neural activity.
In the next section we propose that most of the patterns recognized by a neuron do not directly lead to an action potential, but instead play a role in how networks of neurons make predictions and learn sequences.
2.1.1. Three Sources of Synaptic Input to Cortical Neurons
Neurons receive excitatory input from different sources that are segregated on different parts of the dendritic tree. Fig. 1B shows a typical pyramidal cell, the most common excitatory neuron in the neocortex. We show the input to the cell divided into three zones. The proximal zone receives feedforward input. The basal zone receives contextual input, mostly from nearby cells in the same cortical region (Petreanu et al., 2009; Rah et al., 2013; Yoshimura et al., 2000). The apical zone receives feedback input (Spruston, 2008). (The second most common excitatory neuron in the neocortex is the spiny stellate cell; we suggest they be considered similar to pyramidal cells minus the apical dendrites.) We propose the three zones of synaptic integration on a neuron (proximal, basal, and apical) serve the following purposes.
Proximal Synapses Define the Classic Receptive Field of a Cell The synapses on the proximal dendrites (typically several hundred) have a relatively large effect at the soma and therefore are best situated to define the basic receptive field response of the neuron (Spruston, 2008).
If the coincident activation of a subset of the proximal synapses is sufficient to generate a somatic action potential and If the inputs to the proximal synapses are sparsely active, then the proximal synapses will recognize multiple unique feedforward patterns in the same manner as discussed earlier. Therefore, the feedforward receptive field of a cell can be thought of as a union of feedforward patterns.
Basal Synapses Learn Transitions in Sequences
We propose that basal dendrites of a neuron recognize patterns of cell activity that precede the neuron firing, in this way the basal dendrites learn and store transitions between activity patterns. When a pattern is recognized on a basal dendrite it generates an NMDA spike. The depolarization due to an NMDA spike attenuates in amplitude by the time it reaches the soma, therefore when a basal dendrite recognizes a pattern it will depolarize the soma but not enough to generate a somatic action potential (Antic et al., 2010; Major et al., 2013). We propose this sub-threshold depolarization is an important state of the cell. It represents a prediction that the cell will become active shortly and plays an important role in network behavior. A slightly depolarized cell fires earlier than it would otherwise if it subsequently receives sufficient feedforward input. By firing earlier it inhibits neighboring cells, creating highly sparse patterns of activity for correctly predicted inputs. We will explain this mechanism more fully in a later section.
-- If the coincident activation of a subset of the proximal synapses is sufficient to generate a somatic action potential and If the synapses are active, then the proximal synapses will recognize multiple unique feedforward patterns . If the coincident activation is sufficient and If the inputs to the proximal synapses are sparsely active, then the proximal synapses will recognize multiple unique feedforward patterns in the same manner as discussed earlier.
The apical dendrites of a neuron also generate NMDA spikes when they recognize a pattern (Cichon and Gan, 2015). An apical NMDA spike does not directly affect the soma. Instead it can lead to a Ca2+ spike in the apical dendrite (Golding et al., 1999; Larkum et al., 2009). A single apical Ca2+ spike will depolarize the soma, but typically not enough to generate a somatic action potential (Antic et al., 2010). The interaction between apical Ca2+ spikes, basal NMDA spikes, and somatic action potentials is an area of ongoing research (Larkum, 2013), but we can say that under many conditions a recognized pattern on an apical dendrite will depolarize the cell and therefore have a similar effect as a recognized pattern on a basal dendrite. We propose that the depolarization caused by the apical dendrites is used to establish a top-down expectation, which can be thought of as another form of prediction.
2.1.2. The HTM Model Neuron
Fig. 1C shows an abstract model of a pyramidal neuron we use in our software simulations. We model a cell’s dendrites as a set of threshold coincidence detectors; each with its own synapses. If the number of active synapses on a dendrite/coincidence detector exceeds a threshold the cell detects a pattern. The coincidence detectors are in three groups corresponding to the proximal, basal, and apical dendrites of a pyramidal cell. We refer to this model neuron as an “HTM neuron” to distinguish it from biological neurons and point neurons. HTM is an acronym for Hierarchical Temporal Memory, a term used to describe our models of neocortex (Hawkins et al., 2011). HTM neurons used in the simulations for this paper have 128 dendrite/coincidence detectors with up to 40 synapses per dendrite. For clarity, Fig. 1C shows only a few dendrites and synapses.
-- The interaction is an area of ongoing research , but we can say that under many conditions a recognized pattern .
Because all tissue in the neocortex consists of neurons with active dendrites and thousands of synapses, it suggests there are common network principles underlying everything the neocortex does. This leads to the question, what network property is so fundamental that it is a necessary component of sensory inference, prediction, language, and motor planning?
We propose that the most fundamental operation of all neocortical tissue is learning and recalling sequences of patterns (Hawkins and Blakeslee, 2004), what Karl Lashley famously called “the most important and also the most neglected problem of cerebral physiology” (Lashley, 1951). More specifically, we propose that each cellular layer in the neocortex implements a variation of a common sequence memory algorithm. We propose cellular layers use sequence memory for different purposes, which is why cellular layers vary in details such as size and connectivity. In this paper we illustrate what we believe is the basic sequence memory algorithm without elaborating on its variations.
We started our exploration of sequence memory by listing several properties required of our network in order to model the neocortex. 1) On-line learning Learning must be continuous. If the statistics of the world change, the network should gradually and continually adapt with each new input. 2) High-order predictions Making correct predictions with complex sequences requires the ability to incorporate contextual information from the past. The network needs to dynamically determine how much temporal context is needed to make the best predictions. The term “high-order” refers to “high-order Markov chains” which have this property.
-- We propose that the most fundamental operation of all neocortical tissue is learning and recalling sequences of patterns , what Karl Lashley famously called `` the most important and also the most neglected problem '' .
3) Multiple simultaneous predictions Natural data streams often have overlapping and branching sequences. The sequence memory therefore needs to make multiple predictions at the same time. 4) Local learning rules The sequence memory must only use learning rules that are local to each neuron. The rules must be local in both space and time, without the need for a global objective function. 5) Robustness The memory should exhibit robustness to high levels of noise, loss of neurons, and natural variation in the input. Degradation in performance under these conditions should be gradual.
All these properties must occur simultaneously in the context of continuously streaming data.
4
2.2.1. Mini-columns and Neurons: Two Representations
High-order sequence memory requires two simultaneous representations. One represents the feedforward input to the network and the other represents the feedforward input in a particular temporal context. To illustrate this requirement, consider two abstract sequences “ABCD” and “XBCY”, where each letter represents a sparse pattern of activation in a population of neurons. Once these sequences are learned the network should predict “D” when presented with sequence “ABC” and it should predict “Y” when presented with sequence “XBC”. Therefore, the internal representation during the subsequence “BC” must be different in the two cases; otherwise the correct prediction can’t be made after “C” is presented.
Fig. 2 illustrates how we propose these two representations are manifest in a cellular layer of cortical neurons. The panels in Fig. 2 represent a slice through a single cellular layer in the neocortex (Fig. 2A). The panels are greatly simplified for clarity. Fig. 2B shows how the network represents two input sequences before the sequences are learned. Fig. 2C shows how the network represents the same input after the sequences are learned. Each feedforward input to the network is converted into a sparse set of active mini-columns. (Mini-columns in the neocortex span multiple cellular layers. Here we are only referring to the cells in a mini-column in one cellular layer.) All the neurons in a mini-column share the same feedforward receptive fields. If an unanticipated input arrives, then all the cells in the selected mini-columns will recognize the input pattern and become active. However, in the context of a previously learned sequence, one or more of the cells in the mini-columns will be depolarized. The depolarized cells will be the first to generate an action potential, inhibiting the other cells nearby. Thus a predicted input will lead to a very sparse pattern of cell activation that is unique to a particular element, at a particular location, in a particular sequence.
2.2.2. Basal Synapses Are the Basis of Sequence Memory
In this theory, cells use their basal synapses to learn the transitions between input patterns. With each new feedforward input some cells become active via their proximal synapses. Other cells, using their basal synapses, learn to recognize this active pattern and upon seeing the pattern again, become depolarized, thereby predicting their own feedforward activation in the next input. Feedforward input activates cells, while basal input generates predictions. As long as the next input matches the current prediction, the sequence continues, Fig. 3. Fig. 3A shows both active cells and predicted cells while the network follows a previously learned sequence.
[Figure 3 about here see end of manuscript]
Often the network will make multiple simultaneous predictions. For example, suppose that after learning the sequences “ABCD” and “XBCY” we expose the system to just the ambiguous sub-sequence “BC”. In this case we want the system to simultaneously predict both “D” and “Y”. Fig. 3B illustrates how the network makes multiple predictions when the input is ambiguous. The number of simultaneous predictions that can be made with low chance of error can again be calculated via Eq.(1). Because the predictions tend to be highly sparse, it is possible for a network to predict dozens of patterns simultaneously without confusion. If an input matches any of the predictions it will result in the correct highly-sparse representation. If an input does not match any of the predictions all the cells in a column will become active, indicating an unanticipated input.
Although every cell in a mini-column shares the same feedforward response, their basal synapses recognize different patterns. Therefore cells within a mini-column will respond uniquely in different learned temporal contexts, and overall levels of activity will be sparser when inputs are anticipated. Both of these attributes have been observed (Martin and Schr er, 2013; Vinje and Gallant, 2002; Yen et al., 2007).
For one of the cells in the last panel of Fig. 3A, we show three connections the cell used to make a prediction. In real neurons, and in our simulations, a cell would form 15 to 40 connections to a subset of a larger population of active cells.
2.2.3. Apical Synapses Create a Top-Down Expectation
Feedback axons between neocortical regions often form synapses (in layer 1) with apical dendrites of pyramidal neurons whose cell bodies are in layers 2, 3, and 5. It has long been speculated that these feedback connections implement some form of expectation or bias (Lamme et al., 1998). Our neuron model suggests a mechanism for top-down expectation in the neocortex. Fig. 4 shows how a stable feedback pattern to apical dendrites can predict multiple elements in a sequence all at the same time. When a new feedforward input arrives it will be interpreted as part of the predicted sequence. The feedback biases the input towards a particular interpretation. Again, because the patterns are sparse, many patterns can be simultaneously predicted.
Thus there are two types of prediction occurring at the same time. Lateral connections to basal dendrites predict the next input, and top-down connections to apical dendrites predict multiple sequence elements simultaneously. The physiological interaction between apical and basal dendrites is an area of active research (Larkum, 2013) and will likely lead to a more nuanced interpretation of their roles in inference and prediction. However, we propose that the mechanisms shown in Figs.2, 3 and 4 are likely to continue to play a role in that final interpretation.
2.2.4. Synaptic Learning Rule
Our neuron model requires two changes to the learning rules by which most neural models learn. First, learning occurs by growing and removing synapses from a pool of “potential” synapses (Chklovskii et al., 2004). Second, Hebbian learning and synaptic change occur at the level of the dendritic segment, not the entire neuron (Stuart and H龝sser, 2001).
Potential Synapses For a neuron to recognize a pattern of activity it requires a set of co-located synapses (typically fifteen to twenty) that connect to a subset of the cells that are active in the pattern to be recognized.
5
Learning to recognize a new pattern is accomplished by the formation of a set of new synapses collocated on a dendritic segment.
Learning to recognize a new pattern is accomplished by the formation of a set of new synapses collocated on a dendritic segment.
Figure 5 shows how we model the formation of new synapses in a simulated HTM neuron. For each dendritic segment we maintain a set of “potential” synapses between the dendritic segment and other cells in the network that could potentially form a synapse with the segment (Chklovskii et al., 2004). The number of potential synapses is larger than the number of actual
720 名前： synapses. We assign each potential synapse a scalar value called “permanence” which represents stages of growth of the synapse. A permanence value close to zero represents an axon and dendrite with the potential to form a synapse but that have not commenced growing one. A 1.0 permanence value represents an axon and dendrite with a large fully formed synapse.
[Figure 5 about here see end of manuscript]
The permanence value is incremented and decremented using a Hebbian-like rule. If the permanence value exceeds a threshold, such as 0.3, then the weight of the synapse is 1, if the permanence value is at or below the threshold then the weight of the synapse is 0. The threshold represents the establishment of a synapse, albeit one that could easily disappear. A synapse with a permanence value of 1.0 has the same effect as a synapse with a permanence value at threshold but is not as easily forgotten. Using a scalar permanence value enables on-line learning in the presence of noise. A previously unseen input pattern could be noise or it could be the start of a new trend that will repeat in the future. By growing new synapses, the network can start to learn a new pattern when it is first encountered, but only act differently after several presentations of the new pattern. Increasing permanence beyond the threshold means that patterns experienced more than others will take longer to forget. []
HTM neurons and HTM networks rely on distributed patterns of cell activity, thus the activation strength of any one neuron or synapse is not very important. Therefore, in HTM simulations we model neuron activations and synapse weights with binary states. Additionally, it is well known that biological synapses are stochastic (Faisal et al., 2008), so a neocortical theory cannot require precision of synaptic efficacy. Although scalar states and weights might improve performance, they are not required from a theoretical point of view and all of our simulations have performed well without them. The formal learning rules used in our HTM network simulations are presented in the Materials and Methods section.
3. Simulation Results
Fig. 6 illustrates the performance of a network of HTM neurons implementing a high-order sequence memory. The network used in Fig. 6 consists of 2048 mini-columns with 32 neurons per mini-column. Each neuron has 128 basal dendritic segments, and each dendritic segment has up to 40 actual synapses. Because this simulation is designed to only illustrate properties of sequence memory it does not include apical synapses. The network exhibits all five of the desired properties for sequence memory listed earlier.
Although we have applied HTM networks to many types of real-world data, in Fig. 6 we use an artificial data set to more clearly illustrate the network’s properties. The input is a stream of elements, where every element is converted to a 2% sparse activation of mini-columns (40 active columns out of 2048 total). The network learns a predictive model of the data based on observed transitions in the input stream. In Fig. 6 the data stream fed to the network contains a mixture of random elements and repeated sequences. The embedded sequences are six elements long and require high-order temporal context for full disambiguation and best prediction accuracy, e.g.“XABCDE” and “YABCFG”. For this simulation we designed the input data stream such that the maximum possible average prediction accuracy is 50% and this is only achievable by using high-order representations.
Fig. 6A illustrates on-line learning and high-order predictions. The prediction accuracy of the HTM network over time is shown in red. The prediction accuracy starts at zero and increases as the network discovers the repeated temporal patterns mixed within the random transitions. For comparison, the accuracy of a first-order network (created by using only one cell per column) is shown in blue. After sufficient learning, the high-order HTM network achieves the maximum possible prediction accuracy of 50% whereas the first-order network only achieves about 33% accuracy. After the networks reached their maximum performance the embedded sequences were modified. The accuracy drops at that point, but since the network is continually learning it recovers by learning the new high-order patterns.
Fig. 6B illustrates the robustness of the network. After the network reached stable performance we inactivated a random selection of neurons. At up to about 40% cell death there was minimal impact on performance. This robustness is due to the noise tolerance described earlier that occurs when a dendritic segment forms more synapses than necessary to generate an NMDA spike. At higher levels of cell death the network performance initially declines but then recovers as the network relearns the patterns using the remaining neurons.
4. Discussion
We presented a model cortical neuron that is substantially different than model neurons used in most artificial neural networks. The key feature of the model neuron is its use of active dendrites and thousands of synapses, allowing the neuron to recognize hundreds of unique patterns in large populations of cells. We showed that a neuron can reliably recognize many patterns, even in the presence of large amounts of noise and variation. In this model, proximal synapses define the feedforward receptive field of a cell. The basal and apical synapses depolarize the cell, representing predictions.
We showed that a network of these neurons will learn a predictive model of a stream of data. Basal synapses detect contextual patterns that predict the next feedforward input. Apical synapses detect feedback patterns that predict entire sequences. The operation of the neuron and the network rely on neural activity being sparse. The sequence memory model learns continuously, uses variable amounts of context to make predictions, makes multiple simultaneous predictions, relies on local learning rules, and is robust to failure of network elements, noise, and variation.
Although we refer to the network model as a “sequence memory”, it is actually a memory of transitions. There is no representation or concept of the length of sequences or of the number of stored sequences. The network only learns transitions between inputs. Therefore, the capacity of a network is measured by how many transitions a given network can store. This can be calculated as the product of the expected duty cycle of an individual neuron (cells per column/column sparsity) times the number of patterns each neuron can recognize on its basal dendrites. For example, a network where 2% of the columns are active, each column has 32 cells, and each cell recognizes 200 patterns on its basal dendrites, can store approximately 320,000 transitions ((32/0.02)*200). The capacity scales linearly with the number of cells per column and the number of patterns recognized by the basal synapses of each neuron.
Another important capacity metric is how many times a particular input can appear in different temporal contexts without confusion. This is analogous to how many times a particular musical interval can appear in melodies without confusion, or how many times a particular word can be memorized in different sentences. If mini-columns have 32 cells it doesn’t mean a particular pattern can have only 32 different representations. For example, if we assume 40 active columns per input, 32 cells per column, and one active cell per column, then there are 3240 possible representations of each input pattern, a practically unlimited number. Therefore, the practical limit is not representational but memory-based. The capacity is determined by how many transitions can be learned with a particular sparse set of columns.
So far we have only discussed cellular layers where all cells in the network can potentially connect to all other cells with equal likelihood. This works well for small networks but not for large networks. In the neocortex, it is well known that most regions have a topological organization. For example cells in region V1 receive feedforward input from only a small part of the retina and receive lateral input only from a local area of V1. HTM networks can be configured this way by arranging the columns in a 2D array and selecting the potential synapses for each dendrite using a 2D probability distribution centered on the neuron. Topologically organized networks can be arbitrarily large.
There are several testable predictions that follow from this theory. 1) The theory provides an algorithmic explanation for the experimentally observed phenomenon that overall cell activity becomes sparser during a continuous predictable sensory stream (Martin and Schr er, 2013; Vinje and Gallant, 2002; Yen et al., 2007). In addition, it predicts that unanticipated inputs will result in higher cell activity, which should be correlated vertically within mini-columns. Anticipated inputs on the other hand will result in activity that is uncorrelated within mini-columns. It is worth noting that mini-columns are not a strict requirement of this theory. The model only requires the presence of small groups of cells that share feedforward responses and that are mutually inhibitory. We refer to these groups as mini-columns, but the columnar aspect is not a requirement, and the groupings could be independent of actual mini-columns.
2) A second core prediction of the theory is that the current pattern of cell activity contains information about past stimuli. Early experimental results supporting this prediction have been reported in (Nikoli et al., 2009). Further studies are required to validate the exact nature of dynamic cell activity and the role of temporal context in high order sequences. 3) Synaptic plasticity should be localized to dendritic segments that have been depolarized via synaptic input followed a short time later by a back action potential. This effect has been reported (Losonczy et al., 2008), though the phenomenon has yet to be widely established. 4) There should be few, ideally only one, excitatory synapses formed between a given axon and a given dendritic segment. If an excitatory axon made many synapses in close proximity onto a single dendrite then the presynaptic cell would dominate in causing an NMDA spike. Two, three, or even four synapses from a single axon onto a single dendritic segment could be tolerated, but if axons routinely made more synapses to a single dendritic segment it would lead to errors. Pure Hebbian learning would seem to encourage forming multiple synapses. To prevent this from happening we predict the existence of a mechanism that actively discourages the formation of a multiple synapses after one has been established. An axon can form synapses onto different dendritic segments of the same neuron without causing problems, therefore we predict this mechanism will be spatially localized within dendritic segments or to a local area of an axonal arbor. 5) When a cell depolarized by an NMDA spike subsequently generates an action potential via proximal input, it needs to inhibit all other nearby excitatory cells. This requires a fast, probably single spike, inhibition. Fast-spiking basket inhibitory cells are the most likely source for this rapid inhibition (Hu et al., 2014).
6) All cells in a mini-column need to learn common feedforward responses. This requires a mechanism to encourage all the cells in a mini-column to become active simultaneously while learning feedforward patterns. This requirement for mutual excitation seems at odds with the prior requirement for mutual inhibition when one or more cells are slightly depolarized. We don’t have a specific proposal for how these two requirements are met but we predict a mechanism where sometimes cells in a column are mutually excited and at other times they are mutually inhibited.
Pyramidal neurons are common in the hippocampus. Hence, parts of our neuron and network models might apply to the hippocampus. However, the hippocampus is known for fast learning, which is incompatible with growing new synapses, as synapse formation can take hours in an adult (Holtmaat and Svoboda, 2009; Knott et al., 2002; Niell et al., 2004; Trachtenberg et al., 2002). Rapid learning could be achieved in our model if instead of growing new synapses, a cell had a multitude of inactive, or “silent” synapses (Kerchner and Nicoll, 2008). Rapid learning would then occur by turning silent synapses into active synapses. The downside of this approach is a cell would need many more synapses, which is metabolically expensive.
7
Pyramidal cells in hippocampal region CA2 have several times the number of synapses as pyramidal cells in neocortex (Meg s et al., 2001). If most of these synapses were silent it would be evidence to suggest that region CA2 is also implementing a variant of our proposed sequence memory.
It is instructive to compare our proposed biological sequence memory mechanism to other sequence memory techniques used in the field of machine learning. The most common technique is Hidden Markov Models (HMMs) (Rabiner and Juang, 1986). HMMs are widely applied, particularly in speech recognition. The basic HMM is a first-order model and its accuracy would be similar to the first-order model shown in Fig. 6A. Variations of HMMs can model restricted high order sequences by encoding high-order states by hand. More recently, recurrent neural networks, specifically long short-term memory (LSTM) (Hochreiter and Schmidhuber, 1997), have become popular, often outperforming HMMs. Unlike HTM networks, neither HMMs nor LSTMs attempt to model biology in any detail; as such they provide no insights into neuronal or neocortical functions. The primary functional advantages of the HTM model over both these techniques are its ability to learn continuously, its superior robustness, and its ability to make multiple simultaneous predictions. A more detailed comparison can be found in S1 Table.
A number of papers have studied spiking neuron models (Ghosh-Dastidar and Adeli, 2009; Maass, 1997) in the context of sequences. These models are more biophysically detailed than the neuron models used in the machine learning literature. They show how spike-timing-dependent plasticity (STDP) can lead to a cell becoming responsive to a particular sequence of presynaptic spikes and to a specific time delay between each spike (Rao and Sejnowski, 2000; Ruf and Schmitt, 1997). These models are at a lower level of detail than the HTM model proposed in this paper. They explicitly model integration times of postsynaptic potentials and the corresponding time delays are typically sub-millisecond to a few milliseconds. They also typically deal with a very small subset of the synapses and do not explicitly model non-linear active dendrites. The focus of our work has been at a higher level. The work presented in this paper is a model of the full set of synapses and active dendrites on a neuron, of a networked layer of such neurons and the emergence of a computationally sophisticated sequence memory. An interesting direction for future research is to connect these two levels of modeling, i.e. to create biophysically detailed models that operate at the level of a complete layer of cells. Some progress is reported in (Billaudelle and Ahmad, 2015), but there remains much to do on this front.
A number of papers have studied spiking neuron models (Ghosh-Dastidar and Adeli, 2009; Maass, 1997) in the context A key consideration in learning algorithms is the issue of generalization, or the ability to robustly deal with novel patterns. The sequence memory mechanism we have outlined learns by forming synapses to small samples of active neurons in streams of sparse patterns. The properties of sparse representations naturally allow such a system to generalize. Two randomly selected sparse patterns will have very little overlap. Even a small overlap (such as 20%) is highly significant and implies that the representations share significant semantic meaning. Dendritic thresholds are lower than the actual number of synapses on each segment, thus segments will recognize novel but semantically related patterns as similar. The system will see similarity between different sequences and make novel predictions based on analogy.
Recently we showed that our sequence memory method can learn a predictive model of sensory-motor sequences (Cui et al., 2015). We also see it is likely that cortical motor sequences are generated using a variation of the same network model. Understanding how layers of cells can perform these different functions and how they work together is the focus of our current research.
5. Materials and Methods
Here we formally describe the activation and learning rules for an HTM sequence memory network. There are three basic aspects to the rules: initialization, computing cell states, and updating synapses on dendritic segments. These steps are described below, along with notation and some implementation details.
Notation: Let N represent the number of mini-columns in the layer, M the number of cells per column, and NM the total number of cells in the layer. Each cell can be in an active state, in a predictive (depolarized) state, or in a non-active state. Each cell maintains a set of segments each with a number of synapses. (In this figure we use the term “synapse” to refer to “potential synapses” as described in the body of the paper. Thus at any point in time some of the synapses will have a weight of 0 and some will have a weight of 1.) At any time step t, the set of active cells is represented by the M×N binary matrix A_t , where a_t_ij is the activity of the i’th cell in the j’th column. Similarly, the M×N binary matrix Π_t denotes cells in a predictive state at time t, where π_t_ij is the predictive state of the i’th cell in the j’th column.
Each cell is associated with a set of distal segments, D_ij , such that D_d_ij represents the d’th segment of the i’th cell in the j’th column. Each distal segment contains a number of synapses, representing lateral connections from a subset of the other NM-1 cells. Each synapse has an associated permanence value (see Supplemental Fig. 2). Therefore, D_d_ij itself is also an M×N sparse matrix. If there are s potential synapses associated with the segment, the matrix contains s non-zero elements representing permanence values. A synapse is considered connected if its permanence value is above a connection threshold. We use /D_d_ij to denote a binary matrix containing only the connected synapses. 1) Initialization: the network is initialized such that each segment contains a set of potential synapses (i.e. with non-zero permanence value) to a randomly chosen subset of cells in the layer. The permanence values of these potential synapses are chosen randomly: initially some are connected (above threshold) and some are unconnected.
2) Computing cell states: All the cells in a mini-column share the same feed forward receptive fields. We assume that an inhibitory process has already selected a set of k columns that best match the current feed forward input pattern. We denote this set as W_t . The active state for each cell is calculated as follows:
8
(2)
!!! 1 if ∈ ! and !" = 1 ! ! !!! !" = 1 if ∈ and !" =0 ! 0 otherwise
The first line will activate a cell in a winning column if it was previously in a predictive state. If none of the cells in a winning column were in a predictive state, the second line will activate all cells in that column. The predictive state for the current time step is then calculated as follows:
Threshold θ represents the NMDA spiking threshold and ο represents element-wise multiplication. At a given point in time, if there are more than θ connected synapses with active presynaptic cells, then that segment will be active (generate an NMDA spike). A cell will be depolarized if at least one segment is active.
3) Updating segments and synapses: the HTM synaptic plasticity rule is a Hebbian-like rule. If a cell was correctly predicted (i.e. it was previously in a depolarized state and subsequently became active via feedforward input), we reinforce the dendritic segment that was active and caused the depolarization. Specifically, we choose those segments D_d_ij such that:
(4)
!!! ∀!∈! ! !" > 0 and !!" !!!
The first term selects winning columns that contained correct predictions. The second term selects those segments specifically responsible for the prediction.
If a winning column was unpredicted, we need to select one cell that will represent the context in the future if the current sequence transition repeats. To do this we select the cell with the segment that was closest to being active, i.e. the segment that had the most input even though it was below threshold. Let ・D_d_ij denote a binary matrix containing only the positive entries in D_d_ij . We reinforce a segment where the following is true:
Reinforcing the above segments is straightforward: we wish to reward synapses with active presynaptic cells and punish synapses with inactive cells. To do that we decrease all the permanence values by a small value p- and increase the permanence values corresponding to active presynaptic cells by a larger value p+ :
(6)
!!" = ! !!" !!! ! !!"
The above rules deal with cells that are currently active. We also apply a very small decay to active segments of cells that did not become active. This can happen if segments were mistakenly reinforced by chance:
(7)
!!" = !!" where ! !" = 0 and !!" !!!
The matrices ΔD_d_ij are added to the current matrices of permanence values at every time step.
Implementation details: in our software implementation, we make some simplifying assumptions that greatly speed up simulation time for larger networks. Instead of explicitly initializing a complete set of synapses across every segment and every cell, we greedily create segments on a random cell and initialize potential synapses on that segment by sampling from currently active cells. This happens only when there is no match to any existing segment. In our simulations N = 2048, M = 32, k = 40. We typically connect between 20 and 40 synapses on a segment, and θ is around 15. Permanence values vary from 0 to 1 with a connection threshold of 0.5. p+ and p- are small values that are tuned based on the individual dataset but typically less than 0.1. The full source code for the implementation is available on Github at github.com/numenta/nupic
Ahmad, S., and Hawkins, J. (2015). Properties of Sparse Distributed Representations and their Application to Hierarchical Temporal Memory. arXiv:1503.07469 [q-bio.NC] Antic, S.
D., Zhou, W. L., Moore, A. R., Short, S. M., and Ikonomu, K. D. (2010). The decade of the dendritic NMDA spike. J. Neurosci. Res. 88, 2991-3001. doi:10.1002/jnr.22444.
Billaudelle, S., and Ahmad, S. (2015). Porting HTM Models to the Heidelberg Neuromorphic Computing Platform. arXiv:1505.02142 [q-bio.NC]
Chklovskii, D. B., Mel, B. W., and Svoboda, K. (2004). Cortical rewiring and information storage. Nature 431, 782-8. doi:10.1038/nature03012.
Cichon, J., and Gan, W.-B. (2015). Branch-specific dendritic Ca2+ spikes cause persistent synaptic plasticity. Nature 520, 180-5.
Cui, Y., Ahmad, S., Surpur, C., and Hawkins, J. (2015). Maintaining stable perception during active exploration. in Cosyne Abstracts (Salt Lake City).
Faisal, A. A., Selen, L. P. J., and Wolpert, D. M. (2008). Noise in the nervous system. Nat. Rev. Neurosci. 9, 292-303.
Ghosh-Dastidar, S., and Adeli, H. (2009). Spiking neural networks. Int. J. Neural Syst. 19, 295-308.
Golding, N. L., Jung, H. Y., Mickus, T., and Spruston, N. (1999). Dendritic calcium spike initiation and repolarization are controlled by distinct potassium channel subtypes in CA1 pyramidal neurons. J. Neurosci. 19, 8789-98.
Knott, G. W., Quairiaux, C., Genoud, C., and Welker, E. (2002). Formation of dendritic spines with GABAergic synapses induced by whisker stimulation in adult mice. Neuron 34, 265-73.
Lamme, V. A., Supèr, H., and Spekreijse, H. (1998). Feedforward, horizontal, and feedback processing in the visual cortex. Curr. Opin. Neurobiol. 8, 529-35.
Larkum, M. (2013). A cellular mechanism for cortical associations: an organizing principle for the cerebral cortex. Trends Neurosci. 36, 141-51. doi:10.1016/j.tins.2012.11.006.
Larkum, M. E., and Nevian, T. (2008). Synaptic clustering by dendritic signalling mechanisms. Curr. Opin. Neurobiol. 18, 321- 31. doi:10.1016/j.conb.2008.08.013.
Larkum, M. E., Nevian, T., Sandler, M., Polsky, A., and Schiller, J. (2009). Synaptic integration in tuft dendrites of layer 5 pyramidal neurons: a new unifying principle. Science 325, 756-760. doi:10.1126/science.1171958.
Larkum, M. E., Zhu, J. J., and Sakmann, B. (1999). A new cellular mechanism for coupling inputs arriving at different cortical layers. Nature 398, 338-41.
Lashley, K. (1951). “The problem of serial order in behavior,” in Cerebral mechanisms in behavior, ed. L. Jeffress (New York: Wiley), 112-131.
LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. Nature 521, 436-444.
Losonczy, A., Makara, J. K., and Magee, J. C. (2008). Compartmentalized dendritic plasticity and input feature storage in neurons. Nature 452, 436-41. doi:10.1038/nature06725.
Maass, W. (1997). Networks of spiking neurons: the third generation of neural network models. Neural networks 10, 1659-1671. doi:10.1016/S0893-6080(97)00011-7.
Major, G., Larkum, M. E., and Schiller, J. (2013). Active properties of neocortical pyramidal neuron dendrites. Annu. Rev. Neurosci. 36, 1-24. doi:10.1146/annurev-neuro-062111-150343.
Major, G., Polsky, A., Denk, W., Schiller, J., and Tank, D. W. (2008). Spatiotemporally graded NMDA spike/plateau potentials in basal dendrites of neocortical pyramidal neurons. J. Neurophysiol. 99, 2584-601.
Martin, K. A. C., and Schr er, S. (2013). Functional heterogeneity in neighboring neurons of cat primary visual cortex in response to both artificial and natural stimuli. J. Neurosci. 33, 7325-44.
Meg as, M., Emri, Z., Freund, T. F., and Guly s, A. I. (2001). Total number and distribution of inhibitory and excitatory synapses on hippocampal CA1 pyramidal cells. Neuroscience 102, 527-40.
Niell, C. M., Meyer, M. P., and Smith, S. J. (2004). In vivo imaging of synapse formation on a growing dendritic arbor. Nat. Neurosci. 7, 254-60.
Nikoli, D., H龝sler, S., Singer, W., and Maass, W. (2009). Distributed fading memory for stimulus properties in the primary visual cortex. PLoS Biol. 7, e1000260.
Olshausen, B. A., and Field, D. J. (2004). Sparse coding of sensory inputs. Curr. Opin. Neurobiol. 14, 481-487. doi:10.1016/j.conb.2004.07.007.
Petreanu, L., Mao, T., Sternson, S. M., and Svoboda, K. (2009). The subcellular organization of neocortical excitatory connections. Nature 457, 1142-5.
Poirazi, P., Brannon, T., and Mel, B. W. (2003). Pyramidal neuron as two-layer neural network. Neuron 37, 989-99.
Polsky, A., Mel, B. W., and Schiller, J. (2004). Computational subunits in thin dendrites of pyramidal cells. Nat. Neurosci. 7, 621-7. doi:10.1038/nn1253.
Rabiner, L., and Juang, B. (1986). An introduction to hidden Markov models. IEEE ASSP Mag. 3, 4-16.
Rah, J.-C., Bas, E., Colonell, J., Mishchenko, Y., Karsh, B., Fetter, R. D., et al. (2013). Thalamocortical input onto layer 5 pyramidal neurons measured using quantitative large-scale array tomography. Front. Neural Circuits 7, 177.
Rao, R. P. N., and Sejnowski, T. J. (2000). Predictive Sequence Learning in Recurrent Neocortical Circuits. in in Advances in Neural Information Processing Systems 12, 164-170. doi:10.1.1.45.9739.
Ruf, B., and Schmitt, M. (1997). Learning temporally encoded patterns in networks of spiking neurons. Neural Process. Lett., 9-18. doi:10.1023/A:1009697008681.
Schiller, J., Major, G., Koester, H. J., and Schiller, Y. (2000). NMDA spikes in basal dendrites of cortical pyramidal neurons. Nature 404, 285-9.
Schiller, J., and Schiller, Y. (2001). NMDA receptor-mediated dendritic spikes and coincident signal amplification. Curr. Opin. Neurobiol. 11, 343-8.
Spruston, N. (2008). Pyramidal neurons: dendritic structure and synaptic integration. Nat. Rev. Neurosci. 9, 206-21.
Stuart, G. J., and H龝sser, M. (2001). Dendritic coincidence detection of EPSPs and action potentials. Nat. Neurosci. 4, 63-71.
11
Trachtenbe
742 名前：rg, J. T., Chen, B. E., Knott, G. W., Feng, G., Sanes, J. R., Welker, E., et al. (2002). Long-term in vivo imaging of experience-dependent synaptic plasticity in adult cortex. Nature 420, 788-94.
Vinje, W. E., and Gallant, J. L. (2002). Natural Stimulation of the Nonclassical Receptive Field Increases Information Transmission Efficiency in V1. J. Neurosci. 22, 2904-2915.
Yen, S.-C., Baker, J., and Gray, C. M. (2007). Heterogeneity in the responses of adjacent neurons to natural stimuli in cat striate cortex. J. Neurophysiol. 97, 1326-1341. doi:10.1167/7.9.326.
Yoshimura, Y., Sato, H., Imamura, K., and Watanabe, Y. (2000). Properties of horizontal and vertical inputs to pyramidal cells in the superficial layers of the cat visual cortex. J. Neurosci. 20, 1931-40.
Figure 1: Comparison of neuron models. A) The neuron model used in most artificial neural networks has few synapses and no dendrites. B) A neocortical pyramidal neuron has thousands of excitatory synapses located on dendrites (inset). The co-activation of a set of synapses on a dendritic segment will cause an NMDA spike and depolarization at the soma. There are three sources of input to the cell. The feedforward inputs (shown in green) which form synapses proximal to the soma, directly lead to action potentials. NMDA spikes generated in the more distal basal and apical dendrites depolarize the soma but typically not sufficiently to generate a somatic action potential. C) An HTM model neuron models dendrites and NMDA spikes with an array of coincident detectors each with a set of synapses (only a few of each are shown).
Same columns, but only one cell active per column.
C After learning
A B' C' D' X B'' C'' Y''
Figure 2: Representing sequences in cortical cellular layers. A) The neocortex is divided into cellular layers. The panels in this figure show part of one generic cellular layer. For clarity, the panels only show 21 mini-columns with 6 cells per column. B) Input sequences ABCD and XBCY are not yet learned. Each sequence element invokes a sparse set of mini-columns, only three in this illustration. All the cells in a mini-column become active if the input is unexpected, which is the case prior to learning the sequences. C) After learning the two sequences, the inputs invoke the same mini-columns but only one cell is active in each column, labeled B’, B’’, C’, C’’, D’ and Y’’. Because C’ and C’’ are unique, they can invoke the correct high-order prediction of either Y or D.
Prediction of next input A - input B' - predicted B - input C' - predicted
B
Multiple simultaneous predictions B - input C' and C'' - predicted C -input D' and Y'' - predicted
Figure 3: Basal connections to nearby neurons predict the next input. A) Using one of the sequences from Fig. 2, both active cells (black) and depolarized/predicted cells (red) are shown. The first panel shows the unexpected input A, which leads to a prediction of the next input B’ (second panel). If the subsequent input matches the prediction then only the depolarized cells will become active (third panel), which leads to a new prediction (fourth panel). The lateral synaptic connections used by one of the predicted cells are shown in the rightmost panel. In a realistic network every predicted cell would have 15 or more connections to a subset of a large population of active cells. B) Ambiguous sub-sequence “BC” (which is part of both ABCD and XBCY) is presented to the network. The first panel shows the unexpected input B, which leads to a prediction of both C’ and C’’. The third panel shows the system after input C. Both sets of predicted cells become active, which leads to predicting both D and Y (fourth panel). In complex data streams there are typically many simultaneous predictions.
Figure 4: Feedback to apical dendrites predicts entire sequences. This figure uses the same network and representations as Fig. 2. Area labeled “apical dendrites” is equivalent to layer 1 in neocortex; the apical dendrites (not shown) from all the cells terminate here. In the figure, the following assumptions have been made. The network has previously learned the sequence ABCD as was illustrated in Fig. 2. A constant feedback pattern was presented to the apical dendrites during the learned sequence, and the cells that participate in the sequence B’C’D’ have formed synapses on their apical dendrites to recognize the constant feedback pattern.
After the feedback connections have been learned, presentation of the feedback pattern to the apical dendrites is simultaneously recognized by all the cells that would be active sequentially in the sequence. These cells, shown in red, become depolarized (left pane). When a new feedforward input arrives it will lead to the sparse representation relevant to the predicted sequence (middle panel). If a feedforward pattern cannot be interpreted as part of the expected sequence (right panel) then all cells in the selected columns become active indicative of an anomaly. In this manner apical feedback biases the network to interpret any input as part of an expected sequence and detects if an input does not match any one of the elements in the expected sequence.
Figure 5: Learning by growing new synapses. Learning in an HTM neuron is modeled by the growth of new synapses from a set of potential synapses. A “permanence” value is assigned to each potential synapse and represents the growth of the synapse. Learning occurs by incrementing or decrementing permanence values. The synapse weight is a binary value set to 1 if the permanence is above a threshold.
Figure 6: Simulation results of the sequence memory network. The input stream used for this figure contained high-order sequences mixed with random elements. The maximum possible average prediction accuracy of this data stream is 50%. A) High-order on-line learning. The red line shows the network learning and achieving maximum possible performance after about 2500 sequence elements. At element 3000 the sequences in the data stream were changed. Prediction accuracy drops and then recovers as the model learns the new temporal structure. For comparison, the lower performance of a first-order network is shown in blue. B) Robustness of the network to damage. After the network reached stable performance we inactivated a random selection of neurons. At up to 40% cell death there is almost no impact on performance. At greater than 40% cell death the performance of the network declines but then recovers as the network relearns using remaining neurons.
18
S1 Text.
Chance of Error When Recognizing Large Patterns with a Few Synapses Formula for calculating chance of error
A non-linear dendritic segment can robustly classify a pattern by sub-sampling (forming synapses to) a small number of cells from a large population. Assuming a random distribution of patterns, the exact probability of a false match, following is given by the following equation:
= cell population size = number of active cells = number of synapses on segment = NMDA spike threshold
Table A: Chance of error due to sub-sampling
This table demonstrates the effect of sub-sampling on the probability of a false match using the above equation. The chance of an error drops rapidly as the sampling size increases. A small number of synapses is sufficient for reliable matching.
Table B: Chance of error with addition of 50% noise immunity
This table demonstrates robustness to noise. By forming more synapses than required for an NMDA spike, a neuron can be robust to large amounts of noise and pattern variation and still have low probability of a false match. For example, with s = 2θ the system will be immune to 50% noise. The chance of an error drops rapidly as increases; even with noise a small number of synapses is sufficient for reliable matching.
Table C: Chance of error with addition of mixing synapses on a dendritic segment
This table demonstrates that mixing synapses for m different patterns on a single dendritic segment will still not cause unacceptable errors. By setting s = 2mθ we can see how a segment can recognize m independent patterns and still be robust to 50% noise. It is possible to get very high accuracy with larger m by using a slightly higher threshold.
Robustness and fault tolerance HTM Very high HMMs No LSTM Yes
Detailed mapping to neuroscience HTM Yes HMMs No LSTM No
Probabilistic model HTM No HMMs Yes LSTM No
S1 Table, Comparison of Common Sequence Memory Algorithms Table comparing two common sequence memory algorithms (HMM and LSTM) to proposed model (HTM). * Although weight updated rules are local, LSTMs require computing a global error signal that is then back propagated.
Universal Transformers arxiv-vanity.com/papers/1807.03819/ References [1] Karim Ahmed, Nitish Shirish Keskar, and Richard Socher. Weighted transformer network for machine translation. arXiv preprint arXiv:1711.02132, 2017. [2] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016. [3] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. CoRR, abs/1409.0473, 2014. [4] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. CoRR, abs/1406.1078, 2014. [5] Francois Chollet. Xception: Deep learning with depthwise separable convolutions. arXiv preprint arXiv:1610.02357, 2016. [6] Zewei Chu, Hai Wang, Kevin Gimpel, and David McAllester. Broad context language modeling as reading comprehension. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, volume 2, pages 52–57, 2017. [7] Bhuwan Dhingra, Qiao Jin, Zhilin Yang, William W Cohen, and Ruslan Salakhutdinov. Neural models for reasoning over multiple mentions using coreference. arXiv preprint arXiv:1804.05922, 2018. [8] Bhuwan Dhingra, Zhilin Yang, William W Cohen, and Ruslan Salakhutdinov. Linguistic knowledge as memory for recurrent neural networks. arXiv preprint arXiv:1703.02620, 2017. [9] Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. Convolutional sequence to sequence learning. CoRR, abs/1705.03122, 2017. [10] Edouard Grave, Armand Joulin, and Nicolas Usunier. Improving neural language models with a continuous cache. arXiv preprint arXiv:1612.04426, 2016.
[11] Alex Graves. Generating sequences with recurrent neural networks. CoRR, abs/1308.0850, 2013. [12] Alex Graves. Adaptive computation time for recurrent neural networks. arXiv preprint arXiv:1603.08983, 2016. [13] Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. CoRR, abs/1410.5401, 2014. [14] Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun. Tracking the world state with recurrent entity networks. arXiv preprint arXiv:1612.03969, 2016. [15] Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, and Jürgen Schmidhuber. Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. A Field Guide to Dynamical Recurrent Neural Networks, 2003. [16] A. Joulin and T. Mikolov. Inferring algorithmic patterns with stack-augmented recurrent nets. In Advances in Neural Information Processing Systems, (NIPS), 2015. [17] Łukasz Kaiser and Ilya Sutskever. Neural GPUs learn algorithms. In International Conference on Learning Representations (ICLR), 2016. [18] Nikita Kitaev and Dan Klein. Constituency parsing with a self-attentive encoder. In Proceedings of ACL’18, 2018. [19] Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. A structured self-attentive sentence embedding. arXiv preprint arXiv:1703.03130, 2017. [20] Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. Assessing the ability of lstms to learn syntax-sensitive dependencies. Transactions of the Association of Computational Linguistics, 4(1):521–535, 2016.
[21] Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Ngoc Quan Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernandez. The lambada dataset: Word prediction requiring a broad discourse context. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1525–1534, 2016. [22] Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. A decomposable attention model. In Empirical Methods in Natural Language Processing, 2016. [23] Jack Rae, Jonathan J Hunt, Ivo Danihelka, Timothy Harley, Andrew W Senior, Gregory Wayne, Alex Graves, and Tim Lillicrap. Scaling memory-augmented neural networks with sparse reads and writes. In Advances in Neural Information Processing Systems, pages 3621–3629, 2016. [24] Minjoon Seo, Sewon Min, Ali Farhadi, and Hannaneh Hajishirzi. Query-reduction networks for question answering. arXiv preprint arXiv:1606.04582, 2016. [25] Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929–1958, 2014. [26] Sainbayar Sukhbaatar, arthur szlam, Jason Weston, and Rob Fergus. End-to-end memory networks. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 2440–2448. Curran Associates, Inc., 2015. [27] Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, pages 3104–3112, 2014. [28] Ke Tran, Arianna Bisazza, and Christof Monz. The importance of being recurrent for modeling hierarchical structure. In Proceedings of NAACL’18, 2018.
[29] Ashish Vaswani, Samy Bengio, Eugene Brevdo, Francois Chollet, Aidan N. Gomez, Stephan Gouws, Llion Jones, Łukasz Kaiser, Nal Kalchbrenner, Niki Parmar, Ryan Sepassi, Noam Shazeer, and Jakob Uszkoreit. Tensor2tensor for neural machine translation. CoRR, abs/1803.07416, 2018. [30] Ashish Vaswani, Niki Parmar, Jakob Uszkoreit, Noam Shazeer, and Lukasz Kaiser. Image transformer , 2018. [31] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. CoRR, 2017. [32] Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart van Merriënboer, Armand Joulin, and Tomas Mikolov. Towards ai-complete question answering: A set of prerequisite toy tasks. arXiv preprint arXiv:1502.05698, 2015. [33] Dani Yogatama, Yishu Miao, Gabor Melis, Wang Ling, Adhiguna Kuncoro, Chris Dyer, and Phil Blunsom. Memory architectures in recurrent neural network language models. In International Conference on Learning Representations, 2018. [34] Wojciech Zaremba and Ilya Sutskever. Learning to execute. CoRR, abs/1410.4615, 2015.
Appendix C bAbI Attention Visualization We present visualization of the attention distributions on bAbI tasks for a couple of examples. The visualization of attention weights is over different time steps based on different heads over all the facts in the story and a question. Different color bars on the left side indicate attention weights based on different heads (4 heads in total). An example from tasks 1: (requiring one supportive fact to solve) Story: John travelled to the hallway. Mary journeyed to the bathroom. Daniel went back to the bathroom. John moved to the bedroom Query: Where is Mary? Model’s output: bathroom (a) Step 1 (b) Step 2 (c) Step 3 (d) Step 4
Figure 5: Visualization of the attention distributions, when encoding the question: “Where is Mary?”. An example from tasks 2: (requiring two supportive facts to solve) Story: Sandra journeyed to the hallway. Mary went to the bathroom. Mary took the apple there. Mary dropped the apple. Query: Where is the apple? Model’s output: bathroom (a) Step 1 (b) Step 2 (c) Step 3 (d) Step 4
Figure 6: Visualization of the attention distributions, when encoding the question: “Where is the apple?”. An example from tasks 2: (requiring two supportive facts to solve) Story: John went to the hallway. John went back to the bathroom. John grabbed the milk there. Sandra went back to the office. Sandra journeyed to the kitchen. Sandra got the apple there. Sandra dropped the apple there. John dropped the milk. Query: Where is the milk? Model’s output: bathroom (a) Step 1 (b) Step 2 (c) Step 3 (d) Step 4
Figure 7: Visualization of the attention distributions, when encoding the question: “Where is the milk?”. An example from tasks 3: (requiring three supportive facts to solve) Story: Mary got the milk. John moved to the bedroom. Daniel journeyed to the office. John grabbed the apple there. John got the football. John journeyed to the garden. Mary left the milk. John left the football. Daniel moved to the garden. Daniel grabbed the football. Mary moved to the hallway. Mary went to the kitchen. John put down the apple there. John picked up the apple. Sandra moved to the hallway. Daniel left the football there. Daniel took the football. John travelled to the kitchen. Daniel dropped the football. John dropped the apple. John grabbed the apple. John went to the office. Sandra went back to the bedroom. Sandra took the milk. John journeyed to the bathroom. John travelled to the office. Sandra left the milk. Mary went to the bedroom.
Mary moved to the office. John travelled to the hallway. Sandra moved to the garden. Mary moved to the kitchen. Daniel took the football. Mary journeyed to the bedroom. Mary grabbed the milk there. Mary discarded the milk. John went to the garden. John discarded the apple there. Query: Where was the apple before the bathroom? Model’s output: office (a) Step 1 (b) Step 2 (c) Step 3 (d) Step 4
Figure 8: Visualization of the attention distributions, when encoding the question: “Where was the apple before the bathroom?”. Generated by LaTeXML [LOGO] Want to hear about new tools we're making? Sign up to our mailing list for occasional updates. Subscribe Built by Andreas Jansson and Ben Firshman, with help from LaTeXML. Contribute on GitHub. Kindly sponsored by YLD.
Author: Erin Masterson Category: Disaster Recovery, Data Centers, Infrastructure Availability
Disaster recovery planning is an integral part of any business’s IT strategy, and is becoming more prevalent as security breaches and network outages have become common threats, and the cost of downtime has steadily increased.
In the beginning stages of disaster recovery planning, decision makers are often mistaken about what constitutes a disaster recovery plan. Many times they are misled by the idea that data backup is sufficient precaution in the event of a disaster.
“Customers often come to us seeking disaster recovery services without realizing that simply backing up their data is not enough,” says Joe Palian, Regional Account Executive at Expedient.
While having a backup strategy is important, it is not the same as a disaster recovery strategy; rather, the beginning stages of establishing a proper DR plan. A backup is a copy of your data; a disaster recovery plan is insurance that guarantees its recovery.
So, what makes backups and disaster recovery different?
1.) Data retention requirements Backups are typically performed on a daily basis to ensure necessary data retention at a single location, for the single purpose of copying data.
Disaster recovery requires the determination of the RTO (recovery time objective) in order to designate the maximum amount of time the business can be without IT systems post-disaster. Traditionally, the ability to meet a given RTO requires at least one duplicate of the IT infrastructure in a secondary location to allow for replication between the production and DR site. 2.) Recovery ability Disaster recovery is the process of failing over your primary environment to an alternate environment that is capable of sustaining your business continuity.
Backups are useful for immediate access in the event of the need to restore a document, but does not facilitate the failover of your total environment should your infrastructure become compromised. They also do not include the physical resources required to bring them online.
3.) Additional resource needs A backup is simply a copy of data intended to be restored to the original source.
DR requires a separate production environment where the data can live. All aspects of the current environment should be considered, including physical resources, software, connectivity and security. 4.) Planning process Planning a backup routine is relatively simple, since typically the only goals are to meet the RPO (recovery point objective) and data retention requirements.
A complete disaster recovery strategy requires additional planning, including determining which systems are considered mission critical, creating a recovery order and communication process, and most importantly, a way to perform a valid test.
The overall benefits and importance of a DR plan are to mitigate risk and downtime, maintain compliance and avoid outages. Backups serve a simpler purpose. Make sure you know which solution makes sense for your business needs.
Looking to improve your DR preparedness? Follow these 6 Steps.
the cost of downtime https://thecloudcalculator.com/calculators/cost-of-downtime/ often mistaken about what constitutes a disaster recovery plan https://www.expedient.com/blog/the-differences-between-backups-and-disaster-recovery/ a disaster recovery strategy https://www.expedient.com/blog/what-steps-have-you-left-out-of-your-dr-strategy/ Disaster recovery https://www.expedient.com/services/managed-services/disaster-recovery/ the failover of your total environment https://www.expedient.com/blog/expedient-push-button-dr-zertocon2018/ a way to perform a valid test https://www.expedient.com/blog/with-push-button-dr-disaster-recovery-testing-doesnt-have-to-be-a-four-letter-word/ Follow these 6 Steps bit.ly/1SFm5yp
A scalable pipeline for designing reconfigurable organisms View ORCID ProfileSam Kriegman, Douglas Blackiston, Michael Levin, and Josh Bongard PNAS first published January 13, 2020 doi.org/10.1073/pnas.1910837117
Sam Kriegman aDepartment of Computer Science, U
773 名前：niversity of Vermont, Burlington, VT 05405;
Douglas Blackiston bDepartment of Biology, Tufts University, Medford, MA 02153;cAllen Discovery Center, Tufts University, Medford, MA 02153;
Michael Levin bDepartment of Biology, Tufts University, Medford, MA 02153;cAllen Discovery Center, Tufts University, Medford, MA 02153;dWyss Institute for Biologically Inspired Engineering, Harvard University, Boston, MA 02115
Josh Bongard aDepartment of Computer Science, University of Vermont, Burlington, VT 05405;
Phase offsets stored in the genotype were mutated by adding a number that was drawn randomly from a normal distribution with mean zero and SD s = 0.4π.
Also, contractile tissue incurs a much higher metabolic cost compared to nonmuscle tissue (the human heart consumes 1 mM ATP per second; ref. 31 ). Also, contractile tissue incurs a much higher metabolic cost compared to nonmuscle tissue (the human heart consumes ∝1 mM ATP per second; ref. 31 ). []
The influence of one memory retrieval on a subsequent memory retrieval *
GEOFFREY R. LOFTUS and ELIZABETH F. LOFTUS University of Washington, Seattle, Washington 98195
*Requests for reprints may be sent to either Loftus, Department of Psychology, University of Washington, Seattle, Washington 98195. The research was supported by a National Institute of Mental Health grant to E. Loftus and a National Science Foundation grant to G. Loftus. Appreciation is expressed to Thomas 0. Nelson for his comments on the manuscript.
Ss produced an instance of a category and following zero or two intervening items produced a second instance of the same category. The second instance was produced more quickly than the initial instance. This finding, in conjunction with other data reported in the paper, indicate that the reduction in latency for the second instance is due mostly to a reduction in the rate with which the category is searched.
In an experiment by Freedman and Loftus (1971), Ss were shown a noun category plus a restricting letter or adjective and were asked to name an instance of the category which began with the letter or which was characterized by the adjective. Reaction time to produce the response was measured. The data were discussed in terms of a model that postulated a hierarchical memory composed of noun categories (e.g., animals) with subsets (eg., birds, dogs) and supersets (e.g., living things) of each category. Retrieval from this hierarchical structure was assumed to consist of at least two major steps: (1) entering the appropriate category and (2) searching the category for an appropriate member. The times to execute Step 1 and Step 2 are hereafter denoted t1 and t2, respectively. The duration of t1 was estimated to be about .25 sec by the following reasoning. Ss saw stimuli presented with the category either first (e.g., fruit-P) or second (e.g., P-fruit) and with at least a 1/2-sec interval between the noun and restrictor. Reaction times were measured from the presentation of the second member of the pair. When the category came second, the total retrieval process began only after its presentation and included both t1 and t2, according to the model. When the category came first, however, t1 could be completed before the restrictor was shown. For example, given the stimulus fruit-P, the S could enter the category “fruits” during the interval. Since measured reaction time begins when “P” is presented, measured reaction time excludes t1 in this case. The decrease in reaction time when the category is shown first vs second can therefore be equated with t1, which is excluded in the former case and included in the latter.
More recently, Loftus (1973) asked Ss to produce a member of a category and a short time later asked them to produce a different member of that category. -- This was accomplished by showing a category-letter pair , which asked the S for an appropriate instance, then, following zero, or two intervening items, showing the same category paired with a different letter , which asked for a different instance. This was accomplished by showing a AB pair (eg., cat-P), which asked the S for an appropriate instance, then, following zero, or two intervening items, showing the same BB paired with a different letter (e.g., cat-A), which asked for a different instance. -- Interest centered around the question of whether the speed of retrieving the second instance of a category was affected by the retrieval of the first instance and/or the lag between the two retrievals. The results indicated that response latency for the second instance was shorter than response latency for the first instance and increased monotonically with the number of intervening items. For example, a S’s baseline time to name a fruit beginning with the letter “P” was 1.52 sec. However, it took him 1.22 sec to produce the same response if he had named a different fruit on the previous trial and 1.29 sec to produce the response if he had named a different fruit two trials back.
The results of the Loftus (1973) study thus indicate that the process of retrieving information from a category facilitates a subsequent retrieval from that category. However, in this experiment the S was presented with the category name and restricting letter simultaneously; retrieval time thus included both t1 and t2. Consequently, the facilitation effect could have involved a reduction in t1 or t2 or both. The present experiment is designed to distinguish among these three possibilities.
-- This was accomplished by showing a category-letter pair (eg., fruit-P), which asked the S for an appropriate instance, then, following zero, one, or two intervening items, showing the same category paired with a different letter (e.g., fruit-A), which asked for a different instance. --
In some conditions of the present experiment, an interval was inserted between the category name and the letter and the stimuli were presented either in the order category-letter or in the order letter-category [as in the Freedman & Loftus (1971) study]. As noted above, this procedure allows an estimation of t1 . Additionally in the present experiment, the S was required to name an instance of a category and shortly thereafter was asked to name a second instance of the category [as in the Loftus (1973) study]. This design is sufficient to determine the locus of the reduction in reaction time to name a second category instance.
Figure 1 shows three possible patterns of results. Suppose first that only category entry time, t1, is reduced when a second category instance is produced. In this case, the results shown in Fig.1a should obtain: the letter-category conditions (which include t1) should depend on the prior retrieval, whereas the category-letter conditions (which exclude t1) should not.
467
-- In some conditions of the present experiment, an interval was inserted between the category name and the letter and the stimuli were presented either in the order A in the order B [as in the Freedman & Loftus (1971) study].
RT * letter - category * category - letter 0 2 initial LAG
b
RT
0 2 initial LAG
c
RT
0 2 initial LAG
Fig.1. Three possible patterns of results for the relationship between time and the number of intervening items (lag) between two appearances of a critical category.
Conversely, suppose that only category search time, t2, is reduced when the second category instance is produced. Such a situation would lead to the results shown in Fig.1b. Both the category-letter and the letter-category conditions include t2, so they should be affected equally by the initial retrieval.
The final possibility is that both t1 and t2 are reduced. This situation would predict the results shown in Fig.1c. Here, the category-letter condition (which includes t2 but not t1) should be affected by the initial retrieval, but the letter-category condition (which includes both t, and t2) should be affected to a greater degree.
METHOD
Subjects Eighteen Ss from the New School for Social Research received $5 for their participation in two 1-h sessions, which occurred on 2 consecutive days. No S had previously participated in a memory experiment.
Materials Each stimulus was printed in block letters on a 5 x 8 in. index card. A stimulus always consisted of a category name plus a letter (e.g., fruit-P). Eighty critical category names were selected from the Battig and Montague (1969) and Shapiro and Palermo (1970) category norms. Each of the category names was paired with two different letters. If “dominance" is defined as the frequency with which a word is given as an exemplar of a category, then one of the twu category-letter stimuli will be referred to as more dominant than the other.
In addition to the 160 critical stimuli (80 categories each paired with two letters), 80 filler stimuli were used. The filler stimuli also consisted of a category plus a letter. Some of the filler categories were used only once; others appeared twice with two different letters. Thus, each S saw 240 unique stimuli (80 critical categories, each paired with two letters, plus 80 filler stimuli).
Design There were three within-S factors: order (category-letter vs letter-category), interval (simultaneous presentation of the stimuli vs 2.5-sec interval between the category name and the letter), and lag (Lag 0, Lag 2, and initial presentation). These factors were combined factorially, thereby giving a 2 (orders) by 2 (intervals) by 3 (lags) by 18 (Ss) design.
Each S received a different permutation of the 240 items with the following restrictions:
(1)The initial presentation of a critical category-letter pair was followed after zero or two intervening filler items (i.e., at Lag 0 or at Lag 2) by the presentation of the same category paired with a different letter. Each S received 40 stimuli presented at Lag 0 and 40 at Lag 2.
(2) On half of the trials, Ss saw the stimulus corresponding to the high dominant instance before seeing the stimulus corresponding to the low dominant instance. For the remaining trials, the reverse arrangement held. A given category was presented in the order dominant-nondominant for half the Ss and in the reverse order for the remaining half of the Ss.
Procedure Each S was told that he would see items consisting of categories and letters and that he was to respond with a word in the category that began with the given letter. He was given examples and told to respond as quickly as possible, but to avoid errors.
The S sat in front of a screen with a window covered by half-silvered glass. An index card containing the stimulus was placed in a dark enclosure behind the minor and was presented by illuminating the enclosure. A microphone was placed in front of the S, and he responded by speaking into it.
A trial consisted of the following: (a) a card with the item printed in large type was placed in the darkened enclosure; (b) the E said “ready” and pressed a button which illuminated the first member of the stimulus pair; (c) either simultaneously or after a 2.5-sec interval, the second member of the pair was automatically illuminated and an electric timer started; (d) the S’s verbal response activated a voice key that stopped the timer and temrinated the trial. A warm-up period of 20 trials preceded the experimental trials each day.
Only correct responses (96%) to the critical stimuli were included in the following analyses. Median latencies were obtained for each S’s responses in each of the 12 conditions. For each condition, mean latencies were then obtained by averaging the medians from individual Ss; these means are plotted in Figs. 2 and 3. Figure 2 shows the results when the 2.5-sec interval was inserted between the category and the letter. In both the letter-category and category-letter conditions, a second instance of a category is produced faster than the first instance; furthermore, a second instance is produced faster at Lag 0 than at Lag 2. Figure 3 indicates that the same pattern of results obtains when letter and noun are presented simultaneously.
A 2 (orders) by 2 (intervals) by 3 (lags) analysis of variance was done on the latency data. Significant effects were found for lag [F(2,34) = 6.57, p < .05], category-letter order [F(1,17) = 14.71, p< .01],‘and interval [F(1,17) = 33.52, p <01].
THE INFLUENCE OF ONE MEMORY RETRIEVAL 469
None of the two-way or three-way interactions was significant (F < 1 for all cases).
Dependence of Memory Retrievals A number of studies have indicated that the time to retrieve information from a semantic category is decreased if that category has been accessed a short time previously. Collins and Quillian (1970), for example, have shown that the time required to answer such questions as “Is a canary a bird?” is decreased by as much as 600 msec if information about canaries has been accessed on the previous trial. Using a somewhat different paradigm, Meyer and Schvaneveldt (Meyer & Schvaneveldt, 1971; Meyer, Schvaneveldt, & Ruddy, 1972', Schvaneveldt & Meyer, 1973; Meyer, 1973) have shown the same thing. In these experiments, Ss were required to classify letter strings as words or nonwords. The general finding was that the reaction time to classify a letter string as a word is faster if the S has just classified 3 semantically similar word as opposed to a semantically dissimilar word. Thus, for example, the time it takes to classify “butter” as a word is faster if “butter” is preceded by “bread” than if it is preceded by “nurse.”
Two general classes of models have been proposed to handle such results. A location shifting model (Meyer & Schvaneveldt, 1971) assumes that when a S has finished processing a member of a particular category An activation model, on the other hand, assumes that when items in a category are processed, other items are “excited” or “activated” to the extent that they are semantically similar to the information being processed. Two further assumptions are made: first (Warren, 1970) that activation decays away over time and second that activated items are more readily accessible than nonactivated items.
-- A location shifting model (1971) assumes that when a S has finished processing a member of a particular category and must then shift to begin processing a second category, the shift time is dependent upon the semantic distance between the two categories.
1.90 : * Letter - Category : * Category - Letter 1.60 1.50
0 2 Initial LAG
Fig. 2. Mean reaction time in seconds as a function of the number of intervening items (lag) between two appearances of a critical category. Items were presented with a 2.5-sec interval between the category and the letter.
simult.
RT
2.20 : * Letter - Category : * Category - Letter 1.90 1.80
0 2 Initial LAG
Fig. 3. Mean reaction time in seconds as a function of the number of intervening items (lag) between two appearances of a critical category. The category and letter were presented simultaneously.
The results of the present experiment together with the data of Meyer et al (1972) and Loftus (1973) disconfirm the location shifting model and support the activation model. All of these experiments involve the following sorts of comparisons. Let T represent target information whose time to be processed is the dependent variable of interest. Let R represent information which is semantically related to T, and finally let U1 and U2 represent information which is semantically unrelated to T. Now consider three conditions:
Condition a: Process U1 ; Process U1 ; Process T. Condition b: Process R ; Process U2 ; Process T. Condition c: Process U1 ; Process R ; Process T.
The data show that T is processed fastest in Condition c, next fastest in Condition b, and slowest in Condition a. Both the location shifting model and the activation model correctly predict that reaction time in Condition c would be faster than reaction time in Conditions a and b. However, the predictions of the two models differ with regard to the relationship between Conditions a and b. A location shifting model incorrectly predicts that reaction time would be the same for Conditions a and b, since in both cases the S is shifting from the unrelated category, U2 to T. An activation model, on the other hand, correctly predicts the obtained pattern of results. This is because in Condition b, T is assumed to have been activated by R, and this activation has not decayed by the time T is processed. In Condition a, on the Other hand, T is not assumed to have been activated at all; therefore, time to process T would be longer.
Processing Stages In the outset of this report, it was noted that the semantic retrieval model proposed by Freedman and Loftus (1971) postulates two major processing stages: entering a category (which takes time t1) and searching the category (which takes time t2).
Another stage, taking time k, is a baseline stage, involving response execution, etc. Unfortunately, these stages are not sufficient to handle the data from the present experiment. To see why this is so, consider the reaction times to initially access a category. These reaction times fall into a 2 by 2 design with order (category-letter vs letter-category) and interval (2.5 sec vs simultaneous) as factors. According to the Freedman-Loftus model, the processing times involved in initial access should be as follows:
Thus reaction times for Conditions 2-4 should be equal to each other and should differ (by t1) from the reaction time to Condition 1. However, the data indicate that all four reaction times differ from one another, thereby necessitating the postulation of additional processing stages. First, in Condition 4, the predisposition to encode the category before the letter may conflict with normal left-to-right reading habits. Thus, an additional eye fixation could sometimes occur in Condition4 relative to the other three conditions. We shall label the time for this additional eye fixation t3. Secondly, when category and letter are presented simultaneously (Conditions 3 and 4), reaction time must include the time to encode both stimuli. With a 2.5-sec interval, on the other hand (Conditions 1 and 2), reaction time includes the time to encode only one of the two stimuli. Let the extra encoding time required in Conditions 3 and 4 be designated by t4.
We are now in a position to include the two new stages in the four initial reaction times.
By appropriate manipulations of Eqs. 1a-4a, we find that t1 = 0.27 sec (RT2 - RTI); (t2 + k) = 1.69 sec (RT1), t3 = 0.13 sec (RT4 - RT3); and t4 = 0.22 sec (RT3 - RT2 ). The estimate of 0.27 sec for t1 (category entry time) coincides well with previous estimates obtained by Freedman and Loftus (1971) and Loftus and Freedman (1972). The estimate of 0.22 sec for t4 (encoding time) is far greater than one would expect if “encoding” meant only the process of pattern-recognizing the visual stimulus (cf. Sperling, 1963, who estimated 10msec per item for the pattern-recognition process). Thus the obtained estimate of 0.22 sec must include a great deal more processing, although it is impossible in the present experiment to determine what such encoding might consist of. Finally, since an eye fixation usually lasts on the order of 200-300 msec, the estimate of 0.13 sec for t3 (extra fixation time) is somewhat less than one would expect. A possible reason for this discrepancy is that additional eye fixations may not be made on all of the Condition 4 trials. The notion of an extra eye fixation sometimes occurring in Condition 4 is, of course, easily testable.
One more parenthetical remark should be made. As noted above, the interaction of interval time and category-letter order was not significant. If the null hypothesis of no interaction is accepted, then inspection of Eqs. 1a-4a indicates that t1 = t3. (This can be seen either by the fact that RT3 - RT1 = RT4 - RT2 or by the fact that RT2 - RT1 = RT4 - RT3, both of which are true under the null hypothesis.) However, since nothing in the present experiment necessarily warrants acceptance of the null hypothesis, the equality of t1 and t3 should not be taken very seriously.
What Stage Does Activation Affect? Using the logic outlined above, it is possible to obtain estimates of t1, (t2 + k), t3, and t4 for second category presentations at Lags 0 and 2. These estimates, along with the estimates given above for initial presentation, are shown in Table 1. The statistical analyses of the data indicate that the only parameter which reliably changes over lag condition is t2 + k. If we make the reasonable assumption that k remains constant over lag conditions, then t2, the category search time, constitutes the locus of the activation effect. This finding agrees with the conclusion of Meyer (1973, p.30), who noted that “The semantic distance between categories...may affect the search rate for the second category.”
The invariance of encoding time (t4) over lag condition is somewhat at odds with the finding of Meyer et al (1972, Experiment 3) that encoding time appears to be shortened by prior processing of semantically similar information. The reason for this discrepancy is not entirely clear. A possible explanation may lie in the fact that the processing delay between the two categories was much shorter in the Meyer et al experiment than in the present experiment, and the activation decay function for encoding time may be different from the analogous decay function for search rate.
THE INFLUENCE OF ONE MEMORY RETRIEVAL 47l
-- A possible explanation may lie in the fact that the processing delay between the two categories was shorter than the present experiment,and the activation decay function for encoding time may be different from the analogous decay function for search rate
Battig, W. F., & Montague, W. E. Category norms for verbal items in 56 categories: A replication and extension of the Connecticut category norms. Journal of Experimental Psychology Monograph, 1969. 80(3, Pt.2).
Collins, A. M., & Quillian, M. R. Facilitating retrieval from semantic memory: The effect of repeating part of an inference. In A. F. Sanders (Ed.), Attention and performance III. Amsterdam: North-Holland, 1970.
Freedman, J. L., & Loftus, E. F. Retrieval of words from long-term memory. Journal of Verbal Learning & Verbal Behavior, 1971, 10, 107-115.
Loftus, E. F. Activation of semantic memory. American Journal of Psychology. 1974. in press.
Loftus, E. F., & Freedman, J. L. Effect of category-name frequency on the speed of naming an instance of the category. Journal of Verbal Learning & Verbal Behavior, 1972, 11, 343-347.
Meyer, D. E. Correlated operations in searching stored semantic categories. Journal of Experimental Psychology, 1973, 99, 124-133.
Meyer, D. E., & Schvaneveldt, R. W. Facilitation in recognizing pairs of words: Evidence of a dependence between retrieval operations. Journal of Experimental Psychology, 1971, 90, 227-234.
Meyer, D. E., Schvaneveldt, R. W., & Ruddy, M. G. Activation of lexical memory. Paper presented at the meeting of the Psychonomic Society. St. Louis, November 1972.
Schvaneveldt, R. W., & Meyer, D. E. Retrieval and comparison processes in semantic memory. In S. Kornblum (Ed.), Attention and performance IV. New York: Academic Press, 1973
Shapiro, S. I., & Palermo, D. S. Conceptual organization and class membership: Normative data for representatives of 100 categories. Psychonomic Monograph Supplements, 1970, 3(11, Whole No. 43).
Sperling, G. A model for visual memory tasks. Human Factors, 1963, 5, 19-31.
Warren, R. E. Stimulus encoding and memory. Unpublished doctoral dissertation, University of Oregon, 1970.
(Received for publication September 17, 1973; revision accepted December 6, 1973.)
>>732 If “dominance" is defined as the frequency with which a word is given as an exemplar of a category, then one of the two category-letter stimuli will be referred to as more dominant than the other.
>>741 By appropriate manipulations of Eqs 1a-4a, we find that t1 = 0.27 sec (RT2 - RTI); (t2 + k) = 1.69 sec (RT1), t3 = 0.13 sec (RT4 - RT3); and t4 = 0.22 sec (RT3 - RT2 ).
If the null hypothesis of no interaction is accepted, then inspection of Eqs 1a-4a indicates that t, = t3.
nature.com/articles/s41598-020-58831-9 Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript. : * Article * Open Access * Published: 25 February 2020
Memristive synapses connect brain and silicon spiking neurons
* Alexantrou Serb1, * Andrea Corna2, * Richard George3, * Ali Khiat1, * Federico Rocchi2, * Marco Reato2, * Marta Maschietto2, * Christian Mayr3, * Giacomo Indiveri ORCID: orcid.org/0000-0002-7109-16894, * Stefano Vassanelli ORCID: orcid.org/0000-0003-0389-80232 & * Themistoklis Prodromakis ORCID: orcid.org/0000-0002-6267-69091
Memristors The memristive synapse set-up consisted of an array of memristive devices positioned inside an ArC memristor characterisation and testing instrument33 (Supplementary Fig.5. http:www.arc-instruments.co.uk). The instrument is controlled by a PC, which handles all the communications over UDP; all through a python-based user interface. The software is configured to react to UDP packets carrying information about the firing of either artificial or biological neurons (who fired when). Once a packet is received, the ID of the neuron that emitted it and the time of spiking are both retrieved from the packet payload and the neural connectivity matrix is consulted in order to determine which neurons are pre- and which are post-synaptic to the firing cell. Then, if the plasticity conditions are met, the ArC instrument applies programming pulses that cause the memristive synapses to change their resistive states. Importantly, the set-up can control whether LTP- or LTD-type plasticity is to be applied in each case, but once the pulses have been applied it is the device responses that determine the magnitude of the plasticity. Notably, resistivity transitions of the device are non-volatile, they hold over at least hours27 as also exemplified in our prototype experiment and are therefore fully compatible with typical LTP and LTD time scales of natural synapses. The system is sustained by a specific methodology for handling timing within the overall network (Zurich, Southampton, Padova). The set-up in Southampton being the node that links Zurich and Padova together, controls the overall handling of time.
-- Once a packet is received, the ID of the neuron that emitted it and the time of spiking are retrieved from the neural connectivity matrix (held at the Southampton set-up) is consulted the ID of the neuron that emitted it and the time of spiking are both retrieved from the packet payload and the neural connectivity matrix (held at the Southampton set-up) is consulted
Under this system, one of the partners (in our case Zurich) is labelled as the “primary partner” and all timing information arriving from that partner is treated as a ground truth. Every timing information sent by other partners then has to be related to this ground truth, for example if the primary partner says that neuron 12 fires a spike at time 305, then the secondary partner(s) is informed of this (through Southampton). If then a neuron in the secondary partner set-up fires 5 time units (as measured by a wall-clock) after being informed of the firing of neuron 12, it emits a packet informing Southampton that e.g. neuron 55 fired at time 310. This way the relative timing between spikes arriving from the primary partner and the spikes triggered by the secondary partner(s) in response is maintained despite any network delays. The price is that if the secondary partners wish to communicate spikes to the primary partner, network delays for the entire round-trip are then burdening the secondary-to-primary pathway. The details of timing control at each partner site are fairly complicated and constrained by the set-ups at each partner, but all timing information is eventually encoded in an “absolute time” record held at Southampton. T
796 名前：he rationale behind this design decision was to ensure that at least in the pathway from primary to secondary partner(s) timing control is sufficiently tight to sustain plasticity in the face of network delays. Neuronal culture and electrophysiology Embryonic (E18) rat hippocampal neurons were plated and cultured on the CMEA according to procedures described in detail in34. Recordings were performed on 812 DIV neurons. The experimental setup in UNIPD(Supplementary Fig.1)enabled UDP-triggered capacitive stimulation of neurons13 while simultaneously recording and communicating via UDP the occurrence of depolarisations that were measured by patch-clamp whole-cell recording []
The CMEA (20 × 20 independent TiO2 capacitors, each one of area 50 × 50 μm2) was controlled by a dedicated stimulation board and all the connections to partners, Southampton and Zurich, were managed by a PC running a LabVIEW-based software (National Instruments Corp, Austin, TX, USA). The stimulation protocol was derived from13 and further optimized for non-invasive adjustable stimulation of the neurons. In brief, capacitive stimulation was adjusted to the memristor’s resistance (i.e. the synaptor weight) by varying the repetition number of appropriate stimulation waveforms (Supplementary Fig.1). Patch-Clamp recordings were performed in whole-cell current-clamp configuration using an Axopatch 200B amplifier ( USA) connected to the PC through a BNC-2110 Shielded Connector Block ( TX, USA) along with a PCI-6259 PCI Card ( TX, USA). WinWCP (Strathclyde Electrophysiology Software, University of Strathclyde, Glasgow, UK) was used for data acquisition. Micropipettes were pulled from borosilicate glass capillaries (GB150T-10, Science Products GmbH, Hofheim, Germany) using a P-97 Flaming/Brown Micropipette Puller (Sutter Instruments Corp., Novato, CA, USA). Intracellular pipette solution and extracellular solution used during the experiments were respectively (in mM): 6.0 KCl, 120 K gluconate, 10 HEPES, 3.0 EGTA, 5 MgATP, 20 Sucrose (K); 135.0 NaCl, 5.4 KCl, 1.0 MgCl2, 1.8 CaCl2, 10.0 Glucose, 5.0 HEPES (N). Digitised recordings were analysed by a custom LabVIEW software running on the PC, allowing detection and discrimination of firing and EPSP activity through a thresholding approach. All experiments were performed in accordance with the Italian and European legislation for the use of animals for scientific purposes and protocols approved by the ethical committee of the University of Padova and by the Italian Ministry of Health (authorisation number 522/2018-PR).
-- Molecular Devices, USA National Instruments Corp, Austin, TX, USA adjusted to pH 7.3 with 1N KOH
1. O’Doherty, J. E. et al. Active tactile exploration using a brain-machine-brain interface. Nature 479, 228-231 (2011). * ADS * Article * Google Scholar 2. Hampson, R. E. et al. Developing a hippocampal neural prosthetic to facilitate human memory encoding and recall. J. Neural Eng. 15, 036014 (2018). * ADS * Article * Google Scholar 3. Thakor, N. V. Translating the Brain-Machine Interface. Sci. Transl. Med. 5, 210ps17-210ps17 (2013). * Article * Google Scholar 4. Mead, C. Neuromorphic electronic systems. Proc. IEEE 78, 1629-1636 (1990). * Article * Google Scholar 5. Vassanelli, S. & Mahmud, M. Trends and Challenges in Neuroengineering: Toward “Intelligent” Neuroprostheses through Brain-“Brain Inspired Systems” Communication. Front. Neurosci. 10 (2016). 6. Boi, F. et al. A Bidirectional Brain-Machine Interface Featuring a Neuromorphic Hardware Decoder. Front. Neurosci. 10 (2016).
>>748 Recordings were performed on 8-12 DIV neurons.
7. Wei, S. L. et al. Emulating long-term synaptic dynamics with memristive devices. ArXiV. 1509, 01998 (2015). * Google Scholar 8. Berdan, R. et al. Emulating short-term synaptic dynamics with memristive devices. Scientific reports. 6 (2015). 9. Burr, G. W. et al. Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element. IEEE Trans. Electron Devices 62, 34983507 (2015). * ADS * Article * Google Scholar 10. Yang, J. J., Strukov, D. B. & Stewart, D. R. Memristive devices for computing. Nat. Nanotechnol. 8, 13-24 (2013). * ADS * CAS * Article * Google Scholar 11. Gupta, I. et al. Real-time encoding and compression of neuronal spikes by metal-oxide memristors. Nat. Commun. 7, 12805 (2016). * ADS * CAS * Article * Google Scholar 12. Birmingham, K. et al. Bioelectronic medicines: a research roadmap. Nat. Rev. Drug Discov. 13, 399-400 (2014). * CAS * Article * Google Scholar
>>749 6.0 KCl, 120 K gluconate, 10 HEPES, 3.0 EGTA, 5 MgATP, 20 Sucrose (adjusted to pH 7.3 with 1N KOH); 135.0 NaCl, 5.4 KCl, 1.0 MgCl2, 1.8 CaCl2, 10.0 Glucose, 5.0 HEPES (adjusted to pH 7.4 with 1N NaOH).
13. Schoen, I. & Fromherz, P. Extracellular Stimulation of Mammalian Neurons Through Repetitive Activation of Na+ Channels by Weak Capacitive Currents on a Silicon Chip. J. Neurophysiol. 100, 346-357 (2008). * Article * Google Scholar 14. George, R., Mayr, C., Indiveri, G. & Vassanelli, S. Event-based softcore processor in a biohybrid setup applied to structural plasticity. In 2015 International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP) 1-4, https://doi.org/10.1109/EBCCSP.2015.7300664 (IEEE, 2015). 15. Rast, A. D. et al. A location-independent direct link neuromorphic interface. In The 2013 International Joint Conference on Neural Networks (IJCNN) 1-8, https://doi.org/10.1109/IJCNN.2013.6706887 (IEEE, 2013). 16. Keren, H., Partzsch, J., Marom, S. & Mayr, C. G. A Biohybrid Setup for Coupling Biological and Neuromorphic Neural Networks. Front. Neurosci. 13 (2019). 17. Dudek, S. M. & Bear, M. F. Homosynaptic long-term depression in area CA1 of hippocampus and effects of N-methyl-D-aspartate receptor blockade. Proc. Natl. Acad. Sci. USA 89, 4363-4367 (1992). * ADS * CAS * Article * Google Scholar 18. Cooper, L. N. & Bear, M. F. The BCM theory of synapse modification at 30: interaction of theory with experiment. Nat. Rev. Neurosci. 13, 798-810 (2012). * CAS * Article * Google Scholar
19. Vassanelli, S., Mahmud, M., Girardi, S. & Maschietto, M. On the Way to Large-Scale and High-Resolution Brain-Chip Interfacing. Cogn. Comput. 4, 71-81 (2012). * Article * Google Scholar 20. Giacomello, M. et al. Stimulation of Ca2+ signals in neurons by electrically coupled electrolyte-oxide-semiconductor capacitors. J. Neurosci. Methods 198, 1-7 (2011). * CAS * Article * Google Scholar 21. Spira, M. E. & Hai, A. Multi-electrode array technologies for neuroscience and cardiology. Nat. Nanotechnol. 8, 83 (2013). * ADS * CAS * Article * Google Scholar 22. Alivisatos, A. P. et al. Nanotools for Neuroscience and Brain Activity Mapping. ACS Nano 7, 1850-1866 (2013). * CAS * Article * Google Scholar 23. Angle, M. R., Cui, B. & Melosh, N. A. Nanotechnology and neurophysiology. Curr. Opin. Neurobiol. 32, 132-140 (2015). * CAS * Article * Google Scholar 24. Duan, X. & Lieber, C. M. Nanoscience and the nano-bioelectronics frontier. Nano Res. 8, 1-22 (2015). * Article * Google Scholar
25. Brivio, S. et al. Experimental study of gradual/abrupt dynamics of HfO2-based memristive devices. Appl. Phys. Lett. 109, 133504 (2016). * ADS * Article * Google Scholar 26. Serrano-Gotarredona, T., Masquelier, T., Prodromakis, T., Indiveri, G. & Linares-Barranco, B. STDP and STDP variations with memristors for spiking neuromorphic learning systems. Front. Neurosci. 7 (2013). 27. Serb, A. et al. Unsupervised learning in probabilistic neural networks with multi-state metal-oxide memristive synapses. Nat. Commun. 7 (2016). 28. Qiao, N. et al. A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses. Front. Neurosci. 9, 141 (2015). * Article * Google Scholar 29. Boegerhausen, M., Suter, P. & Liu, S.-C. Modeling Short-Term Synaptic Depression in Silicon. Neural Comput. 15, 331-348 (2003). * Article * Google Scholar 30. Mitra, S., Fusi, S. & Indiveri, G. Real-Time Classification of Complex Patterns Using Spike-Based Learning in Neuromorphic VLSI. IEEE Trans. Biomed. Circuits Syst. 3, 32-42 (2009). * CAS * Article * Google Scholar
31. Livi, P. & Indiveri, G. A current-mode conductance-based silicon neuron for address-event neuromorphic systems. In 2009 IEEE International Symposium on Circuits and Systems 2898-2901 doi.org/10.1109/ISCAS.2009.5118408 (IEEE, 2009). 32. Deiss, S., Douglas, R. & Whatley, A. A pulse-coded communications infrastructure for neuromorphic systems. Pulsed Neural Netw. 157-178 (1999). 33. Berdan, R. et al. A u-Controller-Based System for Interfacing Selectorless RRAM Crossbar Arrays. IEEE Trans. Electron Devices 62, 2190-2196 (2015). * ADS * CAS * Article * Google Scholar 34. Antonucci, D. E., Lim, S. T., Vassanelli, S. & Trimmer, J. S. Dynamic localization and clustering of dendritic Kv2.1 voltage-dependent potassium channels in developing hippocampal neurons. Neuroscience 108, 69-81 (2001). * CAS * Article * Google Scholar 35. Indiveri, G. et al. Neuromorphic silicon neuron circuits. Front. Neurosci. 5, 73 (2011). * PubMed * PubMed Central * Google Scholar 36. Stathopoulos, S. et al. Multibit memory operation of metal-oxide bi-layer memristors. Sci. Rep. 7 (2017).
Download references Author information Affiliations
1. Centre for Electronics Frontiers, University of Southampton, Southampton, SO17 1BJ, UK * Alexantrou Serb * , Ali Khiat * & Themistoklis Prodromakis 2. Biomedical Sciences and Padua Neuroscience Center, University of Padova, Padova, 35131, Italy * Andrea Corna * , Federico Rocchi * , Marco Reato * , Marta Maschietto * & Stefano Vassanelli 3. Institute of Circuits and Systems, TU Dresden, Dresden, 01062, Germany * Richard George * & Christian Mayr 4. Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, 8057, Switzerland * Giacomo Indiveri
Contributions The experiments were jointly conceived by T.P., S.V. and G.I., who share senior authorship. The experiments were jointly designed and ran by A.S., A.C., R.G., who are acknowledged as shared first authors. A.K. manufactured the memristive devices. FR and MR assisted with the biological system set-up and operation. MM cultured neurons on chips. C.M. provided valuable feedback and guidance during the write-up of the paper. The paper was jointly written by all co-authors.
Corresponding authors Correspondence to Stefano Vassanelli or Themistoklis Prodromakis.
Ethics declarations
Competing interests The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format,as long as you give appropriate credit to the original author(s) and the source,provide a link to the Creative Commons license,and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit creativecommons.org/licenses/by/4.0/ .
Reprints and Permissions
About this article
Verify currency and authenticity via CrossMark
Cite this article Serb, A., Corna, A., George, R. et al. Memristive synapses connect brain and silicon spiking neurons. Sci Rep 10, 2590 (2020). https://doi.org/10.1038/s41598-020-58831-9 Download citation
Provided by the Springer Nature SharedIt content-sharing initiative
-- This article is licensed under a Creative Commons Attribution 4.0 License, which permits use, sharing and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) , and indicate if changes .
Mostafa Dehghani* † Stephan Gouws* Oriol Vinyals University of Amsterdam DeepMind DeepMind dehghani@uva.nl sgouws@google.com vinyals@google.com
Jakob Uszkoreit ukasz Kaiser Google Brain Google Brain usz@google.com lukaszkaiser@google.com
D.4. LEARNING TO EXECUTE (LTE). LTE is a set of tasks indicating the ability of a model to learn to execute computer programs and was proposed by Zaremba & Sutskever (2015). These tasks include two subsets: 1) program evaluation tasks (program, control, and addition) that are designed to assess the ability of models for understanding numerical operations, if-statements, variable assignments, the compositionality of operations, and more, as well as 2) memorization tasks (copy, double, and reverse).
The difficulty of the program evaluation tasks is parameterized by their length and nesting. The length parameter is the number of digits in the integers that appear in the programs (so the integers are chosen uniformly from [1, length]), and the nesting parameter is the number of times we are allowed to combine the operations with each other. Higher values of nesting yield programs with deeper parse trees. For instance, here is a program that is generated with length = 4 and nesting = 3. Input: j=8584 for x in range(8): j+=920 b=(1500+j) print((b+7567)) Target: 25011 1) program evaluation tasks (A) that are designed to assess the ability of models for understanding numerical operations, if-statements, variable assignments, the compositionality of operations, and more, as well as 2) memorization tasks (B).
CellVM: A Homogeneous Virtual Machine Runtime System for a Heterogeneous Single-Chip Multiprocessor
Albert Noll ETH Zurich albert.nollaTinf.ethz ch Andreas Gal University of California, Irvine galATuci edu Michael Franz University of California, Irvine franzATuci edu
The assign and method benchmark, on the other hand, include CellVM’s worst case scenario: synchronized methods and data structures.
Figure 4. Performance evaluation of low-level VM operations. Values are normalized to JamVM running on the PPE
