A three-layered model of primate prefrontal cortex encodes identity and abstract categorical structure of behavioral sequences
Highlights
► Prefrontal activity simulation related to sequence discrimination and categorization. ► Provides a functional explanation for primate neurophysiology results. ► We propose an anatomically inspired three layered recurrent network model of cortex. ► Aspects of corticostriatal system can be modeled in reservoir computing framework.
Introduction
In 1989 the first report of prefrontal neuronal activity encoding the sequential properties of sensorimotor sequences was reported by Barone and Joseph (1989). They demonstrated that after the macaque had been shown a spatial sequence of three illuminated push-buttons, and was preparing to reproduce the sequence, neurons in the prefrontal cortex encoded both the spatial location of the targets, and most strikingly, their order or rank in the sequence. Thus, the prefrontal neural population was considered to encode the entire action plan, necessary for the animal to reproduce the visualized sequence. Since then a number of studies have been performed, looking more closely at the sequence specific encoding of visuo-spatial behavioral components of sensorimotor sequences in the prefrontal cortex e.g. (Funahashi et al., 1997, Clower and Alexander, 1998).
Recently, Shima and colleagues (2007) have considered that when the subject is required to encode a large number of sensorimotor sequences, the increasing memory load would potentially require the animal to adopt a categorization strategy, so the similar sequences would be encoded as members of the same category. They employed a sequence reproduction task, where sequences were made up of the motor actions push, pull and turn, using a manipulable lever. The animal was required to be able to observe and reproduce 11 sequences from three categories: “paired” (AABB), “alternate” (ABAB) and “4-repeat” (AAAA), where A and B were each systematically replaced by one of the three elements push, pull and turn. Four sequences were of the category AABB (e.g. Turn, Turn, Push, Push), 4 of the category ABAB and 3 of the category AAAA. In the experiment, the animal was presented one of these sequences under visual guidance five times, and then was required to reproduce the sequence from memory. This was repeated for each of the 11 sequences. Single unit recordings were performed in the lateral prefrontal cortex during the delay preceding the first reproduction of the sequence from memory.
Shima et al. (2007) found a population of neurons that had the traditional coding of spatio-temporal characteristics of individual sequences. Most interestingly, they found a second population of neurons that were active for one of the three categories, without distinguishing the individual sequences within that category (illustrated in Fig. 1). They observed that this categorical information was not directly present in the visual input to the animal. Instead, the brain had somehow extracted the appropriate regularities in order to recognize the repetitive structure that defined the three categories of sequences.
They then raised the crucial question: what are the underlying neuronal processes that would allow these PFC neurons to extract the required properties to form these categorical representations? They propose that the answer to this question will likely be found through the development of neural network models of the PFC applied to this problem (Tanji et al., 2007).
We have previously developed a neural network model of prefrontal cortex (Dominey et al., 1995, Dominey, 1995) with the objective of explaining the primate single unit recordings that reflected the identity and sequential order of sequence elements as initially observed by Barone and Joseph (1989). The principal characteristics of this model were (1) PFC was modeled as two 2-D layers of leaky integrator sigmoidal average firing rate neurons, that were connected by recurrent connections with random weights over the interval [−0.45; 0.55]. This mixture of excitatory and inhibitory connections yielded a dynamical system that was sensitive both to the spatial location of targets on the input array, and the previous sequential context. (2) Importantly, the recurrent connections were not subject to modification by learning, instead, learning was used to link these dynamic patterns of activity with the required output states. The learning was simulated as a form of reinforcement learning, based on reward-related dopamine in the striatum. This was the first simulation of reward-related dopaminergic synaptic modification in the cortico-striatal system, and the first expression of the concept that came to be known as the echo state network (Jaeger, 2003), and the subsequent liquid state machine (Maas et al., 2002), which are now considered under the framework of reservoir computing (Verstraeten et al., 2007). An interesting historical overview is provided by Lukosevicius and Jaeger (2009).
Indeed, this dynamical system was thus able to reproduce the behavior and neurophysiology (Dominey et al., 1995) reported in Barone and Joseph (1989). It also displayed a rich capability to encode and reproduce the serial order and temporal structure of behavioral sequences in a variety of conditions (Dominey, 1998a, Dominey, 1998b, Dominey and Ramus, 2000). When exposed to sequences with abstract structure such as ABCBAC the model failed to characterize this repetitive structure, leading us to propose an additional working memory mechanism which allowed such sequences to be recoded as an abstract structure (Dominey et al., 1998). This model would encode sequences like AABB as {u, n-1, u, n-1} and sequences like ABAB as {u, u, n-2, n-2} (“u” signifies unpredictable, and “n-2” indicates a repetition of the element 2 places behind, etc.), and would immediately generalize to new sequences that conform to the learned rules. In the current research our goal is to re-examine the capabilities of a recurrent network, alone without working memory, to perform categorization as observed by Shima et al. (2007).
Section snippets
Material and methods
Primate cortex is characterized by its laminar structure, and the connectivity patterns within and between layers (Douglas and Martin, 2004). The level of detail in modeling the laminar organization is an important issue. Here we will consider a partition composed of three major layers: supragranular, granular and infragranular. This allows us to implement a more realistic pattern of divergent connectivity between layers than in our less structured two-layered model (Dominey et al., 1995,
Experiment 2: extracting sequential and categorical information from LPFC model
We showed that the model is able to produce patterns of activity that are different for each sequence and category – thus demonstrating its potential capacity to discriminate and categorize. Now we hypothesize that such a model is able to effectively learn to recognize these sequences and categories. In this section we test this hypothesis using supervised learning with a simple associative method.
Discussion
Neocortex provides an adaptive capability for controlling the “reptilian” basal ganglia and brainstem structures. Cortex cannot anticipate all possible configurations of the perceptual world that can face the organism. Instead, it must embody general coding strategies that can in principle become sensitive to any significant regularities that can arise in the perceptual surfaces. In this context it has been demonstrated exhaustively that recurrent networks of dynamic neuron-like computing
Acknowledgments
This work is supported by the FP7 Grant 231267 Organic. We want to thank the Reservoir Lab team of Gent University (Belgium) for its help with the Oger Toolbox (2011), especially David Verstraeten.
References (21)
A shared system for learning serial and temporal structure of sensorimotor sequences? Evidence from simulation and human experiments
Cognitive Brain Research
(1998)- et al.
Delay-period activity in the primate prefrontal cortex encoding multiple spatial positions and their order of presentation
Behavioral Brain Research
(1997) - et al.
Reservoir computing approaches to recurrent neural network training
Computer Science Review
(2009) - et al.
Concept-based behavioral planning and the lateral prefrontal cortex
Trends in Cognitive Sciences
(2007) - et al.
An experimental unification of reservoir computing methods
Neural Networks
(2007) - et al.
Prefrontal cortex and spatial sequencing in the macaque monkey
Experimental Brain Research
(1989) - et al.
Movement sequence-related activity reflecting numerical order of components in supplementary and presupplementary motor areas
Journal of Neurophysiology
(1998) - et al.
Cell-cycle control and cortical development
Nature Reviews Neuroscience
(2007) Complex sensory-motor sequence learning based on recurrent state-representation and reinforcement learning
Biological Cybernetics
(1995)Influences of temporal organization on transfer in sequence learning: comments on Stadler (1995) and Curran and Keele (1993). J. Exp. Psychology: Learning
Memory and Cognition
(1998)
Cited by (10)
Implementing associative memories by Echo State Network for the applications of natural language processing
2023, Machine Learning with ApplicationsCategory learning in a recurrent neural network with reinforcement learning
2022, Frontiers in PsychiatryUsing conceptors to manage neural long-term memories for temporal patterns
2017, Journal of Machine Learning Research