ReviewInternal models for motor control and trajectory planning
Introduction
Internal models are neural mechanisms that can mimic the input/output characteristics, or their inverses, of the motor apparatus. Forward internal models can predict sensory consequences from efference copies of issued motor commands. Inverse internal models, on the other hand, can calculate necessary feedforward motor commands from desired trajectory information.
Fast and coordinated arm movements cannot be executed solely under feedback control, since biological feedback loops are slow and have small gains (Figure 1a). Thus, the internal model hypothesis (Figure 1b) proposes that the brain needs to acquire an inverse dynamics model of the object to be controlled through motor learning, after which motor control can be executed in a pure feedforward manner. In theory, a forward model of the motor apparatus embedded in an internal feedback loop can approximate an inverse model.
The internal model concept has its origin in control theory and robotics, but Ito [1] proposed almost 30 years ago that the cerebellum contains forward models of the limbs and other brain regions. More recently, internal models have attracted a broader range of specialists (e.g. neural network modelers, connectionists and neurophysiologists 2, 3, 4), and have been studied increasingly seriously as one of the major theories of motor control and learning in neuroscience and cognitive science. Accordingly, in the past few years, much more direct and convincing data than ever before have been accumulated. Such data can already show the existence, structures, learning, functions and anatomy of internal models. Of particular importance, we have seen significant theoretical advances in elucidation of the generalization, multiplicity and switching of internal models, and their possible use in trajectory planning.
In this review, I will discuss data supporting the existence of internal models. It has been shown that the behavioral paradigms in use are diverse and include adaptation to force fields, posture control, grip-force–load-force coupling, oculomanual coordination, and the vestibular system. An explanation will be given on points of controversy between the equilibrium point control hypothesis and the internal model hypothesis, and some clues towards their resolution will be presented. Recent neurophysiological and imaging studies that suggest that the cerebellar cortex is a major site of internal models will also be discussed. Furthermore, structures of internal models will be explored by ‘generalization’ experiments; modularity and multiplicity are suggested by the data obtained. Finally, two major approaches to trajectory planning will be reviewed and a new theory will be introduced to integrate them.
Section snippets
Existence of internal models
When subjects first undertake point-to-point arm reaching movements under force fields which effectively change dynamic characteristics of the arm, their hand trajectories are distorted compared with the normal, roughly straight paths; also, the end point errors are large, especially in the direction of the applied force. The force fields generated predetermined forces which depended on the state space point (position, velocity), and were produced by a robot manipulandum [5] or by a rotating
Internal models in the cerebellum
It is conceivable that internal models are located in all brain regions having synaptic plasticity, provided that they receive and send out relevant information for their input and output. We have good reason to believe that at least some internal models are acquired and stored in the cerebellar cortex. For example, there is a new computational theory [27••] that allocates supervised learning, reinforcement learning, and unsupervised learning to the cerebellum, the basal ganglia and the
Structures of internal models
The functional structures of internal models can be probed by the so-called ‘generalization experiment’ 49, 50, 51, 52, 53•, 54, 55•. Humans or animals are trained for a specific set of movement trajectories with an altered kinematic or dynamic perturbation. After sufficient learning, the organism’s ability to cope with different trajectories or movements in a different part of the workspace in which the motor apparatus can move is examined. If ‘generalization’ is considered perfect, new
Trajectory formation
Computational theories on how arm reaching trajectories are planned have been a central issue in motor control since it was shown that they involve roughly straight hand paths and bell-shaped velocity profiles [64]. Many of the different computational models can be classified into two types: kinematic models such as the minimum jerk model [65], and dynamic models such as the minimum torque-change model [66]. Because these two classes of models enable the experimental testing of qualitatively
Conclusions
The concepts concerning internal models have now been well supported by behavioral studies in the field of sensory motor control. Neurophysiological studies have just begun but should be fruitful in the next five years. Theoretically, the concept should be extended from pure sensory motor control to cognitive domains as we have a flood of data suggesting cerebellar involvement in higher cognitive functions such as language 78, 79, 80••. This is especially important because the cerebellar
Acknowledgements
I wish to thank my collaborators in ATR and in the ERATO Dynamic Brain Project as well as those outside, especially, Daniel Wolpert, Chris Miall and Randy Flanagan. This research was partially supported by HFSP grants, special coordination funds of promotion of science.
References and recommended reading
Papers of particular interest, published within the annual period of review, have been highlighted as:
• of special interest
•• of outstanding interest
References (80)
- et al.
Forward models: supervised learning with a distal teacher
Cogn Sci
(1992) - et al.
Reconstruction of shifting elbow joint compliant characteristics during fast and slow movements
Neuroscience
(1991) What are the computations of cerebellum, basal ganglia, and cerebral cortex?
Neural Networks
(1999)A theory of cerebellar function
Math Biosci
(1971)- et al.
The cerebellum and VOR/OKR learning models
Trends Neurosci
(1992) A role for the cerebellum in learning movement coordination
Neurobiol Learn Mem
(1998)- et al.
Visual inputs to cerebellar ventral paraflocculus during ocular following responses
Prog Brain Res
(1996) - et al.
A mathematical model that reproduces vertical ocular following responses from visual stimuli
Neurosci Res
(1997) A role for the cerebellum in the control of limb movement velocity
Curr Opin Neurobiol
(1998)- et al.
Microcircuitry and function of the inferior olive
Trends Neurosci
(1998)
Multiple paired forward and inverse models for motor control
Neural Networks
The cerebellum: an overview
Trends Cog Sci
Neurophysiological aspects of the cerebellar motor control system
Int J Neurol
A hierarchical neural network model for the control and learning of voluntary movements
Biol Cybern
Is the cerebellum a Smith Predictor?
J Motor Behav
Adaptive representation of dynamics during learning of a motor task
J Neurosci
Rapid adaptation to Coriolis force perturbations of arm trajectory
J Neurophysiol
Gravitoinertial force background level affects adaptation to Coriolis force perturbations of reaching movements
J Neurophysiol
Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects
Exp Brain Res
The role of internal models in motion planning and control: evidence from grip force adjustments during movements of hand-held loads
J Neurosci
Activation of the cerebellum in grip force and load force coordination: an fMRI study
Neuroimage
The oculomanual coordination control center takes into account the mechanical properties of the arm
Exp Brain Res
This way up: illusions and internal models in the vestibular system
Nat Neurosci
Oculomotor control of primary eye position discriminates between translation and tilt
J Neurophysiol
Humans use internal models to estimate gravity and linear acceleration
Nature
Functional tuning of nervous system with control of movement or maintenance of a steady posture. 2. Controllable parameters of the muscles
Biophysics
Posture control and trajectory formation during arm movement
J Neurosci
The control of hand equilibrium trajectories in multi-joint arm movements
Biol Cybern
Are complex control signals required for human arm movement?
J Neurophysiol
Virtual trajectory and stiffness ellipse during multijoint arm movement predicted by neural inverse models
Biol Cybern
Equilibrium-point control hypothesis examined by measured arm-stiffness during multi-joint movement
Science
Compensation for interaction torques during single and multi-joint limb movement
J Neurophysiol
Can muscle stiffness alone stabilize upright standing?
J Neurophysiol
Compensation for mechanically unstable loading in voluntary wrist movement
Exp Brain Res
Neural network learning control of robot manipulators using gradually increasing task difficulty
IEEE Trans Robotics Automat
A theory of cerebellar cortex
J Physiol
Temporal firing patterns of Purkinje cells in the cerebellar ventral paraflocculus during ocular following responses in monkeys. II. Complex spikes
J Neurophysiol
Inverse-dynamics model eye movement control by Purkinje cells in the cerebellum
Nature
Temporal firing patterns of Purkinje cells in the cerebellar ventral paraflocculus during ocular following responses in monkeys. I. Simple spikes
J Neurophysiol
Cited by (1958)
Sensory restoration for improved motor control of prostheses
2023, Current Opinion in Biomedical EngineeringThe effects of a single bout of moderate-intensity aerobic exercise on visuomotor adaptation and its savings
2023, Neurobiology of Learning and Memory