Volume 17 Supplement 7

## Selected articles from the 12th Annual Biotechnology and Bioinformatics Symposium: bioinformatics

# Sensorimotor control: computing the immediate future from the delayed present

- Arman Sargolzaei
^{1}, - Mohamed Abdelghani
^{2}Email author, - Kang K. Yen
^{1}and - Saman Sargolzaei
^{3}

**17(Suppl 7)**:245

https://doi.org/10.1186/s12859-016-1098-2

© The Author(s). 2016

**Published: **25 July 2016

## Abstract

### Background

The predictive nature of the primate sensorimotor systems, for example the smooth pursuit system and their ability to compensate for long delays have been proven by many physiological experiments. However, few theoretical models have tried to explain these facts comprehensively. Here, we propose a sensorimotor learning and control model that can be used to (1) predict the dynamics of variable time delays and current and future sensory states from delayed sensory information; (2) learn new sensorimotor realities; and (3) control a motor system in real time.

### Results

This paper proposed a new time-delay estimation method and developed a computational model for a predictive control solution of a sensorimotor control system under time delay. Simulation experiments are used to demonstrate how the proposed model can explain a sensorimotor system’s ability to compensate for delays during online learning and control. To further illustrate the benefits of the proposed time-delay estimation method and predictive control in sensorimotor systems a simulation of the horizontal Vestibulo-Ocular Reflex (hVOR) system is presented.

Without the proposed time-delay estimation and prediction, the hVOR can be unstable and could be affected by high frequency oscillations. These oscillations are reminiscent of a fast correction mechanism, e.g., a saccade to compensate for the hVOR delays. Comparing results of the proposed model with those in literature, it is clear that the hVOR system with impaired time-delay estimation or impaired sensory state predictor can mimic certain outcomes of sensorimotor diseases. Even more, if the control of hVOR is augmented with the proposed time-delay estimator and the predictor for eye position relative to the head, then hVOR control system can be stabilized.

### Conclusions

Three claims with varying degrees of experimental support are proposed in this paper. Firstly, the brain or any sensorimotor system has time-delay estimation circuits for the various sensorimotor control systems. Secondly, the brain continuously estimates current/future sensory states from the previously sensed states. Thirdly, the brain uses predicted sensory states to perform optimal motor control.

### Keywords

Sensorimotor control Time delay estimation Vestibulo-Ocular Reflex (VOR) Predictive control Motor learning## Background

Sensorimotor control system is the most robust and versatile collection of modular, hierarchical and well-organized hybrid of control strategies. With it we can land a plane, drive a car, play sports, and adapt to bizarre sensorimotor transformations [1, 2] (e.g., reversing prisms), or we are allowed to perform in space or in the deep sea. Sensorimotor control system can accomplish all of these tasks in an optimal manner in terms of speed, accuracy and efficiency [3, 4].

The brain’s sensorimotor cortex, as a complex neural sensorimotor control system, inherently finds and implements an optimal decision to a vast range of input conditions. Noises, nonlinearities, delays, uncertainties and redundancies are among the major problems that the sensorimotor control system interacts with [5]. Delays occur in various parts of a sensorimotor control system, ranging from sensory information reception to, information transmission along nerve fibers, computing responses by processing the sensory information, feedback transmission, and finally, motor output in terms of muscle reaction. The delay value is dynamic and varies with sensory modality. For example there is longer delay for vision than proprioception. This is because the complexity of the sensory information processing depends on the task and it is longer for face recognition than motion perception. Efferent control signals are delayed as a result of neural conduction delays and low-pass filtering properties of muscles. Short efferent delays, such as the monosynaptic Stretch Reflex (SR), are in the order of 10–40 milliseconds, depending on the length and the type of nerve fiber. This delay is increased to 30–70 milliseconds for the cortical component of the long latency SR [6]. The electromechanical delay, such as the delay in generating force response in muscles, can take up to 25 milliseconds [7]. So far, the investigation focuses on how a complex neural sensorimotor control system such as the brain’s sensorimotor cortex is capable of performing tasks in the presence of the above-mentioned conditions. However, the study in this paper specifically focuses on developing a mathematical model that will simulate a sensorimotor control system in the presence of noise and delay.

The Vestibulo-Ocular Reflex (VOR), one of the fastest involuntary responses due to the short neural connections and high neural computation speed, is not prone to the occurrence of delay up to 10 milliseconds from stimulus onset [8]. VOR is a motor control system that stabilizes vision during head movements. Smooth pursuit, another efficient visual control system in human beings for target tracking in their visual field, has the ability to process the information with a 80–130 millisecond delay in the brain [9, 10]. Delays make control difficult because information about the current state of the motor system is outdated. A motor control system that does not have delay compensation mechanisms could not correct for errors, leading to potential inefficiencies and instability. The explanation of those controllers such as fuzzy logic control, feedback and adaptive linearization based control, optimal nonlinear trajectory control can be found in the literature [11–13]. Although they have several applications but they should be modified to be used in systems with time delays.

### Delay compensation: facts and current theories

Consider the saccadic eye movement, which is a fast eye movement produced by a visual system that directs the eyes to interesting visual stimuli: the movement duration is shorter than the sensory delay [9]. This means that sensory feedback about the current state of the eye and the visual field cannot be used to correct or guide saccades because the sensory information regarding the movement itself arrives after the completion of the movement. Smooth pursuit eye movements allow a person to track targets in the visual field at a high speed of ~200°/sec, which is a remarkable performance. Recent experiments stated the high performance of the smooth pursuit system, where it was observed that the position of eyes was ahead of visual sensory feedback of the target position. This cannot be achieved by solely implementing standard negative feedback methods based on visual error signals [14–16].

Under such information processing and transmission delays, simple feedback control is affected by significant temporal discrepancies between target signal and current state, suggesting that some form of predictive control must take place to achieve such a high performance [17]. The predictive nature of sensorimotor control systems is explored and demonstrated through multiple experiments. Experiments demonstrated that monkeys have the ability to conduct smooth pursuit movements with zero retinal slip [18, 19] or the ability to maintain smooth pursuit during blink periods (momentary disappearances of the target) [20]. Such predictive compensation was observed both in tracking moving targets with constant velocity or in sinusoidal moving objects. In a hand movement study, it was demonstrated that the cerebellum is involved in predicting the position of the hand during a movement [21]. The predicted state of the limb from the history of motor commands allows the motor control to act on this estimate of state rather than relying solely on a delayed sensory feedback. This suggests that cerebellar output is a signal that can be combined with delayed sensory feedback elsewhere in the brain in order to generate real-time state estimates for motor control.

### Time-delay estimation and control

A primate’s sensorimotor controller is equipped with the ability to predict motor movements, as well as possess the ability to compensate for time delays. Time-delay estimation is a difficult problem, as it renders even the simplest linear systems nonlinear, yet biological control systems are robust enough to deal with time delays. However, it is not known how this is achieved. Current time-delay estimation techniques mainly cover linear systems, including: constant time delays, random time delay with specific noise characteristics, or restricted dynamic time delay [9, 10, 22–27]. However, most biological systems exhibit some degree of variability, nonlinearity, and uncertainty, which may make above mentioned methods developed inapplicable. Furthermore, most delay estimation procedures are not used in the context of predictive control methodology. The Hilbert-Huang Transform-based method is found to be the most efficient delay estimation technique with a focus on practical applicability to the motor control; however, the process is a complex one [10].

A comprehensive computational model to explain time-delay compensation in biological control is lacking. The study in this paper proposes a sensorimotor learning and control model that estimates variable time delays, predicts sensory states from delayed sensory feedback, and controls a motor system in real time. Accurate models of sensorimotor control systems result in a better understanding of the function of the human sensorimotor cortex, with practical applications in understanding the mechanisms underlying neurological disorders such as autism [28] and epilepsy [29].

The next section covers the proposed time-delay estimation method and develops a computational model for a predictive control solution for a sensorimotor control system under time delay. The proposed model is evaluated in a real time, with online learning and control simulation processes. The paper is concluded with findings and suggestions for future research.

## Methods

*x*(

*t*) is the state vector (e.g., the position of the eye or hand in space, etc.),

*u*(

*t*) is the control vector or the neural motor commands (e.g., the firing of motor-neurons or muscle contractions, etc), and

*A*(

*t*) and

*B*(

*t*) are time-varying matrices with appropriate dimensions. The matrix

*A*(

*t*) represents the influence of the current state

*x*(

*t*) of the motor system to its future changes \( \overset{.}{x}(t) \). The matrix

*B*(

*t*) is the sensorimotor controller gain, which determines how motor commands affect \( \overset{.}{x}(t) \). It is common for

*A*(

*t*) and

*B*(

*t*) to change over time. Examples include joint friction, viscosity and elasticity of muscles, etc. All of which change over time.

*x*

_{0}is the initial state [30].

*G(t)*is

*x*(

*t*) is the current state of the sensorimotor system measured by the sensor organs. The motor command vector is

*u*(

*t*). Motor commands are usually sensed at the level of the effector by specialized sensory organs. For example, muscle spindles measure the force generated in the muscle and communicate the information to the brain. Here, we assume that

*x*(

*t*) and

*u*(

*t*) are precisely measured by sensory organs.

**τ**= [

*τ*

_{ i }] (

*i*

^{th}time delay value). For simplicity we assumed

*τ*=

*τ*

_{ i }in the rest of this paper. The solution of Eq. (2) with the time delay is

Before proceeding with a solution for the time-delay problem and an associated predictive control method in the brain, let us state our assumption about the representation of time in the brain. We assume that the brain is a truly autonomous system. In other words, there are no clocks in the brain that count the ticks of *absolute time*. All sensation of time is the result of externally perceived periodic stimuli. This is unlike industrial control systems, where there are synchronized clocks that count the ticking of time and the time variable *t* can be accessed directly. Direct access to a time variable *t* is not possible in the brain’s sensorimotor control system.

We assume the brain keeps an internal estimate of time delays, denoted as \( \widehat{\boldsymbol{\uptau}} \). The error signal is calculated as \( \xi =x\left(t-\tau \right)-x\left(t-\widehat{\tau}\right) \), where *x*(*t* − *τ*) is the delayed sensory signal. The delayed sensory signals are known to the brain, but the brain cannot access the time-delay vector τ directly. On the other hand, \( x\left(t-\widehat{\tau}\right) \) is unknown since \( \widehat{\boldsymbol{\uptau}} \) is unknown. However, *x*(*t* − *τ*) can be computed from the knowledge of *G(t)*, *B(t)* and *u(t)*.

wheres *η* is the learning parameter.

The time delay *τ* can be estimated using Eq. (6). However, there are biological constraints that need to be considered. Equation (6) requires the knowledge of \( x\left(t-\widehat{\tau}\right) \), \( G\left(t-\widehat{\tau}\right) \) and *u* for any \( 0\le \widehat{\tau}\le t-\tau \). But, this is impossible because it needs to store the full history of motor commands *u*(*t*) or all functions, *G(t)* and *x(t)*. Therefore, assuming the biological plausibility of Eq. (6) without boundedness assumptions on the maximum delay *τ* is not possible.

To guarantee stability and limited memory usage, the following condition, *τ* ≤ *τ*
_{
max
} is added. This condition is reasonable and does not in any way limit the generality of the method. Furthermore, most human movements are either repetitive, such as walking, or intermittent with many pauses, such as reaching. In reaching, at the beginning of the movement, the initial position of the arm is known, and the delay is not an issue because the arm is at rest. At the end of the movement, the arm is coming back to rest and the final state of the arm is known. Therefore, delays have no detrimental effects. However, during the motion, the state of the arm keeps on changing which causes the values communicated to the brain with variable delays. It is during the arm’s motion that the delay estimation is paramount. Since movements are finite in time, applying a limit on the maximum number of delays is reasonably justified.

In terms of hardware implantation, it is necessary to store the history of constructed signals in a finite buffer. Actually, the brain automatically stores history about signals like *u*(*t*) and *x*(*t*)*.* One possible scenario is for the brain to learn the dynamics of *G*(*t*) and *B*(*t*) and thereby compute the dynamics of *x* and *u* for any time period.

Here, we assume that the brain stores *u*(*t*) from *t* to *t* − *τ*
_{
max
}, as well as *G*(*t*), *B*(*t*) and *x*(*t*). It should be noted that if delays exceed *τ*
_{
max
}, a complete open-loop control prevails.

It should be noted that *x*(*t* − *τ*) is what is actually measured and delivered to the sensorimotor plant model in the brain, represented by Eq. (4). However, *G*(*t*) and the integral over *u*(*t*) are both dependent on the estimate of the time delay \( \widehat{\tau} \). When the error in the estimate of time delay \( \varepsilon =\widehat{\tau}-\tau \) decreases to zero, the predicted state approaches to the actual state *x*(*t*).

*r*(

*t*) and current state

*x*(

*t*) be the performance error

*e*(

*t*) =

*r*(

*t*) −

*x*(

*t*), and the estimate of the performance error be \( \widehat{e}(t)=r(t)-\widehat{x}(t) \). Here, we define a PID controller input in terms of the estimated error as [31, 32]:

*K*

_{ P },

*K*

_{ D },

*K*

_{ I }and

*K*are proportional gain, derivative gain, integral gain and optimal gain, respectively.

The PID controller and the optimal feedback controller gains can be designed as if there was no delay with information about the predicted state. Essentially, the controller depends on the error *ê*(*t*) that results from the estimate \( \widehat{x}(t) \). So, if the estimate \( \widehat{x}(t) \) converges to *x*(*t*), then *ê*(*t*) converges to *e*(*t*).

Sensorimotor control known variables

Variable | Definition |
---|---|

| Goal or reference |

| Delayed sensory inputs |

| Prediction errors |

\( \widehat{\tau} \) | Estimated time delay |

\( G\left(t-\widehat{\tau}\right) \) | Estimated sensory Jacobian |

| Control Jacobian |

| Motor commands |

Sensorimotor control unknown variable

Variable | Definition |
---|---|

| Real time delay |

| Current sensorimotor state |

To illustrate the benefits of the time-delay estimation method of predictive control in the sensorimotor system, we have chosen to simulate the VOR system. The method has been implemented with MATLAB R2013a.

## Results and Discussion

*x*∈

*ℝ*is the eye position relative to the head, and

*u*∈

*ℝ*is the net motor-neuron signal to the horizontal eye muscles. So, the hVOR system equation in its simplest form [1] is shown as:

*κ*is the coefficient of viscosity and ρ is the coefficient of elasticity, and both are constants. The retinal-image slip velocity is

*y*∈

*ℝ*, which is the sum of eye and head velocities,

*y =*0. Here, the reference signal

*r*is

*-h*and the feedback error signal e is

*x*-

*r*or

*x*+

*h*. Therefore, \( y=\overset{.}{e} \)

_{,}and the feedback control law is basically a derivative control given by

Choosing the appropriate *K*
_{
D
} results in \( \overset{.}{e}=y=0 \).

With sensory delay τ, the measured state of the hVOR control system will be *x*(*t* − *τ*) instead of *x*(*t*), which means a form of time-delay estimation and a plant state predictor.

*u*(0) = 0. The state predictor can be found as

## Conclusion

In this paper, we have proposed a sensorimotor learning and control model that can predict the dynamics of variable time delays and the future sensory states from the delayed sensory information; learned new sensorimotor realities; and controlled motor system in real time. The results have demonstrated that our developed model can explain the ability of a sensorimotor system compensating delays during real-time control.

This development boils down to three claims, with varying degrees of experimental support. Firstly, we claim that the brain or sensorimotor systems possess time-delay estimation circuits. Secondly, the brain continuously estimates current/future sensory states from the previously sensed states. Thirdly, the brain uses predicted sensory states to perform optimal motor control. Essentially, this process requires performing feedback control by using predicted states.

The work makes further predictions that the brain does not need to use any form of clocking mechanism to sync various aspects of motor control systems affected by delays. In other words, the brain is a data-driven asynchronous collection of sensorimotor control systems. Also, fast and random perturbations to the motor control systems cannot be predicted and may cause instability. The predictive nature of the primate sensorimotor system and its ability to compensate for long delays have been shown by several behavioral and physiological experiments.

## Declarations

### Declarations

The publication costs for this article were funded by the Department of Electrical and Computer Engineering, Florida International University.

This article has been published as part of *BMC Bioinformatics* Volume 17 Supplement 7, 2016: Selected articles from the 12th Annual Biotechnology and Bioinformatics Symposium: bioinformatics. The full contents of the supplement are available online at https://bmcbioinformatics.biomedcentral.com/articles/supplements/volume-17-supplement-7.

### Availability of data and materials

This paper does not contain any specific/special data to be shared publicly. The simulation results can be re-done by following the method steps outlined in the paper.

### Authors’ contributions

Developed and implemented the study algorithm, and designed the experiments: MA, AS, SS. Performed the experiments and designed the study protocols: MA, AS, KY, SS. Processed the data, prepared the tests, analyzed and interpreted the study results: MA, AS, KY, SS. Drafted and manuscript: MA, AS, KY, SS. All authors have read and approved the final manuscript.

### Competing interests

The authors declare that they have no competing interests.

### Consent for publication

Not applicable.

### Ethics approval and consent to participate

Not applicable.

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

## Authors’ Affiliations

## References

- Abdelghani M, Lillicrap T, Tweed D. Sensitivity derivatives for flexible sensorimotor learning. Neural Comput. 2008;20(8):2085–111.View ArticlePubMedGoogle Scholar
- Abdelghani MN, Tweed DB. Learning course adjustments during arm movements with reversed sensitivity derivatives. BMC Neurosci. 2010;11(1):150.View ArticlePubMedPubMed CentralGoogle Scholar
- Raibert MH. A model for sensorimotor control and learning. Biol Cybern. 1978;29(1):29–36.View ArticlePubMedGoogle Scholar
- Todorov E, Jordan MI. Optimal feedback control as a theory of motor coordination. Nat Neurosci. 2002;5(11):1226–35.View ArticlePubMedGoogle Scholar
- Franklin DW, Wolpert DM. Computational mechanisms of sensorimotor control. Neuron. 2011;72(3):425–42.View ArticlePubMedGoogle Scholar
- Matthews PB. The human stretch reflex and the motor cortex. Trends Neurosci. 1991;14(3):87–91.View ArticlePubMedGoogle Scholar
- Ito T, Murano EZ, Gomi H. Fast force-generation dynamics of human articulatory muscles. J Appl Physiol. 2004;96(6):2318–24.View ArticlePubMedGoogle Scholar
- Aw ST, Todd MJ, Halmagyi GM. Latency and initiation of the human vestibuloocular reflex to pulsed galvanic stimulation. J Neurophysiol. 2006;96(2):925–30.View ArticlePubMedGoogle Scholar
- Chan Y, Hattin RV, Plant J. The least squares estimation of time delay and its use in signal detection. In: Acoustics, speech, and signal processing, IEEE international conference on ICASSP’78. 1978. p. 665–9. IEEE.Google Scholar
- Shibata T, Tabata H, Schaal S, Kawato M. A model of smooth pursuit in primates based on learning the target dynamics. Neural Netw. 2005;18(3):213–24.View ArticlePubMedGoogle Scholar
- Abaspour A, Sadati SH, Sadeghi M. Nonlinear optimized adaptive trajectory control of helicopter. Control Theory Technol. 2015;13(4):297–310.View ArticleGoogle Scholar
- Abaspour A, Sadeghi M, Sadati H. Using fuzzy logic in dynamic inversion flight controller with considering uncertainties. In: 13th Iranian conference on fuzzy systems (IFSC). 2013.Google Scholar
- Abaspour A, Parsa NT, Sadeghi M. A New Feedback Linearization-NSGA-II based Control Design for PEM Fuel Cell. Int J Comput Appl. 2014;97(10):25–32.Google Scholar
- Dallos P, Jones R. Learning behavior of the eye fixation control system. Autom Control IEEE Trans On. 1963;8(3):218–27.View ArticleGoogle Scholar
- Stark L, Vossius G, Young LR. Predictive control of eye tracking movements. IRE Trans Hum Factors Electron. 1962;2(HFE-3):52–7.View ArticleGoogle Scholar
- Westheimer G. Eye movement responses to a horizontally moving visual stimulus. AMA Arch Ophthalmol. 1954;52(6):932–41.View ArticleGoogle Scholar
- Pavel M. Predictive control of eye movement. Rev Oculomot Res. 1989;4:71–114.Google Scholar
- Morris E, Lisberger S. Signals used to maintain smooth pursuit eye movements in monkeys: Effects of small retinal position and velocity errors. Soc Neroscience Abstract. 1983;9:866.Google Scholar
- Morris E, Lisberger S. Different responses to small visual errors during initiation and maintenance of smooth-pursuit eye movements in monkeys. J Neurophysiol. 1987;58(6):1351–69.PubMedGoogle Scholar
- Churchland MM, Lisberger SG. Apparent motion produces multiple deficits in visually guided smooth pursuit eye movements of monkeys. J Neurophysiol. 2000;84(1):216–35.PubMedPubMed CentralGoogle Scholar
- Miall RC, Christensen LO, Cain O, Stanley J. Disruption of state estimation in the human lateral cerebellum. PLoS Biol. 2007;5(11):e316.View ArticlePubMedPubMed CentralGoogle Scholar
- Cleveland WS, Parzen E. The estimation of coherence, frequency response, and envelope delay. Technometrics. 1975;17(2):167–72.View ArticleGoogle Scholar
- Carter GC. Coherence and time delay estimation. Proc IEEE. 1987;75(2):236–55.View ArticleGoogle Scholar
- Hinich MJ, Wilson GR. Time delay estimation using the cross bispectrum. Sig Process IEEE Trans On. 1992;40(1):106–13.View ArticleGoogle Scholar
- Holm S, Ottesen G. Bias in the cross spectrum and time delay estimates due to misalignment. Acoust Speech Signal Process IEEE Trans On. 1986;34(6):1662–5.View ArticleGoogle Scholar
- Cabot RC. A note on the application of the Hilbert transform to time delay estimation. Acoust Speech Signal Process IEEE Trans On. 1981;29(3):607–9.View ArticleGoogle Scholar
- Müller T, Lauk M, Reinhard M, Hetzel A, Lücking CH, Timmer J. Estimation of delay times in biological systems. Ann Biomed Eng. 2003;31(11):1423–39.View ArticlePubMedGoogle Scholar
- Takarae Y, Luna B, Minshew NJ, Sweeney JA. Visual motion processing and visual sensorimotor control in autism. J Int Neuropsychol Soc. 2014;20(01):113–22.View ArticlePubMedGoogle Scholar
- Koelewijn L, Hamandi K, Brindley LM, Brookes MJ, Routley BC, Muthukumaraswamy SD, Williams N, Thomas MA, Kirby A, te Water Naudé J. Resting‐state oscillatory dynamics in sensorimotor cortex in benign epilepsy with centro‐temporal spikes and typical brain development. Hum Brain Mapp. 2015;36(10):3935–49.View ArticlePubMedGoogle Scholar
- Sargolzaei A, Yen KK, Abdelghani MN. Preventing Time-Delay Switch Attack on Load Frequency Control in Distributed Power Systems. IEEE Transactions on Smart Grid. 2016;7(2):1176–85.Google Scholar
- Sargolzaei A, Yen KK, Abdelghani M. Control of Nonlinear Heartbeat Models under Time-Delay-Switched Feedback Using Emotional Learning Control. International Journal on Recent Trends in Engineering & Technology. 2014;10(2):85–91.View ArticleGoogle Scholar
- Sargolzaei A, Yen K, Abdelghani M. Delayed inputs attack on load frequency control in smart grid. In: Innovative smart grid technologies conference (ISGT), 2014 IEEE PES. 2014. p. 1–5. IEEE.Google Scholar
- Leigh RJ, Zee DS. The neurology of eye movements. USA: Oxford University Press; 2015.View ArticleGoogle Scholar
- Kheradmand A, Zee DS. Cerebellum and ocular motor control. Front Neurol. 2011;2(53):1–15.Google Scholar