## Abstract

Experimental observations indicate that positive feedback plays an important role for maintaining human balance in the upright position. This observation is used to motivate an investigation of a simple switch-like controller for postural sway in which corrective movements are made only when the vertical displacement angle exceeds a certain threshold. This mechanism is shown to be consistent with the experimentally observed variations in the two-point correlation for human postural sway. Analysis of first-passage times for this model suggests that this control strategy may slow escape by taking advantage of two intrinsic properties of a stochastic unstable first-order delay differential equation: (i) time delay and (ii) the possibility that the dynamics can be ‘temporarily confined’ near the origin.

## 1. Introduction

Falling is a leading cause of mortality and morbidity in the elderly. Consequently, as a society ages, it becomes increasingly important to understand how the human nervous system maintains its precarious control of the upright posture so that effective preventive strategies can be developed. Mathematical models for balance control identify three essential components (Ohira & Milton 1995; Eurich & Milton 1996; Newell *et al*. 1997; Stépán & Kollár 2000; Yao *et al*. 2001; Kiemel *et al*. 2004): (i) feedback, (ii) time delays, and (iii) the effects of random uncontrolled perturbations (referred to hereafter as ‘noise’). Consequently, discussions of the dynamics of delay stochastic differential equations figure prominently; an example is(1.1)where *x*(*t*) and *x*(*t*−*τ*) are, respectively, the variables related to the vertical displacement angles measured at times *t* and *t*−*τ*; *f* is the feedback; and the additive noise term, *ξ*(*t*), satisfieswith variance *η*^{2}.

Postural sway refers to the fluctuations in the centre of pressure (COP), which occur as a subject stands quietly with the eyes closed on a pressure platform. Typically, it is assumed that the subject stands in neutral alignment (figure 1*a*). In neutral alignment, the parts of the body are balanced and symmetrical along the line of gravity. This line of gravity passes just anterior to the ankle, through the centre of the knee, hip and shoulder joints, and through the external meatus of the ear. For subjects standing in neutral alignment, it has been suggested that the biomechanical properties of the ankle, knee and hip provide a major passive mechanism for balance control. Active balance control during quiet standing has been thought to be provided by the action of the stretch reflex of the calf muscles. In this hypothesis, forward sway is accompanied by a stretch of the calf muscles, which activates the stretch reflexes to increase tension in the muscles to oppose the forward sway. However, recent non-invasive measurements of calf muscle lengths during postural sway demonstrate that the length changes are opposite to this prediction (Loram *et al*. 2005*a*,*b*): the average calf muscle fibre length shortens during forward sway and lengthens during backward sway. Thus, for a subject standing in neutral alignment, the feedback at the ankle is positive and not negative.

A second surprising observation is that the muscle controlling movements for balance control are not continuous but are intermittent and pulsatile (Loram & Lakie 2002; Loram *et al*. 2006). Similar conclusions concerning positive feedback and intermittent control have been obtained from studies of a closely related experimental paradigm, namely stick balancing at the fingertip (Cabrera *et al*. 2006; Hosaka *et al*. 2006). Fluctuations that exhibit these properties are highly suggestive of ‘chattering’, the dynamic signature of switch-type discontinuous controllers (Bottaro *et al*. 2005). Moreover, pulsatile corrective movements have been demonstrated in models of robotic control in which the digital nature of time-delayed feedback control circuits is taken into account (Haller & Stépán 1996; Enikov & Stépán 1998).

Here, we model the fluctuations in the COP during quiet standing as a delayed stochastic differential equation. In view of the above observations, we examine a switch-type discontinuous feedback controller that incorporates two feedback control mechanisms (figure 2)(1.2)where *α*, *C*>0 and *τ*, *τ*′ are two time delays not necessarily equal. The case when *τ*=0 was discussed previously (Eurich & Milton 1996). Since the vertical displacement angle, *Θ*, is small, we have *x* ∼sin *Θ* and the inverted pendulum is overdamped (hence higher order derivatives are ignored; Werness & Anderson 1984; Chow & Collins 1995). When *x* is less than a threshold *X*, the statistical properties are those of an unstable delayed random walk (Milton *et al*. 2008). This local time-delayed feedback controller reflects either a feedback controller tuned to the unstable side of the edge of stability (Cabrera *et al*. 2006; Hosaka *et al*. 2006) or the thixotropic properties of muscle-elastic tissue (Proske *et al*. 1993; Loram *et al*. 2006). When *x* exceeds *X*, a pulsatile feedback controller is activated with delay *τ*′. The presence of feedback controllers that become active only when the controlled variables exceed a threshold is very well known in balance control, e.g. the difference between the ankle, hip and step strategies for controlling balance (Shumway-Cook & Woollacott 2001). ‘Safety net’-type controllers also arise in the design of strategies to control attractors that have shallow basins of attraction (Guckhenheimer 1995). Despite the obvious over-simplification of this model, it provides a remarkably robust empirical explanation for the statistical properties of the correlated random walks that arise in the fluctuations of COP (Ohira & Milton 1995; Eurich & Milton 1996; this communication).

Here, we show that when *τ*≠0, the interplay between noise, delay and positive feedback yields a mechanism that has two remarkable properties that benefit control: (i) the time delay that increases the time for threshold crossing, and (ii) the corrective action taken upon threshold crossing can lead to a solution that is transiently confined near the origin.

## 2. Methods

Data were collected and analysed for a group of 16 females and 18 males aged 18–89 years, who were free of balance disorders. During testing, participants were alone with the experimenter in a quiet room with minimal visual and auditory distractions. Each participant was asked to stand in stocking feet on a pressure platform (AccuSway, AMTI) looking straight ahead with the eyes closed while remaining as still as possible. This study was approved by the institutional review board at Claremont McKenna College in accordance with the currently applicable US Public Health Service Guidelines. The sampling frequency was 200 Hz. The data were resampled at 100 Hz.

In order to compare our experimental observations, and those obtained in other laboratories (e.g. Collins & De Luca 1994), with the predictions of (1.2), we calculated the two-point correlation function, *K*(*s*), for postural sway where(2.1)for each *s*=|*t*_{1}−*t*_{2}|, the two-point correlations are calculated from *N* data points spanning *N*−*m* data intervals of length *ms*, and where *x* indicates the displacements of the fluctuations in the anterior–posterior direction and *y* indicates the displacements in the medial–lateral direction. Our reasoning for focusing on this statistical property is partly historical and practical. Historically, the fluctuations in COP during quiet standing have been modelled in terms of a correlated random walk (e.g. Collins & De Luca 1994). The complex nature of *K*(*s*) provided the first evidence that balance control for small displacement angles was different from that for larger displacements and moreover motivated the development of delayed random walk models (e.g. Ohira & Milton 1995). Practical concerns arise because reliable estimates of *τ*, *τ*′, *α* and *C* are not yet available and hence quantitative comparisons between (1.2) and observation are premature. Thus, our goal is to determine whether or not *K*(*s*) predicted by (1.2) is qualitatively the same as that observed experimentally.

It is important to note that a great deal of controversy has surrounded the question as to whether (Collins & De Luca 1994) or not (Newell *et al*. 1997; Delignières *et al*. 2003) the COP fluctuations can be modelled in terms of a correlated random walk. It is now generally accepted that it is important to take into account the boundedness of COP fluctuations, as illustrated in figure 3 (Ohira & Milton 1995; Delignières *et al*. 2003). However, completely lost in this debate has been the fact that different types of two-point correlation can be readily observed experimentally and hence must be accounted for in any proposed model for postural sway.

## 3. Results

We first demonstrate that three qualitatively different *K*(*s*) can be observed experimentally and that they probably have a dynamical basis. Then, we show that qualitatively similar *K*(*s*) can be generated by (1.2) by holding all parameters fixed and varying the noise intensity *η*^{2}.

Figure 4*a* shows that the three qualitatively different *K*(*s*) can be identified in postural sway. These are referred to, respectively, from top to bottom, as types I, II and III. The type I pattern has two distinct, non-oscillatory regions with positive slope; the type II pattern has three distinct, non-oscillatory regions with positive slope; and the type III pattern has two distinct regions, one of which is oscillatory. Although the significance of the type II pattern has been extensively discussed previously (Collins & De Luca 1994; Chow & Collins 1995; Eurich & Milton 1996), the other patterns have received comparatively little attention (Ohira & Milton 1995).

Figure 4*b* shows similar types of *K*(*s*) that can be generated by (1.2) by holding the parameters *α*, *τ* and *C* constant and varying the noise intensity *η*^{2}. The parameters were chosen using the Eurich–Milton (1996) conjecture, namely *K*(*s*), qualitatively similar to those observed for postural sway, which can be generated if the parameters are chosen to give rise to bistability and the noise intensity adjusted so that switches can occur between the two attractors.

Three additional observations support the hypothesis of (1.2) that the different *K*(*s*) observed for postural sway have a dynamic basis: (i) the area of the ellipse that encloses the fluctuations of postural sway is in the order (highest to lowest) type I, type II and then type III, (ii) the same person can exhibit different *K*(*s*) on different days, and (iii) application of a 250 Hz vibration whose amplitude was sub-threshold over the Achilles tendon for two subjects with a type III *K*(*s*) produced a type I *K*(*s*) (J. Townsend & J. Milton 2007, unpublished data; table 1).

## 4. First-passage times

The cost for the discontinuous negative feedback loop in controlling balance is directly proportional to the number of times it is activated. We can get insight into this quantity by determining the first-passage time for(4.1)i.e. the equation that describes the dynamics when *x*(*t*−*τ*)<|*X*|. We assume that *α* is not too large and that *τ*<3*π*/2*α*. In this case, there is one real eigenvalue that is positive and an infinite number of complex eigenvalue pairs whose real part is negative. The time to cross a threshold located at ±*X* for (4.1) for a given *α* depends on *τ*, *η*^{2} and the initial function, *x*(*t*)=*ϕ*(*t*), *t*∈[−*τ*,0], denoted by *ϕ*_{0}(*t*) (Milton *et al*. 2008).

Since (4.1) describes an unstable time-delayed dynamical system, it is difficult to obtain expressions for the transient variance and correlation using standard methods. A useful trick for the analysis is to recast (4.1) in terms of a delayed random walk (Ohira & Milton 1995, in press; Ohira & Yamane 2000): the walker takes a discrete step of unit length per unit time in a direction determined by the conditional probabilities that depend on the position of the walker at some time, *τ*, in the past. In particular, when the delayed random walk evolves in a quadratic potential, the Fokker–Planck equation (Milton *et al*. 2008; Ohira & Milton in press) is identical to that obtained starting from (4.1) (Frank 2005), i.e.whereand where *γ* and *D* are constants and *P*(*x*, *t*; *y*, *t*−*τ*) is the joint probability that the walker is at position *x* at time *t* and position *y* at time *t*−*τ*. Since the Fokker–Planck equation does not depend on the starting point of the derivation, we see that the delayed random walk in a quadratic potential and the stochastic first-order delay differential equation given by (4.1) provide equivalent descriptions of the same stochastic process. In other words, we can use a delayed random walk to obtain insights into the properties of (4.1). Two advantages for the use of the delayed random walk are (i) issues related to stochastic integration are circumvented and (ii) the numerical simulations are exact since the numerical simulations do not involve additional approximations.

The delayed random walk in a quadratic potential is a generalization of the Ehrenfest random walk for the case that *τ*≠0 (Kac 1947). In order to construct this walk, we proceed as follows. First, we generalize a position-dependent random walk so that the transition probability depends on its past state, for example(4.2)where the position of the walker at time *t* is *X*(*t*); *P*(*n*, *t*) is the probability for the walker to be at *X*(*t*)=*n*; *P*(*n*, *t*; *m*, *t*−*τ*) is the joint probability such that *X*(*t*)=*n* and *X*(*t*−*τ*)=*m* takes place; and *f*(*x*) and *g*(*x*) are the transition probabilities for the walker to take a step to the negative (−1) and positive (+1) directions, respectively. Since (1.1) describes an unstable dynamical system that evolves on a symmetric (i.e. quadratic) potential surface, *f* and *g* must be chosen to preserve this symmetry with respect to the origin. Thus, we choose(4.3)where *d* is a constant such that 0≤*d*≤0.5 and *k*=2*d*/*a*, so that *f*(*x*)<*g*(*x*) for *x*>0 and *f*(*x*)>*g*(*x*) for *x*<0. Also, *f*(*x*)+*g*(*x*)=1. Thus, the repulsive delayed random walk that corresponds to (4.1) is given by (4.2) together with these choices of *f*, *g*.

The equation for the non-stationary time dependence of the correlation function *C*(*Δ*, *t*)=〈*X*(*t*)*X*(*t*−*Δ*)〉, and hence the change in the variance *C*(0, *t*), can be derived from the equation for the joint probability distribution (Ohira & Yamane 2000; Ohira & Milton in press)(4.4)where the definitions of *f* and *g* are the same. When the initial function is constant zero, i.e. *ϕ*_{0}(*t*)=0, we eventually obtain the following set of coupled delayed partial differential equations:(4.5)

Figure 5 plots the variance predicted by (4.5) and compares this with the variance determined from the numerical simulations of (4.1) when *ϕ*_{0}(*t*)=0. As can be seen, the results are in excellent agreement. The results in figure 5 suggest that to a first approximation the first-passage time, *L*, i.e. the time that it takes a solution of (4.1) with *ϕ*_{0}(*t*)=0 to cross a threshold (identified here with ±*X*), decreases as *τ* increases. In principle, the most probable, or mean, first-passage time can be calculated using the backward Kolmogorov equation that corresponds to (4.1). However, we have been unable to complete this calculation. Therefore, we calculated the distribution of first-passage times using numerical simulations of (4.1).

Figure 6 shows the distribution of *L* for different values of *τ* when *ϕ*_{0}(*t*)=0. The distribution of *L* is clearly bimodal for *τ*≠0. Thus, there exist two pathways of escape from the origin at *x*=0: a relatively slow path and a faster one. This bimodal nature of the distribution in *L* is related to the number of delayed zero crossings, since if *x*(*t*) and *x*(*t*−*τ*) have different signs, then there is an increased probability that the evolution of (4.1) will change direction, thus further increasing the time until threshold crossing (Milton *et al*. 2008). In other words, the slower paths arise because the solution is temporarily confined near the origin as reported previously in delay differential equations in the setting of a saddle point (Pakdaman *et al*. 1998; Grotta-Ragazzo *et al*. 1999).

## 5. Discussion

It is remarkable that simple models, such as (1.2), are able to capture some aspects of the observed dynamics for human postural sway (Ohira & Milton 1995, in press; Eurich & Milton 1996; Newell *et al*. 1997; Yao *et al*. 2001). Thus, not only is it possible to obtain insights into the dynamic basis of the differences in *K*(*s*), but also to understand what the benefits of a discontinuous control strategy might be for the nervous system. We suggest that there is an elegant simplicity for the neural control of balance. By applying a corrective pulse whenever the variable crosses a threshold, such as ±*X*, advantage is taken of two intrinsic properties of (4.1): (i) the time delay that slows escape and (ii) the possibility that the reset can lead to a transient confinement of the dynamics near the origin.

Concepts borrowed from control engineering, such as feedback and feed-forward control, have long dominated the thinking of neuroscientists concerning how the nervous system regulates its activities. Thus, it is surprising that very little attention has been given by neuroscientists to the considerations of discontinuous or ‘bang-bang’ control (Flügge-Lotz 1953, 1968; Insperger 2006; Lenhart & Workman 2007). Such controllers are favoured by engineers when expense is a concern; familiar examples arise in the context of the thermostatic control of room temperature. An important feature of discontinuous control is that it can be shown to be optimal when control is bounded. Certainly, it is true that many sensory receptors exhibit switch-like properties including certain mechanoreceptors (e.g. Grigg & Greenspan 1977) and *α*-motor neurons in the spinal cord (Hounsgaard *et al*. 1988; Shapiro & Lee 2007). Thus, there are no anatomical reasons to exclude *a priori* the possibility of discontinuous control, at least in certain situations.

Time delays play an important role in discontinuous control mechanisms. They arise because the time between when a threshold is crossed and when a controlling movement is triggered is finite. The existence of such time delays is absolutely essential to ensure that the solution is always uniquely determined (Flügge-Lotz 1968). The properties of time-delayed piecewise feedback control have been studied by mathematicians (an der Heiden & Mackey 1982; an der Heiden *et al*. 1990; Bayer & an der Heiden 2007) and exploited by experimentalists (Milton *et al*. 1989; Milton & Longtin 1990). Multistability and other complex dynamics readily arise in these examples (Losson *et al*. 1993; Foss *et al*. 1997). Although the effects of noise have received comparatively little attention (Longtin *et al*. 1990; Eurich & Milton 1996), it is obvious that switching between coexisting attractors will add further complexity. Thus, it is not surprising that the statistical properties of fluctuations in postural sway are complex and would change as noise intensity is varied. Unfortunately, as initially pointed out by Flügge-Lotz (1953), the dynamical complexities introduced by discontinuous feedback can render interpretation of the significance of the observed fluctuations themselves difficult (cf. Loram & Lakie (2002) with Loram *et al*. (2005*a*)). Clearly, more research needs to be done on the properties exhibited by such control mechanisms, especially those that operate in a stochastic environment.

Neuroscientists favour the interpretation that the paradoxical changes in the length of calf muscle fibres during postural sway reflect the action of a predictive-type control strategy (Loram *et al*. 2005*a*,*b*). However, the nature of the changes that occur in calf muscle fibre length with sway clearly depends on the subject's posture. For example, for the common kyphotic-scoliosis posture of today's desk-bound sedentary lifestyle, a posterior displacement of the mass in the thorax is balanced by an anterior displacement of the hip, head and knee (figure 1*b*). Importantly, for our discussion, there is increased ankle dorsiflexion, indicated in figure 1*b* by the anterior displacement of the knee. In this case, forward sway would be accompanied by a decrease in the inward curvature of the back and hence a shortening of the calf muscles.

An important concern is the cost(s) involved in the implementation of a control strategy (Flügge-Lotz 1953, 1968; Lenhart & Workman 2007). Clearly, the human nervous system does not have an infinite supply of resources at its disposal. There is just so much energy available and many of the brain's computation capacities, such as attention and working memory (Just *et al*. 2001; Cabrera & Milton 2004; Beilock & Carr 2005; Milton *et al*. 2007), are finite. Thus, the nervous system must disregard elegant solutions that are costly to implement in favour of those that are effective but cheaper. We anticipate that our demonstration of the effectiveness of inexpensive switch-like controllers for balance will trigger an interest among neuroscientists to evaluate other neural control hypotheses from the point of view of implementation costs.

## Acknowledgments

We thank J. L. Cabrera, S. A. Campbell, C. W. Eurich, A. Nadim and A. Radunskaya for their useful comments. This research was supported by the National Science Foundation (NSF-0617072).

## Footnotes

One contribution of 10 to a Theme Issue ‘Delay effects in brain dynamics’.

- © 2009 The Royal Society