## Abstract

The discovery of chaotic dynamics implies that deterministic systems may not be predictable in any meaningful sense. The best-known source of unpredictability is sensitivity to initial conditions (popularly known as the butterfly effect), in which small errors or disturbances grow exponentially. However, there are many other sources of uncertainty in nonlinear dynamics. We provide an informal overview of some of these, with an emphasis on the underlying geometry in phase space. The main topics are the butterfly effect, uncertainty in initial conditions in non-chaotic systems, such as coin tossing, heteroclinic connections leading to apparently random switching between states, topological complexity of basin boundaries, bifurcations (popularly known as tipping points) and collisions of chaotic attractors. We briefly discuss possible ways to detect, exploit or mitigate these effects. The paper is intended for non-specialists.

## 1. Introduction

The trouble was that ignorance became more interesting, especially big fascinating ignorance about huge and important things such as matter and creation, and people stopped patiently building their little houses of rational sticks in the chaos of the universe and started getting interested in the chaos itself—partly because it was a lot easier to be an expert on chaos, but mostly because it made really good patterns that you could put on a t-shirt.

And instead of getting on with proper science,

^{1} scientists suddenly went around saying how impossible it was to know anything, and there wasn't really anything you could call reality to know anything about, and how all this was tremendously exciting, and incidentally did you know there were possibly these little universes all over the place but no-one can see them because they are all curved in on themselves? Incidentally, don't you think this is a rather good t-shirt?Terry Pratchett [1], ch. 1

On 5 July 1687, Isaac Newton's *Principia mathematica* sparked an intellectual revolution, allowing scientists to write down mathematical descriptions of Nature, and use them to calculate what Newton called ‘the system of the world’. There had been precursors, of course; notably Galileo Galilei and Johannes Kepler. But Newton pulled everything together into a powerful package. He expressed his ideas in terms of classical Euclidean geometry, but behind the scenes, the work was informed by a new area of mathematics: differential equations. This was a natural outgrowth of his ideas on calculus, developed quasi-independently by Gottfried Wilhelm Leibniz: priority was controversial at the time, and remains so among scholars [2]. A differential equation expresses the state of a system in terms of the rate of change of that state. The solution of the equation, if it can be obtained, yields the state of the system at all instants of time. Today, any system with that property is called a ‘dynamical system’.

The mathematicians of continental Europe quickly made Newton's ideas their own, placed calculus at the forefront of the enterprise and devised mathematical equations for heat, light, sound, waves, fluids, electricity and magnetism. Modern physics was born.

‘All instants of time’ includes past, present and *future*. Armed with Newton's equations for gravity and motion, astronomers could predict, with great accuracy, the motion of the Moon and planets, the timing of eclipses and the orbits of asteroids. The heavens appeared to be some gigantic clockwork machine, and like a machine, their workings were entirely determined by their structure and mode of operation. In his 1812 *Essai philosophique sur les probabilités* [3], Pierre-Simon de Laplace wrote
We may regard the present state of the universe as the effect of its past and the cause of its future. An intellect which at a certain moment knew all forces that set nature in motion, and all positions of all items of which nature is composed, if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes.

Laplace [3], p. 4

Laplace was expressing a mathematical fact in dramatic language, and he did not assert that any such intellect did or could exist, certainly outside the realm of the divine. But his views became established as the concept of a *deterministic* universe: one whose future is in principle completely specified, and unique, if its present state is known in perfect detail. This view is parodied in Douglas Adams's *The hitch hiker's guide to the galaxy* as the supercomputer Deep Thought, which ponders the Great Question of Life, the Universe and Everything, and after seven and a half million years gives the answer: 42.

In astronomy, as far as the questions being asked at that time went, Laplace was pretty much right in practice as well as in principle. Deep Thought would have obtained excellent answers, as its real-world counterparts do today. In other branches of science, and eventually in astronomy as well, it started to become apparent that principle and practice were often poles apart. This was, to some extent, disguised by the widespread use of the simplest types of differential equation: linear systems. Models of this kind often work well when key variables are small—waves of small amplitude, for example. Linear models also have a seductive simplicity: they can (usually) be solved in a very explicit manner, often by a formula. The other side of the coin is that linear models can be too simple, and miss important forms of behaviour. This point of view is often paraphrased as a statement attributed to Albert Einstein: ‘As simple as possible, but not simpler’. The source seems to be his remark: ‘It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience’, Einstein [4], p. 165.

In a linear world there are no nasty surprises. Whatever happens on a small scale can be transferred proportionately to larger scales; so twice the cause produces twice the effect. Linear models are far more tractable mathematically than nonlinear ones, and have been widely used for this reason. However, Nature and human affairs are seldom obliging enough to obey linear models, except in quantum mechanics, where the Schrödinger equation is linear and corresponds very closely to experimental observations. So, annoying though they may be, nonlinearities cannot be ignored.

In the late nineteenth century, Henri Poincaré introduced a new way to think about dynamics, based on geometry instead of numbers. His idea, the ‘qualitative theory of differential equations’, triggered a slow revolution in our ability to handle nonlinearity. A major step forward was Poincaré's own work on the three-body problem: how does a system of three bodies—such as the Earth, Moon and Sun—behave under Newtonian gravitation? His eventual answer, once he had corrected a significant mistake [5], was that the behaviour can be extraordinarily complex [6],
One is struck with the complexity of this figure that I am not even attempting to draw. Nothing can give us a better idea of the complexity of the three-body problem.

Poincaré [6], p. 252

In the 1960s, building on Poincaré's ideas, Stephen Smale conceived a programme to develop Poincaré's approach into a systematic and powerful theory of nonlinear dynamical systems [7], based on topology—a flexible kind of geometry that Poincaré had also pioneered. At much the same time, scientists in a variety of disciplines discovered strange phenomena in various nonlinear systems that modelled aspects of Nature. The appearance of fast, powerful computers made it much easier to study nonlinear dynamics in any specific case, by approximating the behaviour of the system numerically. This option had always been available in principle, but performing billions or trillions of calculations by hand was not a viable option. Now, a machine could do the task, and unlike a human calculator, it did not make arithmetical mistakes. However, machine calculations are subject to rounding errors when the decimal or binary representation of a number is truncated at a finite number of places, or otherwise represented in the machine, and this raises serious issues about accuracy. Synergy among these three driving forces—topological insight, the needs of applications, and raw computer power—has created a revolution in our understanding of nonlinear systems, the natural world and the world of human concerns.

The revolution in nonlinear dynamics has opened up a wealth of new phenomena. Some are useful, some are obstacles to be overcome, some turn out to be both of those. Some place limits on what we can hope to achieve in particular circumstances. All are fascinating; most are surprising.

One of the best known among these new phenomena is chaos—more properly, deterministic chaos. Here, simple, non-random rules can generate irregular behaviour: so irregular that in some respects it can appear random. In fact, in some respects it may technically *be* random, when restricted information is considered. In a chaotic system, determinacy and practical predictability are not synonymous. Such a system is predictable in the short term, unlike a truly random system, but there is a ‘prediction horizon’ beyond which inevitable small uncertainties in the initial state grow so large that no reliable prediction can be made. This idea has been popularized as the ‘butterfly effect’, generally associated with Lorenz [8], but actually deriving from Lorenz [9], where the effect described is different. This topic is discussed in more detail in §4.

It is less well known that there are many other sources of uncertainty in deterministic dynamics. In the popular literature, much emphasis is devoted to two of them: ‘tipping points’ [10], technically known as bifurcations [11], and ‘black swan events’ [12], which are unexpected events inadvertently assumed to be rarer than they really are. Both have been the subject of much current speculation and exhortation, but they by no means exhaust the possibilities. The aim of this article is to survey a selection of sources of uncertainty in deterministic systems, to explain in informal terms what they are and why they occur, to indicate what might be done about them, and to illustrate their applicability to real-world problems. The intended audience is essentially that of the conference at which a version of this paper was originally presented: professionals in many different areas with an interest in recent scientific insights into uncertainty. Occasional references to popular phrases are intended to relate the material to discussions that may have been encountered in popular writing or the media.

Our growing understanding of the curious but intriguing behaviour of nonlinear dynamical systems provides a framework for thinking about realistic models of Nature. It constrains the design of experiments and the interpretation of results, and warns us against making assumptions based on traditional linear thinking. It also provides the beginnings of a toolkit for dealing with some aspects of scientific uncertainty. Most importantly, it provides a clearer view of uncertainty itself, while revealing that often there may be hidden order behind Nature's irregularities. The world is mostly nonlinear, and it is the task of science and mathematics to face up to that fact, and discover how to turn it to our advantage.

## 2. Quick primer in nonlinear dynamics

In broad terms, a *dynamical system* is any collection of mathematical rules that determines the future behaviour of some system in terms of its state at any instant. More specifically, we focus on two basic types of dynamical system:

— discrete:

*x*_{t+1}=*f*(*x*_{t}),*t*=0,1,2,…, and— continuous: d

*x*/d*t*=*f*(*x*),*t*∈**R**.

Here, **R** is the set of real numbers, *x* is a point in a manifold (so in coordinate form it may have several components, see below) and *f* is a suitable function (the precise conditions are omitted). In other words, a discrete dynamical system is a difference equation, and a continuous dynamical system is a differential equation (technically, an ordinary differential equation, as opposed to a partial differential equation).

Both kinds are *deterministic*: given *x*_{0}, the sequence *x*_{t} is uniquely determined by the function *f*, and given *x*(0), all *x*(*t*) are uniquely determined by *f*. The proof of determinacy in the discrete case is easy,
2.1
and so on. The proof for ordinary differential equations [13], §6.3 is more technical, and requires restrictive hypotheses on the function *f*, but it yields more: not only is the future determined, but so is the past. In the discrete case, the past is determined provided *f* is invertible. If not, different past states may lead to the same present state, and therefore to the same future for all time.

We have not yet discussed the nature of the ‘variable’ *x*. This is specified by the *phase space* or *state space* of the system, which is the set of all possible values of *x*_{t} or *x*(*t*). Usually this is a manifold—a multi-dimensional space. Often *x* is a vector (*x*_{1},…,*x*_{n}), where the *x*_{i} are real numbers, but the coordinates may also be angles, or more complicated quantities. The dimension *d* of phase space is what engineers call the number of *degrees of freedom* (d.f.) of the system.

In the discrete case, the *orbit* or *trajectory* of an initial point *x*_{0} is the sequence *x*_{0},*x*_{1},*x*_{2},*x*_{3},…. In the continuous case, it is the curve {*x*(*t*):*t*∈**R**}. Phase space lets us draw a picture of the trajectories, called a *phase portrait*. In figure 1, the dimension *d*=2; so we get a picture in the plane.

Typical features of dynamics in the plane include:

— source: a point away from which all nearby trajectories flow;

— sink: a point into which all nearby trajectories flow;

— saddle: a point near which two trajectories flow in, two flow out and the rest come close but then move away again; and

— limit cycle: a closed loop. (In the older literature, ‘limit cycle’ is often reserved for the stable case, in which all nearby trajectories flow towards the loop. This is the reason for the word ‘limit’.)

The phase portrait in figure 1 also illustrates the notion of stability. A state of a dynamical system is *stable* if the system returns to that state after a small disturbance, or ‘perturbation’. Otherwise, it is *unstable*. In a phase portrait, a state is stable if all arrows on nearby trajectories point towards it, and unstable otherwise. So in figure 1, the sink and limit cycle are stable, but the source and saddle (which has both types of arrow) are unstable.

The first three features listed are equilibria or fixed points; the fourth is a stable periodic trajectory. Unstable periodic trajectories are also possible (reverse the arrows in the picture). The Poincaré–Bendixson theorem [14,15] states that in the plane, typically that is all we get.

The big discovery that turned nonlinear dynamics on its head is that three dimensions are different. In continuous systems with at least three d.f., and discrete systems with at least two, more complex behaviour is possible,

— chaos: trajectories converge on to an infinitely complicated shape, known as an attractor. Trajectories on this attractor that start close together diverge rapidly as time passes, but remain confined to the attractor.

A classic example of chaos is the Lorenz attractor, shown in figure 2, and discussed in §4.

In a discrete system, the trajectories are sequences of separate points, but some useful geometric information can still be represented: in particular, the form of attractors.

## 3. Sources of uncertainty

To illustrate the scope of this article, we list the main sources of uncertainty that will be discussed. All technical terms will be explained in later sections; the intention is to showcase the diversity of the phenomena.

— Chaos leads to sensitive dependence on initial conditions (butterfly effect).

— Lack of control over initial conditions can cause uncertainty, even when there is no chaos (for example, in coin tossing).

— The geometry of basins of attraction of competing attractors can be topologically complex, leading to uncertainty not only about the trajectory on an attractor, but also which attractor it is on. Phenomena include Wada basins, in which three or more basins have a common boundary [16], and riddled basins, which have points arbitrarily close to a different basin and have very complex boundaries [17].

— Heteroclinic connections can occur: here, apparent ‘attractors’ are in fact not attractors, but the system stays near them for substantial periods before making a rapid transition elsewhere. These can lead to mixtures of predictability and unpredictability, such as cycling chaos, where the system makes random transitions round a fixed cycle of chaotic attractors.

— Bifurcations (popularly known as tipping points) are not unusual: here, a small change in one or more parameters of the system causes a qualitative change in its behaviour. Parameters are ‘adjustable constants’ that do not change as time passes, in any specific run of the dynamics, but whose values can be adjusted on each run. Bifurcation phenomena causing uncertainty include intermittency, blowout, bubbling, collision of attractors and explosion of an attractor.

## 4. The butterfly effect

The phenomenon now known as the butterfly effect was discovered by the meteorologist Edward Lorenz in 1961 while working on a heavily simplified model of convection in the atmosphere. He published his findings in Lorenz [8], but it took some time before they were appreciated by meteorologists or known to mathematicians. The origin of the name is described shortly, and ironically it arose in connection with a different paper [9], and a different, though related, phenomenon. The technical term for the phenomenon noted in Lorenz [8] is ‘sensitivity to initial conditions’, and contrary to the historical record, the term ‘butterfly effect’ almost always refers to this.

Lorenz's model was a system of differential equations with three state variables *x*,*y*,*z* and three adjustable parameters *r*,*b*,*s*,
4.1

Without the terms *xz* and *xy*, these would be linear equations. Lorenz showed that a little nonlinearity goes a long way. One key discovery was serendipitous: he was using a computer to solve his equations numerically, had to stop in the middle of a run and re-entered the numbers by hand to restart the calculation with some overlap to check all was well. He noticed that after a time, the results diverged from his previous calculations. Suspecting a mistake, he rechecked the numbers, and they were correct. Eventually he discovered that the computer held numbers internally to more digits than it printed out. This tiny difference caused the divergent results. Figure 3 recreates this phenomenon by changing one coordinate of the initial condition by one part in a million.

Lorenz himself noted in the 1972 lecture described below:
One meteorologist remarked that if the theory were correct, one flap of a seagull's wings could change the course of weather forever.

The remark was intended as a put-down, but Lorenz was right. In later speeches and papers, he used the more poetic butterfly, which came about by accident. According to Lorenz, upon failing to provide a title for a talk at the 139th meeting of the American Association for the Advancement of Science in 1972, Philip Merilees concocted ‘Does the flap of a butterfly's wings in Brazil set off a tornado in Texas?’ Ever since, ‘a butterfly flapping its wings’ has remained constant—but the location of the butterfly, the consequences and the location of the consequences have varied widely. Moreover, it is almost certain that the topic of this lecture was Lorenz [9], which discussed a stronger form of uncertainty, related to a limit on the scale of observable disturbances that cannot be avoided however accurate the measurements may be [18].

Lorenz called this behaviour ‘unstable’, but we now see it as a new type of stability. Namely, the Lorenz system has an attractor. Unlike the traditional attractors of classical mathematics, points and closed loops, this attractor has more complex topology. In fact, it is a *fractal*, meaning that it has intricate structure on all scales of magnification [19].

Informally, an *attractor* is a region *A* of phase space such that any initial condition starting near *A* converges towards a trajectory that lies in *A*. The technical definition is more complicated, and several alternatives exist [20]. Because *A* is not a point or a closed loop, this property leaves open the prospect that although distinct trajectories on *A* may diverge, they can still lie on *A*. So *A* is a stable object, and the dynamics on *A* is *robust under perturbations* in the following sense. If the system is subjected to a small disturbance, the trajectory can change dramatically; however, *it still lies on* *A*, and typically fills *A* densely over infinite time. This implies that chaotic behaviour is physically feasible, unlike the usual notion of instability, where unstable states are usually not found in reality—for example, a pencil balancing on its point. But in this more general notion of stability, the details are not repeatable, only the overall ‘texture’. The curves in figure 3 resemble each other qualitatively, but differ quantitatively.

As previously noted, Lorenz's observation is now called ‘sensitivity to initial conditions’. Trajectories on *A* not only can, but typically do, diverge after small perturbations. The divergence is rapid in the sense that any small change in initial conditions grows exponentially, at first, until it becomes comparable to the size of *A* itself. After that, the original trajectory and the perturbed one are effectively independent.

Is sensitivity to initial conditions real? Mathematically, it is as real—and as common—as classical regular behaviour such as a periodic cycle. In fact, sensitive dependence on initial conditions is the basic definition of chaotic dynamics. Such behaviour has been observed in many physical systems; in particular, it is what makes many mixing processes work. It is also known to occur in standard models of weather, used in weather forecasting, which exhibit sensitivity to initial conditions in some circumstances [21], and this has changed the way forecasts are made and presented. We cannot test real weather by running it a second time with the entire planet unchanged except for one flap of a butterfly's wing, but tests on simpler fluid systems support the view that in principle, real weather is sensitive to initial conditions. However, real weather is subject to other influences, and this is not necessarily the most important influence on weather and its unpredictability. Specifically, there is not just one butterfly.

Smale asked whether it can be proved rigorously that the Lorenz system has a chaotic attractor. Tucker [22] proved that it does, using a computer-aided but rigorous proof involving normal form theory and interval arithmetic. The complexity is real, not some numerical artefact.

There are several different technical definitions of ‘chaos’, covering a number of related but distinct phenomena [23].

## 5. Tossing a coin

The archetypal model for statistical uncertainty is the flip of a coin. The earliest textbook of probability theory, Bernoulli's *Ars conjectandi* [24,25], devotes much space to analysing what happens when a coin (fair or biased) is tossed a large number of times. This led to the formulation and proof of the law of large numbers: over a sufficiently large number of repetitions, the frequency with which the coin lands ‘heads’ almost surely tends to the probability that a single flip will give heads.

The irony is that the motion of a coin is an archetypal *deterministic* system. A spinning coin follows a highly predictable trajectory, determined by Newton's laws of motion. Its centre of mass travels along a parabola, and the rotation of the coin is regular and uniform. The spinning coin is not chaotic. So why do we use it as our standard icon for chance?

First, this view is traditional. Second, although the dynamics is non-chaotic, so there is no sensitivity to initial conditions, the end result—heads or tails—depends on initial conditions in a manner that is sensitive to the size of error that is likely in a real coin toss. The human hand cannot be controlled accurately. Model the toss using, say two variables: the initial upward velocity and the initial spin rate. Then, the space of initial conditions—phase space—can be subdivided according to the final resting state, heads or tails. If we colour initial conditions that lead to heads black, and those that lead to tails white, then phase space is filled with very narrow black and white stripes. A small error can shift an initial condition from a white stripe to a black one. Unlike the butterfly effect, an initial error does not grow exponentially fast. But if the error is larger than the typical spacing of the stripes, we cannot predict with certainty whether the outcome will be heads or tails.

However, that is not the full story. Unless the shape or density of the coin introduces bias, we expect it to be ‘fair’: heads and tails should occur with equal probability. The analysis so far does not explain that, and in fact there is a sense in which it is false. Diaconis *et al.* [26] have analysed the dynamics of flipping a coin. Their main conclusion is that even when the coin spins rapidly, it lands the same way up as it started 51 per cent of the time, and the other way up 49 per cent. This discrepancy becomes significant in practice after about a quarter of a million flips.

The reason is that the dynamics of the coin depends on the angle between the coin's axis of rotational symmetry (through its centre at right angles to the plane of the coin) and the axis about which the coin is spinning (figure 4). If these axes coincide, and the coin is tossed vertically, then it remains the same way up throughout its motion. On the other hand, if the spin axis is along a diameter of the coin, it alternates heads up, tails up, very rapidly and we would expect it to be fair if it is spinning sufficiently fast. Most coins will lie between these two extremes, and this introduces a small bias, on average over many tosses. The authors invented a coin-tossing machine to test the results, and confirmed them. For a non-technical description, see [27].

So why do we *really* think that a coin toss is random? The answer has nothing to do with the dynamics of the toss. What matters is what happens to the coin when it is in our pocket before we take it out, slap it on top of our thumb, and flip it. Provided we do not cheat by selecting which way up it starts, the initial configuration—heads or tails up—is randomized by the complex and wildly unpredictable movements that the coin experiences between tosses. Even if it always landed the same way up as it started, we would still have a fair coin.

## 6. Heteroclinic connections

In figure 1, the saddle point acts as an attractor along two incoming trajectories, and as a repeller (like an attractor but with arrows reversed) along two outgoing trajectories. These trajectories form the *stable* and *unstable manifolds* of the saddle point. If the system's initial condition is close to the stable manifold, but not exactly on it, then the state will approach the saddle point, remain near it for a relatively long period of time, and then exit along a trajectory close to the unstable manifold.

The existence of saddle points, and more complex generalizations such as ‘chaotic saddles’, opens the way to a plethora of wild and unpredictable kinds of behaviour. In particular, the unstable manifold of one saddle point may connect to the stable manifold of another. This situation is called a *heteroclinic connection*. If the connection joins a saddle to itself, we get a *homoclinic connection*. If there exist several heteroclinic connections, they can form a network, for example, a *heteroclinic cycle* (figure 5).

Although such a connection can be changed by a small perturbation, so it is not typical, it *is* typical in any system with one or more parameters, in the following sense. If, for some parameter value, the system has such a connection, and the equations are slightly perturbed, then the perturbed system has such a connection for some nearby parameter value. Since most models have many parameters, heteroclinic connections are robust and the associated phenomena are often found in applications. The uncertainty manifests itself in unexpected switching behaviour between apparently stable states.

If a system has a heteroclinic connection, then it will appear to be in some attractor state, but after a fairly long time, it will suddenly and unpredictably switch to another apparent attractor, whose dynamics may be very different. On geological time scales, the Earth's climate appears to switch between two states: warm and cold.

Meteorologists often observe ‘blocked’ patterns of weather, in which the atmosphere remains in much the same state for a week or more, before suddenly switching to another long-lived pattern. The persistent snowstorms of early 2010 are a recent example; common examples on a larger scale are the North Atlantic Oscillation and the Arctic Oscillation, in which periods of east–west flow in these regions alternate with periods of north–south flow. Cromellin [28] provides evidence that blocked atmospheric regimes may be related to a heteroclinic cycle in large-scale atmospheric dynamics. This suggestion explains the apparent stability of the blocked regimes, and the sudden changes that occur when they are unblocked. Heteroclinic cycles are characterized by occasional bursts of activity punctuated by lengthy periods of torpor. The torpid states are predictable: they occur when the system is near an equilibrium. The uncertainty lies in predicting when the torpor will cease.

Cromellin has tested his theory using data on the Northern Hemisphere for the 1948–2000 period. Subject to caveats about the size of datasets used, the analysis provides evidence for the possible existence of a preferred dynamic cycle connecting various regimes of blocked flow in the Atlantic region. The dynamical variable is the deviation of the atmospheric flow from its mean state, which we will refer to simply as the flow. The cycle begins with north–south flows over the Pacific and North Atlantic. These merge to form a single east–west Arctic flow. This in turn elongates over Eurasia and the west coast of North America, leading to a predominantly north–south flow. In the second half of the cycle, the flow patterns revert to their original state, but following a different sequence. The complete cycle takes about 20 days. The results suggest that the North Atlantic and Arctic Oscillations are related—each may act as a partial trigger for the other.

There is a vast catalogue of exotic phenomena associated with heteroclinic connections. Possibilities include:

— connections among chaotic saddles, see §8;

— connections between chaos and non-chaos;

— cycling chaos: the system exhibits bursts of chaos in a predictable order as it cycles round several chaotic saddles in a closed loop [29], see §8; and

— complex networks of connections with ‘random’ switching [30,31].

Figure 6 shows a network of heteroclinic connections arising in an electrochemical system, from Kiss *et al.* [32]. The network is shown in black, the dots are saddles and the coloured region shows an attractor of the system corresponding to a trajectory near the network.

## 7. Basins of attraction

Sensitivity to initial conditions generates uncertainty about the trajectory of the system within a known attractor, given a point on that trajectory. A further level of uncertainty arises when there are several attractors. Now a basic question is: for given initial conditions, which attractor does the system converge towards? The associated geometry focuses on basins of attraction. The *basin* of an attractor is the set of initial conditions in phase space whose trajectories converge to that attractor.

The basins divide phase space into regions, one for each attractor. In simple cases, these regions have simple boundaries, like frontiers between countries on a map (figure 7). Here, the main uncertainty about the final destination arises only for initial states near these boundaries. However, the topology of the basins can be much more complicated, creating uncertainty for a wide range of initial conditions.

### (a) Wada basins

A natural example of more complex basin boundaries occurs in the Newton–Raphson method, which is a time-honoured numerical scheme for finding the zeros of a function *f*, that is, the solutions *x* of the equation *f*(*x*)=0. The Newton–Raphson method can be interpreted as a discrete dynamical system,
Here, *f*′ is the derivative of *f*. An initial value *x*_{0} is chosen, and successive *x*_{t} are computed in turn. If the sequence converges, whatever it converges to must be a solution of the equation *f*(*x*)=0. If we think of the procedure as a dynamical system, the zeros of *f* are its attractors. Since *f* may have several zeros, we can ask what their basins of attraction are.

Consider the example *f*(*z*)=*z*^{4}−1, where *z*=*x*+i*y* is a complex number. Here, we know that the zeros are *z*=1,*i*,−1,−*i*. Apply Newton–Raphson with initial conditions in the square −3/2≤*x*≤3/2,−3/2≤*y*≤3/2. Figure 8 shows the result [33].

Here, the four basins are so complex topologically that they all have the same boundary. This requires the common boundary to be very complex—a simple curve can adjoin at most two basins. In fact, the common basin boundary is fractal. Any basin of this kind is called a *Wada basin*, after the first topological example of three regions in the plane with a common boundary, the Lakes of Wada (figure 9). This was published by Kunizo Yoneyama [34] in 1917, who credited it to his teacher Takeo Wada. The behaviour of a system is unpredictable for initial conditions near the boundary of a Wada basin because very small perturbations can move the state to a different basin [35].

Wada basins can occur in physical systems: an example arises when light is reflected in four touching spheres arranged in a tetrahedron [36]. Here, the four colours correspond to the four openings between spheres, through which the light ray eventually exits (figure 10).

### (b) Riddled basins

In the example of the Newton–Raphson method, the complex boundary occupies a relatively small part of phase space. However, distinct basins can be intertwined to such an extent that this type of uncertainty arises for most initial points. The name for this phenomenon is *riddled basins*. Moreover, many standard and important physical systems exhibit this type of behaviour. An example is the forced damped pendulum, a pendulum driven by a time-periodic force at its pivot, and subject to a small amount of friction. In this system, attractors are various time-periodic states. Kennedy & Yorke [16] have shown that the basins of attraction are densely interwoven (figure 11).

Palmer [37] has used riddled basins to model quantum indeterminacy by deterministic dynamics. Here, the competing attractors correspond to alternative quantum states, such as spins, and the complex topology of the riddled basins is the source of apparent randomness. It is often thought that Bell's theorem [38] rules out all types of ‘hidden variable’ theory for quantum indeterminacy. However, Palmer's suggestion, whether physically correct or not, shows that certain types of hidden variable theory are consistent with Bell's theorem.

## 8. Bifurcations

Bifurcation theory studies changes in the type and number of attractors (and other dynamically important objects) as parameters are varied. Bifurcations usually show up as a sudden change in the system's behaviour resulting from gradual changes to its parameters—for example, external influences. The rather overworked expression ‘tipping point’ refers to the simplest effect of this kind, when some parameter crosses a critical threshold leading to an abrupt change in the system's behaviour. However, bifurcation phenomena can be much richer and more complex than simple thresholds.

Until recently, most of the literature dealt with bifurcations involving equilibria and periodic cycles, and we omit these here. However, there is a growing body of work where the attractors are chaotic, and these bifurcations introduce new types of uncertainty.

### (a) Bubbling

This type of bifurcation occurs when two identical chaotic systems are coupled together. Here, a key feature is *synchrony*, which occurs when both systems behave identically at all times. The synchronous dynamics can be regular or chaotic, and asynchronous dynamics is also possible. Bifurcations occur when small perturbations destroy the synchrony, either temporarily or permanently. In the temporary case, one possibility is *bubbling* [39,40]. Here, the synchronous dynamic is chaotic, but its overall stability depends on the transverse dynamic, that is, the effect of small perturbations that destroy the symmetry.

In coupled systems of this type, with state variables *x*(*t*),*y*(*t*) for the two systems, the states are synchronous if and only if *x*(*t*)=*y*(*t*) for all times *t*. That is, the trajectory lies on the *synchrony subspace*
inside phase space. The synchrony subspace is *flow invariant*: if an initial point (*x*(0),*y*(0)) lies in *Δ*, then the entire trajectory (*x*(*t*),*y*(*t*)) lies in *Δ* for all times *t*. In consequence, the synchronous dynamics obeys its own *restricted* differential equation, obtained by identifying synchronous variables with each other. In order for the synchronous state to be stable, it must in particular lie on an attractor for the restricted system on *Δ*, meaning that after any sufficiently small synchrony-preserving perturbation, the trajectory returns to the attractor. However, this condition is not sufficient: in order to be an attractor for the full system, the synchronous state must also be stable with respect to synchrony-breaking perturbations, which push it away from *Δ*. This second condition depends on *the transverse dynamic*: how the trajectory behaves in the direction at right angles to *Δ*. If this transverse dynamic causes the system to return to a synchronized state, then the loss of synchrony is temporary. If not, it may be permanent, or the system may eventually resynchronize after a lengthy departure from synchrony.

In the bubbling bifurcation, the departure from the synchronized state is temporary, but persistent. The loss of synchrony appears to be random. Bubbling arises because the transverse dynamic can be attracting in some places, but repelling in others. Broadly speaking, the overall effect of the transverse dynamic is its average over the synchronous attractor, with respect to the so-called Sinai-Bowen-Ruelle (SBR) measure [41], also known as the SRB measure. The SBR measure can be interpreted as the probability that a trajectory lies near some given point in the attractor—it measures the local density of the attractor near any given point. It is the ‘natural invariant measure’ for the system, meaning that the dynamics preserves the probabilities.

However, there are other invariant measures. Chaotic attractors typically contain unstable periodic states: indeed, under suitable technical hypotheses [7], these are dense in the attractor, and it is conjectured that these hypotheses can be relaxed considerably. A periodic state of this kind has its own invariant measure, which assigns zero probability to all points other than those on the periodic trajectory. Since the measure differs from the SBR measure, the average with respect to that measure may be different. In particular, if the synchronous state as a whole is attracting on average (with respect to SBR measure) but the embedded periodic state is transversely repelling on average, then the state ‘bubbles’ away from synchrony repeatedly, but is almost always attracted back to synchrony after a small departure (figure 12).

Two related types of bifurcation exist, each causing its own kind of uncertainty: *on–off intermittency* and *blowout* bifurcations [42–46]. A common theme is the occurrence of riddled basins: in this case, the basin of attraction is broken up by infinitely many narrow ‘tongues’ where the state can escape from the synchronous attractor, either temporarily or permanently.

### (b) Heteroclinic chaos

Changing a parameter can cause a bifurcation in which chaotic attractors become saddles, and the unstable directions can resemble riddled basins, so that the state moves between different saddles in an unpredictable way. An example occurs in a network of three symmetrically coupled Rössler systems. Rössler [47] introduced one of the earliest systems with a chaotic attractor, having three variables *x*,*y*,*z*. The network is a slight modification of his equations in which one of the variables, *z*, is replaced by three new ones, *z*_{1},*z*_{2},*z*_{3}. The equations are

Setting *z*_{1}=*z*_{2}=*z*_{3}=*z* yields the original Rössler equations, so the restricted system on the corresponding synchrony space *Δ* is the usual Rössler system.

Figure 13 shows the standard Rössler attractor in (*a*), and a three-dimensional projection of the corresponding dynamics in the network showing *x*,*y*,*z*_{1} in (*b*). Departures from the smooth shape of the standard Rössler attractor are caused by transitions in the variables *z*_{1},*z*_{2},*z*_{3} caused by heteroclinic connections between three distinct chaotic saddles. In these saddles, the variables are synchoronized in pairs: *z*_{1}=*z*_{2}, *z*_{1}=*z*_{3}, or *z*_{2}=*z*_{3}. Figure 14 plots the dynamics in a projection of the space with coordinates *z*_{1},*z*_{2},*z*_{3} onto the plane, in which the three *z*-coordinates correspond to the three straight lines. When the system is near a chaotic saddle, the trajectory lies very close to one of these lines. The curved sections are transitions from one saddle to another.

### (c) Cycling chaos

*Cycling chaos* is similar, but the transitions move the trajectory from one chaotic saddle to another in a fixed order [29]. So here the order in which the states switch is predictable, and so is the ‘texture’ of each state, but the timing of the transitions is unpredictable (figure 15).

### (d) Colliding attractors

Dellnitz *et al.* [48] and Dellnitz [49] have studied a type of bifurcation in which two or more chaotic attractors collide (figure 16). Such collisions are typical in systems with symmetry, but can arise in asymmetric systems if there exist adjustable parameters that can be ‘tuned’ to make the bifurcation happen.

Prior to the collision, the system is confined to one of the colliding attractors, and the existence of the others may not be observed. However, when the attractors grow, collide and merge, the system suddenly explores a much larger region of phase space, the combined attractor. Just after the attractors have merged, there is an ‘overlap’ region, or a ‘bottleneck’, through which a trajectory must pass in order to switch its behaviour from (something close to) one of the original separate attractors, to (something close to) one of the previously unobserved attractors.

Dellnitz's analysis shows that the actual sequence of events is more complicated: as well as the merging attractors, there must be an unstable invariant set that simultaneously comes into contact with all of the attractors (figure 17). So the collision is a heteroclinic phenomenon involving this hidden unstable invariant set.

In practice, when observing such a system, it begins by following a trajectory with a texture that corresponds to a single attractor before the merger. Even after the attractors have merged, it may continue to behave in much the same manner, as if it has not yet ‘noticed’ that the attractor has become much larger. After some random time, however, its trajectory will wander into the bottleneck region, and it will then suddenly begin to explore the enlarged attractor (figure 18).

This type of phenomenon is one possible reason for the occurrence of ‘black swan’ events—unexpected changes in behaviour, radically different from anything observed beforehand.

## 9. Handling these uncertainties

The topic of the meeting to which this paper is contributed is ‘handling uncertainty’ and not just ‘uncertainty’, so we end with a few remarks on what can be done to detect, exploit or mitigate the effects of the above sources of uncertainty.

### (a) Chaotic attractors

In explicit models, numerical methods can be used to locate attractors and approximate their shapes. There also exist good methods for extracting chaotic dynamics from real data. The first such technique was the Packard–Takens method [50,51]. Given a time series of data *x*_{1},*x*_{2},*x*_{3},…, choose a fixed length *m* and plot the sequence of shifted vectors
Then, the Takens embedding theorem [51] states that ‘almost always’ the resulting set is topologically equivalent to the attractor. In practice, care must be taken with this method, fairly large datasets are required and there are technical refinements, notably the use of principal component analysis [52]. The method has been widely used in experimental science, and there are industrial applications, notably the fractal materials (FRACMAT) quality control test for spring wire [53].

Chaos is not always an obstacle—it can also be an opportunity. In *chaotic control*, sensitivity to initial conditions is used to derive efficient methods for constraining a dynamical system to maintain it in an unstable state, notably the Ott, Grebogi, Yorke (OGY) method [54]. In this method, small perturbations to the system are used to keep it close to an unstable periodic orbit, and the method works because such a perturbation, if suitably chosen, can have a large effect on the subsequent dynamics. The basic technique is to choose the perturbation so that the state of the system remains close to the stable manifold of the periodic state concerned—that is, a specific set of trajectories that are attracted to that state. An application to intelligent heart pacemakers can be found in Garfinkel *et al.* [55].

### (b) Heteroclinic connections

Heteroclinic behaviour in models can be found numerically [56].

Cromellin's analysis of blocked weather patterns, discussed in §6, uses the following method. First, he calculates ‘empirical eigenfunctions’, or common flow patterns, in the atmosphere, derived from real data, using principal component analysis. In this technique, the actual flow is approximated by the closest possible combination of a set of independent basic patterns. The complex equations of meteorology are thereby transformed into a system of differential equations, whose variables represent the amplitudes of the component patterns. Heteroclinic connections can be interpreted as the persistence of various apparently stable flow patterns, punctuated by rapid switches from one such pattern to another.

### (c) Riddled basins

There exists a quantitative measure of the extent to which a basin is riddled that describes how the uncertainty about which basin a point occupies varies as the point approaches the basin boundary [17].

### (d) Bubbling

Invariant measures can be found numerically, and average transverse dynamics can be computed, see [39,40].

### (e) Colliding attractors

Experimentally, or numerically in a model, a system that in principle may occupy one of several distinct attractors can sometimes be ‘kicked’ into a new attractor by applying a *large* perturbation. This must be large enough to move the trajectory into a different basin of attraction.

In the symmetric case, where collisions are especially common, there are methods for analysing chaotic time series and determining the symmetry of the attractor [57]. These methods have been used in fluid flow problems, for example.

The Packard–Takens method of phase space reconstruction and its refinements can detect changes in the topology of the attractor as a parameter is varied.

## Footnotes

One contribution of 15 to a Discussion Meeting Issue ‘Handling uncertainty in science’.

↵1 Like finding that bloody butterfly whose flapping wings cause all these storms we have been having lately and getting it to stop.

- This journal is © 2011 The Royal Society