## Abstract

Statistical complexity measures are used to quantify the degree of complexity of the delayed logistic map, with linear and nonlinear feedback. We employ two methods for calculating the complexity measures, one with the ‘histogram-based’ probability distribution function and the other one with ordinal patterns. We show that these methods provide complementary information about the complexity of the delay-induced dynamics: there are parameter regions where the histogram-based complexity is zero while the ordinal pattern complexity is not, and vice versa. We also show that the time series generated from the nonlinear delayed logistic map can present zero missing or forbidden patterns, i.e. all possible ordinal patterns are realized into orbits.

## 1. Introduction

Time-delayed systems have attracted wide interdisciplinary interest, as delays, which arise from finite response and/or propagation times, are ubiquitous in nature. Examples can be found in small-scale systems, such as nano-optical resonators and micro-ring lasers, where many effects depend on the finite speed of propagation of light along the cavity [1]; in large-scale systems, such as the brain, where time delays play central roles in the dynamics of neural networks [2]; in epidemic models, where delays are used to describe various immunity degrees [3,4]; and in vehicular traffic models, where delays arise owing to finite reaction times etc. [5].

A delay adds memory to a system, as the future state of the system depends on the past states over a finite interval of time. Moreover, the presence of delays often results in patterns of oscillatory behaviour that repeat from time to time and that strongly depend on the initial conditions, multi-stability being a well-known feature in delayed systems [6,7]. In addition, time delays add new degrees of freedom, which can result in the delayed system having a high-dimensional dynamics similar to that of spatio-temporal systems [8–12].

These delay-induced features (memory, multi-stability, high dimensionality) are characteristic of complex systems [13]. While phase portraits, power spectra, Lyapunov exponents and fractal dimensions are very useful tools for characterizing low-dimensional dynamics, they can fail to capture the patterns and the correlations that are often present in high-dimensional systems.

The interplay of delay and nonlinearity plays a central role in self-organization and complex phenomena that arise in small-scale (in interacting many-particle systems, such as atoms, molecules and multi-electronic systems, in mesoscopic systems such as quantum dots, etc.) as well as in large-scale systems (cellular automata, neural networks, social sciences, etc.). Meaningful measures that capture the complexity of these systems can shed light on the underlying nonlinear interactions. While we all have an intuitive notion about complexity, as an intermediate situation between full order (certainly) and full randomness (complete unpredictability), there is no general definition of complexity, and its quantitative characterization is a challenging task that has received considerable attention. In the last three decades, many different complexity measures have been proposed that capture various aspects of complex systems, and that are based on data compression algorithms, optimal predictability, recurrence plots, symbolic analysis, wavelet analysis, etc. [14–26].

Complexity measures have been shown to be powerful tools for detecting dynamical changes in time series from epileptic patients [27] and in speech signals [20,28], for distinguishing chaotic signals from stochastic ones, for distinguishing among different degrees of stochasticity [29], for quantifying stochastic and coherence resonances [30], and for classifying spatio-temporal patterns [31,32] and neural networks [33,34] etc.

From the fundamental and applied perspectives, the quantification of patterns and subtle correlations present in nano- and mesoscopic systems is of broad interest. To mention just a few other examples, at the quantum level, complexity measures computed using Hartree–Fock wave functions for neutral atoms in the position and momentum spaces allow us to show the subshell pattern of the periodic table [35]; complexity measures have been used to quantify the disequilibrium associated with the quantum-mechanical probability density of hydrogenic stationary states, allowing for a classification of orbital complexity in terms of quantum numbers [36]. At the mesoscopic level, complexity measures allow us to differentiate between protein-coding and non-coding RNA segments in DNA sequences [37].

Although time-delayed nonlinear systems have received much attention, no systematic quantification of the complexity exhibited by these systems has been done so far to the best of our knowledge. In this article, we carry out the first step towards this by considering a discrete-time nonlinear dynamical system, specifically the delayed logistic map, and study the relationship between nonlinearity, time delay and the complexity of the dynamics.

Our motivation for focusing on the logistic map is twofold: first, because the logistic map is a popular well-known example of how chaotic dynamics and universal scaling laws arise in simple nonlinear dynamical systems and delayed logistic maps; and, second, because one can perform long simulations, even in the presence of long delays, with a great reduction in computational time and memory requirements, when compared with delay-differential equation models. At the small scale, nonlinearities are often fundamental to understand a system’s behaviour, and the delayed logistic map can therefore be useful for studying the interplay of delay and nonlinearity.

Several authors have analysed the complexity of the logistic map, employing various complexity measures [20,22,38,39], but no systematic study of the delayed logistic map has been done.

We consider two ways of including a delay in the logistic map: with linear feedback (‘echo’ type, referred to as ‘model A’ [40]) and with nonlinear feedback (referred to as ‘model B’ [41]) .

In order to quantify the complexity of the time series {*x*(*t*)} generated from the delayed logistic map, we employ two information theory tools: the normalized Shannon’s entropy, *H*, and the MPR-statistical complexity, *C*, proposed by Martín *et al*. [24]. To compute *H* and *C*, one needs to associate with the time series {*x*(*t*)} a probability distribution function (PDF) that characterizes the time series. Here, the PDF is calculated in two different ways: one is the ‘conventional’, frequency count method, by which the PDF is calculated from the histogram of the values of {*x*(*t*)}, and, thus, it does not take into account time correlations arising from repeated oscillatory patterns present in the time series; the second way is by first transforming the time series {*x*(*t*)} in a sequence of ‘ordinal patterns’ [20,42], and then computing the PDF of the ordinal patterns. The PDF constructed in this way characterizes the temporal correlations in the time series {*x*(*t*)}, but neglects the absolute magnitude of the oscillations in the time series.

We show that the normalized entropy and the statistical complexity calculated with histogram-based PDFs, *H*_{his} and *C*_{his}, and with ordinal pattern-based PDFs, *H*_{BP} and *C*_{BP}, give complementary information about the complexity of the delayed logistic map: we find parameter regions where *C*_{his}≈0 while *C*_{BP}≠0, and also regions where *C*_{his}≠0 while *C*_{BP}≈0. These results can be understood by looking at the oscillations present in the time series generated by the delayed logistic map.

This article is organized as follows. The delayed logistic map with linear and nonlinear feedback is introduced in §2, where we also discuss the measures employed, the normalized Shannon entropy and MPR-statistical complexity, and the ordinal pattern methodology. In §3, we present the results, and analyse the complexity of the time series generated by the delayed logistic map as a function of the delay time and the feedback strength. We consider the value of the logistic parameter such that the dynamics of the undelayed map is periodic (an orbit of period 2) and, also, a value of the logistic parameter such that the dynamics of the undelayed map is chaotic. We show that, in both cases, the delay induces a highly complex dynamics, but it has different features that can be captured by the complexity measures. We also show that for long delays there is saturation as the normalized Shannon entropy and the statistical complexity remain nearly constant. Our conclusions are presented in §4.

## 2. Methods

### (a) Models and information theory quantifiers

We consider two schemes for including a time delay in the logistic map. The linear scheme (model A) is
2.1
and the nonlinear scheme (model B) is
2.2
In equations (2.1) and (2.2), *f*(*x*) is the logistic map, *f*(*x*)=*rx*(1−*x*) with parameter *r*, *t* is a discrete time index, *η* is the strength of the feedback and *τ* is the delay time, *τ*=0,1,2,…. To generate a trajectory from equations (2.1) and (2.2) one needs, as the initial condition, not only *x*(0) but also *τ* past values of *x*, i.e. one needs to specify *x*(−*τ*),…,*x*(0). Thus, equations (2.1) and (2.2) are *τ*+1-dimensional dynamical systems.

### (b) Information theory quantifiers

Let us now present the information theory measures employed to analyse a time series {*x*(*t*),*t*=1,…,*M*} generated from either model A or model B.

Statistical complexity measures can be classified into three categories [19]: (i) they grow with increasing disorder, (ii) they are small for large amounts of either order or disorder, and display a maximum at some intermediate stage, and (iii) they grow with increasing order. The MPR-statistical complexity employed here belongs to the second family of complexity measures, and vanishes in the two extreme situations of perfect order and complete randomness, as the two situations possess no structure to speak of.

The MPR-statistical complexity is a modification of the complexity measure proposed by Lopez Ruiz, Mancini and Calbet (LMC-statistical complexity, [17]). The LMC-statistical complexity is the product of the normalized Shannon entropy multiplied by a disequilibrum, *C*=*H*⋅*Q*. The disequilibrum, *Q*, is a function of the probability distribution characterizing the system and captures the system’s ‘architecture’, being different from zero if there are ‘privileged’, or more probable, states among the accessible ones.

Given a PDF, *P* associated with the time series {*x*(*t*)}, the normalized Shannon entropy is
2.3
where , with and is the equilibrium uniform distribution.

The disequilibrium functional, *Q*[*P*,*P*_{e}], for the complexity measure proposed by LMC, is the Euclidean ‘distance’ from *P* to *P*_{e}. Martín *et al*. [24] redefined the disequilibrium, replacing the Euclidean distance by the Jensen–Shannon divergence, which is the symmetric form of the Kullback–Leiber relative entropy [43]. The main idea underlying this notion of distance is that of adequately taking into account statistical fluctuations inherent to any finite sample. The concept of statistical distance originates within the field of quantum mechanics [44]: as a result of the inherent statistical errors, the observed frequencies of occurrence of the various possible outcomes typically differ from the actual probabilities, with the result that, in a given fixed number of ‘trials’, two ‘states’ are indistinguishable if the statistical difference between the actual probabilities is smaller than the size of a typical fluctuation.

Therefore, the MPR-statistical complexity calculated as in Martín *et al*. [24] and Rosso *et al*. [29] is
2.4
where *Q*_{J}[*P*,*P*_{e}]=*Q*_{0}⋅{*S*[(*P*+*P*_{e})/2]−*S*[*P*]/2−*S*[*P*_{e}]/2}, with *Q*_{0} a normalization constant (0≤*Q*_{J}≤1) that reads .

### (c) Ordinal patterns methodology

Various methods can be employed to define a PDF that characterizes the time series {*x*(*t*),*t*=1,…,*M*}. The standard procedure is by computing the histogram of the continuous variable *x*(*t*), 0≤*x*≤1, which defines the PDF *P*(*x*). This can be useful, but it has the drawback that time correlations are lost, i.e. the *causal order* by which the different values appear in the sequence {*x*(*t*),*t*=1,…,*M*} is not taken into account.

An alternative approach is based on computing the PDF of ‘ordinal patterns’. The method, originally proposed by Bandt & Pompe (BP, [20]), is based on the comparison of neighbouring values in the time series. The method is powerful but rather new, and, therefore, we present in this section a discussion of the main ideas.

The first step is to divide the time series {*x*(*t*),*t*=1,…,*M*} into *M*−*D* overlapping vectors of dimension *D*. Then, each element of a vector is replaced by a number from 0 to *D*−1, in accordance with the relative strength of the element in the ordered sequence (0 corresponding to the shortest and *D*−1 to the longest value in each vector). Each vector then has an associated ‘ordinal pattern’ composed of *D* symbols.

For example, with *D*=3, the sequence {*x*(*t*),*x*(*t*+1),*x*(*t*+2)}={5,−1,10} gives the ordinal pattern (1 0 2), as *x*(*t*+1)<*x*(*t*)<*x*(*t*+2).

The number of different ordinal patterns of dimension *D* is *D*! and can be labelled with an index *n*=1,…,*D*!. For *D*=3, the six possible ordinal patterns can be labelled as (0 1 2)=1, (0 2 1)=2, (1 0 2)=3, (1 2 0)=4, (2 0 1)=5 and (2 1 0)=6.

In this way, the time series of *M* real numbers {*x*(*t*),*t*=1,…,*M*} is replaced by a sequence of *M*−*D* integers, {*n*(*t*),*t*=1,…,*M*−*D*,1≤*n*(*t*)≤*D*!}, representing the ordinal patterns.

By counting the number of times a pattern *n*_{i} appears in the sequence {*n*(*t*),*t*=1,…,*M*−*D*}, one can compute the PDF of the ordinal patterns. Since the number of possible ordinal patterns is *D*!, to be statistically correct one must have a long time series, such that (*M*−*D*)≫*D*!.

Figure 1 presents the results of analysing two trajectories generated from model A, starting with random initial conditions (the details of the simulations will be presented §3). In figure 1*a,* we show a small part of one trajectory, and, as examples, we mark with squares three consecutive values that give the ordinal pattern (1 2 0)=4, and with circles three consecutive values that give the ordinal pattern (2 0 1)=5. Figure 1*b* displays the PDF of the *D*=3 ordinal patterns (for comparison, figure 1*c* shows the histogram-based PDF of the same time series) and one can see that the ordinal pattern (2 1 0)=6 has probability zero. The patterns that are not observed in a time series are referred to as ‘missing patterns’ and will be discussed in §3. It has been shown that piecewise monotone one-dimensional maps always have ‘forbidden patterns’, which are permutations that do not appear in any trajectory obtained by iterating the map, starting from any point in the interval [45,46]. Basic or minimal forbidden patterns, which are those for which any proper consecutive subpattern is allowed, have been studied for various maps [47]. Forbidden patterns can, therefore, be used to distinguish random sequences, where every permutation appears with some probability, from deterministic sequences produced by iterating a one-dimensional map [48].

In figure 1*d*–*f*, we present results for a second trajectory, obtained with a different realization of the random initial conditions. One can notice that the ordinal pattern (2 1 0)=6 appears in this time series (see the consecutive values indicated with circles in figure 1*d*). Moreover, comparing the PDFs of the two trajectories one can notice that both the ordinal pattern PDF (figure 1*b*,*e*) and the histogram-based PDF (figure 1*c,f*) are very different, suggesting that the trajectories evolve in different attractors. As was discussed in §1, multi-stability can be expected as it is a typical feature of time-delayed systems.

The ordinal pattern method of associating a PDF with the time series {*x*(*t*),*t*=1,…,*M*} has the main advantage that it takes into account the time correlations in the sequence of values; however, it also has two drawbacks. First, the information about the absolute value of the variable *x*(*t*) is neglected, as it only takes into account the relative strength of *x*(*t*) in the sequence of *D* consecutive values. Second, equal values in the time series are not accounted for, because such values appearing in the sequence, either in a periodic or in a non-periodic way, are neglected. It is in principle possible to use a small random perturbation to break the equality; however, this can be a strong distortion if there is a large number of consecutive equal values [26].

## 3. Results

In this section, we present the results of calculating the normalized Shannon entropy and the MPR-statistical complexity of the time series generated from the delayed logistic map, employing the two methods discussed previously for associating a PDF with the time series: one is referred to as ‘histogram-based’, where *H*_{his} and *C*_{his} are calculated according to equations (2.3) and (2.4), with *P* computed in terms of the histogram of values of *x*(*t*); the other one is referred to as ‘ordinal pattern based’, where *H*_{BP} and *C*_{BP} are calculated according to equations (2.3) and (2.4), but with *P* computed from ordinal patterns (BP methodology).

Time series of length *M*=10^{6} were generated from models A and B, with initial conditions [*x*(−*τ*),…,*x*(0)], such that *x*(−*τ*) is chosen randomly in the interval (0,1) and then the map evolves without feedback in the first *τ* iterations. In each run, the first 2×10^{6} iterations were disregarded in order to let transients disappear.

Figure 2 for model A and figures 3 and 4 for model B display results of simulations for varying delay time, *τ*, and feedback strength, *η*. In figures 2 and 3 the logistic parameter is *r*=3.2, and, thus, the dynamics of the non-delayed map is periodic, in a limit cycle of period 2. For the linear feedback model A, *r* and *η* cannot be too large in order to have trajectories that remain in the interval [0,1]; large values of *r* or *η* result in run away trajectories. With nonlinear feedback, model B, one can explore a wider parameter region. In figure 4, which is for model B, we choose *r*=4.0, and, thus, the dynamics of the undelayed logistic map is chaotic.

Panels (*a,b*) of figures 2–4 display typical bifurcation diagrams when (*a*) *τ* or (*b*) *η* increase. It can be seen that the delay induces complex dynamics, and it can also be seen that the orbit of period 2 is stable for *η* small and *τ* even, while the fixed point is stable for *η* and *τ* small and *τ* odd.

Panels (*c*–*e*) and (*f*–*h*) of figures 2–4 display the averaged values of the normalized Shannon entropy and of the statistical complexity respectively: panels (*c*,*f*) correspond to *H* and *C* calculated with the BP method (ordinal patterns of dimension *D*=6); panels (*d*,*g*) correspond to *H* and *C* calculated with the histogram method. In order to check for delay-induced multi-stability, for each set of parameters (*η*, *τ*), 50 trajectories were generated with different random initial conditions. We computed the standard deviations of *H* and *C* (*σ*_{H,his}, *σ*_{H,BP}, *σ*_{C,his}, *σ*_{C,BP}), and considered that multi-stability occurred if any *σ* value was larger than a certain threshold. In figures 2–4, panel (*e*) displays the regions of multi-stability. Also in figures 2–4, panel (*f*) displays the average number of forbidden patterns, in the sense that ‘missing’ patterns do not appear in the time series.

In figures 2–4 one can see that the normalized entropy and the statistical complexity vary with *η* but tend to saturate with *τ*; for *τ* longer than a few units, *H* and *C* remain nearly constant with increasing *τ*. This saturation effect is seen when *H* and *C* are computed with histograms and with ordinal patterns, and is in agreement with the behaviour seen in other time-delayed systems. For example, in the Mackey and Glass model for blood production [49], in the Ikeda ring-cavity model [50] and in the Lang and Kobayashi model for a semiconductor laser with optical feedback [11], with increasing delay a linear increase in the Lyapunov dimension is observed while the metric entropy remains nearly constant.

In the linear model A, multi-stability (figure 2*c*) occurs for large enough *η*; an example of two trajectories evolving in coexisting attractors was presented in figure 1. As discussed in relation to figure 1, these attractors can be distinguished by different PDFs, and they can have different ‘forbidden’ patterns. One can observe in figure 2*e* that the number of forbidden patterns tends to decrease with increasing *η* and also seems to saturate with increasing *τ*.

For small *η* and *τ*, the bifurcation diagrams shown in figure 2*a* reveal that the trajectories are either orbits of period 2 for even delays, or a stable point for odd delays. Because of the drawback of the BP method discussed in §2, i.e. that it does not take into account identical values, numerical noise can have a strong influence on the values of *H*_{BP} and *C*_{BP} in the parameter region where the trajectory evolves in a periodic orbit or stays at the fixed point, while it leaves rather unaffected *H*_{his} and *C*_{his}. For the non-delayed logistic map, the influence of noise was analysed by Bandt & Pompe [20], who showed that observational noise (noise added to the time series) causes only a small increase in entropy, while dynamical noise, added to *x*(*t*) during each step of the iteration, results in the entropy being a smoother function, approximating the entropy of the undisturbed time series for small noise level. For the delayed logistic map, it will be interesting to study the influence of noise, as the interplay of noise and delay could result in peculiar dynamical features.

For model B, in figure 3, which is shown for *r*=3.2 as in figure 2, one can see that, for strong nonlinear feedback (*η* close to 1), *H* and *C* present different behaviours, depending on whether they are computed with histogram-based PDF or with ordinal pattern-based PDF. For *η*≈1, *H*_{his}≈0 and *C*_{his}≈0 regardless of *τ*, while *H*_{BP}, *C*_{BP} >0.4. Multi-stability also occurs for even delays and *η*≈1.

The different behaviour of (*H*_{BP}, *C*_{BP}) and (*H*_{his}, *C*_{his}) can be understood when looking at the time series {*x*(*t*)}. Typical examples are presented in figure 5, where one can see that a peculiar delay-induced dynamics occurs: the orbit is periodically switching between two values, but the switching is such that, with a certain periodicity, a ‘switch’ is skipped or missed. This behaviour is robust and we have observed it for other values of *r*, such that the dynamics of the undelayed logistic map is periodic. The periodicity of the skippings seems to be related to both the delay time and the ‘natural’ period of the orbit of the undelayed logistic map. Clearly, time series such as those shown in figure 5 have very simple histogram-based PDFs, resulting in *H*_{his}≈0 and *C*_{his}≈0, but have structured ordinal pattern PDFs, resulting in *H*_{BP} and *C*_{BP} different from zero.

In figure 4, which is shown for nonlinear feedback and *r*=4, we also see a parameter region where *C*_{his}≈0 while *C*_{BP}≠0 and *H*_{his}≈1, whereas *H*_{BP}≠1, which can also be interpreted as being due to the fact that the PDF-BP has a non-trivial structure, which results in values of *H*_{BP} and *C*_{BP} different from 1 and 0, respectively, while the PDF histogram is closer to the uniform distribution, giving *C*_{his}≈0 and *H*_{his}≈1.

In figure 4*h* one can see that, for large enough *η* and delays longer than a few units, the number of forbidden patterns is equal to zero, i.e. all the possible ordinal patterns (for *D*=6, 720 patterns) are in fact present in the time series. This is in agreement with the fact that, in this region, *H*_{BP}≈1 and *C*_{BP}≈0, revealing that the PDF of the ordinal patterns is close to the uniform distribution.

As discussed in §2, Amigó and colleagues [42,45,46,48] have shown that, for deterministic maps, not all ordinal patterns can be materialized in the orbits, not even if the dynamics is chaotic, which makes these patterns ‘forbidden’. For example, in time series generated by the undelayed logistic map with *r*=4, if we consider patterns of length *D*=3, the ordinal pattern (2 1 0) is forbidden. That is, the pattern [*x*(*t*+2)<*x*(*t*+1)<*x*(*t*)] never appears in the time series [48].

We can see that in the delayed logistic map, with nonlinear feedback and *r*=4 (figure 4*h*), the number of forbidden patterns decreases with *η* and *τ*, and it can even be zero for large *η*. This is a very interesting observation that merits further investigation.

## 4. Conclusions

To summarize, we studied the complexity of the time series generated from the delayed logistic map, calculating the normalized Shannon entropy and the MPR-statistical complexity. We employed two methods for associating a PDF with the time series, the first one based on a ‘histogram PDF’, and the second one based on an ‘ordinal pattern PDF’. We found that these methods give complementary information about the complexity of the delay-induced dynamics: we found parameter regions where the histogram-based complexity is close to zero while the ordinal pattern complexity is not, and also parameter regions where the ordinal pattern complexity is close to zero while the histogram-based complexity is not. We have shown that the normalized entropy and the statistical complexity vary with *η* but tend to saturate with *τ*, i.e. for delays longer than a few units, *H* and *C* remain nearly constant with increasing *τ*. This is similar to the saturation that occurs in delay differential equations (e.g. in the Mackey and Glass model, in the Ikeda model and in the Lang and Kobayashi model): with increasing delay there is a linear increase in the Lyapunov dimension while the metric entropy remains nearly constant [11,49,50]. We analysed the time series generated from different realizations of the random initial conditions (and the same model parameters) and identified them as evolving on different attractors, which were characterized by different histogram PDFs and ordinal pattern PDFs. Multi-stability of coexisting attractors is also a well-known feature of delayed systems. We also showed that, for strong feedback and nonlinearity, the time series generated from the nonlinear delayed logistic map presents the very interesting feature that it has zero forbidden patterns, i.e. there are no missing or unobserved patterns and all possible ordinal patterns are effectively realized into orbits.

## Acknowledgements

C.M. acknowledges support from the Ministerio de Ciencia e Innovación, Spain, project FIS2009-13360 and the Agencia de Gestio d’Ajuts Universitaris i de Recerca (AGAUR), Generalitat de Catalunya, through project 2009 SGR 1168. O.A.R. acknowledges support from CONICET, Argentina, and CAPES, PVE fellowship, Brazil.

## Footnotes

One contribution of 17 to a Theme Issue ‘Nonlinear dynamics in meso and nano scales: fundamental aspects and applications’.

- This journal is © 2011 The Royal Society