## Abstract

This is a brief survey of quantum feedback control and specifically follows on from the two-day conference *Principles and applications of quantum control engineering,* which took place in the Kavli Royal Society International Centre at Chicheley Hall, on 12–13 December 2011. This was the eighth in a series of principles and applications of control to quantum systems workshops.

## 1. Introduction

Information is information, not matter or energy.

Wiener [1, p. 132]

Information is physical.

Landauer [2, p. 77]

Control theory deals with the optimization of performance of fixed systems through regulation by a controller. Its widespread applicability in classical technology is down to our ability to model dynamical systems and to work with abstracted concepts of information. Self-regulation requires feedback, and it was the use of centrifugal governors by James Watt to stabilize the rotational speed of steam engines that brought them to general attention in the industrial revolution. The first mathematical analysis of governors and their ability to stabilize took place a century later by Maxwell [3]. But it was left to Wiener [1] to develop the abstract theory of feedback to science and society, not just control engineering, and in the process, establish the field of cybernetics. His other key contribution to control was the extraction of signals in the presence of noise [4], with the same problem solved independently and earlier in the discrete time case by Kolmogorov [5,6]: this is the Wiener–Kolmogorov filtering theory, later to be superseded by the Kalman filter [7,8]. Information theory was likewise developed by Shannon to quantify information [9], and thereby demonstrate fundamental limits on signal processing and on information storage. It too quickly outgrew its origins in electrical engineering, and Shannon’s theory was immediately applicable to generic problems in communications systems, and indeed relevant to fundamental issues in physical modelling.

Following standard engineering terminology, the designated system is called the *plant*, and the system used to alter the plant dynamics is the *controller*. The control is *open loop* if we do not use feedback (figure 1); otherwise, it is *closed-loop* control.

In the closed-loop situation, the feedback may be entirely dynamical (i.e. the coupled plant and controller form a single dynamical system). We refer to this as *coherent-feedback control* (figure 2*a*). Alternatively, the feedback may be entirely information theoretic insofar as the controller gains information about the plant owing to measurement of the plant. This is *measurement-based control* (figure 2*b*).

Classically, the distinction is unimportant because the type of information is irrelevant. It is convenient to use block diagram techniques in classical control: here, each block corresponds to a physical system, and connections between the blocks indicate flow of information (either analogue or digital).

The situation in figure 3*a* depicts a plant connected to a controller. Mathematically, one would like to choose the controller so as to optimize some performance criterion for a given plant: the control problem is said to satisfy a *separation principle* if the design of the controller decouples into a filtering problem (how best to extract information from the plant output signal) and an actuation problem (how best to act on the plant based on this information).

At present, there are several competing suggestions as to what will be the most useful physical platform on which to develop quantum computation/information processing, each with significant advantages and drawbacks. It is not unlike the early days in the development of computing where one tried to see beyond the limitations of vacuum tube valves. Time will of course tell, but it is difficult to make predictions on what will emerge as the most appropriate practical quantum hardware. This leads us to the systematic study of quantum control theory. The central problem facing the construction of practical quantum computers is how do we make them robust! This is a question of quantum control engineering. To this end, we need the description of open quantum systems that dates from the mid-1970s. Key concepts introduced here include quantum irreversible systems models, quantum probability and stochastic processes, information theory and communications, estimation and filtering, computation and control to the quantum domain.

Elements of classical control have of course been used in quantum experiments for a long time; however, increasingly sophisticated experiments require a more detailed analysis, and often a fully quantum description. It would seem natural to try and develop a dedicated control theory for quantum systems that would ideally be of a universal character. This would of necessity take into account the features of the quantum world, of the measurement and estimation of quantum systems, the processing of quantum information, and the manipulation and actuation of quantum systems by their environment.

The development of quantum control, to date, is strikingly dissimilar to its classical counterpart: closed-loop constitutes only a relatively small fraction of theoretical work on quantum control, and is even rarer in experiment; also there is as yet no analogue of Wiener–Kolmogorov filtering, despite the fact that Kalman and nonlinear filtering for quantum Markov models are well developed. In these notes, we shall be interested in feedback that affects the dynamics of a given system. We should mention that there are other closed-loop strategies, notably adaptive feedback [10] in quantum chemistry, which uses learning algorithms but where control policies are applied to copies of the given system (molecules).

## 2. Quantum control

Landauer .. emphasized that computers are made out of physical apparatus and not out of Hamiltonians. And he made the point … that it does not constitute a serious model of computation if the imperfections in these apparatuses are not dealt with in the analysis.

DiVincenzo & Loss [11, p. 423]

The open-loop quantum control theory is principally concerned with Hamiltonian control, generically implemented through a controlled Hamiltonian of the form
2.1
with *H*_{0},*H*_{1},…,*H*_{n} fixed self-adjoint operators and *u*_{1},…,*u*_{n} the predetermined control functions, or policies. There are various definitions of controllability—i.e. the extent to which one may steer one state to another using appropriate choices of control policies. This is a bilinear control problem, and the characterization inevitably involves examining the Lie algebra generated by the skew-adjoint operators −i*H*_{α} with the commutator as the Lie bracket [12].

### (a) Quantum governors

In classical closed-loop control, the form of the information fed back in is not important and we do not make the distinction between coherent-feedback and measurement-based control. This reflects the universal nature of classical information: it does not matter if we use digital or analogue. In quantum feedback, coherent-feedback control and measurement-based control are fundamentally different schemes owing to the non-trivial effect of the measurement process.

The simplest implementation of coherent-feedback control is to consider the plant and controller as an isolated system with Hamiltonian
2.2
where *V*_{PC} gives the non-trivial coupling of the plant and controller. This set-up was initially proposed by Lloyd [13], and it is important to note that both plant and controller are quantized.

It is, however, more advantageous to consider the coupling between the plant and controller to be mediated by fields. In Markov models, which we shall present next, this leads to a natural quantum extension to the block diagram approach that was so successful in the classical regime. Formally, we replace the policies *u*_{j} in (2.1) with Boson quantum input processes that act as carriers for the signals between the plant and controller.

### (b) Quantum input–plant–output models

A single component Markov component is parameterized by a triple (*S*,*L*,*H*) consisting of:

— the system Hamiltonian

*H*;— coupling operators

*L*=[*L*_{j}] between the system and the field; and— scattering operators

*S*=[*S*_{jk}], unitary.

The input–output component is sketched in figure 4.

In the case of a single input and output, we associate the unitary adapted quantum stochastic evolution
where *B*^{†}(*t*),*B*(*t*) and *Λ*(*t*) are the fundamental processes of creation, annihilation and conservation introduced by Hudson & Parthasarathy [14]. The non-vanishing products of Itō differentials are [14,15]

The associated Heisenberg equations of motion for yield the *plant dynamics driven by the input processes*
2.3
where the Lindblad superoperator is

The output field is then defined to be and from the quantum Ito calculus, we have [14] the *quantum input–output relations*
2.4
Equations (2.3) and (2.4) are then the quantum mechanical analogues of the system dynamical equation and the output equation, respectively.

We may readily extend the above to the multi-channel input/output case,

The special case where *L*=0 and *H*=0 corresponds to a beam splitter, for example, with *n*=2 (figure 5)

The models presented here cover the physical situations covered in quantum input–output systems in the physics literature [16–18].

### (c) Quantum feedback networks

The efficacy of input–state–output models in classical control theory lies in the fact that one can represent them by block diagrams, as in figure 4, and then connect them up to form networks. The diagram in figure 6 illustrates the feedback set-up found in standard engineering textbooks. However, quantum mechanically we cannot proceed this way!

#### (i) Cascaded systems

The simplest form of a network consists of a pair of cascaded systems, as shown in figure 7. In the instantaneous feedforward limit, the output of the second plant is .

The cascaded system in the *instantaneous feedforward* limit is in fact equivalent to the single component [19]
which is referred to as the *series product* of the two models [19].

#### (ii) Bilinear Hamiltonians

The following construction, based on an idea of Wiseman & Milburn [20], shows how the series product gives rise to bilinear Hamiltonians of the form (2.1) generically [21]: , which is a double pass through the system and is illustrated in figure 8. We find

#### (iii) Beam-splitter feedback

We illustrate (figure 9) briefly the role of beam-splitter feedback, originally introduced for linear models in Yanagisawa & Kimura [22,23]. This is a special case of the more general problem of feedback reduction described in Gough & James [24]. Here, we consider a simple network consisting of a beam-splitter and an in-loop component (*S*_{0},*L*_{0},0).

Again, assuming the instantaneous feedback limit (figure 10), we have , so that , where the feedback loop-renormalized operators are now
This leads to an equivalent component . The form of is given by Gough & James [24]. Some caution is needed here as the internal fields *B*_{2} and do not satisfy canonical commutation relations!

### (d) How do we build arbitrary networks?

We need only two operations, *concatenation* and *feedback reduction*, to construct arbitrary networks.

#### (i) Concatenation

In the situation in figure 11*a*, we have several models (*S*_{i},*L*_{i},*H*_{i}) that can be assembled into a single model,
2.5

#### (ii) Feedback reduction

In the instantaneous feedback limit, the reduced model is obtained by eliminating the internal channels as in figure 11*b*
where we decompose and with respect to the external and internal channels.

It can be shown that basic modelling procedures such as adiabatic elimination may be performed consistently with these operations, see Nurdin & Gough [25] in this issue.

## 3. Measurement-based control

Information is physical is one of the key messages, and, on a fundamental level, it is quantum physical.

Furusawa & van Loock [26, p. 3].

### (a) Quantum filtering

Classically, filtering is the problem of obtaining a best causal estimate of a hidden signal given partial observations. The original problem tackled by Wiener and Kolmogorov in a linear stationary setting was to try and filter out background stationary noise added onto a given stationary signal: this was greatly extended by Kalman to input–plant–output models. The next step was made by Stratonovich [27], who showed that the dynamical state vector of a noisy conditionally Markovian system could be estimated recursively from noisy observations, thereby generalizing the Kalman filter to nonlinear systems. The general theory of nonlinear filtering was subsequently developed in a time continuous setting by Stratonovich, Kallianpur, Striebel, Zakai and others. This was extended at the end of the 1980s by Belavkin to the quantum conditionally Markov setting in a series of papers [28–32]. For a general discussion on continual measurement of quantum systems, see Barchielli & Gregoratti [33] and Barchielli & Belavkin [34], as well as Barchielli & Gregoratti [35].

#### (i) Quantum estimation

In a given experiment (figure 12), we may consistently measure only a commuting set of observables. Let us denote the algebra of observables generated by the measured observables as —this will be a commutative (von Neumann) algebra. The set of all observables will be denoted as , and we may ask when is it possible to estimate an observable from the measured data. In principle, if *X* does not commute with measurement observables, then it cannot be given a joint probability distribution with these observables, and it does not make sense statistically to talk about an estimate for *X* based on the measurements. We therefore restrict the set of observables we may estimate from down to the set of just the observables that are compatible with the measurements: mathematically, this means that we may estimate an observable if and only if it belongs to the commutant .

The error of an estimator is given as and the optimal estimator is the conditional expectation 3.1

Conditional expectations do not always exist in the non-commutative setting of quantum probability; however, in the present context, is well defined so long as its domain is the set of observables compatible with the measurement algebra . In fact, for given , the algebra generated by *X* and the elements of is commutative; so, in a sense, we are just doing ordinary classical conditioning. The construction is however not trivial as the set of measurement compatible observables typically is itself a non-commutative algebra.

#### (ii) Quantum filtering

We now recall the problem of quantum filtering as the causal estimation of a quantum open Markovian system. Here (figure 13), we measure an observable *Y* (*t*) of the output field. Our aim is to estimate causally any observable *X* of the system at time *t*, that is , from the observations up to time *t*,
for all *s*≥*t*. This implies that the causal estimation problem is well posed because all observables of the system at time *t* are then compatible with the measured observables up to that time. That is, the measurement output algebra as the commutative algebra generated by the observations *Y* (*s*), 0≤*s*≤*t*, is contained in the commutant {*j*_{s}(*X*):*s*≥*t*}^{′} for any *X* at each time *t*.

We now must fix a state which we take to be the expectation in the joint state *ρ*_{0}⊗|*Ω*〉〈*Ω*| with *ρ*_{0} a fixed density matrix for the plant system, and |*Ω*〉 the Fock vacuum of the field. The least-squares estimate of the *j*_{t}(*X*) is then the conditional expectation
which satisfies the nonlinear quantum filtering equation first derived by Belavkin for diffusive and counting observations [28–32].

In the case of the standard diffusive measurement *Y* (*t*)=*U*^{†}(*t*)*I*⊗(*B*(*t*)+*B*^{†}(*t*))*U*(*t*), the quantum filter is
where the innovations d*I*(*t*)=d*Y* (*t*)−*π*_{t}(*L*+*L*^{†}) d*t* are a Wiener process. For the counting measurement case *Y* (*t*)=*U*^{†}(*t*)*I*⊗*Λ*(*t*)*U*(*t*), we have
with innovations d*I*(*t*)=d*Y* (*t*)−*π*_{t}(*L*^{†}*L*) d*t*, which are a compensated Poisson process. There is an analogue [34] of the Kallianpur–Striebel representation *π*_{t}(*X*)=*ϖ*_{t}(*X*)/*ϖ*_{t}(1), where the corresponding quantum Zakai equation [34] in the diffusive case is
and in the counting case is
with the appropriate innovations in either case. For the rigorous derivation of the general conditionally Markov quantum linear and nonlinear filtering equations, see Belavkin [28–32]. After the solving of quantum filtering problem, one deals only with the classical stochastic equations driven by *Y* (*t*). Quantum feedback control problems can be tackled then along the usual lines of the classical stochastic control theory by filtering-control separation theorem, see also Bouten & van Handel [36].

#### (iii) Non-selective measurements: the master equation

Suppose we do not record the measurement readout.

It is customary to see the above written in terms of states: we may write *π*_{t}(*X*)=*tr*{*ϱ*_{t}*X*}, which serves to define a density-matrix-valued stochastic process *ϱ*_{t}. The diffusive case leads to the stochastic master equation (SME)
where d*I*(*t*)=d*Y* (*t*)−*λ*_{t} d*t* and *λ*_{t}=*tr*{*ϱ*_{t}(*L*+*L*^{†})}, while we have the adjoint to the Lindbladian (Liouvillian)
The presence of the *λ*_{t} terms means that this is a nonlinear equation in *ϱ*_{t}, but that would be the case classically. The counting measurement SME may be similarly deduced.

If we ignore the measurement readout, then we must average over the possible readout trajectories. This leads to the (traditional) master equation
(Note that we must have the relation .) The basic equations are identical to those encountered in quantum trajectories, but the logic is reversed. Rather than starting from the master equation and seeking unravellings that may be interpreted as measurements, we start from the fully quantum mechanical model of the measurement, derive the filter (SME) and interpret the noise *I*(*t*) as the innovations process. The innovations process is of the form d*I*(*t*)=d*Y* (*t*)−*λ*_{t} d*t*, which is the difference of what we next observe (d*Y*) and what we would next expect to observe (*λ* d*t*). Averaging over the readouts, with respect to the derived probability for the outputs, is then equivalent to the original master equation. The dissipative part of the Liouvillian is then usually referred to as the *measurement back-action*. For more details, see the book of Wiseman & Milburn [37] and the contributions of Doherty *et al.* [38] and Ruskov *et al.* [39] to this issue.

### (b) Filtering in non-classical states

The previous examples dealt with an input that was in the vacuum state, and much of the theory extends in a analogous manner to the case of a Gaussian field. However, there has been considerable interest recently in non-classical field states (figure 14), specifically, states corresponding to a single photon (or more generally Fock states) [40,41] and superpositions of coherent states. The former can now be generated on demand using state-of-the-art experimental techniques, while the latter correspond to the so-called cat states.

The filtering problem for these non-classical states turns out to be tractable, see the contribution of Gough *et al.* [42] in this issue, where one employs an ancilla to extend the filter in an appropriate manner. The role of the ancilla may often be easily understood as a preparation device that takes the vacuum input and feeds out the non-classical field into the system of interest.

### (c) Direct measurement feedback

Finally, it is worth recalling an example introduced by Wiseman & Milburn [20]. This is a double pass of a quantum light field through a plant and can be modelled as the following series product:

— first pass: (

*I*,*L*,*H*_{0}) and— second pass: corresponding to .

Here (figure 15), the detector measures a component *J*(*t*) of the output field from the first pass (either a quadrature or the photon number count), which is then fed in a second time as a direct proportional Hamiltonian term: this was the original interpretation of Wiseman & Milburn [20]. Formally, *J*(*t*) is singular; so we interpret it as generating a stochastic unitary process *U* as outlined earlier.

The cases of homodyne and photon counting may be treated separately.

*Homodyne detection*: *J*_{t}=*B*(*t*)+*B*(*t*)^{†}, (d*J*)^{2}=d*t*, second pass is (*I*,−i*F*,0). The closed-loop model is

*Photon counting*: *J*_{t}=*Λ*_{t}, (d*J*)^{2}=d*J*, second pass is (*S*=e^{−iF},0,0). The closed-loop model is
In either case, the resulting triple (*S*,*L*,*H*) gives the effective model for the quantum input–state–output model.

## 4. Coherent feedback control

Once we have the basic rules for assembling arbitrary quantum feedback networks, we can start to think about complex architectures, analysis and synthesis, and design issues for quantum systems. General conditions for stability, passivity and *L*^{2}-gain for quantum feedback networks have been given by James & Gough [43] in a framework that extends the Willems approach [44,45] to control engineering. A general set-up is sketched in figure 16. The exosystem *W* represents an external system or environment (more generally, one aims for robustness of performance against a class of exosystems). Here, one may look at the ability of the system to store or to dissipate energy. This is a generalization of Lyapunov techniques to open systems with a supply of energy.

Mathematically, one works with operators now replacing state functions. Linear models arise for systems with canonical coordinates for which the triple (*S*,*L*,*H*) leads to a linear system of Heisenberg–Langevin equations and input/output relations: this happens when *H* is quadratic, *L* is linear and *S* is independent of the canonical coordinates. In this case, the performance specification becomes tractable and one may generalize some of the known results for classical linear systems for linear quadratic Gaussian (LQG) problems and control, see the surveys [46–48] for more details. While the robust control problem turns out to be tractable in the quantum case, unfortunately the optimal LQG controller need not correspond to a genuine quantum input–system–output model. This issue of *physical realizability* means that we have a non-convex optimization problem, which is still an open problem.

Physical applications have been proposed by the MabuchiLab group [49,50]. In particular, they have developed a hardware description language to implement the series product, concatenation and feedback reduction operations introduced earlier: see the contribution by Tezak *et al.* [51] to this issue. This allows for the construction of arbitrary quantum optical networks modelled within the Markov assumption.

## 5. Outlook

The potential for truly revolutionary innovation is enormous and so is the likely return on investment. Perhaps one day my grandchildren will be graduating from a Department of Quantum Engineering.

Milburn [52, p. 180]

There has been considerable interest in applying control theory to quantum systems in recent years; however, there is a clear need to abstract the basic elements and to develop a more universal theory similar to what has occurred with modern control engineering for classical systems. Several topics have emerged as being of particular importance. Open-loop control has been successfully applied to molecular dynamics, chemical kinetics, optical interactions with matter and the design of multi-dimensional nuclear magnetic resonance experiments. The role of feedback in quantum dynamical systems is now more apparent with closed-loop models having been developed and implemented experimentally—the analysis and synthesis problems are now a major sphere of activity, mirroring what has been known for classical systems. A network theory for quantum input–output systems has been developed, allowing the same flexibility as in circuit theory. This in turn has opened the way for control by interconnection of quantum controllers, robust quantum control and synthesis, etc. The theory of quantum filtering has been applied to model quantum syndrome error correction in continuous time. In these situations, new mathematical problems appear that are of direct physical application. Many new features arise as a result of the inherently quantum nature of the problem, and an area of intense investigation is in quantum statistics: examples of issues that can be treated within this framework are model reduction and system identification. Again, this requires a significant departure from classical approaches.

We have touched on some of the background themes of quantum feedback and control that were discussed at the Royal Society meeting. We should also mention the other contributions to this volume on important problems such as system identification [53, 54], preparation and stabilization of target states using only local dissipation [55, 56] and quantum dynamical programming [57].

While we may expect that the road to realizing quantum technology will be challenging, it is not unreasonable to believe that developing systematic approaches to quantum control will be a necessary step to achieving this. We finish therefore with figure 17, which we hope will encourage researchers in the years to come.

## Acknowledgements

The author thanks Matthew James, Hendra Nurdin, Hideo Mabuchi, Luc Bouten, Daniel Burgarth and Ian Petersen for valuable comments.

## Footnotes

One contribution of 15 to a Theo Murphy Meeting Issue ‘Principles and applications of quantum control engineering’.

- This journal is © 2012 The Royal Society