Position-dependent and cooperative quantum Parrondo walks

This work is a study on quantum computational formulations of Parrondo walks, that is, positively trending random walks formed as combinations of negatively trending random walks. We reanalyse the position-dependent walk proposed by Košík et al (2007 J. Mod. Opt. 54 2275), correcting the parameter choices in that paper to achieve the Parrondo effect. We also devise a quantum analogue of the cooperative Parrondo walk of Toral (2002 Fluct. Noise Lett. 2 L305), in which it is the interaction between multiple participants, rather than position-dependence, that allows the Parrondo effect to occur. We give a general formulation of a quantum analogue of the classical walk of Toral (2002 Fluct. Noise Lett. 2 L305), and demonstrate the Parrondo effect numerically. Lastly, we highlight a qualitative difference in asymptotic behaviour between quantum Parrondo walks and their classical counterparts. In particular, we draw attention to an intuitive but unreliable assumption, based on classical random walks, which may pose extra challenges for applications of the Parrondo effect in the quantum setting seeking to separate or classify data or particles.


Introduction
Starting with Parrondo in [1], many random walks have been found exhibiting Parrondo's paradox: these are positively biased walks formed by combining, in various ways, sets of negatively biased walks.
For instance, Parrondo's original random walk [1]- [3] involves two-component walks (or subwalks). A particle moves randomly on the set Z of integers. Each subwalk moves the particle one step forward or backward per iteration. One subwalk is homogeneous and slightly negatively biased. The other is inhomogeneous-the probability of increment depends periodically on the current position, and is greater than 1/2 for some positions, and less than 1/2 for others-but with repeated iteration, the particle's expected position decreases. Surprisingly, a combination of these two subwalks, in which the particle moves according to a randomly chosen subwalk at each iteration, yields an increasing expected particle position after multiple iterations.
Several authors [4]- [7] have discussed the possibility of obtaining a Parrondo effect from a quantum walk [8]. A quantum walk on Z is similar to a random walk on Z, but with unitary evolution replacing the random walk's probabilistic transitions. Thus a quantum Parrondo walk is a positively biased combination of individually negatively biased quantum walks on the integers Z. A quantum walk presented in [4] and intended as a Parrondo walk does not, in fact, have negatively trending expectations for its subwalks, but we correct this in section 2.
Toral in [9] introduced an alternative classical Parrondo walk, in which interactions between multiple participants, rather than position-dependent transition probabilities, cause a shared particle moving on Z to display the Parrondo effect. This 'cooperative Parrondo walk' again involves constituent subwalks, and for each subwalk, we imagine the participants arranged in a circle. At each iteration, one participant is selected at random, and moves the particle one unit forward or backward, with probabilities depending on the most recent movements made by the left-and right-hand neighbours of the selected participant. Toral showed that two subwalks, each yielding negative expected particle motion, can be combined randomly to give a new walk with positive expected motion. In section 3, we describe a quantum analogue of Toral's walk, also exhibiting the Parrondo effect.
As pointed out in [10], quantum Parrondo walks are only meaningful in applications where we can in fact achieve coherent evolution. The widespread use of random walks in 3 modern algorithms suggests that quantum walks will also find application, and understanding the Parrondo effect may contribute to that.
Another practical concern lies in the effect of quantum walks' well-known accelerated spreading [11] on the long-run behaviour of quantum Parrondo walks. In classical Parrondo walks, while the expected positions of the combined walk and subwalks grow linearly with time, their standard deviations grow as the square root of time. Thus a difference in expectation will ultimately become dominant, allowing the position distribution of the combined walk to be reliably distinguished from that of the subwalks. For applications, this seems likely to be important. However, the standard deviation of position in a quantum Parrondo walk grows linearly with time, just as the expectation. Consequently, a modest difference in expectations between a combined walk and its subwalks remains modest after many iterations. This seems not to have been emphasized in previous studies.

Quantum position-dependent walk
The original, non-quantum Parrondo walk has a simple structure. It was presented in [1]- [3] in some generality, but the parameter values described here were given as an example. It is an ordinary random walk on Z with positive drift, which can be seen as a mixture of two subwalks, each with negative drift.
Subwalk A is a simple biased random walk on Z: we imagine a particle moving along Z whose position, at each time step, increases by 1 with probability 1/2 − , or decreases by 1 with probability 1/2 + , where is some small positive bias parameter.
Subwalk B is slightly more complicated, because its transition probabilities depend periodically on position. The particle's position increases by 1 with probability 1/10 − if it is divisible by 3, and with probability 3/4 − if it is not. Otherwise, the position decreases by 1.
Subwalks A and B each yield negative drift over time in expected position for positive values of . On the other hand, for reasonably small , there are many ways of combining these two subwalks to produce a third walk with a positive drift in expected position. Perhaps the simplest way (and the only one considered here) is simple stochastic mixing: at each time step, one of the two subwalks is chosen at random to govern the particle's motion. This reversal, a positive drift in the combined walk with negative drifts in the individual walks, is the Parrondo effect.
We turn now to quantum Parrondo walks. Although they have been examined in a number of studies, we will follow the work of [4], where a specific position-dependent quantum walk on Z is described. In a typical quantum walk on Z [8], a quantum system evolves by repeated application of a unitary operator. Both the system and the operator are compound. The system consists of two or more subsystems, called registers: the position register has states corresponding to the elements of Z, and represents the position of an imaginary particle along this line; the remaining registers can be thought of as describing internal degrees of freedom of the particle, such as spin.
The 'internal degrees of freedom' are called chirality, and influence the motion of the particle on Z. These constitute a necessary qualitative difference between quantum walks and random walks, contrasting sharply with the Markovian property of random walks. (It was pointed out in [12] that, without extra degrees of freedom, the only translation-invariant quantum walks on Z are the left-and right-shift operators, up to an overall phase shift.) Likewise, the unitary operator representing a single 'step' of the walk is composed of two operators: one, the step operator, acts translation-invariantly on the position register, to move the particle along Z by an increment or decrement determined by the state of the chirality register or registers; the other acts solely on the chirality registers. (Frequently this latter operator is called the 'coinflip operator', because it is analogous to a coin or other source of randomness in a classical random walk.) In [4], three registers are used: the position register, with Hilbert space H C = span{. . . , |−2 , |−1 , |0 , |1 , |2 , . . .}; a standard chirality register, with Hilbert space H X = span{|0 , |1 }, controlling whether the position register is decremented or incremented; and a subwalk selection register, with Hilbert space H W = span{|0 , |1 , . . . , |r − 1 }, where r is a fixed positive integer. The state of the subwalk selection register evolves independently, creating a coherent combination of r subwalks, each defined on the first two registers. Moreover, the walk's translation invariance is weakened, so that the chirality register's evolution depends periodically on the position register (modulo another fixed positive integer, m).
For convenience, we generalize the walk of [4] in the following definition. The evolution of the subwalk selection register will be described by an operatorÛ . Also, we will assume the operator acting on the chirality qubit to be of the form where θ is a parameter depending on the particle's position modulo m and on the state of the subwalk selection register. Notationally we collect these parameters in an m × r matrix , whose entry jk specifies the rotation angle in (1) when the particle position is congruent to j modulo m and the subwalk selection register is in the state |k . (The rows and columns of parameter matrices in this section and the next will be labelled 0, 1, . . . .) Definition 1. Let m and r be fixed positive integers,Û be an r × r unitary matrix and be an m × r real matrix. On any Hilbert space, letÎ denote the identity andπ k denote |k k|. On H C , letπ ≡ j = z∈Zπ j+zm ,T − = j∈Z | j − 1 j| andT + = j∈Z | j + 1 j|. The quantum positiondependent walk with parametersÛ and is a unitary operation on the space The walk defined above is depicted in figure 1. The space H C represents the particle's position. The space H X is the ordinary quantum coin, or 'chirality qubit,' representing the direction of particle motion. The space H W is the subwalk selection space, determining which position-dependent subwalk is used on each iteration: the state of this register is rotated byÛ at the beginning of each iteration, and then, together with the particle's position, modulo m, determines an entry of the parameter matrix , parameterizing the rotation of the H X register. Thus, at each iteration, firstly the subwalk selection bit |w is rotated byÛ , and then one of the rotationsĜ 0,0 , . . . ,Ĝ m−1,−1 acts on the chirality bit (with the values in |w and |c determining which rotation is used), and lastly the position register is decremented or incremented according to the value of the chirality bit.
The walk described in [4] combines two subwalks, and has modulo 3 position-dependence. Therefore m = 3 and r = 2, and we can write it aŝ (While the paper indicates 3/2 rather than 3π/2, it appears that the latter is intended.) The two columns of the parameter matrix represent the two subwalks. The simpler walk, subwalk 1 (called A in [4]) is represented by the second column, (π − 1/50, π − 1/50, π − 1/50) † ; the equality of all three parameters indicates that subwalk 1 is not position-dependent. On the other hand, subwalk 0 is described in [4] (where it is called B) as a position-dependent combination of two further subwalks: one used when the position is divisible by three, and represented by the upper-left entry π/5 − 1/50 of ; and the other used when the position is not divisible by three, and represented by 's remaining entries, both equal to 3π/2 − 1/50.

Simulation
We have used Matlab to implement a simulation of the quantum position-dependent walk (4). Figure 2 plots the results of a simulation of the walk (4) beginning from the state |0 ⊗ |1 ⊗ |0 . The walk (4) is presented in [4] as an example of a quantum version of a Parrondo walk. However, the analogy is incomplete: the combined walk does exhibit positive expected motion, but so does subwalk 0, with a faster increase in expected position.  By altering the parameters of the walk, we can restore the Parrondo paradox. For simplicity, we have chosen m = r = 2, that is, we consider a walk combining two subwalks, each of which uses different chirality bit rotation angles for odd position values and even position values. Figure 3 shows results from a simulation of the walk beginning from the state 7 Note that the combined walk shows a positive trend in expected position, whereas each of the subwalks displays a negative trend.

Distinguishability of distributions and the evolution of quantile-quantile (Q-Q) plots
In the Parrondo walk studied in [3], the combined walk's expected position increases, while the expected position of each of the subwalks decreases. These effects are both asymptotically linear. Intuition suggests that, eventually, the combined walk must reach a positive position while the subwalks reach negative positions, with any required degree of certainty. Therefore, the probability distribution of the combined walk's position will overlap decreasingly with the probability distributions of the subwalks, eventually lying almost entirely to the right of them. One way to quantify the distributional overlap would be via the cross-entropy or Kullback-Leibler distance [13]. However, the cross-entropy ignores the order relation on Z, whereas we expect applications of quantum Parrondo walks to rely on this ordering, that is, to require that the combined walk's resulting position exceeds the positions resulting from the subwalks. Another common way to quantify distributional distinguishability, which does take order into account, is via Q-Q plots [14].
A Q-Q plot is simply a plot of one cumulative distribution function (CDF) versus another. A CDF maps R to [0, 1] and must be nondecreasing with infimum 0 and supremum 1, and therefore a Q-Q plot maps [0, 1] to [0, 1] nondecreasingly, fixing the values 0 and 1. Figure 4 illustrates the relationship between two distributions and their Q-Q plot. (Q-Q plots are frequently used as an exploratory tool in statistics, with one or both of the CDFs empirical, that is, based directly on proportions of an observed sample, rather than on probabilities of a known distribution. They are also used in hypothesis testing and classification to express the trade-off between the false positive and false negative error rates, where they are called receiver operating characteristic (ROC) curves [15].) Figure 5 illustrates the increasing gap between the classical Parrondo walk of [3] and one of its subwalks. Fifty Q-Q plots are depicted, comparing the position distribution of the combined walk with that of subwalk A at equal intervals from 200 to 10 000 iterations. (Comparison of the combined walk with subwalk B yields a similar picture.) This verifies our intuition: we see that, as the walks progress, the Q-Q plots relating their position distributions become increasingly square. The limit of this process is shown in figure 4(e), and would indicate that the position of the combined walk exceeded the position of the subwalk with probability 1.
The quantum case is rather different. Quantum walks spread quadratically faster than random walks [11], keeping pace with a linear trend in expected position. For a quantum Parrondo walk, the centres of the position distributions of the combined walk and of the subwalks drift apart over time, but the distributions grow in width at the same rate, so that the overlap does not, in general, become negligible. Figure 6 again shows the evolution of the Q-Q plot describing the overlap of the position distributions of a combined walk and a subwalk. This time, the combined walk (5) is compared with its subwalk 0. We observe very little change as the walks progress; the combined walk cannot be reliably distinguished from subwalk 0 on the basis of a single final position, regardless of the number of walk iterations prior to measurement.
Therefore we expect that it will be quite difficult, in practice, to observe a clear separation between the behaviour of a quantum Parrondo walk and of its subwalks. Our intuition that, for instance, a game of chance with positive expected return will reliably return a profit if played In (a), the variable Y is certainly less than X , and so they are perfectly distinguishable. For instance, if we are given a value z, and we know that it is from one of the two distributions, we can easily determine which, by comparing z with a threshold value t lying between the supports of the distributions. Then if z < t, it must be from Y 's distribution, since P[Y t] = 1 and P[X t] = 0. The point (1, 0) on the Q-Q plot derives from such threshold values t. Poorer choices are available too; thresholds within the support of Y describe the segment of the Q-Q plot from the origin to (1, 0), and thresholds within the support of X describe the segment from (1, 0) to (1, 1). In (b), the distributions' supports overlap, and if we are given a value z within this overlap, we cannot determine with certainty from which distribution it arose. Noting that usually Y < X , we would still choose a threshold t, and classify values below t as from Y 's distribution and values above t as from X 's, but now we will have some positive probability of misclassifying one or both of the variables Y and X . Either error probability can be reduced, at the expense of the other. The Q-Q plot depicts this trade-off, for all possible thresholds. In (c), the two distributions are identical, and trying to classify a given value z as from one or the other is futile. The diagonal Q-Q plot indicates perfect indistinguishability. Scenarios (d) and (e) are identical to (b) and (a), respectively, with X and Y exchanged, and in classification, the role of the threshold would be reversed, so that values greater than t would be classified as from Y 's distribution. Thus, a Q-Q plot that is perfectly or nearly square, whether above the diagonal or below it, indicates perfect or near-perfect distinguishability of distributions. often enough cannot be trusted here. It appears that the paradox, while quite spectacular in classical formulations, is considerably attenuated in the quantum setting owing to the quadratic spreading of quantum walks.

A cooperative quantum Parrondo walk
A non-quantum cooperative Parrondo walk was introduced in [9], consisting, like other Parrondo walks, of two subwalks. This walk involves a number p of participants, arranged in a circle. At each iteration, a randomly chosen participant moves a shared particle one unit forward or backward. The probability of moving forward (that is, incrementing the particle's position) depends on which of the two subwalks is selected, and also on the 'states' of the chosen participant's left-and right-hand neighbours; each participant's 'state' is either 'plus' or 'minus', according to whether the participant has most recently moved the particle forward or backward (with initial states chosen arbitrarily). Values were demonstrated in [9] for the set of probabilities parameterizing this walk such that the particle's expected position would increase after multiple iterations, whereas either of the two subwalks alone would decrease the particle's expected position.
Here we present a quantum walk with similar structure and behaviour. The walk evolves on a quantum system comprising registers representing which participant is currently in control of the particle (the participant selection subsystem), which subwalk is used (the subwalk selection subsystem), the state of each participant (the participant's chirality bit) and the particle's position (the position subsystem). A subwalk and participant are chosen at the beginning of each iteration by applying fixed unitary operations to both the subwalk selection subsystem and the participant selection subsystem. Then, the chosen participant's chirality qubit is rotated through an angle depending on the states of the participant's neighbours' chirality qubits, and on the selected subwalk. Lastly, the particle is moved forward or backward according to that chirality qubit. One way to visualize this walk is shown in figure 7.
To be precise, the walk performs a unitary operation on a space where • H S = span{|0 , . . . , | p − 1 } determines which participant moves the particle in the current iteration, • H W = span{|0 , . . . , |r − 1 } determines which subwalk is used (we will assume r = 2), • H X j = span{|0 , |1 } records whether the jth participant most recently moved the particle backward (|0 ) or forward (|1 ), and |0 , |1 , |2 , . . .} records the particle's position. We superscript operators with parenthesized labels for the subsystems on which they act, so that for instanceπ (S) j ,π (W ) k andπ (X j ) 0 are projections acting on the subsystems H S , H W and H X j , andT (C) − is a decrement operator acting on the subsystem H C . One iteration of the walk can now be expressed aŝ wherê with G sk as in (3), and of course the participant labels are to be interpreted periodically, so that 0 is equivalent to p and p + 1 is equivalent to 1. The number of rows of , and hence the number of terms in the right-hand side of (9), are fixed at four, because there are four possible combinations of the states of the selected participant's two neighbours. A circuit diagram appears in figure 8. (7) is called the cooperative quantum walk with parametersÛ w ,Û s and , and denoted byĈ(Û w ,Û s , ).

Numerical work
We have found that, for certain values of the walk parameters, the cooperative quantum walk described above exhibits a Parrondo effect.
To compare a combined walk with one of its subwalks numerically, we may use any of at least three formulations of the subwalk. Suppose for instance we wish to compare the walk C(Û w ,Û s , ) starting in state |ψ 0 with its first subwalk, that is, with the subwalk described by ·0 , the first column of . The system acted on byĈ(Û w ,Û s , ) includes the subwalk selection register, which is obviously not an intrinsic part of the subwalk. We must either omit this register when simulating the subwalk, or else somehow 'deactivate' it. Here are three options: 1. Simply omit the subwalk selection register, that is, simulateĈ(1,Û s , ·0 ). This is mostly straightforward, but since the subwalk and combined walk act on different spaces, we cannot use the same initial state |ψ 0 when simulating the subwalk. A natural approach is to modify this state by tracing out the subwalk selection space.

Active player |sÛ s • •
Active subwalk |wÛ wÂ For the sake of clarity, some grouping of operations has been necessary, and this diagram introduces the operatorsŶ andÂ. The operatorŶ simply permutes the chirality bits cyclically so that the bit indicated by the participant selection register appears in place of |χ 2 . ThusŶ corresponds to the rotation of the wheel about the axis in figure 7 and to the summation over j in (8). The operatorÂ then acts on the selected chirality bit, but is controlled by the two neighbouring chirality bits and by the subwalk selection register. Thus, at each iteration, the first two registers are rotated byÛ s andÛ w to determine which participant will move the particle and which subwalk will determine its motion; then, the selected participant's chirality qubit is rotated by an angle depending on its neighbours' chiralities and on the selected subwalk; then, the resulting chirality is used to decrement or increment the particle's position viaT − orT + .
2. To be able to use the same initial state, we can instead modify . Let ·0 ω denote the 4 × r matrix each of whose columns equals ·0 . ThenĈ(Û w ,Û s , ·0 ω) is a combination of r copies of the given subwalk, and can act directly on |ψ 0 . 3. To be able to use the original and the original space (7), we can trace out the subwalk selection register as in option 1 above, reinstate it by tensoring the resulting state with |0 so that the initial state is entirely within the first-subwalk space, and replaceÛ w withÎ (W ) so that the state remains entirely within the first-subwalk space. This is clearly equivalent to option 1.
In this section, for simplicity, we consider a very symmetric scenario in which all three of the above options are equivalent. The simulations were performed using option 1 above for the sake of computational efficiency. Now consider the walk with p = 7 participants, r = 2 subwalks,Û w equal to the Hadamard operator (11) Figure 9 shows the evolution of the expected position when 100 iterations ofĈ(Û w ,Û s , ) are performed on an initial equal-amplitude, zero-position state and also when 100 iterations of each of the two subwalkŝ are performed on the corresponding initial state | j ⊗ |χ 1 ⊗ · · · ⊗ |χ 7 ⊗ |0 . Figure 10. Probability density functions of position, after 100 iterations, for the walkĈ(Û w ,Û s , ) applied to (12) and for the walks (13) applied to (14). Figure 10 shows the probability density functions for the position after 100 iterations, for the two subwalks and the combined walk. (Densities for even position values only are plotted, since the walks conserve the parity of the sum of time and position.) The positive bias of the combined walk is perceptible; however, we also observe that the combined walk's position distribution strongly overlaps the subwalks' position distributions. For some applications it may be desirable to minimize this overlap. Figure 11 shows the Q-Q plots relating the final position distribution of the two subwalks to that of the combined walk. The area under a Q-Q plot is sometimes used as a scalar measure of distinguishability; both of these curves bound an area just over 63%, indicating that considerable overlap remains between the position distribution of the combined walk and of the subwalks. As in section 2.2, this overlap is undiminished by further iteration of the walks.

Conclusion
This work is a study on quantum computational formulations of Parrondo walks. We have reanalysed the position-dependent walk proposed by [4], correcting the parameter choices in that paper to achieve the Parrondo effect, that is, a coherent combination of quantum walks such that the combination shows an increasing trend in expected position, while each of its constituent walks shows a decreasing trend in expected position.
We have also devised a quantum analogue of the cooperative Parrondo walk of [9], in which interaction between multiple participants replaces position-dependence as an opportunity for the Parrondo effect to occur. We have given a general formulation of a cooperative quantum walk with structure comparable with the classical walk in [9], and we have presented numerical results demonstrating parameter choices yielding the Parrondo effect.
Additionally, we have used Q-Q plots to highlight a qualitative difference in asymptotic behaviour between quantum Parrondo walks and their classical counterparts. In particular, we have drawn attention to the intuitive but unreliable assumption, based on classical random walks, that a combined walk's linearly increasing advantage in expected position over its constituent subwalks asymptotically implies vanishing overlap of the distributions of the combined walk versus the subwalks. This may pose extra challenges for applications of the Parrondo effect in the quantum setting.  The parameters described in this paper are far from unique, in both the classical and quantum Parrondo walks discussed. Loosely speaking, if we chose all of the parameters randomly, then two subwalks and their combined walk would each have equal probabilities of positive and negative drift. Although there is some correlation between the combined walk's drift and those of the subwalks, we do find the Parrondo effect throughout an appreciable region of the parameter space. A thorough survey of the parameter space for the classical positiondependent walk is given in [16].
Lastly, we have not tried to relate the parameter choices for quantum Parrondo walks to their classical counterparts. Some authors have sought quantum Parrondo walks with the aid of parameter choices known to yield the Parrondo effect in the classical setting. We believe, however, that there is no canonical correspondence between the parameters of quantum Parrondo walks and of their classical counterparts, since unitary transformations in the quantum case replace scalar transition probabilities in the classical case. Of course it is possible to reinterpret the transition probabilities as parameterizing unitary transformations, but there is no favoured way to do this, and we feel that it would be difficult to justify.