A Geometric Analysis of Phase Retrieval

Abstract

Can we recover a complex signal from its Fourier magnitudes? More generally, given a set of m measurements, $y_k = \left| \varvec{a}_k^* \varvec{x} \right|$ for $k = 1, \ldots , m$, is it possible to recover $\varvec{x} \in \mathbb C^n$ (i.e., length-n complex vector)? This generalized phase retrieval (GPR) problem is a fundamental task in various disciplines and has been the subject of much recent investigation. Natural nonconvex heuristics often work remarkably well for GPR in practice, but lack clear theoretic explanations. In this paper, we take a step toward bridging this gap. We prove that when the measurement vectors $\varvec{a}_k$’s are generic (i.i.d. complex Gaussian) and numerous enough ($m \ge C n \log ^3 n$), with high probability, a natural least-squares formulation for GPR has the following benign geometric structure: (1) There are no spurious local minimizers, and all global minimizers are equal to the target signal $\varvec{x}$, up to a global phase, and (2) the objective function has a negative directional curvature around each saddle point. This structure allows a number of iterative optimization methods to efficiently find a global minimizer, without special initialization. To corroborate the claim, we describe and analyze a second-order trust-region algorithm.

Appendices

Appendices

Basic Tools and Results

Lemma 25

(Even moments of complex Gaussian) For $a \sim \mathscr {CN}(1)$, it holds that

$$\begin{aligned} \mathbb E\left[ \left| a \right| ^{2p} \right] = p! \quad \forall \; p \in \mathbb N. \end{aligned}$$

Proof

Write $a = x + \mathrm {i}y$, then $x, y \sim _{i.i.d.} \mathscr {N}(0, 1/2)$. Thus,

$$\begin{aligned} \mathbb E\left[ \left| a \right| ^{2p} \right] = \mathbb E_{x, y} \left[ \left( x^2 + y^2\right) ^p\right] = \frac{1}{2^p}\mathbb E_{z \sim \chi ^2(2)} \left[ z^p\right] = \frac{1}{2^p} 2^p p! = p!, \end{aligned}$$

as claimed. $\square$

Lemma 26

(Integral form of Taylor’s theorem) Consider any continuous function $f(\varvec{z}): \mathbb C^n \mapsto \mathbb R$ with continuous first- and second-order Wirtinger derivatives. For any $\varvec{\delta }\in \mathbb C^n$ and scalar $t\in \mathbb R$, we have

$$\begin{aligned} f(\varvec{z}+t\varvec{\delta })&= f(\varvec{z} ) + t \int _0^1 \begin{bmatrix} \varvec{\delta }\\ \overline{\varvec{\delta }}\end{bmatrix}^* \nabla f(\varvec{z}+ s t \varvec{\delta }) \; \mathrm{d}s, \\ f(\varvec{z}+t\varvec{\delta })&= f(\varvec{z} ) + t \begin{bmatrix}\varvec{\delta }\\ \overline{\varvec{\delta }}\end{bmatrix}^* \nabla f(\varvec{z} ) + t^2 \int _0^1 (1-s)\begin{bmatrix}\varvec{\delta }\\ \overline{\varvec{\delta }}\end{bmatrix}^* \nabla ^2 f(\varvec{z}+ st \varvec{\delta }) \begin{bmatrix}\varvec{\delta }\\ \overline{\varvec{\delta }}\end{bmatrix}\; \mathrm{d}s. \end{aligned}$$

Proof

Since f is continuous differentiable, by the fundamental theorem of calculus,

$$\begin{aligned} f(\varvec{z} + t\varvec{\delta }) = f(\varvec{z} ) +\int _0^t \begin{bmatrix}\varvec{\delta }\\ \overline{\varvec{\delta }}\end{bmatrix}^* \nabla f(\varvec{z} + \tau \varvec{\delta }) \; \text {d}\tau . \end{aligned}$$

Moreover, by integral by part, we obtain

$$\begin{aligned} f(\varvec{z}+t \varvec{\delta }) =&f(\varvec{z}) + \left. \left[ (\tau -t) \begin{bmatrix}\varvec{\delta }\\ \overline{\varvec{\delta }}\end{bmatrix} ^* \nabla f(\varvec{z} + \tau \varvec{\delta }) \right] \right| _{0}^t\\&- \int _{0}^t (\tau -t) \; d \left[ \begin{bmatrix}\varvec{\delta }\\ \overline{\varvec{\delta }}\end{bmatrix}^* \nabla f(\varvec{z} + \tau \varvec{\delta })\right] \nonumber \\ =&f(\varvec{x}) + t \begin{bmatrix}\varvec{\delta }\\ \overline{\varvec{\delta }}\end{bmatrix}^* \nabla f(\varvec{z}) +\int _{0}^t (t-\tau ) \begin{bmatrix}\varvec{\delta }\\ \overline{\varvec{\delta }}\end{bmatrix}^* \nabla ^2 f(\varvec{z} + \tau \varvec{\delta }) \begin{bmatrix}\varvec{\delta }\\ \overline{\varvec{\delta }}\end{bmatrix} \text {d}\tau . \end{aligned}$$

Change of variable $\tau = st (0 \le s \le 1)$ gives the claimed result. $\square$

Lemma 27

(Error of quadratic approximation) Consider any continuous function $f(\varvec{z}): \mathbb C^n \mapsto \mathbb R$ with continuous first- and second-order Wirtinger derivatives. Suppose its Hessian $\nabla ^2 f(\varvec{z})$ is $L_h$-Lipschitz. Then the second-order approximation

$$\begin{aligned} \widehat{f}(\varvec{\delta }; \varvec{z}) = f(\varvec{z}) + \begin{bmatrix}\varvec{\delta }\\ \overline{\varvec{\delta }} \end{bmatrix}^* \nabla f(\varvec{z}) + \frac{1}{2}\begin{bmatrix}\varvec{\delta }\\ \overline{\varvec{\delta }} \end{bmatrix}^* \nabla ^2 f(\varvec{z}) \begin{bmatrix}\varvec{\delta }\\ \overline{\varvec{\delta }} \end{bmatrix} \end{aligned}$$

around each point $\varvec{z}$ obeys

$$\begin{aligned} \left| f(\varvec{z}+ \varvec{\delta }) - \widehat{f}(\varvec{\delta }; \varvec{z}) \right| \le \frac{1}{3} L_h \left\| \varvec{\delta } \right\| _{}^3. \end{aligned}$$

Proof

By integral form of Taylor’s theorem in Lemma 26,

$$\begin{aligned} \left| f(\varvec{z}+ \varvec{\delta }) - \widehat{f}(\varvec{\delta }; \varvec{z} ) \right|&= \;\left| \int _0^1 (1-\tau ) \begin{bmatrix}\varvec{\delta }\\ \overline{\varvec{\delta }} \end{bmatrix}^* \left[ \nabla ^2 f(\varvec{x}+ \tau \varvec{\delta })- \nabla ^2 f(\varvec{x}) \right] \begin{bmatrix}\varvec{\delta }\\ \overline{\varvec{\delta }} \end{bmatrix} \; \text {d}\tau \right| \\&\le \; 2\left\| \varvec{\delta } \right\| _{}^2 \int _0^1 (1-\tau )\left\| \nabla ^2 f(\varvec{x}+ \tau \varvec{\delta })- \nabla ^2 f(\varvec{x}) \right\| _{} \;\text {d}\tau \\&\le \; 2L_h\left\| \varvec{\delta } \right\| _{}^3 \int _0^1 (1-\tau ) \tau \;\text {d}\tau = \frac{L_h}{3}\left\| \varvec{\delta } \right\| _{}^3, \end{aligned}$$

as desired. $\square$

Lemma 28

(Spectrum of complex Gaussian matrices) Let $\varvec{X}$ be an $n_1 \times n_2$ ($n_1 > n_2$) matrix with i.i.d. $\mathscr {CN}$ entries. Then,

$$\begin{aligned} \sqrt{n_1} - \sqrt{n_2} \le \mathbb E\left[ \sigma _{\min }(\varvec{X}) \right] \le \mathbb E\left[ \sigma _{\max }(\varvec{X}) \right] \le \sqrt{n_1} + \sqrt{n_2}. \end{aligned}$$

Moreover, for each $t \ge 0$, it holds with probability at least $1 - 2\exp \left( -t^2\right)$ that

$$\begin{aligned} \sqrt{n_1} - \sqrt{n_2} - t \le \sigma _{\min }(\varvec{X}) \le \sigma _{\max }(\varvec{X}) \le \sqrt{n_1} + \sqrt{n_2} + t. \end{aligned}$$

Lemma 29

(Hoeffding-type inequality, Proposition 5.10 of [101]) Let $X_1,\ldots , X_N$ be independent centered sub-Gaussian random variables, and let $K = \max _i \left\| X_i \right\| _{\psi _2}$, where the sub-Gaussian norm

$$\begin{aligned} \left\| X_i \right\| _{\psi _2 } \doteq \sup _{p\ge 1} p^{-1/2} \left( \mathbb E\left[ \left| X \right| ^p \right] \right) ^{1/p}. \end{aligned}$$

(A.1)

Then for every $\varvec{b} = \left[ b_1; \cdots ; b_N \right] \in \mathbb C^N$ and every $t\ge 0$, we have

$$\begin{aligned} \mathbb P\left( \left| \sum _{k=1}^N b_k X_k \right| \ge t \right) \le e \exp \left( - \frac{ct^2}{K^2 \left\| \varvec{b} \right\| _{2}^2 } \right) . \end{aligned}$$

(A.2)

Here c is a universal constant.

Lemma 30

(Bernstein-type inequality, Proposition 5.17 of [101]) Let $X_1,\ldots , X_N$ be independent centered subexponential random variables, and let $K = \max _i \left\| X_i \right\| _{\psi _1}$, where the subexponential norm

$$\begin{aligned} \left\| X_i \right\| _{\psi _1} \doteq \sup _{p\ge 1} p^{-1} \left( \mathbb E\left[ \left| X \right| ^p \right] \right) ^{1/p}. \end{aligned}$$

(A.3)

Then for every $\varvec{b} = \left[ b_1; \ldots ; b_N \right] \in \mathbb C^N$ and every $t\ge 0$, we have

$$\begin{aligned} \mathbb P\left( \left| \sum _{k=1}^N b_k X_k \right| \ge t \right) \le 2 \exp \left( -c \min \left( \frac{t^2}{K^2\left\| \varvec{b} \right\| _{2}^2}, \frac{t}{K\left\| \varvec{b} \right\| _{\infty }}\right) \right) . \end{aligned}$$

(A.4)

Here c is a universal constant.

Lemma 31

(Sub-Gaussian lower tail for nonnegative RV’s, Problem 2.9 of [21]) Let $X_1$, $\ldots$, $X_N$ be i.i.d. copies of the nonnegative random variable X with finite second moment. Then it holds that

$$\begin{aligned} \mathbb P\left[ \frac{1}{N} \sum _{i=1}^N \left( X_i - \mathbb E\left[ X_i \right] \right) < -t \right] \le \exp \left( -\frac{Nt^2}{2\sigma ^2}\right) \end{aligned}$$

for any $t > 0$, where $\sigma ^2 = \mathbb E\left[ X^2 \right]$.

Proof

For any $\lambda > 0$, we have

$$\begin{aligned} \log \mathbb E\left[ \mathrm {e}^{-\lambda (X - \mathbb E\left[ X \right] )} \right] = \lambda \mathbb E\left[ X \right] +\log \mathbb E\left[ \mathrm {e}^{-\lambda X} \right] \le \lambda \mathbb E\left[ X \right] + \mathbb E\left[ \mathrm {e}^{-\lambda X} \right] - 1, \end{aligned}$$

where the last inequality holds thanks to $\log u \le u -1$ for all $u > 0$. Moreover, using the fact $\mathrm {e}^u \le 1 + u + u^2/2$ for all $u \le 0$, we obtain

$$\begin{aligned} \log \mathbb E\left[ \mathrm {e}^{-\lambda (X - \mathbb E\left[ X \right] )} \right] \le \frac{1}{2} \lambda ^2 \mathbb E\left[ X^2 \right] \Longleftrightarrow \mathbb E\left[ \mathrm {e}^{-\lambda (X - \mathbb E\left[ X \right] )} \right] \le \exp \left( \frac{1}{2} \lambda ^2 \mathbb E\left[ X^2 \right] \right) . \end{aligned}$$

Thus, by the usual exponential transform trick, we obtain that for any $t > 0$,

$$\begin{aligned} \mathbb P\left[ \sum _{i=1}^N (X_i - \mathbb E\left[ X_i \right] ) < -t \right] \le \exp \left( -\lambda t + N\lambda ^2 \mathbb E\left[ X^2 \right] /2\right) . \end{aligned}$$

Taking $\lambda = t/(N\sigma ^2)$ and making change of variable for t give the claimed result. $\square$

Lemma 32

(Moment-control Bernstein’s inequality for random variables) Let $X_1, \ldots , X_p$ be i.i.d. copies of a real-valued random variable X Suppose that there exist some positive number R and $\sigma _X^2$ such that

$$\begin{aligned} \mathbb E\left[ X^2 \right] \le \sigma _X^2, \quad \text {and} \quad \mathbb E\left[ \left| X \right| ^m \right] \le \frac{m!}{2} \sigma _X^2 R^{m-2}, \; \; \text {for all integers }m \ge 3. \end{aligned}$$

Let $S \doteq \frac{1}{p}\sum _{k=1}^p X_k$, then for ... , it holds that

$$\begin{aligned} \mathbb P\left[ \left| S - \mathbb E\left[ S \right] \right| \ge t \right] \le 2\exp \left( -\frac{pt^2}{2\sigma _X^2 + 2Rt}\right) . \end{aligned}$$

Lemma 33

(Angles between two subspaces) Consider two linear subspaces $\mathscr {U}$, $\mathscr {V}$ of dimension k in $\mathbb R^n$ ($k \in [n]$) spanned by orthonormal bases $\varvec{U}$ and $\varvec{V}$, respectively. Suppose $\pi /2 \ge \theta _1 \ge \theta _2 \dots \ge \theta _k \ge 0$ are the principal angles between $\mathscr {U}$ and $\mathscr {V}$. Then it holds that

(i) $\min _{\varvec{Q} \in O_k} \left\| \varvec{U} - \varvec{V} \varvec{Q} \right\| _{} \le \sqrt{2-2\cos \theta _1}$;

(ii) $\sin \theta _1 = \left\| \varvec{U}\varvec{U}^* - \varvec{V}\varvec{V}^* \right\| _{}$;

(iii) Let $\mathscr {U}^\perp$ and $\mathscr {V}^\perp$ be the orthogonal complement of $\mathscr {U}$ and $\mathscr {V}$, respectively. Then $\theta _1(\mathscr {U}, \mathscr {V}) = \theta _1(\mathscr {U}^\perp , \mathscr {V}^\perp )$.

Proof

Proof to (i) is similar to that of II. Theorem 4.11 in [95]. For $2k \le n$, w.l.o.g., we can assume $\varvec{U}$ and $\varvec{V}$ are the canonical bases for $\mathscr {U}$ and $\mathscr {V}$, respectively. Then

$$\begin{aligned} \min _{\varvec{Q} \in O_k} \left\| \begin{bmatrix} \varvec{I} - \varvec{\varGamma }\varvec{Q} \\ - \varvec{\Sigma }\varvec{Q} \\ \varvec{0} \end{bmatrix} \right\| _{} \le \left\| \begin{bmatrix} \varvec{I} - \varvec{\varGamma }\\ - \varvec{\Sigma }\\ \varvec{0} \end{bmatrix} \right\| _{} \le \left\| \begin{bmatrix} \varvec{I} - \varvec{\varGamma }\\ - \varvec{\Sigma }\end{bmatrix} \right\| _{}. \end{aligned}$$

Now by definition

$$\begin{aligned} \left\| \begin{bmatrix} \varvec{I} - \varvec{\varGamma }\\ - \varvec{\Sigma }\end{bmatrix} \right\| _{}^2&= \max _{\left\| \varvec{x} \right\| _{} = 1} \left\| \begin{bmatrix} \varvec{I} - \varvec{\varGamma }\\ - \varvec{\Sigma }\end{bmatrix} \varvec{x} \right\| _{}^2 = \max _{\left\| \varvec{x} \right\| _{} = 1} \sum _{i=1}^k (1 - \cos \theta _i)^2 x_i^2 + \sin ^2\theta _i x_i^2 \\&= \max _{\left\| \varvec{x} \right\| _{} = 1} \sum _{i=1}^k (2-2\cos \theta _i) x_i^2 \le 2- 2\cos \theta _1. \end{aligned}$$

Note that the upper bound is achieved by taking $\varvec{x} = \varvec{e}_1$. When $2k > n$, by the results from CS decomposition (see, e.g., I Theorem 5.2 of [95])

$$\begin{aligned} \min _{\varvec{Q} \in O_k} \left\| \begin{bmatrix} \varvec{I}&\varvec{0} \\ \varvec{0}&\varvec{I} \\ \varvec{0}&\varvec{0} \end{bmatrix} - \begin{bmatrix} \varvec{\varGamma }&\varvec{0} \\ \varvec{0}&\varvec{I} \\ \varvec{\Sigma }&\varvec{0} \end{bmatrix} \right\| _{} \le \left\| \begin{bmatrix} \varvec{I} - \varvec{\varGamma }\\ - \varvec{\Sigma }\end{bmatrix} \right\| _{}, \end{aligned}$$

and the same argument then carries through. To prove (ii), note the fact that $\sin \theta _1 = \left\| \varvec{U} \varvec{U}^* - \varvec{V} \varvec{V}^* \right\| _{}$ (see, e.g., Theorem 4.5 and Corollary 4.6 of [95]). Obviously, one also has

$$\begin{aligned} \sin \theta _1 = \left\| \varvec{U} \varvec{U}^* - \varvec{V} \varvec{V}^* \right\| _{} = \left\| (\varvec{I} - \varvec{U} \varvec{U}^*) - (\varvec{I} - \varvec{V} \varvec{V}^*) \right\| _{}, \end{aligned}$$

while $\varvec{I} - \varvec{U} \varvec{U}^*$ and $\varvec{I} - \varvec{V} \varvec{V}^*$ are projectors onto $\mathscr {U}^\perp$ and $\mathscr {V}^\perp$, respectively. This completes the proof. $\square$