Average-Case Rigidity Lower Bounds

Abstract

It is shown that there exists $f : \{0,1\}^{n/2} \times \{0,1\}^{n/2} \rightarrow \{0,1\}$ in E$^\mathbf {NP}$ such that for every $2^{n/2} \times 2^{n/2}$ matrix M of rank $\le \rho$ we have $\mathbb {P}_{x,y}[f(x,y)\ne M_{x,y}] \ge 1/2-2^{-\varOmega (k)}$, whenever $\log \rho \le \delta n/k(\log n + k)$ for a sufficiently small $\delta > 0$, and n is large enough. This generalizes recent results which bound below the probability by $1/2-\varOmega (1)$ or apply to constant-depth circuits.

Supported by NSF CCF award 1813930.

Appendices

A Proof of Lemma 3

Proof

The first column of A is $\langle i,u \rangle$, row-indexed by $i \in \mathrm {\{0,1\}}^m$. The second column of A is all 1, while the third column of A is all b. The second row of B is $\langle j,u \rangle$, column-indexed by $j \in \mathrm {\{0,1\}}^m$, while every other entry in B is 1.

B Proof of Lemma 4

Proof

The algorithm on input $\widetilde{\pi } = \sum _{i = 1}^m \alpha _i \cdot (-1)^{A^{(i)}B^{(i)}}$ runs as follows:

• 1. Initialize the result $\mathsf {res}$ to be 0.

• 2. For each $R_\mathsf {shared}\in \mathrm {\{0,1\}}^{r_\mathsf {shared}}$:

  • (a) Compute the decision function $D = D(1^n;R_{\mathsf {shared}})$ and randomness parity check $(C_1, \dots , C_p) = (C_1(1^n; R_{\mathsf {shared}}), \dots , C_p(1^n; R_{\mathsf {shared}}))$.

  • (b) For each $k \in [q]$, for each $i \in [m]$,

    • i Compute the $2^{r_\mathsf {rect}} \times \rho$ matrices $A^{(k,i)}$ whose $R_\mathsf {row}$-th row is the row of $A^{(i)}$ indexed by $i^{(k)}(1^n; R_\mathsf {row}, R_\mathsf {shared})$ for all $R_\mathsf {row}\in \mathrm {\{0,1\}}^{r_\mathsf {rect}}$.

    • ii Compute the $\rho \times 2^{r_\mathsf {rect}}$ matrices $B^{(k,i)}$ whose $R_\mathsf {col}$-th column is the column of $B^{(i)}$ indexed by $j^{(k)}(1^n; R_\mathsf {col}, R_\mathsf {shared})$ for all $R_\mathsf {col}\in \mathrm {\{0,1\}}^{r_\mathsf {rect}}$.

  • (c) For each $j \in [p]$,

    • i Compute the $2^{r_\mathsf {rect}} \times 3$ matrix $A^{(q+j,1)}$ and the $3 \times 2^{r_\mathsf {rect}}$ matrix $B^{(q+j, 1)}$ with $\left( (-1)^{A^{(q+j,1)}B^{(q+j,1)}}\right) _{R_\mathsf {row},R_\mathsf {col}} = C_j(R_\mathsf {row}, R_\mathsf {col})$ given by Lemma 3.

  • (d) Now we define q m-sums of rank-$\rho$ matrices, $\widetilde{Q}_{k} = C \sum _{i = 1}^m b_i \cdot (-1)^{A^{(k,i)}B^{(k,i)}}$ for each $k \in [q]$, and p 1-sums of rank-3 matrices, $\widetilde{Q}_{q+j} = (-1)^{A^{(q+j,1)} B^{(q+j, 1)}}$ for each $j \in [p]$. Apply Theorem 2 to calculate the following value and add it to $\mathsf {res}$:

    $$ \mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}}\left[ \left( \overline{D}(\widetilde{Q}_1, \dots , \widetilde{Q}_q, \widetilde{Q}_{q+1}, \dots , \widetilde{Q}_{q+p})\right) _{R_\mathsf {row}, R_\mathsf {col}}\right] . $$

• 3. Return $\mathsf {res}$ as the value of $\mathrm {\mathbb {E}}_R\left[ \widetilde{V}^{\widetilde{\pi }}(1^n;R)\right]$.

Correctness of the algorithm follows from Definition 6.

Step 2(b) runs in time $O(2^{r_\mathsf {rect}} \cdot (t + m\rho ))$, while Step 2(c) runs in $O(r 2^{r_\mathsf {rect}})$ by Lemma 3, which is dominated by the runtime of Step 2(b) since $t \ge r$. By Theorem 2, Step 2(d) takes time $O\big (2^{q+p}\cdot m^{q+p}\cdot (T(2^{r_\mathsf {rect}}, (q+p)\rho ) + \mathrm {{poly}}(n, q+p, r))\big ) = O\left( 2^{q+p}\cdot m^{q+p}\cdot T(2^{r_\mathsf {rect}}, (q+p)\rho )\right) \mathrm {{poly}}(n)$, if $(q+p)\rho = \left( 2^{r_\mathsf {rect}}\right) ^{o(1)}$, i.e. $\log ((q+p)\rho ) = o(r)$. Therefore the running time of the above algorithm is

$$\begin{aligned}&O\left( 2^{r_\mathsf {shared}} \cdot \left( 2^{r_\mathsf {rect}} \cdot (t + m\rho ) + 2^{q+p}\cdot m^{q+p}\cdot T(2^{r_\mathsf {rect}}, (q+p)\rho ) \right) \right) \mathrm {{poly}}(n) \\ =~&O\left( 2^{r_\mathsf {shared}+r_\mathsf {rect}} \cdot (t+m\rho ) + m^{q+p} \cdot 2^{q+p+r - \varOmega (r / \log ((q+p)\rho ))} \right) \mathrm {{poly}}(n). \end{aligned}$$

C Proof of Lemma 7

Proof

Fix an arbitrary $R_\mathsf {shared}\in \mathrm {\{0,1\}}^{r_\mathsf {shared}}$. By definition $\mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}} \left[ \left| V^\pi (1^n;R) - \widetilde{V}^{\widetilde{\pi }}(1^n;R)\right| \right]$ is

$$\begin{aligned} \mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}}&\left[ \left| D(\pi _{i^{(1)}, j^{(1)} }, \dots , \pi _{i^{(q)}, j^{(q)} }, C_1(R_\mathsf {row}, R_\mathsf {col}), \dots , C_p(R_\mathsf {row}, R_\mathsf {col}) ) \right. \right. \nonumber \\&\qquad \left. \left. - D(\widetilde{\pi }_{i^{(1)}, j^{(1)} }, \dots , \widetilde{\pi }_{i^{(q)}, j^{(q)} }, C_1(R_\mathsf {row}, R_\mathsf {col}), \dots , C_p(R_\mathsf {row}, R_\mathsf {col}) ) \right| \right] , \end{aligned}$$

(2)

where $D = D(1^n; R_\mathsf {shared})$, $(C_1, \dots , C_p) = (C_1(1^n; R_\mathsf {shared}), \dots , C_p(1^n; R_\mathsf {shared}))$, $i^{(k)} = i^{(k)}(1^n; R_\mathsf {row}, R_\mathsf {shared})$ and $j^{(k)} = j^{(k)}(1^n; R_\mathsf {row}, R_\mathsf {shared})$ for all $k \in [q]$.

We write D in its Fourier expansion $D(z_1, \dots , z_{q+p}) = \sum _{S \subseteq [q+p]} \beta _S \prod _{k \in S} z_k$, where for each $S \subseteq [q+p]$, $\beta _S = \mathrm {\mathbb {E}}_{z \in \mathrm {\{-1,1\}}^{q+p}} \left[ D(z) \prod _{k \in S} z_k\right]$. For all $z\in \mathrm {\{-1,1\}}^{q+p}$, $D(z) \in \mathrm {\{0,1\}}$ and $\prod _{k \in S} z_k \in \mathrm {\{-1,1\}}$, thus $\left| \beta _S\right| \le 1$ for any S. Hence by the triangular inequality we can bound (2) by

$$\begin{aligned}&\sum _{S \subseteq [q+p]} \mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}}\left[ \left| \left( \prod _{k \in S \cap [q]} \pi _{i^{(k)}, j^{(k)}} - \prod _{k \in S \cap [q]} \widetilde{\pi }_{i^{(k)}, j^{(k)}}\right) \prod _{k \in S \setminus [q]} C_k(R_\mathsf {row}, R_\mathsf {col}) \right| \right] \nonumber \\ =~&2^p \sum _{S \subseteq [q]} \mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}}\left[ \left| \prod _{k \in S } \pi _{i^{(k)}, j^{(k)}} - \prod _{k \in S} \widetilde{\pi }_{i^{(k)}, j^{(k)}} \right| \right] , \end{aligned}$$

(3)

as all the $C_k$’s are $\mathrm {\{-1,1\}}$-valued.

Fix any $S \subseteq [q]$. Wlog let $S = \{1, \dots , d\}$ for some $d \le q$, then the expectation in (3) can be written as

$$\begin{aligned}&\mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}}\left[ \left| \prod _{u=1}^d \pi _{i^{(u)}, j^{(u)}} - \prod _{u=1}^d \widetilde{\pi }_{i^{(u)}, j^{(u)}} \right| \right] \\ =~&\mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}}\left[ \left| \sum _{v=1}^d \left( \prod _{u=1}^{v -1} \widetilde{\pi }_{i^{(u)}, j^{(u)}} \prod _{u=v}^d \pi _{i^{(u)}, j^{(u)}} - \prod _{u=1}^{v} \widetilde{\pi }_{i^{(u)}, j^{(u)}} \prod _{u=v+1}^d \pi _{i^{(u)}, j^{(u)}} \right) \right| \right] \\ \le ~&\sum _{v=1}^d \mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}} \left[ \left| \prod _{u=1}^{v -1} \widetilde{\pi }_{i^{(u)}, j^{(u)}} \prod _{u=v}^d \pi _{i^{(u)}, j^{(u)}} - \prod _{u=1}^{v} \widetilde{\pi }_{i^{(u)}, j^{(u)}} \prod _{u=v+1}^d \pi _{i^{(u)}, j^{(u)}} \right| \right] \\ =~&\sum _{v=1}^d \mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}} \left[ \left| \prod _{u=1}^{v -1} \widetilde{\pi }_{i^{(u)}, j^{(u)}} \cdot \left( \pi _{i^{(v)}, j^{(v)}} - \widetilde{\pi }_{i^{(v)}, j^{(v)}} \right) \cdot \prod _{u=v+1}^d \pi _{i^{(u)}, j^{(u)}} \right| \right] \\ \le ~&\sum _{v=1}^d \left( \mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}} \left[ \left( \pi _{i^{(v)}, j^{(v)}} - \widetilde{\pi }_{i^{(v)}, j^{(v)}} \right) ^2 \right] \right) ^{1/2} \left( \mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}} \left[ \prod _{u=1}^{v -1} \widetilde{\pi }^2_{i^{(u)}, j^{(u)}} \prod _{u=v+1}^d \pi ^2_{i^{(u)}, j^{(u)}} \right] \right) ^{1/2} \\ \le ~&\sum _{v=1}^d \left( \mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}} \left[ \left( \pi _{i^{(v)}, j^{(v)}} - \widetilde{\pi }_{i^{(v)}, j^{(v)}} \right) ^2 \right] \right) ^{1/2} \\ =~&\sum _{k \in S} \left( \mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}} \left[ \left( \pi _{i^{(k)}, j^{(k)}} - \widetilde{\pi }_{i^{(k)}, j^{(k)}} \right) ^2 \right] \right) ^{1/2}, \end{aligned}$$

where the first inequality comes from the triangular inequality, the second inequality follows from the Cauchy-Schwarz inequality, and the last inequality follows from the assumptions that $\pi \in \mathrm {\{-1,1\}}^{\ell \times \ell }$ and $\widetilde{\pi }$ is bounded for V.

Summing over S, we can bound (3) by

$$\begin{aligned}&2^p \sum _{S \subseteq [q]} \sum _{k \in S} \left( \mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}} \left[ \left( \pi _{i^{(k)}, j^{(k)}} - \widetilde{\pi }_{i^{(k)}, j^{(k)}} \right) ^2 \right] \right) ^{1/2} \\ =~&2^{p + q - 1} \sum _{k \in [q]} \left( \mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}} \left[ \left( \pi _{i^{(k)}, j^{(k)}} - \widetilde{\pi }_{i^{(k)}, j^{(k)}} \right) ^2 \right] \right) ^{1/2} \\ =~&2^{O(p+q)} \mathrm {\mathbb {E}}_{k \in [q]} \left[ \left( \mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}} \left[ \left( \pi _{i^{(k)}, j^{(k)}} - \widetilde{\pi }_{i^{(k)}, j^{(k)}} \right) ^2 \right] \right) ^{1/2}\right] \\ \le ~&2^{O(p+q)} \left( \mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}, k} \left[ \left( \pi _{i^{(k)}, j^{(k)}} - \widetilde{\pi }_{i^{(k)}, j^{(k)}} \right) ^2 \right] \right) ^{1/2}, \end{aligned}$$

where the first step uses double counting, and the last step follows from Jensen’s inequality.

Therefore by averaging over $R_\mathsf {shared}$, we have

$$\begin{aligned}&\left| \mathrm {\mathbb {E}}_R\left[ V^\pi (1^n;R)\right] - \mathrm {\mathbb {E}}_R\left[ \widetilde{V}^{\widetilde{\pi }}(1^n;R) \right] \right| \\ \le ~&\mathrm {\mathbb {E}}_{R_\mathsf {shared}}\mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}} \left[ \left| V^\pi (1^n;R) - \widetilde{V}^{\widetilde{\pi }}(1^n;R) \right| \right] \\ \le ~&2^{O(p+q)} \mathrm {\mathbb {E}}_{R_\mathsf {shared}} \left[ \left( \mathrm {\mathbb {E}}_{R_\mathsf {row}, R_\mathsf {col}, k} \left[ \left( \pi _{i^{(k)}, j^{(k)}} - \widetilde{\pi }_{i^{(k)}, j^{(k)}} \right) ^2 \right] \right) ^{1/2} \right] \\ \le ~&2^{O(p+q)} \left( \mathrm {\mathbb {E}}_{R_\mathsf {shared}, R_\mathsf {row}, R_\mathsf {col}, k} \left[ \left( \pi _{i^{(k)}, j^{(k)}} - \widetilde{\pi }_{i^{(k)}, j^{(k)}} \right) ^2 \right] \right) ^{1/2} \\ =~&2^{O(p+q)} \left( \mathrm {\mathbb {E}}_{i,j} \left[ \left( \pi _{i,j} - \widetilde{\pi }_{i,j} \right) ^2 \right] \right) ^{1/2} \\ =~&2^{O(p+q)} \left\| \pi - \widetilde{\pi }\right\| _2, \end{aligned}$$

where the first step uses triangular inequality, the second step uses the above bound for every $R_\mathsf {shared}$, the third step comes from Jensen’s inequality, and the fourth step follows from the smoothness of V.

D Proof of Lemma 8

Proof

We prove it by induction on k. For $k=1$ it is trivial as h is bounded. Now we assume that the hypothesis holds for $k-1$, and we are proving for k.

For all $x_1 \in \mathrm {\{0,1\}}^n$, define $g(x_1) = \mathrm {\mathbb {E}}_{y \sim \mathrm {\{0,1\}}^{n(k-1)}}\left[ f^{\oplus k-1}(y)h(x_1, y)\right]$, where we use y for $(x_2, \dots , x_k)$ for convenience. If there exists $x_1 \in \mathrm {\{0,1\}}^n$ such that $|g(x_1)| \ge \mathrm {\varepsilon }_{k-1}$, then we know that $f^{\oplus k-1}$ $\mathrm {\varepsilon }_{k-1}$-correlates with $h'$ defined by $h'(y) = h(x_1,y)$, so we can use the induction hypothesis for $k-1$ to get a bounded m-sum of functions obtained by fixing inputs of $h'$, thus by fixing inputs of h.

Otherwise, for all $x_1 \in \mathrm {\{0,1\}}^n$ we have $\left| g(x_1)\right| \le \mathrm {\varepsilon }_{k-1} = \frac{2\mathrm {\varepsilon }_k}{1+ \mathrm {\varepsilon }}$. We take m i.i.d. samples $y_1, \dots , y_m$ uniformly from $\mathrm {\{0,1\}}^{n(k-1)}$ for $m = O\left( \frac{n}{(\mathrm {\varepsilon }_k)^2}\right)$, then define $\widetilde{g}(x_1) = \mathrm {\mathbb {E}}_{i \in [m]} \left[ f^{\oplus k-1}(y_i)h(x_1, y_i)\right]$. By Chernoff bound,

$$ \Pr _{y_1, \dots , y_m} \left[ \left| g(x_1) - \widetilde{g}(x_1)\right| \ge \frac{1-\mathrm {\varepsilon }}{(1+\mathrm {\varepsilon })^2}\mathrm {\varepsilon }_k \right] \le 2^{-n-1}. $$

By union bound, there exists a fix assignment to $y_1, \dots , y_m$ such that for all $x_1 \in \mathrm {\{0,1\}}^n$,

$$\begin{aligned} \left| g(x_1) - \widetilde{g}(x_1)\right| \le \frac{1-\mathrm {\varepsilon }}{(1+\mathrm {\varepsilon })^2}\mathrm {\varepsilon }_k, \end{aligned}$$

(4)

thus $\left| \widetilde{g}(x_1)\right| \le \left| g(x_1)\right| + \left| g(x_1) - \widetilde{g}(x_1)\right| \le \left( \frac{2}{1+\mathrm {\varepsilon }} + \frac{1-\mathrm {\varepsilon }}{(1+\mathrm {\varepsilon })^2}\right) \mathrm {\varepsilon }_k = \frac{3 + \mathrm {\varepsilon }}{(1 + \mathrm {\varepsilon })^2} \mathrm {\varepsilon }_k$.

Let $r = \frac{3 + \mathrm {\varepsilon }}{(1 + \mathrm {\varepsilon })^2} \mathrm {\varepsilon }_k$. We define $\widetilde{h}$ by

$$ \widetilde{h}(x_1) = \frac{\widetilde{g}(x_1)}{r} = \frac{1}{mr} \sum _{i=1}^m f^{\oplus k-1}(y_i)h(x_1, y_i). $$

Now $|\widetilde{h}(x_1)| \le 1$ for all $x_1$. We can write $\widetilde{h}$ as $C \sum _{i = 1}^m b_i h_i$, where

$$\begin{aligned} C&= \frac{1}{mr},\\ b_i&= f^{\oplus k-1}(y_i), \forall i \in [m], \\ h_i&:x_1 \mapsto h(x_1, y_i), \forall x_1 \in \mathrm {\{0,1\}}^n, \forall i \in [m], \end{aligned}$$

which is a bounded $O(n/\mathrm {\varepsilon }_k^2)$-sum of functions that can be obtained by fixing inputs of h. The bit-complexity of m is $O(k + \log n)$, and O(k) for r, thus the bit-complexity of $\widetilde{h}$ is $O(k + \log n)$.

What remains is to prove that $\mathsf {corr}(f, \widetilde{h}) \ge \mathrm {\varepsilon }$. By the definition of g and assumption we have $\mathsf {corr}(f,g) = \mathsf {corr}(f^{\oplus k}, h) \ge \mathrm {\varepsilon }_k$. Therefore by the definition of $\widetilde{h}$, the fact that $f(x_1) \in \mathrm {\{-1,1\}}$ for all $x_1$, and (4), we have

$$\begin{aligned} \mathsf {corr}(f, \widetilde{h})&= \frac{\mathsf {corr}(f, \widetilde{g})}{r} \\&\ge \frac{1}{r}\left( \mathsf {corr}(f, g) - \mathrm {\mathbb {E}}_{x_1}\left| g(x_1) - \widetilde{g}(x_1)\right| \right) \\&\ge \frac{\mathrm {\varepsilon }_k - \frac{1-\mathrm {\varepsilon }}{(1+\mathrm {\varepsilon })^2}\mathrm {\varepsilon }_k }{ \frac{3 + \mathrm {\varepsilon }}{(1 + \mathrm {\varepsilon })^2} \mathrm {\varepsilon }_k} \\&= \mathrm {\varepsilon }. \end{aligned}$$