Note that the vectors u and v computed in each iteration are orthogonal, which means that \Vert (u+v)/\sqrt{2} \Vert = \Vert (u-v)/\sqrt{2} \Vert = \Vert v \Vert = 1. Hence the actions of the algorithm are always in \mathbb B^d_1. The main argument of the proof is based on an induction to show that with high probability when the execution of the algorithm ends, there exists an s \in \{\pm 1\} such that for all w \in \mathcal E,
(a) w \in \mathcal E, \Vert w \Vert^2 \geq \tau^2; and
(b) s \langle w, \theta_\star \rangle \in [\frac{1}{2}\Vert w \Vert^2, 2 \Vert w \Vert^2].
We proceed in five steps. First, we prove that if the above holds and the algorithm halts before n rounds are over, then the vector returned is a suitable approximation of \theta_\star / \Vert \theta_\star \Vert. Second, we upper bound the probability of certain bad events. In the third and fourth steps we prove the base case and induction step for (a) and (b). In the last step we bound the expected running time.
paragraph4 1.5ex 0.5ex 0.2ex -1em 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 Step 1: Correctness Suppose that (a) and (b) above hold and the algorithm halts at the end of iteration k. Then,
\begin{aligned}
\langle \widehat A_w, \theta_\star \rangle^2
=\left\langle \frac{\sum_{w \in \mathcal E} w}{\Vert \sum_{w \in \mathcal E} w \Vert},\, \theta_\star \right\rangle^2
\geq \frac{1}{4} \sum_{w \in \mathcal E} \Vert w \Vert^2
\geq \frac{r^2}{64}\,.
\end{aligned}
where the first inequality follows from orthogonality of w \in \mathcal E and (b) above. The second inequality follows from the stopping condition in Line ? of ?, (a) above and the definition of \tau. Part (2) of the theorem follows by showing that (a) and (b) above hold with probability at least 1 - 1/n.
paragraph4 1.5ex 0.5ex 0.2ex -1em 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 Step 2: Failure events The algorithm computes some kind of estimator at the end of each do/loop. Since the algorithm cannot play more than n actions, the number of estimators computed is naively upper bounded by n. A union bound over all estimates and the concentration bounds in Lemma 8 and Corollary 11 show that with probability at least 1 - 1/n the following both hold:
For all v sampled in the first iteration and corresponding average rewards \bar X,
\begin{aligned}
\left|\bar X - \langle v, \theta_\star \rangle^2\right| \leq \sqrt{\frac{2 \log(2n^2)}{m}} = \frac{\tau^2}{2}\,,
\end{aligned}
where m is defined in Line ? of ?.
Let \mathcal D = (A_s)_s be the actions played in the inner loop of some iteration k \geq 2 and \hat \theta be the corresponding least-squares estimator. Then,
\begin{aligned}
\sum_{a \in \mathcal D} \langle a, \hat \theta - \theta_\star \rangle^2 \langle a, \hat \theta + \theta_\star \rangle^2 \leq 9(\log(98) + 4 \log(n)) \triangleq \beta\,.
\end{aligned}
We assume both of the above hold in all relevant iterations for the remainder.
paragraph4 1.5ex 0.5ex 0.2ex -1em 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 Step 3: Base case The next step is to show that (a) and (b) hold with high probability after the first iteration. Consider the operation of the algorithm in the inner loop. After sampling v \in \mathbb S^{d-1}_1, the algorithm plays v for m rounds and computes the average reward. Let v be the last sampled action before the iteration halts and w = v \sqrt{\bar X}. By the stopping condition in Line ?, \Vert w \Vert^2 = \bar X \geq \tau^2. Without loss of generality, we choose the sign of \theta_\star so that \langle v, \theta_\star \rangle \geq 0. Then by ?,
\begin{aligned}
\langle w, \theta_\star \rangle
= \sqrt{\bar X} \langle v, \theta_\star \rangle
\in \left[\frac{1}{2} \Vert w \Vert^2, 2 \Vert w \Vert^2\right]\,.
\end{aligned}
This establishes the base case.
paragraph4 1.5ex 0.5ex 0.2ex -1em 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 Step 4: Inductive step Assume that (a) and (b) above hold for \mathcal E at the end of iteration k. Let u be the value computed in Line ? of ?. Then,
\begin{aligned}
\langle u, \theta_\star \rangle
&= \frac{\sum_{w \in \mathcal E} \langle w, \theta_\star \rangle}{\sqrt{\sum_{w \in \mathcal E} \Vert w \Vert^2}}
\geq \frac{1}{2} \sqrt{\sum_{w \in \mathcal E} \Vert w \Vert^2}
\geq \frac{\tau \sqrt{k}}{2}\,.
\end{aligned}
Let \mathcal A = \{(u + v)/\sqrt{2}, (u - v)/\sqrt{2}, v\}, which is the set of actions played in the inner loop of iteration k+1 after sampling v for the last time. Let \hat \theta be the corresponding least-squares estimate. We consider two cases. First, if \langle u, \hat \theta + \theta_\star \rangle \geq 2 |\langle v, \hat \theta + \theta_\star \rangle|, then by ?,
\begin{aligned}
\frac{\beta}{m}
&\geq \sum_{a \in \mathcal A} \langle a, \theta_\star - \hat \theta \rangle^2 \langle a, \theta_\star + \hat \theta \rangle^2 \\
&\geq \frac{1}{16} \langle u + v, \theta_\star - \hat \theta \rangle^2 \langle u, \theta_\star \rangle^2 + \frac{1}{16} \langle u - v, \theta_\star - \hat \theta \rangle^2 \langle u, \theta_\star \rangle^2 \\
&\geq \frac{1}{8} \langle v, \theta_\star - \hat \theta \rangle^2 \langle u, \theta_\star \rangle^2 \geq \frac{k r^2}{16d} \langle v, \theta_\star - \hat \theta \rangle^2\,.
\end{aligned}
Rearranging shows that
\begin{aligned}
\langle v, \theta_\star - \hat \theta \rangle^2 \leq \frac{16d \beta}{m k r^2} \leq \frac{\tau^2}{4}\,.
\end{aligned}
For the second case, \langle u, \hat \theta + \theta_\star \rangle \leq 2 |\langle v, \hat \theta + \theta_\star \rangle|. Then,
\begin{aligned}
\frac{\beta}{m}
\geq \sum_{a \in \mathcal A} \langle a, \theta_\star - \hat \theta \rangle^2 \langle a, \theta_\star + \hat \theta \rangle^2
\geq \frac{1}{4} \langle v, \hat \theta - \hat \theta \rangle^2 \langle u, \hat \theta + \theta_\star \rangle
\geq \frac{k r^2}{8d} \langle v, \theta_\star - \hat \theta \rangle^2\,.
\end{aligned}
And again, ? holds. Summarising, \langle v, \hat \theta \rangle is an estimator of \langle v, \theta_\star \rangle up to accuracy \tau / 2. By the definition of the algorithm, the iteration only ends if |\langle v, \hat \theta \rangle| \geq \tau. Therefore, with w = \langle v, \hat \theta \rangle v, we have \Vert w \Vert^2 = \langle v, \hat \theta \rangle^2 \geq \tau^2. Furthermore, \langle w, \theta_\star \rangle = \langle v, \hat \theta \rangle \langle v, \theta_\star \rangle
\in [\Vert w \Vert^2/2, 2 \Vert w \Vert^2]\,. Therefore if (a) and (b) hold for \mathcal E computed after iteration k, they also hold for \mathcal E computed after iteration k+1.
paragraph4 1.5ex 0.5ex 0.2ex -1em 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 7 2 5 3 4 3 3 Step 5: Running time The length of an iteration is determined by the corresponding value of m and the number of samples of v. The former is an iteration-dependent constant, while the latter depends principally on how many samples are needed before |\langle v, \theta_\star \rangle| is suitably large. The law of \nu is the uniform distribution on \mathbb S^{d-1}_1 \cap \operatorname{span}(\mathcal E)^\perp, which is the uniform distribution on a sphere of dimension d-1-|\mathcal E| embedded in \mathbb R^d. The squared norm of the projection of \theta_\star onto \operatorname{span}(\mathcal E)^\perp is
\begin{aligned}
\Vert \theta_\star \Vert^2 - \sum_{w \in \mathcal E} \frac{\langle \theta_\star, w \rangle^2}{\Vert w \Vert^2}
\geq r^2 - 4\sum_{w \in \mathcal E} \Vert w \Vert^2
\geq \frac{r^2}{2}\,,
\end{aligned}
where we used (a) of the induction and the stopping condition in Line ?. Therefore, when v is sampled uniformly from \mathbb S^{d-1}_1 \cap \operatorname{span}(\mathcal E)^\perp, by Lemma 9,
\begin{aligned}
\mathbb P\left(\langle v, \theta_\star \rangle^2 \geq 3\tau^2/2\right)
&= \mathbb P\left(\langle v, \theta_\star \rangle^2 \geq \frac{3r^2}{2d}\right)
\geq \operatorname{const} > 0\,,
\end{aligned}
Furthermore, by the concentration analysis in the previous step, an iteration will end once a v has been sampled for which \langle v, \theta_\star \rangle^2 \geq 3\tau^2 / 2. Hence, the expected number of times the algorithm samples v per iteration is constant and a simple calculation using the definition of m in Lines ? and ? shows that the expected number of rounds used by the algorithm is at most
\begin{aligned}
\mathbb E[T] &\leq \operatorname{const} \frac{d^2}{r^4} \log\left(n\right) \log(d)\,.
\qedhere
\end{aligned}