Entropy and the growth rate of universal covering trees

To prove that the condition on G is required, assume that the NBRW on G is irreducible. The condition \operatorname{mindeg}(G) \geq 2 must hold since the walk is well defined. In addition, G must be connected, because otherwise one can find edges e, f \in \vec{E} with no non-backtracking path from e to f. It remains to exclude the possibility that G is connected with \operatorname{mindeg}(G) = \operatorname{maxdeg}(G) = 2, namely that G is a cycle. Indeed, in this case, for any e \in \vec{E}, there is no non-backtracking path from e to e'.

To prove that the condition on G implies irreducibility, we first argue that it suffices to find a path P_{h,h'} from some edge h to its inverse h' to imply irreducibility. Indeed, given any pair of edges e,f \in \vec{E}, because G is connected there is a non-backtracking path P_1 connecting e to either h or h' and there is a non-backtracking path P_2 connecting h or h' to f. Therefore, there is a non-backtracking path from e to f formed by concatenating P_1, possibly P_{h,h'} or its inverse, and P_2.

To conclude, observe that the condition \operatorname{mindeg}(G) \geq 2 implies the existence of some non-backtracking cycle C = e_1, e_2, \ldots, e_\ell. As the graph is connected with \operatorname{maxdeg}(G) > 2, there must be a vertex v \in C with \deg(v)>2, and we may assume that t(e_1)=h(e_\ell)=v, as depicted in Figure ?.

Since \deg(v)>2 there is some edge e with h(e)=v distinct from e_1' and e_\ell. Then the path e,e_1, e_2, \ldots, e_\ell,e' is a path from e to e', implying the Markov chain irreducibility.

where the inequality is Jensen’s inequality, and the equality before the last follows by linearity of expectation and the fact that the marginal distribution of e_i is \nu_s for all i.

Therefore by Proposition 7,

The result follows by applying Lemma 9 to lower bound the right hand side.

By definition, R_\ell = \log_2(X_\beta) with \beta(e) = \operatorname{outdeg}(e). Therefore, by the last two equalities in the proof of Lemma 9, we have \mathbb{E}[R_\ell] = \mathbb{E}[\log_2 X_\beta] = \ell \log_2 \Lambda, which is the first equality.

For the second equality, we use Lemma 10 with \beta(e) = \operatorname{outdeg}(e), yielding that \log_2 \rho(B) = \log_2 \lim_{\ell \rightarrow \infty} \left( \mathbb{E}[X_\beta] \right)^\frac{1}{\ell} = \lim_{\ell \rightarrow \infty} \frac{1}{\ell} \log_2 \mathbb{E}[2^{R_\ell}].

2 \implies 1 and 3 \implies 1 are Lemmas 17 and 18; 1 \implies 2 and 1 \implies 3 are Lemmas 23 and 22.

By (3), \mu[\omega]F(\omega)=1/|\vec{E}| for every \omega\in\Omega_\ell, and therefore

By (16) the right hand side lies between c_1\Lambda^\ell and c_2\Lambda^\ell. Taking the \ell-th root and letting \ell\to\infty, Lemma 6 yields \rho(G)=\Lambda(G). The second statement is (16) after taking \log_2.

Given a graph G satisfying the suspended path condition and some non-backtracking walk \omega = (e_0,\ldots,e_\ell)\in \Omega_\ell, let i_1 < \cdots < i_s be the set of indices i<\ell with \operatorname{outdeg}(e_i) > 1. If s>0, then \omega can be expressed as the concatenation P_0 P_1 \cdots P_s, where P_j = (e_{i_j+1},e_{i_j+2},\ldots,e_{i_{j+1}}) for j=1,\ldots,s-1 are suspended paths, P_0 = (e_0,\ldots,e_{i_1}) and P_s = (e_{i_s+1},\ldots,e_\ell) are partial suspended paths. Then:

where the third equality uses the fact that \operatorname{outdeg}(P_j) = \operatorname{indeg}(P_{j+1}), the fourth equality uses the suspended path condition, and where \Theta(1) denotes a factor that is upper and lower bounded by positive constants that depend on G but not on \ell or \omega.

Otherwise, s=0 and F(\omega)=1. As \ell is bounded by the maximal length of a suspended path, it follows that F(\omega) = \Theta(1) \cdot \Lambda(G)^{\ell}.

In both cases we get \rho(G) = \Lambda(G) by Lemma 16.

By Claim 4 the matrix B is irreducible, so for every e,f\in\vec{E} there is a non-backtracking walk e=x_0\to x_1\to\dots\to x_m=f with m\ge 1. Let l_0=l_0(G) be the maximum, over all pairs e,f\in\vec{E}, of the minimal such m.

Let \omega=(e_0,\dots,e_\ell)\in\Omega_\ell, and choose a non-backtracking walk e_\ell=x_0\to x_1\to\dots\to x_m=e_0 with 1\le m\le l_0. Then

is a non-backtracking cycle of length \ell+m. By the cycle condition,

Since both the product \prod_{j=0}^{m-1}\operatorname{outdeg}(x_j) and \Lambda^m are \Theta(1), condition (16) holds, implying that \rho = \Lambda by Lemma 16.

We first note that the matrix \Pi_\beta in Lemma 10, where \beta(e)=\operatorname{outdeg}(e)^t, is equal to M_t. Therefore (10) yields:

On the other hand, by (19)

where the equality holds since \rho(\Pi_G) =1 and \rho(B) = \Lambda(G) by our assumption.

Given an NB-irreducible graph G such that \rho(G) = \Lambda(G), let the matrix M_t be defined as above. Then \rho(M_t)=\rho(G)^t for all 0 \leq t \leq 1, by Lemma 20, and therefore by Theorem 19 we have

where k>0 is constant, and D is a diagonal matrix with positive diagonal entries. Let \lambda = \log k and define \varphi:\vec{E} \to \mathbb{R} by \varphi(e) = \log D_{e,e}.

For any pair of edges e,f such that e \rightarrow f, we have B_{e,f} = 1 and (\Pi_G) _{e,f} = \operatorname{outdeg}(e)^{-1} = \operatorname{indeg}(f)^{-1}. Taking \log of the e,f entry of equation (Proof) yields:

Summing equation (?) over all e \rightarrow f pairs with weight \operatorname{outdeg}(e)^{-1} yields:

Hence, \lambda = -\log \Lambda. Plugging \lambda into (?) yields equation (21), as claimed.

Let C=(e_0,\dots,e_{k-1}) be a non-backtracking cycle. Summing (21) over the transitions e_j\to e_{j+1} for j=0,\dots,k-1, with indices modulo k, the contributions of \varphi cancel, and we obtain \sum_{j=0}^{k-1}\log\operatorname{outdeg}(e_j)=k\log\Lambda, as claimed.

Given an NB-irreducible graph G such that \rho(G)=\Lambda(G), let \varphi be the function given by Lemma 21. For a vertex v \in V(G), denote its incoming and outgoing edges by \vec{E}_{\mathrm{in}}(v) = h^{-1}(v) and \vec{E}_{\mathrm{out}}(v) = t^{-1}(v).

Then, for any pair of edges e \in \vec{E}_{\mathrm{in}}(v) and f \in \vec{E}_{\mathrm{out}}(v) with f\neq e', we have e \rightarrow f and therefore by Lemma 21

If \deg(v) \ge 3, then for any e_1,e_2 \in \vec{E}_{\mathrm{in}}(v), there is an edge f \in \vec{E}_{\mathrm{out}}(v) so that both e_1 \rightarrow f and e_2 \rightarrow f, implying that \varphi(e_1)=\varphi(e_2). Therefore, \varphi is constant on \vec{E}_{\mathrm{in}}(v), and by a similar argument, it is constant on \vec{E}_{\mathrm{out}}(v) as well. We denote these values by \varphi_{\mathrm{in}}(v) and \varphi_{\mathrm{out}}(v) respectively and obtain the following identity, for any v \in V(G) with \deg(v) \ge 3:

Let P = (e_0,\dots,e_{\ell-1}) \in \mathcal{S}, let u = t(e_0) and w = h(e_{\ell-1}), where u,w are not necessarily distinct vertices. By definition, \deg(u),\deg(w)\ge 3 and \operatorname{outdeg}(e_i)=1 for i<\ell-1. Applying equation (21) repeatedly along P and along its inverse P^{-1} gives:

Summing the two equations yields

Applying (?) results in

i.e. (\deg(u)-1)(\deg(w)-1) = \Lambda(G)^{2\ell}, which is the suspended path condition (13) since \operatorname{indeg}(P)=\deg(u)-1 and \operatorname{outdeg}(P)=\deg(w)-1.

Since \mathcal{S} partitions \vec{E}, we have \sum_{P\in \mathcal{S}}|P|=|\vec{E}|, and since all edges of a suspended path except the last one have out-degree one, \prod_{e\in\vec{E}}\operatorname{outdeg}(e)=\prod_{P\in \mathcal{S}}\operatorname{outdeg}(P). As P\mapsto P^{-1} is a bijection of \mathcal{S} with \operatorname{outdeg}(P^{-1})=\operatorname{indeg}(P), we also have \prod_{P\in \mathcal{S}}\operatorname{outdeg}(P)=\prod_{P\in \mathcal{S}}\operatorname{indeg}(P). Therefore

If \rho=\Lambda then g\equiv\Lambda by Theorem 15. Conversely, if g\equiv\alpha on \mathcal{S}, then as (24) is a weighted geometric average of g(P), we must have \alpha = \Lambda, so the suspended path condition holds, and \rho=\Lambda by Theorem 15.

If G has no degree two vertices then all suspended paths have length one. Therefore, for any edge e \in \vec{E} the product \operatorname{outdeg}(e) \cdot \operatorname{indeg}(e) is fixed. This implies that for any vertex v, all its neighbors must have the same degree. As G is connected, G must be either regular or bipartite bi-regular.

It is easy to verify that G^m is NB-irreducible. As any suspended path P \in \mathcal{S}(G) corresponds to a suspended path P^m \in \mathcal{S}(G^m), where \operatorname{indeg}(P^m) = \operatorname{indeg}(P), \operatorname{outdeg}(P^m) = \operatorname{outdeg}(P) and |P^m| = m|P|, it follows that g(P^m)=(g(P))^{1/m}, where g is defined by (?). Therefore, as g(P) = \Lambda(G) for all P, we have g(P^m) = \Lambda(G)^{1/m}. The claim follows using Corollary 24.

Since \rho=\Lambda, Theorem 15 implies that the suspended path condition holds. Therefore, given an NB-path \omega = (e_0,\ldots,e_\ell)\in \Omega_\ell, by Lemmas 17 and 16, \left\vert R_\ell(\omega) - \ell \cdot \log_2 \Lambda \right\vert = O(1). As \mathbb{E}[R_\ell] = \ell \cdot \log_2 \Lambda, this implies that \operatorname{var}[R_\ell] = O(1), as claimed.

Since \rho>\Lambda, Theorem 15 implies that the cycle condition does not hold. Therefore, there is a cycle C with \prod_{e \in C} \operatorname{outdeg}(e) \neq \Lambda(G)^{|C|}. We only prove the claim for the case

as proof for the other case is virtually identical. Let f be some edge in C. Then, the first step is to find a second cycle C' with a strictly larger average log out-degree such that f \in C'. Indeed, since \mathbb{E}[R_\ell] = \ell \log_2 \Lambda, then for any \ell, there is some specific walk \omega=(e_0,e_1,\ldots,e_\ell) \in \Omega_\ell with R_\ell(\omega) \geq \ell \log_2 \Lambda. Therefore, since G is NB-irreducible, there are paths P_1,P_2 respectively from f to e_0 and from e_\ell to f, whose length is bounded by some constant l_0 that depends only on G. Therefore, C' = P_1 \omega P_2 is a cycle with f \in C'. Furthermore,

By taking a sufficiently large \ell, one can ensure that:

We end the construction by looping through C and C' the required amount of times to obtain the cycles C_1 and C_2, whose length is the LCM of |C| and |C'|, meeting the requirements of the lemma.

Given an NB-irreducible graph G with \rho > \Lambda, by Lemma 29 there are two cycles C_1 and C_2 of the same length \ell_0=|C_1| = |C_2|, with different average out-degree, and with some edge f in their intersection. Recall that:

In order to lower-bound the variance, we use the exposure method. We expose some of the edges of \omega = (e_0,e_1,\ldots,e_\ell) and argue that the conditional expectation of {(R_\ell(\omega) - \ell \log_2\Lambda)}^2 given the exposed edges is \Omega(\ell) with high probability. Define the mapping \phi:\vec{E}\to\mathbb{N}, where \phi(e) is the length of the shortest NB-path from e to f if e \neq f and \phi(f) = \ell_0. Also, let \ell_{\max} be the maximum of \phi(e) on e \in \vec{E}. We build the partial exposure mapping \chi from \{0,\ldots,\ell\} to \vec{E} by the following randomized procedure:

The domain of \chi is the set of exposed edge indices, i_1=0 < i_2 < \cdots < i_k=\ell. Therefore, conditioned on \chi, one may write R_\ell as the sum of independent random variables:

where R(\ell',e',e'') is the random variable counting the number of random bits used by a length \ell' NBRW, conditioned on it beginning at e' and ending at e''. Therefore:

where I = I(\chi) = \{i \in \operatorname{dom}(\chi) :\, i+\ell_0 \in \operatorname{dom}(\chi) \mbox{ and } \chi(i) = \chi(i+\ell_0) = f\}. As the number of exposed edges is lower bounded by \ell/\ell_{\max} = \Omega(\ell), it follows that |I| is lower bounded by a binomial random variable X \sim B(\ell/(2\ell_{\max}),p_{\min}), where p_{\min} is a positive constant defined as the minimum of (\Pi^{\phi(e)})_{e,f} \cdot (\Pi^{\ell_0})_{f,f} on e \in \vec{E}. Therefore, \mathbb{E}[|I|] \geq \mathbb{E}[X] = c_1 \ell, where c_1 = p_{\min}/(2 \ell_{\max}). The Chernoff bound on the lower tail of X yields \Pr[|I| \leq c_1 \ell/2] \leq \Pr[X \leq c_1 \ell/2] \leq \exp[-c_1 \ell/2], which is o(1) as \ell grows.

Recall that by our assumption on the cycles C_1, C_2, the variance of R(\ell_0,f,f) is some positive constant c_2. Therefore:

We define R'_\ell = R_\ell - \ell \log_2\Lambda = \sum_{i=0}^{\ell-1} R'_{\ell,i}, where R'_{\ell,i} are dependent, zero mean, identically distributed random variables with R'_{\ell,i}(\omega) = \log_2 \operatorname{outdeg}(e_i) - \log_2\Lambda. Therefore:

where in the last step we use the fact that the NBRW starting from the stationary distribution \nu_s is time-invariant.

Let f denote a column vector where f_e = \log_2 \operatorname{outdeg}(e) - \log_2\Lambda and note that \sum_e f_e = 0. Recall that \Pi=\Pi_G is the NBRW transition matrix. Then:

Let J be the Jordan normal form of \Pi, written as the direct sum of its blocks J=J_1 \oplus J_2 \ldots \oplus J_k, where block J_i corresponds to the eigenvalue \lambda_i. Assume further that \lambda_1 = 1 \geq |\lambda_2| \geq \cdots \geq |\lambda_k| and that \Pi = Q^{-1} J Q. Then:

where in the last equality we can replace J_1 by the scalar zero. This is justified because the leftmost column of Q^{-1} is a multiple of the all ones vector, orthogonal to f^T. The next step is to employ the following identity, holding for any matrix A provided that I-A is invertible:

As I-J_i is invertible for all i>1, it follows that the normalized variance can be written as:

The matrices J_i^{\ell+1} are upper triangular Toeplitz, where (J_i^{\ell+1})_{u,v} = \lambda_i^{\ell+1-v+u} \binom{\ell+1}{v-u} for u \leq v. Therefore, if |\lambda_i| < 1, all entries of the matrix J_i^{\ell+1} are bounded in absolute value by some constant that is a function of G only. Otherwise, if |\lambda_i|=1, then necessarily \lambda_i has algebraic multiplicity of one (see [horn2012matrix], Corollary 8.4.6), so J_i is a scalar and |J_i^{\ell+1}|=1. It follows that the normalized variance converges when \ell\to\infty:

As the right-hand side does not depend on \ell this concludes the proof.

As in Lemma 32, we define R'_\ell = \sum_{i=0}^{\ell-1} R'_{\ell,i}, where R'_{\ell,i} are random variables with R'_{\ell,i}(\omega) = \log_2 \operatorname{outdeg}(e_i) - \log_2\Lambda. Therefore:

In order to avoid 10x10 matrices, we exploit the fact that K_4 minus an edge has only three edge types. Then, the stationary distribution, the NBRW transition matrix, and the vector f with f_e = \log_2 \operatorname{outdeg}(e) - \log_2\Lambda, with row and column order of: uu, vu and uv, are as follows:

Using N_s to denote the matrix with diagonal \nu_s we write:

We next observe that the matrix \Pi is diagonalizable, \Pi = Q^{-1} D Q, with:

where the first row of Q is the left Perron eigenvector, equal to \nu_s up to scaling. In addition, the right Perron eigenvector, which is the first column of Q^{-1}, is necessarily the all one vector up to scaling. Therefore, (f^T N_s Q^{-1})_1 = 0, implying that f^T N_s Q^{-1} D = f^T N_s Q^{-1} E, where E is the matrix obtained by zeroing the first diagonal entry of D. Plugging this into the variance expression yields:

Using (?), the normalized variance expression converges to:

Plugging in the numbers, yields 2/125 on the right hand side, as claimed.