Deploying graph neural networks (GNNs) on whole-graph classification or regression tasks is known to be challenging: it often requires computing node features that are mindful of both local interactions in their neighbourhood and the global context of the graph structure. GNN architectures that navigate this space need to avoid pathological behaviours, such as bottlenecks and oversquashing, while ideally having linear time and space complexity requirements. In this work, we propose an elegant approach based on propagating information over expander graphs. We leverage an efficient method for constructing expander graphs of a given size, and use this insight to propose the EGP model. We show that EGP is able to address all of the above concerns, while requiring minimal effort to set up, and provide evidence of its empirical utility on relevant graph classification datasets and baselines in the Open Graph Benchmark. Importantly, using expander graphs as a template for message passing necessarily gives rise to negative curvature. While this appears to be counterintuitive in light of recent related work on oversquashing, we theoretically demonstrate that negatively curved edges are likely to be required to obtain scalable message passing without bottlenecks. To the best of our knowledge, this is a previously unstudied result in the context of graph representation learning, and we believe our analysis paves the way to a novel class of scalable methods to counter oversquashing in GNNs.
For a finite connected graph G = (V(G), E(G)), we consider functions f \colon V(G) \rightarrow \mathbb{R}. The Laplacian L f \colon V(G)\rightarrow \mathbb{R} of such a function is defined to be
where \mathrm{deg}(v) is the degree of the vertex v.
An infinite collection \{ G_i \} of finite connected graphs is an expander family if there is a constant c > 0 such that for all G_i in the collection, \lambda_1(G_i) \geq c.
Let G be a finite graph. For A \subset V(G), its boundary \partial A is the collection of edges with one endpoint in A and one endpoint not in A. The Cheeger constant h(G) is defined to be
Let \{ G_i \} be an infinite collection of finite connected graphs with a uniform upper bound on their vertex degrees. Then the following are equivalent:
\{ G_i \} is an expander family;
there is a constant \epsilon > 0 such that for all graphs in the collection, h(G_i) \geq \epsilon.
The diameter \mathrm{diam}(G) of a graph G satisfies
where \Delta(G) is the maximal degree of any vertex of G. Hence, if \{ G_i \} is an expander family of finite graphs with a uniform upper bound on their vertex degrees, then there is a constant k > 0 such that for all graphs in the family,
A group (\Gamma, \circ) is a set \Gamma equipped with a composition operation \circ : \Gamma \times \Gamma \rightarrow \Gamma (written concisely by omitting \circ, i.e. g\circ h = gh, for g, h \in\Gamma), satisfying the following axioms:
(Associativity) (gh)l = g(hl), for g, h, l\in\Gamma.
(Identity) There exists a unique e\in\Gamma satisfying eg=ge=g for all g\in\Gamma.
(Inverse) For every g\in\Gamma there exists a unique g^{-1}\in\Gamma such that gg^{-1} = g^{-1}g=e.
Let \Gamma be a group. A subset S\subseteq\Gamma is a generating set for \Gamma if it can be used to “generate” all of \Gamma via composition. Concretely, any element g\in\Gamma can be expressed by composing elements in the generating set, or their inverses; that is, we can express g=s_1^{\pm 1}s_2^{\pm 1}s_3^{\pm 1}\cdots s_{n-1}^{\pm 1}s_n^{\pm 1} for s_i\in S.
Let \Gamma be a group with a finite generating set S. Then the associated Cayley graph \mathrm{Cay}(\Gamma;S) has vertex set \Gamma and it has an edge g\rightarrow gs for each g \in \Gamma and each s \in S. We say that s is the label on this edge. This is a potentially non-simple graph, as it may have edges with both endpoints on the same vertex and it may have multiple edges between a pair of vertices. In particular, when s has order 2, then we view the edge g \rightarrow gs and the edge gs \rightarrow gs^2 = g as distinct edges.
The family of Cayley graph \mathrm{Cay}(\mathrm{SL}(2,\mathbb{Z}_n); S_n ) forms an expander family.
Let s be one of
Let n , n' > 18 and let e and e' be s-labelled edges in G_n and G_{n'}. Then there is a graph isomorphism between N_2(e) and N_2(e') taking e to e'.
The balanced Forman curvatures \mathrm{Ric}(n), and the Ollivier curvatures \kappa(n) of all edges of Cayley graphs G_n are given by:
Proposition 11 implies that the balanced Forman and Ollivier curvatures are all equal for n > 18. Their values for 2 \leq n \leq 19 can all be empirically computed, and are given as above.
For any \delta > 0 and \Delta > 0, there are only finitely many graphs with maximum vertex degree \Delta, Cheeger constant at least \delta and non-negative Ollivier curvature.
Note first that, by the homogeneity of the Cayley graphs G_n and G_{n'}, we may assume that e and e' emanate from the identity vertex of each graph.
Let G_\infty be the Cayley graph of \mathrm{SL}(2, \mathbb{Z}) with respect to the generators
Let e_\infty be the s-labelled edge emanating from the identity vertex of G_\infty. The quotient homomorphism
induces a graph homomorphism G_\infty \rightarrow G_n sending e_\infty to e. We will show that it restricts to a graph isomorphism
As there is a similar graph isomorphism N_2(e_\infty) \rightarrow N_2(e'), the proposition will follow.
Note that two elements of \mathrm{SL}(2, \mathbb{Z}) map to the same element of \mathrm{SL}(2, \mathbb{Z}_n) if and only if they differ by multiplication by an element of the kernel K_n. This is
The graph homomorphism sends edges to edges, and so it is distance non-increasing. Hence it certainly sends N_2(e_\infty) to N_2(e). It is also clearly surjective, because any element of N_2(e) is reached from an endpoint of e by a path of length at most 2, and there is a corresponding path in N_2(e_\infty).
We just need to show that this is an injection. If not, then two distinct vertices g_1 and g_2 in N_2(e_\infty) map to the same vertex in N_2(e). Note then that as elements of \mathrm{SL}(2, \mathbb{Z}), g_2 = g_1 k for some k \in K_n. There are paths with length at most 3 joining the identity 1 to g_1 and g_2 respectively. Hence, the distance in G_\infty between g_1 and g_2 is at most 6. Therefore, the distance between 1 and g_1^{-1} g_2 is at most 6. This element g_1^{-1} g_2 lies in K_n. We will show that when n > 18, the only element of K_n that has distance at most 6 from the identity is the identity itself. This will imply that g_1^{-1} g_2 = 1 and hence g_1 = g_2. But this contradicts the assumption that g_1 and g_2 are distinct vertices. Our argument follows that of [margulis1982explicit].
The operator norm ||A|| of a matrix A \in \mathrm{SL}(2, \mathbb{Z}) is
This is submultiplicative: ||AB|| \leq ||A|| \, ||B|| for matrices A and B. It can be calculated as the square root of the largest eigenvalue of A^t A. In our case, the operator norms satisfy
Consider an element
of K_n that is not the identity. Since a \equiv d \equiv 1 modulo n and b \equiv c \equiv 0 modulo n, we deduce that at least one |a|, |b|, |c| and |d| is at least n-1. Therefore, this matrix acts on one of the vectors (1,0)^t or (0,1)^t by scaling its length by at least n-1. Therefore, ||K|| \geq n-1. Suppose now that K has distance at most 6 from the identity. Then K can be written as a word in the generators of \mathrm{SL}(2, \mathbb{Z}) with length at most 6. Therefore, we obtain the inequality
Hence, n < 18.95 and therefore, as n is integral, n \leq 18.
This is a consequence of the main result of Salez [salez2021sparse, Theorem 3]. This states if G_n = (V_n, E_n) is a sequence of graphs with the following properties:
then
Here, \kappa(e) is the Ollivier curvature of an edge e and
are the eigenvalues of the lazy random walk operator. To prove the theorem, we suppose that on the contrary, there are infinitely many distinct graphs G_n = (V_n, E_n) with maximum vertex degree \Delta, Cheeger constant at least \delta and non-negative Olliver curvature. Then
and so condition Proof is satsfied. Condition Proof is trivially satisfied because the Ollivier curvature of each graph is non-negative. Thus, we deduce that the conclusion of Salez’ theorem holds. Setting \rho = 1- (\delta^2/4 \Delta^2), we deduce that a definite proportion of the eigenvalues of the lazy random walk operator are at least 1 - (\delta^2/4\Delta^2). In particular, \mu_1(G_n) \geq 1 - (\delta^2/4\Delta^2). Denote the eigenvalues of the normalised Laplacian by
These are related to the eigenvalues of the lazy random walk operator by \lambda'_i (G_n) = 2 - 2 \mu_i (G_n). Hence, \lambda'_1 (G_n) \leq \delta^2/(2\Delta^2). There is a variation of Cheeger’s inequality that relates \lambda'_1 to the conductance of the graph. To define this, one considers subsets A of the vertex set, and defines their volume to be \mathrm{vol}(A) = \sum_{v \in A} \mathrm{deg}(v). The conductance \phi(G) of a graph G is
Then, by [Chung, Theorem 2.2],
Hence, in our case,
Consider any subset A_n of the vertex set that realises \phi(G_n). Thus 0 < \mathrm{vol}(A_n) \leq \mathrm{vol}(V_n)/2 and |\partial A_n|/\mathrm{vol}(A_n) = \phi(G_n) \leq \delta / \Delta. If A_n is at most half the vertices of G_n, then this implies that the Cheeger constant h(G_n) \leq \delta. On the other hand, if A_n is more than half the vertices of G_n, we consider its complement A_n^c. Its cardinality |A_n^c| satisfies
Hence,
In either case, we deduce that the Cheeger constant of G_n is at most \delta, contradicting one of our hypotheses. Hence, there must have been only finitely many graphs satisfying the conditions of the theorem.
Let r be a positive integer satisfying
Then there is a graph isomorphism between the r-neighbourhood of the identity vertex in G_n and the r-neighbourhood of the identity vertex in G_\infty. This isomorphism takes the identity vertex to the identity vertex.
As shown in the proof of Proposition 11, there is a graph homomorphsm from N_r(1) in G_\infty to N_r(1) in G_n that is a surjection. If it fails to be an injection, then there is a non-trivial element K in the kernel K_n of \mathrm{SL}(2, \mathbb{Z}) \rightarrow \mathrm{SL}(2, \mathbb{Z}_n) satisfying
But any non-trivial element K in K_n satisfies
Rearranging gives the required inequality.
A quasi-isometry between two metric spaces (X_1, d_1) and (X_2, d_2) is a function f \colon X_1 \rightarrow X_2 that satisfies the following two conditions:
there are constants c, C > 0 such that, for every x,x' \in X_1
there is a constant K \geq 0 such that for every y \in X_2, there is an x \in X_1 with d_2(f(x), y) \leq K.
If there is such a quasi-isometry, we say that (X_1, d_1) and (X_2, d_2) are quasi-isometric.
The mixing time for a regular graph G is the minimum value of \ell such that for any starting probability distribution \pi on the vertex set of G,
Here, u is the uniform probability distribution on the vertex set, and M is the lazy random walk operator.
For any k > 0 and \delta > 0, there is a constant c > 0 with the following property. If G is a connected k-regular graph on n vertices with Cheeger constant at least \delta > 0, then the mixing time for G is at most c \log (n).
The text on this page is from arXiv:2210.02997v2 by Andreea Deac, Marc Lackenby and Petar Veličković, under the licence CC BY 4.0. The corpus made these changes: it read the items out of the LaTeX source, it expanded the macros of the paper, it put each proof after its statement, and it converted the text to HTML. The formulas are the TeX of the paper, and KaTeX renders them. A formula or a reference can look different from the PDF. The page is published under CC BY 4.0.
The record of each item is in analysis/objects/2210/2210.02997.jsonl.