Persistent Gaussian Perturbations Prevent Oversmoothing in Recurrent Graph Neural Networks

Mostafa Haghir Chehreghani

arXiv: 2607.28185v1 · PDF · 30 July 2026
Categories: cs.LG, cs.AI, cs.IT, math.IT
Licence: CC BY 4.0 The site shows the text.

Abstract

Oversmoothing is a fundamental limitation of deep graph neural networks (GNNs), where repeated message passing causes node representations to become increasingly similar, eventually collapsing toward a low-dimensional subspace. This phenomenon limits the effective depth of message-passing architectures and motivates the search for mechanisms that preserve representation diversity. In this paper, we study a recurrent graph neural network in which independent Gaussian noise is injected after every propagation step and analyze the resulting architecture as a stochastic dynamical system. Under a standard global contraction assumption on the deterministic update, we prove that the hidden representations form a geometrically ergodic Markov chain admitting a unique invariant probability measure. Our main theoretical result establishes an explicit positive lower bound on the expected stationary Dirichlet energy, proportional to both the noise variance and the spectral gap of the underlying graph. Consequently, the stationary representations cannot collapse onto the constant manifold, providing a rigorous guarantee that asymptotic oversmoothing is prevented in the sense of non-vanishing Dirichlet energy. Our analysis reveals persistent stochastic perturbations as a fundamentally different mechanism for combating oversmoothing, complementing existing deterministic approaches based on residual connections, normalization, and graph rewiring. Finally, numerical experiments on both linear and nonlinear recurrent graph neural networks closely match the theoretical predictions, illustrating the emergence of a stationary distribution and the predicted dependence of the limiting Dirichlet energy on the noise intensity.

Items 30

Lemma

Lemma 3.1 (Projected Gaussian noise)

47KZ

Let

P = I_N - \frac1N \mathbf1\mathbf1^\top,

and let

\Xi \in \mathbb R^{N\times d}

be a random matrix whose entries are independent standard Gaussian random variables,

\Xi_{ij} \stackrel{\mathrm{i.i.d.}}{\sim} \mathcal N(0,1).

Define the projected noise \bar\Xi = P\Xi. Then the following properties hold.

The projected noise is centered, i.e., \mathbb E[\bar\Xi] = 0.
Each feature column has covariance:
\operatorname{Cov}(\bar\Xi_{:,k}) = P, \qquad k=1,\dots,d.
The expected squared Frobenius norm satisfies:
\boxed{\mathbb E\|\bar\Xi\|_F^2 = (N-1)d.}

Used by Proof, Proof

Proof of Lemma 3.1

Since \bar\Xi = P\Xi and \mathbb E[\Xi]=0, linearity gives \mathbb E[\bar\Xi]=0. For each k=1,\dots,d, the k-th column \Xi_{:,k} is distributed as \mathcal N(0,I_N). Therefore,

\bar\Xi_{:,k} = P\,\Xi_{:,k} \sim \mathcal N(0,PIP) = \mathcal N(0,P),

where we used the symmetry and idempotence of P, i.e., P^2 = P = P^\top. Hence \operatorname{Cov}(\bar\Xi_{:,k}) = P.

Finally,

\mathbb E\|\bar\Xi\|_F^2 = \sum_{k=1}^{d} \mathbb E\|\bar\Xi_{:,k}\|^2 = \sum_{k=1}^{d} \operatorname{Tr}(P) = d\operatorname{Tr}(P).

The projection matrix P = I_N - \frac1N\mathbf1\mathbf1^\top has eigenvalues 1 (multiplicity N-1) and 0 (multiplicity 1), so \operatorname{Tr}(P) = N-1. Therefore:

\mathbb E\|\bar\Xi\|_F^2 = (N-1)d.

Definition

Definition 3.1 (Dirichlet energy)

KYYP

For a representation matrix H\in\mathbb{R}^{N\times d},

\mathcal E(H) = \frac1m \operatorname{\operatorname{Tr}} (H^{\top}LH).

Used by Remark 5.1, Remark 5.2, Proof

Proposition

Proposition 3.2

XM8Y

Assume that G is connected. Then, \mathcal E(H)=0 if and only if all node representations are identical.

Proof of Proposition 3.2

Since L is positive semidefinite,

\mathcal E(H)\ge0.

Moreover,

\ker(L) = \operatorname{span}\{\mathbf1\}.

Hence,

\mathcal E(H)=0

if and only if \bar H=0, which is equivalent to all rows of H being identical.

Definition

Definition 3.2 (Oversmoothing)

09VI

Let

(H^{(t)})_{t\ge0}

be a stochastic sequence of node representations. The sequence is said to exhibit oversmoothing if

\lim_{t\rightarrow\infty} \mathbb E \left[ \mathcal E(H^{(t)}) \right] = 0.

Used by Remark 5.2, Theorem 5.6

Definition

Definition 3.3 (Escape from oversmoothing)

EAUX

The sequence (H^{(t)}) is said to escape oversmoothing if there exists a constant

\delta>0

such that

\liminf_{t\rightarrow\infty} \mathbb E \left[ \mathcal E(H^{(t)}) \right] \ge \delta.

Lemma

Lemma 3.3

Z2X2

For every H\in\mathbb R^{N\times d},

\mathcal E(H) \ge \frac{\lambda_2(L)}{m} \, \|\bar H\|_F^2.

Used by Proof

Proof of Lemma 3.3

Since

\bar H \perp \ker(L),

the Courant–Fischer characterization of the second eigenvalue gives

\operatorname{\operatorname{Tr}} ( \bar H^{\top} L \bar H ) \ge \lambda_2(L) \, \|\bar H\|_F^2.

Dividing by m yields the result.

Assumption

Assumption 4.1 (Graph connectivity)

3E4J

The graph G is finite, connected, and undirected.

Assumption

Assumption 4.2 (Projection invariance)

IO4D

The deterministic update satisfies

PF(H) = PF(PH), \qquad H\in\mathbb R^{N\times d}.

Equivalently, two representations that differ only by a constant vector shared by all nodes produce the same projected update.

Assumption

Assumption 4.3 (Global contraction)

VMX5

The mapping

\Delta:\mathcal X\rightarrow\mathcal X

is globally Lipschitz with constant 0\le\alpha<1, that is,

\| \Delta(H_1)-\Delta(H_2) \|_F \le \alpha \| H_1-H_2 \|_F

for every H_1,H_2\in\mathcal X.

In addition, the origin is a fixed point of the deterministic dynamics,

\boxed{\Delta(0)=0.}

Used by Proof, Proof

Assumption

Assumption 4.4 (Additive noise)

E40M

The random matrices (\Xi^{(t)})_{t\ge0} are independent and identically distributed, independent of the initial condition, centered,

E[\Xi^{(t)}]=0,

and satisfy

E\|\Xi^{(t)}\|_F^2<\infty.

Used by Proof

Proposition

Proposition 4.1 (Projected dynamics define a Markov chain)

TGTO

Suppose Assumption 4.2 holds. Assume further that

F:\mathbb{R}^{N\times d}\rightarrow\mathbb{R}^{N\times d}

is Borel measurable. Then the projected process

(\bar H^{(t)})_{t\ge0}

defined by

\bar H^{(t+1)} = \Delta(\bar H^{(t)}) + \sigma\bar\Xi^{(t)}

is a time-homogeneous Markov chain on the state space \mathcal X=\operatorname{Im}(P).

Proof of Proposition 4.1

Since P is linear and F is Borel measurable, the mapping

\Delta=P\circ (F|_{\mathcal X}):\mathcal X\to\mathcal X,

is Borel measurable. For every t\ge0,

\bar H^{(t+1)} = \Delta(\bar H^{(t)}) + \sigma\bar\Xi^{(t)},

where the random matrices (\bar\Xi^{(t)})_{t\ge0} are independent and identically distributed and are independent of

\sigma\!\left( \bar H^{(0)}, \ldots, \bar H^{(t)} \right).

Let A\subset\mathcal X be a Borel set. Conditioning on the past gives

\begin{aligned} & \mathbb P \!\left( \bar H^{(t+1)} \in A \,\middle|\, \bar H^{(0)}, \ldots, \bar H^{(t)} \right) \\ &= \mathbb P \!\left( \Delta(\bar H^{(t)}) + \sigma\bar\Xi^{(t)} \in A \,\middle|\, \bar H^{(0)}, \ldots, \bar H^{(t)} \right). \end{aligned}

Since \bar\Xi^{(t)} is independent of the entire history,

\mathbb P \!\left( \bar H^{(t+1)} \in A \,\middle|\, \bar H^{(0)}, \ldots, \bar H^{(t)} \right) = K(\bar H^{(t)},A),

where

K(x,A) = \mathbb P \!\left( \Delta(x) + \sigma\bar\Xi \in A \right).

Thus the conditional distribution of \bar H^{(t+1)} depends only on the current state \bar H^{(t)}. Hence (\bar H^{(t)})_{t\ge0} is a Markov chain. Moreover, the kernel K does not depend on t because both the mapping \Delta and the law of the additive noise are time-independent. Therefore the chain is time homogeneous.

Theorem

Theorem 5.1 (Ergodicity of the projected Markov chain)

K977

Suppose that Assumptions 4.1, 4.2, 4.3, and 4.4 hold.

Then the projected Markov chain (\bar H^{(t)})_{t\ge0} with transition kernel K:

admits a unique invariant probability measure \mu;
converges to \mu from every initial condition;
converges geometrically in the Wasserstein metric.

Used by Proof

Proof of Theorem 5.1

The projected dynamics define a contractive iterated random function on the complete metric space

\mathcal X=\operatorname{Im}(P),

endowed with the Frobenius norm. By Assumption 4.3, the mapping

\Delta:\mathcal X\rightarrow\mathcal X

is globally Lipschitz with contraction constant \alpha<1. Moreover, by Assumption 4.4, the projected noise sequence

(\bar\Xi^{(t)})_{t\ge0}

is independent and identically distributed with finite second moments.

Therefore, the hypotheses of the classical ergodicity theorem for contractive iterated random functions are satisfied (see, e.g., [diaconis1999iterated, Theorem 1]; see also [hairer2011]). Consequently, the projected Markov chain admits a unique invariant probability measure, converges to it from every initial condition, and the convergence is geometric in the Wasserstein distance.

Uses Assumption 4.3, Assumption 4.4

Remark

Remark 5.1

7MK7

The Gaussian assumption is not essential for Theorem 5.1. The result remains valid for any independent additive noise sequence having finite second moments. Gaussianity becomes important only in the subsequent analysis, where the covariance structure of the noise is used to derive an explicit stationary variance identity and the lower bound on the Dirichlet energy.

Uses Definition 3.1

Proposition

Proposition 5.2 (Exact stationary second-moment identity)

KCOY

Suppose that Assumptions 4.1, 4.2, 4.3, and 4.4 hold. Then

\boxed{ \mathbb E_\mu \!\left[ \|\bar H\|_F^2 \right] = \mathbb E_\mu \!\left[ \|\Delta(\bar H)\|_F^2 \right] + \sigma^2 (N-1)d. }

Used by Proof

Proof of Proposition 5.2

Since \mu is an invariant probability measure,

\bar H \stackrel{d}{=} \Delta(\bar H) + \sigma\bar\Xi,

where \bar\Xi is independent of \bar H. Therefore,

\begin{aligned} \mathbb E_\mu \!\left[ \|\bar H\|_F^2 \right] &= \mathbb E_\mu \!\left[ \| \Delta(\bar H) + \sigma\bar\Xi \|_F^2 \right] \\ &= \mathbb E_\mu \!\left[ \|\Delta(\bar H)\|_F^2 \right] + 2\sigma \, \mathbb E \!\left[ \left\langle \Delta(\bar H), \bar\Xi \right\rangle_F \right] \\ &\qquad + \sigma^2 \, \mathbb E \!\left[ \|\bar\Xi\|_F^2 \right]. \end{aligned}

It remains to evaluate the last two terms. Since \bar\Xi is independent of \bar H and satisfies \mathbb E[\bar\Xi]=0 by Lemma 3.1,

\begin{aligned} \mathbb E \!\left[ \left\langle \Delta(\bar H), \bar\Xi \right\rangle_F \right] &= \mathbb E \!\left[ \left\langle \Delta(\bar H), \mathbb E[\bar\Xi] \right\rangle_F \right] \\ &= 0. \end{aligned}

Furthermore, Lemma 3.1 establishes that

\mathbb E \!\left[ \|\bar\Xi\|_F^2 \right] = (N-1)d.

Substituting these two identities into the previous expansion yields

\mathbb E_\mu \!\left[ \|\bar H\|_F^2 \right] = \mathbb E_\mu \!\left[ \|\Delta(\bar H)\|_F^2 \right] + \sigma^2(N-1)d,

which completes the proof.

Uses Lemma 3.1

Lemma

Lemma 5.3 (Contractive variance estimate)

T1RL

Suppose that Assumptions 4.1, 4.2, 4.3, and 4.4 hold.

Then

\boxed{ \mathbb E_\mu \!\left[ \|\Delta(\bar H)\|_F^2 \right] \le \alpha^2 \, \mathbb E_\mu \!\left[ \|\bar H\|_F^2 \right]. }

Used by Proof

Proof of Lemma 5.3

By Assumption 4.3,

\|\Delta(X)-\Delta(Y)\|_F \le \alpha \|X-Y\|_F \qquad (\alpha<1).

Taking Y=0 gives,

\| \Delta(H)-\Delta(0) \|_F \le \alpha \|H-0\|_F.

Since \Delta(0)=0,

\|\Delta(H)\|_F \le \alpha \|H\|_F.

Squaring both sides yields

\|\Delta(H)\|_F^2 \le \alpha^2 \|H\|_F^2.

Finally, taking expectations with respect to the invariant probability measure \mu gives

\mathbb E_\mu \!\left[ \|\Delta(\bar H)\|_F^2 \right] \le \alpha^2 \, \mathbb E_\mu \!\left[ \|\bar H\|_F^2 \right].

Uses Assumption 4.3

Theorem

Theorem 5.4 (Positive lower bound on the stationary variance)

76EG

Suppose that Assumptions 4.1, 4.2, 4.3, and 4.4 hold. Then the unique invariant probability measure satisfies

\boxed{ \mathbb E_\mu \!\left[ \|\bar H\|_F^2 \right] \ge \frac{\sigma^2 (N-1)d} {1-\alpha^2} >0. }

Consequently, the projected hidden representations possess a strictly positive stationary variance.

Used by Proof

Proof of Theorem 5.4

By Proposition 5.2,

\mathbb E_\mu \!\left[ \|\bar H\|_F^2 \right] = \mathbb E_\mu \!\left[ \|\Delta(\bar H)\|_F^2 \right] + \sigma^2(N-1)d.

Applying Lemma 5.3 yields

\mathbb E_\mu \!\left[ \|\bar H\|_F^2 \right] \le \alpha^2 \mathbb E_\mu \!\left[ \|\bar H\|_F^2 \right] + \sigma^2(N-1)d.

Rearranging gives

(1-\alpha^2) \, \mathbb E_\mu \!\left[ \|\bar H\|_F^2 \right] \ge \sigma^2(N-1)d.

Since \alpha<1,

1-\alpha^2>0,

and therefore

\mathbb E_\mu \!\left[ \|\bar H\|_F^2 \right] \ge \frac{\sigma^2(N-1)d} {1-\alpha^2}.

Because \sigma>0, the right-hand side is strictly positive, proving the claim.

Uses Proposition 5.2, Lemma 5.3

Remark

Remark 5.2

HYRF

Theorem 5.4 is purely probabilistic and does not depend on the graph topology. It shows that persistent additive noise prevents the projected representations from collapsing to zero variance, regardless of the particular form of the contractive mapping \Delta.

This result is the fundamental stochastic ingredient of the subsequent analysis. To relate stationary variance to oversmoothing, one must additionally connect the projected variance to the graph Dirichlet energy through the spectral properties of the graph Laplacian.

Uses Definition 3.2, Definition 3.1

Lemma

Lemma 5.5 (Uniform boundedness of second moments)

S46H

Suppose that Assumptions 4.1, 4.2, 4.3, and 4.4 hold. Then the projected process satisfies

\sup_{t\ge0} \mathbb E \!\left[ \|\bar H^{(t)}\|_F^2 \right] < \infty.

More precisely,

\mathbb E \!\left[ \|\bar H^{(t)}\|_F^2 \right] \le \alpha^{2t} \|\bar H^{(0)}\|_F^2 + \frac{\sigma^2 (N-1)d} {1-\alpha^2}, \qquad t\ge0.

Proof of Lemma 5.5

The projected dynamics satisfy

\bar H^{(t+1)} = \Delta(\bar H^{(t)}) + \sigma\bar\Xi^{(t)}.

Expanding the squared Frobenius norm gives

\begin{aligned} \|\bar H^{(t+1)}\|_F^2 &= \|\Delta(\bar H^{(t)})\|_F^2 + 2\sigma \langle \Delta(\bar H^{(t)}), \bar\Xi^{(t)} \rangle_F \\ &\qquad + \sigma^2 \|\bar\Xi^{(t)}\|_F^2. \end{aligned}

Taking expectations and using the independence and zero mean of \bar\Xi^{(t)} yields

\mathbb E \!\left[ \|\bar H^{(t+1)}\|_F^2 \right] = \mathbb E \!\left[ \|\Delta(\bar H^{(t)})\|_F^2 \right] + \sigma^2 (N-1)d,

where Lemma 3.1 has been used.

Since \Delta(0)=0 and \Delta is globally contractive,

\|\Delta(H)\|_F \le \alpha \|H\|_F.

Therefore,

\mathbb E \!\left[ \|\bar H^{(t+1)}\|_F^2 \right] \le \alpha^2 \mathbb E \!\left[ \|\bar H^{(t)}\|_F^2 \right] + \sigma^2 (N-1)d.

Define

M_t = \mathbb E \!\left[ \|\bar H^{(t)}\|_F^2 \right].

Then

M_{t+1} \le \alpha^2 M_t + \sigma^2 (N-1)d.

Iterating this inequality gives

M_t \le \alpha^{2t}M_0 + \sigma^2 (N-1)d \sum_{k=0}^{t-1} \alpha^{2k}.

Since \alpha<1,

\sum_{k=0}^{t-1} \alpha^{2k} \le \frac{1}{1-\alpha^2},

and hence

M_t \le \alpha^{2t} \|\bar H^{(0)}\|_F^2 + \frac{\sigma^2 (N-1)d} {1-\alpha^2}.

Taking the supremum over t completes the proof.

Uses Lemma 3.1

Theorem

Theorem 5.6 (Escape from oversmoothing under contractive noisy recurrent dynamics)

Q5T1

Suppose that Assumptions 4.1, 4.2, 4.3, and 4.4 hold. Let

(H^{(t)})_{t\ge0}

denote the noisy recurrent graph neural network defined in Section ?. Then

\boxed{ \lim_{t\rightarrow\infty} \mathbb E \!\left[ \mathcal E(H^{(t)}) \right] = \mathbb E_\mu \!\left[ \mathcal E(\bar H) \right] \ge \frac{\lambda_2(L)}{m} \, \frac{\sigma^2(N-1)d} {1-\alpha^2} >0. }

Consequently, the noisy recurrent graph neural network escapes oversmoothing in the sense of Definition 3.3.

Uses Definition 3.2 · Used by Corollary 5.7

Proof

ZJFA

By Theorem 5.1, the projected chain (\bar H^{(t)})_{t\ge0} converges to the unique invariant measure \mu in the 2‑Wasserstein metric. The Dirichlet energy satisfies the quadratic estimate

\mathcal E(H) = \frac1m\operatorname{Tr}(H^{\top}LH) \le \frac{\lambda_N(L)}{m}\|H\|_F^2,

and it depends only on the projected representation, \mathcal E(H^{(t)}) = \mathcal E(\bar H^{(t)}). Because \mathcal E is continuous and has at most quadratic growth, convergence in the W_2 metric implies convergence of expectations (see, e.g., [villani2009, Chapter 6]). Consequently,

\lim_{t\to\infty} \mathbb E\bigl[\mathcal E(H^{(t)})\bigr] = \mathbb E_\mu\bigl[\mathcal E(\bar H)\bigr].

By the spectral lower bound (Lemma 3.3),

\mathcal E(\bar H) \ge \frac{\lambda_2(L)}{m}\|\bar H\|_F^2.

Taking expectations with respect to \mu and applying the stationary variance lower bound from Theorem 5.4 gives

\mathbb E_\mu\bigl[\mathcal E(\bar H)\bigr] \ge \frac{\lambda_2(L)}{m} \frac{\sigma^2(N-1)d}{1-\alpha^2} \;>\; 0,

which completes the proof.

Uses Theorem 5.1, Definition 3.1, Lemma 3.3, Theorem 5.4

Corollary

Corollary 5.7

LSQS

Under the assumptions of Theorem 5.6, the limiting hidden representations remain separated from the consensus manifold in expectation. Consequently, stochastic perturbations provide a persistent mechanism preventing asymptotic feature collapse.

Uses Theorem 5.6

Source and licence

The text on this page is from arXiv:2607.28185v1 by Mostafa Haghir Chehreghani, under the licence CC BY 4.0. The corpus made these changes: it read the items out of the LaTeX source, it expanded the macros of the paper, it put each proof after its statement, and it converted the text to HTML. The formulas are the TeX of the paper, and KaTeX renders them. A formula or a reference can look different from the PDF. The page is published under CC BY 4.0.

The record of each item is in analysis/objects/2607/2607.28185.jsonl.