Differential privacy (DP) has recently emerged as a definition of privacy to release private estimates. DP calibrates noise to be on the order of an individuals contribution. Due to the this calibration a private estimate obscures any individual while preserving the utility of the estimate. Since the original definition, many alternate definitions have been proposed. These alternates have been proposed for various reasons including improvements on composition results, relaxations, and formalizations. Nevertheless, thus far nearly all definitions of privacy have used a divergence of densities as the basis of the definition. In this paper we take an information geometry perspective towards differential privacy. Specifically, rather than define privacy via a divergence, we define privacy via the Rao distance. We show that our proposed definition of privacy shares the interpretation of previous definitions of privacy while improving on sequential composition.
A random mechanism f_D is said to satisfy \epsilon-differentially private, pure differential privacy, if for all D\sim D',
where \epsilon is a pre-specified parameter referred as the privacy budget, S is any measurable set, and \mu is the base measure.
A mechanism is said to be (\epsilon,\delta)-DP if for all adjacent datasets D\sim D',
where \epsilon>0 and \delta\in(0,1) are pre-specified parameters.
For two probability densities f_1 and f_2, the Rényi divergence of order \alpha>1 from f_1 is,
For two probability densities f_1 and f_2, the Kullback-Leibler divergence from f_1 is defined as
A random mechanism f_D said to satisfy Kullback Leibler DP if
for all D\sim D' and a pre-specified \alpha.
A randomized mechanism f_D is said to satisfy (\xi, \rho)-zero concentrated differentially privacy if for all D\sim D' and all \alpha\in(1,\infty),
where D_{\alpha}(\cdot\|\cdot) is the \alpha-Rényi divergence. For (0,\rho) a mechanism satisfies \rho-zCDP.
A randomized mechanism f_D is said to satisfy \zeta-Rényi differential privacy of order \alpha
for all D\sim D' and pre-specified \zeta.
[rao1945information, costa2015fisher, miyamoto2024closed] Consider the univariate Gaussian density f(x)=\frac{1}{\sqrt{2\pi\sigma^2}}\exp(-(x-\mu)^2/(2\sigma^2)). Here \theta=(\mu,\sigma) is the set of parameters. The entries of the information matrix are g_{11}=\mathbb{E}[\frac{\partial}{\partial\mu}\log f\frac{\partial}{\partial\mu}\log f]=\mathbb{E}[(x-\mu)^2/\sigma^4]=1/\sigma^4\mathbb{E}[(\mu-x)^2]=1/\sigma^2, g_{12}=g_{21}=\mathbb{E}[\frac{\partial}{\partial\mu}\log f\frac{\partial}{\partial\sigma}\log f]=0, and g_{22}=\mathbb{E}[\frac{\partial}{\partial\sigma}\log f\frac{\partial}{\partial\sigma}\log f]=\mathbb{E}[-1/\sigma^2+(x-\mu)^4/\sigma^6]=2/\sigma^2. We have the information matrix, and hence the Riemannian metric is:
The coordinate pair in the parameter space is (\mu,\sigma), and the entries of the metric tensor give us information on how these parameters effect the distance. For instance, we note that g_{11} says that, for two Gaussian densities with the same \sigma, they are increasingly similar for larger \sigma. This matches intuition as the distributions become infinitely flat as \sigma\rightarrow\infty, so they locally become more and more similar. Computing the distance between two normal densities is not immediately straightforward as we need to integrate
We can identify this line element with that of the upper Poincare half-plane so we have that
We note this can be algebraically rewritten and include equivalent reformulations in the appendix. The two special cases, as noted by Rao are d_R((\mu_1,\sigma),(\mu_2,\sigma))=\frac{|\mu_1-\mu_2|}{\sigma} and d_R((\mu,\sigma_1),(\mu,\sigma_2))=\sqrt{2}|\log(\sigma_2/\sigma_1)|.
A random mechanism f_D satisfies \theta-Rao differential privacy if for all adjacent datasets D\sim D' we have that
where d_R(\cdot,\cdot) is the Rao distance for densities and \theta is the privacy budget, a pre-specified parameter.
Given the above, two mechanisms f_{1,D} and f_{2,D} with privacy budgets \theta_1 and \theta_2, respectively, the total privacy budget is \sqrt{\theta_1^2+\theta_2^2}.
Let f_D be a random mechanism that satisfies Rao DP for some \theta and \varphi be an arbitrary random function. We have that \varphi\circ f_D also satisfies Rao DP under the same \theta.
The \eta-sensitivity of a function h:D\rightarrow\mathcal{S} and d_\eta being a distance function is defined \Delta_\eta(h)=\sup_{D\sim D'} d_\eta(h(D),h(D')).
Let f_1(x;\mu_1,\sigma) and f_2(x;\mu_2,\sigma) be two Laplace densities with the same scale parameter. The Rao distance between f_1 and f_2 is,
The proof is nearly identical to Example 1 with the information matrix being g_{11}=\frac{1}{\sigma^2}.
The Laplace mechanism to release a private version of summary h(D) with the form f(x;D)=(2\sigma)^{-1}\exp\{|x-h(D)|/\sigma\}.
The Laplace mechanism with \sigma\geq\Delta/\theta satisfies \theta-Rao Privacy.
This follows semi-directly from Theorem 4.2. We have that d_R(f_D,f_{D'})=\frac{|h(D)-h(D')|}{\sigma}. We thus need \frac{|h(D)-h(D')|}{\sigma}\leq\theta for all D\sim D'.
This is the required result.
The Laplace mechanism that satisfies \epsilon pure DP satisfies Rao DP with \theta=\epsilon.
Let f_1(x;\mu_1,\sigma) and f_2(x;\mu_2,\sigma) be two Gaussian densities with the same scale parameter. The Rao distance between f_1 and f_2 is,
The Gaussian mechanism to release a private version of summary h(D) takes the form f(x;D)=(2\pi\sigma^2)^{-1/2}\exp\{(x-h(D))^2/2\sigma^2\}.
The Gaussian mechanism with \sigma\geq\Delta/\theta satisfies \theta-Rao DP.
The Gaussian mechanism that satisfies (\epsilon,\delta)-DP satisfies Rao DP with \theta=\epsilon/\sqrt{2\log(1.25/\delta)}. The Gaussian mechanism that satisfies \mu-GDP satisfies Rao DP with \theta=\mu.
Let f^N_1(x;\mu_1,\sigma) and f^N_2(x;\mu_2,\sigma) be two Generalized Gaussian densities with the same scale parameter. That is f_i(x;\mu,\sigma)=(2\sigma\Gamma(1/N)/N)^{-1}\exp\{|x-\mu_i|^N/\sigma\}. The Rao distance between f_1 and f_2 is
The generalized Gaussian mechanism of degree N to release a private version of summary h(D) takes the form f^N(x;D)=(2\sigma\Gamma(1/N)/N)^{-1}\exp\{|x-h(D)|^N/\sigma\}.
The Generalized Gauss mechanism with \sigma\geq\Delta \left(\frac{N\Gamma(2-1/N)}{\Gamma(1+1/N)}\right)^{1/2}/\theta satisfies \theta-Rao Privacy.
Let \varphi be an arbitrary deterministic function and f_D be our mechanism. By definition we have d_R(f_D,f_{D'})\leq\theta. We have that,
Thus we have that
Note that (\sqrt{x}+\sqrt{y})^2-2\sqrt{xy}=x+y. Since this holds for all D\sim D' we have by symmetry that \|\varphi(f(x;D))-\varphi(f(x;D')) \| \leq \|f(x;D)-f(x;D') \|.
We thus have the required result.
A mechanism f_D is said to satisfy \mu-Gaussian differential privacy if for all adjacent datasets D\sim D'
where G_{\mu}=:T(N(0,1),N(\mu,1)), T(\cdot,\cdot) is a trade-off function, and \mu is the privacy parameter.
The text on this page is from arXiv:2508.17135v1 by Carlos Soto, under the licence CC BY 4.0. The corpus made these changes: it read the items out of the LaTeX source, it expanded the macros of the paper, it put each proof after its statement, and it converted the text to HTML. The formulas are the TeX of the paper, and KaTeX renders them. A formula or a reference can look different from the PDF. The page is published under CC BY 4.0.
The record of each item is in analysis/objects/2508/2508.17135.jsonl.