On the Subadditivity of Mutual f-Information

The proof can be found in Appendix ?.

See Appendix ?.

The proof can be found in Appendix ?.

The proof can be found in Appendix ?.

Since r_X is a probability distribution, it suffices to show that for every function v_X satisfying \sum_{x}v_X(x)=0, the following quantity is non-positive:

Here, \mathcal{H}\!\left(\Theta_{p_X,p_{Y|X}}\left(r_X\right)\right) denotes the Hessian matrix of \Theta_{p_X,p_{Y|X}}\left(r_X\right), and \vec{v}_{X} is the column vector corresponding to the function v_X, while T denotes the transpose operator. Let g(x)\triangleq x^2f''(x) and set

Then, routine calculations show that

For every real-valued function u_X and all distributions p_X and q_X, define the function

Then, one can observe that

Thus, it suffices to show that \mathcal{L}_{u_X,p_X}(q_X) is concave in q_X for any u_X and p_X.

As before, consider an arbitrary real-valued function k_X such that \sum_{x\in \mathcal{X}}k_X(x)=0. To show that \mathcal{L}_{u_X,p_X}(q_X) is concave over probability distributions, it suffices to show that

for all distributions p_X and q_X and all functions u_X and k_X satisfying \sum_{x\in\mathcal{X}}k_X(x)=0. Routine calculations show that

where

Now, we wish to design a function g(x) such that the expression in (?) is non-positive. Let \eta_g be a constant. Using g'=(g'-\eta_g)+\eta_g together with (?), we obtain

Assuming g''(x)\neq 0, rearranging (?) yields

where

It is clear that if g''(x)<0 for all x>0, then \mathtt{A}_1\leq 0. Moreover, if we assume that for all x>0 the function g(x) satisfies

for some constants \theta_1 and \theta_2, then \mathtt{A}_2\leq \theta_1. Thus, we have

By choosing \theta_1= 2\eta_g, we obtain \frac{d^2}{d\epsilon^2}\mathcal{L}(q_X+\epsilon k_X)\big|_{\epsilon=0}\leq 0. Recall that f is a convex function. Since x^2f''(x)=g(x) and we assume that g''(x)<0, the left-hand side of (?) is a positive function, and thus \mathtt{A}_{2} is positive. Since \mathtt{A}_{2}\leq \theta_1, we conclude that \eta_g must be positive. This completes the proof of Lemma 4.

Let \lambda=\frac{a}{m}\in[0,1] and \eta=\frac{b}{m}\in[0,1]. We write a=\lambda m+0\cdot(1-\lambda) and b=\eta m+0\cdot(1-\eta). Concavity of G^{-1}\!\bigl(G(a)+G(b)\bigr) with respect to each of its arguments yields

Using G(0)=0, G^{-1}(G(m))=m, \lambda=\frac{a}{m}, and \eta=\frac{b}{m} in the above yields

According to Theorem 3 in [abin2026class:isit], if the function x^2f'' belongs to the set \mathcal{T}^{+} defined as

then the pair (\log(1+x),f) is divergence-subadditive. Let \gamma be a positive constant. Since

if x^2(\gamma f)''=\gamma\, x^2 f''\in \mathcal{T}^{+}, then the pair (\log(1+x),\gamma f), and equivalently the pair (\log(1+\gamma x),f), is divergence-subadditive. Let x^2f''=g(x). The condition x^2(\gamma f)''\in \mathcal{T}^{+} implies

Thus, if x^2f''\in \mathcal{T}_{\gamma}, then the pair (\log(1+\gamma x),f) is divergence-subadditive. Divergence-subadditivity of the pair (\log(1+\gamma x),f) implies

which is equivalent to

According to Theorem 4 in [abin2026class:isit], if the function x^2f'' belongs to the set \mathcal{T}^{-} defined as

then the pair (-\log(1-x),f) is divergence-subadditive. Let \gamma be a positive constant. Since

if x^2(\gamma f)''=\gamma\, x^2 f''\in \mathcal{T}^{-}, then the pair (-\log(1-x),\gamma f), and equivalently the pair (-\log(1-\gamma x),f), is divergence-subadditive. Let x^2f''=g(x). The condition x^2(\gamma f)''\in \mathcal{T}^{-} implies

Thus, if x^2f''\in \mathcal{T}_{\gamma}^{-}, then the pair (-\log(1-\gamma x),f) is divergence-subadditive. Divergence-subadditivity of the pair (-\log(1-\gamma x),f) implies

which is equivalent to