Hierarchical Federated ADMM
Seyed Mohammad Azimi-Abarghouyi, Nicola Bastianello, Karl H. Johansson and Viktoria Fodor
- arXiv
- 2409.18796v1 · PDF · 27 September 2024
- Categories
- cs.LG, cs.AI, cs.DC, cs.IT, cs.SY, eess.SY, math.IT
- Licence
- CC0 1.0 The site shows the text.
Abstract
In this paper, we depart from the widely-used gradient descent-based hierarchical federated learning (FL) algorithms to develop a novel hierarchical FL framework based on the alternating direction method of multipliers (ADMM). Within this framework, we propose two novel FL algorithms, which both use ADMM in the top layer: one that employs ADMM in the lower layer and another that uses the conventional gradient descent-based approach. The proposed framework enhances privacy, and experiments demonstrate the superiority of the proposed algorithms compared to the conventional algorithms in terms of learning convergence and accuracy. Additionally, gradient descent on the lower layer performs well even if the number of local steps is very limited, while ADMM on both layers lead to better performance otherwise.
Items 2
Assume f_{kc} in (?) to be closed, proper and convex for all k, c. Consider a version of lmttHierFADMM or lmttHierF2ADMM in which the number of intra-set iterations (\tau^t) changes over time according to \lim_{t \to \infty} \tau^t = \infty. Then \mathbf{w}^t and \mathbf{w}_c^t, \forall c, converge to a solution of (?).
Proof of Proposition 1
As discussed above, we can interpret lmttHierFADMM and lmttHierF2ADMM as the inexact ADMM in (?). Additionally, if the number of intra-set iterations grows according to \lim_{t \to \infty} \tau^t = \infty, then \lim_{t \to \infty} \mathbf{e}^t_c = \mathbf{0}, as the intra-set updates become increasingly accurate. Therefore, we can apply [eckstein, Proposition 1] to guarantee convergence. \square
Source and licence
The text on this page is from arXiv:2409.18796v1 by Seyed Mohammad Azimi-Abarghouyi, Nicola Bastianello, Karl H. Johansson and Viktoria Fodor, under the licence CC0 1.0. The corpus made these changes: it read the items out of the LaTeX source, it expanded the macros of the paper, it put each proof after its statement, and it converted the text to HTML. The formulas are the TeX of the paper, and KaTeX renders them. A formula or a reference can look different from the PDF. The page is published under CC BY 4.0.
The record of each item is in analysis/objects/2409/2409.18796.jsonl.