Distributed Optimization Method Based On Optimal Control

The directed graph \mathcal{G} is balanced and is strongly connected.

Compared with the traditional method, we convert the optimization problem into an optimal control problem, where the objective of each agent is to design the current control input minimizing the sum of the original objective function and updated size for the future time instant. On the one hand, the problem studied in this paper includes the traditional results with multi-agents system, if we let f_i(x_i(k))=f_i(x_i(N+1))=0 while the system (?) can be modified as x_i(k+1)=Ax_i(k)+B_iu_i(k). On the other hand, in the existing optimization algorithms to handle the problem (?), there is a typical distributed first-order gradient descent algorithm [ruder2016overview]:

where \eta(k) is the step size.

Noting that in convex optimization, it has been proven that using distributed algorithm (1), the global optimal solution for problem (?) can be obtained. However, distributed algorithm (1) has limitations in terms of selecting the step size. The step size \eta(k) in distributed algorithm (1) should theoretically satisfy the following two conditions:

In practical applications, the common practice for selecting the \eta is to start with a small constant value. Alternatively, even if a time-varying step size \eta(k) is chosen to satisfy the two conditions (1)-(1), the results obtained from distributed algorithm (1) are evidently influenced by the \eta(k). The distributed algorithm (1) has limitations in terms of selecting the step size and the results obtained from distributed algorithm (1) are evidently influenced by the \eta(k).

In our recent work [Zhang2024], we implemented a centralized optimization algorithm, which causes us to design the system and cost function (?)-(?). Naturally, we considered the possibility of designing a distributed algorithm based on the centralized optimization algorithm from the maximum principle. It should be noted that if F({x(k)})=\sum\limits_{i = 1}^nf_i({x_{i}(k)})=0, (?) can be reduced to a cost function of consensus problem based on optimal control in our recent work [zhang2023distributed].

It should be noted that the analytical solution u(k) obtained in Lemma 1 contains the future information, i.e., it depends on \{\nabla f(i), i \in [k + 1,N + 1]\}, which is difficult to realize in practical life, thus a modified algorithm is presented.

The connection between algorithm (3) and our previous work presented in [Zhang2024] will be discussed. On the one hand, if each agent does not have a local function to optimize, i.e., f_i(x_i(k))=f_i(x_i(N+1))=0, it is evident that the terms g(k) and h(k) in (3) will not exist. That is to say, (3) transforms into the following form:

In this case, d_k corresponds to the form of the controller u_k from Lemma 1 in [zhang2023distributed] when the matrix A is the identity matrix. On the other hand, if we only consider a centralized optimization problem, or if the states during the iteration process of each agent are same, then e(k) and the related P will not exist. Moreover, (3) degenerates into the following form:

where l=1,\dots,k means the required number of cycles. In this case, (4) corresponds to Algorithm I in the previous work [Zhang2024], where g(k) and h(k) correspond to the (average) gradient and Hessian matrix, respectively.

Now the connection between algorithm (?) and our previous work in [wang2024superlinear] is discussed. If we only consider a centralized optimization problem, or if the states during the iteration process of each agent are the same, then e(k) and the related P will not exist. In this case, (?) degrades into the following form:

which is exactly Algorithm II in the previous work [wang2024superlinear].

The local objective functions f_i, i=1,...,n are twice continuously differentiable, and there exist constants 0<m_1\le m_2<\infty such that for any x \in {\mathbb{R}^n}, m_1I\preceq \nabla^2f_i(x)\preceq m_2I

Assumption 2 is standard in the convergence analysis , see [boyd2004convex]. The lower bound m_1 of \nabla^2f_i(x) implies that the local objective function f_i is strongly convex, and thus the objective function f_i has a unique global minimum point. The upper bound m_2 of \nabla^2f_i(x) means that the local objective function f_i is smooth.