In safety-critical decision-making, the environment may evolve over time, and the learner adjusts its risk level accordingly. This work investigates risk-averse online optimization in dynamic environments with varying risk levels, employing Conditional Value-at-Risk (CVaR) as the risk measure. To capture the dynamics of the environment and risk levels, we employ the function variation metric and introduce a novel risk-level variation metric. Two information settings are considered: a first-order scenario, where the learner observes both function values and their gradients; and a zeroth-order scenario, where only function evaluations are available. For both cases, we develop risk-averse learning algorithms with a limited sampling budget and analyze their dynamic regret bounds in terms of function variation, risk-level variation, and the total number of samples. The regret analysis demonstrates the adaptability of the algorithms in non-stationary and risk-sensitive settings. Finally, numerical experiments are presented to demonstrate the efficacy of the methods.
Let \mathcal{J} be the set of cost functions J_t, for t=1,\dots,T. The function variation V_f is defined as drifts of the function sequences \{J_t\}_{t=1}^T \in \mathcal{J} over the horizon T, i.e., V_f= \sum_{t=2}^T \sup_{x\in\mathcal{X}}\mathbb{E}_\xi\big[|J_t(x,\xi) - J_{t-1}(x,\xi)| \big].
The cost function J_t(x,\xi) is convex in x for every \xi \in \Xi.
The cost function J_t(x,\xi) is Lipschitz continuous in x for every \xi \in \Xi. That is, there exists a positive real constant L_0 such that, for all x, y \in \mathcal{X}, we have |J_t(x,\xi)-J_t(y,\xi)| \le L_0\|x-y\|.
[rockafellar2000optimization] Given Assumption 1, \mathrm{CVaR}_{\alpha}\left[J_t(x,\xi)\right] is convex in x.
Let the risk level be time-varying, denoted as \alpha_t\in (0,1], for t=1,\dots,T. The risk-level variation V_\alpha is defined as the drifts of the risk level sequences \{\alpha_t\}_{t=1}^T over the horizon T , i.e., V_\alpha= \sum_{t=2}^T |\alpha_t - \alpha_{t-1}|.
[hong2009simulating] It holds that
where \nabla C_t(x) denotes the gradient of C_t(x) with respect to x, and \nabla_x H_t(x,\nu) denotes the gradient of H_t(x,\nu) with respect to x.
Let \|\nabla J_t(x,\xi_t)\| \le G, for all t=1,\dots,T, where G is a positive scalar.
Let F_t^x(y) = \mathbb{P}\{J_t(x,\xi) \le y \} and \mathcal{Y}_x = \text{Range}(J_t(x,\xi)). For every x \in \mathcal{X}, the distribution function F_t^x is continuously differentiable and L_g-Lipschitz continuous. Moreover, its probability density function is lower bounded by \underline{p}, i.e., {F_t^x}'(y) \ge \underline{p}>0 for all y \in \mathcal{Y}_x.
Let Assumptions 3 and 4 hold. Assume that n_t samples are available at time t. Then, it holds that
Given a confidence level \gamma, for t=1,\dots,T, the error of the CVaR gradient estimate (?) is bounded as
with probability 1-\gamma.
Let X be a random variable with distribution \mathcal{D}_x, and let \alpha_1, \alpha_2 \in (0,1] be two risk levels. Let |X|\leq U. Then, we have that
Let J_1(X) and J_2(X) be two functions of a random variable X with distribution \mathcal{D}_x. Then, we have that
Let Assumptions 1–4 hold. Suppose that the total number of samples over the horizon T satisfies (?) with a tuning parameter a>0. By selecting \eta = \left(\frac{V_\alpha+V_f}{T}\right)^{\frac{1}{3}}, Algorithm ? achieves {\rm DR}(T) = \tilde{\mathcal{O}}(T^{\frac{2}{3}}(V_\alpha+V_f)^{\frac{1}{3}}+ T^{1-\frac{a}{2}}) when a\in(0,\frac{2}{3}) and {\rm DR}(T) = \tilde{\mathcal{O}}(T^{\frac{2}{3}}(V_\alpha+V_f)^{\frac{1}{3}}) when a\ge \frac{2}{3}, with high probability.
To account for the time-varying function and risk level, we partition the horizon T into s = \left\lceil\frac{T}{\Delta_T}\right\rceil intervals of length \Delta_T \in (1,T) and examine the performance of Algorithm ? on each interval. Each interval is defined as
where j=\left\lceil\frac{t}{\Delta_T}\right\rceil denotes the interval that time t belongs to. Denote \tilde{x}_j^\ast as the optimal action over the interval j, i.e., \tilde{x}_j^\ast: = \min_{x\in \mathcal{X}}\sum_{t \in \mathcal{T}_j}C_t(x). Then, the dynamic regret (?) can be written as
with \mathcal{R}_1 = \sum_{j=1}^s\sum_{t\in \mathcal{T}_j} \big(C_t(x_t)- C_t(\tilde{x}_j^\ast) \big) and \mathcal{R}_2 = \sum_{j=1}^s\sum_{t\in \mathcal{T}_j} \big( C_t(\tilde{x}_j^\ast) - C_t(x_t^\ast) \big). For \mathcal{R}_1, we have
where the inequality follows from the convexity of C_t(x), as shown in Lemma 1, and the equality from substituting (?) into (Proof). By the gradient descent rule (?), for t\in \mathcal{T}_j, we have that
where the inequality holds as \tilde{x}_j^{\ast} \in \mathcal{X}. Then, we obtain
for t \in \mathcal{T}_j. Substituting (Proof) into \mathcal{R}_1, we obtain
with \mathcal{R}_{11} =\sum_{t=1}^T \frac{\eta}{2}\mathbb{E}[\|g_t^1\|^2] and \mathcal{R}_{12} = \sum_{j=1}^s \sum_{t\in \mathcal{T}_j} \mathbb{E}[\| e_t^1\|\| x_t -\tilde{x}_j^{\ast}\| ]. The second inequality of (Proof) follows from the diameter definition D_x = \sup_{x,y \in \mathcal{X}} \| x-y\| and \left\lceil\frac{T}{\Delta_T}\right\rceil \le \frac{2T}{\Delta_T}. By Assumption 3, we have \mathcal{R}_{11} \le \frac{\eta G^2T}{2}. For \mathcal{R}_{12}, we have
with probability 1-\gamma. In (Proof), the first inequality follows from D_x = \sup_{x,y \in \mathcal{X}} \| x-y\|, the second inequality from Lemma 3, and the last inequality from the sampling requirement (?).
We next bound \mathcal{R}_2. For t=1,\dots, T, replacing the function f_t of Lemma 8 (in the Appendix) with C_t, we obtain
where
By Lemma 4, we have
where \alpha_{\min} := \min\{a_1,\dots,\alpha_T \}, and the second inequality holds by Definition 2. Furthermore, by Lemma 5, we have
where the second inequality holds by Definition 1. Substituting (Proof)–(?) into (Proof), we obtain
with probability at least 1- \gamma. We select \eta = \left(\frac{V_\alpha+V_f}{T}\right)^{\frac{1}{3}} and \Delta_T = \left(\frac{T}{V_\alpha+V_f}\right)^{\frac{2}{3}} to minimize the regret bound. Then, {\rm DR}(T) = \tilde{\mathcal{O}}(T^{\frac{2}{3}}(V_\alpha+V_f)^{\frac{1}{3}}, T^{1-\frac{a}{2}}), which reduces to {\rm DR}(T) = \tilde{\mathcal{O}}(T^{\frac{2}{3}}(V_\alpha+V_f)^{\frac{1}{3}}) when a\ge \frac{2}{3}, with probability at least 1- \gamma. $
When the risk level is static, i.e., \alpha_t = \alpha for all t = 1,\dots, T, we have V_\alpha = 0. Additionally, when {\alpha_t} is monotonically nonincreasing or nondecreasing with time, we have V_\alpha = |\alpha_1- \alpha_T|, which is constant and independent of T. In these two cases, Algorithm ? achieves {\rm DR}(T) = \tilde{\mathcal{O}}(T^{\frac{2}{3}}V_f^{\frac{1}{3}}+ T^{1-\frac{a}{2}}) when a\in(0,\frac{2}{3}) and {\rm DR}(T) = \tilde{\mathcal{O}}(T^{\frac{2}{3}}V_f^{\frac{1}{3}}) when a\ge \frac{2}{3}, with probability at least 1- \gamma. This argument also applies to a static function setting. Namely, when the function is static, i.e., J_t(\cdot) = J(\cdot) for all t = 1,\dots, T, we have V_f = 0. Then, Algorithm ? achieves {\rm DR}(T) = \tilde{\mathcal{O}}(T^{\frac{2}{3}}V_\alpha^{\frac{1}{3}}+ T^{1-\frac{a}{2}}) when a\in(0,\frac{2}{3}) and {\rm DR}(T) = \tilde{\mathcal{O}}(T^{\frac{2}{3}}V_\alpha^{\frac{1}{3}}) when a\ge \frac{2}{3}, with probability at least 1- \gamma.
We characterize the dynamic regret in terms of the variation in the cost function and the variation in the risk level, rather than the variation in CVaR. As noted, the CVaR value is generally not directly observable and has to be estimated from samples. Moreover, variations in both the cost function and risk level will lead to variations in CVaR. In Theorem 1, V_f and V_\alpha influence the regret bound independently, and the dominant-order term governs the asymptotic rate. Consequently, defining the performance metric using these two variations yields a tighter regret bound than that based on the variation in CVaR.
A natural question arises: Can the optimal decisions remain unchanged as the CVaR risk level \alpha_t varies? Characterizing such regimes is nontrivial because CVaR depends on the loss quantile, i.e., the inverse cumulative distribution function, which rarely admits a closed-form expression. This dependence makes sensitivity analysis with respect to \alpha_t analytically challenging. Nonetheless, some tractable special cases exist. For instance, when the random component is separable from the decision variable, i.e., when J(x, \xi) = f(x) + g(\xi), the optimal decision is unaffected by changes in \alpha_t. Similarly, in constrained dynamic portfolio problems [rockafellar2000optimization] with linear transaction costs and Gaussian returns, varying \alpha_t merely rescales the risk term, rendering the optimizer invariant across different risk levels.
Theorem 1 reveals that the regret bound decreases as the total number of samplings (controlled by the parameter a) increases when a \in (0,\frac{2}{3}], and ceases to improve when a > \frac{2}{3}. This occurs because the VaR estimate error becomes negligible compared to other error terms in (?) for a > \frac{2}{3}. Furthermore, the sampling strategy in [rockafellar2000optimization] corresponds to the special case a=1 in (?), which yields a convergence rate of \tilde{\mathcal{O}}(T^{-1/2}) for a static problem. Under identical static conditions, Algorithm ? achieves a regret bound of \mathcal{R}(T) = \tilde{\mathcal{O}}(T^{1/2} + T^{1 - a/2}), which simplifies to \tilde{\mathcal{O}}(T^{1/2}) when a \ge 2/3. Consequently, our algorithm matches the \tilde{\mathcal{O}}(T^{1/2}) bound of [rockafellar2000optimization] with fewer samples.
[cardoso2019risk] Given Assumptions 1 and 2, we have that
C_t^\delta\left(x\right) is convex in x;
C_t^\delta (x) is L_0-Lipschitz in x and |C_t^\delta(x) - C_t(x)| \le \delta L_0.
[wang2022zeroth] Let F and G be the cumulative distribution functions of two random variables, respectively. Let the random variables be bounded by U. Then, we have that
Let Assumptions 1 and 2 hold. Suppose that the total sampling number over the horizon T satisfy (?) with a constant a>0.
When a \in (0,\frac{4}{5}], by selecting \delta = T^{-\frac{a}{4}}(V_\alpha+V_f)^{\frac{1}{5}} and \eta =T^{-\frac{3a}{4}}(V_\alpha+V_f)^{\frac{3}{5}}, Algorithm ? achieves \rm{DR}_0(T) = \tilde{\mathcal{O}}(T^{1-\frac{a}{4}}(V_\alpha+V_f)^{\frac{1}{5}}) with high probability.
When a >\frac{4}{5}, by selecting \delta = T^{-\frac{1}{5}}(V_\alpha+V_f)^{\frac{1}{5}} and \eta =T^{-\frac{3}{5}}(V_\alpha+V_f)^{\frac{3}{5}}, Algorithm ? achieves \rm{DR}_0(T) = \tilde{\mathcal{O}}(T^{\frac{4}{5}}(V_\alpha+V_f)^{\frac{1}{5}}) with high probability.
The dynamic regret (?) can be further written as
where the first inequality follows from the Lipschitzness of C_t and the definition of C_t^\delta, as in (?). The second inequality follows from the Lipschitzness of C_t^\delta, as shown in Lemma 6. Furthermore, for t=1,\dots,T, we have
where the first inequality follows from the convexity of C_t^\delta and the second inequality from Lipschitzness of C_t^\delta, as shown in Lemma 6. Denote x_t^{\delta,\ast} = {\textrm{argmin}}_{x \in \mathcal{X}^\delta} C_t^\delta (x) as the single-step optimal decision, for t=1,\dots,T. Substituting (Proof) into (Proof) and following the derivation of (Proof), we have that
with
and \bar{\tilde{x}}_j^{\delta,\ast}=\min_{x\in\mathcal{X}^\delta}\sum_{t\in \mathcal{T}_j}C_t^\delta(x) is the optimal action over the interval j. From (?) and (?), we have that
Furthermore, for \tilde{R}_1, we have that
with \tilde{\mathcal{R}}_{11} = \sum_{t=1}^T \frac{\eta}{2}\mathbb{E}[\|g_t^0\|^2], \tilde{\mathcal{R}}_{12} = \sum_{j=1}^s \sum_{t\in \mathcal{T}_j} \frac{d}{\delta} \mathbb{E}[\| e_t^0\|\| x_t -\bar{\tilde{x}}_j^{\delta,\ast}\| ], where the first inequality follows from the convexity of C_t^\delta, the equality from substituting (Proof) into (Proof), and the second inequality follows the derivation of (Proof)-(Proof). By the definition of the CVaR gradient estimate (?), we have
We next bound \tilde{\mathcal{R}}_{12}. By the Dvoretzky–Kiefer–Wolfowitz (DKW) inequality [dvoretzky1956asymptotic], we have that
Denote the event in (Proof) as B_t, and \mathbb{P}\{B_t\} the occurrence probability of B_t, for t=1,\dots, T. By Lemma 7 and the definition of e_t^0, as in (?), we have
with probability at least 1 -\bar{\gamma}, for t = 1,\dots, T. Let \gamma = \bar{\gamma}T. Then, subsituting (Proof) into \tilde{\mathcal{R}}_{12}, we obtain
with probability 1-\gamma, which establishes as 1 - \mathbb{P}\{\bigcup_{t=1}^{T} B_t \} \ge 1 - \sum_{t=1}^{T} \mathbb{P}\{B_t\} \ge 1 - T\frac{\gamma}{T} \ge 1-\gamma. The second inequality of (Proof) follows from the sampling requirement (?). Furthermore, following the derivation of \mathcal{R}_2, as in (?)–(?), we have
Substituting (Proof), (Proof), (Proof), and (Proof) into (Proof), we obtain
with probability 1-\gamma. To minimize the regret \text{DR}_0(T), we select \delta = T^{-\frac{a}{4}}(V_\alpha+V_f)^{\frac{1}{5}}, \eta =T^{-\frac{3a}{4}}(V_\alpha+V_f)^{\frac{3}{5}}, and \Delta_T = T^{a}(V_\alpha+V_f)^{-\frac{4}{5}} when a \in (0,\frac{4}{5}] and \delta = T^{-\frac{1}{5}}(V_\alpha+V_f)^{\frac{1}{5}}, \eta =T^{-\frac{3}{5}}(V_\alpha+V_f)^{\frac{3}{5}}, and \Delta_T = T^{\frac{4}{5}}(V_\alpha+V_f)^{-\frac{4}{5}} when a >\frac{4}{5}. Hence, \text{DR}_0(T) = \tilde{\mathcal{O}}(T^{1-\frac{a}{4}}(V_\alpha+V_f)^{\frac{1}{5}}) when a\in(0,\frac{4}{5}] and \text{DR}_0(T) = \tilde{\mathcal{O}}(T^{\frac{4}{5}}(V_\alpha+V_f)^{\frac{1}{5}}) when a > \frac{4}{5}, with probability at least 1- \gamma. $
[besbes2015non] Consider the function f_t(x): \mathcal{X} \rightarrow \mathbb{R}, define the function sequences over iteration horizon T as \{f_t(x_t)\}_{t=1}^{T}, we have that
where \tilde{x}_j^\ast = \min_{x\in\mathcal{X}}\sum_{t\in\mathcal{T}_j}f_t(x) and x_t^\ast = \min_{x\in\mathcal{X}}f_t(x).
The proof of the CVaR gradient estimation error bound builds upon the approach presented in [wang2024learning]. The concentration inequality (3) follows from [shao1999mathematical]. Let \bar{\gamma} = 2e^{-2n_t\varepsilon^2\underline{p}^2} in (3), we have
Denote the event in (Proof) as A_t, and \mathbb{P}\{A_t\} the occurrence probability of A_t, for t=1,\dots, T.
Denote \nu_m = \min\{\nu_{t}, \nu^{\ast}_{t}\}, and \nu_M = \max\{\nu_{t}, \nu^{\ast}_{t}\}. We obtain
Since \| \mathbb{E}[XY] \| \leq \mathbb{E}[\|X\|\|Y\|] holds for random variables X,Y, we have
for t=1,\dots,T with probability 1-\gamma, which establishes 1 - \mathbb{P}\{\bigcup_{t=1}^{T} A_t \} \ge 1 - \sum_{t=1}^{T} \mathbb{P}\{A_t\} \ge 1 - T\frac{\gamma}{T} \ge 1-\gamma. In (?), the first inequality follows from \|\nabla_x J_t(x_t, \xi) \|\le G, as in Assumption 3. The second inequality follows from F_t(\cdot) is L_g-Lipschitz, as in Assumption 4, and the last inequality follows from (Proof). $
Define the augmented functions
As shown in [rockafellar2000optimization], we have
Let v_{\alpha_1} = \text{argmin}_{v}L_{\alpha_1}(v) and v_{\alpha_2} = \text{argmin}_{v}L_{\alpha_2}(v). Then, we have
Furthermore, we have
where the first inequality follows from v_{\alpha_1} = \text{argmin}_{v}L_{{\alpha_1}}(v) and L_{\alpha_1}(v_{\alpha_1}) \le L_{\alpha_1}(v_{\alpha_2}), and the second inequality from |J(\cdot)| \le U. $
Define the augmented functions
As shown in [rockafellar2000optimization], we have
Let v_1 = \text{argmin}_{v}L_1(v) and v_2 = \text{argmin}_{v}L_2(v). Then, we have
Furthermore, we have
where the first inequality follows from v_1 = \text{argmin}_{v}L_1(v) and L_1(v_1)\le L_1(v_2), the second inequality from a_+ - b_+ \leq [a-b]_+, and the last inequality from the 1-Lipschitz property of [\cdot]_{+}. A similar result appears in [pichler2017quantitative]. However, we extend their analysis by relaxing the convexity assumption. $
The text on this page is from arXiv:2512.22986v1 by Siyi Wang, Zifan Wang and Karl H. Johansson, under the licence CC0 1.0. The corpus made these changes: it read the items out of the LaTeX source, it expanded the macros of the paper, it put each proof after its statement, and it converted the text to HTML. The formulas are the TeX of the paper, and KaTeX renders them. A formula or a reference can look different from the PDF. The page is published under CC BY 4.0.
The record of each item is in analysis/objects/2512/2512.22986.jsonl.