arXiv reader

Learning Disturbances Online for Risk-Aware Control: Risk-Aware Flight with Less Than One Minute of Data

Prithvi Akella, Skylar X. Wei, Joel W. Burdick and Aaron D. Ames

arXiv
2212.06253v1 · PDF · 12 December 2022
Categories
eess.SY, cs.IT, cs.LG, cs.RO, cs.SY, math.IT
Licence
CC BY 4.0 The site shows the text.

Abstract

Recent advances in safety-critical risk-aware control are predicated on apriori knowledge of the disturbances a system might face. This paper proposes a method to efficiently learn these disturbances online, in a risk-aware context. First, we introduce the concept of a Surface-at-Risk, a risk measure for stochastic processes that extends Value-at-Risk – a commonly utilized risk measure in the risk-aware controls community. Second, we model the norm of the state discrepancy between the model and the true system evolution as a scalar-valued stochastic process and determine an upper bound to its Surface-at-Risk via Gaussian Process Regression. Third, we provide theoretical results on the accuracy of our fitted surface subject to mild assumptions that are verifiable with respect to the data sets collected during system operation. Finally, we experimentally verify our procedure by augmenting a drone’s controller and highlight performance increases achieved via our risk-aware approach after collecting less than a minute of operating data.

Items 10

Theorem

Theorem 1

PDOB

Let f:X \to \mathbb{R}, \operatorname{\mathbb{X}} = \{x_i\}_{i=1}^N be a set of N points x \in X, \operatorname{\mathbb{Y}} = \{y_i = f(x_i) + \xi\}_{x_i \in \operatorname{\mathbb{X}}} be a set of noisy observations y_i of f(x_i) with R sub-gaussian noise \xi, and k: X \times X \to \mathbb{R} be a positive-definite kernel function. If f has B-bounded RKHS norm for some B > 0, i.e. \|f\|_{RKHS} \leq B, then, with \mu_N and \sigma_N as per (?) and with minimum probability 1-\delta,

|\mu_N(x) - f(x)| \leq \left(B + R \sqrt{2 \ln{\frac{\sqrt{\det\left((1+\frac{2}{N})I_N + \mathbb{K}_N\right)}}{\delta}}}\right) \sigma_N(x),~\forall~x \in X.
Used by Proof
Definition

Definition 2

VSQE

The Value-at-Risk level \epsilon \in [0,1] of a scalar random variable X defined over the probability space (\Omega,\mathcal{F},\mathbb{P}) with distribution \pi is defined as the (1-\epsilon)-th quartile of X, i.e.

\operatorname{VaR}_{\epsilon}(X) \triangleq c\, \operatorname*{~\mathrm{s.t.}~}\, c = \inf\{z \in \mathbb{R}~|~ \operatorname{\mathbb{P}}_{\pi}[X\leq z] \geq 1-\epsilon\}.
Definition

Definition 3

DO2X

The Surface-at-Risk level \epsilon \in [0,1] of a scalar stochastic process S indexed by the set \mathcal{X} is the indexed collection of the Values-at-Risk level \epsilon of each random variables S_x comprising S:

\operatorname{SaR}_{\epsilon}(S,x) = \operatorname{VaR}_{\epsilon}(S_x).
Definition

Definition 4

M9BB

The disturbance-norm stochastic process S = \{S_{\hat x}\}_{\hat x \in \hat{\mathcal{X}}} where samples of each random variable S_{\hat{x}} correspond to norms \operatorname{\delta} of disturbances d as defined in equation (?). The variability in norm samples \delta arises through the assumed process noise \xi in the true system dynamics in (?).

Problem

Problem 1

FZEQ

Identify an upper bound to the Surface-at-Risk at some risk-level \epsilon \in [0,1] for the disturbance-norm stochastic process S as per Definition 4 with Surfaces-at-Risk as defined in Definition 3. Specifically, identify an estimate \operatorname{\mathbb{SR}}_{\epsilon} such that,

\operatorname{\mathbb{SR}}_{\epsilon}(S,\hat{x}) \geq \operatorname{SaR}_{\epsilon}(S,\hat{x}),~\forall~\hat x \in \hat{\mathcal{X}}.
Assumption

Assumption 1

SCND

For the disturbance-norm stochastic process S in Definition 4, the Surface-at-Risk at a given risk-level \epsilon \in [0,1] has bounded discrepancy. I.e. \exists~\alpha,\beta \in \mathbb{R}_{\geq 0} such that,

\forall~\hat{x}_i, \hat{x}_j \in \hat{\mathcal{X}},~\|\hat{x}_i - \hat{x}_j\| \leq \alpha \implies |\operatorname{SaR}_{\epsilon}(S,\hat{x}_i) - \operatorname{SaR}_{\epsilon}(S,\hat{x}_j)| \leq \beta.
Proposition

Proposition 1

U0UQ

Let \{X_i\}_{i=1}^N be a collection of N scalar random variables with (perhaps) different distributions \{\pi_i\}_{i=1}^N, and let \{x_i\}_{i=1}^N be a set of N samples of these random variables, one sample per each random variable, i.e. x_i is a sample of X_i. Then, for any \epsilon \in [0,1], the probability that at least one sample x_{\ell} \in \{x_i\}_{i=1}^N is greater than the Value-at-Risk level \epsilon of its corresponding random variable X_{\ell} is equivalent to 1-(1-\epsilon)^N, i.e. with \operatorname{VaR} as per Definition 2 and \forall~\epsilon \in [0,1],

\operatorname{\mathbb{P}}_{\pi_1,\pi_2,\dots,\pi_N}\left[\exists~x_{\ell} \in \{x_i\}_{i=1}^N \operatorname*{~\mathrm{s.t.}~} x_{\ell} \geq \operatorname{VaR}_{\epsilon}(X_{\ell}) \right] \geq 1-(1-\epsilon)^N.
Proof of Proposition 1

Consider a random variable X_{\ell} \in \{X_i\}_{i=1}^N. The probability of taking a sample x_{\ell} of X_{\ell} such that x_{\ell} \geq \operatorname{VaR}_{\epsilon}(X_{\ell}) is less than or equal to \epsilon by Definition 2. The same line of reasoning holds \forall~X_{\ell} \in \{X_i\}_{i=1}^N. As such, the probability that no sample x_{\ell} \in \{x_i\}_{i=1}^N is greater than the corresponding Value-at-Risk level \epsilon is less than or equal to (1-\epsilon)^N, yielding our result.

Theorem

Theorem 2

H7W6

Let \alpha,\beta,\operatorname{N_{RV}},\operatorname{s},\mu_{\operatorname{s}},\sigma_{\operatorname{s}}, and k be as defined in Algorithm ?, let B > 0, let \operatorname{SaR} be the Surface-at-Risk measure as per Definition 3 for some risk-level \epsilon \in [0,1], let S be the disturbance-norm stochastic process as per Definition 4, and let Assumption 1 hold for each data set \mathcal{D}_{\operatorname{s}} in lines 5-7 of Algorithm ? with respect to the given parameters \alpha,\beta. If \|\operatorname{\mathbb{SR}}_{\epsilon}(S)\|_{RKHS} \leq B, then with minimum probability \left(1-(1-\epsilon)^{\operatorname{N_{RV}}}\right)^{\operatorname{s}} the following holds \forall~\hat{x} \in \hat{\mathcal{X}} and \forall~\operatorname{s} = 1,2,\dots:

|\mu_{\operatorname{s}}(\hat{x}) - \operatorname{\mathbb{SR}}_{\epsilon}(S,\hat{x})| \leq B \sigma_{\operatorname{s}}(\hat{x}), \qquad \mu_{\operatorname{s}}(\hat{x}) + B \sigma_{\operatorname{s}}(\hat{x}) \geq \operatorname{SaR}_{\epsilon}(S,\hat{x}).
Proof of Theorem 2

First, by the assumptions above, we know that for each data set \mathcal{D}_{\operatorname{s}} in lines 5-7 of Algorithm ?, we have taken one sample \operatorname{\delta}_j of \operatorname{N_{RV}} (potentially) different random variables S_{\hat{x}_j}. By Proposition 1, we know that with minimum probability 1-(1-\epsilon)^{\operatorname{N_{RV}}}, the maximum sample \operatorname{\delta}^*_j \triangleq \max\{\operatorname{\delta}_{\ell} \in \mathcal{D}\} is greater than the Value-at-Risk of its corresponding random variable \operatorname{VaR}_{\epsilon}(S_{\hat{x}^*_j}) (\operatorname{VaR} is defined in Definition 2). Since we assume Assumption 1 holds for each such set of random variables, then we know that with minimum probability 1-(1-\epsilon)^{\operatorname{N_{RV}}}, the sum \operatorname{\delta}^*_j + \beta is greater than the value-at-risk level \epsilon of any sampled random variable, i.e. the sum \operatorname{\delta}^*_j + \beta is a non-noisy estimate of \operatorname{\mathbb{SR}}_{\epsilon}(S,\hat{x}),~\forall~\hat{x} \in \mathcal{D}_{\operatorname{s}}. Hence, repeating this same argument for each data point in \operatorname{\mathbb{X}},\operatorname{\mathbb{Y}} and setting R=0, as each sampled point is a non-noisy sample of our upper-bounding surface, we recover the results of Theorem 1 with minimum probability (1-(1-\epsilon)^{\operatorname{N_{RV}}})^{\operatorname{s}}:

|\mu_{\operatorname{s}}(\hat{x}) - \operatorname{\mathbb{SR}}_{\epsilon}(S,\hat{x})| \leq B \sigma_{\operatorname{s}}(\hat{x}),~\forall~\hat{x} \in \hat{\mathcal{X}}.

Our final result holds by unraveling the absolute-value inequality in (Proof), as \operatorname{\mathbb{SR}}_{\epsilon}(S) is an upper-bounding surface for \operatorname{SaR}_{\epsilon}(S).

Source and licence

The text on this page is from arXiv:2212.06253v1 by Prithvi Akella, Skylar X. Wei, Joel W. Burdick and Aaron D. Ames, under the licence CC BY 4.0. The corpus made these changes: it read the items out of the LaTeX source, it expanded the macros of the paper, it put each proof after its statement, and it converted the text to HTML. The formulas are the TeX of the paper, and KaTeX renders them. A formula or a reference can look different from the PDF. The page is published under CC BY 4.0.

The record of each item is in analysis/objects/2212/2212.06253.jsonl.