In this work, we present a novel class of parallelizable high-order time integration schemes for the approximate solution of additive ODEs. The methods achieve high order through a combination of a suitable quadrature formula involving multiple derivatives of the ODE’s right-hand side and a predictor-corrector ansatz. The latter approach is designed in such a way that parallelism in time is made possible. We present thorough analysis as well as numerical results that showcase scaling opportunities of methods from this class of solvers.
All occuring functions \Phi, \Phi_{\text{I}}, \Phi_{\text{E}} and their temporal derivatives \overset{\boldsymbol .}{\Phi},\overset{\boldsymbol .}{\Phi}_{\text{I}}, \overset{\boldsymbol .}{\Phi}_{\text{E}} are assumed to be smooth and Lipschitz continuous.
Let w^n denote an approximate solution to w at point t^n, and let w^{n,l}, 1 \leq l \leq s denote the stage values to be computed. A two-derivative Runge-Kutta method is any method that can be cast as
and update given by
where the B^{(1)}, B^{(2)}, b^{(1)}, and b^{(2)} are the Butcher tableaux that define the scheme. We say the method is globally stiffly accurate if b^{(1)}_j = B^{(1)}_{sj} and b^{(2)}_j = B^{(2)}_{sj}, i.e., the last stage defines the update:
In this work, we use the following two-derivative Runge-Kutta methods:
40 A fourth-order method (q=4) with two stages (s=2, one being fully explicit), which exactly corresponds to the method used in [SealSchuetz19]:
40 A sixth-order method (q=6) with three stages (s=3, one being fully explicit), as also used in [SSJ2017]:
40 An eigth-order method (q=8) with four stages (s=4, one being fully explicit):
For each Hermite-Birkhoff Runge-Kutta method, there holds,
This is due to the construction: The Hermite-Birkhoff polynomial is of order 2s-1=q-1, which gives an integral approximation of order q+1. (Note that the length of the integration area is \Delta t.)
\sqcap\sqcup 501em\sqcap\sqcup =0pt=0
To advance the solution to Eq. (?) from time level t^{n} to time level t^{n+1}, fill the values w^{n,[0],{l}} using a second-order IMEX-Taylor method:
40 Predict. Solve the following expression for w^{n,[0],{l}} and 1 \leq l \leq s:
Subsequently:
40 Correct. Solve the following for w^{n,[k+1],{l}}, for each 2 \leq l \leq s and each 0 \leq k < k_{\max}:
In order to close the recursion, if k = k_{\max} -1, then replace each of the k+2 with k_{\max}.
40 Update. In order to preserve the first-same-as-last property, we put
Finally, to seed initial conditions for this solver, we define
The difference between this algorithm and our previous one found in [SealSchuetz19] lies in the highlighted red terms:
In our original work each of these would have been evaluated at w^n. Here, we work with different iterate numbers. This potent modification makes it possible to parallelize the method in time without sacrificing the order of accuracy. The idea that will be layed out in this publication is very similar to – and in fact inspired by – the strategy proposed in [ChriOng11, ChriOng10].
Please note that Alg. 1 is not a Runge-Kutta method.
There is no implicitness in the Runge-Kutta quadrature \mathcal I_l. The only implicitness is in the difference of the \Phi_{\text{I}} and \overset{\boldsymbol .}{\Phi}_{\text{I}}. This term is used to introduce the necessary stability for stiff problems.
The matrix A is power-bound by one, i.e., for each n \in \operatorname{\mathbb{N}}, there holds
Define the matrix \operatorname{\mathfrak{E}} \in \operatorname{\mathbb{R}}^{s\times s} that is zero except for the last column, which is filled with ones, i.e.,
Then, A is given as the block matrix
There holds \|Av\| \leq \|v\|, which can be seen due to the fact that for each 1 \leq i \leq s\cdot (k_{\max}+1), there is a 1 \leq j \leq s \cdot (k_{\max}+1) such that (Av)_i = v_j. This concludes the proof.
\sqcap\sqcup 501em\sqcap\sqcup =0pt=0
There exists a constant C \in \operatorname{\mathbb{R}} such that for each j \in \operatorname{\mathbb{N}}^{\geq 0}, there holds
Subdivide E^n into k_{\max}+1 blocks of size s each; call the individual blocks E^{n,[k]}, 0 \leq k \leq k_{\max}. Then, there holds
The proof goes along the same lines as in [SealSchuetz19] and is hence omitted.
\sqcap\sqcup 501em\sqcap\sqcup =0pt=0
The recursion for Z^n given in (?) can be explicitly written as:
This can be easily proved using (?) and an induction over n. Note that one has to take into account that Z^0 = 0 by definition.
\sqcap\sqcup 501em\sqcap\sqcup =0pt=0
Consider the following linear recursion:
with \alpha, \beta_n \in \operatorname{\mathbb{R}} given for all n \in \operatorname{\mathbb{N}}. (As indicated, \beta_n is allowed to depend on n, while \alpha is not.) Also x_0 is supposed to be known. The analytical solution to this recursion is given by
We proceed by induction. For n=1, we have x_1 = \alpha x_0 + \beta_1, which agrees with formula (5).
Now assume that (5) is correct for all n < N. Then, there holds:
We compute the terms separately:
Now for II, there holds via a switching of the summation that
All together, this shows that also x_N can be formed according to (5). The statement is hence true.
\sqcap\sqcup 501em\sqcap\sqcup =0pt=0
Given that \Delta t is sufficiently small, and given the assumptions on the functions \Phi, \Phi_{\text{I}} and \Phi_{\text{E}} as done in Assumption 1, there is a constant C such that
and hence, the method is second-order convergent.
Note that due to Corollary 1, we can estimate terms like \|A^j B_E\| against fixed constants. Using the expression (4), we can hence compute
Given that C \Delta t is smaller than one, we get the inequality
with another constant C that does not depend on \Delta t or N. Note that summation of the first term begins at j=1. Now there holds that \|Z^n\| \leq \varepsilon_n, with \varepsilon_0 = 0 and \varepsilon_n, n > 0, defined as
This is exactly of form (5), and has hence the analytical solution
We can estimate (1+C\Delta t)^{n-j-1} against e^{C T_{end}}, and because of La. 3, there then holds that
Note that the property \sum_{k=0}^{j-1} \|E^{n-j}\| can be estimated against C\Delta t^2. Note also that constants are subject to change. They do however never depend on N or \Delta t.
\sqcap\sqcup 501em\sqcap\sqcup =0pt=0
The structure of the matrix A as given in (Proof) is very interesting: it “pushes elements in vectors downwards." To facilitate the understanding of the following analysis, we show different exponents of A. Obviously, A^0 = \operatorname{Id}, the identiy matrix, and A^1 is given in (Proof). There holds, always given that k_{\max} is large enough:
Eventually, there is a limit matrix A^{\infty}, given as
More formally, we have A^j = A^{\infty} for all j \geq k_{\max} -1. This structure is important, because it significantly simplifies the convergence analysis. Essentially, it states that most contributions to the overall error come from higher corrections. Intuitively, this is desirable, because higher corrections are supposed to have smaller error contributions, at least asymptotically.
Subdivide Z^n into k_{\max}+1 blocks of size s each, similarly to La. 3. Define \varepsilon_k^n, 0 \leq k \leq k_{\max} to be the infinity-norm of the k-th block. For n = 0, there is no error, and hence
Furthermore, for k = 0, the underlying consistency analysis is trivial, and hence \varepsilon_0^n can be bound by a positive constant C times \Delta t^2, i.e.,
The constant C depends of course on the analytical solution, the fluxes \Phi, the initial conditions w_0 and the like, but it does not depend on \Delta t or n. Furthermore, there is a constant C such that there holds for 1 \leq k < k_{\max} and 1 \leq n \leq N:
Before proving this lemma, we point out a couple of salient features:
40 Note that \varepsilon^n_{k_{\max}} has a global (i.e., over all n) dependence on \varepsilon^n_{k_{\max}-1}, while all the other \varepsilon^n_k do not have this kind of dependence on \varepsilon^n_{k-1}. 40 We have chosen to take the same constant C for both the expression of \varepsilon_0^n as well as for all the terms in the inequality for \varepsilon_k^n. This can be done without loss of generality, as we take the largest of all these constants.
To prove (?), we consider the analytical expression of Z^n given in (4). Let us first treat the last term in (4), \sum_{j=0}^{n-1} A^j E^{n-j}. Due to La. 3, we know that there is a constant C such that E^{n-j,[k]} \leq C \Delta t^{\min\{2+k,q\}+1}. There holds
By (\cdot)^{[k]}, we denote the k-the block of a vector, again, 0 \leq k \leq k_{\max}. We have used the fact that \Delta t = \frac{T_{end}}{N} and that A^j “pushes elements downwards," so there will be no \Delta t contributions of lower orders than the ones given here. This explains why we carefully chose the red terms in Alg. 1 the way we did.
In a similar way, we treat the other terms. They can all be handled alike, so we only consider the term
Note that the first term couples errors at correction level k-1 to those of k. Estimated, this gives, independent of whether k=k_{\max} or not, a contribution of
For k < k_{\max}, due to the peculiar structure of A^j, A^jB_I remains a block-matrix; on the k-th block-row, only two blocks are filled.(Let us clarify what we mean by block-rows and block-columns: The matrices we are dealing with here are of size s \cdot (k_{\max}+1) \times s \cdot (k_{\max}+1). They can hence be subdivided into (k_{\max}+1)^2 blocks of size s\times s. According to the notation in Alg. 1, we begin counting by k=0. Block-row k means hence in the big matrix the rows from k(s+1) + 1 to (k+1)(s+1).) The block-columns thereof are j and j+1, with j \geq k. A generous estimate yields the contribution
(In fact, for j \geq k_{\max} -1, the block columns are k_{\max}-1 and k_{\max}, so sharper estimates could be used.) For k=k_{\max}, the non-zero block-columns are k_{\max}-1 and k_{\max}. This then gives the contribution
This yields the desired result.
\sqcap\sqcup 501em\sqcap\sqcup =0pt=0
From the results in La. 6, we conjecture that the method is convergent with the k-dependent orders. That is, we postulate that each of the correction steps satisfy
and that the final correction step satisfies
Please note that the last correction step does not increase the order. This is different to the behavior of the algorithm in [SealSchuetz19]; it is a consequence of the fact that we have modified the values at time level t^n to make the algorithm ready for parallelism.
Although we only present numerical results that support Proposition 1 – and we only know from Thm. 0.1 that the method is at-least second order convergent – we can very well see the underlying structure from Eqs. (?)-(?): For k < k_{\max}, \varepsilon_k^n essentially possesses the contributions \Delta t \varepsilon_{k-1}^n, \Delta t \sum_{j=1}^{n-1}\sum_{h=k}^{k_{\max}} \varepsilon_h^{n-j} and \Delta t^{\min\{2+k,q\}}. (We have neglected the constants here.) If we assume that for increasing k, the order in \Delta t of \varepsilon_k^n would also increase or at least not decrease, the contribution \Delta t \sum_{j=1}^{n-1}\sum_{h=k}^{k_{\max}} \varepsilon_h^{n-j} scales as \mathcal{O}(\varepsilon_k^n). Furthermore, if the order between \varepsilon_{k}^n and \varepsilon_{k-1}^n differs by one, then \Delta t \varepsilon_{k-1}^n scales again as \varepsilon_k^n. The final constant term \Delta t^{\min\{2+k,q\}} then gives the overall order.
Note that a similar strategy can be pursued to obtain an adaptation criterion for the IMEX-MD scheme. Since the exact solution is not known for general problems, one could think about evaluating the change of the numerical solution after each iteration. If the iterates do not differ more than a user-defined threshold, no further correction steps are performed. Also timestep adaptation could be taken into account. We leave the detailed analysis of an adaptation of Alg. 1 in this direction to future investigations.
To advance the solution to Eq. (?) in time, we compute values w^{n,[k ],{l}}. The meaning of the parameters n, k and l is explained in Eq. (?) and thereafter. To account for the initial conditions, define
First, the values w^{n,[0],{l}} are filled using a straightforward second-order IMEX-Taylor method with an improved initial condition.
40 Predict. Solve the following expression for w^{n,[0],{l}} and 1 \leq l \leq s:
The modified terms ({w^{n-1,[1],{s}}}, {\Phi_{\text{E}}^{n-1,[1],{s}}}, and {\overset{\boldsymbol .}{\Phi}_{\text{E}}^{n-1,[1],{s}}}, marked in blue) simply use one higher iterate than the one found in Alg. 1. This makes the predictor third-order accurate due to its improved local truncation error.
40 Correct. Next, the correction steps take place to fill the values of w^{n,[k],{l}} for 1 \leq k \leq k_{\max}. Solve the following for w^{n,[k+1],{l}}, for each 2 \leq l \leq s and each 0 \leq k < k_{\max}:
The red term(s) (w^{n-1,[k+2],{s}}) are the same modified values we implemented in Alg. 1. The blue terms \Phi^{n,[k+1],{0}}, \ldots,\Phi^{n,[k+1],{l-1}} are evaluated at iterate k+1 instead of k. This produces smaller errors and requires less storage. Again, if k = k_{\max} -1, then the k+2 in the red terms are replaced by k_{\max} in order to close the recursion.
40 Update. In order to retain a first-same-as-last property, we update the solution with
The text on this page is from arXiv:2101.07846v1 by Jochen Schütz, David C. Seal and Jonas Zeifang, under the licence CC BY 4.0. The corpus made these changes: it read the items out of the LaTeX source, it expanded the macros of the paper, it put each proof after its statement, and it converted the text to HTML. The formulas are the TeX of the paper, and KaTeX renders them. A formula or a reference can look different from the PDF. The page is published under CC BY 4.0.
The record of each item is in analysis/objects/2101/2101.07846.jsonl.