Elsevier

Automatica

Volume 102, April 2019, Pages 1-9
Automatica

Brief paper
Distributed model predictive control—Recursive feasibility under inexact dual optimization

https://doi.org/10.1016/j.automatica.2018.12.037Get rights and content

Abstract

We propose a novel model predictive control (MPC) formulation, that ensures recursive feasibility, stability and performance under inexact dual optimization. Dual optimization algorithms offer a scalable solution and can thus be applied to large distributed systems. Due to constraints on communication or limited computational power, most real-time applications of MPC have to deal with inexact minimization. We propose a modified optimization problem inspired by robust MPC which offers theoretical guarantees despite inexact dual minimization. The approach is not tied to any particular optimization algorithm, but assumes that the feasible optimization problem can be solved with a bounded suboptimality and constraint violation. In combination with a distributed dual gradient method, we obtain a priori upper bounds on the number of required online iterations. The design and practicality of this method are demonstrated with a benchmark numerical example.

Keywords

Predictive control
Control of constrained systems
Large scale systems
Distributed dual optimization

1. Introduction

Model predictive control (MPC) is a well-established control method, that can be used to control complex dynamical systems and guarantee constraint satisfaction (Rawlings & Mayne, 2009). One of the main limitations to control a system with MPC comes from computational issues, since in each time step an optimization problem has to be solved. In order to apply MPC to large-scale systems, we have to consider distributed approaches, which fall in the domain of distributed MPC (DMPC) (Maestre et al., 2014, Müller and Allgöwer, 2017). If we want to facilitate DMPC applications to fast (physically) interconnected networks, we typically need scalable distributed optimization algorithms with bounds on the number of required iterations.
Dual optimization algorithms such as the alternating direction method of multipliers (ADMM), dual gradient methods and proximal decomposition have been studied to solve DMPC optimization problems online (Kögel and Findeisen, 2012, Necoara and Nedelcu, 2015, Necoara and Suykens, 2008). While these algorithms enable a fully distributed implementation and asymptotically converge to the optimal central solution, real-time requirements lead to early termination and an inexact solution. Contrary to primal decomposition methods (Stewart, Venkat, Rawlings, Wright, & Pannocchia, 2010), these inexact solutions based on dual optimization do not necessarily satisfy the posed constraints (dynamic, state and input constraints) in the MPC optimization problem. This necessitates additional modifications to ensure recursive feasibility and stability of the resulting MPC scheme.

Related work

In Giselsson and Rantzer (2014) DMPC without terminal constraints is investigated and a sufficient stopping condition for the distributed iteration based on a candidate solution is presented. For this approach no prior bound on the number of required iterations can be given.
In Kögel and Findeisen (2014) a primal optimization algorithm with constraint violations in the dynamic equality constraints is investigated. Recursive feasibility is ensured with an appropriate state and input constraint tightening.
In Necoara, Ferranti, and Keviczky (2015) and Rubagotti, Patrinos, and Bemporad (2014) constraint violations in the inequality constraints due to inexact dual optimization are addressed with an appropriate (constant or adaptive) constraint tightening. Constraint violations in the posed dynamic equality constraints are avoided by using a condensed formulation (Necoara et al., 2015) or projecting the intermediate solution to the set of dynamically feasible trajectories (Rubagotti et al., 2014). Both approaches are, however, unsuited for distributed large-scale systems.
In Ferranti and Keviczky (2015) constraint violations in inequality constraints and dynamic equality constraints are considered by using an appropriate constraint tightening. Recursive feasibility is ensured by choosing the tolerance and thus the constraint tightening adaptively. As a consequence, the number of iterations can vary and global communication is required to enable this adaptation. In Doan, Keviczky, and Schutter (2011) a similar constraint tightening is used for a distributed hierarchical MPC scheme.

Contribution

We propose a new framework to ensure recursive feasibility of inexact DMPC resulting from finite dual iterations. This consists of a constant constraint tightening and a stabilizing controller, motivated by robust MPC (Chisci, Rossiter, & Zappa, 2001). To avoid an overly conservative constraint tightening, we propose a modified optimization problem and employ a different candidate solution, that explicitly takes the inexactness into account. This presents a general procedure which is applicable to different MPC setups. By combining this framework with a dual distributed gradient algorithm, we obtain an a priori upper bound for the number of dual iterations to ensure recursive feasibility. Compared to Ferranti and Keviczky (2015), Giselsson and Rantzer (2014) and Necoara et al. (2015), no adaptive constraint tightening is required. Furthermore, compared to Doan et al. (2011), Ferranti and Keviczky (2015), Kögel and Findeisen (2014) and Rubagotti et al. (2014), no centralized operations are necessary, thus allowing a fully distributed implementation for large-scale systems.

Outline

The remainder of this paper is structured as follows: Section 2 presents the nominal distributed MPC formulation and explains the problem inherent in inexact dual optimization. Section 3 presents the modified formulation, derives closed-loop properties under inexact minimization and presents a corresponding distributed dual iteration scheme. Section 4 illustrates the practicality and simplicity of the proposed framework with a numerical example. Section 5 concludes the paper.
In the extended version (Köhler, Müller, & Allgöwer, 2018a), these results are generalized to MPC without terminal ingredients, unreachable setpoints, multi-step MPC and the distributed offline computation of the terminal ingredients is detailed.

2. Distributed model predictive control

Notation

The real numbers are R, the positive real numbers are R>0={rR|r>0} and the natural numbers are N. Given vectors aiRni, we abbreviate the column vector [a1,,an]=(a1,,an). The quadratic norm with respect to a positive definite matrix Q=Q is denoted by xQ2=xQx and the minimal and the maximal eigenvalue of Q are denoted by λmin(Q) and λmax(Q), respectively. For a polytopic constraint Ayb, we define an ϵ-feasible solution as any vector y that satisfies Ayb+ϵ1, with ϵR>0 and the vector of ones 1=[1,,1]. We call a vector ϵ-strictly feasible if it satisfies Aybϵ1. The Minkowski sum of two sets S,TRn is denoted by ST={x|sS,tT:x=s+t}.A distributed system is represented as a graph G=(N,E) with nodes N and edges E. Each node iN corresponds to a subsystem with local state xiRni and local input uiRmi. The neighborhood of a subsystem i is given by Ni={j|(i,j)E}{i}, with xNiRnNi, nNi=jNinj.

2.1. Problem setup

The distributed linear discrete-time system is given by (1)xi(t+1)=ANixNi(t)+Biui(t),iN,with polytopic state and input constraints of the form (2)xNiXNi={xNi|HNixNihNi},(3)uiUi={ui|Liuili}, where hNiR>0pi and liR>0qi. We consider the general case, where the control input is given by (4)ui(t)=KNixNi(t)+vi(t),where K is some existing distributed controller and v is the input calculated using distributed MPC. If no such feedback is known, we can always set K=0. However, including this feedback can reduce the conservatism and mitigate the deteriorating effects of suboptimality on closed-loop stability. The overall system is given by (5)x(t+1)=Ax(t)+Bu(t)=(A+BK)AKx(t)+Bv(t),with the polytopic constraints X={x|xNiXNiiN}={x|Hxh}Rn,U=U1×U|N|={u|Lul}Rm, where lR>0q and hR>0p. We consider a structured quadratic stage cost (x,v)=xQ2+vR2, with block diagonal positive definite matrices Q and R. We consider an MPC framework including a terminal cost and terminal set. To this end, we make the following assumption.

Assumption 1

There exists a terminal cost Vf(x)=iNxiPi2=xP2 with a block diagonal matrix P, a distributed terminal controller ui=Kf,NixNi, and a distributed compact polytopic set Xf={x|Fxf}, such that the following conditions hold for each xfXf (6a)Vf((A+BKf)xf)Vf(x)(xf,(KfK)xf),(6b)xfX,uf=KfxfU,(6c)(A+BKf)xfXf.

Remark 2

In Conte, Jones, Morari, and Zeilinger (2016) distributed linear matrix inequalities (LMIs) are presented that can be used to compute a distributed terminal cost and an ellipsoidal terminal set Xf. Ellipsoidal terminal constraints lead to a (distributed) quadratically constrained quadratic program (QCQP), which makes the online optimization more complex. Methods to obtain a distributed polytopic terminal set Xf are for example given in Kögel and Findeisen (2012) and Trodden (2016). The offline computation of the distributed terminal ingredients is discussed in more detail in the extended version (Köhler et al., 2018a A.1). The proposed framework can also be used without such terminal ingredients, which is discussed in Köhler et al. (2018a, A.2, A.3).
The open-loop cost of a state sequence x(|t)Rn×N+1 and an input sequence v(|t)Rm×N with the prediction horizon NN is defined as JN(x(|t),v(|t))k=0N1(x(k|t),v(k|t))+Vf(x(N|t)). The standard MPC optimization problem is given by (7)VN(x(t))=minv(|t),x(|t)JN(x(|t),v(|t))s.t. x(k+1|t)=AKx(k|t)+Bv(k|t),x(0|t)=x(t),x(N|t)Xf,x(k|t)X,u(k|t)=v(k|t)+Kx(k|t)U. The solution to this optimization problem is the value function VN and optimal state and input trajectories (x(|t),v(|t)) that satisfy the dynamic equality constraint and the state and input constraints. Problem (7) is a distributed quadratic program, the solution of which is discussed in Sections 2.23.5.
For the closed-loop operation the first step of the optimal input v(|t) is applied to the system (5), resulting in the following closed-loop system dynamics: (8)x(t+1)=AKx(t)+v(0|t)=x(1|t).The following theorem is a standard result in MPC and establishes the desired properties.

Theorem 3

Rawlings & Mayne, 2009

Let Assumption 1 hold and assume that Problem (7) is feasible att=0. Then Problem (7) is recursively feasible and the origin x=0 is asymptotically stable for the resulting closed-loop system (8).

2.2. Distributed (dual) optimization

In the following, we motivate why we consider inexact dual optimization and explain why it necessitates modifications to Problem (7). Most theoretical results for MPC (such as Theorem 3) assume that the optimal solution to (7) is obtained in real time, which is rarely achievable in practice.
If primal optimization methods are used, Theorem 3 remains valid with inexact optimization assuming a suitable initialization (Scokaert et al., 1999, Stewart et al., 2010). However, an application of primal optimization methods to large-scale distributed systems suffers from various difficulties, including initialization and scalability.
Thus, we consider dual optimization algorithms (Kögel and Findeisen, 2012, Necoara and Nedelcu, 2015, Necoara and Suykens, 2008), which only require neighbor-to-neighbor communication and can be implemented in a fully distributed manner. The main drawback of dual optimization is that the constraints (dynamic, state and input) are not necessarily satisfied after finite iterations. This necessitates additional modifications to enable theoretical guarantees after finite iterations, compare (Ferranti and Keviczky, 2015, Rubagotti et al., 2014). In the following, we provide a novel MPC formulation which is suitable for distributed computation and explicitly takes the inexact dynamics of approximate solutions into account.

3. Inexact distributed MPC

In the following, we consider bounds on the accuracy ϵ, interpret them as disturbances and use tools from robust MPC (Chisci et al., 2001) to compensate the effects of inexact minimization. The proposed modifications are inspired by Ferranti and Keviczky (2015) and directly take the inexactness of the solver into account. By making use of an inexact candidate solution, we obtain a formulation that requires no adaptation and thus no global communication.

3.1. Inexact MPC and constraint tightening

Define an accuracy for the dynamic, state, input and terminal constraints and strict feasibility ϵz,ϵx,ϵu,ϵf,ϵλR>0, given by the user. Consider relaxation parameters (9)ϵz,kϵλ+(N1k)(ϵλ+ϵz),k=0,,N1,and the sets Wk={wRn|wϵz,k+ϵz}. We tighten the constraints using the k-step support function (Conte, Zeilinger, Morari, & Jones, 2013), which for some aRn and kN is defined as (10)σW(a,k)=supwW0kay(k),s.t. y(l+1)=AKy(l)+w(l),y(0)=0. The tightened state and input constraints are given by X¯k={x|Hxh¯k},U¯k={u|Lul¯k},with (11)h¯j,k=hjσW(Hj,k)ϵxk(ϵx+ϵλ),(12)l¯j,k=ljσW(KLj,k)ϵuk(ϵu+ϵλ). Here, h¯j,k denotes the jth component of h¯k, hj the jth component of h and Hj the jth row of H, jp. The evaluation of the k-step support function amounts to solving a distributed linear program (LP) offline. The resulting tightened constraints preserve the distributed structure and can equally be represented with the local polytopic sets X¯Ni,k,U¯i,k.

Assumption 4

Consider the terminal cost and controller from Assumption 1. There exists a compact tightened terminal set X¯f={x|Fxf¯}, such that the following conditions hold (13a)X¯f,ϵk=0N1AKN1kWkXf,(13b)X¯f,ϵAKN1W0{x|Hxh¯N11pϵλ},(13c)Kf(X¯f,ϵAKN1W0){u|Lul¯N11qϵλ},(13d)(A+BKf)X¯f,ϵAKN1W0X¯f,λ,X¯f,ϵ{x|Fxf¯+1rϵf},X¯f,λ{x|Fxf¯1rϵλ}.
The sets X¯f,ϵ,X¯f,λ are needed to study strict recursive feasibility (ϵλ) under inexact minimization (ϵf). A sufficient condition for (13b) is X¯f,ϵX¯N. In case Kf=K, KfX¯f,ϵU¯N is a sufficient condition for (13c). Condition (13d) requires contractivity of the terminal set, despite the additive disturbance w0.
If the terminal set in Assumption 1 is contractive, Assumption 4 can be satisfied with the following design procedure: for a fixed accuracy ϵ and prediction horizon N, compute the tightened constraints (11). Then scale the terminal set Xf such that conditions (13a)(13c) are satisfied. Finally, verify that condition (13d) is satisfied. If this is not the case, decrease ϵ and start over. In Köhler et al. (2018a), we show that the proposed framework can also be used without constructing a terminal set.
With this, we define the modified optimization problem (14a)minv(|t),z(|t)JN(z(|t),v(|t))(14b)s.t. AKz(k|t)+Bv(k|t)z(k+1|t)ϵz,k,(14c)z(k|t)X¯k,v(k|t)+Kz(k|t)U¯k,k=0,,N1,(14d)z(N|t)X¯f,(14e)z(0|t)=x(t). Compared to the original optimization Problem (7), the state and input constraints are tightened and the dynamic equality constraints are relaxed to inequality constraints. We do not try to find a solution that exactly satisfies the dynamic constraints, but only consider a relaxed dynamic constraint with the parameter ϵz,k. This relaxation will allow us to construct a feasible candidate solution which again does not exactly satisfy the dynamic constraints. This is the key insight and novelty in order to prove recursive feasibility and stability under inexact minimization. The resulting Problem (14) is a distributed quadratic program with linear inequality constraints.
To study recursive feasibility of (14) under the inexact DMPC we introduce the notion of ϵ-feasible solutions.

Definition 5

An ϵ-feasible solution to (14) is any pair (zϵ(|t),vϵ(|t)), that satisfies (15)AKzϵ(k|t)+Bvϵ(k|t)zϵ(k+1|t)ϵz,k+ϵz,Hzϵ(k|t)h¯k+1pϵx,L(vϵ(k|t)+Kzϵ(k|t))l¯k+1qϵu,Fzϵ(N|t)f¯+1rϵf,zϵ(0|t)=x(t).
This formulation allows a constraint violation for the posed constraints (14b)(14d) by ϵz,ϵx,ϵu, and ϵf, respectively. A corresponding algorithm to ensure an ϵ-feasible solution with finite iterations is presented in Section 3.5.

3.2. Feasible consolidated trajectory

In order to characterize the feasibility of on an ϵ-feasible solution, we consider the consolidated1 trajectory (Ferranti & Keviczky, 2015).

Proposition 6

Let Assumption 1, Assumption 4 hold. Given anϵ-feasible solution (15)zϵ(|t),vϵ(|t) at timet, the consolidated state and input trajectories x¯ϵ(|t),u¯ϵ(|t) x¯ϵ(k+1|t)AKx¯ϵ(k|t)+Bvϵ(k|t),x¯ϵ(0|t)x(t),(16)u¯ϵ(k|t)Kx¯ϵ(k|t)+vϵ(k|t), satisfy x¯ϵ(k|t)X,u¯ϵ(k|t)=vϵ(k|t)+Kx¯ϵ(k|t)U,x¯ϵ(N|t)Xf.

Proof

The inexact relaxed dynamic constraint (15) can be equivalently written as a dynamic equality constraint with an additive disturbance (17)zϵ(k+1|t)=AKzϵ(k|t)+Bvϵ(k|t)+wk,wkWk.The consolidated trajectory (16) satisfies (18)x¯ϵ(k|t){zϵ(k|t)}l=0k1AKkl1Wl(9){zϵ(k|t)}l=0k1AKlW0,which implies Hjx¯ϵ(k|t)(10)Hjzϵ(k|t)+σW(Hj,k)Def. 5h¯j,k+ϵx+σW(Hj,k)(11)hj,Lju¯ϵ(k|t)(10)Lj(vϵ(k|t)+Kzϵ(k|t))+σW(KLj,k)Def. 5l¯j,k+ϵu+σW(KLj,k)(12)lj. Terminal constraint satisfaction follows by condition (13a) in combination with the characterization (18) for k=N.  □
Proposition 6 shows that the consolidated trajectory based on the inexact optimization has all the desirable properties of the standard optimal solution x(|t),u(|t) to Problem (7). The closed-loop system resulting from an inexact DMPC is given by (19)u(t)=Kx(t)+vϵ(0|t)=u¯ϵ(0|t),x(t+1)=AKx(t)+vϵ(0|t)=x¯ϵ(1|t). Thus, Proposition 6 implies that the closed loop based on an ϵ-feasible solution satisfies the state and input constraints.

Remark 7

In order to show feasibility of the consolidated trajectory, the constraint tightening (11), (12) could be formulated without the term k(ϵx+ϵλ) and the support function could be defined based on the smaller set W0××Wk, compare (Ferranti & Keviczky, 2015). The more restrictive constraint tightening will be crucial in order to establish recursive feasibility of Problem (14) for the closed-loop system (19) based on an ϵ-feasible solution. The issue of using a more conservative constraint tightening to establish recursive feasibility is also addressed in Kögel and Findeisen (2014) and Rubagotti et al. (2014).

3.3. Recursive feasibility under inexact minimization

The following Theorem is the main contribution of this paper. It establishes recursive feasibility of Problem (14) under the inexact MPC control law with a suitable candidate solution.

Theorem 8

Let Assumption 1, Assumption 4 hold. Given anϵ-feasible solution (15) zϵ(|t),vϵ(|t) at time t, the candidate sequence (20)ṽ(k|t+1)=vϵ(k+1|t),k=0,,N2,ṽ(N1|t+1)=(KfK)z̃(N1|t+1),z̃(0|t+1)=x(t+1)=zϵ(1|t)+w0,w0W0,z̃(k|t+1)=zϵ(k+1|t)+AKkw0,k=0,,N1,z̃(N|t+1)=(A+BKf)z̃(N1|t+1), is an ϵλ-strictly feasible solution to the optimization Problem (14) at timet+1. Problem (14) is recursively feasible for the closed-loop system (19).

Proof

The proof is composed of three parts. First, we show strict satisfaction of the relaxed dynamic constraints. Then we show strict satisfaction of the tightened state and input constraints. Finally, we show strict satisfaction of the terminal constraint and thus establish recursive feasibility.
Part I: Show that the candidate sequence z̃(|t+1),ṽ(|t+1) in (20) strictly satisfies the relaxed dynamic constraint (14b): The candidate input ṽ(|t+1) (20) is constructed by shifting the previous input sequence vϵ(|t) by one time step and appending the terminal controller Kf. The state sequence zϵ(|t) is shifted with an additional error term w0 propagated through the system dynamics to ensure satisfaction of the initial state constraint (14e). Substituting (17) in z̃(|t+1) yields z̃(k|t+1)=zϵ(k+1|t)+AKkw0=AKzϵ(k|t)+Bvϵ(k|t)+AKkw0+wk=AKz̃(k1|t+1)+Bṽ(k1|t+1)+wk, for k=1,,N1, with wkϵz,k+ϵz=ϵz,k1ϵλ. Similarly, the last dynamic constraint (k=N) is satisfied with equality, which implies that all relaxed dynamic constraints are strictly satisfied with ϵz,N1=ϵλ.
Part II: Show that the candidate sequence (20) strictly satisfies the state and input constraints (14c): Due to the definition of the support function2 and linear superposition we have σW(Hj,k+1)σW(Hj,k)+HjAKkw0,w0W0,which implies h¯j,k+1+HjAKkw0h¯j,k(ϵx+ϵλ).The candidate sequence satisfies Hjz̃(k|t+1)=Hjzϵ(k+1|t)+HjAKkw0h¯j,k+1+HjAKkw0+ϵxh¯j,kϵλ,k=0,,N2. and hence the state constraints are strictly satisfied. For the input constraints the same argument holds with Lj(ṽ(k|t+1)+Kz̃(k|t+1))=Lj(vϵ(k+1|t)+Kzϵ(k+1|t)+KAKkw0)l¯j,k+1+ϵu+LjKAKkw0l¯j,kϵλ,k=0,,N2. Given zϵ(N|t)X¯f,ϵ, conditions (13b) and (13c) imply strict satisfaction of the state and input constraints at k=N1.
Part III: Show that the terminal state of the candidate sequence (20) strictly satisfies the terminal constraint (14d): Condition (13d) ensures z̃(N|t+1)=(A+BKf)z̃(N1|t+1)=(A+BKf)(zϵ(N|t)+AKN1w0)(A+BKf)(X¯f,ϵAKN1W0)X¯f,λX¯f.
This theorem ensures recursive feasibility under inexact dual optimization with bounded constraint violation. The candidate solution z̃ with the corresponding tightened (and shifted) constraint set X¯k is sketched in Fig. 1. The tightened constraint set X¯k is constructed, such that ϵ-feasibility of zϵ(|t) implies ϵλ-strict feasibility of z̃(|t+1) w.r.t. the shifted constraint set X¯k, despite the error w0.

3.4. Closed-loop stability

To study stability properties of the closed-loop system, we use the following definition regarding the suboptimality of the inexact solution.
  1. Download: Download high-res image (124KB)
  2. Download: Download full-size image

Fig. 1. Illustration of the strictly feasible candidate sequence z̃(|t+1) in relation to the previous solution zϵ(|t), the error in the first dynamic constraint w0 and the (shifted) tightened constraints X¯k over the prediction horizon.

Definition 9

Given an ϵ-feasible solution (Definition 5), the suboptimality η w.r.t. the optimal solution is defined as (21)JN(x¯ϵ(|t),vϵ(|t))VN(x(t))+η.The inexact optimal solution is given by (22)VN,ϵ(x(t))minz(|t),v(|t)JN(z(|t),v(|t))s.t. z(|t),v(|t) satisfy (15). The suboptimality ηϵ with respect to this inexact optimal solution is given by (23)JN(zϵ(|t),vϵ(|t))VN,ϵ(x(t))+ηϵ.Solutions satisfying (15), (21), (23) are called (ϵ,η,ηϵ)-approximate solutions.
Corresponding bounds on the suboptimality for inexact dual optimization will be established in Proposition 12. The following proposition shows that the proposed inexact DMPC approximately preserves the stability properties of nominal MPC based on exact optimization.

Proposition 10

Let Assumption 1, Assumption 4 hold. Given an(ϵ,η,ηϵ)-approximate solutionzϵ(|t),vϵ(|t) (Definition 9) at time t, the candidate sequence z̃(|t+1),ṽ(|t+1) in Theorem 8 implies (24)VN(x(t+1))VN(x(t))(x(t),v(t))+η.Hence the origin x=0 is practically asymptotically stable (Grüne & Pannek, 2017 Def. 2.15) for the closed-loop system (19) based on(ϵ,η,ηϵ)-approximate solutions at each time t. Given a sufficiently small ϵ,ηϵ, the additional bound (25)VN,ϵ(x(t+1))VN,ϵ(x(t))(x(t),v(t))+ηϵ+β1holds with β1 according to (26).

Proof

Part I: Consolidated cost VN: The candidate input sequence ṽ(|t+1) from Theorem 8 with the corresponding consolidated state trajectory x¯(|t+1) is a feasible solution to (7) (Proposition 6). Using suboptimality η according to Definition 9, this implies VN(x(t+1))JN(x¯(|t+1),v(|t+1))(6a)JN(x¯ϵ(|t),vϵ(|t))(x(t),v(t))(21)VN(x(t))+η(x(t),v(t)). Practical asymptotic stability follows from standard Lyapunov arguments.
Part II: Inexact optimal cost VN,ϵ: There exist constants α̲,α¯, such that zϵX¯f implies Vf(zϵ)α¯, and Vf(zϵ)α̲ implies zϵX¯f. In the following we consider a bound VN,ϵ(x(t))V¯ϵηϵ, with some V¯ϵα̲+ηϵ, which is recursively established in the end. This bound in combination with the suboptimality implies JN(zϵ(|t),vϵ(|t))V¯ϵ. The stage cost and terminal cost of the candidate solution satisfy