# Semiconcavity for the value function of a minimum time problem with time
delay

Elisa Continelli & Cristina Pignotti
Dipartimento di Ingegneria e Scienze dell’Informazione e Matematica
Università degli Studi di L’Aquila
Via Vetoio, Loc. Coppito, 67100 L’Aquila Italy

###### Abstract

In this paper, we deal with a minimum time problem in presence of a time delay
$\tau.$ The value function of the considered optimal control problem is no
longer defined in a subset of $\mathbb{R}^{n}$, as it happens in the undelayed
case, but its domain is a subset of the Banach space
$C([-\tau,0];\mathbb{R}^{n})$. For the undelayed minimum time problem, it is
known that the value function associated with it is semiconcave in a subset of
the reachable set and is a viscosity solution of a suitable Hamilton-Jacobi-
Belmann equation. The Hamilton-Jacobi theory for optimal control problems
involving time delays has been developed by several authors. Here, we are
rather interested in investigating the regularity properties of the minimum
time functional. Extending classical arguments, we are able to prove that the
minimum time functional is semiconcave in a suitable subset of the reachable
set.

##  1 Introduction

Consider the nonlinear system

$\begin{cases}y^{\prime}(t)=f(y(t),u(t)),\quad&\,t\geq 0,\\\
y(0)=x\in\mathbb{R}^{n},\end{cases}$ (1.1)

where $U$ is a compact subset of $\mathbb{R}^{m}$ and $f:\mathbb{R}^{n}\times
U\rightarrow\mathbb{R}^{n}$ is a given function. A measurable function
$u:[0,+\infty)\rightarrow U$ is called a control and the solution to the state
equation (1.1) with the initial datum $x$ corresponding to the control $u$ is
denoted by $y(\cdot;x,u)$. Given a nonempty closed set
$\mathcal{K}\subseteq\mathbb{R}^{n}$, called the target, the reachable set
$\mathcal{R}$ is the set of all points $x\in\mathbb{R}^{n}$ that can be
steered to $\mathcal{K}$ in finite time. The minimum time problem for the
system (1.1) is the following:

$\text{minimize }\theta(x,u),\text{ over all controls }u,$ (1.2)

where $\theta(x,u)$ is the first time at which the trajectory starting at
$x\in\mathbb{R}^{n}$ with control $u$ reaches the target $\mathcal{K},$ i.e.

$\theta(x,u):=\min\\{t\geq 0:y(t,x,u)\in\mathcal{K}\\}.$

The value function associated to the optimal control problem (1.2) is called
the minimum time function and it is the function
$T:\mathcal{R}\rightarrow\mathbb{R}$ defined as

$T(x):=\inf_{u}\theta(x,u),\quad\forall x\in\mathcal{R}.$ (1.3)

The minimum time problem (1.2) is a classical topic in control theory that has
caught the attention of many researchers in the past decades. In particular,
the regularity properties of the minimum time function $T$ given by (1.3) have
been largely investigated. It is known that, under a controllability
assumption known as the Petrov condition (see [27, 28]), the minimum time
function is locally Lipschitz continuous in the reachable set. However, there
are simple examples showing that the minimum time function fails to be
differentiable everywhere. In [12], it has been proved that, under suitable
assumptions, the minimum time function is semiconcave in
$\mathcal{R}\setminus\mathcal{K}.$ The semiconcavity result has been then
extended to general exit time problems in [11]. Moreover, it has been proved
in [4] that the minimum time function is the unique viscosity solution of a
suitable Hamilton-Jacobi-Belmann equation. Semiconcavity is a very useful
property to get uniqueness results for weak solutions of Hamilton-Jacobi
equations. Also, the semiconcavity property allows deriving some optimality
conditions (see e.g. [10] for the minimum time function, [26, 29] for exit
time problems). For more details on semiconcave functions, Hamilton-Jacobi
theory and viscosity solutions we refer to [13, 5, 14, 15]. For other results
regarding semiconcavity of different types of equations see for instance [1,
2, 3, 9, 21].
In this paper, we deal with the minimum time problems for the delayed control
system

$\begin{cases}y^{\prime}(t)=f(y(t-\tau),u(t)),\quad&\,t\geq 0,\\\
y(s)=x(s),\quad&s\in[-\tau,0],\end{cases}$ (1.4)

where $x\in C([-\tau,0];\mathbb{R}^{n})$, being $C([-\tau,0];\mathbb{R}^{n})$
the Banach space of all continuous functions defined in $[-\tau,0]$ with
values in $\mathbb{R}^{n}$. For a complete understanding of differential
equations of the type (1.4), we refer the reader to [19, 20]. Due to the fact
that the initial data are functions whose domain is the time interval
$[-\tau,0]$, the minimum time function associated to the minimum time problem
for system (1.4) is no longer defined in a subset of $\mathbb{R}^{n}$. Indeed,
the reachable set $\mathcal{R}$ is now a subset of the inifinite dimensional
space $C([-\tau,0];\mathbb{R}^{n})$.
The Hamilton-Jacobi theory for optimal control problems involving time delays
has been developed by several authors ([6, 7, 8, 23, 24, 25, 30, 31, 32, 33,
34]). In all of these works the optimal control problems under consideration
are with finite time horizon. Also, in [23, 24, 25, 30, 31, 34], optimal
control problems associated to control systems with dynamics depending not
just on a past instant but on a whole interval of the past history of the
trajectory (in the literature one refers to such systems as hereditary
systems) are investigated and a notion of differentiability for functionals
belonging to the Banach space $C([-\tau,0];\mathbb{R}^{n})$, the so-called
coinvariant (or ci)-differentiability, is exploited. In [6, 7, 33], optimality
conditions, with particular attention to the Pontryagin maximum principle, are
given for optimal control problems in presence of time delays. We also mention
[16, 17, 18] for a general study of Hamilton-Jacobi equations in infinite
dimension and [22] for the analysis of a class of Hamilton-Jacobi equations in
Hilbert spaces.
Motivated by this, we analyze the regularity properties of the minimum time
function associated with the delayed control system (1.4). Extending the
classical arguments employed in [12], we are able to prove that, also in this
functional setting, the Petrov controllability condition implies the local
Lipschitz continuity of the minimum time function. Furthermore, we show that
the minimum time function is semiconcave in a suitable subset of the reachable
set.

The present paper is organized as follows. In Sect. 2 we present some
preliminary definitions and notations and we introduce rigorously the minimum
time problem for systems involving time delays. In Sect. 3, we show that the
minimum time function is locally Lipschitz continuous in
$\mathcal{R}_{\bar{M}}$, where $\mathcal{R}_{\bar{M}}$ is the set of all
functions $x$ in the reachable set $\mathcal{R}$ that are Lipschitz continuous
of constant $\bar{M}$, being $\bar{M}$ a positive constant greater than the
constant of boundness $M$ of the dynamics $f$. In Sect. 4 we prove our main
result, which ensures that the minimum time function is semiconcave in a
suitable subset of $\mathcal{R}_{M}$, where $\mathcal{R}_{M}$ is the set of
all functions $x$ in the reachable set $\mathcal{R}$ that are Lipschitz
continuous of constant $M.$ Finally, in Sect. 5 we make some conclusions and
comments about the analysis carried out throughout this paper.

##  2 Preliminaries

Let $\lvert\cdot\rvert$ be the usual norm on $\mathbb{R}^{n}$. Given a
nonempty closed set $\mathcal{K}\subseteq\mathbb{R}^{n}$, we denote with
$d_{\mathcal{K}}(\cdot)$ the distance function from $\mathcal{K}$, namely

$d_{\mathcal{K}}(z)=\inf_{y\in\mathcal{K}}\lvert y-z\rvert,\quad\forall
z\in\mathbb{R}^{n}.$

Moreover, for any $\rho>0$, we set

$\mathcal{K}_{\rho}=\\{z\in\mathbb{R}^{n}:d_{\mathcal{K}}(z)<\rho\\}.$

We shall denote with $C([-\tau,0];\mathbb{R}^{n})$ the Banach space of all
continuous functions defined in $[-\tau,0]$ with values in $\mathbb{R}^{n}$.
We endow $C([-\tau,0];\mathbb{R}^{n})$ with the uniform norm

$\lVert x\rVert_{\infty}=\sup_{s\in[-\tau,0]}\lvert x(s)\rvert.$

We recall the definition of a semiconcave function.

######  Definition 2.1.

A continuous function $v:\Omega\rightarrow\mathbb{R}$, with
$\Omega\subseteq\mathbb{R}^{n}$, is called semiconcave if, for any convex set
$K\subset\subset\Omega$, there exists $c_{K}>0$ such that

$v(x+h)+v(x-h)-2v(x)\leq c_{k}\lvert h\rvert^{2},$

for any $x,h$ such that $x,x+h,x-h\in K$.

For the analysis we carry out in this paper, we need to extend the above
definition to functionals whose domain is a subset of the Banach space
$C([-\tau,0];\mathbb{R}^{n})$.

######  Definition 2.2.

A continuous functional $v:S\rightarrow\mathbb{R}$, with $S\subseteq
C([-\tau,0];\mathbb{R}^{n})$, is called semiconcave if, for any convex set
$K\subset\subset S$, there exists $c_{K}>0$ such that

$v(x+h)+v(x-h)-2v(x)\leq c_{k}\lVert h\rVert_{\infty}^{2},$

for any $x,h$ such that $x,x+h,x-h\in K$.

###  2.1 The minimum time problem with time delay

Let $\mathcal{K}$ be a nonempty closed subset of $\mathbb{R}^{n}$ and $U$ be a
subset of $\mathbb{R}^{m}$. Let $f:\mathbb{R}^{n}\times
U\rightarrow\mathbb{R}^{n}$ be a given function.
Consider the delayed control system

$\begin{cases}y^{\prime}(t)=f(y(t-\tau),u(t)),\quad&\,t\geq 0,\\\
y(s)=x(s),\quad&s\in[-\tau,0],\end{cases}$ (2.5)

where $x\in C([-\tau,0];\mathbb{R}^{n})$.
In this paper, the set $\mathcal{K}$ plays the role of the target to which one
wants to steer the trajectories of the nonlinear control system (2.5).

We assume that the following conditions hold.

  * $(H_{1})$

$f:\mathbb{R}^{n}\times U\rightarrow\mathbb{R}^{n}$ is continuous and
satisfies

$\lvert f(z,u)\rvert\leq M,\quad\forall z\in\mathbb{R}^{n},u\in U,$ $\lvert
f(z,u)-f(y,u)\rvert\leq L\lvert z-y\rvert,\quad\forall
z,y\in\mathbb{R}^{n},u\in U.$

  * $(H_{2})$

$U\subseteq\mathbb{R}^{m}$, $\mathcal{K}\subseteq\mathbb{R}^{n}$ are compact
sets.

  * $(H_{3})$

The function $f$ is semiconcave with respect to the variable $x$, uniformly
with respect to $u$.

A control for equation (2.5) is a measurable function $u:[0,\infty)\rightarrow
U$.
Given a control $u:[0,\infty)\rightarrow U$ and an initial datum $x\in
C([-\tau,0];\mathbb{R}^{n})$, adopting a step-by-step procedure and using the
assumption $(H_{1})$, we get that there exists a unique solution to the
control system (2.5) associated to the control $u$ for the initial condition
$x$, denoted by $y(\cdot;x,u)$. For further details about existence and
uniqueness of functional differential equations, we refer to [19], [20].
Given a solution $y(\cdot;x,u)$ of the control system (2.5), for every $t\geq
0$, let $y_{t}(\cdot;x,u)$ be the function in $C([-\tau,0];\mathbb{R}^{n})$
given by

$y_{t}(s;x,u)=y(t+s;x,u),\quad\forall s\in[-\tau,0].$

For any initial datum $x\in C([-\tau,0];\mathbb{R}^{n})$ and for any control
$u$, we set

$\theta(x,u)=\min\\{t\geq 0:y(t;x,u)\in\mathcal{K}\\},$

i.e. $\theta(x,u)\in[0,+\infty]$ is the first time at which the trajectory
$y(\cdot;x,u)$ reaches the target $\mathcal{K}$.
Moreover, we define the reachable set $\mathcal{R}$ as the set of all initial
data starting from which the target can be reached in a finite time, namely

$\mathcal{R}=\\{x\in
C([-\tau,0];\mathbb{R}^{n}):\theta(x,u)<+\infty,\,\text{for some
control}\,u\\}.$

The optimal control problem we deal with, called the minimum time problem, is
the following:

$\quad\text{minimize }\theta(x,u)\text{ over all controls }u.$ (2.6)

The value function associated to (2.6) is called minimum time function and it
is the functional $T:\mathcal{R}\rightarrow[0,+\infty)$ defined as

$T(x)=\inf_{u}\theta(x,u),\quad\forall x\in\mathcal{R}.$

Our interest is in investigating the regularity properties of the minimum time
functional over a certain subset of the reachable set. For this purpose, we
will assume the following controllability condition of Petrov type:

  * $(H_{4})$

There exist $\mu,\sigma>0$ such that, for any $x\in\mathbb{R}^{n}$ with
$x\in\mathcal{K}_{\sigma}\setminus\mathcal{K}$, there exists $u_{x}\in U$ such
that

$f(x,u_{x})\cdot\frac{x-\pi(x)}{\lvert x-\pi(x)\rvert}\leq-\mu,$

for some $\pi(x)\in\mathcal{K}$ such that $d_{\mathcal{K}}(x)=\lvert
x-\pi(x)\rvert.$

##  3 Local Lipschitz continuity of the value function

Now, we investigate the Lipschitz continuity of the minimum time functional.
We first present a version of the dynamic programming principle that is
suitable for our functional setting.

######  Proposition 3.1 (Dynamic programming principle).

Let $x\in\mathcal{R}$. Then, for every $t\in[0,T(x)]$ and for every control
$u$ such that $\theta(x,u)<+\infty$, it holds

$T(x)\leq t+T(y_{t}(\cdot;x,u)).$ (3.7)

###### Proof.

By contradiction, we assume that there exist $\tilde{t}\in[0,T(x)]$ and a
control $\tilde{u}$ satisfying $\theta(x,\tilde{u})<+\infty$ such that

$T(x)>\tilde{t}+T(y_{\tilde{t}}(\cdot;x,\tilde{u})).$

Then, we define the control $\bar{u}:[0,+\infty)\rightarrow U$ given by

$\bar{u}(t)=\begin{cases}\tilde{u}(t),\quad t\in[0,\tilde{t}],\\\ u(t),\quad
t>\tilde{t},\end{cases}$

where $u$ is an optimal control for $y_{\tilde{t}}(\cdot;x,\tilde{u})$, i.e.
$T(y_{\tilde{t}}(\cdot;x,\tilde{u}))=\theta(y_{\tilde{t}}(\cdot;x,\tilde{u}),u)$.
Then, by definition of $\bar{u}$ we get

$\tilde{t}+T(y_{\tilde{t}}(\cdot;x,\tilde{u}))=\theta(y_{\tilde{t}}(\cdot;x,\tilde{u}),\bar{u})<T(x),$

contradicting the optimality of $T(x)$. Thus, (3.7) is valid. ∎

For our purpose, given $\bar{M}>0$, we set

$\text{Lip}_{\bar{M}}([-\tau,0];\mathbb{R}^{n})=\\{x\in
C([-\tau,0];\mathbb{R}^{n}):\lvert x(s)-x(t)\rvert\leq\bar{M}\lvert
s-t\rvert,\,\forall s,t\in[-\tau,0]\\},$
$\mathcal{R}_{\bar{M}}=\\{x\in\mathcal{R}:x\in\text{Lip}_{\bar{M}}([-\tau,0];\mathbb{R}^{n})\\}.$

Our goal is to prove that the minimum time functional is locally Lipschitz
continuous in $\mathcal{R}_{\bar{M}}$, for any $\bar{M}\geq M$, where $M$ is
the positive constant in $(H_{1})$. To this aim, we need the following result
ensuring that the Petrov condition $(H_{4})$ implies an estimate of the
minimum time functional in terms of the distance from the target.

######  Proposition 3.2.

Assume $(H_{1})$, $(H_{2})$, $(H_{4})$ hold. Let $\bar{M}\geq M$ and assume
that $\displaystyle{\tau<\frac{\mu}{2\bar{M}L}}$. Then, there exist
$\delta,C>0$ such that

$T(x)\leq Cd_{\mathcal{K}}(x(0)),$ (3.8)

for every $x\in\mathcal{R}_{\bar{M}}$ with $x(0)\in\mathcal{K}_{\delta}$.

###### Proof.

From our assumptions, eventually taking a larger $M$, we may assume
$M\geq\mu$. Let $x_{0}\in\mathcal{R}_{\bar{M}}$ be such that
$x_{0}(0)\in\mathcal{K}_{\delta}$, where

$\delta=\min\left\\{\frac{M}{L},\sigma\right\\}.$

We can suppose that $x_{0}(0)\in\mathcal{K}_{\delta}\setminus\mathcal{K}$,
since otherwise inequality (3.8) is trivially satisfied for any positive
constant $C$.
Following [12], we define inductively a sequence $(x_{j})_{j}\subset
C([-\tau,0];\mathbb{R}^{n})$ such that
$x_{j}(0)\in\mathcal{K}_{\delta}\setminus\mathcal{K}$, for all $j\geq 0$, in
the following way. For all $j\geq 0$, we set

$x_{j+1}=y_{t_{j}}(\cdot;x_{j},u_{x_{j}}),$ (3.9)

where

$t_{j}=\frac{\mu-2\bar{M}L\tau}{4M^{2}}d_{\mathcal{K}}(x_{j}(0)),$ (3.10)

and $u_{x_{j}}$ is chosen according to $(H_{3})$ related to $x_{j}(0),$ i.e.

$f(x_{j}(0),u_{x_{j}})\cdot\frac{x_{j}(0)-\pi(x_{j}(0))}{\lvert
x_{j}(0)-\pi(x_{j}(0))\rvert}\leq-\mu.$

First of all, we have that $x_{j}\in\mathcal{R}_{\bar{M}}$, for all $j\geq 0$.
Indeed, by induction, if $j=0$ we know that $x_{0}\in\mathcal{R}_{\bar{M}}$.
Now, we assume that $x_{j}\in\mathcal{R}_{\bar{M}}$, for some $j\geq 0$, and
we show that also $x_{j+1}\in\mathcal{R}_{\bar{M}}$. For every
$s,t\in[-\tau,0]$, by definition of $x_{j+1}$, it holds

$\lvert x_{j+1}(s)-x_{j+1}(t)\rvert=\lvert
y(t_{j}+s;x_{j},u_{x_{j}})-y(t_{j}+t;x_{j},u_{x_{j}})\rvert.$

Then, if both $t_{j}+s,t_{j}+t<0$, by using the fact that
$x_{j}\in\mathcal{R}_{\bar{M}}$, we get

$\lvert x_{j+1}(s)-x_{j+1}(t)\rvert=\lvert
x_{j}(t_{j}+s)-x_{j}(t_{j}+t)\rvert\leq\bar{M}\lvert s-t\rvert.$

On the other hand, if both $t_{0}+s,t_{0}+t>0$, we can write

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert
x_{j+1}(s)-x_{j+1}(t)\rvert}\\\ \vspace{0.3cm}\displaystyle{\hskip
28.45274pt=\left\lvert
y(0;x_{j},u_{x_{j}})+\int_{0}^{t_{j}+s}y^{\prime}(r;x_{j},u_{x_{j}})dr-y(0;x_{j},u_{x_{j}})-\int_{0}^{t_{j}+t}y^{\prime}(r;x_{j},u_{x_{j}})dr\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip
28.45274pt\leq\int_{t_{j}+t}^{t_{j}+s}\lvert
y^{\prime}(r,x_{j},u_{x_{j}})\rvert dr=\int_{t_{j}+t}^{t_{j}+s}\lvert
f(y(r-\tau;x_{j},u_{x_{j}}),u_{x_{j}})\rvert dr}\\\ \displaystyle{\hskip
28.45274pt\leq M\lvert s-t\rvert\leq\bar{M}\lvert s-t\rvert.}\end{array}$

Finally, if $t_{j}+s>0$ and $t_{j}+t<0$ (or, analogously, if $t_{j}+s<0$ and
$t_{j}+t>0$), it turns out that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert
x_{j+1}(s)-x_{j+1}(t)\rvert=\left\lvert
y(0;x_{j},u_{x_{j}})+\int_{0}^{t_{j}+s}y^{\prime}(r;x_{j},u_{x_{j}})dr-
x_{j}(t_{j}+t)\right\rvert}\\\ \vspace{0.3cm}\displaystyle{\hskip
71.13188pt\leq\lvert x_{j}(0)-x_{j}(t_{j}+t)\rvert+\int_{0}^{t_{j}+s}\lvert
f(y(r-\tau;x_{j},u_{x_{j}}),u_{x_{j}})\rvert dr}\\\
\vspace{0.3cm}\displaystyle{\hskip 71.13188pt\leq\bar{M}\lvert
t_{j}+t\rvert+M(t_{j}+s)}\\\ \vspace{0.3cm}\displaystyle{\hskip
71.13188pt\leq\bar{M}\lvert t_{j}+t\rvert+\bar{M}(t_{j}+s)}\\\
\vspace{0.3cm}\displaystyle{\hskip 71.13188pt=\bar{M}(t_{j}+s-t_{j}-t)}\\\
\displaystyle{\hskip 71.13188pt=\bar{M}\lvert s-t\rvert.}\end{array}$

Therefore $x_{j+1}\in\mathcal{R}_{\bar{M}}$ as desidered.
Next, we claim that

$d_{\mathcal{K}}(x_{j+1}(0))\leq kd_{\mathcal{K}}(x_{j}(0)),\quad\forall j\geq
0,$ (3.11)

where

$k=\sqrt{1-\mu\frac{\mu-2\bar{M}L\tau}{4M^{2}}}\in[0,1).$ (3.12)

To this aim, for all $j\geq 0$, we define

$y_{j}(t)=y(t;x_{j},u_{x_{j}}),\quad\forall t\in[0,t_{j}].$ (3.13)

Given $j\geq 0$, for every $t\in[0,t_{j}]$, we have that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\frac{1}{2}\frac{d}{dt}\lvert
y_{j}(t)-\pi(x_{j}(0))\rvert^{2}=\langle
f(y_{j}(t-\tau),u_{x_{j}}),y_{j}(t)-\pi(x_{j}(0))\rangle}\\\
\vspace{0.3cm}\displaystyle{\hskip 71.13188pt=\langle
f(x_{j}(0),u_{x_{j}}),x_{j}(0)-\pi(x_{j}(0))\rangle}\\\
\vspace{0.3cm}\displaystyle{\hskip 79.6678pt+\langle
f(y_{j}(t-\tau),u_{x_{j}})-f(x_{j}(0),u_{x_{j}}),x_{j}(0)-\pi(x_{j}(0))\rangle}\\\
\displaystyle{\hskip 79.6678pt+\langle
f(y_{j}(t-\tau),u_{x_{j}}),y_{j}(t)-x_{j}(0)\rangle.}\end{array}$

Therefore, using $(H_{1})$, $(H_{3})$ and the Cauchy-Schwarz inequality, we
get

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\frac{1}{2}\frac{d}{dt}\lvert
y_{j}(t)-\pi(x_{j}(0))\rvert^{2}\leq-\mu
d_{\mathcal{K}}(x_{j}(0))+Ld_{\mathcal{K}}(x_{j}(0))\lvert
y_{j}(t-\tau)-x_{j}(0)\rvert}\\\ \displaystyle{\hskip 130.88284pt+M\lvert
y_{j}(t)-x_{j}(0)\rvert.}\end{array}$

We first estimate

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert
y_{j}(t)-x_{j}(0)\rvert=\lvert y(t;x_{j},u_{x_{j}})-x_{j}(0)\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 73.97733pt=\left\lvert
x_{j}(0)+\int_{0}^{t}y^{\prime}(s;x_{j},u_{x_{j}})ds-x_{j}(0)\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 73.97733pt\leq\int_{0}^{t}\lvert
f(y(s-\tau);x_{j},u_{x_{j}})\rvert ds}\\\ \displaystyle{\hskip 73.97733pt\leq
Mt.}\end{array}$

Thus,

$\frac{1}{2}\frac{d}{dt}\lvert y_{j}(t)-\pi(x_{j}(0))\rvert^{2}\leq-\mu
d_{\mathcal{K}}(x_{j}(0))+Ld_{\mathcal{K}}(x_{j}(0))\lvert
y_{j}(t-\tau)-x_{j}(0)\rvert+M^{2}t.$ (3.14)

Moreover,

$\lvert y_{j}(t-\tau)-x_{j}(0)\rvert=\lvert
y(t-\tau;x_{j},u_{x_{j}})-x_{j}(0)\rvert.$

We can distinguish two different situations. Assume first that $t-\tau\leq 0$.
Then, since $x_{j}\in\mathcal{R}_{\bar{M}}$, we have

$\lvert y_{j}(t-\tau)-x_{j}(0)\rvert=\lvert
x_{j}(t-\tau)-x_{j}(0)\rvert\leq\bar{M}\lvert
t-\tau\rvert=\bar{M}(\tau-t)\leq\bar{M}\tau.$

Therefore, (3.14) becomes

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\frac{1}{2}\frac{d}{dt}\lvert
y_{j}(t)-\pi(x_{j}(0))\rvert^{2}\leq-\mu
d_{\mathcal{K}}(x_{j}(0))+\bar{M}L\tau d_{\mathcal{K}}(x_{j}(0))+M^{2}t}\\\
\vspace{0.3cm}\displaystyle{\hskip 56.9055pt\leq-\mu
d_{\mathcal{K}}(x_{j}(0))+\bar{M}L\tau
d_{\mathcal{K}}(x_{j}(0))+M^{2}t_{j}}\\\ \vspace{0.3cm}\displaystyle{\hskip
56.9055pt=-\mu d_{\mathcal{K}}(x_{j}(0))+\bar{M}L\tau
d_{\mathcal{K}}(x_{j}(0))+M^{2}\frac{\mu-2\bar{M}L\tau}{4M^{2}}d_{\mathcal{K}}(x_{j}(0))}\\\
\vspace{0.3cm}\displaystyle{\hskip 56.9055pt\leq-\mu
d_{\mathcal{K}}(x_{j}(0))+\bar{M}L\tau
d_{\mathcal{K}}(x_{j}(0))+M^{2}\frac{\mu-2\bar{M}L\tau}{2M^{2}}d_{\mathcal{K}}(x_{j}(0))}\\\
\vspace{0.3cm}\displaystyle{\hskip 56.9055pt=-\mu
d_{\mathcal{K}}(x_{j}(0))+\bar{M}L\tau
d_{\mathcal{K}}(x_{j}(0))+\frac{\mu}{2}d_{\mathcal{K}}(x_{j}(0))-\bar{M}L\tau
d_{\mathcal{K}}(x_{j}(0))}\\\ \displaystyle{\hskip
56.9055pt=-\frac{\mu}{2}d_{\mathcal{K}}(x_{j}(0)).}\end{array}$

On the other hand, assume that $t-\tau>0$. Then,

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert
y_{j}(t-\tau)-x_{j}(0)\rvert=\left\lvert
x_{j}(0)+\int_{0}^{t}y^{\prime}(s;x_{j},u_{x_{j}})ds-x_{j}(0)\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 96.73918pt\leq\int_{0}^{t}\lvert
f(y(s-\tau;x_{j},u_{x_{j}}),u_{x_{j}})\rvert ds}\\\ \displaystyle{\hskip
96.73918pt\leq M(t-\tau)\leq Mt.}\end{array}$

Using this and the fact that $x_{j}(0)\in\mathcal{K}_{\delta}$ with
$\delta\leq\frac{M}{L}$ in (3.14), we get

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\frac{1}{2}\frac{d}{dt}\lvert
y_{j}(t)-\pi(x_{j}(0))\rvert^{2}\leq-\mu
d_{\mathcal{K}}(x_{j}(0))+LMtd_{\mathcal{K}}(x_{j}(0))+M^{2}t}\\\
\vspace{0.3cm}\displaystyle{\hskip 56.9055pt\leq-\mu
d_{\mathcal{K}}(x_{j}(0))+M^{2}t+M^{2}t=-\mu
d_{\mathcal{K}}(x_{j}(0))+2M^{2}t}\\\ \vspace{0.3cm}\displaystyle{\hskip
56.9055pt\leq-\mu d_{\mathcal{K}}(x_{j}(0))+2M^{2}t_{j}}\\\
\vspace{0.3cm}\displaystyle{\hskip 56.9055pt\leq-\mu
d_{\mathcal{K}}(x_{j}(0))+2M^{2}\frac{\mu}{4M^{2}}d_{\mathcal{K}}(x_{j}(0))}\\\
\displaystyle{\hskip
56.9055pt=-\frac{\mu}{2}d_{\mathcal{K}}(x_{j}(0)).}\end{array}$

We have so proven that

$\frac{1}{2}\frac{d}{dt}\lvert
y_{j}(t)-\pi(x_{j}(0))\rvert^{2}\leq-\frac{\mu}{2}d_{\mathcal{K}}(x_{j}(0)),\quad\forall
t\in[0,t_{j}].$ (3.15)

Thus, integrating (3.15) over $(0,t)$, we have that

$\begin{split}\lvert y_{j}(t)-\pi(x_{j}(0))\rvert^{2}&\leq-\mu
d_{\mathcal{K}}(x_{j}(0))t+\lvert y_{j}(0)-\pi(x_{j}(0))\rvert^{2}\\\ &=-\mu
d_{\mathcal{K}}(x_{j}(0))t+\lvert x_{j}(0)-\pi(x_{j}(0))\rvert^{2}\\\ &=-\mu
d_{\mathcal{K}}(x_{j}(0))t+d^{2}_{\mathcal{K}}(x_{j}(0)),\end{split}$ (3.16)

for all $t\in[0,t_{j}]$. In particular, for $t=t_{j}$, (3.16) yields

$\lvert y_{j}(t_{j})-\pi(x_{j}(0))\rvert^{2}\leq-\mu
d_{\mathcal{K}}(x_{j}(0))t_{j}+d^{2}_{\mathcal{K}}(x_{j}(0)).$

As a consequence, we have

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{d^{2}_{\mathcal{K}}(x_{j+1}(0))=d^{2}_{\mathcal{K}}(y_{j}(t_{j}))}\\\
\vspace{0.3cm}\displaystyle{\hskip 62.59596pt\leq\lvert
y_{j}(t_{j})-\pi(x_{j}(0))\rvert^{2}}\\\ \vspace{0.3cm}\displaystyle{\hskip
62.59596pt\leq-\mu
d_{\mathcal{K}}(x_{j}(0))t_{j}+d^{2}_{\mathcal{K}}(x_{j}(0))}\\\
\vspace{0.3cm}\displaystyle{\hskip 62.59596pt=-\mu
d_{\mathcal{K}}(x_{j}(0))\frac{\mu-2\bar{M}L\tau}{4M^{2}}d_{\mathcal{K}}(x_{j}(0))+d^{2}_{\mathcal{K}}(x_{j}(0))}\\\
\displaystyle{\hskip
62.59596pt=\left(1-\mu\,\frac{\mu-2\bar{M}L\tau}{4M^{2}}\right)d_{\mathcal{K}}^{2}(x_{j}(0)),}\end{array}$

from which follows that

$d_{\mathcal{K}}(x_{j+1}(0))\leq\sqrt{1-\mu\,\frac{\mu-2\bar{M}L\tau}{4M^{2}}}d_{\mathcal{K}}(x_{j}(0))=kd_{\mathcal{K}}(x_{j}(0)),$

i.e. (3.11) is fulfilled.
Now, thanks to an induction argument, (3.11) implies that

$d_{\mathcal{K}}(x_{j}(0))\leq k^{j}d_{\mathcal{K}}(x_{0}(0)),$ (3.17)

for all $j\geq 0$. Hence, using (3.17), it turns out that

$d_{\mathcal{K}}(x_{j}(0))\to 0\quad\text{as}\,\,j\to\infty.$ (3.18)

Furthermore, from (3.10), (3.12) and (3.17), we have that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\sum_{j=0}^{\infty}t_{j}=\frac{\mu-2\bar{M}L\tau}{4M^{2}}\sum_{j=0}^{\infty}d_{\mathcal{K}}(x_{j}(0))\leq\frac{\mu-2\bar{M}L\tau}{4M^{2}}d_{\mathcal{K}}(x_{0}(0))\sum_{j=0}^{\infty}k^{j}}\\\
\displaystyle{\hskip
85.35826pt=\frac{\mu-2\bar{M}L\tau}{4M^{2}}d_{\mathcal{K}}(x_{0}(0))\frac{1}{1-k}.}\end{array}$

So, setting

$C=\frac{\mu-2\bar{M}L\tau}{4M^{2}}\frac{1}{1-k},$

it holds

$\sum_{j=0}^{\infty}t_{j}\leq Cd_{\mathcal{K}}(x_{0}(0)).$ (3.19)

Now, let us define the control $\bar{u}:[0,+\infty[\rightarrow U$ as follows

$\bar{u}(t)=\begin{cases}u_{x_{h}},\quad\quad\quad\hskip
11.38092pt\text{if}\,\sum_{j=0}^{h-1}t_{j}\leq
t<\sum_{j=0}^{h}t_{j},\,\text{for\, some}\,h\geq 0,\\\
\text{arbitrary},\,\,\,\quad\text{if}\,\,t\geq\sum_{j=0}^{\infty}t_{j}.\end{cases}$

We can note that, for every $j\geq 0$,

$y\left(\sum_{h=0}^{j}t_{j};x_{0},\bar{u}\right)=y\left(\sum_{h=0}^{j}t_{j};x_{0},u_{x_{j+1}}\right)=x_{j+1}(0).$

Therefore, using (3.18) and (3.19), we can conclude that

$T(x_{0})\leq\theta(x_{0},\bar{u})=\sum_{h=0}^{\infty}t_{j}\leq
Cd_{\mathcal{K}}(x_{0}(0)).$

∎

Now, we are able to prove that the minimum time functional is locally
Lipschitz continuous in $\mathcal{R}_{\bar{M}},$ for all $\bar{M}\geq M.$

######  Theorem 3.3.

Assume that $(H_{1})$, $(H_{2})$, $(H_{4})$ hold. Let $\bar{M}\geq M$ and
assume $\displaystyle{\tau<\frac{\mu}{2\bar{M}L}}$. Then, the minimum time
functional $T$ is locally Lipschitz continuous in $\mathcal{R}_{\bar{M}}$.

###### Proof.

Fix $x_{0}\in\mathcal{R}_{\bar{M}}$. Let $x$ be any function in
$\mathcal{R}_{\bar{M}}$ such that

$\lVert x-x_{0}\rVert_{\infty}<\delta\frac{e^{-L(T(x_{0})+1)}}{(1+L\tau)}.$
(3.20)

We claim that

$\lvert T(x)-T(x_{0})\rvert\leq C(1+L\tau)e^{LT(x_{0})}\lVert
x-\tilde{x}\rVert_{\infty},$ (3.21)

where $C>0$ is the positive constant in (3.8).
We first assume that $T(x)>T(x_{0})$. Then, for $\epsilon>0$ small enough, we
have that $T(x)\geq T(x_{0})+\epsilon$. By definition of $T(x_{0})$, in
correspondence of $\epsilon$, there exists a control $u_{\epsilon}$ such that

$\theta_{\epsilon}:=\theta(x_{0},u_{\epsilon})<T(x_{0})+\epsilon.$ (3.22)

Therefore, $T(x)\geq\theta_{\epsilon}$. Moreover, using $(H_{1})$, for every
$t\in[0,\theta_{\epsilon}]$, we get

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert
y(t;x,u_{\epsilon})-y(t;x_{0},u_{\epsilon})\rvert=\left\lvert
y(0;x,u_{\epsilon})+\int_{0}^{t}y^{\prime}(s;x,u_{\epsilon})ds-y(0;x_{0},u_{\epsilon})-\int_{0}^{t}y^{\prime}(s;x_{0},u_{\epsilon})ds\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 122.34692pt\leq\lvert
x(0)-x_{0}(0)\rvert+\int_{0}^{t}\lvert
y^{\prime}(s;x,u_{\epsilon})-y^{\prime}(s;x_{0},u_{\epsilon})\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip 122.34692pt\leq\lVert
x-x_{0}\rVert_{\infty}+\int_{0}^{t}\lvert
f(y(s-\tau;x,u_{\epsilon}),u_{\epsilon})-f(y(s-\tau;x_{0},u_{\epsilon}),u_{\epsilon})\rvert
ds}\\\ \displaystyle{\hskip 122.34692pt\leq\lVert
x-x_{0}\rVert_{\infty}+L\int_{0}^{t}\lvert
y(s-\tau;x,u_{\epsilon})-y(s-\tau;x_{0},u_{\epsilon})\rvert ds.}\end{array}$

Now, we distinguish two situations. Assume first that $t\leq\tau$. Then, for
all $s\in[0,t]$, it holds that $s-\tau\in[-\tau,0]$ and

$\lvert y(s-\tau;x,u_{\epsilon})-y(s-\tau;x_{0},u_{\epsilon})\rvert=\lvert
x(s-\tau)-x_{0}(s-\tau)\rvert\leq\lVert x-x_{0}\rVert_{\infty}.$

Thus,

$\lvert y(t;x,u_{\epsilon})-y(t;x_{0},u_{\epsilon})\rvert\leq\lVert
x-x_{0}\rVert_{\infty}+L\lVert x-x_{0}\rVert_{\infty}t=(1+L\tau)\lVert
x-x_{0}\rVert_{\infty}.$

On the contrary, assume that $t>\tau$. Then, using a change of variable, we
can write

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\int_{0}^{t}\lvert
y(s-\tau;x,u_{\epsilon})-y(s-\tau;x_{0},u_{\epsilon})\rvert
ds=\int_{-\tau}^{t-\tau}\lvert
y(s;x,u_{\epsilon})-y(s;x_{0},u_{\epsilon})\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip 39.83368pt=\int_{-\tau}^{0}\lvert
y(s;x,u_{\epsilon})-y(s;x_{0},u_{\epsilon})\rvert ds+\int_{0}^{t-\tau}\lvert
y(s;x,u_{\epsilon})-y(s;x_{0},u_{\epsilon})\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip 39.83368pt=\int_{-\tau}^{0}\lvert
x(s)-x_{0}(s)\rvert ds+\int_{0}^{t-\tau}\lvert
y(s;x,u_{\epsilon})-y(s;x_{0},u_{\epsilon})\rvert ds}\\\ \displaystyle{\hskip
39.83368pt\leq\lVert x-x_{0}\rVert_{\infty}\tau+\int_{0}^{t}\lvert
y(s;x,u_{\epsilon})-y(s;x_{0},u_{\epsilon})\rvert ds.}\end{array}$

As a consequence, it comes that, for all $t\in[0,\theta_{\epsilon}],$

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert
y(t;x,u_{\epsilon})-y(t;x_{0},u_{\epsilon})\rvert\leq\lVert
x-x_{0}\rVert_{\infty}}\\\ \vspace{0.3cm}\displaystyle{\hskip
150.79968pt+L\tau\lVert x-x_{0}\rVert_{\infty}+L\int_{0}^{t}\lvert
y(s;x,u_{\epsilon})-y(s;x_{0},u_{\epsilon})\rvert ds}\\\ \displaystyle{\hskip
136.5733pt\leq(1+L\tau)\lVert x-x_{0}\rVert_{\infty}+L\int_{0}^{t}\lvert
y(s;x,u_{\epsilon})-y(s;x_{0},u_{\epsilon})\rvert ds.}\end{array}$

Therefore, the Gronwall’s inequality yields

$\lvert
y(t;x,u_{\epsilon})-y(t;x_{0},u_{\epsilon})\rvert\leq(1+L\tau)e^{Lt}\lVert
x-x_{0}\rVert_{\infty},\quad\forall t\in[0,\theta_{\epsilon}].$ (3.23)

In particular, for $t=\theta_{\epsilon}$, we have

$\lvert
y(t;x,u_{\epsilon})-y(t;x_{0},u_{\epsilon})\rvert\leq(1+L\tau)e^{L\theta_{\epsilon}}\lVert
x-x_{0}\rVert_{\infty}.$

Thus, using (3.22), we get

$\lvert
y(\theta_{\epsilon};x,u_{\epsilon})-y(\theta_{\epsilon};x_{0},u_{\epsilon})\rvert\leq(1+L\tau)e^{L(T(x_{0})+\epsilon)}\lVert
x-x_{0}\rVert_{\infty}.$ (3.24)

Using the dynamic programming principle (Proposition 3.1), it turns out that

$T(x)\leq\theta_{\epsilon}+T(y_{\theta_{\epsilon}}(\cdot;x,u_{\epsilon})).$
(3.25)

Let us note that
$y_{\theta_{\epsilon}}(\cdot;x,u_{\epsilon})\in\mathcal{R}_{\bar{M}}$. Indeed,
let $s,t\in[-\tau,0]$. We can assume without loss of generality that $s\geq
t$. Then, if both $\theta_{\epsilon}+s$, $\theta_{\epsilon}+t\leq 0$, using
the fact that $x\in\mathcal{R}_{\bar{M}}$, we have that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert
y_{\theta_{\epsilon}}(s;x,u_{\epsilon})-y_{\theta_{\epsilon}}(t;x,u_{\epsilon})\rvert=\lvert
y(\theta_{\epsilon}+s;x,u_{\epsilon})-y(\theta_{\epsilon}+t;x,u_{\epsilon})\rvert}\\\
\displaystyle{\hskip 56.9055pt=\lvert
x(\theta_{\epsilon}+s)-x(\theta_{\epsilon}+t)\rvert\leq\bar{M}\lvert
s-t\rvert.}\end{array}$

On the other hand, if both $\theta_{\epsilon}+s,\theta_{\epsilon}+t>0$, from
$(H_{1})$ we can write

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert
y_{\theta_{\epsilon}}(s;x,u_{\epsilon})-y_{\theta_{\epsilon}}(t;x,u_{\epsilon})\rvert=\lvert
y(\theta_{\epsilon}+s;x,u_{\epsilon})-y(\theta_{\epsilon}+t;x,u_{\epsilon})\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 136.5733pt=\left\lvert
x(0)+\int_{0}^{\theta_{\epsilon}+s}y^{\prime}(r;x,u_{\epsilon})dr-x(0)-\int_{0}^{\theta_{\epsilon}+t}y^{\prime}(r;x,u_{\epsilon})dr\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip
136.5733pt\leq\int_{\theta_{\epsilon}+t}^{\theta_{\epsilon}+s}\lvert
y^{\prime}(r;x,u_{\epsilon})\rvert dr}\\\ \vspace{0.3cm}\displaystyle{\hskip
136.5733pt=\int_{\theta_{\epsilon}+t}^{\theta_{\epsilon}+s}\lvert
f(y(r-\tau;x,u_{\epsilon}),u_{\epsilon})\rvert dr}\\\ \displaystyle{\hskip
136.5733pt\leq M(s-t)\leq\bar{M}(s-t)=\bar{M}\lvert s-t\rvert.}\end{array}$

Finally, if $\theta_{\epsilon}+s>0$ and $\theta_{\epsilon}+t\leq 0$, using
$(H_{1})$ and the fact that $x\in\mathcal{R}_{\bar{M}}$, we have

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert
y_{\theta_{\epsilon}}(s;x,u_{\epsilon})-y_{\theta_{\epsilon}}(t;x,u_{\epsilon})\rvert=\lvert
y(\theta_{\epsilon}+s;x,u_{\epsilon})-y(\theta_{\epsilon}+t;x,u_{\epsilon})\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 136.5733pt=\left\lvert
x(0)+\int_{0}^{\theta_{\epsilon}+s}y^{\prime}(r;x,u_{\epsilon})dr-x(\theta_{\epsilon}+t)\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 136.5733pt\leq\lvert
x(0)-x(\theta_{\epsilon}+t)\rvert+\int_{0}^{\theta_{\epsilon}+s}\lvert
f(y(r-\tau;x,u_{\epsilon}),u_{\epsilon})\rvert dr}\\\
\vspace{0.3cm}\displaystyle{\hskip
136.5733pt\leq\bar{M}(-\theta_{\epsilon}-t)+M(\theta_{\epsilon}+s)}\\\
\displaystyle{\hskip
136.5733pt\leq\bar{M}(-\theta_{\epsilon}-t+\theta_{\epsilon}+s)=\bar{M}\lvert
s-t\rvert.}\end{array}$

Thus, $y_{\theta_{\epsilon}}(\cdot;x,u_{\epsilon})\in\mathcal{R}_{\bar{M}}$ as
claimed before. In addition, since $\epsilon$ is small, inequalities (3.20)
and (3.24) imply that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{d_{\mathcal{K}}(y_{\theta_{\epsilon}}(0;x,u_{\epsilon}))=d_{\mathcal{K}}(y(\theta_{\epsilon};x,u_{\epsilon}))\leq\lvert
y(\theta_{\epsilon};x,u_{\epsilon})-y(\theta_{\epsilon};x_{0},u_{\epsilon})\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip
173.56198pt\leq(1+L\tau)e^{L(T(x_{0})+\epsilon)}\lVert
x-x_{0}\rVert_{\infty}}\\\ \vspace{0.3cm}\displaystyle{\hskip
173.56198pt\leq(1+L\tau)e^{L(T(x_{0})+1)}\lVert x-x_{0}\rVert_{\infty}}\\\
\displaystyle{\hskip
173.56198pt<(1+L\tau)e^{L(T(x_{0})+1)}\delta\frac{e^{-L(T(x_{0})+1)}}{1+L\tau}=\delta,}\end{array}$

i.e. $y_{\theta_{\epsilon}}(0;x,u_{\epsilon})\in\mathcal{K}_{\delta}$. As a
consequence, (3.8) yields

$T(y_{\theta}(\cdot;x,u_{\epsilon}))\leq
Cd_{\mathcal{K}}(y_{\theta}(0;x,u_{\epsilon}))=Cd_{\mathcal{K}}(y(\theta_{\epsilon};x,u_{\epsilon})).$
(3.26)

Therefore, combining (3.22), (3.24) (3.25) and (3.26), we can conclude that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{T(x)\leq\theta_{\epsilon}+Cd_{\mathcal{K}}(y(\theta_{\epsilon};x,u_{\epsilon}))}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt\leq
T(x_{0})+\epsilon+Cd_{\mathcal{K}}(y(\theta_{\epsilon};x,u_{\epsilon}))}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt\leq T(x_{0})+\epsilon+C\lvert
y(\theta_{\epsilon};x,u_{\epsilon})-y(\theta_{\epsilon};x_{0},u_{\epsilon})\rvert}\\\
\displaystyle{\hskip 28.45274pt\leq
T(x_{0})+\epsilon+C(1+L\tau)e^{L(T(x_{0})+\epsilon)}\lVert
x-x_{0}\rVert_{\infty}.}\end{array}$

Letting $\epsilon\to 0$, we finally get

$T(x)-T(x_{0})\leq C(1+L\tau)e^{LT(x_{0})}\lVert x-x_{0}\rVert_{\infty},$

i.e. (3.21) is fulfilled.
Now, we assume that $T(x_{0})>T(x)$. In this case, we exchange the roles of
$x_{0}$ and $x$ in the above arguments. To be precise, for $\epsilon>0$ small
enough, we have that $T(x_{0})\geq T(x)+\epsilon$. In addition, in
correspondence of $\epsilon$, by definition of the value function, there
exists a control $u_{\epsilon}$ such that

$\theta_{\epsilon}:=\theta(x,u_{\epsilon})\leq T(x)+\epsilon.$

Hence, $T(x_{0})\geq\theta_{\epsilon}$. In addition, arguing exactly as we did
in the previous case, (3.23) holds. Then, for $t=\theta_{\epsilon}$, (3.23)
yields

$\lvert
y(t;x,u_{\epsilon})-y(t;x_{0},u_{\epsilon})\rvert\leq(1+L\tau)e^{L\theta_{\epsilon}}\lVert
x-x_{0}\rVert_{\infty}\leq(1+L\tau)e^{LT(x_{0})}\lVert
x-x_{0}\rVert_{\infty}.$ (3.27)

Also, by using the dynamic programming principle (Proposition 3.1), we have
that

$T(x_{0})\leq\theta_{\epsilon}+T(y_{\theta_{\epsilon}}(\cdot;x_{0},u_{\epsilon})).$
(3.28)

Therefore, since
$y_{\theta_{\epsilon}}(\cdot;x_{0},u_{\epsilon})\in\mathcal{R}_{\bar{M}}$ from
the fact that $x_{0}\in\mathcal{R}_{\bar{M}}$ and since
$y(\theta_{\epsilon};x_{0},u_{\epsilon})\in\mathcal{K}_{\delta}$ from (3.20)
and (3.27), estimate (3.8) implies that

$T(y_{\theta_{\epsilon}}(\cdot;x_{0},u_{\epsilon}))\leq
Cd_{\mathcal{K}}(y(\theta_{\epsilon};x_{0},u_{\epsilon})).$

Finally, combining this last fact together with (3.27) (3.28), we get

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{T(x_{0})\leq\theta_{\epsilon}+Cd_{\mathcal{K}}(y(\theta_{\epsilon};x_{0},u_{\epsilon}))}\\\
\vspace{0.3cm}\displaystyle{\hskip 34.14322pt\leq
T(x)+\epsilon+Cd_{\mathcal{K}}(y(\theta_{\epsilon};x_{0},u_{\epsilon}))}\\\
\vspace{0.3cm}\displaystyle{\hskip 34.14322pt\leq T(x)+\epsilon+\lvert
y(\theta_{\epsilon};x_{0},u_{\epsilon}))-y(\theta_{\epsilon};x,u_{\epsilon}))\rvert}\\\
\displaystyle{\hskip 34.14322pt\leq
T(x)+\epsilon+C(1+L\tau)e^{LT(x_{0})}\lVert
x-x_{0}\rVert_{\infty}.}\end{array}$

Letting $\epsilon\to 0$, we can conclude that

$T(x_{0})-T(x)\leq C(1+L\tau)e^{LT(x_{0})}\lVert x-x_{0}\rVert_{\infty},$

i.e. (3.21) holds also in this case.
Now, for every $x\in\mathcal{R}_{\bar{M}}$ satisfying (3.20), from (3.21) it
holds

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{T(x)\leq
T(x_{0})+C(1+L\tau)e^{LT(x_{0})}\lVert x-x_{0}\rVert_{\infty}}\\\
\vspace{0.3cm}\displaystyle{\hskip
28.45274pt<T(x_{0})+C(1+L\tau)e^{LT(x_{0})}\delta\frac{e^{-L(T(x_{0})+1)}}{1+L\tau}}\\\
\displaystyle{\hskip 28.45274pt=T(x_{0})+Ce^{-L}.}\end{array}$

So, setting $\bar{c}:=T(x_{0})+Ce^{-L},$ we have that

$\lvert T(x)\rvert\leq\bar{c},\quad\forall x\in
B_{C([-\tau,0];\mathbb{R}^{n})}\Big{(}x_{0},\delta\frac{e^{-L(T(x_{0})+1)}}{1+L\tau}\Big{)}\cap\mathcal{R}_{\bar{M}},$
(3.29)

where
$B_{C([-\tau,0];\mathbb{R}^{n})}\left(x_{0},\delta\frac{e^{-L(T(x_{0})+1)}}{1+L\tau}\right)$
is the open ball in the Banach space $C([-\tau,0];\mathbb{R}^{n})$ centered at
$x_{0}$ with radius $\delta\frac{e^{-L(T(x_{0})+1)}}{1+L\tau}$. We claim that

$\lvert T(\bar{x})-T(\tilde{x})\rvert\leq
c\lVert\bar{x}-\tilde{x}\rVert_{\infty},\quad\forall\bar{x},\tilde{x}\in
B_{C([-\tau,0];\mathbb{R}^{n})}\Big{(}x_{0},\delta\frac{e^{-L(\bar{c}+1)}}{2(1+L\tau)}\Big{)}\cap\mathcal{R}_{\bar{M}}.$
(3.30)

for some positive constant $c$.
To this aim, let $\bar{x},\tilde{x}\in
B_{C([-\tau,0];\mathbb{R}^{n})}\left(x_{0},\delta\frac{e^{-L(\bar{c}+1)}}{2(1+L\tau)}\right)\cap\mathcal{R}_{\bar{M}}$.
We first assume that $T(\bar{x})>T(\tilde{x})$. Then, for $\epsilon>0$ small,
$T(\bar{x})\geq T(\tilde{x})+\epsilon$. Also, in correspondence of $\epsilon$,
by definition of the value function, there exists a control $u_{\epsilon}$
such that

$\theta_{\epsilon}:=\theta(\tilde{x},u_{\epsilon})<T(\tilde{x})+\epsilon.$
(3.31)

With the same arguments employed for the proof of (3.23), it holds

$\lvert
y(t;\bar{x},u_{\epsilon})-y(t;\tilde{x},u_{\epsilon})\rvert\leq(1+L\tau)e^{Lt}\lVert\bar{x}-\tilde{x}\rVert_{\infty}.$
(3.32)

In addition, the dynamic programming principle (Proposition 3.1) yields

$T(\bar{x})\leq\theta_{\epsilon}+T(y_{\theta_{\epsilon}}(\cdot;\bar{x},u_{\epsilon})).$
(3.33)

Let us note that, from (3.29) and (3.32) it comes that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{d_{\mathcal{K}}(y_{\theta_{\epsilon}}(0;\bar{x},u_{\epsilon}))=d_{\mathcal{K}}(y(\theta_{\epsilon};\bar{x},u_{\epsilon}))\leq\lvert
y(\theta_{\epsilon};\bar{x},u_{\epsilon})-y(\theta_{\epsilon};\tilde{x},u_{\epsilon})\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip
173.56198pt\leq(1+L\tau)e^{L\theta_{\epsilon}}\lVert\bar{x}-\tilde{x}\rVert_{\infty}}\\\
\vspace{0.3cm}\displaystyle{\hskip
173.56198pt\leq(1+L\tau)e^{L(T(\tilde{x})+\epsilon)}\lVert\bar{x}-\tilde{x}\rVert_{\infty}}\\\
\displaystyle{\hskip
173.56198pt\leq(1+L\tau)e^{L(\bar{c}+1)}\lVert\bar{x}-\tilde{x}\rVert_{\infty}.}\end{array}$

Thus, since

$\lVert\bar{x}-\tilde{x}\rVert_{\infty}\leq\lVert\bar{x}-x_{0}\rVert_{\infty}+\lVert
x_{0}-\tilde{x}\rVert_{\infty}<\delta\frac{e^{L(\bar{c}+1)}}{2(1+L\tau)}+\delta\frac{e^{L(\bar{c}+1)}}{2(1+L\tau)}=\delta\frac{e^{-L(\bar{c}+1)}}{1+L\tau},$

we get

$d_{\mathcal{K}}(y_{\theta_{\epsilon}}(0;\bar{x},u_{\epsilon}))=d_{\mathcal{K}}(y(\theta_{\epsilon};\bar{x},u_{\epsilon}))<(1+L\tau)e^{L(\bar{c}+1)}\delta\frac{e^{-L(\bar{c}+1)}}{1+L\tau}=\delta,$

i.e. $y_{\theta_{\epsilon}}(0,\bar{x},u_{\epsilon})\in\mathcal{K}_{\delta}$.
Therefore, being
$y_{\theta_{\epsilon}}(\cdot,\bar{x},u_{\epsilon})\in\mathcal{R}_{\bar{M}}$
from the fact that $\bar{x}\in\mathcal{R}_{\bar{M}}$, (3.8) yields

$T(y_{\theta_{\epsilon}}(\cdot,\bar{x},u_{\epsilon}))\leq
Cd_{\mathcal{K}}(y_{\theta_{\epsilon}}(0;\bar{x},u_{\epsilon})).$

Finally, combining this last fact with (3.29), (3.31), (3.32), we can write

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{T(\bar{x})\leq
T(\tilde{x})+\epsilon+T(y_{\theta_{\epsilon}}(\cdot,\bar{x},u_{\epsilon}))}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt\leq
T(\tilde{x})+\epsilon+Cd_{\mathcal{K}}(y_{\theta_{\epsilon}}(0;\bar{x},u_{\epsilon}))}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt\leq
T(\tilde{x})+\epsilon+C\lvert
y(\theta_{\epsilon};\bar{x},u_{\epsilon})-y(\theta_{\epsilon};\tilde{x},u_{\epsilon})\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt\leq
T(\tilde{x})+\epsilon+C(1+L\tau)e^{L\theta_{\epsilon}}\lVert\bar{x}-\tilde{x}\rVert_{\infty}}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt\leq
T(\tilde{x})+\epsilon+C(1+L\tau)e^{L(T(\tilde{x})+\epsilon))}\lVert\bar{x}-\tilde{x}\rVert_{\infty}}\\\
\displaystyle{\hskip 28.45274pt\leq
T(\tilde{x})+\epsilon+C(1+L\tau)e^{L(\bar{c}+\epsilon))}\lVert\bar{x}-\tilde{x}\rVert_{\infty}.}\end{array}$

Letting $\epsilon\to 0$, we get

$T(\bar{x})-T(\tilde{x})\leq
C(1+L\tau)e^{L\bar{c}}\lVert\bar{x}-\tilde{x}\rVert_{\infty}.$

So, setting $c:=C(1+L\tau)e^{L\bar{c}}$, we can conclude that

$T(\bar{x})-T(\tilde{x})\leq c\lVert\bar{x}-\tilde{x}\rVert_{\infty}.$

Now, if $T(\bar{x})<T(\tilde{x})$, exchanging the roles of
$\bar{x},\tilde{x}$, in the above arguments, it also holds that

$T(\tilde{x})-T(\bar{x})\leq\lVert\bar{x}-\tilde{x}\rVert_{\infty}.$

Thus, (3.30) is fulfilled and $T$ is Lipschitz continuous in a neighborhood of
$x_{0}$. From the arbitrariness of $x_{0}$, the minimum time functional is
locally Lipschitz continuous in $\mathcal{R}_{\bar{M}}$. ∎

######  Remark 3.1.

Note that the Lipschitz continuity of the minimum time function in
${\mathcal{R}}_{\bar{M}}$ holds under the smallness assumption on the time
delay $\displaystyle{\tau<\frac{\mu}{2\bar{M}L}}.$ To the aim of our analysis,
in the sequel, we will need the Lipschitz continuity of $T(\cdot)$ in
${\mathcal{R}}_{3M}.$ As a consequence, our semiconcavity result will be
proved under the following condition on the time delay size:
$\displaystyle{\tau<\frac{\mu}{6ML}}.$

##  4 Semiconcavity of the value function

In this section, we present our main result, which guarantees that the minimum
time functional $T$ is semiconcave in a suitable subset of the reachable set
$\mathcal{R}$. The semiconcavity result we will prove is the following.

######  Theorem 4.1.

Assume that $\tau<\frac{\mu}{6ML}$. Moreover, suppose that assumptions
$(H_{1})$, $(H_{2})$, $(H_{3})$, $(H_{4})$ are satisfied and that the target
$\mathcal{K}$ is such that

$d_{\mathcal{K}}\,\text{is semiconcave in
}\,\mathbb{R}^{n}\setminus\mathring{\mathcal{K}},$ (4.34)

where $\mathring{\mathcal{K}}$ denotes the interior of $\mathcal{K}$. Then,
the minimum time function $T$ is semiconcave in
$\mathcal{R}_{M}\setminus\mathcal{R}_{M}^{\mathcal{K}}$, where
$\mathcal{R}_{M}^{\mathcal{K}}=\\{x\in\mathcal{R}_{M}:x(0)\notin\mathcal{K}\\}$.

For the proof of Theorem 4.1, we need the following auxiliary lemma.

######  Lemma 4.2.

Let the hypotheses of Theorem 4.1 hold. Then, there exist $\rho>0$, $k>0$ such
that

$T(x+2h)-2T(x+h)\leq k\lVert h\rVert_{\infty}^{2},$

for every $x\in\mathcal{R}$ and $h\in C([-\tau,0];\mathbb{R}^{n})$ satisfying
the following conditions: $x(0)\in\partial\mathcal{K}$; $\lVert
h\rVert_{\infty}\leq\rho$; $(x+h)(0)\notin\mathcal{K}$;
$x,x+h\in\mathcal{R}_{M}$, $x+2h\in\mathcal{R}$.

###### Proof.

First of all, let us note that, from (4.34) and from the compactness of
$\mathcal{K}$, there exist two constants $r>0$, $c>0$ such that

$d_{\mathcal{K}}(z_{1})+d_{\mathcal{K}}(z_{2})-2d_{\mathcal{K}}\left(\frac{z_{1}+z_{2}}{2}\right)\leq
c\lvert z_{1}-z_{2}\rvert^{2},$ (4.35)

for every $z_{1},\,z_{2}$ satisfying
$z_{1},\,z_{2},\,\frac{z_{1}+z_{2}}{2}\in\mathcal{K}_{r}\setminus\mathring{\mathcal{K}}.$
Now, we pick $\rho>0$ such that

$2(1+2MC)\rho<\min\\{\delta,r\\},$ (4.36)

where $C$ and $\delta$ are the constants of Proposition 3.2 in the case
$\bar{M}=M.$ Let $x\in\mathcal{R}$ and $h\in C([-\tau,0];\mathbb{R}^{n})$
satisfying the following properties: $x(0)\in\partial\mathcal{K}$, $\lVert
h\rVert_{\infty}\leq\rho$, $(x+h)(0)\notin\mathcal{K}$,
$x,\,x+h\in\mathcal{R}_{M}$, $x+2h\in\mathcal{R}$.
Being $(x+h)(0)\notin\mathcal{K}$, we can thereby consider

$T(x+h)=\inf_{u}\theta(x+h,u).$

By definition of $T$, fixed $\epsilon\in(0,C\lVert h\rVert_{\infty})$, there
exists a control $u$ such that

$\theta^{*}:=\theta(x+h,u)<T(x+h)+\epsilon.$ (4.37)

Let us define

$\bar{u}(t)=u\left(\frac{t}{2}\right),\quad\tilde{y}(t)=y(t;x+h,u),\quad\bar{y}(t)=y(t;x+2h,\bar{u}).$

We claim that
$\tilde{y}(t),\,\bar{y}(t)\in\mathcal{K}_{\delta}\cap\mathcal{K}_{r}$, for
every $t\in[0,2\theta^{*}]$. Indeed, fix $t\in[0,2\theta^{*}]$. Then, since
$x(0)\in\partial\mathcal{K}$, we can write

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{d_{\mathcal{K}}(\tilde{y}(t))\leq\lvert\tilde{y}(t)-x(0)\rvert=\lvert
y(t;x+h,u)-x(0)\rvert}\\\ \vspace{0.3cm}\displaystyle{\hskip
45.52458pt=\left\lvert
y(0;x+h,u)+\int_{0}^{t}\tilde{y}^{\prime}(s)ds-x(0)\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 45.52458pt=\left\lvert
x(0)+h(0)+\int_{0}^{t}\tilde{y}^{\prime}(s)ds-x(0)\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 45.52458pt\leq\lvert
h(0)\rvert+\int_{0}^{t}\lvert\tilde{y}^{\prime}(s)\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip 45.52458pt\leq\lVert
h\rVert_{\infty}+\int_{0}^{t}\lvert f(\tilde{y}(s-\tau),u)\rvert ds}\\\
\displaystyle{\hskip 45.52458pt\leq\lVert h\rVert_{\infty}+Mt\leq\lVert
h\rVert_{\infty}+2M\theta^{*}.}\end{array}$

Now, using (4.37), we get

$2M\theta^{*}<2M(T(x+h)+\epsilon),$

from which, being $\epsilon\in(0,C\lVert h\rVert_{\infty})$,

$2M\theta^{*}<2M(T(x+h)+C\lVert h\rVert_{\infty}).$ (4.38)

In addition, we have that
$(x+h)(0)\in\mathcal{K}_{\delta}\setminus\mathcal{K}$. Indeed, by using (4.36)
and the fact that $x(0)\in\partial\mathcal{K}$, it turns out that

$d_{\mathcal{K}}((x+h)(0))\leq\lvert x(0)+h(0)-x(0)\rvert\leq\lVert
h\rVert_{\infty}\leq\rho<\delta.$

Thus, $x+h\in\mathcal{R}_{M}$ is such that
$(x+h)(0)\in\mathcal{K}_{\delta}\backslash\mathcal{K}$ and (3.8) yields

$T(x+h)\leq Cd_{\mathcal{K}}((x+h)(0))\leq C\lVert h\rVert_{\infty}.$

Therefore, (4.38) becomes

$2M\theta^{*}<2M(C\lVert h\rVert_{\infty}+C\lVert h\rVert_{\infty})=4MC\lVert
h\rVert_{\infty},$

and so

$\theta^{*}<2C\lVert h\rVert_{\infty}.$ (4.39)

As a consequence, using (4.36) and (4.39), we can write

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{d_{\mathcal{K}}(\tilde{y}(t))\leq\lVert
h\rVert_{\infty}+4MC\lVert h\rVert_{\infty}}\\\
\vspace{0.3cm}\displaystyle{\hskip 45.52458pt\leq 2\lVert
h\rVert_{\infty}+4MC\lVert h\rVert_{\infty}}\\\
\vspace{0.3cm}\displaystyle{\hskip 45.52458pt=2(1+2MC)\lVert
h\rVert_{\infty}}\\\ \displaystyle{\hskip 45.52458pt\leq
2(1+2MC)\rho<\min\\{\delta,r\\},}\end{array}$

i.e. $\tilde{y}(t)\in\mathcal{K}_{\delta}\cap\mathcal{K}_{r}$.
On the other hand, since $x(0)\in\partial\mathcal{K}$, we have that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{d_{\mathcal{K}}(\bar{y}(t))\leq\lvert\bar{y}(t)-x(0)\rvert=\lvert
y(t;x+2h,\bar{u})-x(0)\rvert}\\\ \vspace{0.3cm}\displaystyle{\hskip
45.52458pt=\left\lvert
y(0;x+2h,\bar{u})+\int_{0}^{t}\bar{y}^{\prime}(s)ds-x(0)\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 45.52458pt\leq 2\lvert
h(0)\rvert+\int_{0}^{t}\lvert f(\bar{y}(s-\tau),\bar{u})\rvert ds}\\\
\displaystyle{\hskip 45.52458pt\leq 2\lVert h\rVert_{\infty}+Mt\leq 2\lVert
h\rVert_{\infty}+2M\theta^{*}.}\end{array}$

Thus, using (4.39), we get

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{d_{\mathcal{K}}(\bar{y}(t))\leq
2\lVert h\rVert_{\infty}+4MC\lVert h\rVert_{\infty}=2(1+2MC)\lVert
h\rVert_{\infty}}\\\ \displaystyle{\hskip 56.9055pt\leq
2(1+2MC)\rho<\min\\{\delta,r\\},}\end{array}$

i.e. $\bar{y}(t)\in\mathcal{K}_{\delta}\cap\mathcal{K}_{r}$.
Next, we claim that

$T(x+2h)\leq 2(T(x+h)+\epsilon)+k\lVert h\rVert_{\infty}^{2}$ (4.40)

for a suitable constant $k>0$, indipendent of $\epsilon,\,x,\,h$. Let us note
that we can assume $2\theta^{*}<\theta(x+2h,\bar{u})$. Indeed, if this is not
the case, it holds

$T(x+2h)\leq\theta(x+2h,\bar{u})\leq 2\theta^{*}<2(T(x+h)+\epsilon)$

and (4.40) is satisfied for any positive constant $k$.
Therefore, using the dynamic programming principle, we can write

$T(x+2h)\leq 2\theta^{*}+T(\bar{y}_{2\theta^{*}}(\cdot)).$ (4.41)

Now, for every $t_{1}\in[0,\theta^{*}]$ and $t_{2}\in[0,2\theta^{*}]$, we
compute

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert\tilde{y}(t_{1})-\bar{y}(t_{2})\rvert=\lvert
y(t_{1};x+h,u)-y(t_{2};x+2h,\bar{u})\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt=\left\lvert
x(0)+h(0)+\int_{0}^{t_{1}}y^{\prime}(s;x+h,u)ds-x(0)-2h(0)-\int_{0}^{t_{2}}y^{\prime}(s;x+2h,\bar{u})ds\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt\leq\lvert
h(0)\rvert+\int_{0}^{t_{1}}\lvert f(\tilde{y}(s-\tau),u)\rvert
ds+\int_{0}^{t_{2}}\lvert f(\bar{y}(s-\tau),\bar{u})\rvert ds}\\\
\displaystyle{\hskip 28.45274pt\leq\lVert
h\rVert_{\infty}+Mt_{1}+Mt_{2}\leq\lVert
h\rVert_{\infty}+M\theta^{*}+2M\theta^{*}=\lVert
h\rVert_{\infty}+3M\theta^{*}.}\end{array}$

Therefore, since (4.39) implies that

$3M\theta^{*}<6MC\lVert h\rVert_{\infty},$

we can write

$\lvert\tilde{y}(t_{1})-\bar{y}(t_{2})\rvert\leq(1+6MC)\lVert
h\rVert_{\infty}.$ (4.42)

Similarly, it holds

$\lvert\tilde{y}(\theta^{*})-x(0)\rvert\leq{\|h\|_{\infty}}+M\theta^{*},$

from which, by using again (4.39), it comes that

$\lvert\tilde{y}(\theta^{*})-x(0)\rvert\leq(1+2MC)\lVert h\rVert_{\infty}.$
(4.43)

Also, for every $t\in[0,2\theta^{*}]$, we have

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert\bar{y}(t)-x(0)\rvert=\lvert
y(t;x+2h,\bar{u})-x(0)\rvert}\\\ \vspace{0.3cm}\displaystyle{\hskip
65.44142pt=\left\lvert
x(0)+2h(0)+\int_{0}^{t}\bar{y}^{\prime}(s)ds-x(0)\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 65.44142pt\leq 2\lVert
h\rVert_{\infty}+\int_{0}^{t}\lvert f(\bar{y}(s-\tau),\bar{u})\rvert}\\\
\displaystyle{\hskip 65.44142pt\leq 2\lVert h\rVert_{\infty}+Mt\leq 2\lVert
h\rVert_{\infty}+2M\theta^{*}.}\end{array}$

Thus, from (4.39) it follows that

$\lvert\bar{y}(t)-x(0)\rvert\leq 2\lVert h\rVert_{\infty}+4MC\lVert
h\rVert_{\infty}=2(1+2MC)\lVert h\rVert_{\infty}.$ (4.44)

Now, let $\tilde{x}$, $\bar{x}$ be the functions in
$C([-\tau,0];\mathbb{R}^{n})$ defined as follows

$\tilde{x}(\cdot)=\tilde{y}_{\theta^{*}}(\cdot),\quad\bar{x}(\cdot)=\bar{y}_{2\theta^{*}}(\cdot).$

Then,

$\tilde{x}(0)=\tilde{y}_{\theta^{*}}(0)=y(\theta^{*};x+h,u)=y(\theta(x+h,u);x+h,u)\in\partial\mathcal{K}.$

Moreover, it holds

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert\bar{x}(0)+x(0)-2\tilde{x}(0)\rvert=\lvert\bar{y}(2\theta^{*})+x(0)-2\tilde{y}(\theta^{*})\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt=\lvert
y(2\theta^{*};x+2h,\bar{u})+x(0)-2y(\theta^{*};x+h,u)\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt=\left\lvert
x(0)+2h(0)+\int_{0}^{2\theta^{*}}\bar{y}^{\prime}(s)ds+x(0)-2x(0)-2h(0)-2\int_{0}^{\theta^{*}}\tilde{y}^{\prime}(s)ds\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt=\left\lvert
2\int_{0}^{\theta^{*}}\bar{y}^{\prime}(2s)ds-2\int_{0}^{\theta^{*}}\tilde{y}^{\prime}(s)ds\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt\leq 2\int_{0}^{\theta^{*}}\lvert
f(\bar{y}(2s-\tau),\bar{u}(2s))-f(\tilde{y}(s{-\tau}),u(s))\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt=2\int_{0}^{\theta^{*}}\lvert
f(\bar{y}(2s-\tau),u(s))-f(\tilde{y}(s{-\tau}),u(s)\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt\leq
2L\int_{0}^{\theta^{*}}\lvert\bar{y}(2s-\tau)-\tilde{y}(s-\tau)\rvert ds}\\\
\displaystyle{\hskip
28.45274pt=2L\int_{-\tau}^{\theta^{*}-\tau}\lvert\bar{y}(2s+\tau)-\tilde{y}(s)\rvert
ds.}\end{array}$

Now, we distinguish two different situations. Assume first that
$2\theta^{*}\leq\tau$. Then, also $\theta^{*}\leq\tau$ and we can write

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert\bar{x}(0)+x(0)-2\tilde{x}(0)\rvert\leq
2L\int_{-\tau}^{\theta^{*}-\tau}\lvert x(2s+\tau)+2h(2s+\tau)-x(s)-h(s)\rvert
ds}\\\ \displaystyle{\hskip 28.45274pt\leq
2L\int_{-\tau}^{\theta^{*}-\tau}\lvert x(2s+\tau)-x(s)\rvert
ds+2L\int_{-\tau}^{\theta^{*}-\tau}\lvert 2h(2s+\tau)-h(s)\rvert
ds.}\end{array}$

Thus, since $x\in\mathcal{R}_{M}$, it comes that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert\bar{x}(0)+x(0)-2\tilde{x}(0)\rvert\leq
2LM\int_{-\tau}^{\theta^{*}-\tau}\lvert 2s+\tau-s\rvert ds+6L\lVert
h\rVert_{\infty}\theta^{*}}\\\ \vspace{0.3cm}\displaystyle{\hskip
56.9055pt=2LM\int_{-\tau}^{\theta^{*}-\tau}(s+\tau)ds+6L\lVert
h\rVert_{\infty}\theta^{*}}\\\ \displaystyle{\hskip
56.9055pt{=LM(\theta^{*})^{2}}+6L\lVert
h\rVert_{\infty}\theta^{*}.}\end{array}$

So, using (4.39), we can conclude that

$\lvert\bar{x}(0)+x(0)-2\tilde{x}(0)\rvert\leq{4}MLC^{2}\lVert
h\rVert_{\infty}^{2}+12LC\lVert h\rVert_{\infty}^{2}=4LC({MC}+3)\lVert
h\rVert_{\infty}^{2}.$ (4.45)

On the contrary, assume that $2\theta^{*}>\tau$. In this case, we have to
examine two further situations. Firstly, if $\theta^{*}\leq\tau$, using again
the fact that $x\in\mathcal{R}_{M}$, we have that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert\bar{x}(0)+x(0)-2\tilde{x}(0)\rvert\leq
2L\int_{-\tau}^{-\frac{\tau}{2}}\lvert\bar{y}(2s+\tau)-\tilde{y}(s)\rvert
ds+2L\int_{-\frac{\tau}{2}}^{\theta^{*}-\tau}\lvert\bar{y}(2s+\tau)-\tilde{y}(s)\rvert
ds}\\\ \vspace{0.3cm}\displaystyle{\hskip
28.45274pt=2L\int_{-\tau}^{-\frac{\tau}{2}}\lvert(x+2h)(2s+\tau)-(x+h)(s)\rvert
ds+2L\int_{-\frac{\tau}{2}}^{\theta^{*}-\tau}\lvert\bar{y}(2s+\tau)-(x+h)(s)\rvert
ds}\\\ \vspace{0.3cm}\displaystyle{\hskip 28.45274pt\leq
2L\int_{-\tau}^{-\frac{\tau}{2}}\lvert x(2s+\tau)-x(s)\rvert
ds+2L\int_{-\tau}^{-\frac{\tau}{2}}\lvert 2h(2s+\tau)-h(s)\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip
36.98866pt+2L\int_{-\frac{\tau}{2}}^{\theta^{*}-\tau}\lvert\bar{y}(2s+\tau)-x(0)\rvert
ds+2L\int_{-\frac{\tau}{2}}^{\theta^{*}-\tau}\lvert x(0)-x(s)-h(s)\rvert
ds,}\end{array}$

and so,

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert\bar{x}(0)+x(0)-2\tilde{x}(0)\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt\leq
2LM\int_{-\tau}^{-\frac{\tau}{2}}\lvert 2s+\tau-s\rvert ds+6L\lVert
h\rVert_{\infty}\frac{\tau}{2}+2L\int_{-\frac{\tau}{2}}^{\theta^{*}-\tau}\lvert\bar{y}(2s+\tau)-x(0)\rvert
ds}\\\ \vspace{0.3cm}\displaystyle{\hskip
36.98866pt+2L\int_{-\frac{\tau}{2}}^{\theta^{*}-\tau}\lvert x(0)-x(s)\rvert
ds+2L\int_{-\frac{\tau}{2}}^{\theta^{*}-\tau}\lvert h(s)\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip
28.45274pt=2LM\int_{-\tau}^{-\frac{\tau}{2}}(s+\tau)ds+6L\lVert
h\rVert_{\infty}\frac{\tau}{2}+2L\int_{-\frac{\tau}{2}}^{\theta^{*}-\tau}\lvert\bar{y}(2s+\tau)-x(0)\rvert
ds}\\\ \vspace{0.3cm}\displaystyle{\hskip
36.98866pt+2L\int_{-\frac{\tau}{2}}^{\theta^{*}-\tau}(-s)ds+2L\lVert
h\rVert_{\infty}\left(\theta^{*}-\frac{\tau}{2}\right)}\\\
\displaystyle{\hskip 28.45274pt\leq LM\left(\frac{\tau}{2}\right)^{2}+3L\lVert
h\rVert_{\infty}\tau+2L\int_{-\frac{\tau}{2}}^{\theta^{*}-\tau}\lvert\bar{y}(2s+\tau)-x(0)\rvert
ds+L\theta^{*}\tau+2L\lVert h\rVert_{\infty}\theta^{*}.}\end{array}$

Thus, using (4.39) and (4.44) and the fact that
$\frac{\tau}{2}\leq\theta^{*}$, it comes that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert\bar{x}(0)+x(0)-2\tilde{x}(0)\rvert\leq
2LM\left(\frac{\tau}{2}\right)^{2}+6L\lVert h\rVert_{\infty}\frac{\tau}{2}}\\\
\vspace{0.3cm}\displaystyle{\hskip 116.65646pt+4L(1+2MC)\lVert
h\rVert_{\infty}\left(\theta^{*}-\frac{\tau}{2}\right)+2L\theta^{*}\frac{\tau}{2}+2L\lVert
h\rVert_{\infty}\theta^{*}}\\\ \vspace{0.3cm}\displaystyle{\hskip
56.9055pt\leq 2LM\left(\frac{\tau}{2}\right)^{2}+6L\lVert
h\rVert_{\infty}\frac{\tau}{2}+4L(1+2MC)\lVert
h\rVert_{\infty}\theta^{*}+2L\theta^{*}\frac{\tau}{2}+2L\lVert
h\rVert_{\infty}\theta^{*}}\\\ \vspace{0.3cm}\displaystyle{\hskip
56.9055pt\leq 2LM\left(\theta^{*}\right)^{2}+6L\lVert
h\rVert_{\infty}\theta^{*}+4L(1+2MC)\lVert
h\rVert_{\infty}\theta^{*}+2L(\theta^{*})^{2}+2L\lVert
h\rVert_{\infty}\theta^{*}}\\\ \vspace{0.3cm}\displaystyle{\hskip
56.9055pt=2L(1+M)(\theta^{*})^{2}+8L(1+MC)\lVert
h\rVert_{\infty}\theta^{*}}\\\ \displaystyle{\hskip 56.9055pt\leq
8LC^{2}\lVert h\rVert_{\infty}^{2}+16LC(1+MC)\lVert
h\rVert_{\infty}^{2}=8LC(C+2(1+MC))\lVert h\rVert_{\infty}^{2}.}\end{array}$

Hence,

$\lvert\bar{x}(0)+x(0)-2\tilde{x}(0)\rvert\leq 8LC(C+2(1+MC))\lVert
h\rVert_{\infty}^{2}.$ (4.46)

Finally, assume that also $\theta^{*}>\tau$. Then,

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert\bar{x}(0)+x(0)-2\tilde{x}(0)\rvert\leq
2L\int_{-\tau}^{-\frac{\tau}{2}}\lvert\bar{y}(2s+\tau)-\tilde{y}(s)\rvert
ds+2L\int_{-\frac{\tau}{2}}^{0}\lvert\bar{y}(2s+\tau)-\tilde{y}(s)\rvert
ds}\\\ \vspace{0.3cm}\displaystyle{\hskip
119.50148pt+2L\int_{0}^{\theta^{*}-\tau}\lvert\bar{y}(2s+\tau)-\tilde{y}(s)\rvert
ds}\\\ \vspace{0.3cm}\displaystyle{\hskip
56.9055pt=2L\int_{-\tau}^{-\frac{\tau}{2}}\lvert
x(2s+\tau)+2h(2s+\tau)-x(s)-h(s)\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip
65.44142pt+2L\int_{-\frac{\tau}{2}}^{0}\lvert\bar{y}(2s+\tau)-x(s)-h(s)\rvert
ds+2L\int_{0}^{\theta^{*}-\tau}\lvert\bar{y}(2s+\tau)-\tilde{y}(s)\rvert
ds}\\\ \vspace{0.3cm}\displaystyle{\hskip 56.9055pt\leq
2L\int_{-\tau}^{-\frac{\tau}{2}}\lvert x(2s+\tau)-x(s)\rvert
ds+2L\int_{-\tau}^{-\frac{\tau}{2}}\lvert 2h(2s+\tau)-h(s)\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip
65.44142pt+2L\int_{-\frac{\tau}{2}}^{0}\lvert\bar{y}(2s+\tau)-x(0)\rvert
ds+2L\int_{-\frac{\tau}{2}}^{0}\lvert x(0)-x(s)\rvert
ds+2L\int_{-\frac{\tau}{2}}^{0}\lvert h(s)\rvert ds}\\\ \displaystyle{\hskip
65.44142pt+2L\int_{0}^{\theta^{*}-\tau}\lvert\bar{y}(2s+\tau)-\tilde{y}(s)\rvert
ds.}\end{array}$

Therefore, from (4.39), (4.42), (4.44) and from the fact that
$x\in\mathcal{R}_{M}$, we can write

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert\bar{x}(0)+x(0)-2\tilde{x}(0)\rvert\leq
8LC(C+2(1+MC))\lVert h\rVert_{\infty}^{2}+2L(1+6MC)\lVert
h\rVert_{\infty}\theta^{*}}\\\ \vspace{0.3cm}\displaystyle{\hskip
71.13188pt\leq 8LC(C+2(1+MC))\lVert h\rVert_{\infty}^{2}+4LC(1+6MC)\lVert
h\rVert_{\infty}^{2}}\\\ \displaystyle{\hskip
71.13188pt=4LC(C+2+2MC+2+12MC)\lVert h\rVert_{\infty}^{2},}\end{array}$

from which

$\lvert\bar{x}(0)+x(0)-2\tilde{x}(0)\rvert\leq 4LC(C+4+14MC)\lVert
h\rVert_{\infty}^{2}.$ (4.47)

Combining (4.45), (4.46) and (4.47), we can conclude that

$\lvert\bar{x}(0)+x(0)-2\tilde{x}(0)\rvert\leq 4LC(C+4+14MC)\lVert
h\rVert_{\infty}^{2}.$ (4.48)

Now, we set

$x_{1}=2\tilde{x}(0)-x(0),\quad x_{2}=x(0).$

Then, $x_{2}\in\partial\mathcal{K}$ and also

$\frac{x_{1}+x_{2}}{2}=\tilde{x}(0)\in\partial\mathcal{K}.$

Moreover, we have that $x_{1}\in\mathcal{K}_{r}$. Indeed, since
$x(0)\in\partial\mathcal{K}$, (4.43) yields

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{d_{\mathcal{K}}(2\tilde{x}(0)-x(0))\leq\lvert
2\tilde{x}(0)-x(0)-x(0)\rvert}\\\ \vspace{0.3cm}\displaystyle{\hskip
42.67912pt=2\lvert\tilde{x}(0)-x(0)\rvert=2\lvert\tilde{y}(\theta^{*})-x(0)\rvert}\\\
\displaystyle{\hskip 42.67912pt\leq 2(1+2MC)\lVert h\rVert_{\infty}\leq
2(1+2MC)\rho<r.}\end{array}$

Thus, if $x_{1}\notin\mathring{\mathcal{K}}$, using (4.35) and (4.43) we get

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{d_{\mathcal{K}}(2\tilde{x}(0)-x(0))=d_{\mathcal{K}}(2\tilde{x}(0)-x(0))+d_{\mathcal{K}}(x(0))-2d_{\mathcal{K}}(\tilde{x}(0))}\\\
\displaystyle{\hskip 42.67912pt\leq 4c\lvert\tilde{x}(0)-x(0)\rvert^{2}\leq
4c(1+2MC)^{2}\lVert h\rVert^{2}_{\infty}.}\end{array}$

This last inequality still holds whenever $x_{1}\in\mathring{\mathcal{K}}$
since, in this case, $d_{\mathcal{K}}(2\tilde{x}(0)-x(0))=0$. Combining the
above inequality with (4.47), it comes that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{d_{\mathcal{K}}(\bar{x}(0))\leq\lvert\bar{x}(0)-\pi(2\tilde{x}(0)-x(0))\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip
48.36958pt{\leq}\lvert\bar{x}(0)+x(0)-2\tilde{x}(0)\rvert+d_{\mathcal{K}}(2\tilde{x}(0)-x(0))}\\\
\displaystyle{\hskip 48.36958pt\leq 4LC(C+4+14MC)\lVert
h\rVert_{\infty}^{2}+4c(1+2MC)^{2}\lVert h\rVert_{\infty}^{2}.}\end{array}$

So, setting

$\bar{k}:=4LC(7+16MC)+4c(1+2MC)^{2},$

we can write

$d_{\mathcal{K}}(\bar{x}(0))\leq\bar{k}\lVert h\rVert_{\infty}^{2}.$ (4.49)

Let us note that, being $x,x+h\in\mathcal{R}_{M}$, $x+2h\in\mathcal{R}_{3M}$.
This implies that
$\bar{x}=\bar{y}_{2\theta^{*}}(\cdot)=y_{2\theta^{*}}(\cdot;x+2h,\bar{u})\in\mathcal{R}_{3M}$.
Therefore, $\bar{x}\in\mathcal{R}_{3M}$ with
$\bar{x}(0)=\bar{y}(2\theta^{*})\in\mathcal{K}_{\delta}\setminus\mathcal{K}$
and (3.8) yields

$T(\bar{x})\leq Cd_{\mathcal{K}}(\bar{x}(0))).$

This last fact together with (4.37), (4.41) and (4.49) implies that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{T(x+2h)\leq
2\theta^{*}+T(\bar{y}_{2\theta^{*}}(\cdot))}\\\
\vspace{0.3cm}\displaystyle{=2\theta^{*}+T(\bar{x})\leq
2\theta^{*}+Cd_{\mathcal{K}}(\bar{x}(0))}\\\ \vspace{0.3cm}\displaystyle{\leq
2\theta^{*}+C\bar{k}\lVert h\rVert_{\infty}^{2}=2\theta^{*}+k\lVert
h\rVert_{\infty}^{2}}\\\ \displaystyle{\leq 2(T(x+h)+\epsilon)+k\lVert
h\rVert_{\infty}^{2},}\end{array}$

showing that (4.40) is valid. Finally, letting $\epsilon\to 0$ in (4.40), we
get

$T(x+2h)-2T(x+h)\leq k\lVert h\rVert_{\infty}^{2},$

and the proof is concluded. ∎

Now, we are able to prove Theorem 4.1. We assume for simplicity that for any
initial data $x\in\mathcal{R}$ there exists an optimal control. Indeed, if no
such trajectory exists, one can use an approximation argument.

###### proof of Theorem 4.1.

Let $Q\subset\subset\mathcal{R}_{M}\setminus\mathcal{R}_{M}^{\mathcal{K}}$ be
a convex set and take $x,h\in C([-\tau,0],\mathbb{R}^{n})$ such that
$x,x+h,x-h\in Q$. Let $u$ be the optimal control associated with $x$, i.e. for
which $\theta(x,u)=T(x)$. We set

$\bar{y}(t)=y(t;x,u),\quad y_{1}(t)=y(t;x-h,u),\quad y_{2}(t)=y(t;x+h,u).$

Then, by definition, $\bar{y}(T(x))\in\partial\mathcal{K}$.
Now, let $0\leq t\leq T(x)$. Then, we have that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert\bar{y}(t)-y_{1}(t)\rvert=\lvert
y(t;x,u)-y(t;x-h,u)\rvert}\\\ \vspace{0.3cm}\displaystyle{\hskip
68.28644pt=\left\lvert
x(0)+\int_{0}^{t}\bar{y}^{\prime}(s)ds-x(0)+h(0)-\int_{0}^{t}y_{1}^{\prime}(s)ds\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 68.28644pt\leq\lvert
h(0)\rvert+\int_{0}^{t}\lvert f(\bar{y}(s-\tau),u)-f(y_{1}(s-\tau),u)\rvert
ds}\\\ \vspace{0.3cm}\displaystyle{\hskip 68.28644pt\leq\lvert
h(0)\rvert+L\int_{0}^{t}\lvert\bar{y}(s-\tau)-y_{1}(s-\tau)\rvert ds}\\\
\displaystyle{\hskip 68.28644pt=\lvert
h(0)\rvert+L\int_{-\tau}^{t-\tau}\lvert\bar{y}(s)-y_{1}(s)\rvert
ds.}\end{array}$

We distinguish two different situations. Assume first that $T(x)\leq\tau$.
Then, $t\leq\tau$ and, for every $s\in[-\tau,t-\tau]$, it holds $s\leq 0$.
Thus,

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert\bar{y}(t)-y_{1}(t)\rvert\leq\lvert
h(0)\rvert+L\int_{-\tau}^{t-\tau}\lvert x(s)-x(s)+h(s)\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip 68.28644pt\leq\lVert
h\rVert_{\infty}+L\lVert h\rVert_{\infty}t}\\\
\vspace{0.3cm}\displaystyle{\hskip 68.28644pt\leq\lVert
h\rVert_{\infty}+L\lVert h\rVert_{\infty}T(x)}\\\ \displaystyle{\hskip
68.28644pt=(1+T(x))\lVert h\rVert_{\infty}.}\end{array}$

On the contrary, assume that $T(x)>\tau$. Then, if $t\in[0,\tau]$, for every
$s\in[-\tau,t-\tau]$, we have that $s\leq 0$ and and

$\lvert\bar{y}(t)-y_{1}(t)\rvert\leq\lvert
h(0)\rvert+L\int_{-\tau}^{t-\tau}\lvert x(s)-x(s)+h(s)\rvert ds\leq\lVert
h\rVert_{\infty}+L\lVert h\rVert_{\infty}t\leq(1+T(x))\lVert
h\rVert_{\infty}.$

If $t>\tau$, it rather comes that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert\bar{y}(t)-y_{1}(t)\rvert\leq\lVert
h\rVert_{\infty}+L\int_{-\tau}^{0}\lvert\bar{y}(s)-y_{1}(s)\rvert
ds+L\int_{0}^{t-\tau}\lvert\bar{y}(s)-y_{1}(s)\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip 68.28644pt=\lVert
h\rVert_{\infty}+L\int_{-\tau}^{0}\lvert x(s)-x(s)+h(s)\rvert
ds+L\int_{0}^{t-\tau}\lvert\bar{y}(s)-y_{1}(s)\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip 68.28644pt\leq\lVert
h\rVert_{\infty}+L\lVert
h\rVert_{\infty}\tau+L\int_{0}^{t-\tau}\lvert\bar{y}(s)-y_{1}(s)\rvert ds}\\\
\displaystyle{\hskip 68.28644pt\leq(1+LT(x))\lVert
h\rVert_{\infty}+L\int_{0}^{t}\lvert\bar{y}(s)-y_{1}(s)\rvert ds.}\end{array}$
(4.50)

Then, (4.50) holds, as long as $\bar{y}$ and $y_{1}$ remain outside the
target, for $t\in[0,T(x)].$ Thus, from Gronwall’s inequality, we get

$\lvert\bar{y}(t)-y_{1}(t)\rvert\leq(1+LT(x))\lVert
h\rVert_{\infty}e^{Lt}\leq(1+LT(x))e^{LT(x)}\lVert h\rVert_{\infty},\quad{\
t\leq T(x).}$

We have so proved that, in both cases,

$\lvert\bar{y}(t)-y_{1}(t)\rvert\leq k_{1}\lVert h\rVert_{\infty},\quad t\leq
T(x),$ (4.51)

for some positive constant $k_{1}$, depending only on $Q$.
Arguing in the same way, it also holds that

$\lvert\bar{y}(t)-y_{2}(t)\rvert\leq k_{1}\lVert h\rVert_{\infty},\quad t\leq
T(x).$ (4.52)

Now, from assumption $(H_{4})$, the nonlinearity $f$ is semiconcave with
respect to the first variable, uniformly in $u$. Thus, for every
$t\in[-\tau,T(x)]$, there exists a constant $k_{2}>0$, depending only on $Q$,
such that

$\left\lvert
f(y_{1}(t),u)+f(y_{2}(t),u)-2f\left(\frac{y_{1}(t)+y_{2}(t)}{2},u\right)\right\rvert\leq
k_{2}\lvert y_{1}(t)-y_{2}(t)\rvert^{2}.$ (4.53)

Therefore, for $t\in[0,T(x)]$, we have that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert
y_{1}(t)+y_{2}(t)-2\bar{y}(t)\rvert=\lvert
y(t;x-h,u)+y(t;x+h,u)-2y(t;x,u)\rvert}\\\ \vspace{0.3cm}\displaystyle{\hskip
48.36958pt=\left\lvert
x(0)-h(0)+\int_{0}^{t}y_{1}^{\prime}(s)ds+x(0)+h(0)+\int_{0}^{t}y_{2}^{\prime}(s)ds-2x(0)-2\int_{0}^{t}\bar{y}^{\prime}(s)ds\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 48.36958pt\leq\int_{0}^{t}\lvert
y_{1}^{\prime}(s)+y_{2}^{\prime}(s)-2\bar{y}^{\prime}(s)\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip 48.36958pt=\int_{0}^{t}\lvert
f(y_{1}(s-\tau),u)+f(y_{2}(s-\tau),u)-2f(\bar{y}(s-\tau),u)\rvert ds}\\\
\displaystyle{\hskip 48.36958pt=\int_{-\tau}^{t-\tau}\lvert
f(y_{1}(s),u)+f(y_{2}(s),u)-2f(\bar{y}(s),u)\rvert ds.}\end{array}$

Thus, $(H_{1})$ and (4.53) imply that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert
y_{1}(t)+y_{2}(t)-2\bar{y}(t)\rvert\leq\int_{-\tau}^{t-\tau}\left\lvert
f(y_{1}(s),u)+f(y_{2}(s),u)-2f\left(\frac{y_{1}(s)+y_{2}(s)}{2},u\right)\right\rvert
ds}\\\ \vspace{0.3cm}\displaystyle{\hskip
122.34692pt+2\int_{-\tau}^{t-\tau}\left\lvert
f\left(\frac{y_{1}(s)+y_{2}(s)}{2},u\right)-f(\bar{y}(s),u)\right\rvert
ds.}\\\ \vspace{0.3cm}\displaystyle{\hskip 99.58464pt\leq
k_{2}\int_{-\tau}^{t-\tau}\lvert
y_{1}(t)-y_{2}(t)\rvert^{2}ds+2L\int_{-\tau}^{t-\tau}\left\lvert\frac{y_{1}(s)+y_{2}(s)}{2}-\bar{y}(s)\right\rvert
ds}\\\ \displaystyle{\hskip 99.58464pt=k_{2}\int_{-\tau}^{t-\tau}\lvert
y_{1}(s)-y_{2}(s)\rvert^{2}ds+L\int_{-\tau}^{t-\tau}\left\lvert
y_{1}(s)+y_{2}(s)-2\bar{y}(s)\right\rvert ds.}\end{array}$

Now, we distinguish two situations. Assume first that if $t\leq\tau$. Then,

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert
y_{1}(t)+y_{2}(t)-2\bar{y}(t)\rvert\leq k_{2}\int_{-\tau}^{t-\tau}\lvert
x(s)-h(s)-x(s)-h(s)\rvert^{2}ds}\\\ \vspace{0.3cm}\displaystyle{\hskip
125.19194pt+L\int_{-\tau}^{t-\tau}\left\lvert
x(s)-h(s)+x(s)+h(s)-2x(s)\right\rvert ds}\\\ \displaystyle{\hskip
113.81102pt=4k_{2}\int_{-\tau}^{t-\tau}\lvert h(s)\rvert^{2}\leq 4k_{2}\lVert
h\rVert_{\infty}^{2}t,}\end{array}$

and

$\lvert y_{1}(t)+y_{2}(t)-2\bar{y}(t)\rvert\leq 4k_{2}T(x)\lVert
h\rVert_{\infty}^{2}.$ (4.54)

On the other hand, assume that $t>\tau$. Then, in this case,

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert
y_{1}(t)+y_{2}(t)-2\bar{y}(t)\rvert\leq{k_{2}}\int_{-\tau}^{0}\lvert
y_{1}(s)-y_{2}(s)\rvert^{2}ds+L\int_{-\tau}^{0}\left\lvert
y_{1}(s)+y_{2}(s)-2\bar{y}(s)\right\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip 56.9055pt+\int_{0}^{t-\tau}\lvert
y_{1}(s)-y_{2}(s)\rvert^{2}ds+L\int_{0}^{t-\tau}\left\lvert
y_{1}(s)+y_{2}(s)-2\bar{y}(s)\right\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip 28.45274pt=4k_{2}\int_{-\tau}^{0}\lvert
h(s)\rvert^{2}ds+L\int_{-\tau}^{0}\lvert x(s)-h(s)+x(s)+h(s)-2x(s)\rvert
ds}\\\ \vspace{0.3cm}\displaystyle{\hskip 56.9055pt+\int_{0}^{t-\tau}\lvert
y_{1}(s)-y_{2}(s)\rvert^{2}ds+L\int_{0}^{t-\tau}\left\lvert
y_{1}(s)+y_{2}(s)-2\bar{y}(s)\right\rvert ds}\\\ \displaystyle{\hskip
28.45274pt\leq 4k_{2}\lVert h\rVert_{\infty}^{2}\tau+\int_{0}^{t-\tau}\lvert
y_{1}(s)-y_{2}(s)\rvert^{2}ds+L\int_{0}^{t-\tau}\left\lvert
y_{1}(s)+y_{2}(s)-2\bar{y}(s)\right\rvert ds.}\end{array}$

Now, using (4.51) and (4.52), for every $s\in[0,t-\tau]$,

$\lvert y_{1}(s)-y_{2}(s)\rvert\leq\lvert
y_{1}(s)-\bar{y}(s)\rvert+\lvert\bar{y}(s)-y_{2}(s)\rvert\leq 2k_{1}\lVert
h\rVert_{\infty},$

from which

$\lvert y_{1}(s)-y_{2}(s)\rvert^{2}\leq 4k_{1}^{2}\lVert
h\rVert_{\infty}^{2}.$

As a consequence, we get

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert
y_{1}(t)+y_{2}(t)-2\bar{y}(t)\rvert\leq 4k_{2}\lVert
h\rVert_{\infty}^{2}\tau+4k_{1}^{2}\lVert h\rVert_{\infty}^{2}(t-\tau)}\\\
\vspace{0.3cm}\displaystyle{\hskip 128.0374pt+L\int_{0}^{t-\tau}\left\lvert
y_{1}(s)+y_{2}(s)-2\bar{y}(s)\right\rvert ds}\\\
\vspace{0.3cm}\displaystyle{\hskip 113.81102pt\leq
4(k_{2}+k_{1}^{2})T(x)\lVert
h\rVert_{\infty}^{2}+L\int_{0}^{t-\tau}\left\lvert
y_{1}(s)+y_{2}(s)-2\bar{y}(s)\right\rvert ds}\\\ \displaystyle{\hskip
113.81102pt\leq 4(k_{2}+k_{1}^{2})T(x)\lVert
h\rVert_{\infty}^{2}+L\int_{0}^{t}\left\lvert
y_{1}(s)+y_{2}(s)-2\bar{y}(s)\right\rvert ds.}\end{array}$ (4.55)

Then, (4.55) holds for any $t\leq T(x).$ Thus, Gronwall’s inequality yields

$\lvert y_{1}(t)+y_{2}(t)-2\bar{y}(t)\rvert\leq
e^{LT(x)}4(k_{2}+k_{1}^{2})T(x)\lVert h\rVert_{\infty}^{2},\quad t\leq T(x).$

This last fact together with (4.54) implies that

$\lvert y_{1}(t)+y_{2}(t)-2\bar{y}(t)\rvert\leq k_{3}\lVert
h\rVert_{\infty}^{2},\quad t\leq T(x),$ (4.56)

for a positive constant $k_{3}$, dependent only of $Q$.
Now, we can consider three cases.
(Case I) Suppose that one of the two paths $y_{1}$ or $y_{2}$, namely $y_{1}$,
reaches $\mathcal{K}$ at a time $t^{*}<T(x)$, i.e. $t^{*}=\theta(x-h,u)<T(x)$.
We define the functions $x_{1}$, $x_{2}$, $\bar{x}$ in
$C([-\tau,0];\mathbb{R}^{n})$ as follows

$x_{1}(\cdot)=y_{1_{t^{*}}}(\cdot),\quad
x_{2}(\cdot)=y_{2_{t^{*}}}(\cdot),\quad\bar{x}(\cdot)=\bar{y}_{t^{*}}(\cdot).$

We can immediately notice that $x_{1}(0)=y_{1}(t^{*})\in\partial\mathcal{K}$,
being $\theta(x-h,u)=t^{*}$. Also, by definition of $T$,

$T(x-h)\leq t^{*}.$ (4.57)

Moreover, using the dynamic programming principle (Proposition 3.1) and since
$u$ is an optimal control for $x$, we can write

$T(x+h)\leq t^{*}+T(x_{2}),\quad T(x)=t^{*}+T(\bar{x}).$ (4.58)

Now, we want to apply Lemma 4.2. First of all, we have that
$x_{1}(0)\in\partial\mathcal{K}$ and that
$x_{1}=y_{1_{t^{*}}}(\cdot)=y_{t^{*}}(\cdot;x-h,u)\in\mathcal{R}_{M}$. In
addition, $\bar{x}-x_{1}\in C([-\tau,0];\mathbb{R}^{n})$ satisfies:
$(x_{1}+\bar{x}-x_{1})(0)=\bar{x}(0)=\bar{y}_{t^{*}}(0)=y(t^{*};x,u)\notin\mathcal{K}$,
since from our assumptions $t^{*}<T(x)$;
$x_{1}+\bar{x}-x_{1}=\bar{x}=\bar{y}_{t^{*}}(\cdot)=y_{t^{*}}(\cdot;x,u)\in\mathcal{R}_{M}.$
Therefore, from Lemma 4.2, for $\lVert\bar{x}-x_{1}\rVert_{\infty}$ small it
comes that

$T(2\bar{x}-x_{1})-2T(\bar{x})\leq
k_{4}\lVert\bar{x}-x_{1}\rVert_{\infty}^{2},$ (4.59)

for a suitable positive constant $k_{4}$. Moreover, for every $s\in[-\tau,0]$,
we have

$\lvert\bar{x}(s)-x_{1}(s)\rvert=\lvert\bar{y}(t^{*}+s)-y_{1}(t^{*}+s)\rvert.$

Thus, if $t^{*}+s>0$, from (4.51) with $t^{*}+s\in[0,T(x)]$ we get

$\lvert\bar{x}(s)-x_{1}(s)\rvert\leq k_{1}\lVert h\rVert_{\infty}.$

On the other hand, if $t^{*}+s\leq 0$, it rather holds

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\hskip
14.22636pt\lvert\bar{x}(s)-x_{1}(s)\rvert=\lvert
y(t^{*}+s;x,u)-y(t^{*}+s;x-h,u)\rvert}\\\ \displaystyle{=\lvert
x(t^{*}+s)-x(t^{*}+s)+h(t^{*}+s)\rvert=\lvert h(t^{*}+s)\rvert\leq\lVert
h\rVert_{\infty}}\end{array}$

So, setting $k_{5}:=\max\\{k_{1},1\\}$, it comes that

$\lvert\bar{x}(s)-x_{1}(s)\rvert\leq k_{5}\lVert h\rVert_{\infty},$

for all $s\in[-\tau,0]$. Therefore,

$\lVert\bar{x}-x_{1}\rVert_{\infty}\leq k_{5}\lVert h\rVert_{\infty}.$

and (4.59) becomes

$T(2\bar{x}-x_{1})-2T(\bar{x})\leq k_{4}k_{5}^{2}\lVert h\rVert_{\infty}^{2}.$
(4.60)

Furthermore, since $T$ is locally Lipschitz continuous from Theorem 3.3, we
have that

$\lvert T(2\bar{x}-x_{1})-T(x_{2})\rvert\leq k_{6}\lVert
2\bar{x}-x_{1}-x_{2}\rVert_{\infty},$ (4.61)

for some positive constant $k_{6}$. Let us note that, for every
$s\in[-\tau,0]$,

$\lvert 2\bar{x}(s)-x_{1}(s)-x_{2}(s)\rvert=\lvert
2\bar{y}(t^{*}+s)-y_{1}(t^{*}+s)-y_{2}(t^{*}+s)\rvert.$

Then, if $t^{*}+s>0$, from (4.56) it follows that

$\lvert 2\bar{x}(s)-x_{1}(s)-x_{2}(s)\rvert\leq k_{3}\lVert
h\rVert_{\infty}^{2}.$

On the contrary, if $t^{*}+s\leq 0$, it turns out that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{\lvert
2\bar{x}(s)-x_{1}(s)-x_{2}(s)\rvert=\lvert
2y(t^{*}+s;x,u)-y(t^{*}+s;x-h,u)-y(t^{*}+s;x+h,u)\rvert}\\\
\displaystyle{\hskip 19.91684pt=\lvert
2x(t^{*}+s)-x(t^{*}+s)+h(t^{*}+s)-x(t^{*}+s)-h(t^{*}+s)\rvert=0\leq
k_{3}\lVert h\rVert_{\infty}^{2}.}\end{array}$

Thus, we can write

$\lvert 2\bar{x}(s)-x_{1}(s)-x_{2}(s)\rvert\leq{\color[rgb]{1,0,0}k_{3}}\lVert
h\rVert_{\infty}^{2},$

for every $s\in[-\tau,0]$. Hence,

$\lVert 2\bar{x}-x_{1}-x_{2}\rVert_{\infty}\leq k_{3}\lVert
h\rVert_{\infty}^{2},$

from which (4.61) becomes

$\lvert T(2\bar{x}-x_{1})-2T(\bar{x})\rvert\leq k_{6}k_{3}\lVert
h\rVert_{\infty}^{2}.$ (4.62)

As a consequece, using (4.57), (4.58), (4.60) and (4.62), we finally get

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{T(x+h)+T(x-h)-2T(x)\leq
t^{*}+T(x_{2})+t^{*}-2t^{*}-2T(\bar{x})}\\\ \vspace{0.3cm}\displaystyle{\hskip
56.9055pt=T(x_{2})-T(2\bar{x}-x_{1})+T(2\bar{x}-x_{1})-2T(\bar{x})}\\\
\displaystyle{\hskip 56.9055pt\leq(k_{4}k_{5}^{2}+k_{6}k_{3})\lVert
h\rVert_{\infty}^{2}=k_{7}\lVert h\rVert_{\infty}^{2}.}\end{array}$

(Case II) Suppose that neither $y_{1}$ nor $y_{2}$ reach the target before
$\bar{y}$ and that

$\frac{y_{1}(T(x))+y_{2}(T(x))}{2}\in\mathcal{K}.$ (4.63)

Then, there exists $t^{*}\leq T(x)$ such that

$\frac{y_{1}(t^{*})+y_{2}(t^{*})}{2}\in\partial\mathcal{K}.$

Let us note that, being $t^{*}\leq T(x)$,
$y_{1}(t^{*}),\,y_{2}(t^{*})\notin\mathcal{K}$. Thus, from (4.34) there exists
a positive constant $k_{8}$ such that

$d_{\mathcal{K}}(y_{1}(t^{*}))+d_{\mathcal{K}}(y_{2}(t^{*}))-2d_{\mathcal{K}}\left(\frac{y_{1}(t^{*})+y_{2}(t^{*})}{2}\right)\leq
k_{8}\lvert y_{2}(t^{*})-y_{1}(t^{*})\rvert^{2}.$ (4.64)

Now, using (4.51) and (4.52), it holds that

$\lvert y_{2}(t^{*})-y_{1}(t^{*})\rvert\leq\lvert
y_{2}(t^{*})-\bar{y}(t^{*})\rvert+\lvert\bar{y}(t^{*})-y_{1}(t^{*})\rvert\leq
2k_{1}\lVert h\rVert_{\infty},$

from which

$\lvert y_{2}(t^{*})-y_{1}(t^{*})\rvert^{2}\leq 4k_{1}^{2}\lVert
h\rVert_{\infty}^{2}.$

Therefore, combining this last fact with (4.63) and (4.64), it follows that

$\begin{split}d_{\mathcal{K}}(y_{1}(t^{*}))+d_{\mathcal{K}}(y_{2}(t^{*}))&=d_{\mathcal{K}}(y_{1}(t^{*}))+d_{\mathcal{K}}(y_{2}(t^{*}))-2d_{\mathcal{K}}\left(\frac{y_{1}(t^{*})+y_{2}(t^{*})}{2}\right)\\\
&\leq 4k_{8}k_{1}^{2}\lVert h\rVert_{\infty}^{2}.\end{split}$ (4.65)

Also, the dynamic programming principle (Proposition 3.1) yields

$T(x-h)\leq t^{*}+T(x_{1}),\quad T(x+h)\leq t^{*}+T(x_{2}),$ (4.66)

where $x_{1},x_{2}$ are the functions in $C([-\tau,0];\mathbb{R}^{n})$ given
by

$x_{1}(\cdot)=y_{1_{t^{*}}}(\cdot),\quad x_{2}(\cdot)=y_{2_{t^{*}}}(\cdot).$

Now, from (3.8) it comes that

$T(x_{1})\leq Cd_{\mathcal{K}}(x_{1}(0)),\quad T(x_{2})\leq
Cd_{\mathcal{K}}(x_{2}(0)).$ (4.67)

As a consequence, putting together (4.65), (4.66) and (4.67), we can conclude
that

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{T(x+h)+T(x-h)-2T(x)\leq
t^{*}+T(x_{1})+t^{*}+T(x_{2})-2T(x)}\\\ \vspace{0.3cm}\displaystyle{\hskip
156.49014pt\leq 2T(x)+T(x_{1})+T(x_{2})-2T(x)}\\\
\vspace{0.3cm}\displaystyle{\hskip 156.49014pt\leq
C(d_{\mathcal{K}}(x_{1}(0))+d_{\mathcal{K}}(x_{2}(0)))}\\\
\vspace{0.3cm}\displaystyle{\hskip
156.49014pt=d_{\mathcal{K}}(y_{1}(t^{*}))+d_{\mathcal{K}}(y_{2}(t^{*}))}\\\
\vspace{0.3cm}\displaystyle{\hskip 156.49014pt\leq 4k_{8}k_{1}^{2}\lVert
h\rVert_{\infty}^{2}}\\\ \displaystyle{\hskip 156.49014pt=k_{9}\lVert
h\rVert_{\infty}^{2}.}\end{array}$

(Case III) Suppose that neither $y_{1}$ nor $y_{2}$ reach the target before
$\bar{y}$ and that

$\frac{y_{1}(T(x))+y_{2}(T(x))}{2}\notin\mathcal{K}.$

Let $x_{1}$, $x_{2}$, $\bar{x}$ be the functions in
$C([-\tau,0];\mathbb{R}^{n})$ defined as follows

$x_{1}(\cdot)=y_{1_{T(x)}}(\cdot),\quad
x_{2}(\cdot)=y_{2_{T(x)}}(\cdot),\quad\bar{x}(\cdot)=\bar{y}_{T(x)}(\cdot).$

Then, $x_{1}(0),\,x_{2}(0),\frac{x_{1}(0)+x_{2}(0)}{2}\notin\mathcal{K}$.
Moreover, from (4.34), (4.51) and (4.52), we can write

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{d_{\mathcal{K}}(x_{1}(0))+d_{\mathcal{K}}(x_{2}(0))-2d_{\mathcal{K}}\left(\frac{x_{1}(0)+x_{2}(0)}{2}\right)\leq
k_{9}\lvert x_{1}(0)-x_{2}(0)\rvert^{2}}\\\ \displaystyle{\hskip
56.9055pt=k_{8}\lvert y_{1}(T(x))-y_{2}(T(x))\rvert^{2}\leq
4k_{8}k_{1}^{2}\lVert h\rVert_{\infty}^{2}.}\end{array}$

Therefore, since $\bar{x}(0)=\bar{y}(T(x))=y(T(x);x,u)\in\mathcal{K}$, using
(4.56) we get

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{d_{\mathcal{K}}(x_{1}(0))+d_{\mathcal{K}}(x_{2}(0))}\\\
\vspace{0.3cm}\displaystyle{\hskip
42.67912pt=d_{\mathcal{K}}(x_{1}(0))+d_{\mathcal{K}}(x_{2}(0))-2d_{\mathcal{K}}\left(\frac{x_{1}(0)+x_{2}(0)}{2}\right)+2d_{\mathcal{K}}\left(\frac{x_{1}(0)+x_{2}(0)}{2}\right)}\\\
\vspace{0.3cm}\displaystyle{\hskip 42.67912pt\leq 4k_{8}k_{1}^{2}\lVert
h\rVert_{\infty}^{2}+2\left\lvert\frac{x_{1}(0)+x_{2}(0)}{2}-\bar{x}(0)\right\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 42.67912pt=4k_{8}k_{1}^{2}\lVert
h\rVert_{\infty}^{2}+\lvert x_{1}(0)+x_{2}(0)-2\bar{x}(0)\rvert}\\\
\vspace{0.3cm}\displaystyle{\hskip 42.67912pt=4k_{8}k_{1}^{2}\lVert
h\rVert_{\infty}^{2}+\lvert y_{1}(T(x))+y_{2}(T(x))-2\bar{y}(T(x))\rvert}\\\
\displaystyle{\hskip 42.67912pt\leq 4k_{8}k_{1}^{2}\lVert
h\rVert_{\infty}^{2}+k_{3}\lVert
h\rVert_{\infty}^{2}=(4k_{8}k_{1}^{2}+k_{3})\lVert
h\rVert_{\infty}^{2}.}\end{array}$

Finally, from Proposition 3.1 and from (3.8),

$\begin{array}[]{l}\vspace{0.3cm}\displaystyle{T(x+h)+T(x-h)-2T(x)\leq
T(x)+T(x_{1})+T(x)+T(x_{2})-2T(x)}\\\ \vspace{0.3cm}\displaystyle{\hskip
153.6447pt\leq C(d_{\mathcal{K}}(x_{1}(0))+d_{\mathcal{K}}(x_{2}(0)))}\\\
\displaystyle{\hskip 153.6447pt\leq C(4k_{8}k_{1}^{2}+k_{3})\lVert
h\rVert_{\infty}^{2}=k_{10}\lVert h\rVert_{\infty}^{2},}\end{array}$

which concludes our proof. ∎

##  5 Conclusions

In this paper, we have investigated the regularity properties of the value
function associated to a minimum time problem for a delayed control system.
Time delays in differential equations are very useful for the description of
several phenomena. There are situations in which the dynamics does not depend
on the present state of the trajectory but is rather influenced by what has
happened in some previous instants. For instance, in stock exchange, one might
have to look at the past market trend in order to make the right investment.
Other situations in which it is more convenient to look at the past history of
the dynamics can be found in epidemiological, biological, social, and economic
models.

The semiconcavity result we establish in this work is an extension of one
achieved in [12] by Cannarsa and Sinestrari. Semiconcavity in the classical
undelayed case has many applications. The analysis of the regularity
properties of the minimum time function is relevant since it is connected with
the study of Hamilton-Jacobi-Belmann equations.

The Hamilton-Jacobi theory in infinite dimension has been developed by [16,
17, 18]. Also, for optimal control problems involving time delays and with
finite horizon, the Hamilton-Jacobi-Belmann equations have been investigated
by [6, 7, 8, 23, 24, 25, 30, 31, 32, 33, 34]. It is, then, interesting to
discuss the Hamilton-Jacobi theory related to the minimum time problem for
control systems presenting time delays in the state space and to deepen the
application of the semiconcavity also in the delayed setting. We leave this
for further research.

## References

  * [1] P. Albano,  On the local semiconcavity of the solutions of the eikonal equation,  Nonlinear Anal., 73 (2010), 458–464.
  * [2] P. Albano, V. Basco, and P. Cannarsa, Piermarco,  The distance function in the presence of an obstacle, Calc. Var. Partial Differential Equations 61 (2022), no. 1, Paper No. 13, 26 pp.
  * [3] P. Albano, P. Cannarsa, and T. Scarinci,  Regularity results for the minimum time function with Hörmander vector fields,  J. Differential Equations 264 (2018), 3312–3335.
  * [4] M. Bardi,  A boundary value problem for the minimum-time function,  SIAM J. Control Optim., 27 (1989), 776–785.
  * [5] M. Bardi and I. Capuzzo Dolcetta,  Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations,  Systems & Control: Foundations & Applications. Birkhäuser Boston, Inc., Boston, MA, 1997.
  * [6] E.N. Barron,  Application of viscosity solutions of infinite-dimensional Hamilton-Jacobi-Bellman equations to some problems in distributed optimal control,  J. Optim. Theory Appl., 64 (1990), 245–268.
  * [7] A. Boccia and R.B. Vinter,  The maximum principle for optimal control problems with time delays,  SIAM J. Control Optim., 55 (2017), 2905–2935.
  * [8] R. Bonalli, B. Hérissé, and E. Trélat,  Continuity of Pontryagin extremals with respect to delays in nonlinear optimal control,  SIAM J. Control Optim., 57 (2019), 1440–1466.
  * [9] B. Bonnet and H. Frankowska,  Semiconcavity and sensitivity analysis in mean-field optimal control and applications,  J. Math. Pures Appl., (9) 157 (2022), 282–345.
  * [10] P. Cannarsa, H. Frankowska, and C. Sinestrari,  Optimality conditions and synthesis for the minimum time problem. Set-valued analysis in control theory,  Set-Valued Anal., 8 (2000), 127–148.
  * [11] P. Cannarsa, C. Pignotti, C. Sinestrari,  Semiconcavity for optimal control problems with exit time,  Discrete Contin. Dynam. Systems, 6 (2000), 975–997.
  * [12] P. Cannarsa and C. Sinestrari,  Convexity properties of the minimum time function,  Cal. Var., 3 (1995), 273–298.
  * [13] P. Cannarsa and C. Sinestrari,  Semiconcave functions, Hamilton-Jacobi equations, and optimal control,  Progress in Nonlinear Differential Equations and their Applications, 58. Birkhäuser Boston, Inc., Boston, MA, 2004.
  * [14] M.G. Crandall and P.-L. Lions,  Viscosity solutions of Hamilton-Jacobi equations,  Trans. Amer. Math. Soc., 277 (1983), 1–42.
  * [15] M.G. Crandall, L.C. Evans, and P.-L. Lions,  Some properties of viscosity solutions of Hamilton-Jacobi equations,  Trans. Amer. Math. Soc., 282 (1984), 487–502.
  * [16] M.G. Crandall and P.-L. Lions,  Hamilton-Jacobi equations in infinite dimensions. I. Uniqueness of viscosity solutions,  J. Funct. Anal., 62 (1985), 379–396.
  * [17] M.G. Crandall and P.-L. Lions,  Hamilton-Jacobi equations in infinite dimensions. II. Existence of viscosity solutions,  J. Funct. Anal., 65 (1986), 368–405.
  * [18] M.G. Crandall and P.-L. Lions,  Hamilton-Jacobi equations in infinite dimensions. III ,  J. Funct. Anal., 68 (1986), 214–247.
  * [19] A. Halanay,  Differential Equations: Stability, Oscillations, Time Lags,  Academic Press, New York and London, 1966.
  * [20] J.K. Hale and S.M.V. Lunel,  Introduction to functional differential equations,  Applied Mathematical Sciences, 99, Springer, 1993.
  * [21] Y. Han,  Global semiconcavity of solutions to first-order Hamilton-Jacobi equations with state constraints,  preprint arXiv:2205.01615, 2022.
  * [22] H. Ishii,  Viscosity solutions for a class of Hamilton-Jacobi equations in Hilbert spaces,  J. Funct. Anal., 105 (1992), 301–341.
  * [23] N.Y. Lukoyanov,  A minimax solution of functional equations of Hamilton-Jacobi type for hereditary systems, (Russian)  Differ. Uravn., 37 (2001), 228–237; translation in Differ. Equ., 37 (2001), 246–256.
  * [24] N.Y. Lukoyanov,  Viscosity solution of nonanticipating equations of Hamilton-Jacobi type, (Russian)  Differ. Uravn. 43 (2007), 1674–1682; translation in Differ. Equ., 43 (2007), 1715–1723.
  * [25] N.Y. Lukoyanov,  The Hamilton-Jacobi equation for hereditary systems: minimax and viscosity solutions, (Russian)  Dokl. Akad. Nauk, 418 (2008), 300–303; translation in Dokl. Math., 77 (2008), 51–54.
  * [26] L.V. Nguyen,  A note on optimality conditions for optimal exit time problems,  Math. Control Relat. Fields, 5 (2015), 291–303.
  * [27] N.N. Petrov,  Controllability of autonomous systems,  Differ. Equ., 4 (1968), 311-317.
  * [28] N.N. Petrov,  On the Bellman function for the time-optimal process problem,  J. Appl. Math. Mech., 34 (1970), 785-791.
  * [29] C. Pignotti,  Rectifiability results for singular and conjugate points of optimal exit time problems,  J. Math. Anal. Appl., 270 (2002), 681–708.
  * [30] A. Plaksin,  Minimax and viscosity solutions of Hamilton-Jacobi-Bellman equations for time-delay systems,  J. Optim. Theory Appl., 187 (2020), 22–42.
  * [31] A. Plaksin,  Viscosity solutions of Hamilton-Jacobi-Bellman-Isaacs equations for time-delay systems,  SIAM J. Control Optim., 59 (2021), 1951–1972.
  * [32] H.M. Soner,  On the Hamilton-Jacobi-Bellman equations in Banach spaces,  J. Optim. Theory Appl., 57 (1988), 429–437.
  * [33] R.B. Vinter,  Optimal control problems with time delays: constancy of the Hamiltonian,  SIAM J. Control Optim., 57 (2019), n 2574–2602.
  * [34] J. Zhou,  Delay optimal control and viscosity solutions to associated Hamilton-Jacobi-Bellman equations,  Internat. J. Control, 92 (2019), 2263–2273.