remarkRemark hypothesisHypothesis claimClaim assumptionAssumption
exampleExample Average cost per unit time impulse controlŁ. Stettner

#  On an approximation of average cost per unit time impulse control of Markov
processes

Łukasz Stettner Institute of Mathematics, Polish Academy of Sciences, Warsaw,
Poland, (); research supported by NCN grant 2016/23/B/ST1/00479.
<EMAIL_ADDRESS>
###### Abstract

In this paper we consider impulse control of continuous time Markov processes
with average cost per unit time functional. This problem is approximated using
impulse control problems stopped at the first exit time from increasing
sequence of open sets. We find solution to Bellman equation corresponding to
the original problem and show that stopped impulse control problems
approximate optimal value of the cost functional.

###### keywords:

Optimal stopping, Bellman equation, impulse control, average cost per unit
time

93E20, 49N60, 93C10, 60J25

##  1 Introduction

Let $(X_{t})$ be a Feller-Markov process on $(\Omega,F,(F_{t}))$ taking values
in a locally compact space $E$ with metric $\rho$ and Borel $\sigma$ field
${\cal E}$. The process starting from $x$ at time $0$ generates a probability
measure $\mathbb{P}_{x}$ and we denote by $\mathbb{E}_{x}$ a related
expectation operator. We shall assume that Markov semigroup $T_{t}$ defined
for $h\in C(E)$, the space of continuous bounded functions on $E$, as
$T_{t}h(x)=\mathbb{E}_{x}\left\\{h(X_{t})\right\\}$ transforms continuous
functions vanishing at infinity $C_{0}(E)$ into itself. Process $(X_{t})$ is
controlled by impulses $V=(\tau_{i},\xi_{i})$ consisting of an increasing
sequence of stopping times $\tau_{i}$ and impulses $\xi_{i}$, which is adapted
to the available observation till time $\tau_{i}$: at time $\tau_{i}$ the
process is shifted from the state $X_{\tau_{i}}$ to the state $\xi_{i}$ at the
cost of $c(X_{\tau_{i}},\xi_{i})\geq c>0$ and follows its dynamics until the
next impulse. We assume that impulses shift the process to a compact set
$U\subseteq E$, i.e., $\xi\in U$ and the cost function $c(x,\xi)$ is positive,
continuous bounded and uniformly bounded away from zero by a constant $c>0$.
Furthermore to avoid multiple impulses we assume that for $x\in E$ and
$\xi,\xi^{\prime}\in U$ we have $c(x,\xi)\leq
c(x,\xi^{\prime})+c(\xi^{\prime},\xi)$. A strategy $V=(\tau_{i},\xi_{i})$ is
_admissible_ for $x\in E$ if $\tau_{i}$ form an increasing sequence of
stopping times (possibly taking the value $\infty$) with
$\lim_{i\to\infty}\tau_{i}=\infty$, $\mathbb{P}_{x}$-a.s. To describe the
evolution of the controlled process we introduce a construction of [26,
Section 2], which follows ideas of [23]. Namely, we consider
$\Omega=D(R^{+},E)^{\infty}$, where $D(R^{+},E)$ is a canonical space of right
continuous, left limited functions on $R^{+}$ taking values in $E$. We assume
that $(F^{1}_{t})$ is a canonical filtration on $D(R^{+},E)$ and inductively
define $F^{n+1}_{t}=F^{n}_{t}\otimes F_{t}$. The stopping times $\tau_{i}$ are
adapted $F^{i}_{t}\times\left\\{\emptyset,D(R^{+},E)\right\\}^{\infty}$ while
the impulses $\xi_{i}$ are adapted to
$F^{i}_{\tau_{i}}\times\left\\{\emptyset,D(R^{+},E)\right\\}^{\infty}$. The
trajectory of the controlled process $(Y_{t})$ is defined using coordinates
$x^{n}$ of the canonical space $\Omega$, i.e. $Y_{t}=x^{n}_{t}$ for
$t\in[\tau_{n-1},\tau_{n})$, with $\tau_{0}=0$. Given an impulse strategy $V$
following [26, Section 2] and [23, Chapter 5 and Appendix 2] we define a
probability measure $\mathbb{P}^{V}$ on $\Omega$. Our goal is to minimize over
all admissible strategies the functional

(1)
${J}\big{(}x,(\tau_{i},\xi_{i})\big{)}=\liminf_{n\to\infty}\frac{1}{\mathbb{E}_{x}^{V}\\{\tau_{n}\\}}\mathbb{E}_{x}^{V}\Big{\\{}\int_{0}^{\tau_{n}}f(Y_{s})ds+\sum_{i=1}^{n}c(x_{\tau_{i}}^{i},\xi_{i})\Big{\\}},$

assuming that $(\tau_{n})$ are such that
$\mathbb{E}_{x}\\{\tau_{n}\\}<\infty$, where $f$ is a continuous bounded
function and $x_{\tau_{i}}^{i}$ is the state of the process before the $i$-th
impulse with a natural meaning if there is more than one impulse at the same
time. Alternatively we can consider average cost per unit time functional

(2)
$\hat{J}\big{(}x,(\tau_{i},\xi_{i})\big{)}=\liminf_{T\to\infty}\frac{1}{T}\mathbb{E}_{x}^{V}\Big{\\{}\int_{0}^{T}f(Y_{s})ds+\sum_{i=1}^{\infty}1_{\tau_{i}\leq
T}c(x_{\tau_{i}}^{i},\xi_{i})\Big{\\}}.$

Impulse control of random systems is intensively studied from various points
of view. Such control problems have many applications: in finance (cash
management, portfolio optimization) [13], [4], control of the exchange rate
[18], optimal harvesting [29], [1], inventory control [5] (see also the first
chapter of [6] for other applications). Impulse control of diffusions and
diffusions with jumps was studied in [6], [3]. Average cost per unit time
impulse control problems were studied first for fixed cost of impulses in
[24], [25] under uniform ergodicity assumption and then for so called
separated cost in [26] and in [27] under quasicompactness of transition
semigroup. The problem was also studied under some compactness conditions in
[12]. Ergodic impulse control of diffusion processes in a bounded domain was
studied in [15] and [22]. An extension to an unbounded domain (real line) was
then considered in [14]. Average cost per unit time impulse control for Levy
process was studied in [8]. In the case of compact state space and constraints
the average cost per unit time impulse control problem was studied in [17].
This kind of cost functional is also important in mathematics of finance under
the name of Kelly criterion - see [16]. In particular impulse control appear
in the case of growth optimal portfolio under proportional transaction costs
in [10] and [7] and references there in. Average cost per unit time impulse
control in a general (locally compact separable) state space was considered in
[21]. This paper is an extension of the last paper. Namely in the paper [21]
discounted approximation approach was considered. In this paper we consider
another approach based on stopped impulse control with increasing sequence of
domains. We use stopped impulse control approach i.e. we stop the controlled
process when we enter a boundary of an open set. We consider a sequence of
open sets increasing to the whole space and obtain a sequence of impulse
control problems which as we show approximate original nonrestricted control
problem. This way we obtain a nice approximation procedure to original impulse
control problem. As one can see e.g. in [5] unrestricted impulse control
problem with average cost per unit time functional is difficult to solve
numerically even for rather simple inventory models. The method proposed in
the paper seems to be natural approximation procedure which justifies
computational intuition. In the paper we use stopping results from the paper
[20]. We consider first the functional in a weak form (1) and then study its
general form (2). Several nontrivial results concerning ergodic optimal
stopping are of independent interest and are shown in the appendix.

##  2 Stopped impulse control problems

Let $\cal{O}$ be an open set in $E$ such that
$\sup_{x\in\cal{O}}\mathbb{E}_{x}\left\\{(\tau_{\cal{O}})^{2}\right\\}<\infty$,
for $\tau_{\cal{O}}=\inf\left\\{s\geq 0:X_{s}\in E\setminus\cal{O}\right\\}$,
which means that $\tau_{\cal{O}}$ is the first exit time from $\cal{O}$. We
shall assume that stopped semigroup
$T_{t}^{\cal{O}}h(x):=\mathbb{E}_{x}\left\\{h(X_{t})1_{t<\tau_{\cal{O}}}\right\\}$
transforms $C(E)$ into itself. Consider stopped impulse functional

(3)
${J}^{\cal{O}}\big{(}x,(\tau_{i},\xi_{i})\big{)}=\liminf_{n\to\infty}\frac{1}{\mathbb{E}_{x}^{V}\left\\{\tau_{n}\wedge\tau_{\cal{O}}\right\\}}\mathbb{E}_{x}^{V}\Big{\\{}\int_{0}^{\tau_{n}\wedge\tau_{\cal{O}}}f(Y_{s})ds+\sum_{i=1}^{n}1_{\tau_{i}<\tau_{\cal{O}}}c(x_{\tau_{i}}^{i},\xi_{i})\Big{\\}}.$

Let for $\alpha>0$

(4) $\lambda_{\alpha}:=\inf_{x\in
U}\inf_{V}{\mathbb{E}_{x}^{V}\Big{\\{}\int_{0}^{\tau_{\cal{O}}}e^{-\alpha
s}f(Y_{s})ds+\sum_{i=1}^{\infty}1_{\tau_{i}<\tau_{\cal{O}}}e^{-\alpha\tau_{i}}c(x_{\tau_{i}}^{i},\xi_{i})\Big{\\}}\over\mathbb{E}_{x}^{V}\Big{\\{}\int_{0}^{\tau_{\cal{O}}}e^{-\alpha
s}ds\Big{\\}}},$ (5)
$w_{\alpha}(x):=\inf_{V}\mathbb{E}_{x}^{V}\Big{\\{}\int_{0}^{\tau_{\cal{O}}}e^{-\alpha
s}(f(Y_{s})-\lambda_{\alpha})ds+\sum_{i=1}^{\infty}1_{\tau_{i}<\tau_{\cal{O}}}e^{-\alpha\tau_{i}}c(x_{\tau_{i}}^{i},\xi_{i})\Big{\\}},$

where $V$ stands for impulse strategy $(\tau_{i},\xi_{i})$. We clearly have
that

######  Lemma 2.1.

For each $\alpha>0$ we have

(6) $\inf_{x\in U}w_{\alpha}(x)=0.$

######  Proof 2.2.

For $x\in U$ and any impulse strategy $V$ from (4) we have that

(7) $0\leq\mathbb{E}_{x}^{V}\Big{\\{}\int_{0}^{\tau_{\cal{O}}}e^{-\alpha
s}(f(Y_{s})-\lambda_{\alpha})ds+\sum_{i=1}^{\infty}1_{\tau_{i}<\tau_{\cal{O}}}e^{-\alpha\tau_{i}}c(x_{\tau_{i}}^{i},\xi_{i})\Big{\\}}.$

Consider now $x_{\epsilon}\in U$ and $V^{\epsilon}$ such that

(8)
$\lambda_{\alpha}\geq-\epsilon+{\mathbb{E}_{x_{\epsilon}}^{V^{\epsilon}}\Big{\\{}\int_{0}^{\tau_{\cal{O}}}e^{-\alpha
s}f(Y_{s})ds+\sum_{i=1}^{\infty}1_{\tau_{i}<\tau_{\cal{O}}}e^{-\alpha\tau_{i}}c(x_{\tau_{i}}^{i},\xi_{i})\Big{\\}}\over\mathbb{E}_{x_{\epsilon}}^{V^{\epsilon}}\Big{\\{}\int_{0}^{\tau_{\cal{O}}}e^{-\alpha
s}ds\Big{\\}}}.$

Then

(9)
$0\leq\mathbb{E}_{x_{\epsilon}}^{V^{\epsilon}}\Big{\\{}\int_{0}^{\tau_{\cal{O}}}e^{-\alpha_{s}}(f(Y_{s})-\lambda_{\alpha})ds+\sum_{i=1}^{\infty}1_{\tau_{i}<\tau_{\cal{O}}}e^{-\alpha\tau_{i}}c(x_{\tau_{i}}^{i},\xi_{i})\Big{\\}}\leq\epsilon\mathbb{E}_{x_{\epsilon}}^{V^{\epsilon}}\left\\{\int_{0}^{\tau_{\cal{O}}}e^{-\alpha
s}ds\right\\}$

and since
$\mathbb{E}_{x_{\epsilon}}^{V^{\epsilon}}\left\\{\int_{0}^{\tau_{\cal{O}}}e^{-\alpha
s}ds\right\\}\leq{1\over\alpha}$ we complete the proof.

We characterize now the discounted value function $w_{\alpha}$.

######  Proposition 2.3.

Function $w_{\alpha}$ defined in (5) is a solution to the following Bellman
equation

(10)
$w_{\alpha}(x)=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\tau_{\cal{O}}}e^{-\alpha
s}(f(X_{s})-\lambda_{\alpha})ds+1_{\tau<\tau_{\cal{O}}}e^{-\alpha\tau}Mw_{\alpha}(X_{\tau})\right\\},$

where $Mv(x)=\inf_{\xi\in U}\left[c(x,\xi)+v(\xi)\right]$ for Borel measurable
function $v$. Furthermore $w_{\alpha}\in C(E)$,

(11)
$|w_{\alpha}(x)|\leq\|f-\lambda_{\alpha}\|\mathbb{E}_{x}\left\\{\tau_{\cal{O}}\right\\}$

and $|\lambda_{\alpha}|\leq\|f\|$.

######  Proof 2.4.

Let for $v\in C(E)$

(12)
$Fv(x):=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\tau_{\cal{O}}}e^{-\alpha
s}(f(X_{s})-\lambda_{\alpha})ds+1_{\tau<\tau_{\cal{O}}}e^{-\alpha\tau}Mv(X_{\tau})\right\\}.$

When $v(x)\geq 0$ for $x\in U$ then also $Mv(x)\geq 0$ for $x\in E$ and by
Theorem 4.1 we have that $Fv\in C(E)$. Consider a sequence
$w_{\alpha}^{0}(x)=\mathbb{E}_{x}\left\\{\int_{0}^{\tau_{\cal{O}}}e^{-\alpha
s}(f(X_{s})-\lambda_{\alpha})ds\right\\}$,
$w_{\alpha}^{1}(x)=Fw_{\alpha}^{0}(x)$,
$w_{\alpha}^{n+1}(x)=Fw_{\alpha}^{n}(x)$ for $n=1,\ldots$. One can show that

(13)
$w_{\alpha}^{n}(x)=\inf_{V^{n}}\mathbb{E}_{x}^{V^{n}}\Big{\\{}\int_{0}^{\tau_{\cal{O}}}e^{-\alpha
s}(f(Y_{s})-\lambda_{\alpha})ds+\sum_{i=1}^{n}1_{\tau_{i}<\tau_{\cal{O}}}e^{-\alpha\tau_{i}}c(x_{\tau_{i}}^{i},\xi_{i})\Big{\\}},$

where $V^{n}$ is an impulse strategy consisting of at most $n$ impulses.
Clearly $w_{\alpha}^{n}(x)$ is a decreasing sequence converging to
$w_{\alpha}(x)$.

Since

(14)
$w_{\alpha}^{n+1}(x)=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\tau_{\cal{O}}}e^{-\alpha
s}(f(X_{s})-\lambda_{\alpha})ds+1_{\tau<\tau_{\cal{O}}}e^{-\alpha\tau}Mw_{\alpha}^{n}(X_{\tau})\right\\}$

letting $n\to\infty$ we obtain (10). Notice that $\inf_{x\in
U}w_{\alpha}(x)=0$ and therefore $Mw_{\alpha}(x)\geq 0$ for $x\in E$ and
$Mw_{\alpha}\in C(E)$ and finally using Theorem 4.1 we have that
$w_{\alpha}\in C(E)$. Furthermore since $c(x,\xi)\geq 0$ we have

$\displaystyle
w_{\alpha}(x)\leq\inf_{V}\mathbb{E}_{x}^{V}\Big{\\{}\int_{0}^{\tau_{\cal{O}}}e^{-\alpha
s}\|f-\lambda_{\alpha}\|ds+\sum_{i=1}^{\infty}1_{\tau_{i}<\tau_{\cal{O}}}e^{-\alpha\tau_{i}}c(x_{\tau_{i}}^{i},\xi_{i})\Big{\\}}=$
(15) $\displaystyle\mathbb{E}_{x}\Big{\\{}\int_{0}^{\tau_{\cal{O}}}e^{-\alpha
s}\|f-\lambda_{\alpha}\|ds\Big{\\}}\leq\|f-\lambda_{\alpha}\|\mathbb{E}_{x}\left\\{\tau_{\cal{O}}\right\\}$

and similarly

$\displaystyle
w_{\alpha}(x)\geq\inf_{V}\mathbb{E}_{x}^{V}\Big{\\{}-\int_{0}^{\tau_{\cal{O}}}e^{-\alpha
s}\|f-\lambda_{\alpha}\|ds+\sum_{i=1}^{\infty}1_{\tau_{i}<\tau_{\cal{O}}}e^{-\alpha\tau_{i}}c(x_{\tau_{i}}^{i},\xi_{i})\Big{\\}}=$
(16) $\displaystyle\mathbb{E}_{x}\Big{\\{}-\int_{0}^{\tau_{\cal{O}}}e^{-\alpha
s}\|f-\lambda_{\alpha}\|ds\Big{\\}}\geq-\|f-\lambda_{\alpha}\|\mathbb{E}_{x}\left\\{\tau_{\cal{O}}\right\\},$

where in both approximations the infimum was obtained for a strategy without
impulses, so that we have (11). The fact that $|\lambda_{\alpha}|\leq\|f\|$
follows directly from (4).

We consider now undiscounted impulse control problem.

######  Theorem 2.5.

We have that

(17) $\lim_{\alpha\to 0}\lambda_{\alpha}=\lambda:=\inf_{V}\inf_{x\in
U}{J}^{\cal{O}}\big{(}x,(\tau_{i},\xi_{i})\big{)}$

and there is a function $w\in C(E)$ which is a solution to the equation

(18)
$w(x)=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\tau_{\cal{O}}}(f(X_{s})-\lambda)ds+1_{\tau<\tau_{\cal{O}}}Mw(X_{\tau})\right\\}$

such that $\inf_{x\in U}w(x)=0$.

######  Proof 2.6.

Since $|\lambda_{\alpha}|\leq\|f\|$ one can choose a subsequence
$\alpha_{n}\to 0$ and $\lambda$ such that $\lambda_{\alpha_{n}}\to\lambda$, as
$n\to\infty$. By (11) we know that $w_{\alpha}$ are uniformly (in $\alpha$)
bounded by a constant $L$ (since
$\sup_{x}\mathbb{E}_{x}\left\\{\tau_{\cal{O}}\right\\}<\infty$). Since
$|Mw_{\alpha}(x)-Mw_{\alpha}(y)|\leq\sup_{\xi\in U}|c(x,\xi)-c(y,\xi)|$ the
family $\left\\{Mw_{\alpha}(x),\alpha>0\right\\}$ is equicontinuous at each
point and bounded by $\|c\|+L$. Therefore there is a function $v\in C(E)$ and
a further subsequence of $\alpha_{n}\to 0$ for simplicity again denoted by
$\alpha_{n}$ such that $Mw_{\alpha_{n}}$ converges uniformly on compact sets
to $v$. Let

(19)
$\hat{w}_{\alpha}(x)=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\tau_{\cal{O}}}e^{-\alpha
s}(f(X_{s})-\lambda_{\alpha})ds+1_{\tau<\tau_{\cal{O}}}e^{-\alpha\tau}v(X_{\tau})\right\\}.$

For $x$ from a compact set $K$ and given $\varepsilon>0$ and $T>0$ such that
$\sup_{x\in E}{\mathbb{E}_{x}\left\\{\tau_{\cal{O}}\right\\}\over
T}\leq\varepsilon$ one can find by Proposition 2.1 of [19] $R>0$ such that

(20) $\sup_{x\in K}$

where $\rho$ is the metric on $E$. Since $B(K,R)=\left\\{x:\rho(x,K)\leq
R\right\\}$ is a compact set, for a sufficiently large $n$, say $n\geq n_{0}$
we have $\sup_{y\in B(K,R)}|Mw_{\alpha_{m}}(y)-v(y)|\leq\varepsilon$.
Therefore for $x\in K$

$\displaystyle|w_{\alpha_{n}}(x)-\hat{w}_{\alpha_{n}}(x)|\leq\sup_{\tau}\mathbb{E}_{x}\left\\{1_{\tau<\tau_{\cal{O}}}|Mw_{\alpha_{n}}(X_{\tau})-v(X_{\tau})|\right\\}\leq$
$\displaystyle\sup_{\tau}\mathbb{E}_{x}\left\\{1_{\tau<\tau_{\cal{O}}}1_{\tau_{\cal{O}}\leq
T}1_{X_{\tau}\in B(K,R)}\varepsilon+(1_{\tau_{\cal{O}}\geq T}+\right.$
$\displaystyle 1_{\tau<\tau_{\cal{O}}}1_{\tau_{\cal{O}}\leq
T}1_{X_{\tau}\notin B(K,R)})\left.(\|c\|+L)\right\\}$ (21)
$\displaystyle\leq\varepsilon(1+2(\|c\|+L)).$

Moreover since $|\lambda_{\alpha}|\leq\|f\|$ and $v$ is bounded we have

$\displaystyle|\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\tau_{\cal{O}}}e^{-\alpha
s}(f(X_{s})-\lambda_{\alpha})ds+1_{\tau<\tau_{\cal{O}}}e^{-\alpha\tau}v(X_{\tau})\right\\}-$
$\displaystyle\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\tau_{\cal{O}}}(f(X_{s})-\lambda_{\alpha})ds+1_{\tau<\tau_{\cal{O}}}v(X_{\tau})\right\\}|\leq$
$\displaystyle\sup_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\tau_{\cal{O}}}|e^{-\alpha
s}-1||f(X_{s})-\lambda_{\alpha}|ds+\right.$ (22)
$\displaystyle\left.1_{\tau<\tau_{\cal{O}}}|e^{-\alpha\tau}-1||v(X_{\tau})|\right\\}\to
0$

uniformly as $\alpha\to 0$. Since

$\displaystyle|\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\tau_{\cal{O}}}(f(X_{s})-\lambda_{\alpha})ds+1_{\tau<\tau_{\cal{O}}}v(X_{\tau})\right\\}-$
$\displaystyle\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\tau_{\cal{O}}}(f(X_{s})-\lambda)ds+1_{\tau<\tau_{\cal{O}}}v(X_{\tau})\right\\}|$
(23)
$\displaystyle\leq|\lambda_{\alpha}-\lambda|\mathbb{E}_{x}\left\\{\tau_{\cal{O}}\right\\}$

we finally have that $w_{\alpha_{n}}(x)$ converges uniformly on compact
subsets to $w(x)$ of the form

(24)
$w(x)=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\tau_{\cal{O}}}(f(X_{s})-\lambda)ds+1_{\tau<\tau_{\cal{O}}}v(X_{\tau})\right\\}.$

Then also $Mw_{\alpha_{n}}$ converges uniformly to $Mw$, which means that
$v(x)=Mw(x)$ and therefore $w$ is a solution to (18) and $w\in C(E)$. Since
$\inf_{x\in U}w_{\alpha}(x)=0$ we also have that $\inf_{x\in U}w(x)=0$.

Iterating (18) we obtain inductively for each positive integer $n$

$\displaystyle{w}(x)=\inf_{V^{n}}\mathbb{E}_{x}^{V^{n}}\left\\{\int_{0}^{\tau_{\cal{O}}\wedge\tau_{n}}(f(Y_{s})-\lambda)ds+\right.$
(25)
$\displaystyle\left.\sum_{i=1}^{n}1_{\tau_{i}<\tau_{\cal{O}}}c(x_{\tau_{i}}^{i},\xi_{i})+1_{\tau_{n}<\tau_{\cal{O}}}{w}(\tau_{n},\xi_{n})\right\\},$

where $V^{n}$ denotes impulse strategy consisting of at most $n$ impulses. We
restrict here practically to impulses which at time $\tau_{i+1}$ depend only
on the behavior of the controlled process after time $\tau_{i}$. Using the
procedure as in section 7 of [2] we can show that extension to general impulse
strategies (depending on the whole history) does not change the value of $w$.
Then similarly as in the proof of Lemma 2.1 for each $n$ we have

(26)
$\lambda=\inf_{V^{n}}{-w(x)+\mathbb{E}_{x}^{V^{n}}\left\\{\int_{0}^{\tau_{\cal{O}}\wedge\tau_{n}}f(Y_{s})ds+\sum_{i=1}^{n}1_{\tau_{i}<\tau_{\cal{O}}}c(x_{\tau_{i}}^{i},\xi_{i})+1_{\tau_{n}<\tau_{\cal{O}}}{w}(\xi_{n})\right\\}\over\mathbb{E}_{x}^{V^{n}}\left\\{\tau_{\cal{O}}\wedge\tau_{n}\right\\}}.$

Let for each $\varepsilon>0$ the strategy $V_{\varepsilon}^{n}$ be an
$\varepsilon$ optimal in (26) i.e.

(27)
$\lambda+\varepsilon\geq{-w(x)+\mathbb{E}_{x}^{V^{n}}\left\\{\int_{0}^{\tau_{\cal{O}}\wedge\tau_{n}}f(Y_{s})ds+\sum_{i=1}^{n}1_{\tau_{i}<\tau_{\cal{O}}}c(x_{\tau_{i}}^{i},\xi_{i})+1_{\tau_{n}<\tau_{\cal{O}}}{w}(\xi_{n})\right\\}\over\mathbb{E}_{x}^{V^{n}}\left\\{\tau_{\cal{O}}\wedge\tau_{n}\right\\}}.$

Then either

  * •

(i)
$\liminf_{n\to\infty}\mathbb{E}_{x}^{V_{\varepsilon}^{n}}\left\\{\tau_{\cal{O}}\wedge\tau_{n}\right\\}=\infty$
or

  * •

(ii)
$\liminf_{n\to\infty}\mathbb{E}_{x}^{V_{\varepsilon}^{n}}\left\\{\tau_{\cal{O}}\wedge\tau_{n}\right\\}<\infty$.

In the case $(i)$ taking into account that $w$ is bounded we have

(28)
$\lambda+\varepsilon\geq\liminf_{n\to\infty}{\mathbb{E}_{x}^{V_{\varepsilon}^{n}}\left\\{\int_{0}^{\tau_{\cal{O}}\wedge\tau_{n}}f(Y_{s})ds+\sum_{i=1}^{n}1_{\tau_{i}<\tau_{\cal{O}}}c(x_{\tau_{i}}^{i},\xi_{i})\right\\}\over\mathbb{E}_{x}^{V_{\varepsilon}^{n}}\left\\{\tau_{\cal{O}}\wedge\tau_{n}\right\\}}\geq\lambda.$

Let $N(0,\tau)$ be the number of impulses in the time interval $[0,\tau)$.
Then since $w$ is bounded and $c(x,\xi)\geq c>0$ we can restrict ourselves to
stopping times $\tau_{i}$ and strategy $V_{\varepsilon}^{n}$ such that for
each $n$

(29)
$\lambda+\varepsilon\geq-\|f\|+{-w(x)+c\mathbb{E}_{x}^{V_{\varepsilon}^{n}}\left\\{N(0,\tau_{\cal{O}}\wedge\tau_{n})\right\\}\over\mathbb{E}_{x}^{V_{\varepsilon}^{n}}\left\\{\tau_{\cal{O}}\wedge\tau_{n}\right\\}}$

Then in the case $(ii)$ we have that
$\mathbb{E}_{x}^{V_{\varepsilon}^{n}}\left\\{N(0,\tau_{\cal{O}}\wedge\tau_{n})\right\\}$
for a suitably chosen subsequence $n_{k}$ is bounded and therefore

(30)
$\mathbb{E}_{x}^{V^{n_{k}}}\left\\{1_{\tau_{n_{k}}<\tau_{\cal{O}}}{w}(\xi_{n_{k}})\right\\}\to
0$

as $k\to\infty$. Consequently for $\hat{x}$ such that $w(\hat{x})=0$ we have

(31)
$\lambda+\varepsilon\geq\liminf_{n\to\infty}{\mathbb{E}_{\hat{x}}^{V_{\varepsilon}^{n}}\left\\{\int_{0}^{\tau_{\cal{O}}\wedge\tau_{n}}f(Y_{s})ds+\sum_{i=1}^{n}1_{\tau_{i}<\tau_{\cal{O}}}c(x_{\tau_{i}}^{i},\xi_{i})\right\\}\over\mathbb{E}_{\hat{x}}^{V_{\varepsilon}^{n}}\left\\{\tau_{\cal{O}}\wedge\tau_{n}\right\\}}\geq\lambda.$

Summarizing (28) and (31) we obtain (17). Since $\lambda$ is defined in a
unique way in (17), any subsequence of $\lambda_{\alpha}$ converges to the
same value $\lambda$ equal to (17) therefore we have convergence of
$\lambda_{\alpha}$ to $\lambda$ as $\alpha\to 0$.

######  Remark 2.7.

Notice that it may happen that for an optimal impulse strategy we have
$\mathbb{E}_{x}^{V}\left\\{\tau_{\cal{O}}\right\\}=\infty$ as we can see in
the example below.

######  Example 2.8.

Let $E=[0,\infty)$, we have deterministic movement to the right $X_{t}=x+t$,
$f(x)=(f+x^{2})\wedge M$ with $M>\sqrt{c}$, $c(x,\xi)=c>0$ and $U=[0,1]$ and
${\cal{O}}=[0,M)$. An optimal strategy is to make shift to $0$ as soon as we
are at $\sqrt{c}$ or above it. Starting from $0$ we reach $\sqrt{c}$ at time
$\sqrt{c}$ and $\lambda=f+{c\over 3}+\sqrt{c}+{f\over\sqrt{c}}$. Controlled
process under this optimal strategy never exits $\cal{O}$.

##  3 Bellman equation

Denote by ${\cal{O}}_{m}$ an increasing sequence of open balls in $E$ with
radius $m$. In this section we shall need the following set of assumptions:

  1. (A.1)

Markov process $(X_{t})$ has a unique invariant probability measure $\mu$ and
there is a continuous solution $q$ to the additive Poisson equation (APE)
associated with $f$ such that for any bounded stopping time $\tau$ we have

(32)
$q(x)=\mathbb{E}_{x}\left\\{\int_{0}^{\tau}(f(X_{s})-\mu(f))ds+q(X_{\tau})\right\\},$

and additionally $q$ is bounded from above by a constant $K$ and $(q(X_{t}))$
is uniformly integrable.

  2. (A.2)

for each $m$ the stopped semigroup $T_{t}^{{\cal{O}}_{m}}$ transforms $C(E)$
into itself.

  3. (A.3)

for each $m$ for the first exit time $\tau_{{\cal{O}}_{m}}$ we have
$\sup_{x\in
E}\mathbb{E}_{x}\left\\{(\tau_{{\cal{O}}_{m}})^{2}\right\\}<\infty$,

  4. (A.4)

for each $T>0$ we have that converges to $0$ uniformly in $x$ from compact
sets, as $m\to\infty$.

######  Remark 3.1.

When there is a bounded on compact subset function $K$ and $\gamma>0$ such
that for $f\in C(E)$ we have
$|\mathbb{E}_{x}\left\\{f(X_{t})\right\\}-\mu(f)|\leq\|f\|K(x)e^{-\gamma t}$
then by Lemma 2.2 of [21] there is a continuous solution to (APE) and uniform
integrability of $q(X_{t})$ corresponds to uniform integrability of
$K(X_{t})$. Then

$z_{t}:=\int_{0}^{t}(f(X_{s})-\mu(f))ds+q(X_{t})$

is a uniformly integrable martingale. Let
$\Gamma=\left\\{x:f(x)\geq\mu(f)\right\\}$ be a compact set and suppose that
for each $x\in E$ we have $\mathbb{E}_{x}\left\\{T_{\Gamma}\right\\}<\infty$,
where $T_{\Gamma}=\inf\left\\{s\geq 0:X_{s}\in\Gamma\right\\}$. Then for
$x\notin\Gamma$

(33)
$q(x)=\mathbb{E}_{x}\left\\{\int_{0}^{T_{\Gamma}}(f(X_{s})-\mu(f))ds+q(X_{T_{\Gamma}})\right\\}\leq\mathbb{E}_{x}\left\\{q(X_{T_{\Gamma}})\right\\}$

and by continuity $q$ is bounded from above. More general sufficient
conditions for existence of solutions to (APE) are formulated in Theorem 2.3
of [9]. Sufficient conditions for ((A.1)) were studied in Lemma 2.2 and Lemma
3.18 in [21].

Let following Theorem 2.5

(34) $\lambda_{(m)}=\inf_{x\in
U}\inf_{V}\liminf_{T\to\infty}{\mathbb{E}_{x}^{V}\Big{\\{}\int_{0}^{\tau_{{\cal{O}}_{m}}\wedge
T}f(Y_{s})ds+\sum_{i=1}^{\infty}1_{\tau_{i}<\tau_{\tau_{{\cal{O}}_{m}}\wedge
T}}c(x_{\tau_{i}}^{i},\xi_{i})\Big{\\}}\over\mathbb{E}_{x}^{V}\Big{\\{}\tau_{{\cal{O}}_{m}}\wedge
T\Big{\\}}},$ (35)
$w^{m}(x)=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau_{{\cal{O}}_{m}}\wedge\tau}(f(X_{s})-\lambda_{(m)})ds+1_{\tau<\tau_{{\cal{O}}_{m}}}Mw^{m}(x_{\tau})\right\\}.$

and $\inf_{\xi\in U}w^{m}(\xi)=0$.

######  Remark 3.2.

There is a usual problem how to understand

$\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau}g(X_{s})ds+h(X_{\tau})\right\\}$

for $g,h\in C(E)$. We can consider it as

$\liminf_{T\to\infty}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge
T}g(X_{s})ds+h(X_{\tau\wedge T})\right\\},$

which coincides with
$\mathbb{E}_{x}\left\\{\int_{0}^{\tau}g(X_{s})ds+h(X_{\tau})\right\\}$ with
random non necessarily bounded $\tau$, when we assume that
$\mathbb{E}_{x}\left\\{\tau\right\\}<\infty$ and take into account quasi
leftcontinuity of $(X_{t})$ (see Theorem 3.13 of [11]). Consequently if we
know that for optimal stopping times we can restrict ourselves to stopping
times $\tau$ such that $\mathbb{E}_{x}\left\\{\tau\right\\}\leq M$ then
$\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau}g(X_{s})ds+h(X_{\tau})\right\\}$
is the same as the infimum over all bounded stopping times.

We have the following main result

######  Theorem 3.3.

Under ((A.1))-((A.4)) for every convergent subsequence of $\lambda_{(m_{k})}$
such that $\lim\lambda_{(m_{k})}=\lambda<\mu(f)$ we have that

(36) $\lambda=\inf_{V}{J}\big{(}x,V\big{)}$

and there is a continuous bounded function $w$ such that $\inf_{\xi\in
U}w(\xi)=0$ and

(37)
$w(x)=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau}(f(X_{s})-\lambda)ds+Mw(X_{\tau})\right\\}.$

Furthermore (37) defines an optimal strategy
$\hat{V}=(\hat{\tau}_{i},\hat{\xi}_{i})$ for $J$:

$\hat{\tau}_{1}=\inf\left\\{s\geq 0:w(Y_{s})=Mw(Y_{s})\right\\}$,
$\hat{\tau}_{n+1}=\hat{\tau}_{n}+\hat{\tau}_{1}\circ\theta_{\hat{\tau}_{n}}$,
where $\theta_{t}$ is a shift operator corresponding to the controlled process
$(Y_{t})$ and $\hat{\xi}_{n}=\operatorname*{arg\,min}_{z\in
U}(c(x_{\hat{\tau}_{n}}^{n},z)+w(z)$.

######  Proof 3.4.

Let $w^{m}$ be a solution to Bellman equation (35) (which exists by Theorem
2.5). Assume that $\lambda_{(m_{k})}\to\lambda\leq\mu(f)-a$, with $a>0$, as
$k\to\infty$. Since $M{w}^{m_{k}}(x)\leq\|c\|+\inf_{z\in
U}{w}^{m_{k}}(z)=\|c\|$ and $M{w}^{m_{k}}(x)\geq c+\inf_{z\in
U}{w}^{m_{k}}(z)=c>0$ as well as
$|M{w}^{m_{k}}(x)-M{w}^{m_{k}}(y)|\leq\sup_{\xi\in U}|c(x,\xi)-c(y,\xi)|$ the
family of functions $\left(M{w}^{m_{k}}\right)$ is bounded and uniformly
continuous. Therefore there is a further subsequence, for simplicity still
denoted by $m_{k}$ and a function $v\in C(E)$ such $M{w}^{m_{k}}$ converges
uniformly on compact subsets to $v$ such that $c\leq v\leq\|c\|$. By ((A.1))
and the fact that $M{w}^{m}\geq 0$

$\displaystyle{w}^{m}(x)=\inf_{\tau}\mathbb{E}_{x}\left\\{(\tau_{{\cal{O}}_{m}}\wedge\tau)(\mu(f)-\lambda_{(m)})-q(X_{\tau_{{\cal{O}}_{m}}\wedge\tau})+\right.$
$\displaystyle\left.1_{\tau<\tau_{{\cal{O}}_{m}}}M{w}^{m}(x_{\tau})\right\\}+q(x)\geq
q(x)+$ (38)
$\displaystyle\inf_{\tau}\mathbb{E}_{x}\left\\{(\tau_{{\cal{O}}_{m}}\wedge\tau)(\mu(f)-\lambda_{(m)})-q(X_{\tau_{{\cal{O}}_{m}}\wedge\tau})\right\\},$

so that assuming that $\lambda_{m}<\mu(f)$, taking into account that
${w}^{m}(x)\leq M{w}^{m}(x)\leq\|c\|$ and $q(\cdot)\leq K$, in infimum in
(3.4) we can restrict ourselves to stopping times $\tau$ such that

(39)
$\mathbb{E}_{x}\left\\{(\tau_{{\cal{O}}_{m}}\wedge\tau)\right\\}\leq{\|c\|-q(x)+K\over\mu(f)-\lambda_{m}}.$

Consequently for a sufficiently large $k$, say $k\geq k_{0}$ we see that we
can restrict ourselves in $w^{m}$ to stopping times $\tau$ such that

(40)
$\mathbb{E}_{x}\left\\{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau\right\\}\leq{\|c\|-q(x)+K\over
a}.$

Let

(41)
$z_{m}(x)=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau_{{\cal{O}}_{m}}\wedge\tau}(f(X_{s})-\lambda)ds+v(X_{\tau_{{\cal{O}}_{m}}\wedge\tau})\right\\}.$

By the same arguments as above we can restrict in (41) to stopping times such
that (40) is satisfied. Clearly $z_{m}(x)$ is a nonincreasing sequence and let
$z(x):=\lim_{m\to\infty}z_{m}(x)$. We are going to show that $z(x)$ is of the
form

(42)
$z(x)=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau}(f(X_{s})-\lambda)ds+v(X_{\tau})\right\\}.$

We also can restrict in (42) to stopping times such that

(43) $\mathbb{E}_{x}\left\\{\tau\right\\}\leq{\|c\|-q(x)+K\over a}.$

Therefore there is an $\epsilon$-optimal stopping time $\tau^{*}_{\epsilon}$
for (42) such that for some deterministic $T$ we have $\tau^{*}_{\epsilon}\leq
T$, a.s.. Then

$\displaystyle 0\leq
z_{m}(x)-z(x)\leq\mathbb{E}_{x}\left\\{\int_{0}^{\tau_{{\cal{O}}_{m}}\wedge{\tau^{*}_{\epsilon}}}(f(X_{s})-\lambda)ds+v(X_{\tau_{{\cal{O}}_{m}}\wedge{\tau^{*}_{\epsilon}}})\right\\}-$
$\displaystyle\mathbb{E}_{x}\left\\{\int_{0}^{{\tau^{*}_{\epsilon}}}(f(X_{s})-\lambda)ds+v(X_{{\tau^{*}_{\epsilon}}})\right\\}+\epsilon\leq$
$\displaystyle\mathbb{E}_{x}\left\\{{\tau^{*}_{\epsilon}}-\tau_{{\cal{O}}_{m}}\wedge{\tau^{*}_{\epsilon}}\right\\}\|f-\lambda\|+\|c\|$
(44) $\displaystyle\leq$

which by ((A.4)) converges to $\epsilon$. Consequently $z$ is of the form
(42). By Theorem 4.4 we have that $z\in C(E)$ and since by Proposition 4.2
functions $z_{m}$ are continuous, using Dini’s lemma we have that $z_{m}$
converges uniformly on compact subsets to $z$. We shall evaluate now the
difference between ${w}^{m_{k}}(x)$ and $z_{m_{k}}(x)$. Using ((A.1)) we see
that we can restrict ourselves in $z_{m_{k}}$ to stopping times $\tau$ such
that (40) is satisfied. For such stopping times we have for $k\geq k_{0}$

$\displaystyle|\mathbb{E}_{x}\left\\{\int_{0}^{\tau_{{\cal{O}}_{m}}\wedge\tau}(f(X_{s})-\lambda_{(m_{k})})ds\right\\}-\mathbb{E}_{x}\left\\{\int_{0}^{\tau_{{\cal{O}}_{m}}\wedge\tau}(f(X_{s})-\lambda)ds\right\\}|\leq$
(45)
$\displaystyle|\mathbb{E}_{x}\left\\{(\tau_{{\cal{O}}_{m}}\wedge\tau)\right\\}|\lambda_{(m_{k})}-\lambda|\leq{\|c\|-q(x)+K\over
a}|\lambda_{(m_{k})}-\lambda|.$

For stopping times $\tau$ satisfying (40) we also have (recall that $c\leq
M\bar{w}^{m_{k}}\leq\|c\|$)

$\displaystyle|\mathbb{E}_{x}\left\\{1_{\tau<\tau_{{\cal{O}}_{m_{k}}}}M{w}^{m_{k}}(X_{\tau})\right\\}-\mathbb{E}_{x}\left\\{v(X_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau})\right\\}|$
$\displaystyle\leq\mathbb{E}_{x}\left\\{1_{\tau\geq\tau_{{\cal{O}}_{m_{k}}}}\right\\}\|c\|+$
$\displaystyle|\mathbb{E}_{x}\left\\{M{w}^{m_{k}}(X_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau})-v(X_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau})\right\\}|\leq$
$\displaystyle\mathbb{E}_{x}\left\\{1_{\tau\geq\tau_{{\cal{O}}_{m_{k}}}}1_{\tau_{{\cal{O}}_{m_{k}}}\leq
T}\right\\}\|c\|+\mathbb{E}_{x}\left\\{1_{\tau\geq\tau_{{\cal{O}}_{m_{k}}}}1_{\tau_{{\cal{O}}_{m_{k}}}>T}\right\\}\|c\|+$
$\displaystyle|\mathbb{E}_{x}\left\\{1_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau\leq
T}(M{w}^{m_{k}}(X_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau})-v(X_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau}))\right\\}|+$
$\displaystyle\mathbb{E}_{x}\left\\{1_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau\geq
T}\|c\|\right\\}\leq\mathbb{E}_{x}\left\\{1_{\tau_{{\cal{O}}_{m_{k}}}\leq
T}\right\\}\|c\|+$
$\displaystyle\mathbb{E}_{x}\left\\{1_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau\geq
T}\right\\}2\|c\|+$
$\displaystyle|\mathbb{E}_{x}\left\\{1_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau\leq
T}1_{\rho(x,X_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau})\leq
R}(M{w}^{m_{k}}(X_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau})-\right.$
$\displaystyle\left.v(X_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau})\right\\})|+\mathbb{E}_{x}\left\\{1_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau\leq
T}1_{\rho(x,X_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau})\geq R}\right\\}\|c\|$ (46)
$\displaystyle=b_{k}+e_{k}+g_{k}+h_{k},$

where
$e_{k}\leq{\mathbb{E}_{x}\left\\{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau\right\\}\|c\|\over
T}$ and $b_{k}$ converges to $0$ uniformly on compact sets by ((A.4)). For $x$
from a compact set $K$ and given $\varepsilon>0$ and $T>0$ one can find by
Proposition 2.1 of [19] $R>0$ such that

(47) $\sup_{x\in K}$

where $\rho$ is the metric on $E$. Therefore for $T$ sufficiently large
$e_{k}\leq\epsilon$ for all $k$ and $x\in K$ and for fixed $T$ for large
enough $k$ we have that $b_{k}\leq\epsilon$ for $x$ from $K$. From (47) we
have that $h_{k}\leq\varepsilon\|c\|$. Finally for $x\in K$ there is $k_{1}$
such that for $k\geq k_{1}$ on the set
$\left\\{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau\leq
T\right\\}\cap\left\\{\rho(x,X_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau})\leq
R\right\\}$ we have
$|M{w}^{m_{k}}(X_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau})-v(X_{\tau_{{\cal{O}}_{m_{k}}}\wedge\tau})|\leq\varepsilon$.
Therefore for $x\in K$ and sufficiently large $k$ we can make
$b_{k}+e_{k}+g_{k}+h_{k}$ arbitrarily small.

Summarizing (3.4) and (3.4) we obtain that $|{w}^{m_{k}}(x)-z_{m_{k}}(x)|$
converges to $0$ uniformly on compact sets. Consequently ${w}^{m_{k}}(x)$
converges to $z(x)$ uniformly on compact sets and additionally $\inf_{\xi\in
U}z(\xi)=0$. Therefore $M{w}_{m_{k}}(x)$ converges uniformly to $Mz(x)=v(x)$
and

(48)
$z(x)=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau}(f(X_{s})-\lambda)ds+Mz(X_{\tau})\right\\},$

which completes the proof of (37) with $z\equiv w$. Iterating the last formula
we obtain inductively for each positive integer $n$

(49)
${w}(x)=\inf_{V^{n}}\mathbb{E}_{x}^{V^{n}}\left\\{\int_{0}^{\tau_{n}}(f(Y_{s})-\lambda)ds+\sum_{i=1}^{n}c(x_{\tau_{i}}^{i},\xi_{i})+{w}(\xi_{n})\right\\}.$

where $V^{n}$ denotes impulse strategy consisting of at most $n$ impulses. In
fact, we have first (49) for so called shifted impulse strategies (see the
definition in section 2 of [2]), which then can be extended using technics of
section 7 of [2] to any impulse strategies consisting of $n$ impulses. Notice
that we can restrict ourselves to stopping times such that (43) is satisfied.
Then

(50)
$\lambda=\inf_{V^{n}}{1\over\mathbb{E}_{x}^{V^{n}}\left\\{\tau_{n}\right\\}}\mathbb{E}_{x}^{V^{n}}\left\\{\int_{0}^{\tau_{n}}f(Y_{s})ds+\sum_{i=1}^{n}c(x_{\tau_{i}}^{i},\xi_{i})+{w}(\xi_{n})\right\\}$

and then for any impulse strategy $V$ (since $w$ is bounded on $U$)

(51)
$\lambda\leq\liminf_{n\to\infty}{1\over\mathbb{E}_{x}^{V}\left\\{\tau_{n}\right\\}}\mathbb{E}_{x}^{V}\left\\{\int_{0}^{\tau_{n}}f(Y_{s})ds+\sum_{i=1}^{n}c(x_{\tau_{i}}^{i},\xi_{i})\right\\},$

with equality for an optimal impulse strategy $\hat{V}$ defined by (37). This
completes the proof of (36).

######  Remark 3.5.

Note first that by ((A.4)) we have that

(52) $\mathbb{E}_{x}\Big{\\{}\tau_{{\cal{O}}_{m}}\Big{\\}}\to\infty$

as $m\to\infty$. In fact,
$\mathbb{E}_{x}\Big{\\{}\tau_{{\cal{O}}_{m}}\Big{\\}}\geq T$. Letting
$m\to\infty$, using ((A.4)) we obtain
$\liminf_{m\to\infty}\mathbb{E}_{x}\Big{\\{}\tau_{{\cal{O}}_{m}}\Big{\\}}\geq
T$ for any $T>0$ form which (52) follows. Then for no impulse strategy we have
using ((A.1))

(53)
${\mathbb{E}_{x}\Big{\\{}\int_{0}^{\tau_{{\cal{O}}_{m}}}f(X_{s})ds\Big{\\}}\over{\mathbb{E}_{x}\Big{\\{}\tau_{{\cal{O}}_{m}}\Big{\\}}}}={q(x)-\mathbb{E}_{x}\left\\{q(X_{\tau_{{\cal{O}}_{m}}})\right\\}\over{\mathbb{E}_{x}\Big{\\{}\tau_{{\cal{O}}_{m}}\Big{\\}}}}+\mu(f).$

Letting $m\to\infty$, taking into account that $q$ is bounded from above we
obtain

(54)
$\liminf_{m\to\infty}{\mathbb{E}_{x}\Big{\\{}\int_{0}^{\tau_{{\cal{O}}_{m}}}f(X_{s})ds\Big{\\}}\over{\mathbb{E}_{x}\Big{\\{}\tau_{{\cal{O}}_{m}}\Big{\\}}}}\geq\mu(f)$

When $q$ is bounded we have equality in (54) and in Theorem 3.3 we can write
$\lim_{m\to\infty}\lambda_{m}=\lambda\leq\mu(f)$, and when $\lambda<\mu(f)$ we
have (36) and (37).

We can now formulate a result concerning the functional (2).

######  Proposition 3.6.

Under the assumptions of Theorem 3.3 for any impulse strategy $V$ we have that

(55)
$\lambda\leq\liminf_{T\to\infty}\frac{1}{T}\mathbb{E}_{x}^{V}\Big{\\{}\int_{0}^{T}f(Y_{s})ds+\sum_{i=1}^{\infty}1_{\tau_{i}\leq
T}c(x_{\tau_{i}}^{i},\xi_{i})\Big{\\}}.$

If for the strategy $\hat{V}$ defined in Theorem 3.3 we have that
$\lim_{T\to\infty}{1\over
T}\mathbb{E}_{x}^{\hat{V}}\left\\{w(Y_{T})\right\\}=0$ then

(56) $\lambda=\inf_{V}\hat{J}(x,V)=\hat{J}(x,\hat{V}).$

######  Proof 3.7.

Iterating (37) for any $T>0$ we have

(57) $w(x)\leq\mathbb{E}_{x}^{V}\left\\{\int_{0}^{\tau_{n}\wedge
T}(f(Y_{s})-\lambda)ds+\sum_{i=1}^{n}1_{\tau_{i}\leq
T}c(x_{\tau_{i}}^{i},\xi_{i})+w(\tau_{n}\wedge Y_{T})\right\\}$

with equality for the strategy $\hat{V}$. Taking into account that any nearly
optimal strategy $V$ consists of at most a finite number of impulses on the
time interval $[0,T]$ we obtain for any such strategy $V$

(58)
$w(x)\leq\mathbb{E}_{x}^{V}\left\\{\int_{0}^{T}(f(Y_{s})-\lambda)ds+\sum_{i=1}^{\infty}1_{\tau_{i}\leq
T}c(x_{\tau_{i}}^{i},\xi_{i})+w(Y_{T})\right\\}$

with equality for the strategy $\hat{V}$. Then since $w(Y_{T})\leq\|c\|$ we
have

$\displaystyle\lambda\leq\liminf_{T\to\infty}{1\over
T}\mathbb{E}_{x}^{V}\left\\{\int_{0}^{T}f(Y_{s})ds+\sum_{i=1}^{\infty}1_{\tau_{i}\leq
T}c(x_{\tau_{i}}^{i},\xi_{i})+w(Y_{T})\right\\}$ (59)
$\displaystyle\leq\liminf_{T\to\infty}{1\over
T}\mathbb{E}_{x}^{V}\left\\{\int_{0}^{T}f(Y_{s})ds+\sum_{i=1}^{\infty}1_{\tau_{i}\leq
T}c(x_{\tau_{i}}^{i},\xi_{i})\right\\}$

and (55) follows. For the strategy $\hat{V}$ when $\lim_{T\to\infty}{1\over
T}\mathbb{E}_{x}^{\hat{V}}\left\\{w(Y_{T})\right\\}=0$ we obtain equality in
(3.7), which completes the proof of (55).

######  Corollary 3.8.

Under the assumptions of Theorem 3.3 assuming additionally that the set
$G=\left\\{x:f(x)\leq\lambda\right\\}$ is compact we have that the function
$w$ is bounded and consequently we have (56).

######  Proof 3.9.

We use an analog of (75), namely

(60) $w(x)=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge
T_{G}}(f(X_{s})-\lambda)ds+1_{\tau<T_{G}}Mw(X_{\tau})+1_{T_{G}\leq\tau}w(X_{T_{G}})\right\\}.$

Then since $Mw(x)\geq c>0$

(61) $w(x)\geq
c+\inf_{\tau}\mathbb{E}_{x}\left\\{1_{T_{G}\leq\tau}w(X_{T_{G}})\right\\},$

which is bounded from below by continuity of $w$ and compactness of $G$. Since
$w\leq\|c\|$ function $w$ is bounded.

##  4 Appendix

We recall Theorem 4.3 of [20] (formulated there for supremum of stopping
times)

######  Theorem 4.1.

Assume that $f,G,H\in C(E)$, $G\geq H$ and $\alpha>0$. Under the assumptions
that $T_{t}C_{0}(E)\subset C_{0}(E)$, $T_{t}^{\cal{O}}C(E)\subset C(E)$ we
have that

(62) $\displaystyle w_{\alpha}(x)$ $\displaystyle:=$
$\displaystyle\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\tau_{\cal{O}}}e^{-\alpha
s}f(X_{s})ds+1_{\tau<\tau_{\cal{O}}}e^{-\alpha\tau}G(X_{\tau})+\right.$
$\displaystyle\left.1_{\tau\geq\tau_{\cal{O}}}e^{-\alpha\tau_{\cal{O}}}H(X_{\tau_{\cal{O}}})\right\\}$

is continuous and bounded and an optimal stopping time $\tau^{*}_{\alpha}$ is
given by the formula

(63) $\tau^{*}_{\alpha}=\inf\left\\{s\geq 0:w_{\alpha}(X_{s})\geq G(X_{s})\ \
or\ \ X_{s}\notin\cal{O}\right\\}.$

For an undiscounted case we have

######  Proposition 4.2.

Under the assumptions of Theorem 4.1 and $\sup_{x\in
E}\mathbb{E}_{x}\left\\{(\tau_{\cal{O}})^{2}\right\\}<\infty$ we have that
$w_{\alpha}$ converges uniformly to $w$, as $\alpha\to 0$, where

(64)
$w(x):=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\tau_{\cal{O}}}f(X_{s})ds+1_{\tau<\tau_{\cal{O}}}G(X_{\tau})+1_{\tau\geq\tau_{\cal{O}}}H(X_{\tau_{\cal{O}}})\right\\}$

is continuous bounded and an optimal stopping time $\tau^{*}$ is given by the
formula

(65) $\tau^{*}=\inf\left\\{s\geq 0:w(X_{s})\geq G(X_{s})\ \ or\ \
X_{s}\notin\cal{O}\right\\}.$

######  Proof 4.3.

Since $1-e^{-\alpha s}\leq\alpha s$, we have

(66) $\displaystyle|w(x)-w_{\alpha}(x)|$ $\displaystyle\leq$
$\displaystyle\mathbb{E}_{x}\left\\{\int_{0}^{\tau_{\cal{O}}}(1-e^{-\alpha
s})\|f\|ds+(1-e^{-\alpha\tau_{\cal{O}}})(\|G\|\vee\|H\|)\leq\right.$
$\displaystyle\left.{\alpha\over
2}\mathbb{E}_{x}\left\\{(\tau_{\cal{O}})^{2}\right\\}\|f\|+\alpha\mathbb{E}_{x}\left\\{\tau_{\cal{O}}\right\\}(\|G\|\vee\|H\|)\right\\}$

from which the convergence and continuity of $w$ follows. To show optimality
of $\tau^{*}$ recall so called penalty equation corresponding to $w_{\alpha}$,
see (4) of [20]:

(67)
$w_{\alpha}^{\beta}(x)=\mathbb{E}_{x}\left\\{\int_{0}^{\tau_{\cal{O}}}e^{-\alpha
s}\left(f(X_{s})+\beta(w_{\alpha}^{\beta}(X_{s})-G(X_{s}))^{+}\right)ds+e^{-\alpha\tau_{\cal{O}}}H(X_{\tau_{\cal{O}}})\right\\}.$

By Lemma 2.2 of [20] such function $w_{\alpha}^{\beta}$ exists and by
Proposition 2.9 of [20] it decreases to $w_{\alpha}$. For any stopping time
$\tau$ we can write

(68) $\displaystyle w_{\alpha}^{\beta}(x)$ $\displaystyle=$
$\displaystyle\mathbb{E}_{x}\left\\{\int_{0}^{\tau_{\cal{O}}\wedge\tau}e^{-\alpha
s}\left(f(X_{s})+\beta(w_{\alpha}^{\beta}(X_{s})-G(X_{s}))^{+}\right)ds+1_{\tau<\tau_{\cal{O}}}e^{-\alpha\tau}w_{\alpha}^{\beta}(X_{\tau})+\right.$
$\displaystyle\left.1_{\tau_{\cal{O}}\leq\tau}e^{-\alpha\tau_{\cal{O}}}H(X_{\tau_{\cal{O}}})\right\\}.$

By Theorem 4.3 we know that

(69) $\tau_{\alpha}^{*}=\inf\left\\{s\geq 0:w_{\alpha}(X_{s})\geq G(X_{s})\
or\ X_{s}\notin\cal{O}\right\\}$

is an optimal time for $w_{\alpha}$. Let $\tau^{\epsilon}=\inf\left\\{s\geq
0:w(X_{s})+\epsilon\geq G(X_{s})\ or\ X_{s}\notin\cal{O}\right\\}$. When
$\|w_{\alpha}-w\|\leq\epsilon$ we have that whenever $w_{\alpha}(X_{s})\geq
G(X_{s})$ then also $w(X_{s})+\epsilon\geq G(X_{s})$ which means that
$\tau^{\epsilon}\leq\tau_{\alpha}^{*}$. Consequently from (68) we obtain

(70)
$w_{\alpha}^{\beta}(x)=\mathbb{E}_{x}\left\\{\int_{0}^{\tau_{\cal{O}}\wedge\tau^{\epsilon}}e^{-\alpha
s}f(X_{s})ds+1_{\tau^{\epsilon}<\tau_{\cal{O}}}e^{-\alpha\tau^{\epsilon}}w_{\alpha}^{\beta}(X_{\tau^{\epsilon}})+1_{\tau_{\cal{O}}\leq\tau^{\epsilon}}e^{-\alpha\tau_{\cal{O}}}H(X_{\tau_{\cal{O}}})\right\\}.$

Letting in the last equation $\beta\to\infty$ and then $\alpha\to 0$ we
finally obtain

(71)
$w(x)=\mathbb{E}_{x}\left\\{\int_{0}^{\tau_{\cal{O}}\wedge\tau^{\epsilon}}f(X_{s})ds+1_{\tau^{\epsilon}<\tau_{\cal{O}}}w(X_{\tau^{\epsilon}})+1_{\tau_{\cal{O}}\leq\tau^{\epsilon}}H(X_{\tau_{\cal{O}}})\right\\}.$

It remains now to notice that $\tau^{\epsilon}$ with $\epsilon\to 0$ form an
increasing sequence of stopping times which by quasileftcontinuity of
$(X_{s})$ (see Theorem 3.13 of [11]) converges to $\tau^{*}$ and therefore by
continuity of $w$ we obtain

(72)
$w(x)=\mathbb{E}_{x}\left\\{\int_{0}^{\tau_{\cal{O}}\wedge\tau^{*}}f(X_{s})ds+1_{\tau^{*}<\tau_{\cal{O}}}w(X_{\tau^{*}})+1_{\tau_{\cal{O}}\leq\tau^{*}}H(X_{\tau_{\cal{O}}})\right\\}.$

completing this way the proof of optimality of $\tau^{*}$.

We finally formulate an ergodic stopping result

######  Theorem 4.4.

Under ((A.1)) let $f,w\in C(E)$ and $\mu(f)>0$. Then

(73)
$z(x):=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau}f(X_{s})ds+w(X_{\tau})\right\\}$

is continuous and an optimal stopping time $\tau^{*}$ is of the form

(74) $\tau^{*}=\inf\left\\{s\geq 0:z(X_{s})\geq w(X_{s})\right\\}.$

Furthermore for any stopping time $\sigma$ we have the following version of
Bellman equation

(75)
$z(x)=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\sigma}f(X_{s})ds+1_{\tau<\sigma}w(X_{\tau})+1_{\sigma\leq\tau}z(X_{\sigma})\right\\}.$

######  Proof 4.5.

By ((A.1)) we can write

(76)
$z(x)=q(x)+\inf_{\tau}\mathbb{E}_{x}\left\\{\mu(f)\tau+w(X_{\tau})-q(X_{\tau})\right\\}.$

Since $q(x)\leq K$ for $x\in E$, to show continuity of $z$ it suffices to
prove continuity of the function

(77) $v(x)=\inf_{\tau}\mathbb{E}_{x}\left\\{\mu(f)\tau+g(X_{\tau})\right\\},$

where $g(x)=-q(x)+K+w(x)+\|w\|\geq 0$. To show continuity of $v$ we consider
discrete time problem

(78) $v_{\delta}(x)=\inf_{\tau\in{\cal
T}_{\delta}}\mathbb{E}_{x}\left\\{\mu(f)\tau+g(X_{\tau})\right\\},$

where ${\cal T}_{\delta}$ is a family of stopping times taking values in
$\left\\{0,\delta,\ldots,n\delta,\ldots\right\\}$. Consider Bellman equation
corresponding to (78). We are looking for a function $r$ such that

(79) $r(x)=(\mu(f)\delta+P_{\delta}r(x))\wedge g(x):=T_{\delta}r(x).$

Notice that by ((A.1)) function $q$ is continuous and for each $n$ also
$P_{\delta}^{n}q$ is continuous. Consider two sequences $(\underline{r}_{n})$,
$(\bar{r}_{n})$ of functions approximating solutions to (79) defined as
follows $\underline{r}_{1}(x)=T_{\delta}(0)(x)$,
$\underline{r}_{2}(x)=T_{\delta}\underline{r}_{1}(x)$,
$\bar{r}_{1}(x)=T_{\delta}g(x)$, $\bar{r}_{2}(x)=T_{\delta}\bar{r}_{1}(x)$ and
inductively $\underline{r}_{n+1}(x)=T_{\delta}\underline{r}_{n}(x)$ and
$\bar{r}_{n+1}(x)=T_{\delta}\bar{r}_{n}(x)$ for positive integer $n$. Now,
$\underline{r}_{1}(x)\geq 0$ and $\bar{r}_{1}(x)\leq g(x)$ and by monotonicity
of $T_{\delta}$, we have $\underline{r}_{n+1}(x)\geq\underline{r}_{n}(x)$ and
$\bar{r}_{n+1}(x)\leq T_{\delta}\bar{r}_{n}(x)$ so that $\underline{r}_{n}(x)$
is a nondecreasing sequence while $\bar{r}_{n}(x)$ is a nonincreasing sequence
of continuous functions. Let
$\underline{r}(x)=\lim_{n\to\infty}\underline{r}_{n}(x)$ and
$\bar{r}(x)=\lim_{n\to\infty}\bar{r}_{n}(x)$. Clearly $\underline{r}$ is lower
semicontinuous while $\bar{r}$ is upper semicontinuous. Moreover one can show
that

(80) $\underline{r}_{n}(x)=\inf_{\tau\in{\cal
T}_{\delta}}\mathbb{E}_{x}\left\\{\mu(f)(\tau\wedge
n\delta)+1_{\tau<n\delta}g(X_{\tau})\right\\}$

and

(81) $\bar{r}_{n}(x)=\inf_{\tau\in{\cal
T}_{\delta}}\mathbb{E}_{x}\left\\{\mu(f)(\tau\wedge n\delta)+g(X_{\tau\wedge
n\delta})\right\\}.$

In both problems (80) and (81) we can restrict ourselves to stopping times
$\tau$ such that

(82) $\mathbb{E}_{x}\left\\{\tau\wedge n\delta\right\\}\leq{g(x)\over\mu(f)},$

so that letting $n\to\infty$, by Fatou lemma we have

(83) $\mathbb{E}_{x}\left\\{\tau\right\\}\leq{g(x)\over\mu(f)}.$

Therefore choosing $\varepsilon$-optimal stopping times for
$\underline{r}_{n}$ satisfying (83) we have

(84)
$0\leq\bar{r}_{n}(x)-\underline{r}_{n}(x)\leq\varepsilon+\mathbb{E}_{x}\left\\{1_{\tau\geq
n\delta}g(X_{n\delta})\right\\}$

and since by (83) using uniform integrability of $q(X_{n\delta})$ and
therefore also of $g(X_{n\delta})$ we obtain that
$\underline{r}(x)=\bar{r}(x)$ is continuous and coincides with
$v_{\delta}(x)$. Let now $\tau$ a $\varepsilon$-optimal stopping time for
$v(x)$. We can restrict ourselves to stopping time $\tau$ such that (83) is
satisfied. Let

$\tau_{\delta}=\inf\left\\{(n+1)\delta:n\delta<\tau\leq(n+1)\delta,n=0,1,\ldots\right\\}$
and $\tau_{\delta}=0$, whenever $\tau=0$. Then

$\displaystyle 0\leq v_{\delta}(x)-v(x)\leq\varepsilon+\mu(f)\delta+$
$\displaystyle\mathbb{E}_{x}\left\\{w(X_{\tau_{\delta}})-w(X_{\tau})+q(X_{\tau})-q(X_{\tau_{\delta}})\right\\}\leq$
(85)
$\displaystyle\varepsilon+(\mu(f)+2\|f\|)\delta+\mathbb{E}_{x}\left\\{1_{\tau\leq
T}(w(X_{\tau_{\delta}})-w(X_{\tau}))+1_{\tau>T}2\|w\|\right\\}$

since by the definition of $q$ we have that
$\mathbb{E}_{x}\left\\{q(X_{\tau})-q(X_{\tau_{\delta}})\right\\}\leq\delta
2\|f\|$. For $x$ from a compact set $K$ and given $\varepsilon>0$ one can find
by Proposition 2.1 of [19] $R>0$ such that (see also (47))

(86) $\sup_{x\in K}$

where $\rho$ is the metric on $E$. Furthermore for a given compact set
$B(K,R)=\left\\{y:\rho(y,K)\leq R\right\\}$ for a given $\varepsilon>0$ and
$\gamma>0$ by Proposition 6.4 of [2] there is $\kappa>0$ such that

(87) $\sup_{x\in B(K,R)}$

With the use of (86) and (87) we can evaluate for $\delta\leq\kappa$

$\displaystyle|\mathbb{E}_{x}\left\\{1_{\tau\leq
T}(w(X_{\tau_{\delta}})-w(X_{\tau}))\right\\}|\leq$
$\displaystyle|\mathbb{E}_{x}\left\\{1_{\tau\leq T}1_{X_{\tau}\in
B(K,R)}(w(X_{\tau_{\delta}})-w(X_{\tau}))\right\\}|+2\|w\|\varepsilon\leq$
(88) $\displaystyle|\mathbb{E}_{x}\left\\{1_{\tau\leq T}1_{X_{\tau}\in
B(K,R)}1_{\rho(X_{\tau},X_{\tau_{\delta}})\leq\gamma}(w(X_{\tau_{\delta}})-w(X_{\tau}))\right\\}|+4\|w\|\varepsilon.$

By continuity of $w$ it is uniformly continuous in a compact set so that for a
sufficiently small $\gamma$ we have that

(89) $|\mathbb{E}_{x}\left\\{1_{\tau\leq T}1_{X_{\tau}\in
B(K,R)}1_{\rho(X_{\tau},X_{\tau_{\delta}})\leq\gamma}(w(X_{\tau_{\delta}})-w(X_{\tau}))\right\\}|\leq\varepsilon.$

Summarizing (4.5), (4.5) and (89) taking also into account (83) we finally
obtain

(90) $0\leq
v_{\delta}(x)-v(x)\leq\varepsilon+(\mu(f)+2\|f\|)\delta+4\|w\|\varepsilon+\varepsilon+2\|w\|{g(x)\over\mu(f)T},$

so that $v_{\delta}(x)\to v(x)$, as $\delta\to 0$, uniformly in $x$ from
compact sets. Consequently $v$ and then also $z$ is a continuous function.
This completes the first part of the proof of Theorem 4.4.

Consider a finite horizon stopping problem

(91) $z_{T}(x):=\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge
T}f(X_{s})ds+w(X_{\tau\wedge T})\right\\}.$

By Corollary 1 of [28] we have that $z_{T}\in C(E)$ and therefore an optimal
stopping $\tau_{T}$ time is of the form

(92) $\tau_{T}=\inf\left\\{s\geq 0:z_{T-s}(X_{s})\geq w(X_{s})\right\\}.$

Clearly $z_{T}(x)$ is decreasing to $z(x)$, as $T\to\infty$ and since both
functions are continuous, by Dini’s lemma we have convergence uniform on
compact sets. Stopping times $\tau_{T}$ are increasing in $T$ and
$\tau_{T}\leq\tau^{*}$. We know that we can restrict ourselves in (73) to
stopping times $\tau$ such that

(93) $\mathbb{E}_{x}\left\\{\tau\right\\}\leq{\|w\|-q(x)+K\over\mu(f)}.$

Clearly $\tau^{*}$ also satisfies (93). Furthermore $\tau_{T}\to\tau^{*}$ as
$T\to\infty$. Then we have, using quasilefcontinuity of $(X_{t})$ (see Theorem
3.13 of [11]) and (93)

(94)
$z_{T}(x)=\mathbb{E}_{x}\left\\{\int_{0}^{\tau_{T}}f(X_{s})ds+w(X_{\tau_{T}})\right\\}\to\mathbb{E}_{x}\left\\{\int_{0}^{\tau^{*}}f(X_{s})ds+w(X_{\tau^{*}})\right\\}$

as $T\to\infty$, which means that $\tau^{*}$ is an optimal stopping time for
$z$.

It remains to show (75). Notice first that by (79)
$m_{\delta}(n\delta)=\mu(f)n\delta+v_{\delta}(X_{n\delta})$ is a submartingale
and letting $\delta\to 0$ also $\hat{m}(t)=\mu(f)t+v(t)$ is a right continuous
submartingale. Consequently also $m(t)=\int_{0}^{t}f(X_{s})ds+z(X_{t})$ is a
right continuous submartingale. Therefore for any bounded stopping time $\tau$
and stopping time $\sigma$ we have
$\mathbb{E}_{x}\left\\{m(\tau)|F_{\tau\wedge\sigma}\right\\}\geq
m(\tau\wedge\sigma)$ and

$\displaystyle\int_{0}^{\tau\wedge\sigma}f(X_{s})ds+z(X_{\tau\wedge\sigma})\leq\mathbb{E}_{x}\left\\{\int_{0}^{\tau}f(X_{s})ds+z(X_{\tau})|F_{\tau\wedge\sigma}\right\\}\leq$
(95)
$\displaystyle\mathbb{E}_{x}\left\\{\int_{0}^{\tau}f(X_{s})ds+1_{\tau<\sigma}z(X_{\tau})+1_{\sigma\leq\tau}w(X_{\tau})|F_{\tau\wedge\sigma}\right\\}$

Subtracting from both sides of $1_{\tau<\sigma}z(X_{\tau})$ and then adding to
both sides $1_{\tau<\sigma}w(X_{\tau})$ we obtain that

(96)
$\mathbb{E}_{x}\left\\{\int_{0}^{\tau}f(X_{s})ds+w(X_{\tau})\right\\}\geq\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\sigma}f(X_{s})ds+1_{\tau<\sigma}w(X_{\tau})+1_{\sigma\leq\tau}z(X_{\sigma})\right\\}.$

Since in the definition of $z$ we may restrict to stopping times $\tau$ such
that (93) is satisfied such stopping times can be approximated by bounded
stopping times we have that

(97)
$z(x)\geq\inf_{\tau}\mathbb{E}_{x}\left\\{\int_{0}^{\tau\wedge\sigma}f(X_{s})ds+1_{\tau<\sigma}w(X_{\tau})+1_{\sigma\leq\tau}z(X_{\sigma})\right\\}$

for any stopping time $\sigma$. The inverse inequality is rather obvious so
that we finally obtain (75).

## References

  * [1] L.H.R. Alvarez, Stochastic forest stand value and optimal timber harvesting, SIAM J. Control Optim. 42 (6) (2004), pp. 1972–1993.
  * [2] A. Basu and Ł. Stettner, Zero-sum Markov games with impulse controls, SIAM J. Control Optim. 58 (2020), pp. 580–604.
  * [3] E. Bayraktar, T. Emmerling and J.L. Menaldi, On the impulse control of jump duffusions, SIAM J. Control Optim. 51 (2013), pp. 2612–2637.
  * [4] Ch. Belak, S. Christensen, Utility maximisation in a factor model with constant and proportional transaction costs, Finance Stoch. 23 (1) (2019), 29–96.
  * [5] A. Bensoussan, Dynamic programming and inventory control. Studies in Probability, Optimization and Statistics, 3, IOS Press, Amsterdam, 2011.
  * [6] A. Bensoussan and J.-L. Lions, Impulse Control And Quasi-Variational Inequalities, Gauthier-Villars, Montrouge, 1984.
  * [7] S. Christensen, A. Irle and A. Ludwig, Optimal portfolio selection under vanishing fixed transaction costs, Adv. in Appl. Probab. 49 (2017), pp. 1116–1143.
  * [8] S. Christensen and T. Sohr, A solution technique for Lévy driven long term average impulse control problems, Stochastic Process Appl. 130 (2020), pp. 7303–7337.
  * [9] A. Devraj, I. Kontoyiannis and S. Meyn, Geometric Ergodicity in a Weighted Sobolev Space, Annals of Prob. 48 (2020), pp. 380–403.
  * [10] T. Duncan, B. Pasik-Duncan and L. Stettner, Growth optimal portfolio selection under proportional transaction costs with obligatory diversification, App. Math. Optim. 63 (2010), pp. 107–132.
  * [11] E.B. Dynkin, Markov processes, Springer, 1965.
  * [12] D. Gatarek and L. Stettner, On the compactness method in general ergodic impulsive control of Markov processes, Stoch. Stoch. Rep. 31 (1990), pp. 15–26.
  * [13] R. Korn, Some applications of impulse control in mathematical finance, Math. Methods Oper. Res. 50 (3) (1999), 493–518.
  * [14] A. Jack and M. Zervos, Impulse control of one-dimensional Ito diffusions with an expected and a pathwise ergodic criterion, Appl. Math. Optim. 54 (2006), pp. 71–93
  * [15] P.L. Lions and B. Perthame, Quasi-variational inequalities and ergodic impulse control, SIAM J. Control Optim., 24 (1986), pp. 604–615.
  * [16] L.C. MacLean, E.O. Thorp and W.T. Ziemba, The Kelly Capital Growth Investment Criterion, World Scientific, River Edge, NJ, 2011.
  * [17] J. Menaldi and M. Robin, On some ergodic impulse control problems with constraints, SIAM J. Control Optim., 56 (2018), pp. 2690–2711.
  * [18] G. Mundaca and B. Øksendal, Optimal stochastic intervention control with application to the exchange rate, J. Math. Econom., 29 (2) (1998), pp. 225–243.
  * [19] J. Palczewski and Ł. Stettner, Finite horizon optimal stopping of time-discontinuous functionals with applications to impulse control with delay, SIAM J. Control Optim., 48 (2010), pp. 4874–4909.
  * [20] J. Palczewski, Ł. Stettner, Stopping of discontinuous functionals with the first exit time discontinuity, Stochastic Process. Appl.121, no.10, (2011), pp. 2361–2392.
  * [21] J. Palczewski and Ł. Stettner, Impulse control maximizing average cost per unit time: A nonuniformly ergodic case, SIAM J. Control Optim., 55 (2017), pp. 936–960.
  * [22] B. Perthame, Vanishing impulse control in the quasivariational inequality for ergodic impulse control, Asymptot. Anal., 1 (1988), pp. 13–21.
  * [23] M. Robin, Contrôle impulsionnel des processus de Markov, thèse d’état, Université Paris Dauphine, 1978.  Available at https://tel.archives-ouvertes.fr/tel-00735779.
  * [24] M. Robin, On some impulse control problems with long run average cost, SIAM J. Control Optim., 19 (1981), pp. 333–358.
  * [25] M. Robin, Long-term average control problems for continuous time Markov Processes: A survey, Acta Appl. Math. 1 (1983), pp. 281–299.
  * [26] Ł. Stettner, On Impulsive Control with Long Run Average Cost Criterion, Studia Math. 76 (1983), pp. 279–298.
  * [27] Ł. Stettner, On ergodic impulse control problems, Stochastics, 18 (1986), pp. 49–72.
  * [28] Ł. Stettner, Penalty method for finite horizon stopping problems, SIAM J. Control Optim., 49 (2011), pp. 1078–1999.
  * [29] Y. Willassen, The stochastic rotation problem: A generalization of Faustmann’s formula to stochastic forest growth, J. Econom. Dynam. Control, 22 (1998), pp. 573–596.


# Spin-Frame Field Theory of a Three-Sublattice Antiferromagnet

Bastian Pradenas  Oleg Tchernyshyov  William H. Miller III Department of
Physics and Astronomy, Johns Hopkins University, Baltimore, Maryland 21218,
USA

###### Abstract

We present a nonlinear field theory of a three-sublattice hexagonal
antiferromagnet. The order parameter is the spin frame, an orthogonal triplet
of vectors related to sublattice magnetizations and spin chirality. The
exchange energy, quadratic in spin-frame gradients, has three coupling
constants, only two of which manifest themselves in the bulk. As a result, the
three spin-wave velocities satisfy a universal relation. Vortices generally
have an elliptical shape with the eccentricity determined by the Lamé
parameters.

Theory of magnetic solids is often described as a lattice problem, exemplified
by the Heisenberg model of atomic spins. However, discrete models are
notoriously difficult to solve outside of the simplest tasks, such as finding
the spectrum of linear spin waves. An analytic theory of nonlinear
solitons—such as domain walls or vortices—in the framework of a lattice model
is often not feasible or cumbersone [1]. Continuum theories have the clear
advantage of being more amenable to analytic treatment. By design, they focus
on the physics of long distances and times, capturing the universal aspects of
low-energy physics at the expense of microscopic details.

In magnetism, a well-known example is micromagnetics, the continuum theory of
a ferromagnet going back to Landau and Lifshits [2]. It is usually formulated
through the equation of motion for the magnetization field $\mathbf{m}$ of
unit length parallel to the local direction of spins,

$\mathcal{S}\,\frac{\partial\mathbf{m}}{\partial
t}=-\mathbf{m}\times\frac{\delta U}{\delta\mathbf{m}}.$ (1)

Here $\mathcal{S}$ is the spin density, $U[\mathbf{m}]=\int
d^{d}r\,\mathcal{U}$ is a potential-energy functional, whose functional
derivative $-\delta U/\delta\mathbf{m}$ acts as an effective magnetic field.
The energy density is usually dominated by the Heisenberg exchange interaction
of strength $A$,

$\mathcal{U}=\frac{A}{2}\partial_{i}\mathbf{m}\cdot\partial_{i}\mathbf{m}+\ldots$
(2)

Doubly repeated indices imply summation. The omitted terms represent weaker
anisotropic interactions of dipolar and relativistic origin. We use the
calligraphic font to indicate intensive quantities (densities).

The Landau-Lifshitz equation (1) provides a starting point for understanding
the dynamics of ferromagnetic solitons. A further coarse-graining eliminates
fast internal modes of a soliton and focuses on its slow collective motion
whose seminal achievements; primary examples are Thiele’s equation of rigid
motion [3] and Walker’s dynamical model of a domain wall [4].

The continuum approach has also been applied to simple antiferromagnets, in
which adjacent spins are (nearly) antiparallel and can be split into two
magnetic sublattices 1 and 2, each with its own magnetization field
$\mathbf{m}_{1}$ and $\mathbf{m}_{2}$ of unit length. Because at low energies,
the sublattice magnetizations are (nearly) antiparallel, both can be
approximated by a single field of staggered magnetization
$\mathbf{n}\approx\mathbf{m}_{A}\approx-\mathbf{m}_{B}$, whose dynamics is
described by an $O(3)$ $\sigma$-model with the Lagrangian density [5, 6, 7]

$\mathcal{L}=\mathcal{K}-\mathcal{U}=\frac{\rho}{2}\partial_{t}\mathbf{n}\cdot\partial_{t}\mathbf{n}-\frac{A}{2}\partial_{i}\mathbf{n}\cdot\partial_{i}\mathbf{n}-\ldots$
(3)

The first term
$\mathcal{K}=\frac{\rho}{2}\partial_{t}\mathbf{n}\cdot\partial_{t}\mathbf{n}$
is the kinetic energy of staggered magnetization and $\rho$ is a measure of
inertia. The second, potential term comes from the Heisenberg exchange energy
and has the same functional form as in a ferromagnet (2). The omitted terms
represent various weak anisotropic interactions. Minimization of the action
with the constraint $\mathbf{n}^{2}=1$ yields the equation of motion

$\rho\,\partial_{t}(\mathbf{n}\times\partial_{t}\mathbf{n})=-\mathbf{n}\times\frac{\delta
U}{\delta\mathbf{n}}.$ (4)

As with ferromagnets, the antiferromagnetic Landau–Lifshitz equation (4) can
be translated into equations of motion for solitons [8, 9] and extended to
include the effects of spin transfer and dissipation [10, 11].

The primary goal of this paper is to introduce a universal field theory for an
antiferromagnet with three magnetic sublattices whose magnetization fields
satisfy the relation $\mathbf{m}_{1}+\mathbf{m}_{2}+\mathbf{m}_{3}\approx 0$.
Such magnetic states are typically realized in antiferromagnetic solids of
hexagonal symmetry with triangular motifs. Although such magnets have been
studied for decades [12, 13], recent experimental studies of metallic
antiferromagnets Mn3Sn and Mn3Ge [14, 15] have rekindled theoretical interest
in these frustrated magnets [16, 17, 18].

The existing field theory for 3-sublattice antiferromagnets by Dombre and Read
[12] has a couple of drawbacks. First, it is formulated specifically for the
triangular lattice, which has a higher spatial symmetry than other hexagonal
lattices (such as kagome) and therefore misses some of the universal features
of 3-sublattice antiferromagnets. Second, their mathematical formalism
represents the magnetic order parameter as a $3\times 3$ rotation matrix, an
abstract, and not very intuitive mathematical object.

We derive a nonlinear field theory of a 3-sublattice antiferromagnet with the
order parameter represented by a _spin frame_ , i.e., a triad of orthonormal
vectors
$\hat{\mathbf{n}}\equiv\\{\mathbf{n}_{x},\mathbf{n}_{y},\mathbf{n}_{z}\\}$,
directly related to sublattice magnetizations $\mathbf{m}_{1}$,
$\mathbf{m}_{2}$, and $\mathbf{m}_{3}$. At low energies, the magnetization
dynamics reduce to rigid rotations of the spin frame,
$\partial_{t}\mathbf{n}_{i}=\boldsymbol{\Omega}\times\mathbf{n}_{i}$, at a
local angular frequency $\boldsymbol{\Omega}$. One of our main results is the
Landau–Lifshitz equation for a 3-sublattice antiferromagnet,

$\rho\,\partial_{t}\boldsymbol{\Omega}=-\mathbf{n}_{i}\times\frac{\delta
U}{\delta\mathbf{n}_{i}}.$ (5)

A sum over doubly repeated Roman indices, $i=x,y,z$, is implied hereafter.
Like its analogs (1) and (4), it equates the rate of change of the local
density of angular momentum with the torque density from conservative forces
expressed by a potential energy functional $U(\hat{\mathbf{n}})$. The
transparent physical meaning of the Landau–Lifshitz equation makes it easy to
add other relevant perturbations.

Figure 1: (a) A ground state of the kagome antiferromagnet and its sublattice
magnetizations $\mathbf{m}_{1}$, $\mathbf{m}_{2}$, and $\mathbf{m}_{3}$. (b)
The corresponding spin-frame vectors $\mathbf{n}_{x}$, $\mathbf{n}_{y}$, and
$\mathbf{n}_{z}$.

To define the spin frame $\hat{\mathbf{n}}$, we first switch from the three
unit-vector fields of sublattice magnetizations $\mathbf{m}_{1}$,
$\mathbf{m}_{2}$, and $\mathbf{m}_{3}$ to uniform magnetization $\mathbf{m}$
and two staggered magnetizations $\mathbf{n}_{x}$ and $\mathbf{n}_{y}$ (Fig.
1):

$\displaystyle\mathbf{m}$ $\displaystyle=$
$\displaystyle\mathbf{m}_{1}+\mathbf{m}_{2}+\mathbf{m}_{3},$
$\displaystyle\mathbf{n}_{x}$ $\displaystyle=$
$\displaystyle(\mathbf{m}_{2}-\mathbf{m}_{1})/\sqrt{3},$ (6)
$\displaystyle\mathbf{n}_{y}$ $\displaystyle=$
$\displaystyle(2\mathbf{m}_{3}-\mathbf{m}_{2}-\mathbf{m}_{1})/3.$

To them, we add the vector spin chirality [19]

$\mathbf{n}_{z}=\frac{2}{3\sqrt{3}}(\mathbf{m}_{1}\times\mathbf{m}_{2}+\mathbf{m}_{2}\times\mathbf{m}_{3}+\mathbf{m}_{3}\times\mathbf{m}_{1}).$
(7)

As long as $\mathbf{m}=0$, sublattice fields $\mathbf{m}_{1}$,
$\mathbf{m}_{2}$, and $\mathbf{m}_{3}$ are coplanar and thus define the _spin
plane_. Staggered magnetizations $\mathbf{n}_{x}$ and $\mathbf{n}_{y}$ lie in
the spin plane, whereas spin chirality $\mathbf{n}_{z}$ is orthogonal to it.
The three unit vectors $\mathbf{n}_{i}$ form a right-oriented orthonormal spin
frame:

$\mathbf{n}_{i}\cdot\mathbf{n}_{j}=\delta_{ij},\quad\mathbf{n}_{i}\times\mathbf{n}_{j}=\epsilon_{ijk}\mathbf{n}_{k}.$
(8)

Figure 2: Hexagonal lattices and their magnetic sublattices: (a) kagome, (b)
triangular lattice. (c) Spatial coordinate axes $x$, $y$, and $z$. Red, green,
and blue colors indicate magnetic sublattices 1, 2, and 3. Filled triangles
and dotted lines denote $C_{3}$ and $C_{2}$ rotation axes, respectively.

To derive the dynamics of the spin frame, we follow the standard Lagrangian
approach [6, 12, 20] and integrate out the hard field of uniform magnetization
$\mathbf{m}$ to obtain the dynamics of the spin frame. Our starting point is
the Landau–Lifshitz equations for sublattice magnetizations,

$\mathcal{S}\,\partial_{t}\mathbf{m}_{1}=-\mathbf{m}_{1}\times\frac{\delta
V}{\delta\mathbf{m}_{1}},$ (9)

and similarly for sublattices 2 and 3. Like in 2-sublattice antiferromagnets
[21], the potential energy functional $V$ is dominated by the
antiferromagnetic exchange interaction imposing a penalty for $\mathbf{m}\neq
0$ [22],

$\mathcal{V}(\mathbf{m},\hat{\mathbf{n}})=\frac{\mathbf{m}^{2}}{2\chi}+\mathcal{U}(\hat{\mathbf{n}}),$
(10)

where $\chi$ is the paramagnetic susceptibility. The subdominant term
$U[\hat{\mathbf{n}}]$, expressing the energy of the antiferromagnetic order
parameter, will be discussed below.

With the aid of Eqs. (6), (9), and (10) we find that, like in 2-sublattice
antiferromagnets [5], staggered magnetizationss $\mathbf{n}_{x}$ and
$\mathbf{n}_{y}$ precess about the direction of uniform magnetization
$\mathbf{m}$ at the angular velocity [20]

$\boldsymbol{\Omega}\approx\frac{\mathbf{m}}{\chi\mathcal{S}}.$ (11)

The linear proportionality between the precession frequency and uniform
magnetization (11) can be derived as the equation of motion for uniform
magnetization $\mathbf{m}$ from the following Lagrangian for fields
$\mathbf{m}$ and $\hat{\mathbf{n}}$ [20]:

$\mathcal{L}(\mathbf{m},\hat{\mathbf{n}})=\mathcal{S}\mathbf{m}\cdot\boldsymbol{\Omega}-\frac{\mathbf{m}^{2}}{2\chi}-\mathcal{U}(\hat{\mathbf{n}}).$
(12)

The angular velocity can be expressed explicitly in terms of
$\hat{\mathbf{n}}$ via the kinematic identity

$\boldsymbol{\Omega}=\frac{1}{2}\mathbf{n}_{i}\times\partial_{t}\mathbf{n}_{i},$
(13)

The first term in the Lagrangian (12) is linear in the velocities
$\partial_{t}\mathbf{n}_{i}$, so its action represnts the spin Berry phase. It
yields the expected density of angular momentum $\mathcal{S}\mathbf{m}$.

Lagrangian (12) is quadratic in uniform magnetization $\mathbf{m}$.
Integrating out this field with the aid of its equation of motion (11) yields
an effective Lagrangian for the remaining fields $\hat{\mathbf{n}}$ endowed
with kinetic energy of rotation with the inertia density
$\rho=\chi\mathcal{S}^{2}$:

$\mathcal{L}(\hat{\mathbf{n}})=\frac{\rho\Omega^{2}}{2}-\mathcal{U}(\hat{\mathbf{n}})=\frac{\rho}{4}\partial_{t}\mathbf{n}_{i}\cdot\partial_{t}\mathbf{n}_{i}-\mathcal{U}(\hat{\mathbf{n}}).$
(14)

Minimizing the action in the presence of holonomic constraints (8) yields
equations of motion with undetermined Lagrange multipliers [23]
$\Lambda_{ij}=\Lambda_{ji}$:

$\frac{I}{2}\partial_{t}^{2}\mathbf{n}_{i}=-\frac{\delta
U}{\delta\mathbf{n}_{i}}-\Lambda_{ij}\mathbf{n}_{j}.$ (15)

Finally, we take a cross product with $\mathbf{n}_{i}$, sum over $i$, and use
Eq. (13) to obtain the Landau–Lifshitz equation (5).

The energy functional $U[\hat{\mathbf{n}}]$ is usually dominated by the
Heisenberg exchange interaction. The latter respects the SO(3) symmetry of
global spin rotations and therefore depends not on the orientation of the spin
frame $\hat{\mathbf{n}}$ but rather on its spatial gradients. Like in
ferromagnets and 2-sublattice antiferromagnets, the exchange energy is
quadratic in $\partial_{\alpha}\mathbf{n}_{\beta}$, where Greek indices take
on values $\alpha=x$ and $y$ only. The form of these quadratic terms is
restricted by the $D_{3}$ point-group rotational symmetry of a hexagonal
lattice (Fig. 2), including $\pm 2\pi/3$ spatial rotations about a $C_{3}$
axis normal to the $xy$ plane and $\pi$ spatial rotations about $C_{2}$ axes
lying in the $xy$ plane. Under these transformations, the staggered
magnetizations $(\mathbf{n}_{x},\mathbf{n}_{y})$ transform in terms of each
other in the same way as the in-plane components $(k_{x},k_{y})$ of a spatial
vector $\mathbf{k}$ do; chirality $\mathbf{n}_{z}$ transforms as $k_{z}$. This
observation helps to construct energy terms quadratic in the gradients of
staggered magnetizations and invariant under both spin rotations and lattice
symmetries. To that end, we may start with a rank-4 spatial tensor and spin
scalar
$\partial_{\alpha}\mathbf{n}_{\beta}\cdot\partial_{\gamma}\mathbf{n}_{\delta}$
and contract its spatial indices pairwise to form a spatial scalar. This
procedure yields our second main result, the three possible gradient terms for
the exchange energy density,

$\mathcal{U}=\frac{\lambda}{2}\,\partial_{\alpha}\mathbf{n}_{\alpha}\cdot\partial_{\beta}\mathbf{n}_{\beta}+\frac{\mu}{2}\,\partial_{\alpha}\mathbf{n}_{\beta}\cdot\partial_{\alpha}\mathbf{n}_{\beta}+\frac{\nu}{2}\,\partial_{\alpha}\mathbf{n}_{\beta}\cdot\partial_{\beta}\mathbf{n}_{\alpha}.$
(16)

This expression resembles the elastic energy density of an isotropic solid
[24], albeit with 3 Lamé constants.

A triangular lattice has an extra spatial symmetry. Under primitive lattice
translations, sublattice indices undergo cyclic permutations, see Fig. 2(b).
Staggered magnetizations $\mathbf{n}_{\alpha}$ effectively undergo $\pm
2\pi/3$ spatial rotations, whereas gradients $\partial_{\alpha}$ do not. Thus
translational symmetry forbids the $\lambda$ and $\nu$ terms for a triangular
lattice.

The Landau–Lifshitz equation (5) for a 3-sublattice Heisenberg antiferromagnet
reads

$\rho\,\partial_{t}\mathbf{\Omega}=(\lambda+\nu)\,\mathbf{n}_{\alpha}\times\partial_{\alpha}\partial_{\beta}\mathbf{n}_{\beta}+\mu\,\mathbf{n}_{\alpha}\times\partial_{\beta}\partial_{\beta}\mathbf{n}_{\alpha}.$
(17)

Note that the exchange coupling constants $\lambda$ and $\nu$ enter the
equation of motion through a combination $\lambda+\nu$, rather than
individually. More on that below.

An antiferromagnet with nearest-neighbor exchange interaction $J$ has
$\lambda+\nu=0$ and $\mu=JS^{2}\sqrt{3}/4$ on a triangular lattice; on kagome,
$\lambda+\nu=\sqrt{3}JS^{2}/4$ and $\mu=0$ on kagome. See Supplemental
Material [25] for contributions of further-neighbor interactions.

In what follows, we use the spin-frame formulation of the field theory to
obtain the properties of excitations: spin waves and vortices.

_Spin waves._ Linear spin waves on top of a uniform ground state
$\mathbf{n}_{i}=\text{const}$ can be parametrized in terms of infinitesimal
rotations of the spin frame,
$\delta\mathbf{n}_{i}=\delta\boldsymbol{\phi}\times\mathbf{n}_{i}$. Here
$\delta\boldsymbol{\phi}=\mathbf{n}_{i}\delta\phi_{i}$ is a triplet of
infinitesimal rotation angles $\delta\phi_{i}$ about the corresponding spin
axes; $\partial_{t}\delta\boldsymbol{\phi}=\boldsymbol{\Omega}$.

Assuming a plane wave with wavenumber $k$ travelling along the $x$ direction,
$\delta\boldsymbol{\phi}(t,x)=\mathbf{e}\,\delta\phi\,e^{i(kx^{\prime}-\omega
t)}$, we obtain three spin waves with $\omega=ck$ and the following
polarizations $\mathbf{e}$ and velocities $c$:

$\begin{split}\mathbf{e}_{\text{I}}=\mathbf{n}_{x},\quad&c_{\text{I}}=\sqrt{\mu/\rho},\\\
\mathbf{e}_{\text{II}}=\mathbf{n}_{y},\quad&c_{\text{II}}=\sqrt{(\lambda+\mu+\nu)/\rho},\\\
\mathbf{e}_{\text{III}}=\mathbf{n}_{z},\quad&c_{\text{III}}=\sqrt{(\lambda+2\mu+\nu)/\rho}.\end{split}$
(18)

The velocities satisfy the identity

$c_{\text{I}}^{2}+c_{\text{II}}^{2}=c_{\text{III}}^{2}.$ (19)

Modes I, II are analogs of transverse and longitudinal sound in a hexagonal
solid in two dimensions.

_Vortices._ The existence of topologically stable point defects—vortices—in a
3-sublattice Heisenberg antiferromagnet was first pointed out by Kawamura and
Miyashita [19]. A $2\pi$ rotation of the spin frame corresponds to a loop in
the order-parameter space that cannot be continuously deformed to a point. It
is convenient to parametrize the orientation of the spin frame by starting
with a reference uniform configuration $\mathbf{n}_{x}=(1,0,0)$,
$\mathbf{n}_{y}=(0,1,0)$, $\mathbf{n}_{z}=(0,0,1)$, and applying consecutive
Euler rotations through angles $\phi$ about $\mathbf{n}_{z}$, $\theta$ about
$\mathbf{n}_{y}$, and $\psi$ about $\mathbf{n}_{z}$. On a triangular lattice
($\lambda+\nu=0$), a vortex configuration with the lowest energy is described
by the Euler angles $\phi$, $\theta$, and $\phi$ given by

$e^{i\phi}=\frac{x+iy}{|x+iy|},\quad\theta=\frac{\pi}{2},\quad\psi=\mbox{const}.$
(20)

This expression agrees well with a numerically obtained vortex configuration
for a triangular lattice, Fig. 3(a).

Figure 3: Vortices in 3-sublattice antiferromagnets. (a) Triangular lattice
with nearest-neighbor interactions only. (b) kagome lattice with first and
third-neighbor interactions, $J_{3}=J_{3}^{\prime}=-J_{1}/20$. See
Supplemental Material [25] for the definition of further-neighbor
interactions. Red, green, and blue arrows are spins of the three magnetic
sublattices. Spins on sublattice 3 (blue) point away from the viewer; spins on
sublattices 1 (red) and 2 (green) have components pointing toward the reader.
The circle and ellipse reflect the expected shape of the vortex with the major
axis ratio $b=\sqrt{(\lambda+\mu+\nu)/\mu}$.

Although we have not been able to find an exact vortex solution for a generic
hexagonal antiferromagnet, we can understand the effect of the $\lambda+\nu$
term on the vortex shape perturbatively. Starting with the isotropic solution
(20) for $\lambda+\nu=0$, we keep $\theta=\pi/2$ and choose for simplicity
$\psi=\pi/2$ to obtain the following energy density:

$\mathcal{U}=\frac{\lambda+\mu+\nu}{2}(\partial_{x}\phi)^{2}+\frac{\mu}{2}(\partial_{y}\phi)^{2}.$
(21)

The vortex acquires an elliptical shape with the major axis ratio
$b=\sqrt{(\lambda+\mu+\nu)/\mu}$:

$e^{i\phi}=\frac{bx+iy}{|bx+iy|},\quad\theta=\frac{\pi}{2},\quad\psi=\frac{\pi}{2}.$
(22)

Although this result is obtained in the limit $\lambda+\nu\ll\mu$, our
numerical calculations on a kagome lattice demonstrate its accuracy even in
the opposite limit. The right panel of Fig. 3(b) shows a vortex in a kagome
antiferromagnet with the ratio of first and third-neighbor exchange
interactions $J_{1}/J_{3}=-20$, or $(\lambda+\nu)/\mu=5/3$. Its shape agrees
with the extrapolated semiaxis ratio $b=\sqrt{8/3}$. See Fig. S2 [25] for
other orientations of the major axes of a vortex.

_Discussion._ In this paper, we have presented a universal field theory of a
hexagonal antiferromagnet with 3 magnetic sublattices. The order parameter is
a spin frame constructed from the sublattice magnetizations and vector
chirality. Its mechanics is fully specified by the inertia of the spin frame
$\rho$ and three Lamé constants $\lambda$, $\mu$, and $\nu$. The simple and
versatile field theory enabled us to establish a Pythagorean identity for the
three spin-wave velocities (19) and to predict a generally elliptical shape
for vortices (22).

It is worth noting that the three Lamé constants enter the equations of motion
(17) in the form of two linear combinations, $\lambda+\nu$ and $\mu$. As a
result of that, the three spin-wave velocities are constrained by a
Pythagorean identity (19). The origin of this behavior can be understood by
examining the exchange energy density (16). The $\lambda$ and $\nu$ terms in
it can be obtained from one another through integration by parts. Thus their
infinitesimal variations are the same—up to boundary terms—and so they make
identical contributions to classical dynamics. The difference $\lambda-\nu$ is
a “silent” coupling constant that does not manifest itself in the classical
dynamics of magnetization. It can be seen that it has an intriguing
topological nature with the aid of the identity

$\frac{1}{2}(\partial_{\alpha}\mathbf{n}_{\alpha}\cdot\partial_{\beta}\mathbf{n}_{\beta}-\partial_{\alpha}\mathbf{n}_{\beta}\cdot\partial_{\beta}\mathbf{n}_{\alpha})=\mathbf{n}_{z}\cdot(\partial_{x}\mathbf{n}_{z}\times\partial_{y}\mathbf{n}_{z}).$
(23)

We now see that the energy term (23) is a topological quantity proportional to
the skyrmion density of the spin chirality (7). Its topological nature assures
that it gives a quantized contribution to the exchange energy that does not
change under continuous variations of the order parameter and therefore does
not affect classical dynamics. However, it may lead to interesting boundary
effects such as the presence of edge modes, as discussed recently in a
different context by Dong  _et al._ [26]. We will address the topological
aspects of this field theory in a separate publication [27].

_Acknowledgments._ We thank Boris Ivanov and Se Kwon Kim for useful
discussions. The research has been supported by the U.S. Department of Energy
under Award No. DE-SC0019331 and by the U. S. National Science Foundation
under Grants No. NSF PHY-1748958 and PHY-2309135.

## References

  * Gochev [1983] I. G. Gochev, Zh. Eksp. Teor. Fiz. 85, 199 (1983), [Sov. Phys. JETP 58, 115–119 (1983)].
  * Landau and Lifshits [1935] L. Landau and E. Lifshits, Phys. Z. Sowjet. 8, 153 (1935), reprinted in _Collected Papers of L. D. Landau_ , Pergamon, New York, pp. 101-114 (1965).
  * Thiele [1973] A. A. Thiele, Phys. Rev. Lett. 30, 230 (1973).
  * Schryer and Walker [1974] N. L. Schryer and L. R. Walker, J. Appl. Phys. 45, 5406 (1974).
  * Bar’yakhtar and Ivanov [1979] I. V. Bar’yakhtar and B. A. Ivanov, Fiz. Nizk. Temp. 5, 759 (1979), [Sov. J. Low Temp. Phys. 5, 361 (1979)].
  * Andreev and Marchenko [1980] A. F. Andreev and V. I. Marchenko, Usp. Fiz. Nauk 130, 39 (1980), [Sov. Phys. Usp. 23, 21–34 (1980)].
  * Haldane [1983] F. D. M. Haldane, Phys. Rev. Lett. 50, 1153 (1983).
  * Bar’yakhtar  _et al._ [1979] I. V. Bar’yakhtar, B. A. Ivanov, and A. L. Sukstanskii, Zh. Eksp. Teor. Fiz. 51, 757 (1979), [Sov. Phys. JETP 78, 1509–1522 (1979)].
  * Bar’yakhtar  _et al._ [1985] V. G. Bar’yakhtar, B. A. Ivanov, and M. V. Chetkin, Usp. Fiz. Nauk 146, 417 (1985), [Sov. Phys. Usp. 28, 563–588 (1985)].
  * Hals  _et al._ [2011] K. M. D. Hals, Y. Tserkovnyak, and A. Brataas, Phys. Rev. Lett. 106, 107206 (2011).
  * Tveten  _et al._ [2013] E. G. Tveten, A. Qaiumzadeh, O. A. Tretiakov, and A. Brataas, Phys. Rev. Lett. 110, 127208 (2013).
  * Dombre and Read [1989] T. Dombre and N. Read, Phys. Rev. B 39, 6797 (1989).
  * Harris  _et al._ [1992] A. B. Harris, C. Kallin, and A. J. Berlinsky, Phys. Rev. B 45, 2899 (1992).
  * Nakatsuji  _et al._ [2015] S. Nakatsuji, N. Kiyohara, and T. Higo, Nature 527, 212 (2015).
  * Chen  _et al._ [2020] Y. Chen, J. Gaudet, S. Dasgupta, G. G. Marcus, J. Lin, T. Chen, T. Tomita, M. Ikhlas, Y. Zhao, W. C. Chen, M. B. Stone, O. Tchernyshyov, S. Nakatsuji, and C. Broholm, Phys. Rev. B 102, 054403 (2020).
  * Liu and Balents [2017] J. Liu and L. Balents, Phys. Rev. Lett. 119, 087202 (2017).
  * Yamane  _et al._ [2019] Y. Yamane, O. Gomonay, and J. Sinova, Phys. Rev. B 100, 054415 (2019).
  * Dasgupta and Tchernyshyov [2020] S. Dasgupta and O. Tchernyshyov, Phys. Rev. B 102, 144417 (2020).
  * Kawamura and Miyashita [1984] H. Kawamura and S. Miyashita, J. Phys. Soc. Jpn. 53, 4138 (1984).
  * Ochoa  _et al._ [2018] H. Ochoa, R. Zarzuela, and Y. Tserkovnyak, Phys. Rev. B 98, 054424 (2018).
  * Kim  _et al._ [2014] S. K. Kim, Y. Tserkovnyak, and O. Tchernyshyov, Phys. Rev. B 90, 104406 (2014).
  * Halperin and Saslow [1977] B. I. Halperin and W. M. Saslow, Phys. Rev. B 16, 2154 (1977).
  * Landau and Lifshitz [1976] L. D. Landau and E. M. Lifshitz,  _Mechanics_ , 3rd ed., Course of Theoretical Physics, Vol. 1 (Butterworth-Heinemann, New York, 1976).
  * Landau and Lifshitz [1986] L. D. Landau and E. M. Lifshitz,  _Theory of Elasticity_ , 3rd ed., Course of Theoretical Physics, Vol. 7 (Butterworth-Heinemann, New York, 1986).
  * [25] Supplemental Material.
  * Dong  _et al._ [2023] Z. Dong, O. Ogunnaike, and L. Levitov, Phys. Rev. Lett. 130, 206701 (2023).
  * Pradenas  _et al._ [shed] B. Pradenas, E. G. Ozaktas, and O. Tchernyshyov (unpublished), in preparation.

##  Appendix A Supplemental Material

##  Appendix B Further neighbors exchange interactions

Convention for further-neighbor interactions is shown in Fig. S1.

Figure S1: First, second, and third-neighbors exchange interactions. The
kagome lattice has two distinct types of third neighbors.

In antiferromagnets with up to third-neighbor exchange interactions, the
field-theory parameters are as follows. For the triangular lattice,

$\lambda+\nu=0,\quad\mu=\frac{\sqrt{3}}{4}(J_{1}-6J_{2}+4J_{3})S^{2},\quad\rho^{-1}=\frac{9\sqrt{3}}{2}(J_{1}+J_{3})a^{2}.$
(S1)

Note that $\lambda+\nu$ vanishes in accordance with the higher spatial
symmetry of the triangular lattice.

For kagome,

$\lambda+\nu=\frac{\sqrt{3}}{4}(J_{1}-3J_{2}-2J_{3}-2J_{3}^{\prime})S^{2},\quad\mu=\frac{3\sqrt{3}}{4}(J_{2}-J_{3}-J_{3}^{\prime})S^{2},\quad\rho^{-1}=4\sqrt{3}(J_{1}+J_{2})a^{2}.$
(S2)

In both cases, $a$ is the distance between first neighbors.

For a triangular lattice with first-neighbor interactions only, we obtain

$c_{\text{I}}=c_{\text{II}}=\frac{3\sqrt{3}}{2\sqrt{2}}J_{1}Sa,\quad
c_{\text{III}}=\frac{3\sqrt{3}}{2}J_{1}Sa,$ (S3)

in agreement with Dombre and Read [12]. For a kagome antiferromagnet with
$J_{3}^{\prime}=0$, we find

$c_{\text{I}}=3\sqrt{(J_{1}+J_{2})(J_{2}-J_{3})}Sa,\quad
c_{\text{II}}=\sqrt{3(J_{1}+J_{2})(J_{1}-5J_{3})}Sa,\quad
c_{\text{III}}=\sqrt{3(J_{1}+J_{2})(J_{1}+3J_{2}-8J_{3})}Sa,$ (S4)

in agreement with Harris  _et al._ [13].

##  Appendix C Vortices

Figure S2: Same as Fig. 3 but for $\psi=\pi/2\pm 2\pi/3$, with spins on
sublattice 2 (green) or 1 (red) pointing directly toward the reader.

The energy density (16) for $\theta=\pi/2$ reads

$\mathcal{U}=\frac{\mu}{2}(\nabla\phi)^{2}+\frac{\lambda+\nu}{2}\left(\sin\psi\partial_{x}\phi+\cos\psi\partial_{y}\phi\right)^{2}\textrm{(double-
checked)}.$ (S5)

For constant $\psi$, one can define two new axes

$x^{\prime}=x\cos\psi-y\sin\psi,\quad y^{\prime}=y\cos\psi+x\sin\psi,$ (S6)

(S5) becomes

$\mathcal{U}=\frac{\mu}{2}(\partial_{x^{\prime}}\phi)^{2}+\frac{\mu+\lambda+\nu}{2}(\partial_{y^{\prime}}\phi)^{2}\textrm{(double-
checked)}.$ (S7)

Non-trivial solutions that minimize the above energy correspond to:

$\phi=\arctan\left(\frac{y^{\prime}}{bx^{\prime}}\right),\quad\theta=\frac{\pi}{2},\quad\psi=\textrm{constant}.$
(S8)

Where $b=\sqrt{(\lambda+\mu+\nu)/\mu}$ is the major to minor semiaxis ratio.
Notice that the orientation of the ellipse axes depends on the angle $\psi$,
through the coordinates $x^{\prime},y^{\prime}$ (S6); figures (3) and (S2)
illustrate this effect.