# Trapping wave fields in an expulsive potential by means of linear coupling

Nir Hacker1 and Boris A. Malomed1,2 1Department of Physical Electronics,
School of Electrical Engineering, Faculty of Engineering, and Center for
Light-Matter interaction, Tel Aviv University, Tel Aviv 69978, Israel
2Instituto de Alta Investigación, Universidad de Tarapacá, Casilla 7D, Arica,
Chile

###### Abstract

We demonstrate the existence of confined states in one- and two-dimensional
(1D and 2D) systems of two linearly-coupled components, with the confining
harmonic-oscillator (HO) potential acting upon one component, and an expulsive
anti-HO potential acting upon the other. The systems can be implemented in
optical and BEC dual-core waveguides. In the 1D linear system, codimension-one
solutions are found in an exact form for the ground state (GS) and dipole mode
(the first excited state). Generic solutions are produced by means of the
variational approximation, and are found in a numerical form. Exact
codimension-one solutions and generic numerical ones are also obtained for the
GS and vortex states in the 2D system (the exact solutions are found for all
values of the vorticity). Both the trapped and anti-trapped components of the
bound states may be dominant ones, in terms of the norm. The localized modes
may be categorized as bound states in continuum, as they coexist with
delocalized ones. The 1D states, as well as the GS in 2D, are weakly affected
and remain stable if the self-attractive or repulsive nonlinearity is added to
the system. The self-attraction makes the vortex states unstable against
splitting, while they remain stable under the action of the self-repulsion.

##  I Introduction

The combination of the harmonic-oscillator (HO) trapping potential and cubic
nonlinearity is a ubiquitous setting which occurs in diverse physical
realizations. A well-known one is offered by Bose-Einstein condensates (BECs)
with the mean-field nonlinearity [1, 2, 3], loaded in a magnetic or optical
trap – see, e.g., Refs. [4]-[27]. A similar combination of the effective
confinement, approximated by the parabolic profile of the local refractive
index, and the Kerr term is relevant as a model of optical waveguides
[28]-[32]. Models of the same type find other physical realizations too, such
as polariton condensates [33] and networks of Josephson oscillators [34].

The interplay of the self-attractive nonlinearity (self-focusing, in terms of
optics) and trapping potential gives rise to localized modes, which may be
considered as bright solitons confined by the external potential [35, 36]. On
the other hand, dynamics of solitons in expulsive potentials, such as an
inverted HO (anti-HO), is also relevant in various physical settings
[37]-[46]. In terms of optics, expulsive potentials represents anti-
waveguiding setups, which are used in the design of photonic data-processing
schemes [47]-[51]. It is worthy to mention that the interplay of the self-
repulsive nonlinearity and a combination of spatially periodic (lattice) and
HO or anti-HO potentials gives rise to gap solitons with a negative effective
mass, which are expelled by the normal HO potential, and stay trapped under
the action of the anti-HO one [52].

The objective of this work is to introduce a system of linearly coupled one-
and two-dimensional (1D and 2D) “cores” (conduits for photonic or matter
waves), with opposite signs of HO potentials acting in them, one confining and
one expulsive. In BEC, such a system can be built as a pair of parallel cigar-
or pancake-shaped condensates coupled by tunneling of atoms, which may be
enhanced by an additional optical link [53]. Then, the HO and anti-HO
potentials in the cores can be induced by properly shaped red- and blue-
detuned laser beams illuminating the cores (see. e.g., Ref. [54]). In optics,
an effectively 1D coupler composed of planar waveguiding and antiwaveguiding
cores can be fabricated in a straightforward way [55], although a 2D version
of such an optical system is not a realistic one.

We consider these systems both in the linear regime and with inclusion of
intra-core self-attractive or repulsive cubic nonlinearity. In fact, the
linear version of the system is the most relevant one, as the existence of a
trapped states in the system with linearly-coupled components subject to the
action of the confining and expulsive potentials is not obvious – in
particular, because the straightforward definition of spectra does not
directly apply to such a system. A half-trapped linearly-coupled system,
including the trapping HO potential in one component and no potential in the
other, was considered recently [56], but, to the best of our knowledge,
trapped-antitrapped systems were not addressed before.

It is obvious that, irrespective of the existence of confined states in the
system with the anti-trapping component, it also gives rise to a continuous
family of delocalized states, therefore the confined ones, if they exist, may
be considered as a specific case of _bound states in the continuum_ (BIC) [57,
58, 59], alias _embedded modes_ [64]. Recently, applications of BIC of various
types have drawn much interest in photonics [60]-[63].

The presentation is organized below as follows. The 1D and 2D systems are
introduced in Section II. Then, section III reports exact analytical solutions
of the codimension-one type (with one constraint imposed on the system’s
parameters), for both 1D and 2D systems in their linear form. These are
ground-state (GS) and dipole-mode (DM) solutions in 1D, and 2D states with all
integer values of vorticity, $S\geq 0$ ($S=0$ represents the GS in 2D, while
$S=1$ is the 2D counterpart of the DM). The exact solutions play an important
role, as they provide an exact proof of the existence of the bound state in
the presence of the expulsive potential in one component. Generic, although
approximate, analytical results and their numerical counterparts are produced,
for the linear systems, in Section IV. The analytical results provide
eigenvalues of the propagation constant, in terms of optics (or chemical
potential, in BEC) for the GS and DM states in 1D, predicted by the
variational (Rayleigh-Ritz) approximation. Another analytical finding is an
asymptotic form, at $|x|\rightarrow\infty$ (or $r\rightarrow\infty$) of the 1D
(or 2D) delocalized states. Effects of the nonlinearity are considered, in the
numerical form, in Section V, which addresses stationary GS, DM, and 2D vortex
modes in the nonlinear systems, and simulations of the evolution of inputs
produced by exact eigenmodes of the linear system under the action of the
nonlinearity. In the latter case, the system demonstrates the emergence of
robust breathers for moderate nonlinearity, and chaotization when it is too
strong. The vortex states in 2D are, as usual [65], unstable against
spontaneous splitting under the action of self-attraction, and remain stable
in the case of self-repulsion. The paper is concluded by Section VI.

##  II The one- and two-dimensional systems

The 1D system is introduced, in the scaled form, as a pair of linearly-coupled
Schrödinger equations with the HO and anti-HO potentials in the first and
second components. The equations are written in the notation adopted in
optics, with propagation distance $z$ and transverse coordinate $x$:

$\displaystyle iu_{z}+\frac{1}{2}u_{xx}+\lambda
v-\frac{1}{2}x^{2}u+\sigma|u|^{2}u$ $\displaystyle=$ $\displaystyle-\omega u,$
(1) $\displaystyle iv_{z}+\frac{1}{2}v_{xx}+\lambda u+\frac{1}{2}\kappa
x^{2}v+\sigma|v|^{2}v$ $\displaystyle=$ $\displaystyle 0.$ (2)

Here the diffraction coefficients and strength of the HO trapping potential
acting on the $u$ component are normalized to be $1$, $\lambda$ is the
coefficient of the linear coupling, which may be set to be positive,
$\kappa>0$ is the strength of the expulsive potential acting on $v$, and
$\omega$ is a possible propagation-constant mismatch between the components.
Finally, $\sigma=\pm 1$ or $0$ represents the scaled nonlinearity coefficient,
$\sigma=+1$ and $-1$ corresponding to the self-focusing and defocusing signs
of the nonlinearity.

Stationary solutions to Eqs. (1) and (2) are looked for as

$\left\\{u,v\right\\}=\left\\{U(x),V(x)\right\\}\exp\left(-i\mu z\right),$ (3)

with real propagation constant $-\mu$ (in BEC, with $z$ replaced by scaled
time $t$, $\mu$ is the chemical potential), and real functions $U(x)$ and
$V(x)$ satisfying equations

$\displaystyle\left(\mu+\omega\right)U+\frac{1}{2}\frac{d^{2}U}{dx^{2}}+\lambda
V-\frac{1}{2}x^{2}U+\sigma U^{3}=0,$ (4) $\displaystyle\mu
V+\frac{1}{2}\frac{d^{2}V}{dx^{2}}+\lambda U+\frac{1}{2}\kappa x^{2}V+\sigma
V^{3}=0.$ (5)

In fact, the nonlinearity plays a secondary role in the present context, the
most important issue being the existence of trapped modes in the linear
version of Eqs. (4) and (5), which corresponds to $\sigma=0$. The main
objective of the analysis presented below is finding spectra of eigenvalues
$\mu$ for such modes.

The 2D version of the system, written in the polar coordinates
$\left(r,\theta\right)$, is

$\displaystyle iu_{z}+\frac{1}{2}\left(\frac{\partial^{2}}{\partial
r^{2}}+\frac{1}{r}\frac{\partial}{\partial
r}-\frac{1}{r^{2}}\frac{\partial^{2}}{\partial\theta^{2}}\right)u+\lambda
v-\frac{1}{2}r^{2}u+\sigma|u|^{2}u$ $\displaystyle=$ $\displaystyle-\omega u,$
(6) $\displaystyle iv_{z}+\frac{1}{2}\left(\frac{\partial^{2}}{\partial
r^{2}}+\frac{1}{r}\frac{\partial}{\partial
r}-\frac{1}{r^{2}}\frac{\partial^{2}}{\partial\theta^{2}}\right)v+\lambda
u+\frac{1}{2}\kappa r^{2}v+\sigma|v|^{2}v$ $\displaystyle=$ $\displaystyle 0.$
(7)

Stationary solutions to Eqs. (6) and (7) with integer vorticity $S$ are looked
for as

$\left\\{u,v\right\\}=\exp\left(-i\mu
z+iS\theta\right)\left\\{U(r),V(r)\right\\},$ (8)

where real functions $U$ and $V$ satisfy radial equations

$\displaystyle\left(\mu+\omega\right)U+\frac{1}{2}\left(\frac{d^{2}}{dr^{2}}+\frac{1}{r}\frac{d}{dr}-\frac{S^{2}}{r^{2}}\right)U+\lambda
V-\frac{1}{2}r^{2}U+\sigma U^{3}=0,$ (9) $\displaystyle\mu
V+\frac{1}{2}\left(\frac{d^{2}}{dr^{2}}+\frac{1}{r}\frac{d}{dr}-\frac{S^{2}}{r^{2}}\right)V+\lambda
U+\frac{1}{2}\kappa r^{2}V+\sigma V^{3}=0.$ (10)

Conserved quantities of the 1D system are the integral power (norm) and
Hamiltonian

$\displaystyle
P=\int_{-\infty}^{+\infty}\left[\left|u(x)\right|^{2}+\left|v(x)\right|^{2}\right]dx\equiv
P_{u}+P_{v},$ (11) $\displaystyle
H=\int_{-\infty}^{+\infty}\left[\frac{1}{2}\left(\left|u_{x}\right|^{2}+\left|v_{x}\right|^{2}+|u|^{2}-\kappa|v|^{2}\right)-\lambda\left(u^{\ast}v+uv^{\ast}\right)+\frac{\sigma}{2}\left(|u|^{4}+|v|^{4}\right)\right]dx.$
(12)

The 2D system conserves obvious counterparts of integral quantities (11) and
(12), and also the angular momentum,

$M=-i\int_{0}^{\infty}rdr\int_{0}^{2\pi}d\theta\left(u^{\ast}\frac{\partial
u}{\partial\theta}+v^{\ast}\frac{\partial v}{\partial\theta}\right),$ (13)

where $\ast$ stands for the complex conjugate. Obviously, for eigenstates (8)
Eq. (13) have $M=SP$.

The evolution of the wave fields was simulated, in the framework of Eqs. (1),
(2) and (6), (7), with the help of the MATLAB’s ode15s program, which is a
variable-step, variable-order solver based on the numerical differentiation
formulas (NDFs) of orders up to $5$. The spatial discretization was performed
with a variable mesh size $\Delta x=\Delta y\lesssim 0.01$, and the
propagation distance was discretized with a variable step size $\Delta
z\lesssim 0.1$. Stationary equations (4), (5) and (9), (10) were solved
numerically by means of the collocation method, based on the Runge-Kutta
algorithm, with the same values of $\Delta x$ and $\Delta y$. All the
equations were solved with the zero (Dirichlet) boundary conditions.

The size of $\Delta x$, $\Delta y$, and $\Delta z$, as well as the NDF order,
were dynamically adjusted at each iteration step of the simulations, according
to local error estimation, which had to satisfy the predefined tolerance. In
the present work, the relative tolerance (pertaining to the local solution
value) and the absolute tolerance were fixed as $10^{-3}$ and $10^{-6}$,
respectively. This choice made it possible to secure the stability of the
numerical scheme. In particular, no visible change in the solutions was
observed if the calculations were rerun with smaller values of $\Delta x$,
$\Delta y$ (up to $0.001$), and $\Delta z$. In fact, some numerical results
are plotted with values of $\Delta x$, $\Delta y$, and $\Delta z$ somewhat
larger than those which were used in the simulations, as very small values
were not necessary for plotting.

##  III Exact solutions for bound states in the linear system

###  III.1 The ground state (GS)

The linear version of the 1D stationary system, represented by Eqs. (4) and
(5) with $\sigma=0$, admits a codimension-one (non-generic) exact solution for
the system’s GS:

$\displaystyle
U(x)=\left(U_{0}+U_{2}x^{2}\right)\exp\left(-\frac{x^{2}}{2}\right),$ (14)
$\displaystyle V(x)=V_{0}\exp\left(-\frac{x^{2}}{2}\right),$ (15)
$\displaystyle U_{0}=\frac{1+\kappa-2\lambda^{2}}{4\lambda}V_{0},$ (16)
$\displaystyle U_{2}=-\frac{1+\kappa}{2\lambda}V_{0},$ (17)
$\displaystyle\mu_{\mathrm{GS}}=\frac{1}{2}\left(\lambda^{2}+\frac{1}{2}\right)-\frac{\kappa}{4},$
(18)

where $V_{0}$ is an arbitrary amplitude. The codimension-one character of this
solution is determined by the fact that it exists only if the following
relation is imposed on parameters $\omega,$ $\kappa$, and $\lambda$:

$\omega_{\mathrm{GS}}=\frac{9}{4}-\frac{\lambda^{2}}{2}+\frac{\kappa}{4}$ (19)

(cf. Refs. [64] and [66], as concerns the definition of the codimensionality
one). Note that, as it follows from Eqs. (14), (16), and (17), component
$U(x)$ of the exact solution crosses zero at
$x^{2}=\left(1+\kappa-2\lambda^{2}\right)/[2(1+\kappa)]$, under the condition
of

$\lambda^{2}<(1/2)(1+\kappa),$ (20)

Further, in the interval of

$(1/2)(1+\kappa)<\lambda^{2}<(5/2)(1+\kappa),$ (21)

$U(x)$ does not change its sign, but exhibits a local minimum at $x=0$. An
example of this feature is shown below, in the right panel of Fig. 1.

A “naive assumption” may be that the confined mode would be maintained by the
linear coupling between $U$ and $V$, in spite of the action of the expulsive
potential upon the $V(x)$ component, if the naturally trapped one, $U(x)$,
dominates in the system, i.e., if its integral power exceeds the power of $V$,

$P_{u}/P_{v}>1,$ (22)

see Eq. (11). Nevertheless, the confined mode may exist even if condition (22)
does not hold. Indeed, for the exact solution (14)-(17) the power ratio is

$\left(\frac{P_{u}}{P_{v}}\right)_{\mathrm{GS}}=\frac{\lambda^{2}}{4}+\frac{(1+\kappa)^{2}}{8\lambda^{2}}.$
(23)

Straightforward analysis demonstrates that Eq. (23) always yields
$P_{u}/P_{v}>1$ if the strength of the expulsive potential is large enough,
viz.,

$\kappa>2\sqrt{2}-1\approx 1.83,$ (24)

which agrees with the above-mentioned “naive” expectation. On the other hand,
if condition (24) does not hold, Eq. (23) gives $P_{u}/P_{v}<1$ in the
following interval of values of the coupling constant:

$2-\sqrt{4-(1+\kappa)^{2}/2}<\lambda^{2}<2+\sqrt{4-(1+\kappa)^{2}/2}.$ (25)

###  III.2 The dipole mode (first excited state)

In addition to the spatially-even GS solution, an exact odd one (DM), which
represents the first excited state of the system, can be found too:

$\displaystyle
U(x)=\left(U_{1}x+U_{3}x^{3}\right)\exp\left(-\frac{x^{2}}{2}\right),$ (26)
$\displaystyle V(x)=V_{1}x\exp\left(-\frac{x^{2}}{2}\right),$ (27)
$\displaystyle U_{1}=\frac{3(1+\kappa)-2\lambda^{2}}{4\lambda}V_{1},$ (28)
$\displaystyle U_{3}=-\frac{1+\kappa}{2\lambda}V_{1},$ (29)
$\displaystyle\mu_{\mathrm{DM}}=\frac{1}{2}\left(\lambda^{2}+\frac{3}{2}\right)-\frac{3\kappa}{4},$
(30)

where $V_{1}$ is an arbitrary amplitude. It is also a codimension-one
solution, which exists if the following constraint is imposed onto the
parameters:

$\omega_{\mathrm{DM}}=\frac{11}{4}-\frac{\lambda^{2}}{2}+\frac{3\kappa}{4},$
(31)

cf. Eq. (19). Note that the special values of $\omega$ necessary for the
existence of the GS and DM exact solutions, as given by Eqs. (19) and (31),
respectively, cannot be equal for $\kappa>0$. The wave function (26) crosses
zero at finite values of $x$ under condition

$\lambda^{2}<(3/2)(1+\kappa),$ (32)

cf. Eq. (20).

For the exact DM solution, the $P_{u}/P_{v}$ ratio is given by

$\left(\frac{P_{u}}{P_{v}}\right)_{\mathrm{DM}}=\frac{\lambda^{2}}{4}+\frac{3(1+\kappa)^{2}}{8\lambda^{2}},$
(33)

cf. Eq. (23). Accordingly, the $U$ mode is always the dominant one
($P_{u}/P_{v}>1$) in the case of

$\kappa>2\sqrt{2/3}-1\approx 0.63,$ (34)

cf. Eq. (24). At smaller values of $\kappa$, there exist exact DM states with
the dominance of the $V$ mode, i.e., with $P_{u}/P_{v}<1$.

###  III.3 2D states

A particular exact solution of 2D stationary equations (9) and (10) can also
be found, for all integer values of vorticity $S$:

$\displaystyle
U(r)=\left(U_{0}^{\mathrm{(2D)}}+U_{2}^{\mathrm{(2D)}}r^{2}\right)r^{S}\exp\left(-\frac{r^{2}}{2}\right),$
(35) $\displaystyle
V(r)=V_{0}^{\mathrm{(2D)}}r^{S}\exp\left(-\frac{r^{2}}{2}\right),$ (36)
$\displaystyle
U_{0}^{\mathrm{(2D)}}=\frac{(S+1)(1+\kappa)-\lambda^{2}}{2\lambda}V_{0}^{\mathrm{(2D)}},$
(37) $\displaystyle
U_{2}^{\mathrm{(2D)}}=-\frac{1+\kappa}{2\lambda}V_{0}^{\mathrm{(2D)}},$ (38)
$\displaystyle\mu_{\mathrm{2D}}=\frac{1}{2}\left[\lambda^{2}+\left(S+1\right)\left(1-\kappa\right)\right],$
(39)

where $V_{0}^{\mathrm{(2D)}}$ is an arbitrary amplitude. This solution too is
of the codimension-one type, as it exists under the respective constraint:

$\omega_{\mathrm{2D}}=\frac{1}{2}\left[5+S-\lambda^{2}+\left(S+1\right)\kappa\right].$
(40)

It is relevant to mention that the radial wave function (35) crosses zero at
finite $r$ under condition $\lambda^{2}<(S+1)(1+\kappa)$, cf. similar
conditions (20) and (32) in the 1D version of the system.

For the 2D GS solution with $S=0$, the $U$ component dominates, as per Eq.
(22), at $\kappa>1$, cf. Eqs. (24) and (34). In the opposite case, the $V$
component is the dominant one in the interval of

$2-\sqrt{4-(1+\kappa)^{2}}<\lambda^{2}<2+\sqrt{4-(1+\kappa)^{2}},$ (41)

cf. Eq. (25).

Finally, we note that all the exact solutions given by Eqs. (14)-(40) can be
extended to the case of $\kappa<0$, i.e., for the system in which both
components are subject to the action of trapping potentials. In particular,
codimension-one exact solutions of the half-trapped system, corresponding to
$\kappa=0$, were recently found in Ref. [56]. In the latter case, all the
exact solutions are of the BIC type.

##  IV Generic spectra of the bound states

###  IV.1 The ground state (GS) of the 1D system

####  IV.1.1 The variational approximation (VA)

The codimension-one exact solutions for the trapped modes with propagation
constants (18) and (30), obtained above for the 1D system, are valid only
under constraints (19) or (31), respectively. A possibility to predict generic
trapped modes and the spectrum of their propagation constants is offered by
the variational approximation (VA). Usually, it is applied to solitons and
other modes in nonlinear systems [67], but its original form, known as the
Rayleigh-Ritz approximation, was developed for finding spectra of linear
systems, such as the Schrödinger equation in quantum mechanics [68, 69]. In
the present context, it is natural to apply it to the prediction of spectra of
bound states in the system of Eqs. (4) and (5) in the linear limit,
$\sigma=0$.

In the framework of the VA, 1D wave functions of the GS may be approximated by
the simplest ansatz

$\left\\{U_{\mathrm{GS}}^{\mathrm{(VA)}}(x),V_{\mathrm{GS}}^{\mathrm{(VA)}}(x)\right\\}=\pi^{-1/4}\left\\{\cos\eta,\sin\eta\right\\}\exp\left(-\frac{x^{2}}{2}\right),$
(42)

where $\eta$ is a variational parameter, the Gaussian shape is suggested by
the HO trapping potential in Eq. (4), and the ansatz is subject to the
normalization condition,

$\int_{-\infty}^{+\infty}\left[\left(U(x)\right)^{2}+\left(V(x)\right)^{2}\right]dx=1.$
(43)

Next, one multiplies Eq. (4) with $\sigma=0$ by $U(x)$, Eq. (5) by $V(x)$,
integrates each one as $\int_{-\infty}^{+\infty}dx$, and takes the sum of the
two. With regard to normalization condition (43), this procedure yields an
expression for the eigenvalue sought for:

$\displaystyle\mu$ $\displaystyle=$
$\displaystyle\int_{-\infty}^{+\infty}dx\left\\{U(x)\left[-\omega
U(x)-\frac{1}{2}\frac{d^{2}U}{dx^{2}}+\frac{1}{2}x^{2}U(x)\right]\right.$ (44)
$\displaystyle\left.+V(x)\left[-\frac{1}{2}\frac{d^{2}V}{dx^{2}}-\frac{\kappa}{2}x^{2}V(x)\right]-2\lambda
U(x)V(x)\right\\}.$

Formally, Eq. (44) gives an exact expression for $\mu$, but written in terms
of unknown wave functions $U(x)$ and $V(x)$. To obtain an actual result, we
insert ansatz (42) and perform the integration, which yields the following
approximate expression:

$\mu_{\mathrm{GS}}^{\mathrm{(VA)}}=\left(\frac{1}{2}-\omega\right)\cos^{2}\eta+\frac{1}{4}\left(1-\kappa\right)\sin^{2}\eta-\lambda\sin\left(2\eta\right).$
(45)

Now, the VA means minimization of this expression with respect to the free
parameter, $\eta$, i.e., setting

$\frac{d}{d\eta}\left(\mu^{\mathrm{(VA)}}\right)=0.$ (46)

The substitution of expression (45) in Eq. (46) produces the value of $\eta$
sought for:

$\tan\left(2\eta\right)=-\lambda/q_{\mathrm{GS}},$ (47)

where

$q_{\mathrm{GS}}\equiv\frac{1}{2}\left[\frac{1}{4}\left(1+\kappa\right)-\omega\right].$
(48)

Finally, the substitution of value (48) in the VA expression for $\mu$, given
by Eq. (45), leads to the following prediction for the eigenvalue:

$\mu_{\mathrm{GS}}^{{}_{\mathrm{(VA)}}}=\left(\frac{1}{2}-\omega-
q_{\mathrm{GS}}\right)+\sqrt{q_{\mathrm{GS}}^{2}+\lambda^{2}}.$ (49)

In particular, in the limit of $\kappa\rightarrow\infty$, i.e., in the limit
of the very strong expulsive potential acting upon the $V$ component, Eq. (49)
yields

$\mu_{\mathrm{GS}}^{{}_{\mathrm{(VA)}}}\approx\frac{1}{2}-\omega+\frac{4\lambda^{2}}{\kappa},$
(50)

which is close to the GS eigenvalue of the $U$ component decoupled from $V$.
Accordingly, in this limit Eqs. (42), (47), and (48) show that the relative
amplitude of the $V$ component in the bimodal bound state decreases as
$V/U\approx\eta\approx-4\lambda/\kappa$. This result explains why the bound
state can exist even under the action of the extremely strong expulsive
potential acting upon the $V$ component. On the other hand, the VA also admits
GS solutions dominated by the $V$ component, i.e., with $P_{v}>P_{u}$, see Eq.
(22). The boundary of this situation corresponds to $\eta=\pi/4$ in ansatz
(42), i.e., $q_{\mathrm{GS}}=0$, according to Eq. (47). Thus, it follows from
Eq. (48) that the VA predicts the domination of the $V$ component at
$q_{\mathrm{GS}}<0$. i.e., at

$\kappa<4\omega-1.$ (51)

####  IV.1.2 Numerical results for the GS

Comparison of the VA-predicted profile of the GS, produced by Eqs. (42), (43),
(47) and (48) at a generic point in the parameter space,

$\kappa=3,\lambda=2,\omega=1,$ (52)

which satisfies condition (19), with the exact profile, as given by Eqs.
(14)-(17), is displayed in Fig. 1. Unlike the variational ansatz, the exact
analytical solution features a local minimum at $x=0$, as parameters (52) fall
in interval (21). In spite of the discrepancy in the shape, in this case Eqs.
(49) and (48) produce the VA eigenvalue
$\mu_{\mathrm{GS}}^{{}_{\mathrm{(VA)}}}=1.5$, which _precisely_ coincides with
the exact eigenvalue given by Eq. (18).

Figure 1: The right and left panels display, respectively, exact 1D GS
(ground-state) solution of the linearized ($\sigma=0$) version of Eqs. (4) and
(5), and its VA-produced counterpart, at parameter values (52). Both the exact
solution and VA yield, in this case, equal eigenvalues,
$\mu_{\mathrm{GS}}=\mu_{\mathrm{GS}}^{{}_{\mathrm{(VA)}}}=1.5$. In these
plots, the amplitude of the $V$ component is set to be $V_{0}=-1$. The top and
bottom profiles depict components $U$ and $V$, respectively.

The comprehensive results for the eigenvalue of the GS, as predicted by the VA
and produced by the numerical solution of Eqs. (4) and (5) with $\sigma=0$,
are summarized, respectively, in the right and left panels of Fig. 2. It is
seen that the trapped GS exists at all values of the strength of the expulsive
potential ($\kappa$). Further, the VA produces quite accurate results, unless
the strength $\lambda$ of the linear coupling between the components $U$ and
$V$, upon which the HO and anti-HO potentials act, is too small. If $\lambda$
is small, it is not relevant to adopt the same functional form of both
components in the ansatz, therefore the variational ansatz (42) is inaccurate.

Figure 2: Heatmaps (gray-scale maps, in the black-and-white rendition) of the
GS eigenvalue in planes of strength $\kappa$ of the expulsive potential and
mismatch $\omega$ between the components, as produced by the numerical
solution of the linearized equations (4) and (5), and by the VA (the right and
left panels, respectively). The top, middle, and bottom panels pertain,
severally, to values of the inter-component coupling constant $\lambda=0.1$,
$1.0$, and $2.0$. The black line in the right bottom panel represents relation
(19), at which the exact solution (14)-(18) is available.

The straight black line in the right bottom panel of Fig. 2 is the locus of
points defined by Eq. (19), at which the exact solution is provided by Eqs.
(14)-(18). Along this line, the eigenvalue of the numerically found GS
solution is precisely equal (up to the numerical accuracy) to the eigenvalue
of the exact solution, as given by Eq. (18).

###  IV.2 The dipole mode (DM) in the 1D system

####  IV.2.1 The VA for the DM

The variational ansatz for the spatially odd DM solution is adopted as

$\left\\{U_{\mathrm{DM}}^{\mathrm{(VA)}}(x),V_{\mathrm{DM}}^{\mathrm{(VA)}}(x)\right\\}=\sqrt{2}\pi^{-1/4}\left\\{\cos\eta,\sin\eta\right\\}x\exp\left(-\frac{x^{2}}{2}\right),$
(53)

cf. Eq. (42), which is also subject to normalization (43). Substituting this
in expression (44) yields

$\mu_{\mathrm{DM}}=\left(\frac{3}{2}-\omega\right)\cos^{2}\eta+\frac{3}{4}\left(1-\kappa\right)\sin^{2}\eta-\lambda\sin\left(2\eta\right),$
(54)

cf. Eq. (45). Then, the variational equation (46), applied to expression (54),
produces the result

$\tan\left(2\eta\right)=-\lambda/q_{\mathrm{DM}},$ (55)
$q_{\mathrm{DM}}\equiv\frac{1}{2}\left[\frac{3}{4}\left(1+\kappa\right)-\omega\right],$
(56)

cf. Eqs. (47) and (48). The substitution of this in Eq. (54) leads to the
following eigenvalue:

$\mu_{\mathrm{DM}}^{\mathrm{(VA)}}=\frac{3}{2}-\omega-
q_{\mathrm{DM}}+\sqrt{q_{\mathrm{DM}}^{2}+\lambda^{2}},$ (57)

cf. Eq. (49). In particular, in the limit of $\kappa\rightarrow\infty$, Eq.
(57) gives

$\mu_{\mathrm{DM}}^{\mathrm{(VA)}}\approx\frac{3}{2}-\omega+\frac{4\lambda^{2}}{3\kappa}.$
(58)

Similar to Eq. (50), the latter result is close to the eigenvalue of the first
excited state of the $U$ component decoupled from $V$.

Also similar to what is considered above for the GS solution, the VA predicts
that the $V$ component is the dominant one in the DM ($P_{v}>P_{u}$) at
$q_{\mathrm{DM}}<0$, i.e., at $\kappa<(4/3)\omega-1$, cf. Eq. (51).

####  IV.2.2 Numerical results for the DM

At characteristic values of parameters,

$\kappa=1/3,\lambda=2,\omega=1,$ (59)

which satisfy condition (31), comparison of the variational profile of the DM,
as produced by Eqs. (53), (43), (55) and (56), with the exact profile, as
given by Eqs. (26)-(29) is displayed in Fig. 3. It is worthy to note that, in
comparison with the GS (see Fig. 1), the VA profile for the DM is closer to
its exact counterpart, although the two are not identical [ratios of
amplitudes of the $U$ and $V$ components at parameter values (59) are $1$ and
$1.004$, as given by the VA and exact solution, respectively]. Further, in
this case Eqs. (57) and (56) produce the VA eigenvalue
$\mu_{\mathrm{DM}}^{\mathrm{(VA)}}=2.5$, which precisely coincides with the
exact eigenvalue given by Eq. (30).

Figure 3: The right and left panels display, respectively, the exact 1D DM
(dipole-mode) solution of the linearized ($\sigma=0$) version of Eqs. (4) and
(5), and its variational counterpart, for parameters given by Eq. (59). Both
the exact solution and VA give, in this case, equal eigenvalues,
$\mu_{\mathrm{DM}}=\mu_{\mathrm{DM}}^{{}_{\mathrm{(VA)}}}=2.5$. In these
plots, the amplitude of the $V$ component is $V_{0}=-1$. In each panel, the
right and left profiles depict components $U$ and $V$, respectively.

Full results for the DM eigenvalue, as predicted by the VA, and as produced by
the numerical solution of Eqs. (4) and (5) with $\sigma=0$, are collected in
the left and right panels of Fig. 4, respectively. As well as the GS, the DM
trapped states exist at all values of the expulsive-potential’s strength,
$\kappa$. Also similar to what is observed in Fig. 2 for the GS, the VA
predicts accurate results, unless the linear coupling $\lambda$ is very small.
The explanation of the latter point is the same as for the GS, viz., ansatz
(53) is inaccurate for small $\lambda$.

Figure 4: Heatmaps (gray-scale maps, in the black-and-white rendition) of the
DM eigenvalue in the $\left(\kappa,\omega\right)$ planes, as produced by the
numerical solution of the linearized equations (4) and (5), and by the VA
(right and left panels, respectively). The top, middle, and bottom panels
pertain to values of the coupling constant $\lambda=0.1$, $1.0$, and $2.0$.
The black line in the right bottom panel represents relation (31), at which
the exact solution is given by Eqs. (26)-(30).

The straight black line in the right bottom panel of Fig. 4 is defined by Eq.
(31), as the locus of points at which the exact DM solution exists. Along this
line, the numerically found GS eigenvalue is precisely equal to its
counterpart given by the exact solution, as per Eq. (18).

###  IV.3 Numerical results for the 2D modes

It is commonly known that, due to the separability of the HO potential, the
wave functions trapped in the 2D HO potential can be constructed as products
of 1D eigenstates, and the 2D eigenvalue spectrum can be constructed,
respectively, from its 1D counterpart. However, in the present system the
factorization principle for the eigenstates is broken by linear-coupling
terms. Therefore, it is necessary to compute eigenvalues of the 2D states
numerically. It was thus found that, similar to the 1D case, the solutions for
the bound states (at least, with vorticities $S=0$ and $1$) are produced by
the numerical solution of the linear version of Eqs. (9) and (10) (with
$\sigma=0$) at all values of parameters $\kappa$, $\lambda$, and $\omega$. A
heatmap of the eigenvalues of the 2D states with $S=0$ is displayed in Fig. 5.

Figure 5: The heatmap (gray-scale map, in the black-and-white rendition) of
the eigenvalue of the 2D trapped state with $S=0$ (zero vorticity) in the
$\left(\kappa,\omega\right)$ plane at $\lambda=1$, as produced by the
numerical solution of the linearized equations (9) and (10). The black line
represents relation (40), at which the exact 2D solution is given by Eqs.
(35)-(39).

The straight black line crossing the map in Fig. 5 is defined by Eq. (40).
Along this line, the exact solution is given by Eqs. (35)-(38). The respective
eigenvalue, given by Eq. (39), exactly coincides with its numerically found
counterpart.

###  IV.4 Delocalized states

The localized states considered above are dominated by the trapped component,
$U$. The same system admits delocalized states, dominated by the anti-trapped
component, $V$. Exact solutions for delocalized states cannot be found even if
the system is taken in the linearized form, with $\sigma=0$, but it is
possible to construct an asymptotic form of such states at
$|x|\rightarrow\infty$. The consideration of Eqs. (4) and (5) yields:

$\displaystyle
V_{\mathrm{deloc}}(x)\underset{|x|\rightarrow\infty}{\approx}V_{0}|x|^{-1/2}\cos\left(\frac{\sqrt{\kappa}}{2}x^{2}+\frac{\mu}{\sqrt{\kappa}}\ln\left(|x|\right)\right),$
(60) $\displaystyle
U_{\mathrm{deloc}}(x)\underset{|x|\rightarrow\infty}{\approx}V_{0}\frac{2\lambda}{1+\kappa}|x|^{-5/2}\cos\left(\frac{\sqrt{\kappa}}{2}x^{2}+\frac{\mu}{\sqrt{\kappa}}\ln\left(|x|\right)\right),$
(61)

where $V_{0}$ is an arbitrary constant. Obviously, the total power of the
$V$-component of this solution, defined as per Eq. (11), diverges as $\int
dx/|x|$ at $|x|\rightarrow\infty$, therefore such states are categorized as
delocalized ones. The principal difference of the delocalized states from the
trapped ones, in addition to the divergence of the norm, is that they form a
continuous spectrum, with the delocalized solutions existing at all values of
$\mu$, as seen from Eqs. (60) and (61), while the localized states form the
discrete spectrum: GS, DM, etc. Furthermore, the trapped states may be
considered as BIC ones, as discrete values of $\mu$ corresponding to them are
embedded into the continuous spectrum of the delocalized states.

A typical example of the delocalized states is displayed in Fig. 6(a), along
with the analytical asymptotic form, predicted by Eqs. (60) and (61), for
parameters

$\kappa=0.5,\lambda=2,\omega=0.375,\mu=2.125$ (62)

These values are chosen because they satisfy Eq. (19), hence the exact GS
solution, given by Eqs. (14)-(17), is available in this case, with the
corresponding propagation constant $\mu=2.125$ given by Eq. (18). This GS
solution is plotted in Fig. 6(b). Thus, the existence of the delocalized state
with exactly the same value of $\mu$ directly corroborates that the localized
states may be considered as ones of the BIC type. Further, it is seen that the
overall shape of the analytical prediction of the delocalized state is quite
accurate, some discrepancy being represented by a phase shift of the
oscillating carrier waves. In particular, the numerical solution corroborates
the fact that the dominant role is played by the $V$ component.

Figure 6: (a) Numerically found shapes of the two components of a delocalized
mode with parameters given by Eq. (62), and their counterparts predicted by
the asymptotic expressions (60) and (61). (b) The exact GS solution given by
Eqs. (14)-(18) for the same parameters, with the top and bottom profiles
depicting $U(x)$ and $V(x)$, respectively.

Similarly, in the 2D case the asymptotic form of solutions to Eqs. (9) and
(10) for delocalized states is

$\displaystyle
V_{\mathrm{deloc}}^{\mathrm{(2D)}}(r)\underset{r\rightarrow\infty}{\approx}V_{0}r^{-1}\cos\left(\frac{\sqrt{\kappa}}{2}r^{2}+\frac{\mu}{\sqrt{\kappa}}\ln
r\right),$ (63) $\displaystyle
U_{\mathrm{deloc}}^{\mathrm{(2D)}}(r)\underset{r\rightarrow\infty}{\approx}V_{0}\frac{2\lambda}{1+\kappa}r^{-3}\cos\left(\frac{\sqrt{\kappa}}{2}r^{2}+\frac{\mu}{\sqrt{\kappa}}\ln
r\right),$ (64)

As well as in the 1D case, the 2D delocalized states form a continuous
spectrum, existing at all values of $\mu$, and their total power (norm),

$P_{\mathrm{2D}}=2\pi\int_{0}^{\infty}\left[U^{2}(r)+V^{2}(r)\right]rdr,$ (65)

diverges at $r\rightarrow\infty$ as $\int dr/r$.

##  V Bound states and their dynamics in nonlinear systems

###  V.1 Ground states and dipole modes in 1D

As mentioned above, the results obtained for the linear systems are most
essential for the analysis of the concept of trapped states in the expulsive
potential, maintained by the linear coupling to the component confined by the
trapping potential. Nevertheless, it is also interesting to consider the role
of the nonlinearity in Eqs. (1), (2) and (4), (5).

First, taking into regard the nonlinear terms with $\sigma=+1$ or $-1$ (self-
attraction or repulsion) in Eqs. (4) and (5) leads to moderate deformation of
the 1D GS solution, as shown in Fig. 7 for the set of parameters

$\kappa=1,\lambda=5,\omega=-10.$ (66)

These values satisfy condition (19), hence the solution with $\sigma=0$ is
given in the exact form by Eqs. (14)-(17), with the respective eigenvalue
$\mu=12.5$ given by Eq. (18). Naturally, the self-compression ($\sigma=+1$)
and repulsion ($\sigma=-1$) tend to make the trapped states slightly narrower
or broader, respectively. Direct simulations of the perturbed evolution in the
framework of Eqs. (1) and (2) demonstrate that that the nonlinearly deformed
solutions remain stable (not shown here in detail).

The nonlinear shift of the eigenvalues can be roughly estimated, in the
Thomas-Fermi approximation, by combining Eqs. (4) and (5) at the central point
and neglecting the second-derivative terms,

$\delta\mu\simeq-\sigma\left(U_{0}^{4}+V_{0}^{4}\right)/\left(U_{0}^{2}+V_{0}^{2}\right)\simeq\mp
0.85$ (67)

for $\sigma=\pm 1$. This estimate is consistent with the numerically found
shift $\left|\delta\mu\right|\simeq 0.7$ for the nonlinear solutions displayed
in Fig. 7. Furthermore, the sign of $\delta\mu$ in Eq. (67) implies that the
GS family in the nonlinear system with $\sigma=+1$ satisfies the Vakhitov-
Kolokolov (VK) criterion, $d\mu/dP<0$ [recall $P$ is the total power defined
as per Eq. (11)], which is the well-known condition necessary for the
stability of GS modes in systems of the nonlinear-Schrödinger type with self-
attraction [70, 71, 72]. For $\sigma=-1$, Eq. (67) implies $d\mu/dP>0$, which
means that, in the case of the self-repulsion, the GS solutions satisfy the
anti-VK criterion, which is a necessary stability condition for ground states
in the case of self-repulsion [52].

Figure 7: The left, central, and right panels show, respectively, the exact 1D
GS solution at parameter values (66) and its counterparts produced by the
numerical solution of Eqs. (4) and (5) for $\sigma=+1$ and $-1$. The
eigenvalues of the three solutions are $\mu=12.5$, $11.9735$, and $13.1925$.
All the solutions are produced with the amplitude of the $U$ component
$U(x=0)=1$. The top and bottom profiles depict components $U(x)$ and $V(x)$,
respectively.

The nonlinearity produces a still smaller change of the shape and eigenvalue
of the trapped DM. As an example, Fig. 8 displays it for parameters

$\kappa=2.5,\lambda=10,\omega=-45.375,$ (68)

which satisfy condition (40), hence the linear DM solution (shown in the left
panel of Fig. 8) is available in the exact form, according to Eqs. (26)-(29),
with the respective eigenvalue $\mu=48.875$ given by Eq. (30). Note that the
sign of the nonlinearity-induced shift of the eigenvalue is the same as in Eq.
(67).

Figure 8: The left and right panels show, respectively, the exact 1D DM
solution at parameter values (68) and its counterpart produced by the
numerical solution of Eqs. (4) and (5) for $\sigma=+1$. The eigenvalues of the
linear and nonlinear solutions are, respectively, $\mu=48.875$ and $48.3634$.
The solutions are produced with the amplitude of the $U$ component
$U_{\max}=1$. The large- and small-amplitude profiles depict components $U$
and $V$, respectively. The solution for $\sigma=-1$ (not shown here) is very
close to these ones.

To explore robustness of bound states in the present setting, it is also
relevant to simulate evolution of an input, taken as an analytical or
numerical eigenstate of the linear system, in the framework of the full
nonlinear system of Eqs. (1) and (2). The conclusion is that, if the
nonlinearity is moderately strong (in other words, if the input’s amplitude is
not too large), the linear eigenstate spontaneously transforms into a robustly
oscillating breather. The respective examples for the GS and DM are presented,
severally, in Fig. 9 for

$\kappa=1,\lambda=6,\omega=-15.5,$ (69)

and in Fig. 10 for

$\kappa=0.5,\lambda=5,\omega=-9.375.$ (70)

Figure 9: Robust oscillations produced by simulations of Eqs. (1) and (2) with
parameters (69) and $\sigma=+1$, and the input given by the linear GS
eigenmode (14)-(17) with amplitude $V_{0}=-0.05$ (it corresponds to
$U_{0}=0.146$). Figure 10: The same as in Fig. 9, but for parameters (70) and
the input given by the linear DM eigenmode (26)-(29) with amplitude
$V_{1}=-0.5$ (it corresponds to amplitude $U_{\max}=0.716$).

The regular oscillations carry over into chaotic dynamics when the amplitude
exceeds a certain critical value. For instance, the GS input, given by Eqs.
(14)-(17) with parameters (69) and amplitude $V_{0}$, evolves into a chaotic
state at
$\left|V_{0}\right|>\left(\left|V_{0}\right|\right)_{\mathrm{crit}}\simeq
0.75$ for $\sigma=+1$, and
$\left|V_{0}\right|>\left(\left|V_{0}\right|\right)_{\mathrm{crit}}\simeq 1$
for $\sigma=-1$ (these values correspond to amplitudes $U_{0}\simeq 2.3$ and
$U_{0}\simeq 3$, respectively). It is natural that the chaotic behavior
commences later in the case of the self-defocusing nonlinearity. Similarly, in
the case of the DM input given by Eqs. (14-(17) with parameters (70), the
chaotization threshold is found at
$\left(\left|V_{0}\right|\right)_{\mathrm{crit}}\simeq 2$ [which corresponds
to $\left(\left|U(x)\right|\right)_{\max}\simeq 3$] for both signs of the
nonlinearity, $\sigma=\pm 1$.

An example of the spontaneously established chaos is displayed in Fig. 11. It
demonstrates that the chaotization effectively destroys the coupling between
the two components; as a result, the $v$ field expands over the entire
integration domain, under the action of the anti-OH potential, while the $u$
field stays confined by the HO potential acting upon it.

Figure 11: The same as in Fig. 10, but for amplitude $V_{1}=2.5$ of the DM
input.

###  V.2 The ground and vortical states in 2D

As shown in Figs. (12) and (13), the moderate nonlinearity included in 2D
stationary equations (9) and (10) leads to weak deformation of the radial
profile of the GS ($S=0$) and vortices, and a very small shift of their
propagation constants. In particular, Fig. (12) displays the GS radial
profiles for parameters

$\kappa=1,\lambda=5,\omega=-12,$ (71)

which satisfy condition (40) with $S=0$, thus supplying the exact solution for
$\sigma=0$, as given by Eqs. (35)-(39).

Figure 12: The left and right panels display, respectively, the radial profile
of the exact solution (35)-(39) for the 2D GS ($S=0$) in the linear system
($\sigma=0$), with parameters (71), $\mu=12.5$, and its counterpart produced
by the numerical solution of Eqs. (9) and (10) with $\sigma=+1$, the
respective eigenvalue being $\mu=12.10$. The amplitude of the $U$ component of
both solutions is set to be $U_{0}=1$. The top and bottom profiles depict
components $U(r)$ and $V(r)$, respectively.

Note that the difference between the eigenvalues of the nonlinear and linear
solutions in Fig. (12), $\delta\mu\approx-0.4$, demonstrates that the 2D GS
solutions also satisfy the VK criterion in the case of the self-focusing
nonlinearity. In agreement with this fact, direct simulations corroborate the
stability of these solutions. Similarly, in the case of self-defocusing,
$\sigma=-1$, the nonlinear deformation and eigenvalue shift are very small
too, and the respective GS solutions satisfy the anti-VK criterion. Their
stability was also verified in direct simulations (not shown here).

Typical examples of the exact linear solution for the vortex mode with $S=1$
and its nonlinear counterpart, produced by the numerical solution of Eqs. (9)
and (10) with $\sigma=+1$, are presented in Fig. 13. The parameters are chosen
as

$\kappa=0.5,\lambda=10,\omega=-46.5,$ (72)

satisfying condition (40) In this case too, the nonlinearity-induced
deformation of the radial profile and eigenvalue shift, $\delta\mu$, are
relatively small, and the sign of $\delta\mu$ is the same as given by Eq.
(67).

Figure 13: The same as in Fig. 12, but for vortex states with $S=1$ and
parameters (72). The eigenvalues are $\mu=50.5$ and $48.5213$ for the linear
and nonlinear solutions, respectively. The amplitude of the $U$ component of
both solutions is set to be $\left(U(r)\right)_{\max}=1$. The top and bottom
profiles depict components $U(r)$ and $V(r)$, respectively.

While direct simulations demonstrate that the 2D states with $S=0$ are
completely stable under the action of the nonlinearity of either sign (not
shown here in detail), a well-known problem for localized vortex states in
systems with the self-attractive nonlinearity is their instability against
spontaneous splitting [74, 12, 75, 76] (see also a review in Ref. [65]). For
the present system, we addressed this issue by means of direct simulations of
the evolution of 2D states with $S=1$ in the framework of Eqs. (6) and (7).
While we did not aim to explore the entire family of the solutions, the result
is that the vortices are indeed unstable against splitting into a pair of
fragments in the self-focusing system (with $\sigma=+1$), which is a generic
outcome of the instability development known in other systems [65]. A typical
example of the splitting is displayed in Fig. 14. The rotation of the pair,
observed in the figure, provides conservation of the angular momentum (13).

Figure 14: Spontaneous splitting of the vortex state with $S=1$, produced by
simulations of Eqs. (6) and (7) with parameters (72). The input was provided
by the numerical solution of the stationary equations which is plotted in the
right panel of Fig. 13. The three plots display distributions of density
$\left|u\left(x,y;z\right)\right|^{2}$ at values of the propagation distance
$z=0$ (input), $3.8$, and $6.2$.

On the other hand, direct simulations demonstrate that vortices with $S=1$ are
stable in the self-defocusing system ($\sigma=-1$). An example of the radial
profile of the vortex in this case is plotted (along with its counterpart
provided by the exact solution of the linear system) in Fig. 15, for
parameters

$\kappa=1,\lambda=7,\omega=-20.5,$ (73)

that satisfy Eq. (40). In this case, the nonlinear deformation and shift of
the propagation constant are quite small too.

Figure 15: The same as in Fig. 13, but for vortex states with $\sigma=-1$ and
parameters (73). The eigenvalues are $\mu=24.5$ and $27.011$ for the linear
and nonlinear solutions, respectively. The amplitude of the $U$ component of
both solutions is set to be $\left(U(r)\right)_{\max}=1$. The top and bottom
profiles depict components $U(r)$ and $V(r)$, respectively.

Stability of vortices under the action of the self-repulsion is corroborated
by direct simulations, see Fig. 16.

Figure 16: Stability of the vortex state with $S=1$, whose stationary profile
is shown in Fig. 15(b), corroborated by simulations of Eqs. (6) and (7) with
parameters (73) and $\sigma=-1$. Left and right plots display, respectively,
distributions of density $\left|u\left(x,y;z\right)\right|^{2}$ at $z=0$ and
$10$.

##  VI Conclusion

The objective of this work is to demonstrate the existence of stable bound
states in the 1D and 2D linearly-coupled two-component systems, with the
trapping HO (harmonic-oscillator) and expulsive anti-HO potentials acting upon
the components. The systems can be realized in optics and BEC. Exact
analytical solutions, which are subject to the codimension-one constraints,
and generic solutions, produced by the VA (variational approximation) and
obtained in the numerical form, demonstrate that the linear systems support
the GS (ground state) and DM (dipole mode, i.e., the first excited state), as
well as 2D vortex states with all integer values of the vorticity, at all
values of the system’s parameters. Thus, the linear coupling to the trapped
component makes it possible to maintain robust localized states in the
component subject to the action of the expulsive potential. In most cases, the
trapped component is the dominant one, in terms of the integral power (norm).
Nevertheless, there is a parameter region in which the anti-trapped component
has a larger power. The confined modes supported by this system may be
considered as BIC (bound states in continuum), as they coexist with
delocalized states which form the continuous spectrum. The asymptotic form of
the delocalized solutions is obtained analytically. The inclusion of the self-
attractive or repulsive nonlinearity slightly deforms the 1D states, as well
as the 2D GS, which remain stable solutions. On the other hand, the self-
attraction leads to the splitting instability of vortices, while they remain
stable under the action of self-repulsion.

The work may be extended by considering higher-order eigenstates in 1D, and
performing a more systematic analysis of 2D vortex states. It may also be
interesting to consider coherent motion of self-trapped states in the 1D and
2D systems, cf. Refs. [77]) and [10].

## Acknowledgment

This work was supported, in part, by the Israel Science Foundation through
grant No. 1286/17.

## References

  * [1] C. J. Pethick and H. Smith, Bose-Einstein Condensation in Dilute Gases (Cambridge University Press, Cambridge, 2002).
  * [2] L. P. Pitaevskii and S. Stringari, Bose-Einstein Condensation (Oxford University Press, Oxford 2003).
  * [3] P. G. Kevrekidis, D. J. Frantzeskakis, and R. Carretero-González, Emergent Nonlinear Phenomena in Bose-Einstein Condensates: Theory and Experiment (Springer: Heidelberg, 2008).
  * [4] S. Grossmann and M. Holthaus, On Bose-Einstein condensation in harmonic traps, Phys. Lett. A 208, 188-192 (1995).
  * [5] Y. Castin and R. Dum, Bose-Einstein condensates in time dependent traps, Phys. Rev. Lett. 77, 5315-5319 (1996).
  * [6] A. L. Fetter and D. L. Feder, Beyond the Thomas-Fermi approximation for a trapped condensed Bose-Einstein gas, Phys. Rev. A 58, 3185-3194 (1998).
  * [7] H. Pu, C. K. Law, J. H. Eberly, and N. P. Bigelow, Coherent disintegration and stability of vortices in trapped Bose condensates, Phys. Rev. A 59, 1533-1537 (1999).
  * [8] B. I. Schneider and D. L. Feder, Numerical approach to the ground and excited states of a Bose-Einstein condensed gas confined in a completely anisotropic trap, Phys. Rev. A 59, 2232-2242 (1999).
  * [9] S. K. Adhikari, Numerical solution of the two-dimensional Gross-Pitaevskii equation for trapped interacting atoms, Phys. Lett. A 265, 91-96 (2000).
  * [10] T. Busch and J. R. Anglin, Motion of dark solitons in trapped Bose-Einstein condensates, Phys. Rev. Lett. 84, 2298-2301 (2000).
  * [11] Y. S. Kivshar, T. J. Alexander, and S. K. Turitsyn, Nonlinear modes of a macroscopic quantum oscillator, Phys. Lett. 278, 225-230 (2001).
  * [12] T. J. Alexander and L. Bergé, Ground states and vortices of matter-wave condensates and optical guided waves, Phys. Rev. E 65, 026611 (2002).
  * [13] G. Huang, J. Szeftel and S. Zhu, Dynamics of dark solitons in quasi-one-dimensional Bose-Einstein condensates, Phys. Rev. A 65, 053605 (2002).
  * [14] N. G. Parker, N. P. Proukakis, M. Leadbeater, and C. S. Adams, Soliton-sound interactions in quasi-one-dimensional Bose-Einstein condensates, Phys. Rev. Lett. 90, 220401 (2003).
  * [15] D. E. Pelinovsky, D. J. Frantzeskakis, and P. G. Kevrekidis, Oscillations of dark solitons in trapped Bose-Einstein condensates, Phys. Rev. E 72, 016615 (2005).
  * [16] V. A. Brazhnyi and V. V. Konotop, Stable and unstable vector dark solitons of coupled nonlinear Schrödinger equations: Application to two-component Bose-Einstein condensates, Phys. Rev. E 72, 026616 (2005).
  * [17] M. I. Merhasin, B. A. Malomed, and R. Driben, Transition to miscibility in a binary Bose-Einstein condensate induced by linear coupling, J. Physics B 38, 877-892 (2005).
  * [18] D. Mihalache, D. Mazilu, B. A. Malomed, and F. Lederer, Vortex stability in nearly-two-dimensional Bose-Einstein condensates with attraction, Phys. Rev. A 73, 043615 (2006).
  * [19] H. E. Nistazakis, D. J. Frantzeskakis, P. G. Kevrekidis, B. A. Malomed, and R. Carretero-Gonzalez, Bright-dark soliton complexes in spinor Bose-Einstein condensates, Phys. Rev. A 77, 033612 (2008).
  * [20] N. G. Parker, N. G. Proukakis, and C. S. Adams, Dark soliton decay due to trap anharmonicity in atomic Bose-Einstein condensates, Phys. Rev. A 81, 033606 (2010).
  * [21] S. Alama, L. Bronsard, A. Contreras, D. E. Pelinovsky, Domains walls in the coupled Gross-Pitaevskii equations, Arch. Rat. Mech. Appl. 215, 579-615 (2015).
  * [22] Z. Chen, Y. Li, B. A. Malomed, and L. Salasnich, Spontaneous symmetry breaking of fundamental states, vortices, and dipoles in two and one-dimensional linearly coupled traps with cubic self-attraction, Phys. Rev. A 96, 033621 (2017).
  * [23] T. Bland, N. G. Parker, N. P. Proukakis, and B. A. Malomed, Probing quasi-integrability of the Gross-Pitaevskii equation in a harmonic-oscillator potential, J. Phys. B: At. Mol. Opt. Phys. 51, 205303 (2018).
  * [24] H. Sakaguchi and B. A. Malomed, Symmetry breaking in a two-component system with repulsive interactions and linear coupling, Comm. Nonlin. Sci. Num. Sim. 92, 105496 (2020).
  * [25] P. Bizoń, F. Ficek, D. E. Pelinovsky, and S. Sobieszek, Ground state in the energy super-critical Gross-Pitaevskii equation with a harmonic potential, Nonlinear Anal. 210, 112358 (2021).
  * [26] A. Biasi, O. Evnin, and B. A. Malomed, Fermi-Pasta-Ulam phenomena and persistent breathers in the harmonic trap, Phys. Rev. E 104, 034210 (2021).
  * [27] B. A. Malomed, New findings for the old problem: Exact solutions for domain walls in coupled real Ginzburg-Landau equations, Phys. Lett. A 432, 127802 (2021).
  * [28] S. Raghavan and G. P. Agrawal, Spatiotemporal solitons in inhomogeneous nonlinear media, Opt. Commun. 180, 377-382 (2000).
  * [29] D. A. Zezyulin, G. L. Alfimov, and V. V. Konotop, Nonlinear modes in a complex parabolic potential, Phys. Rev. A 81, 013606 (2010).
  * [30] E. G. Charalampidis, P. G. Kevrekidis, D. J. Frantzeskakis, and B. A. Malomed, Dark-bright solitons in coupled NLS equations with unequal dispersion coefficients, Phys. Rev. E 91, 012924 (2015).
  * [31] Y. Zhang, X. Liu, M. R. Belić, W. Zhong, Y. Zhang, and M. Xiao, Propagation dynamics of a light beam in a fractional Schrödinger equation, Phys. Rev. Lett. 115, 180403 (2015).
  * [32] T. Mayteevarunyoo, B. A. Malomed, and D. V. Skryabin, One- and two-dimensional modes in the complex Ginzburg-Landau equation with a trapping potential, Opt. Exp. 26, 8849-8865 (2018).
  * [33] R. Balili, V. Hartwell, D. Snoke, L. Pfeiffer, and K. West, Bose-Einstein condensation of microcavity polaritons in a trap, Science 316, 1007-1010 (2007).
  * [34] M. Leib, F. Deppe, A. Marx, R. Gross, and M. J. Hartmann, Networks of nonlinear superconducting transmission line resonators, New J. Phys. 14, 075024 (2012).
  * [35] Y. S. Kivshar and G. P. Agrawal, Optical Solitons: From Fibers to Photonic Crystals (Academic Press: San Diego, 2003).
  * [36] T. Dauxois, M. Peyrard, Physics of Solitons (Cambridge University Press: Cambridge, 2006).
  * [37] L. D. Carr and Y. Castin, Dynamics of a matter-wave bright soliton in an expulsive potential, Phys. Rev. A 66, 063602 (2002).
  * [38] L. Salasnich, Dynamics of a Bose-Einstein-condensate bright soliton in an expulsive potential, Phys. Rev. A 70, 053617 (2004).
  * [39] Z. X. Liang, Z. D. Zhang, and W. M. Liu, Dynamics of a bright soliton in Bose-Einstein condensates with time-dependent atomic scattering length in an expulsive parabolic potential, Phys. Rev. Lett. 94, 050402 (2005).
  * [40] W. B. Cardoso, A. T. Avelar, and D. Bazeia, Bright and dark solitons in a periodically attractive and expulsive potential with nonlinearities modulated in space and time, Nonlin. Analysis Real World Appl. 11, 4269-4274 (2010).
  * [41] S. Rajendran, P. Muruganandam, M. Lakshmanan, Bright and dark solitons in a quasi-1D Bose-Einstein condensates modelled by 1D Gross-Pitaevskii equation with time-dependent parameters, Physica D 239, 366-386 (2010).
  * [42] N.H. Xuan, M. Zuo, Matter-wave solitons in two-component Bose-Einstein condensates with tunable interactions and time varying potential, Comm. Theor. Phys. 56, 1035-1040 (2011).
  * [43] S. Rajendran, M. Lakshmanan, P. Muruganandam, Matter wave switching in Bose-Einstein condensates via intensity redistribution soliton interactions, J. Math. Phys. 52, 023515 (2011).
  * [44] A. del Campo, M.G. Boshier, Shortcuts to adiabaticity in a time-dependent box, Sci. Rep. 2, 648 (2012).
  * [45] R. Radha, P. S. Vinayagam, J. B. Sudharsan, and B. A. Malomed, Persistent bright solitons in sign-indefinite coupled nonlinear Schrödinger equations with a time-dependent harmonic trap, Comm. Nonlin. Science Num. Sim. 31, 30-39 (2016).
  * [46] Y. V. Kartashov, V. V. Konotop, Stable nonlinear modes sustained by gauge fields, Phys. Rev. Lett. 125, 054101 (2020).
  * [47] B. V. Gisin and A. A. Hardy, Stationary solutions of plane nonlinear-optical antiwaveguides, Opt. Quant. Elect. 27, 565-575 (1995).
  * [48] B. V. Gisin, A. Kaplan, and B. A. Malomed, Spontaneous symmetry breaking and switching in planar nonlinear optical antiwaveguides, Phys. Rev. E 62, 2804-2809 (2000).
  * [49] D. Bortman-Arbiv, A. D. Wilson-Gordon, and H. Friedmann, Strong parametric amplification by spatial soliton-induced cloning of transverse beam profiles in an all-optical antiwaveguide, Phys. Rev. A 63, 031801(R) (2001).
  * [50] O. N. Verma and T. N. Dey, Steering, splitting, and cloning of an optical beam in a coherently driven Raman gain system, Phys. Rev. A 91, 013820 (2015).
  * [51] A. Kaplan, B. V. Gisin, and B. A. Malomed, Stable propagation and all-optical switching in planar waveguide-antiwaveguide periodic structures. J. Opt. Soc. Am. B 19, 522-527 (2002).
  * [52] H. Sakaguchi and B. A. Malomed, Dynamics of positive- and negative-mass solitons in optical lattices and inverted traps, J. Phys. B 37, 1443-1459 (2004).
  * [53] Y. Shin, G.-B. Jo, M. Saba, T. A. Pasquini, W. Ketterle, and D. E. Pritchard, Optical weak link between two spatially separated Bose-Einstein condensates, Phys. Rev. Lett. 95, 170402 (2005).
  * [54] S. J. Park, J. Noh, and J. Mun, Cold atomic beam from a two-dimensional magneto-optical trap with two-color pushing laser beams, Opt. Commun. 285, 3950-3954 (2012).
  * [55] H. Ma, A. K. Y. Jen, and L. R. Dalton, Polymer-based optical waveguides: Materials, processing, and devices, Advanced Materials 14, 1339-1365 (2002).
  * [56] N. Hacker and B. A. Malomed, Nonlinear dynamics of wave packets in tunnel-coupled harmonic-oscillator traps, Symmetry 13, 372 (2021).
  * [57] F. H. Stillinger and D. R. Herrick, Bound states in continuum, Phys. Rev. A 11, 446-454 (1975).
  * [58] A. Kodigala, T. Lepetit, Q. Gu, B. Bahari, Y. Fainman, and B. Kante, Lasing action from photonic bound states in continuum, Nature 541, 196-199 (2017).
  * [59] Y. V. Kartashov, V. V. Konotop, and L. Torner, Bound states in the continuum in spin-orbit-coupled atomic systems, Phys. Rev. A 96, 033619 (2017).
  * [60] Y. Guo, M. Xiao, S. Fan, Topologically protected complete polarization conversion, Phys. Rev. Lett. 119, 167401 (2017).
  * [61] B. Midya, and V. V. Konotop, Coherent-perfect-absorber and laser for bound states in a continuum, Opt. Lett. 43, 607-610 (2018).
  * [62] L. Xu, K. Z. Kamali, L. Huang, M. Rahmani, A. Smirnov, R. Camacho-Morales, Y. X. Ma, G. Q. Zhang, M. Woolley, D. Neshev, and A. E. Miroshnichenko, Dynamic nonlinear image tuning through magnetic dipole quasi-BIC ultrathin resonators, Advanced Science 6, 1802119 (2019).
  * [63] S. Romano, G. Zito, S. N. L. Yepez, S. Cabrini, E. Penzo, G. Coppola, I. Rendina, and V. Mocella, Tuning the exponential sensitivity of a bound-state-in-continuum optical sensor, Opt. Exp. 27, 18776-18786 (2019).
  * [64] A. R. Champneys, B. A. Malomed, J. Yang, and D. J. Kaup, “Embedded solitons”: solitary waves in resonance with the linear spectrum, Physica D 152-153, 340-354 (2001).
  * [65] B. A. Malomed, (INVITED) Vortex solitons: Old results and new perspectives, Physica D 399, 108-137 (2019).
  * [66] A. Avila, Global theory of one-frequency Schrödinger operators, Acta Mathematica 215, 1-54 (2015).
  * [67] B. A. Malomed, Variational methods in nonlinear fiber optics and related fields, Progr. Optics 43, 71-193 (2002).
  * [68] L. D. Landau and E. M. Lifshitz, Quantum Mechanics (Nauka Publishers, Moscow, 1974).
  * [69] R. N. Hill. Rates of convergence and error estimation formulas for the Rayleigh-Ritz variational method, J. Chem. Phys. 83, 1173-1196 (1985).
  * [70] N. G. Vakhitov and A. A. Kolokolov, Stationary solutions of the wave equation in a medium with nonlinearity saturation, Radiophys. Quant. Electron. 16, 783-789 (1973).
  * [71] L. Bergé, Wave collapse in physics: principles and applications to light and plasma waves, Phys. Rep. 303, 259-370 (1998).
  * [72] G. Fibich, The Nonlinear Schrödinger Equation: Singular Solutions and Optical Collapse (Springer: Heidelberg, 2015).
  * [73] H. Sakaguchi and B. A. Malomed, Solitons in combined linear and nonlinear lattice potentials, Phys. Rev. A 81, 013624 (2010).
  * [74] M. Quiroga-Teixeiro and H. Michinel, Stable azimuthal stationary state in quintic nonlinear optical media, J. Opt. Soc. Amer. B 14, 2004-2009 (1997).
  * [75] L. D. Carr and C. W. Clark, Vortices in attractive Bose–Einstein condensates in two dimensions, Phys. Rev. Lett. 97 010403 (2006).
  * [76] Y. Li, Z. Chen, Z. Luo, C. Huang, H. Tan, W. Pang, and B. A. Malomed, Two-dimensional vortex quantum droplets, Phys. Rev. A 98, 063602 (2018).
  * [77] M. R. Belić, A. Stepken, and F. Kaiser, Spatial screening solitons as particles, Phys. Rev. Lett. 84, 83-86 (2000).


# About every convex set in any
generic Riemannian manifold

Alexander Lytchak and Anton Petrunin

###### Abstract

We give a necessary condition on a geodesic in a Riemannian manifold that can
run in some convex hypersurface. As a corollary, we obtain peculiar properties
that hold true for _every_ convex set in any _generic_ Riemannian manifold
$(M,g)$. For example, if a convex set in $(M,g)$ is bounded by a smooth
hypersurface, then it is strictly convex.

##  1 Introduction

Let $\mathfrak{C}$ be the convex hull of a compact subset $Q$ in the Euclidean
space $\mathbb{R}^{m}$. By Carathéodory’s theorem [6], $\mathfrak{C}$ is the
set of all convex combinations of at most $(m+1)$-tuples of points on $Q$.
Thus, $\mathfrak{C}$ is a compact convex subset. Any point in
$\mathfrak{C}\setminus Q$ is an inner point of a line segment contained in
$\mathfrak{C}$; that is, the complement $\mathfrak{C}\setminus Q$ does not
contain extreme points of $\mathfrak{C}$.

The compactness of the convex hull and, therefore, the existence of a huge
variety of convex subsets with many non-extreme points on the boundary, admits
a straightforward generalization to the sphere and the Lobachevsky space;
moreover, it holds _locally_ in any two-dimensional Riemannian manifold.

Recall that a set $\mathfrak{C}$ in a Riemannian manifold $(M,g)$ is called
_convex_ if for any pair of points $x,y\in\mathfrak{C}$ any minimizing
geodesic $[x,y]$ lies in $\mathfrak{C}$. A point in $\mathfrak{C}$ is called
_extreme_ if it does not lie in an interior of a geodesic in $\mathfrak{C}$.
The _convex hull_ of a set $Q\subset M$ is the minimal convex subset of $M$
that contains $Q$.

It seems to be a folklore belief that a version of the statement above should
hold true in all Riemannian manifolds; see the discussion at mathoverflow
[15]. In the present note we prove that the somewhat counter-intuitive
opposite is the case for _generic_ Riemannian manifolds. It agrees with the
pattern: _a typical object in your favorite theory looks like nothing you have
ever seen before_.

Further Riemannian manifolds will be assumed to be connected and
$\mathcal{C}^{\infty}$-smooth. Given a positive integer $k$, we say that a
property $\mathcal{P}$ holds for _$\mathcal{C}^{k}$ -generic_ Riemannian
metric $g$ on a manifold $M$ if the property $\mathcal{P}$ holds for a dense
_G-delta set_ (that is, a countable intersection of open subsets) of metric
tensors in the $\mathcal{C}^{k}$-topology.

1.1. Main theorem. Let $\mathfrak{C}$ be an arbitrary convex subset of a
$\mathcal{C}^{2}$-generic Riemannian manifold $(M,g)$. Then the set of non-
extreme points in $\mathfrak{C}$ is the union of an open set and at most
countable family of geodesics in $(M,g)$.

In particular, if $\operatorname{\rm dim}M\geqslant 3$ and no connected
component of $\mathfrak{C}$ is a geodesic, then the set of extreme points of
$\mathfrak{C}$ is dense in $\partial\mathfrak{C}$.

Note that our definition of convexity does not require connectedness. However,
any convex subset $\mathfrak{C}$ is locally connected, and $\mathfrak{C}$ is
connected if the manifold $M$ is complete or $\mathfrak{C}$ is contained in
some compact convex subset of $M$.

If $\operatorname{\rm dim}M=2$, the statement is rather trivial and holds true
for _all_ Riemannian metrics not only the generic ones. As a consequence of
the main theorem for $\operatorname{\rm dim}M\geq 3$, we obtain the following:

1.2. Corollary. Let $Q$ be a closed subset of a $\mathcal{C}^{2}$-generic
Riemannian manifold $(M,g)$ of dimension at least $3$. If $Q$ does not lie in
a geodesic and the convex hull $\mathfrak{C}$ of $Q$ is closed and connected,
then $\partial\mathfrak{C}\subset Q$.

The corollary gives a positive resolution of a conjecture formulated by Marcel
Berger [2, Note 6.1.3.1], stating that convex hulls of $3$ points in most
Riemannian manifolds do not need to be compact. Probably the following more
exact form of Berger’s conjecture might be squeezed out from our key
lemma.111More open questions are listed in Appendix B.

1.3. Conjecture. Let $(M,g)$ be an arbitrary Riemannian manifold of dimension
at least 3. If the convex hull of any 3-point subset is compact, then $(M,g)$
has constant curvature.

The following corollary is essentially known, for $\operatorname{\rm dim}M=3$
its proof has been sketched by Robert Bryant [4] and, for $\operatorname{\rm
dim}M\geq 4$, it was proved by Thomas Murphy and Frederick Wilhelm [13].

1.4. Corollary. Let $(M,g)$ be a $\mathcal{C}^{2}$-generic Riemannian
manifold. Then any connected convex subset $\mathfrak{C}$ of $(M,g)$ is either
contained in a geodesic or _full-dimensional_ ; that is, the interior of
$\mathfrak{C}$ is nonempty.

The proofs are built on the following proposition. Its formulation uses the
notion of _rank_ of a point $p$ in a closed convex set $\mathfrak{C}$; we
define it as the dimension of the maximal linear subspace in the tangent cone
to $\mathfrak{C}$ at $p$.

1.5. Main proposition. Suppose $\mathfrak{C}$ is a closed convex set in a
$\mathcal{C}^{2}$-generic $m$-dimensional Riemannian manifold $(M,g)$. Then
all non-extreme points of $\mathfrak{C}$ have rank either $1$ or $m$.

In particular, if $\operatorname{\rm dim}M\geqslant 3$ and $\mathfrak{C}$ is
bounded by a $\mathcal{C}^{1}$-smooth hypersurface, then $\mathfrak{C}$ is
_strictly convex_ ; that is, all boundary points of $\mathfrak{C}$ are
extreme.

The proof relies on the key lemma stated in the following section; it
describes a necessary condition on a geodesic in a Riemannian manifold that
stays in convex subset $\mathfrak{C}$. If the geodesic lies in
$\partial\mathfrak{C}$ and contains a point of rank at least 2, then this
condition implies a non-trivial property of the curvature tensor. Then we show
that the curvature tensor of a generic Riemannian manifold does not meet this
property. The latter part is technical but straightforward; it is done by
applying the Thom transversality theorem; see Appendix A.

Acknowledgments. We thank Mohammad Ghomi and Frederick Wilhelm for their
interest in our result, the anonymous referee for helpful criticism. Alexander
Lytchak was partially supported by the DFG grant, no. 281071066, TRR 191.
Anton Petrunin was partially supported by the NSF grant, DMS-2005279.

##  2 Key lemma

Let $\mathfrak{C}$ be a closed convex set in an $m$-dimensional Riemannian
manifold $(M,g)$. Recall that $\mathrm{T}_{x}=\mathrm{T}_{x}M$ denotes the
_tangent space_ of $M$ at $x$. The _tangent cone_
$\mathrm{K}_{x}=\mathrm{K}_{x}\mathfrak{C}\subset\mathrm{T}_{x}$ at
$x\in\mathfrak{C}$ is defined as the closure of the set of all velocity
vectors of geodesics that start at $x$ and run in $\mathfrak{C}$.

Given $x\in\mathfrak{C}$, denote by
$\mathrm{L}_{x}=\mathrm{L}_{x}\mathfrak{C}$ the _maximal linear subspace_ of
$\mathrm{K}_{x}$. We define the _rank_ of $x$ in $\mathfrak{C}$ as the
dimension of $\mathrm{L}_{x}$.

Note that $\mathrm{K}_{x}$ is a convex cone in $\mathrm{T}_{x}$; in
particular, $\mathrm{L}_{x}=\mathrm{K}_{x}\cap(-\mathrm{K}_{x})$. Further
$\mathrm{K}_{x}$ coincides with $\mathrm{T}_{x}$ if and only if a neighborhood
of $x$ lies in the interior of $\mathfrak{C}$. In other words, $x$ has rank
$m$ if and only if $\mathfrak{C}$ contains a neighborhood of $x$.

Given a tangent vector $\text{\sc{x}}\in\mathrm{T}_{p}M$, consider the _Jacobi
operators_ of order $k$

$R^{k}_{\text{\sc{x}}}\colon\text{\sc{v}}\mapsto\nabla^{k-2}_{\text{\sc{x}}}\operatorname{\rm
Rm}(\text{\sc{v}},\text{\sc{x}})\text{\sc{x}},$

where $\operatorname{\rm Rm}$ denotes the curvature tensor of $g$; we set
$R^{1}=0$. Note that (i)
$R^{k}_{\text{\sc{x}}}\colon\mathrm{T}_{p}\to\nobreak\mathrm{T}_{p}$ is a
self-adjoint operator, (ii) $\text{\sc{x}}\mapsto R^{k}_{\text{\sc{x}}}$ is a
homogeneous polynomial of degree $k$, and (iii)

$R^{k}_{\text{\sc{x}}}{\hskip 0.5pt\cdot\hskip 0.5pt}\text{\sc{x}}=0$ ➊

for any $k$ and $\text{\sc{x}}\in\mathrm{T}_{p}$. The Jacobi equation along a
geodesic $\gamma$ takes the form

$\nabla^{2}_{\gamma^{\prime}}{\hskip 0.5pt\cdot\hskip
0.5pt}\text{\sc{i}}+R^{2}_{\gamma^{\prime}}{\hskip 0.5pt\cdot\hskip
0.5pt}\text{\sc{i}}=0.$

2.1. Key lemma. Let $(M,g)$ be a Riemannian manifold and
$\gamma\colon(a_{0},b_{0})\to M$ be a geodesic that runs in a closed convex
set $\mathfrak{C}\subset(M,g)$. Then the tangent cones of $\mathfrak{C}$ are
parallel along $\gamma$; that is, the parallel translation along $\gamma$
defines a bijection between the tangent cones
$\mathrm{K}_{\gamma(a)}\mathfrak{C}$ and $\mathrm{K}_{\gamma(b)}\mathfrak{C}$
for any $a,b\in(a_{0},b_{0})$.

Moreover, for any $a\in(a_{0},b_{0})$ the following conditions hold:

  * (i)

For any $\text{\sc{v}}\in\mathrm{K}_{\gamma(a)}\mathfrak{C}$ we have

$R^{2}_{\gamma^{\prime}(a)}{\hskip 0.5pt\cdot\hskip
0.5pt}\text{\sc{v}}\in\mathrm{K}_{\text{\sc{v}}}[\mathrm{K}_{\gamma(a)}\mathfrak{C}].$

  * (ii)

$\mathrm{L}_{\gamma(a)}\mathfrak{C}$ is an invariant subspace of
$R^{2}_{\gamma^{\prime}(a)}\colon\mathrm{T}_{\gamma(a)}\to\mathrm{T}_{\gamma(a)}$.

The proof uses the fact that the parallel translation can be defined via
geodesics. In a similar way, this observation was used in [1, Section 13] and
[14]. In fact the main part of the key lemma follows from [14].

Proof of 2. Since all statements are local, we may replace $(M,g)$ by its
small open convex subset. By doing so we may assume that any pair of points of
$(M,g)$ is connected by a unique geodesic and there are no conjugate points.
In particular, for any subinterval $[a,b]\subset(a_{0},b_{0})$ and any tangent
vectors $\text{\sc{v}}\in\mathrm{T}_{\gamma(a)}$ and
$\text{\sc{w}}\in\mathrm{T}_{\gamma(b)}$ there exists a unique Jacobi field i
along $\gamma$ such that $\text{\sc{i}}(a)=\text{\sc{v}}$ and
$\text{\sc{i}}(b)=\text{\sc{w}}$.

Since Jacobi fields are variational fields of geodesic variations, the
convexity of $\mathfrak{C}$ implies the following:

2.2. Observation. Suppose i is a Jacobi field along $\gamma$ and
$a_{0}<a<t<b<b_{0}$. If
$\text{\sc{i}}(a)\in\mathrm{K}_{\gamma(a)}\mathfrak{C}$ and
$\text{\sc{i}}(b)\in\mathrm{K}_{\gamma(b)}\mathfrak{C}$, then
$\text{\sc{i}}(t)\in\mathrm{K}_{\gamma(t)}\mathfrak{C}$.

Choose a subinterval $[a,b]\subset(a_{0},b_{0})$. Given a large positive
integer $k$, consider the arithmetic progression $t_{0},\dots,t_{k+1}$ such
that $t_{0}=a$ and $t_{k}=b$.

Choose a tangent vector $\text{\sc{v}}_{0}\in\mathrm{T}_{\gamma(a)}$. Consider
the sequence of vectors
$\text{\sc{v}}_{i}\in\nobreak\mathrm{T}_{\gamma(t_{i})}$ defined recursively
by $\text{\sc{v}}_{i+1}=2{\hskip 0.5pt\cdot\hskip
0.5pt}\text{\sc{i}}_{i}(t_{i+1})$, where $t\mapsto\text{\sc{i}}_{i}(t)$
denotes the Jacobi field along $\gamma$ such that
$\text{\sc{i}}_{i}(t_{i})=\text{\sc{v}}_{i}$ and
$\text{\sc{i}}_{i}(t_{i+2})=0$.

Define $\iota_{k}\colon\mathrm{T}_{\gamma(a)}\to\mathrm{T}_{\gamma(b)}$ by
setting $\iota_{k}(\text{\sc{v}}_{0})\mathrel{{:}{=}}\text{\sc{v}}_{k}$.
According to the observation, if
$\text{\sc{v}}_{0}\in\mathrm{K}_{\gamma(a)}\mathfrak{C}$, then
$\iota_{k}(\text{\sc{v}}_{0})\in\mathrm{K}_{\gamma(b)}\mathfrak{C}$. As
observed in [1] and [14], $\iota_{k}(\text{\sc{v}}_{0})$ converges to the
parallel translation of $\text{\sc{v}}_{0}$ along $\gamma$ as $k\to\infty$.
Since $\mathrm{K}_{\gamma(b)}\mathfrak{C}$ is closed, the parallel translation
along $\gamma$ maps $\mathrm{K}_{\gamma(a)}\mathfrak{C}$ in
$\mathrm{K}_{\gamma(b)}\mathfrak{C}$. Switching the direction of $\gamma$, we
get the opposite inclusion. That is, the tangent cones
$\mathrm{K}_{\gamma(t)}\mathfrak{C}$ are parallel along $\gamma$ — the main
part is proved.

Let us use the parallel translation along $\gamma$ to identify the tangent
spaces at points on $\gamma$. This way we identify the tangent cones
$\mathrm{K}_{\gamma(t)}\mathfrak{C}$ for all $t$; denote the obtained cone by
$\mathrm{K}$.

For $\text{\sc{v}}\in\mathrm{K}$ and small $\varepsilon>0$, consider the
unique Jacobi field $\text{\sc{i}}_{\varepsilon}$ along $\gamma$ with
$\text{\sc{i}}_{\varepsilon}(a+\varepsilon)=\nobreak\text{\sc{i}}_{\varepsilon}(a-\varepsilon)=\text{\sc{v}}$.
Due to the Jacobi equation,

$\text{\sc{i}}_{\varepsilon}(a)=\text{\sc{v}}+\tfrac{1}{2}{\hskip
0.5pt\cdot\hskip 0.5pt}\varepsilon^{2}{\hskip 0.5pt\cdot\hskip
0.5pt}R^{2}_{\gamma^{\prime}(a)}{\hskip 0.5pt\cdot\hskip
0.5pt}\text{\sc{v}}+o(\varepsilon^{2}).$

According to the observation, $\text{\sc{i}}_{\varepsilon}(a)\in\mathrm{K}$
for any $\varepsilon>0$. Since $\mathrm{K}$ is a closed convex cone, we get
$R^{2}_{\gamma^{\prime}}{\hskip 0.5pt\cdot\hskip
0.5pt}\text{\sc{v}}\in\nobreak\mathrm{K}_{\text{\sc{v}}}\mathrm{K}$ — 2 is
proved.

Finally, $\text{\sc{v}}\in\mathrm{L}_{\gamma(a)}\mathfrak{C}$ $\iff$
$\text{\sc{v}},-\text{\sc{v}}\in\mathrm{K}$ $\iff$
$\mathrm{K}_{\text{\sc{v}}}\mathrm{K}=\mathrm{K}$. Therefore, if
$\text{\sc{v}}\in\mathrm{L}_{\gamma(a)}\mathfrak{C}$, then $\pm
R^{2}_{\gamma^{\prime}(a)}{\hskip 0.5pt\cdot\hskip
0.5pt}\text{\sc{v}}\in\mathrm{K}$, and hence
$R^{2}_{\gamma^{\prime}(a)}{\hskip 0.5pt\cdot\hskip
0.5pt}\text{\sc{v}}\in\mathrm{L}_{\gamma(a)}\mathfrak{C}$. That is,
$\mathrm{L}_{\gamma(a)}\mathfrak{C}$ is an invariant subspace of
$R^{2}_{\gamma^{\prime}(a)}$ — 2 is proved. ∎

##  3 Main proposition

In this section we will prove the main proposition 1 modulo one claim; let us
introduce notations to state it.

Let $M$ be a smooth $m$-dimensional manifold with a Riemannian metric $g$.
Suppose x is a nonzero tangent vector at a point $p\in M$. Recall that
$R^{k}_{\text{\sc{x}}}\colon\mathrm{T}_{p}\to\nobreak\mathrm{T}_{p}$ denotes
the Jacobi operators of $g$ of order $k$ for a tangent vector
$\text{\sc{x}}\in\mathrm{T}_{p}$. An invariant subspace
$V\subset\mathrm{T}_{p}$ of $R^{k}_{\text{\sc{x}}}$ will be called
_exceptional_ if $V\ni\text{\sc{x}}$ and $1<\operatorname{\rm dim}V<m$.
(Recall that $R^{k}_{\text{\sc{x}}}{\hskip 0.5pt\cdot\hskip
0.5pt}\text{\sc{x}}=0$ for any $k$ and $\text{\sc{x}}\in\mathrm{T}_{p}$.
Therefore, the subspace spanned by x is always an invariant subspace of
$R^{k}_{\text{\sc{x}}}$ for any $k$.)

We will say that a metric $g$ on a manifold $M$ is $k$-exceptional if there
exists a point $p\in M$ and a non-zero vector $\text{\sc{x}}\in T_{p}M$, such
that the operators $R^{2}_{\text{\sc{x}}},\dots,R^{k}_{\text{\sc{x}}}$ have a
common exceptional invariant subspace.

3.1. Claim. For any smooth manifold $M$, there exists an integer $k$ such that
the $\mathcal{C}^{k}$-generic Riemannian metric is not $k$-exceptional.

For $k=2$ (and, probably, also for $k=3$) every Riemannian metric is
$k$-exceptional. However, for larger $k$, the $k$-exceptionality defines more
and more restrictions on the curvature tensor. Therefore, it is not surprising
that _most_ Riemannian metrics are not $k$-exceptional, for sufficiently large
$k$. A formal proof of this claim is built on Thom transversality theorem; it
will be derived in Appendix A.

Proof of 1 modulo 3. Suppose that $p$ is a nonextreme point of $\mathfrak{C}$;
that is, $p$ lies on a nonconstant geodesic
$\gamma\colon(a,b)\to\mathfrak{C}$.

According to the key lemma (2), the family of _maximal linear subspaces_
$\mathrm{L}_{\gamma(t)}\mathfrak{C}$ of $\mathrm{K}_{\gamma(t)}\mathfrak{C}$
is parallel along $\gamma$ and invariant for $R^{2}_{\gamma(t)}$. Note that
$\mathrm{L}_{p}$ is exceptional if and only if the rank of $p$ is neither $1$,
nor $m$.

Further, if a nontrivial geodesic $\gamma$ admits a parallel family
$\mathrm{L}_{t}\subset\mathrm{T}_{\gamma(t)}$ of exceptional invariant
subspaces for all $R^{2}_{\gamma(t)}$, then we say that $\gamma$ is
_exceptional_. So, it is sufficient to show that $\mathcal{C}^{2}$-generic
Riemannian manifolds $(M,g)$ do not have exceptional geodesics.

Choose a compact subset $K\subset M$ and $\varepsilon>0$. Consider the set
$Z(K,\varepsilon)$ of all Riemannian metrics $g$ on $M$ such that there exists
an exceptional geodesic $\gamma$ in $(M,g)$ that starts at a point in $K$ and
has length $\varepsilon$. Observe that the geodesics and the curvature tensor
depend continuously on the Riemannian metric in $\mathcal{C}^{2}$-topology.
Therefore, the set $Z(K,\varepsilon)$ is closed with respect to the
$\mathcal{C}^{2}$-topology.

Suppose $\gamma$ is an exceptional geodesic that passes thru $p$ in the
direction x. By taking covariant derivatives along $\gamma$, we get that the
Jacobi operators $R^{k}_{\text{\sc{x}}}$ have a common exceptional invariant
subspace $\mathrm{L}_{p}$, for all $k\geq 2$. In other words, for any integer
$k\geq 2$ we have

$Z^{k}(K)\supset Z(K,\varepsilon),$ ➋

where $Z^{k}(K)$ denotes the set of all smooth Riemannian metrics on $M$ such
that for some $p\in K$ and $\text{\sc{x}}\in\mathrm{T}_{p}\setminus\\{0\\}$
the operators $R^{2}_{\text{\sc{x}}},\dots,R^{k}_{\text{\sc{x}}}$ have a
common exceptional invariant subspace.

By the very definition of $Z^{k}(K)$, it is closed with respect to
$\mathcal{C}^{k}$-topology on the space of all Riemannian metrics on $M$. By
Claim 3, we can choose $k$ so that $Z^{k}(K)$ is $\mathcal{C}^{k}$-meager for
any $K$; that is, its complement is a dense G-delta set in the space of all
Riemannian metrics on $M$ with $\mathcal{C}^{k}$-topology.

Since $Z(K,\varepsilon)$ is closed with respect to the
$\mathcal{C}^{2}$-topology, ➋ ‣ 3 implies that $Z(K,\varepsilon)$ is
$\mathcal{C}^{2}$-meager in the space of all Riemannian metrics on $M$.

Choose a nested sequence of compact sets $K_{1}\subset\nobreak
K_{2}\subset\dots$ that cover $M$ and set $\varepsilon_{n}=\tfrac{1}{n}$. Set

$Z(M)=\bigcup_{n}Z(K_{n},\varepsilon_{n});$

since $Z(K_{n},\varepsilon_{n})$ is $\mathcal{C}^{2}$-meager for every $n$, so
is $Z(M)$.

It remains to note that $g\in Z(M)$ if and only if $(M,g)$ has an exceptional
geodesic. ∎

##  4 Main theorem

The following proposition is a special case of a result of Nan Li and Aaron
Naber [10, Theorem 1.6]. It also can be deduced from the result of Luděk
Zajíček [16].

4.1. Proposition. Let $\mathfrak{C}$ be a closed convex set in a Riemannian
manifold $(M,g)$. Then the set of points in $\mathfrak{C}$ with rank at most
$k$ is countably _$k$ -rectifiable_; that is, this set can be covered by
images of a countable set of Lipschitz maps $\mathbb{R}^{k}\to(M,g)$. In
particular, this set contains at most countably many disjoint Borel sets with
positive $k$-dimensional Hausdorff measure.

Proof of 1 and 1. We may assume that $\mathfrak{C}$ is connected. Assume, in
addition, that $\mathfrak{C}$ is closed.

According to [5, Theorem 1.6], a connected closed convex set $\mathfrak{C}$ in
a Riemannian manifold $(M,g)$ is homeomorphic to a manifold with boundary, say
$\mathfrak{B}$. Moreover, the complement $\mathfrak{C}\backslash\mathfrak{B}$
is a totally geodesic submanifold of $(M,g)$; denote its dimension by $d$.

The tangent cone $\mathrm{K}_{p}\mathfrak{C}$ at any
$p\in\mathfrak{C}\setminus\mathfrak{B}$ is a $d$-dimensional linear space. By
the main proposition (1), $d=0,1$, or $m$. If $d=0$, then $\mathfrak{C}$ is a
single point. If $d=1$, then $\mathfrak{C}\setminus\mathfrak{B}$ is a geodesic
in $(M,g)$; hence $\mathfrak{C}$ is contained in a geodesic as well. If $d=m$,
then by the invariance of domain we have $\mathfrak{C}\setminus\mathfrak{B}$
is open in $M$; that is, $\mathfrak{C}$ is full-dimensional — 1 is proved.

By the main proposition (1) any non-extreme point $x\in\partial\mathfrak{C}$
has rank 1. Thus, there is a unique line in $\mathrm{K}_{x}\mathfrak{C}$ and
it is the tangent line of a geodesic $\gamma\subset\mathfrak{C}$ that has $p$
as an inner point.

Let us extend $\gamma$ to a maximal open interval so that $\gamma$ stays in
$\mathfrak{C}$; note that $p$ uniquely defines $\gamma$. By the main statement
of the key lemma, all points on $\gamma$ lie on $\partial\mathfrak{C}$. By
definition, all such geodesics consist of non-extreme points.

It gives a subdivision of non-extreme points of $\partial\mathfrak{C}$ into
geodesics with positive lengths. By 4, there are only countably many such
geodesics.

If $\mathfrak{C}$ is not closed, consider its closure $\bar{\mathfrak{C}}$;
denote by $\bar{\mathfrak{B}}$ its boundary. Note that $\bar{\mathfrak{C}}$ is
locally convex and the above arguments apply to closed locally convex subsets
without changes. Observe that any nonextreme point of $\mathfrak{C}$ is a
nonextreme point of $\bar{\mathfrak{C}}$ and
$\bar{\mathfrak{C}}\backslash\bar{\mathfrak{B}}\subset\mathfrak{C}$ [5, Lemma
1.5]. Hence, the statement follows. ∎

Few words before the proof of 1. Let $Q$ be a subset of a Riemannian manifold
$M$. Set $Q=Q_{0}$ and let inductively $Q_{i+1}$ to be the union of all
minimizing geodesics between pairs of points of $Q_{i}$. By definition, the
increasing countable union $\mathfrak{C}=\bigcup_{i}Q_{i}$ is the convex hull
of $Q$. By this description, any point in $\mathfrak{C}\setminus Q$ is a non-
extreme point of the convex set $\mathfrak{C}$.

Note, that if $M$ is complete and $Q$ is compact, then each $Q_{i}$ is
compact. In the Euclidean space $M=\mathbb{R}^{m}$ (as well as in the round
sphere or in the Lobachevsky space) Carathéodory’s theorem [6] implies
$\mathfrak{C}=Q_{m}$. As a consequence of Corollary 1, we will know that in a
generic Riemannian manifold the convex hull $\mathfrak{C}$ of $Q$ is strictly
larger than $Q_{i}$, for all $i$.

Proof of 1. Without loss of generality we can assume that $\mathfrak{C}$ is a
proper subset of $M$; in particular $\partial\mathfrak{C}\neq\varnothing$.
Since $Q$ is not contained in a geodesic, by the main theorem, $\mathfrak{C}$
has a non-trivial interior. By the construction of $\mathfrak{C}$ above, any
point $x\in\mathfrak{C}\setminus Q$ is not an extreme point of $\mathfrak{C}$.

Assume $\partial\mathfrak{C}\not\subset Q$. By the main theorem, the
topological manifold $\partial\mathfrak{C}$ is the union of the closed subset
$Q\cap\partial\mathfrak{C}$ and a countable union of geodesics. But
$\partial\mathfrak{C}\setminus Q$ is an $(m-1)$-dimensional topological
manifold. By dimensional reasons, it is not a union of countably many
rectifiable curves — a contradiction. ∎

##  Appendix A Normalization of metrics

This appendix is devoted to the algebra of curvature tensor and its covariant
derivatives that leads to a proof of Claim 3.

Choose an $m$-dimensional Euclidean space $\mathrm{T}$. Denote by
$\mathcal{S}$ the space of self-adjoint operators on $\mathrm{T}$.

Consider the space $\mathcal{G}$ of germs of Riemannian metrics on
$\mathrm{T}$ at $0$ that coincide with the canonical metric at $0$. Any germ
in $\mathcal{G}$ can be described by $\langle G{\hskip 0.5pt\cdot\hskip
0.5pt}\text{\sc{v}},\text{\sc{w}}\rangle$, where $\text{\sc{x}}\mapsto
G_{\text{\sc{x}}}$ is a smooth function $\mathrm{T}\to\mathcal{S}$ such that
$G_{0}=\mathrm{id}$.

The $k$-jet of $G$ is defined by the Taylor polynomial of $G$ of degree $k$

$G_{\text{\sc{x}}}=\mathrm{id}+G^{1}_{\text{\sc{x}}}+\dots+G^{k}_{\text{\sc{x}}}+o(|\text{\sc{x}}|^{k}),$
➌

where $\text{\sc{x}}\mapsto G^{i}_{\text{\sc{x}}}$ is a homogeneous polynomial
$\mathrm{T}\to\mathcal{S}$ of degree $i$.

Note that every array of homogeneous polynomials
$G^{1},\dots,G^{k}\colon\mathrm{T}\to\mathcal{S}$ such that $\deg
G^{i}=\nobreak i$ appears in ➌ ‣ A for the germ in $\mathcal{G}$ defined by

$G_{\text{\sc{x}}}=\mathrm{id}+G^{1}_{\text{\sc{x}}}+\dots+G^{k}_{\text{\sc{x}}}.$
➍

The space of $k$-jets of germs in $\mathcal{G}$ will be denoted by
$\mathcal{G}^{k}$.

A germ in $\mathcal{G}$ will be called _normal_ if the standard coordinates on
$\mathrm{T}$ coincide with normal coordinates of the germ in a neighborhood of
the origin. By the Gauss lemma, a germ defined by $G$ is normal if and only if

$G_{\text{\sc{x}}}{\hskip 0.5pt\cdot\hskip 0.5pt}\text{\sc{x}}=\text{\sc{x}}$
➎

for all small $\text{\sc{x}}\in\mathrm{T}$. The subspace of normal germs in
$\mathcal{G}$ and their $k$-jets will be denoted by $\mathcal{N}$ and
$\mathcal{N}^{k}$, respectively.

Suppose that $G$ describes a germ in $\mathcal{N}$ and $G^{1},\dots,G^{k}$ be
as in ➌ ‣ A. By ➎ ‣ A

$G^{i}_{\text{\sc{x}}}{\hskip 0.5pt\cdot\hskip 0.5pt}\text{\sc{x}}=0$ ➏

for any $i$. Moreover, for an array of polynomials
$G^{1},\dots,G^{k}\colon\mathrm{T}\to\mathcal{S}$ such that $G^{i}$ is
homogeneous of degree $i$ and ➏ ‣ A holds for each $i$, the sum ➍ ‣ A defines
a normal $k$-jet; that is, ➏ ‣ A is the only condition on the normality of
jets.

Christoffel symbols vanish in normal coordinates, thus, $G^{1}=0$, for
$G\in\mathcal{N}$.

Choose $\text{\sc{x}}\in\mathrm{T}$; denote by $\mathcal{S}_{\text{\sc{x}}}$
the subspace of the operators $S\in\mathcal{S}$ such that $S{\hskip
0.5pt\cdot\hskip 0.5pt}\text{\sc{x}}=0$. By ➏ ‣ A,
$G^{i}_{\text{\sc{x}}}\in\mathcal{S}_{\text{\sc{x}}}$ for any germ in
$\mathcal{N}$. The following claim says that $G^{i}_{\text{\sc{x}}}$ can be
chosen arbitrarily in $\mathcal{S}_{\text{\sc{x}}}$ for $i\geqslant 2$ and
$\text{\sc{x}}\neq 0$.

A.1. Claim. Given $\text{\sc{x}}\neq 0$ in $\mathrm{T}$ and a sequence of
operators $A_{2},\dots A_{k}\in\mathcal{S}_{\text{\sc{x}}}$ there is a germ
$(G^{1},\dots,G^{k})$ in $\mathcal{N}^{k}$ such that
$G^{i}_{\text{\sc{x}}}=A_{i}$ for any $i\geqslant 2$.

Proof. For any unit vector y in $\mathrm{T}$ perpendicular to x, consider the
orthogonal projection $P^{\text{\sc{y}}}$ in $T$ onto the line generated by y.
Diagonalizing operators in $\mathcal{S}_{\text{\sc{x}}}$, we see that such
projections $P^{\text{\sc{y}}}$ generate $\mathcal{S}_{\text{\sc{x}}}$ as a
vector space.

$\mathcal{N}^{k}$ is described by ➏ ‣ A, hence it defines a linear subspace of
$\mathcal{S}_{\text{\sc{x}}}^{k}$. Thus, it suffices to verify the following:
For any $2\leq j\leq k$ and any unit vector y in $\mathrm{T}$ perpendicular to
x, there exists a germ $(G^{1},\dots,G^{k})$ in $\mathcal{N}^{k}$ such that
$G^{j}_{\text{\sc{x}}}=P^{\text{\sc{y}}}$ and $G^{i}=0$ for $i\neq j$.

Such a normal germ can be constructed as a product of a surface of revolution
(corresponding to the $(\text{\sc{x}},\text{\sc{y}})$-plane) and a Euclidean
space. ∎

Suppose that a germ in $\mathcal{G}$ is described by
$G\colon\mathrm{T}\to\mathcal{S}$. Consider its array of Jacobi operators
$(R^{1},\dots,R^{k})$ at the origin; recall that $R^{1}=0$. The identities in
Section 2 imply that any such array $(R^{1},\dots,R^{k})$ belongs to the space
$\mathcal{R}^{k}$ defined by the following conditions: (i) each
$R^{i}\colon\mathrm{T}\to\mathcal{S}$ is a homogeneous polynomial, (ii) $\deg
R^{i}=i$, and (iii) $R^{i}_{\text{\sc{x}}}{\hskip 0.5pt\cdot\hskip
0.5pt}\text{\sc{x}}=0$ for any $i$ and $\text{\sc{x}}\in\mathrm{T}$. Note that
these conditions are exactly the same as for $G^{i}$ in $\mathcal{N}^{k}$.
Therefore, $\mathcal{R}^{k}$ can be identified with $\mathcal{N}^{k}$, but we
will keep separate notations for them.

The expression of the curvature tensor in terms of the metric and its
derivatives defines a natural algebraic map

$\rho_{k}\colon\mathcal{G}^{k}\to\mathcal{R}^{k}.$

For any $k\geqslant 2$, any $G\in\mathcal{N}^{k}$ and
$(R^{1},\dots,R^{k})=\rho_{k}(G)$, we have

$G^{k}=a_{k}{\hskip 0.5pt\cdot\hskip 0.5pt}R^{k}+A^{k},$ ➐

where $a_{k}$ is a nonzero constant and $A^{k}$ is a field of self-adjoint
operators that can be written as a polynomial of $R^{2},\dots R^{k-2}$. This
statement follows easily from the formula derived by Oldřich Kowalski and
Martin Belger [9, Proposition 2.2]. (In fact, $a_{k}=-2{\hskip
0.5pt\cdot\hskip 0.5pt}\tfrac{k-1}{k+1}$, but we will not need it.)

Hence, the map $\rho_{k}$ admits an algebraic inverse map::

A.2. Claim. The restriction $\rho_{k}|_{\mathcal{N}^{k}}$ is an algebraic
diffeomorphism $\mathcal{N}^{k}\leftrightarrow\mathcal{R}^{k}$.

Applying A, we get the following:

A.3. Corollary. Given $\text{\sc{x}}\neq 0$ in $\mathrm{T}$ and a sequence of
operators $A_{2},\dots A_{k}\in\mathcal{S}_{\text{\sc{x}}}$ there is a germ
$\mathcal{N}^{k}$ with Jacobi operators $R^{i}_{\text{\sc{x}}}=A_{i}$ for any
$i\geqslant 2$.

A.4. Proposition. The map $\rho_{k}:\mathcal{G}^{k}\to\mathcal{R}^{k}$ is an
algebraic submersion.

$\mathcal{G}$$\mathcal{N}$$\mathcal{G}^{k}$$\mathcal{N}^{k}$$\mathcal{R}^{k}$$\nu$$\iota$$\rho_{k}$$\nu_{k}$$\iota_{k}$$\rho_{k}$

Proof. Evidently, $\rho_{k}$ is algebraic.

Any germ in $\mathcal{G}$ becomes normal if the space $\mathrm{T}$ is
reparametrized by its exponential map. This defines the normalization map
$\mathcal{G}\xrightarrow{\nu}\mathcal{N}$. Since the curvature tensors does
not change under this (or any other) coordinate change, $\nu$ commutes with
$\rho_{k}:\mathcal{G},\mathcal{N}\to\mathcal{R}^{k}$.

By A, $\mathcal{N}^{k}\ext@arrow
0099\arrowfill@\leftarrow\relbar\rightarrow{}{\rho_{k}}\mathcal{R}^{k}$ is a
diffeomorphism. The maps
$\mathcal{G}^{k}\xrightarrow{\rho_{k}}\mathcal{R}^{k}\ext@arrow
0099\arrowfill@\leftarrow\relbar\rightarrow{}{\rho_{k}}\nobreak\mathcal{N}^{k}$
together with the forgetful maps $\mathcal{G}\to\mathcal{G}^{k}$ and
$\mathcal{N}\to\mathcal{N}^{k}$ commute. In particular, we get a map
$\mathcal{G}^{k}\xrightarrow{\nu_{k}}\mathcal{N}^{k}$ that commutes with the
forgetful maps and the normalization $\nu$. Hence, $\nu_{k}$ and $\rho_{k}$
commute.

Note that the inclusion
$\mathcal{N}\stackrel{{\scriptstyle\iota}}{{\hookrightarrow}}\mathcal{G}$ is a
right inverse of $\nu$. Moreover, by changing the parametrization on
$\mathrm{T}$ to normal coordinates of a given germ $G$ in $\mathcal{G}$, we
may assume that $G$ lies in the image of $\iota$. Therefore, there is an
inclusion
$\mathcal{N}^{k}\stackrel{{\scriptstyle\iota_{k}}}{{\hookrightarrow}}\mathcal{G}^{k}$
that is a right inverse of $\nu$ such that its image contains any given jet in
$\mathcal{G}^{k}$. It follows that
$\mathcal{G}^{k}\xrightarrow{\nu^{k}}\mathcal{N}^{k}$ is a submersion, hence
the result. ∎

Proof of 3. Denote by $\tilde{\mathcal{G}}^{k}$ the space of all $k$-jets of
Riemannian metrics at a given point $p$. Denote by $\tilde{\Sigma}^{k}$ all
jets in $\tilde{\mathcal{G}}^{k}$ such that for some nonzero tangent vector
$\text{\sc{x}}\in\mathrm{T}_{p}$ the Jacobi operators
$R^{2}_{\text{\sc{x}}},\dots,R^{k}_{\text{\sc{x}}}$ have a common exceptional
invariant subspace.

By Tarski–Seidenberg theorem, $\tilde{\Sigma}^{k}$ is semialgebraic; in
particular it is stratified. Due to the Thom transversality theorem [7,
2.3.2], it is sufficient to show that for any point $p$ the codimension of
$\tilde{\Sigma}^{k}$ in $\tilde{\mathcal{G}}^{k}$ is larger than
$m=\operatorname{\rm dim}M$.

This is a pointwise statement; therefore we may fix $p$ from now on.

A jet in $\tilde{\mathcal{G}}^{0}$ is described by the metric tensor $g_{0}$
on $\mathrm{T}=\mathrm{T}_{p}M$. Note that the forgetful map
$\tilde{\mathcal{G}}^{k}\to\tilde{\mathcal{G}}^{0}$ is a fiber bundle.
Furthermore, the restriction of this forgetful map to $\tilde{\Sigma}$ is also
a fiber bundle. Thus, it suffices to prove that the intersection $\Sigma^{k}$
of $\tilde{\Sigma}^{k}$ with a fiber of
$\tilde{\mathcal{G}}^{k}\to\tilde{\mathcal{G}}^{0}$ has codimension at least
$m$. Note that the fiber of the forgetful map over the Euclidean structure on
$\mathrm{T}$ given by $g_{0}$ is exactly the space $\mathcal{G}^{k}$
investigated above.

In other words, if we choose a chart $\mathrm{T}\to M$, then $g_{0}$ defines
an inclusion $\mathcal{G}^{k}\hookrightarrow\tilde{\mathcal{G}}^{k}$, and it
is sufficient to show that

$\operatorname{\rm codim}\Sigma^{k}\to\infty\quad\text{as}\quad k\to\infty;$ ➑

here we consider $\Sigma^{k}=\nobreak\tilde{\Sigma}^{k}\cap\mathcal{G}^{k}$ as
a subset of $\mathcal{G}^{k}$.

Denote by $\mathcal{L}$ the semialgebraic set of all pairs $(L,\text{\sc{x}})$
where $L$ is a subspace of $\mathrm{T}$ such that $1<\operatorname{\rm
dim}L<m$, and $\text{\sc{x}}\in L\backslash\\{0\\}$. Given
$(L,\text{\sc{x}})\in\mathcal{L}$, denote by $\Sigma^{k}(L,\text{\sc{x}})$ the
subset of jets in $\mathcal{G}^{k}$ such that $L$ is an invariant subspace of
all Jacobi operators $R^{i}_{\text{\sc{x}}}$ for any $i\leqslant k$.

Choose $(L,\text{\sc{x}})\in\mathcal{L}$. We claim that A implies

$\operatorname{\rm
codim}\Sigma^{k}(L,\text{\sc{x}})\to\infty\quad\text{as}\quad k\to\infty.$ ➒

Indeed, a normal germ $(G^{1},\dots G^{k})$ belongs to
$\Sigma^{k}(L,\text{\sc{x}})$ if and only if all the Jacobi operators
$R^{2}_{\text{\sc{x}}},\dots,R^{k}_{\text{\sc{x}}}\in\mathcal{S}_{\text{\sc{x}}}$
have invariant subspace $L$. The codimension of the space of $(k-1)$-tuples in
$S_{x}$ that all have $L$ as invariant subspace grows with $k$. By A and A the
composition
$\mathcal{G}^{k}\to\nobreak\mathcal{R}^{k}\to\nobreak\mathcal{S}^{k-1}_{\text{\sc{x}}}$
that sends a germ to the array of its Jacobi operators
$(R^{2}_{\text{\sc{x}}},\dots,R^{k}_{\text{\sc{x}}})$ is a submersion.
Therefore, ➒ ‣ A follows.

Observe that

$\operatorname{\rm codim}\Sigma^{k}\geqslant\operatorname{\rm
codim}\Sigma^{k}(L,\text{\sc{x}})-\operatorname{\rm dim}\mathcal{L}.$

Therefore, ➑ ‣ A follows. ∎

##  Appendix B Final remarks

We expect that the following question admits an affirmative answer.

B.1. Question. Is it true that any Riemannian manifold $(M,g)$ contains a
nontrivial geodesic that runs in the boundary of some convex subset?

There is a good chance that the argument of Albert Borbély [3, Lemma 2.1] can
be modified to answer the following question. Assuming that the answer is
affirmative, it can be combined with the main proposition to derive further
restrictions on convex hulls in generic Riemannian manifolds.

B.2. Question. Let $\mathfrak{C}$ be the closure of a convex hull of a set $Q$
in a Riemannian manifold. Then all points of $\mathfrak{C}$ with rank at most
$1$ lie on minimizing geodesics between points in $Q$.

The presented argument, when properly extended to infinite-dimensional
manifolds, might lead to a negative answer to the following question of
Mikhael Gromov [8, 6.B${}_{1}\text{(f)}$].

B.3. Question. Let $X$ be a complete $\operatorname{\rm CAT}(0)$ space (not
necessarily locally compact). Is it true that any compact set of $X$ lies in a
compact convex subset?

A surprising behavior of convex sets in complete (but not locally compact)
$\operatorname{\rm CAT}(0)$ spaces is discussed by Nicolas Monod [12].

Finally let us mention that there is a result of Anatoliy Milka [11, § 4]
about rank of points on geodesics in the intrinsic metric of convex surfaces;
it is closely related to our main proposition but goes in the opposite
direction.

## References

  * [1] V.. Berestovskij and I.. Nikolaev  “Multidimensional generalized Riemannian spaces”  In _Geometry, IV_ 70, Encyclopaedia Math. Sci.  Springer, Berlin, 1993, pp. 165–243245–250  DOI: 10.1007/978-3-662-02897-1_2
  * [2] M. Berger  “A panoramic view of Riemannian geometry”, 2003  DOI: 10.1007/978-3-642-18245-7
  * [3] A. Borbély  “On the smoothness of the convex hull in negatively curved manifolds”  In _J. Geom._ 54.1-2, 1995, pp. 3–14  DOI: 10.1007/BF01222848
  * [4] R. Bryant  “Existence of totally geodesic hypersurfaces”, MathOverflow  URL: https://mathoverflow.net/q/209702
  * [5] J. Cheeger and D. Gromoll  “On the structure of complete manifolds of nonnegative curvature”  In _Ann. of Math. (2)_ 96, 1972, pp. 413–443  DOI: 10.2307/1970819
  * [6] J. Eckhoff  “Helly, Radon, and Carathéodory type theorems”  In _Handbook of convex geometry, Vol. A, B_ North-Holland, Amsterdam, 1993, pp. 389–448
  * [7] Y. Eliashberg and N. Mishachev  “Introduction to the $h$-principle” 48, Graduate Studies in Mathematics, 2002  DOI: 10.1090/gsm/048
  * [8] M. Gromov  “Asymptotic invariants of infinite groups”  In _Geometric group theory, Vol. 2 (Sussex, 1991)_ 182, London Math. Soc. Lecture Note Ser.  Cambridge Univ. Press, Cambridge, 1993, pp. 1–295
  * [9] O. Kowalski and M. Belger  “Riemannian metrics with the prescribed curvature tensor and all its covariant derivatives at one point”  In _Math. Nachr._ 168, 1994, pp. 209–225  DOI: 10.1002/mana.19941680113
  * [10] N. Li and A. Naber  “Quantitative estimates on the singular sets of Alexandrov spaces”  In _Peking Math. J._ 3.2, 2020, pp. 203–234  DOI: 10.1007/s42543-020-00026-2
  * [11] A.. Milka  “Shortest paths on convex surfaces”  In _J. Soviet Math._ 33.2, 1986, pp. 910–933
  * [12] N. Monod  “Extreme points in non-positive curvature”  In _Studia Math._ 234.3, 2016, pp. 265–270  DOI: 10.4064/sm8549-6-2016
  * [13] T. Murphy and F. Wilhelm  “Random manifolds have no totally geodesic submanifolds”  In _Michigan Math. J._ 68.2, 2019, pp. 323–335  DOI: 10.1307/mmj/1555034652
  * [14] A. Petrunin  “Parallel transportation for Alexandrov space with curvature bounded below”  In _Geom. Funct. Anal._ 8.1, 1998, pp. 123–148  DOI: 10.1007/s000390050050
  * [15] A. Petrunin  “Example of non-closed convex hull in a CAT(0) space”, MathOverflow  URL: https://mathoverflow.net/q/8505
  * [16] L. Zajíček  “On the differentiation of convex functions in finite and infinite dimensional spaces”  In _Czechoslovak Math. J._ 29(104).3, 1979, pp. 340–348