# Interaction-driven topological phase diagram of twisted bilayer MoTe2

Wen-Xuan Qiu  School of Physics and Technology, Wuhan University, Wuhan
430072, China  Bohao Li  School of Physics and Technology, Wuhan University,
Wuhan 430072, China  Xun-Jiang Luo  School of Physics and Technology, Wuhan
University, Wuhan 430072, China  Fengcheng Wu<EMAIL_ADDRESS>School of
Physics and Technology, Wuhan University, Wuhan 430072, China  Wuhan Institute
of Quantum Technology, Wuhan 430206, China

###### Abstract

Twisted bilayer MoTe2 is a promising platform to investigate the interplay
between band topology and many-body interaction. We present a theoretical
study of its interaction-driven quantum phase diagrams based on a three-
orbital model, which can be viewed as a generalization of the Kane-Mele-
Hubbard model with one additional orbital and long-range Coulomb repulsion. We
predict a cascade of phase transitions tuned by the twist angle $\theta$. At
the hole filling factor $\nu=1$ (one hole per moiré unit cell), the ground
state can be in the multiferroic phase with coexisting spontaneous layer
polarization and magnetism, the quantum anomalous Hall phase, and finally the
topologically trivial magnetic phases, as $\theta$ increases from
$1.5^{\circ}$ to $5^{\circ}$. At $\nu=2$, the ground state can have a second-
order phase transition between an antiferromagnetic phase and the quantum spin
Hall phase as $\theta$ passes through a critical value. The dependence of the
phase boundaries on model parameters such as the gate-to-sample distance, the
dielectric constant, and the moiré potential amplitude is examined. The
predicted phase diagrams can guide the search for topological phases in
twisted transition metal dichalcogenide homobilayers.

##  I introduction

Moiré superlattices formed by group-VI transition metal dichalcogenides (TMD)
provide a new solid-state platform to simulate model Hamiltonians on honeycomb
and triangular lattices [1, 2, 3, 4, 5, 6]. TMD monolayers are semiconductors
with sizable spin-orbit splitting in the valence band [7], and therefore, have
a reduced number of low-energy degrees of freedom compared to monolayer
graphene. This theoretical simplification makes moiré TMD systems an appealing
platform for quantum simulation. In twisted TMD homobilayers, the low-energy
moiré valence bands can be mapped to the Kane-Mele model on a honeycomb
lattice for $\pm K$ valley states [2], or a honeycomb tight-binding model with
negligible spin-orbit coupling for $\Gamma$ valley states [3, 4, 5]. A variety
of physical effects have been theoretically predicted for the twisted TMD
homobilayers [2, 3, 4, 5, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]. Experimental
studies have reported correlated insulating states as well as signatures of
interaction-induced topological states in twisted bilayer WSe2 ($t$WSe2) [18,
19, 20] and twisted bilayer MoTe2 ($t$MoTe2) [21, 22, 23]. In moiré TMD
heterobilayers, low-energy states reside primarily in one layer due to the
band offset between two different TMD materials and can simulate the
generalized Hubbard model on triangular lattices [1]. Experimental
observations of Mott insulators [24, 25], generalized Wigner crystals [26,
27], metal-insulator transitions [28, 29], and heavy fermion behaviors [30] in
moiré TMD heterobilayers support the Hubbard model physics. An out-of-plane
electric field can reduce the band offset in the heterbilayer and induce band
inversion, which can result in topological moiré bands [31]. The electric
field-induced topological states have been experimentally realized in AB-
stacked MoTe2/WSe2 heterobilayer [32, 33, 34], which stimulated active
theoretical studies on the nature of the states [35, 36, 37, 38, 39, 40, 41,
42, 43, 44].

Figure 1: (a) Moiré superlattices formed in $t$MoTe2. The dots with green,
red, and blue colors indicate, respectively, $O$, $A$, and $B$ sites, which
also correspond to the centers of the three Wannier orbitals. The black dashed
lines mark a moiré unit cell. (b), (c) Moiré band structure at
$\theta=1.5^{\circ}$ and $2.5^{\circ}$ in $+K$ valley. The integer numbers at
$\bm{\gamma}$ and $\bm{\kappa}_{\pm}$ label the $C_{3z}$ angular momentum of
the first three bands (see Appendix A), while the color represents the
relative weight of the three Wannier orbitals. The band structures are plotted
along high-symmetry paths in momentum space with
$\bm{\kappa}^{\prime}_{+}=-\bm{\kappa}_{-}$.

In this paper, we present a theoretical study of the interaction-driven
quantum phase diagram of $t$MoTe2, where the two layers are rotated by an
angle $\theta$ relative to the rhombohedral-stacked bilayer. As predicted in
Ref. 2, low-energy moiré valence bands of $t$MoTe2 originate from $\pm K$
valleys of monolayer MoTe2, where $\pm K$ refer to the monolayer Brillouin
zone corners. The intralayer moiré potentials confine low-energy states to two
different locations [i.e., $A$ and $B$ sites of the moiré superlattice shown
in Fig. 1(a)] in the bottom and top layers, where the two locations exactly
correspond to the two sublattices of a honeycomb lattice. The rotation between
the two layers generates layer-dependent momentum shifts in the kinetic
energy, which induces effective valley-contrasting and sublattice-dependent
fluxes on the honeycomb lattice model. Because of the lattice geometry and the
flux pattern, the first two moiré valence bands realize the Haldane model [45]
within one valley and the Kane-Mele model [46, 47] when both valleys are
considered [2]. Moreover, the moiré bands have narrow bandwidth, and electron
interaction effects are enhanced. Therefore, $t$MoTe2 can serve as a model
system to study the interplay between band topology and many-body interaction.
However, $t$MoTe2 has not been studied experimentally until very recently [21,
22, 23]. Two independent experiments reported signatures of integer and
fractional quantum anomalous Hall states in $t$MoTe2 with $\theta\approx
3.7^{\circ}$ [22] and $3.4^{\circ}$ [23], respectively. This exciting
experimental progress makes the theoretical prediction of Ref. 2 to be of
experimental relevance, and calls for thorough studies of the many-body
interaction effects in $t$MoTe2.

The mapping between the first two moiré bands and the Kane-Mele model allows
to study the interaction effects within a lattice model, which should be a
generalized Kane-Mele-Hubbard model since the realistic interaction is the
long-range Coulomb repulsion. However, such a mapping is limited to the small
$\theta$ regime of $\theta<\theta_{1}^{*}$, where $\theta_{1}^{*}$ is
estimated to be about $1.74^{\circ}$ in Ref. 2. For $\theta>\theta_{1}^{*}$,
there is a topological band inversion between the second and the third bands.
To describe the low-energy moiré bands in a unified manner regardless of the
exact values of $\theta$, we construct a three-orbital model based on the
Wannier states of the first three bands. The three-orbital model faithfully
captures the energy, symmetry, and topology of the first three bands. The
interacting model is then obtained by projecting the Coulomb interaction onto
the Bloch-like states derived from the Wannier orbitals. The interacting
Hamiltonian is formulated in momentum space and implicitly includes all
microscopic interactions such as density-density interaction and Hund’s
coupling.

We calculate the quantum phase diagrams of the interacting three-orbital model
at hole filling factors $\nu=1$ and $\nu=2$, where $\nu$ counts the number of
doped holes per moiré unit cell. The phase diagrams are obtained by comparing
the energy of multiple competing states within self-consistent Hartree-Fock
approximation. We construct the phase diagram as a function of $\theta$ and
examine its dependence on the gate-to-sample distance $d$, the dielectric
constant $\epsilon$, and the moiré potential amplitude $V$. We predict a
cascade of phase transitions tuned by $\theta$ in the range between
$1.5^{\circ}$ and $5^{\circ}$. At $\nu=1$, the ground state is in the
multiferroic phase with spontaneous layer polarization and magnetism for small
$\theta$, the quantum anomalous Hall phase with a Chern number of 1 for
intermediate $\theta$, and topologically trivial magnetic phases for large
$\theta$. At $\nu=2$, the ground state is in interaction-driven magnetic
phases for small $\theta$ and turns into the quantum spin Hall phase for large
$\theta$. We also discuss how the phase boundaries depend on parameters such
as $d$, $\epsilon$, and $V$. The predicted phase diagrams demonstrate the
remarkable richness of $t$MoTe2, and are expected to guide further
experimental and theoretical studies.

There are several benefits of doing the many-body calculation using the three-
orbital model compared to the full continuum model. First, the constructed
Wannier states provide a clear real-space picture of the low-energy degrees of
freedom and valuable intuition for studying interaction physics. The localized
Wannier orbitals allow us to devise various mean-field states with different
layer polarization, magnetization, and topology. Second, the symmetry and
topology of electronic states become more apparent in the Wannier basis.
Third, the three-orbital model can be further studied using techniques (e.g.,
exact diagonalization) beyond mean-field theory.

The paper is organized as follows. In Sec. II, we construct the noninteracting
as well as the interacting three-orbital model. Construction of the Wannier
states is informed by symmetry analysis of the moiré bands, which is detailed
in Appendix A. In Sec. III, we present the phase diagrams at $\nu=1$ and
$\nu=2$. Additional discussion on the phase diagram and the model is given,
respectively, in Appendices B and C. In Sec. IV, we conclude with a summary
and a discussion. Our theoretical results are compared with the recent
experimental findings, and future directions are discussed.

##  II Model

###  II.1 Moiré Hamiltonian

The moiré superlattices of $t$MoTe2 have $D_{3}$ point-group symmetry
generated by a threefold rotation $C_{3z}$ around the out-of-plane $\hat{z}$
axis and a twofold rotation $C_{2y}$ around the in-plane $\hat{y}$ axis that
exchanges the bottom ($b$) and top ($t$) layers. As illustrated in Fig. 1(a),
the moiré superlattices have three distinct high-symmetry locations labeled by
$O$, $A$, and $B$, where the latter two positions are related by the $C_{2y}$
symmetry.

The single-particle moiré Hamiltonian for valence band states in $t$MoTe2 has
been constructed in Ref. 2 and is given by,

$\displaystyle\mathcal{H}_{\tau}=\left(\begin{array}[]{cc}-\frac{\hbar^{2}\left(\bm{k}-\tau\bm{\bm{\kappa}}_{+}\right)^{2}}{2m^{*}}+\Delta_{+}(\bm{r})&\Delta_{\mathrm{T,\tau}}(\bm{r})\\\
\Delta_{\mathrm{T,\tau}}^{\dagger}(\bm{r})&-\frac{\hbar^{2}\left(\bm{k}-\tau\bm{\kappa}_{-}\right)^{2}}{2m^{*}}+\Delta_{-}(\bm{r})\end{array}\right),$
(1)
$\displaystyle\Delta_{\pm}(\bm{r})=2V\sum_{j=1,3,5}\cos\left(\bm{g}_{j}\cdot\bm{r}\pm\psi\right),$
$\displaystyle\Delta_{\mathrm{T,\tau}}(\bm{r})=w\left(1+e^{-i\tau\bm{g}_{2}\cdot\bm{r}}+e^{-i\tau\bm{g}_{3}\cdot\bm{r}}\right).$

Here $\tau=\pm$ represents $\pm K$ valleys and also spin up ($\uparrow$) and
down ($\downarrow$), since spin and valley are locked for valence band states
in TMD[7]. The moiré Hamiltonian $\mathcal{H}_{\tau}$ in Eq. (1) is
represented by a $2\times 2$ matrix in the layer-pseudospin space. The
diagonal terms in $\mathcal{H}_{\tau}$ describe an electron moving in the
layer-dependent potential $\Delta_{\pm}(\bm{r})$ with an amplitude $V$ and
phase parameters $\pm\psi$, and the off-diagonal terms account for the
interlayer tunneling with a strength $w$. In $\mathcal{H}_{\tau}$, $\bm{r}$
and $\bm{k}$ are, respectively, position and momentum operators, $m^{*}$ is
the effective mass, the momentum shifts
$\bm{\kappa}_{\pm}=\left[4\pi/(3a_{M})\right](-\sqrt{3}/2,\mp 1/2)$ are
located at corners of the moiré Brillouin zone, and
$\bm{g}_{j}=\left[4\pi/(\sqrt{3}a_{M})\right]\\{\cos[(j-1)\pi/3],\sin[(j-1)\pi/3]\\}$
for $j=1,...,6$ are the moiré reciprocal lattice vectors. Here $a_{M}\approx
a_{0}/\theta$ is the moiré period, and $a_{0}$ is the monolayer lattice
constant. The moiré Hamiltonian $\mathcal{H}_{\tau}$ is constructed based on
continuum approximation. Effects of in-plane lattice relaxation, which can be
important at small twist angles ($\theta\lesssim 1^{\circ}$), are not taken
into account in the Hamiltonian $\mathcal{H}_{\tau}$. The continuum
approximation can also become less accurate at large twist angles
($\theta\gtrsim 5^{\circ}$) where the superlattices become small.
Nevertheless, the Hamiltonian $\mathcal{H}_{\tau}$ qualitatively captures the
evolution of electronic states as a function of $\theta$ in a convenient way.

The moiré Hamiltonian $\mathcal{H}_{\tau}$ with a fixed $\tau$ index is
invariant under the $C_{3z}$ and the combined $C_{2y}\mathcal{T}$ symmetry
operations, where $\mathcal{T}$ is the time-reversal symmetry. The two valleys
are related by the $C_{2y}$ and the $\mathcal{T}$ symmetries, which map
$\mathcal{H}_{+}$ to $\mathcal{H}_{-}$. Moiré bands of $\mathcal{H}_{\tau}$
can be characterized by Chern numbers $C_{\tau}^{(n)}$, where $n$ is the band
index (counting in descending order of energy). Because of the $\mathcal{T}$
symmetry, $C_{-}^{(n)}=-C_{+}^{(n)}$.

A systematic study on the single-particle topological phase diagram of
$\mathcal{H}_{\tau}$ as a function of the model parameters $(V,\psi,w)$ can be
found in Ref. 9. Unless otherwise stated, we take the parameter values
estimated for $t$MoTe2 from Ref. 2, $a_{0}=3.472$Å, $m^{*}=0.62m_{e}$, $V=8$
meV, $\psi=-89.6^{\circ}$, and $w=-8.5$ meV, where $m_{e}$ is the electron
bare mass. The calculated moiré band structures are plotted in Figs. 1(b) and
1(c) for two representative twist angles $\theta=1.5^{\circ}$ and
$2.5^{\circ}$, respectively. For twist angles $\theta\leq 5^{\circ}$ studied
in this work, $C_{+}^{(1)}$ of the first band in $+K$ valley is always
quantized to 1. However, $(C_{+}^{(2)},C_{+}^{(3)})$ change from $(-1,0)$ for
$\theta<\theta_{1}^{*}$ to $(1,-2)$ for $\theta>\theta_{1}^{*}$, where
$\theta_{1}^{*}\approx 1.74^{\circ}$. The topological phase transition at
$\theta_{1}^{*}$ is accompanied by the gap closing between the second and the
third bands at the $\bm{\gamma}$ point (i.e., the center of the moiré
Brillouin zone). The first three moiré bands with a total Chern number of 0
provide a low-energy approximation of the full moiré band structure and are
the focus of this work.

Figure 2: The amplitude of each layer component of Wannier states
$W^{n\tau}(\bm{r})=\left[W_{b}^{n\tau}(\bm{r}),W_{t}^{n\tau}(\bm{r})\right]^{T}$
for $\theta=2.5^{\circ}$ and $\tau=+$. The black lines mark the effective
triangular lattice formed by $O$ sites.

###  II.2 Wannier states

We construct Wannier states for the first three moiré bands so that the bands
can be represented in terms of local orbitals. The center of the Wannier
states can be determined by the $C_{3z}$ eigenvalues of Bloch states at the
high-symmetry momenta (i.e., $\bm{\gamma}$ and $\bm{\kappa}_{\pm}$). Symmetry
analysis of the moiré Hamiltonian and Bloch states is performed in Appendix A
following the approach developed in Ref. 44. The $C_{3z}$ angular momenta of
the Bloch states at $\bm{\gamma}$ and $\bm{\kappa}_{\pm}$ in the first three
bands are labeled in Figs. 1(b) and 1(c). The symmetry eigenvalues indicate
that Wannier states for the first three bands should be centered at the $A$,
$B$, and $O$ positions, respectively.

We complement the symmetry analysis with a physical picture of the Wannier
states. We first consider the case with $\theta<\theta_{1}^{*}$. In this first
case, the first two bands have opposite Chern numbers $(\pm 1)$, and the third
band has a Chern number of 0. Previous studies [2, 11] have shown that the
first two bands in one valley effectively realize a generalized Haldane model
on a honeycomb lattice with two Wannier states located at the $A$ and $B$
positions, respectively. For the third band, the $C_{3z}$ eigenvalues take the
same value at $\bm{\gamma}$ and $\bm{\kappa}_{\pm}$ ( Fig. 1 (b)), which
implies that the corresponding Wannier states are centered at the $O$
positions. These considerations are consistent with the results obtained from
the symmetry analysis. We then consider the second case with
$\theta>\theta_{1}^{*}$, which is separated from the first case by a band
inversion between the second and the third bands. This band inversion does not
change the centers of Wannier states for the three bands as a whole.

We construct the Wannier states for $\theta<\theta_{1}^{*}$ and
$\theta>\theta_{1}^{*}$ in a unified manner. The Wannier states in the moiré
unit cell associated with the lattice site $\bm{R=0}$ can be formally
expressed as,

$W^{n\tau}(\bm{r})=\frac{1}{\sqrt{N}}\sum_{\bm{k}}\phi_{\bm{k}}^{n\tau}(\bm{r}),$
(2)

where $\tau$ is the valley index, $n$ labels the three Wannier orbitals
($n=$1,2,3), and $N$ is the number of $\bm{k}$ points included in the
summation. Wannier states at a generic site $\bm{R}$ are obtained through
lattice translation, $W^{n\tau}_{\bm{R}}(\bm{r})=W^{n\tau}(\bm{r}-\bm{R})$. In
Eq. (2), $\phi_{\bm{k}}^{n\tau}(\bm{r})$ is a Bloch-like state associated with
the Wannier states and related to Bloch states of $\mathcal{H}_{\tau}$ by a
unitary transformation,

$\phi_{\bm{k}}^{n\tau}(\bm{r})=\sum_{n^{\prime}=1,2,3}U_{\bm{k}\tau}^{n^{\prime}n}\psi_{\bm{k}}^{n^{\prime}\tau}(\bm{r}),$
(3)

where $\psi_{\bm{k}}^{n^{\prime}\tau}(\bm{r})$ is the Bloch state of the
$n^{\prime}$th moiré band at momentum $\bm{k}$ and $U_{\bm{k}\tau}$ is a
3$\times$3 unitary matrix. We obtain $U_{\bm{k}\tau}$ by requiring that
$\phi_{\bm{k}}^{1\tau}$ and $\phi_{\bm{k}}^{3\tau}$ are, respectively,
maximally polarized to the top and bottom layers. This maximum value problem
is equivalent to find the eigenstates of the layer polarization operator
$\sigma_{z}$ (i.e., the $z$ Pauli matrix in the layer pseudospin space)
projected to the subspace spanned by $\psi_{\bm{k}}^{n\tau}$ with $n=1,2,3$,

$\Pi_{\bm{k}\tau}=\left(\begin{array}[]{ccc}\left\langle\psi_{\bm{k}}^{1\tau}\left|\sigma_{z}\right|\psi_{\bm{k}}^{1\tau}\right\rangle&\left\langle\psi_{\bm{k}}^{1\tau}\left|\sigma_{z}\right|\psi_{\bm{k}}^{2\tau}\right\rangle&\left\langle\psi_{\bm{k}}^{1\tau}\left|\sigma_{z}\right|\psi_{\bm{k}}^{3\tau}\right\rangle\\\
\left\langle\psi_{\bm{k}}^{2\tau}\left|\sigma_{z}\right|\psi_{\bm{k}}^{1\tau}\right\rangle&\left\langle\psi_{\bm{k}}^{2\tau}\left|\sigma_{z}\right|\psi_{\bm{k}}^{2\tau}\right\rangle&\left\langle\psi_{\bm{k}}^{2\tau}\left|\sigma_{z}\right|\psi_{\bm{k}}^{3\tau}\right\rangle\\\
\left\langle\psi_{\bm{k}}^{3\tau}\left|\sigma_{z}\right|\psi_{\bm{k}}^{1\tau}\right\rangle&\left\langle\psi_{\bm{k}}^{3\tau}\left|\sigma_{z}\right|\psi_{\bm{k}}^{2\tau}\right\rangle&\left\langle\psi_{\bm{k}}^{3\tau}\left|\sigma_{z}\right|\psi_{\bm{k}}^{3\tau}\right\rangle\end{array}\right),$
(4)

where
$\psi_{\bm{k}}^{n\tau}=[\psi_{\bm{k}b}^{n\tau},\psi_{\bm{k}t}^{n\tau}]^{T}$ is
a two-component spinor in the layer-pseudospin space. $U_{\bm{k}\tau}$ is then
determined by,

$U_{\bm{k}\tau}^{\dagger}\Pi_{\bm{k}\tau}U_{\bm{k}\tau}=\left(\begin{array}[]{ccc}\rho_{\bm{k}}^{1\tau}&0&0\\\
0&\rho_{\bm{k}}^{2\tau}&0\\\ 0&0&\rho_{\bm{k}}^{3\tau}\end{array}\right),$ (5)

where $\rho_{\bm{k}}^{1\tau}<\rho_{\bm{k}}^{2\tau}<\rho_{\bm{k}}^{3\tau}$. We
further fix the phase of $\phi_{\bm{k}}^{n\tau}$ in the following way. The
phase of $\phi_{\bm{k}}^{1\tau}$ is chosen by requiring that its top layer
component $\phi_{\bm{k}t}^{1\tau}$ is real and positive at
$\bm{r}_{A}=a_{M}(1/\sqrt{3},0)$, which is the $A$ site associated with
$\bm{R}=0$. Similarly, we take the bottom layer component of
$\phi_{\bm{k}}^{3\tau}$ to be real and positive at the $B$ site
$\bm{r}_{B}=a_{M}(2/\sqrt{3},0)$. For $\phi_{\bm{k}}^{2\tau}$, we first fix
the gauge at $\bm{k}=\bm{\gamma}$ according to
$\phi_{\bm{\gamma}b}^{2\tau}(\bm{r}_{O})>0$, where $\bm{r}_{O}=a_{M}(0,0)$;
then, at a generic $\bm{k}$, we require that
$\langle\phi_{\bm{\gamma}}^{2\tau}(\bm{r}_{O})|\phi_{\bm{k}}^{2\tau}(\bm{r}_{O})\rangle>0$.
For convenience, hereafter we denote the Bloch-like states
$\phi_{\bm{k}}^{n\tau}$ with $n=1$, $2$, and $3$, respectively, as
$\phi_{\bm{k}}^{A\tau}$, $\phi_{\bm{k}}^{O\tau}$, and $\phi_{\bm{k}}^{B\tau}$,
and similarly for the Wannier states $W^{n\tau}$.

The Wannier states constructed for $\theta=2.5^{\circ}$ using the above
methods are shown in Fig. 2, where the amplitude of each layer component of
$W^{n\tau}$ is plotted. The Wannier states $W^{A\tau}$ and $W^{B\tau}$ are,
respectively, polarized mainly to the top and bottom layer, and transformed to
each other through the $C_{2y}\mathcal{T}$ operation (plus a lattice
translation). The Wannier state $W^{O\tau}$ has significant weights on both
layers and is an eigenstate of the $C_{2y}\mathcal{T}$ symmetry.

A tight-binding Hamiltonian based on the Wannier states can be formally
constructed in real space and then Fourier transformed to obtain the momentum-
space Bloch Hamiltonian [44], which has the following second-quantized form in
the basis of $\phi_{\bm{k}}^{n\tau}$,

$\displaystyle\hat{\mathcal{H}}_{0}$
$\displaystyle=\sum_{\bm{k},\tau,n,n^{\prime}}h_{\bm{k}\tau}^{nn^{\prime}}c_{\bm{k}n\tau}^{\dagger}c_{\bm{k}n^{\prime}\tau},$
(6) $\displaystyle h_{\bm{k}\tau}$
$\displaystyle=U_{\bm{k}\tau}^{\dagger}\left(\begin{array}[]{ccc}E_{\bm{k}}^{1\tau}&0&0\\\
0&E_{\bm{k}}^{2\tau}&0\\\
0&0&E_{\bm{k}}^{3\tau}\end{array}\right)U_{\bm{k}\tau}.$

Here $c_{\bm{k}n\tau}^{\dagger}$ ($c_{\bm{k}n\tau}$) is the electron creation
(annihilation) operator of the Bloch-like state $\phi_{\bm{k}}^{n\tau}$, and
$E_{\bm{k}}^{n\tau}$ is the energy of the Bloch state $\psi_{\bm{k}}^{n\tau}$
with respect to $\mathcal{H}_{\tau}$.

The Hamiltonian $\hat{\mathcal{H}}_{0}$ presents a noninteracting three-
orbital model. It faithfully captures the symmetry and topology of the first
three moiré bands. By diagonalizing $h_{\bm{k}\tau}$, we reproduce the band
structure of the first three bands and also obtain their weights projected
onto the Wannier orbitals, as shown in Figs. 1 (b) and (c). For
$\theta<\theta_{1}^{*}$, the first two moiré bands are mainly derived from the
hybridization of $A$ and $B$ orbitals, while the third band is mainly composed
of $O$ orbital. For $\theta>\theta_{1}^{*}$, the second and the third bands
exchange their orbital characters at the $\bm{\gamma}$ point, which results in
the band inversion.

###  II.3 Interacting model

We now construct the interacting three-orbital model. For the charge-neutral
twisted homobilayer, all valence band states are below the Fermi energy. When
the system is doped with holes, it is more convenient to use the hole basis.
We define the hole operator as $b_{\bm{k}n\tau}=c_{\bm{k}n\tau}^{\dagger}$. In
the hole basis, the noninteracting Hamiltonian $\hat{\mathcal{H}}_{0}$ is
equivalent to

$\hat{\mathcal{H}}_{0}=-\sum_{\bm{k},\tau,n,n^{\prime}}[h_{\bm{k}\tau}^{\top}]^{nn^{\prime}}b_{\bm{k}n\tau}^{\dagger}b_{\bm{k}n^{\prime}\tau},$
(7)

where a constant term is dropped.

Figure 3: (a), (b) Top panel: The quantum phase diagram at $\nu=1$ as a
function of $\theta$ and $d$. The color map plots the charge gap. White solid
lines mark the first-order phase transitions. Bottom panel: Energy per moiré
unit cell of competing states relative to the layer unpolarized but valley
polarized (LUP-VP) states at $d=20$ nm. $\epsilon$ is 10 in (a) and 15 in (b).
(c) Schematic illustration of competing states. In the $\textrm{LP-FM}_{x}$,
$\textrm{LP-FM}_{z}$, $\textrm{LP-120}^{\circ}\textrm{AF}^{+}$ and
$\textrm{LP-120}^{\circ}\textrm{AF}^{-}$ states, the $A$ and $B$ sublattices
have unequal hole densities. In $\textrm{LP-FM}_{x}$ ($\textrm{LP-FM}_{z}$)
state, $A$ (or $B$) sublattices have ferromagnetism along $x$ ($z$) direction.
In $\textrm{LP-120}^{\circ}\textrm{AF}^{+}$
($\textrm{LP-120}^{\circ}\textrm{AF}^{-}$) state, the Néel order has an
anticlockwise (clockwise) arrangement on $A$ (or $B$) sublattices along the
$y$ direction. Similar notations are used for $O$-FMx and $O$-120∘AF± states,
of which magnetic moments are developed on the $O$ sublattice. In QAHI and
$\textrm{FM}_{z}$ states, the $A$ and $B$ sublattices have equal density and
the magnetization is along the $z$ direction. The total Chern number is 1 in
QAHI and 0 in $\textrm{FM}_{z}$ states, both of which can be classified as the
LUP-VP states. Figure 4: (a)-(c) The HF band structure at $\nu=1$ in (a) the
LP-FMz phase at $\theta=1.5^{\circ}$, (b) the QAHI phase at
$\theta=2.5^{\circ}$, and (c) the FMz phase at $\theta=3.5^{\circ}$. The band
structures are presented in the basis defined by $c^{\dagger}_{\bm{k}n\tau}$
and $c_{\bm{k}n\tau}$ operators. The solid and dotted lines, respectively,
plot bands from $+K$ and $-K$ valleys. The color represents the weight of the
three Wannier orbitals. The middle of the interaction-induced gap, marked by
the black dashed line, is set to 0 in each plot. The integer numbers at
$\bm{\gamma}$ and $\bm{\kappa}_{\pm}$ label the $C_{3z}$ angular momentum of
the band above the Fermi energy. (d) The sublattice polarization $P_{AB}$ (red
line), the hole number $n_{O}$ on each $O$ site (blue dashed line), and the
charge gap (black line) in $\nu=1$ ground states as a function of $\theta$.
$P_{AB}$ characterizes the layer polarization and is defined as
$|n_{A}-n_{B}|/(n_{A}+n_{B})$, where $n_{A}$ ($n_{B}$) is the hole number on
each $A$ ($B$) site. The vertical dashed lines mark the transitions between
different phases. $d$ is 20 nm and $\epsilon$ is 15 for (a)-(d).

The Coulomb interaction projected onto the three Wannier orbitals in the hole
representation is expressed as

$\begin{split}\hat{\mathcal{H}}_{\textrm{int}}=\frac{1}{2}\sum
V_{\bm{k}_{1}\bm{k}_{2}\bm{k}_{3}\bm{k}_{4}}^{n_{1}n_{2}n_{3}n_{4}}\left(\tau,\tau^{\prime}\right)b_{\bm{k}_{1}n_{1}\tau}^{+}b_{\bm{k}_{2}n_{2}\tau^{\prime}}^{+}b_{\bm{k}_{3}n_{3}\tau^{\prime}}b_{\bm{k}_{4}n_{4}\tau},\end{split}$
(8)

where the summation is over the momentum $\bm{k}_{j}$ (summed over the moiré
Brillouin zone), the orbital index $n_{j}$, and the valley index $\tau$. The
Coulomb matrix element is given by,

$V_{\bm{k}_{1}\bm{k}_{2}\bm{k}_{3}\bm{k}_{4}}^{n_{1}n_{2}n_{3}n_{4}}\left(\tau,\tau^{\prime}\right)=\frac{1}{\mathcal{A}}\sum_{\bm{q}}V_{\bm{q}}M_{\bm{k}_{1}\bm{k}_{4}}^{n_{1}n_{4}}\left(\tau,\bm{q}\right)M_{\bm{k}_{2}\bm{k}_{3}}^{n_{2}n_{3}}\left(\tau^{\prime},-\bm{q}\right),$
(9)

where $\mathcal{A}$ is the system area. The structure factor
$M_{\bm{k}_{1}\bm{k}_{4}}^{n_{1}n_{4}}\left(\tau,\bm{q}\right)$ is

$M_{\bm{k}_{1}\bm{k}_{4}}^{n_{1}n_{4}}\left(\tau,\bm{q}\right)=\sum_{\ell}\int
d\bm{r}e^{i\bm{q}\cdot\bm{r}}{[{\tilde{\phi}}_{\bm{k}_{1}\ell}^{n_{1}\tau}\left(\bm{r}\right)]}^{*}{\tilde{\phi}}_{\bm{k}_{4}\ell}^{n_{4}\tau}\left(\bm{r}\right),$
(10)

where $\ell$ is the layer index, and
$\tilde{\phi}_{\bm{k}}^{n\tau}\left(r\right)=[\phi_{\bm{k}}^{n\tau}\left(r\right)]^{*}$
due to the particle-hole transformation. We use the dual-gate screened Coulomb
interaction with the momentum-dependent potential $V_{\bm{q}}=2\pi
e^{2}\tanh{(|\bm{q}|d)}/(\epsilon|\bm{q}|)$, where $d$ is the gate-to-sample
distance and $\epsilon$ is the dielectric constant. Here we neglect the effect
of the vertical separation $d_{0}$ between the two MoTe2 layers on the Coulomb
potential since $d_{0}$ is much smaller than the typical interparticle
distance.

By combining the noninteracting Hamiltonian $\hat{\mathcal{H}}_{0}$ in Eq. (7)
with the Coulomb interaction term $\hat{\mathcal{H}}_{\textrm{int}}$ in Eq.
(8), we obtain the full Hamiltonian of the interacting three-orbital model,

$\begin{split}\hat{\mathcal{H}}=\hat{\mathcal{H}}_{0}+\hat{\mathcal{H}}_{\textrm{int}}.\end{split}$
(11)

Here we formulate the Hamiltonian $\hat{\mathcal{H}}$ in momentum space. An
alternative approach would be to construct the Hamiltonian in real space by
projecting the Coulomb interaction onto the Wannier states. This latter
approach works well to capture onsite Hubbard interactions and offsite
density-density interactions [10], but is not suited to keep track of all
relevant non-local interactions such as intersite Hund’s coupling, pair
hopping, and interaction-assisted hopping terms [48, 49, 50]. As shown in Fig.
2, neighboring Wannier states can have significant spatial overlap, and can
lead to enhanced non-local interactions that play an important role in
determining the ground state properties [48, 49, 50]. Therefore, we do not use
the real-space formalism. The momentum-space Hamiltonian $\hat{\mathcal{H}}$
implicitly includes all microscopic interactions, and has been widely employed
in moiré systems such as twisted bilayer graphene [51] to study correlated
states [52, 53].

##  III Phase diagram

We perform self-consistent mean-field studies of the Hamiltonian $\mathcal{H}$
using the Hartree-Fock (HF) approximation for integer filling factors $\nu=1$
and $2$ at zero temperature. Here
$\nu=\frac{1}{N}\sum_{\bm{k},n,\tau}b_{\bm{k}n\tau}^{\dagger}b_{\bm{k}n\tau}$
is the number of holes per moiré unit cell. At a given filling factor, we
study multiple mean-field ansatzes that can break various symmetries of the
system. The self-consistent calculation starting from different ansatz states
often generates different mean-field solutions, whose energetic competitions
determine the mean-field ground state.

We calculate the phase diagram as a function of the twist angle $\theta$, the
gate-to-sample distance $d$, and the dielectric constant $\epsilon$. The
parameter $d$ can be experimentally varied by controlling the thickness of the
encapsulating hBN layer. The screening effect by metallic gates is enhanced by
reducing $d$. The dielectric constant $\epsilon$ accounts for the
environmental screening from hBN as well as internal screening from remote
moiré bands. Here we take $\epsilon$ as a phenomenological parameter and vary
it theoretically to reveal the most generic phase diagram.

###  III.1 $\nu=1$ phase diagram

The quantum phase diagram at $\nu=1$ as a function of
$\theta\in(1.5^{\circ},5^{\circ})$ and $d\in(5~{}\text{nm},30~{}\text{nm})$ is
shown in Figs. 3(a) and 3(b) for $\epsilon=10$ and $15$, respectively. We
start by describing the $\theta$ dependence of the $\epsilon=15$ phase
diagram. Along a representative line cute with $d=$ 20 nm and $\epsilon=$ 15,
the ground state is in (1) the LP-FMz phase for $\theta<2^{\circ}$, (2) the
quantum anomalous Hall insulator (QAHI) phase for
$2^{\circ}<\theta<2.9^{\circ}$, (3) the topologically trivial FMz phase for
$2.9^{\circ}<\theta<3.9^{\circ}$, and (4) the $O-120^{\circ}$AF± phase for
$3.9^{\circ}<\theta<5.0^{\circ}$. Here the LP-FMz phase is a multiferroic
state with coexisting ferroelectricity and ferromagnetism. The
ferroelectricity arises from spontaneous layer polarization (LP) induced by
imbalanced hole densities on $A$ and $B$ Wannier orbitals [red line in Fig.
4(d)], which is driven by the Coulomb repulsion between $A$ and $B$ sites. The
magnetism is a result of valley polarization, which leads to out-of-plane
ferromagnetism (FMz) since valleys are locked to out-of-plane spins in the
system. We note that the LP-FMz phase has been previously obtained in an
exact-diagonalization study of $t$MoTe2 for $\theta$ limited to be below
$1.5^{\circ}$ [16], and multiferroic states have also been studied in
moiréless graphene systems [54, 55]. The next two phases in our phase diagram,
QAHI and FMz, are both valley polarized but layer unpolarized; they are
distinguished by the total Chern number $C$ of the ground state, which is 1 in
QAHI and 0 in FMz. The QAHI and FMz phases are separated by a weakly first-
order phase transition from our numerical calculations, which is manifested by
a slight jump of the occupation number of $O$ orbitals across the transition
[Fig. 4(d)]. Since this transition is first order, the gap does not need to
completely close at the transition, as shown in Fig. 4(d). The topological
transition is also revealed by the HF band structure in the two phases, as
shown in Figs. 4(b) and 4(c). The band (in the original basis defined by
$c^{\dagger}_{\bm{k}n\tau}$ and $c_{\bm{k}n\tau}$ operators) above the Fermi
level is derived from $A$ and $B$ orbitals at $\bm{\gamma}$ and
$\bm{\kappa}_{\pm}$ momenta in the QAHI phase, but is composed of $O$ orbital
at $\bm{\gamma}$ point in the FMz phase. This band inversion results in the
change of the Chern number. Lastly, the $O-120^{\circ}$AF± phase has the in-
plane 120∘ antiferromagnetic (AF) Néel order on the $O$ sites, and the
superscript $\pm$ indicates the vector chirality of the antiferromagnetism
[9], as illustrated in Fig. 3c. This AF state with a given vector chirality
spontaneously breaks the $C_{2y}$ symmetry, but states with opposite
chiralities are related by the $C_{2y}T$ symmetry and therefore, energetically
degenerate. Because of the $C_{2y}$ symmetry breaking, the $O-120^{\circ}$AF±
states also have spontaneous layer polarization controlled by the vector
chirality of the antiferromagnetism. This magnetoelectric coupling allows
electric tuning of the vector chirality. The layer polarization is numerically
confirmed by the imbalanced densities on $A$ and $B$ sites in the
$O-120^{\circ}$AF± phase, as demonstrated by the red line in Fig. 4(d).

Figure 5: The quantum phase diagram at $\nu=1$ and $d=20$ nm as a function of
$\theta$ and $\epsilon$. The color map plots the charge gap. The symbol “M”
stands for the metallic state without any symmetry breaking. The white dashed
line separates gapped and gapless $O$-120∘AF± states. Figure 6:  (a), (b) Top
panel: The quantum phase diagram at $\nu=2$ as a function of $\theta$ and $d$.
The color map plots the charge gap. White solid (dashed) lines mark the first
(second)-order phase transitions. Bottom panel: Energy per moiré unit cell of
competing states relative to the symmetric (sym) state at $d=20$ nm.
$\epsilon$ is 10 in (a) and 15 in (b). (c) Schematic illustration of competing
states at $\nu=2$.

The phase diagram is determined by comparing the energies of multiple
competing states, as shown in the lower panel of Figs. 3(a) and 3(b). In
addition to the states described above, we also find LP-FMx,
LP-$120^{\circ}$AF±, and $O$-FMx states as mean-field solutions in certain
parameter regimes, although these latter states are less energetically
favorable and do not appear in the ground-state phase diagram. Here LP-FMx
refers to a state with spontaneous layer polarization and in-plane
ferromagnetism (FMx) on either $A$ or $B$ sites. LP-$120^{\circ}$AF± is also
layer polarized, but has in-plane 120∘ AF order on either $A$ or $B$ sites;
the superscript $\pm$ again indicate the vector chirality. The $O$-FMx has in-
plane ferromagnetism on $O$ sites. The in-plane magnetism arises from
intervalley coherent order and spontaneously breaks the valley $U(1)$
symmetry.

The phase transition between competing phases is generically first order in
the $\nu=1$ phase diagram. Across the transitions, the energy of the ground
state is continuous, but other quantities, such as the sublattice-resolved
hole density, layer polarization, and gap, can have discontinuous jumps. The
evolution of these quantities as a function of $\theta$ for the ground state
is shown in Fig. 4(d). The $\theta$ dependence of the phase diagram reflects
the competition between the kinetic energy and the Coulomb interaction. As the
moiré period $a_{M}$ decreases by increasing $\theta$, the ratio of the
characteristic Coulomb repulsion $e^{2}/\epsilon a_{M}$ to the single-particle
bandwidth typically decreases, and different phases can appear.

We turn to the $\epsilon=10$ phase diagram in Fig. 3(a), which includes only
the LP-FMz phase for $\theta\lesssim 2.4^{\circ}$ and the FMz phase for
$\theta\gtrsim 2.4^{\circ}$. The QAHI state is a solution to the HF equations
for $\theta\lesssim 2^{\circ}$, but it is energetically unfavorable compared
to the LP-FMz phase in this small angle regime. The stronger Coulomb
interaction for $\epsilon=10$ tends to localize carriers by repulsion. The
holes are localized at $A$ or $B$ sites in the LP-FMz phase, but around $O$
sites in the FMz phase. The existence of the QAHI phase requires an
intermediate strength of the Coulomb repulsion.

The evolution of the phase diagram as a function of $\epsilon$ is shown in
Fig. 5. The region of the FMz phase in the parameter space of
$(\theta,\epsilon)$ shrinks as $\epsilon$ increases and finally vanishes for
$\epsilon>18$, where there is a first-order phase transition between the QAHI
phase and the $O-120^{\circ}$ AF± phase. The phase boundary between the QAHI
phase and the LP-FMz ($O-120^{\circ}$ AF±) phase shifts to lower $\theta$ as
$\epsilon$ increases. The reason is that the interaction effects become weaker
with increasing $\epsilon$ but stronger with decreasing $\theta$. The
$O-120^{\circ}$ AF± phase can become gapless at very large $\epsilon$ and
finally give way to the metallic state without any symmetry breaking.

We now discuss the $d$ dependence of the phase diagrams. The screening from
the gates mainly suppresses the Coulomb repulsion for the interaction range
larger than $d$. When $d\gg a_{M}$, the system should have a weak dependence
on $d$ since the gates only screen the long-range tail of the Coulomb
repulsion in this parameter regime. Numerically, we find that the phase
boundaries indeed have a relatively weak $d$ dependence for $d$ down to 10 nm,
as shown in the phase diagrams of Figs. 3(a) and 3(b). For small $d$, the
phase boundary between the LP-FMz and QAHI phases in the $\epsilon=15$ phase
diagram of Fig. 3(b) shifts towards the LP-FMz side as $d$ decreases from 10
nm to 5 nm; the Coulomb repulsion between the nearest $A$ and $B$ sites is
reduced under such small values of $d$, and the LP-FMz phase becomes less
energetically favorable. A similar effect is found for the phase boundary
between the FMz and QAHI phases, as shown in Fig. 3(b). The numerical results
indicate that the parameter regimes of the QAHI phase can be slightly enlarged
by reducing $d$ down to below $a_{M}$.

We present additional discussion on the $\nu=1$ phase diagram in Appendices B
and C. In Appendix B, we show that (1) the QAHI state is captured by the Kane-
Mele-Hubbard model, where the Hubbard interaction combined with the spin-orbit
coupling drives the out-of-plane spin polarization at $\nu=1$; (2) the
transition from the LP-FMz to the QAHI state is phenomenologically described
by a spinless Haldane model with nearest-neighbor repulsion; (3) the
$O-120^{\circ}$AF± state can be understood as a result of Fermi surface
instability. These analyses deepen the physical understanding of the phase
diagram. In Appendix C, we compare results obtained, respectively, from one-
band, two-band, and three-orbital models, which show that the three-orbital
model is essential to capture all the competing states.

###  III.2 $\nu=2$ phase diagram

The quantum phase diagram at $\nu=2$ as a function of $\theta$ and $d$ is
shown in Figs. 6(a) and 6(b) for $\epsilon=10$ and 15, respectively. The phase
diagram again has a weak $d$ dependence. Therefore, we focus on the $\theta$
dependence of the phase diagram. The ground state along the line cut with $d=$
20 nm and $\epsilon=$ 15 is in (1) the FMz phase for $\theta<1.7^{\circ}$, (2)
the AFx phase for $1.7^{\circ}<\theta<4.5^{\circ}$, and (3) the quantum spin
Hall insulator (QSHI) phase for $\theta>4.5^{\circ}$. The phase diagram is
obtained by comparing the energy of multiple competing states, including FMz,
FMx, AFx, AFz, and symmetric states. All these states have equal hole
densities on $A$ and $B$ sublattices. The symmetric state does not break any
symmetry of the system and always realizes the QSHI state for the $\theta$
range $(1.5^{\circ},5^{\circ})$ under consideration. In the FMz (FMx) state,
the $A$ and $B$ sublattices have ferromagnetic moments along $z$ ($x$)
direction. The AFz (AFx) state has collinear antiferromagnetic Néel order with
opposite magnetic moments on $A$ and $B$ sublattices along $z$ ($x$)
direction. The HF band structures for the FMx, AFx, and QSHI phases are,
respectively, shown in Figs. 7(a), 7(b), and 7(c), where the color encodes the
weight on the $O$ orbital. The results show that the holes at $\nu=2$ are
primarily doped to the $A$ and $B$ sites.

Figure 7: (a)-(c) The HF band structure at $\nu=2$ in (a) the FMz phase at
$\theta=1.5^{\circ}$, (b) the AFx phase at $\theta=2.5^{\circ}$, and (c) the
QSHI phase at $\theta=4.7^{\circ}$. The band structures are presented in the
basis defined by $c^{\dagger}_{\bm{k}n\tau}$ and $c_{\bm{k}n\tau}$ operators.
The color represents the weight of the $O$ orbital. In (a), the solid and
dotted lines, respectively, plot bands from $+K$ and $-K$ valleys. In (b) and
(c), each band is doubly degenerate. (d) The charge gap (black line) and the
in-plane magnetic moment $M_{x}^{A}$ on $A$ sublattice (blue line) in $\nu=2$
ground states as a function of $\theta$. $d$ is 20 nm and $\epsilon$ is 15 for
(a)-(d).

In the small $\theta$ regime of $\theta<1.7^{\circ}$, the energy of the FMz
state is lower than that of the AFx state only by a very small margin, as
shown by the bottom panel of Fig. 6(b). The transition between the FMz and AFx
phases at $\theta\approx 1.7^{\circ}$ is first order. By contrast, the
transition between the AFx and QSHI phases at $\theta\approx 4.5^{\circ}$ is
second order. This is manifested by the continuous suppression of the Néel
order as $\theta$ approaches 4.5∘ from the AFx phase [blue line in Fig. 7(d)].
This phase transition between the AFx and QSHI phases is reminiscent of the
same transition in the Kane-Mele-Hubbard model at a critical onsite Hubbard
interaction [56]. While our realistic model differs from the Kane-Mele-Hubbard
model due to complications such as the additional $O$ orbital and the long-
range Coulomb interaction, the latter model provides important intuitions in
understanding the phase diagram. Mean-field theory underestimates the critical
Hubbard repulsion in the Kane-Mele-Hubbard model compared to that obtained
from quantum Monte Carlo simulation [57]. It is likely that our mean-field
phase diagram also overestimates the tendency towards the formation of the
Néel order, and the QSHI phase could appear in a larger parameter space.

In the QSHI phase, the occupied bands in opposite valleys have opposite Chern
numbers of $\pm 1$. Since the two valleys also act as spin up and down that
are related by the $\mathcal{T}$ symmetry, the QSHI state is characterized by
nontrivial $Z_{2}$ topological invariant. In the single-particle band
structure of the noninteracting moiré Hamiltonian, the first and the second
band overlap in energy for $\theta>3.1^{\circ}$, and the system would then be
a metallic state at $\nu=2$ without taking into account of interaction
effects. However, the QSHI phase of the interacting model has a full charge
gap as indicated by the HF band structure shown in Fig. 7(c). Thus, the
Coulomb repulsion is crucial in opening up the charge gap in the QSHI phase.

When $\epsilon$ is reduced to 10, the corresponding phase diagram in Fig. 6(a)
only has the FMz and AFx phases for $\theta\in(1.5^{\circ},5^{\circ})$. The
stronger Coulomb interaction for $\epsilon=10$ enhances the antiferromagnetic
order and therefore, the QSHI state no longer appears in the phase diagram of
Fig. 6(a). The existence of the QSHI state requires an intermediate
interaction strength, which should be strong enough to open a full charge gap
but weak enough to avoid the instability toward magnetism. This is clearly
shown by the phase diagram as a function of $\theta$ and $\epsilon$ plotted in
Fig. 8.

Figure 8: The quantum phase diagram at $\nu=2$ and $d=20$ nm as a function of
$\theta$ and $\epsilon$. The color map plots the charge gap. The symbol “M”
stands for metallic state.

##  IV Discussion and Conclusion

In summary, we present a systematic study of the quantum phase diagrams of
$t$MoTe2 that result from the interplay between band topology and many-body
interaction. At the single-particle level, we construct a three-orbital model
that can be viewed as a generalization of the two-orbital Kane-Mele model. The
three-orbital model faithfully captures the band dispersion, symmetry, and
topology of the low-energy moiré bands for a large range of $\theta$. For the
interacting physics, we obtain the quantum phase diagrams based on
comprehensive many-body calculations by comparing the energy of multiple
competing states. At $\nu=1$, we find a cascade of phase transitions as
$\theta$ increases. In the small $\theta$ regime, the system is in a layer-
polarized phase with spontaneous imbalanced hole densities on $A$ and $B$
sites. The QAHI phase appears in an intermediate range of $\theta$ and is
superseded by topologically trivial phases (i.e., FMz and $O-120^{\circ}$AF±
phases) in the large $\theta$ regime. At $\nu=2$, the QSHI phase can appear in
the large $\theta$ regime, but undergo an instability towards an
antiferromagnetic state below a critical $\theta$.

The obtained phase diagrams serve as a guide for further experimental and
theoretical studies, but the results should be taken to be qualitative instead
of quantitative. There are several theoretical difficulties in establishing
truly quantitative phase diagrams. First, parameters in the moiré Hamiltonian
are subjected to numerical uncertainties. We take the parameter values from
Ref. [2], where parameters were obtained by fitting to the density-functional-
theory (DFT) band structures of the untwisted homobilayer at different high-
symmetry stackings. The DFT calculations are likely not accurate enough to pin
down the exact values of small quantities such as $V$ and $w$ that are on the
order of 10 meV. Second, mean-field theory can capture the quantum phase
diagram qualitatively but may overestimate the tendency towards symmetry-
breaking states. In spite of these quantitative issues that are generic
challenges for moiré systems, we expect that our study captures the
qualitative features of the phase diagrams.

Figure 9: Quantum phase diagram at $\nu=1$ as a function of $\theta$ and $V$
for $d=20$ nm and $\epsilon=15$. Different phases are separated by blue solid
lines. The blue dashed line is a continuation of the phase boundary between
the FMz and QAHI phases if the $O-120^{\circ}$AF± phase is fully suppressed by
an out-of-plane magnetic field.

Recent experiments on $t$MoTe2 reported optical signatures of a ferromagnetic
correlated insulator in a $\theta=3.9^{\circ}$ sample [21] and a quantum
anomalous Hall state with Chern number of 1 in another $\theta=3.7^{\circ}$
sample [22], where the hole filling factor $\nu$ is 1 in both cases. The
topological character of the $\nu=1$ ferromagnetic insulator in the
$\theta=3.9^{\circ}$ sample was not reported [21]. An independent experiment
reported the quantum anomalous Hall state in $t$MoTe2 with
$\theta=3.4^{\circ}$ at $\nu=1$ [23]. Our $\nu=1$ phase diagram featuring both
the topological ferromagnetic phase (i.e., the QAHI phase ) and the
topologically trivial ferromagnetic phase (i.e., the FMz phase) is
qualitatively consistent with these experimental findings. In the phase
diagram of Fig. 3(b), the QAHI phase appears for $\theta$ in the range of
about $(2^{\circ},2.9^{\circ})$, which is below the experimental twist angle
$3.4^{\circ}$. This discrepancy can be remedied by slightly adjusting the
parameters used in the moiré Hamiltonian. In Fig. 9, we present the $\nu=1$
phase diagram as a function of $\theta$ and $V$, where $V$ is the moiré
potential amplitude as defined in Eq. (1). In the calculation of Fig. 9, we
keep other parameters (except $\theta$ and $V$) of the moiré Hamiltonian in
Eq. (1) fixed, and take $\epsilon=15$ and $d=20$ nm. As shown in Fig. 9, the
$\theta$ range for the QAHI phase is shifted to $(2.35^{\circ},4.05^{\circ})$
if $V$ is increased to 9 meV, which leads to better consistency with the
experimental finding [22]. Increasing the intralayer moiré potential amplitude
$V$ has two effects. First, a deeper moiré potential leads to stronger spatial
confinement of $A$ and $B$ orbitals, and therefore, weaker hopping parameters
between $A$ and $B$ sites. The doped holes then have a stronger tendency to be
localized by Coulomb repulsion, which shifts the phase boundary between the
LP-FMz and QAHI phase to larger twist angles. Second, the deeper intralayer
moiré potential also increases the single-particle energy difference between
$A$/$B$ orbital and $O$ orbital, and effectively pushes the $O$ orbital away
from the Fermi level. Therefore, the topologically trivial FMz phase gradually
shrinks as $V$ increases and finally disappears at $V\approx 8.7$ meV. For
$V>8.7$ meV, there is a first-order phase transition between the QAHI phase
and the $O-120^{\circ}$AF± phase, where the corresponding phase boundary also
shifts towards larger twist angles as $V$ increases. We note that the moiré
potential could in principle be tuned by pressure [58, 59], but the exact
dependence of the model parameters and the phase diagrams on the pressure
requires future investigations.

We comment on the implication of our results for $t$WSe2. Early transport
studies of $t$WSe2 reported topologically trivial correlated insulator at
$\nu=1$ in the twist angle range from $4^{\circ}$ to $5.5^{\circ}$ [18, 19].
Recent local electronic compressibility measurements of $t$WSe2 at $\nu=1$
revealed QAHI state in a narrow twist angle range between $1.2^{\circ}$ and
$1.25^{\circ}$, but topologically trivial correlated insulator for $\theta$
either below or above this range [20]. Qualitatively, our $\nu=1$ phase
diagram with the QAHI phase appearing in an intermediate range of $\theta$ is
consistent with these experimental observations. The $O-120^{\circ}$AF± phase
and the LP phase can capture the topologically trivial correlated insulators
observed at large and small twist angles, respectively. Quantitatively, one
peculiar experimental finding is that the QAHI state in $t$WSe2 only appears
at a very narrow twist angle range [20], which calls for further
investigation.

Signatures of fractional quantum anomalous Hall (FQAH) states at fractional
filling factors were also reported in $t$MoTe2 [22, 23]. Exact diagonalization
studies of the FQAH states have recently been performed, where the
calculations were done in the truncated Hilbert space projected to the first
moiré valence band [60, 61]. Our study highlights the importance of including
multiple bands in the many-body problem. For example, the fully valley
polarized state in the truncated Hilbert space projected to the first moiré
valence bands is always a quantum anomalous Hall state at $\nu=1$, but this is
not the case in the three-orbital model (see more discussion in Appendix C).
It is desirable to examine the energy competition between the FQAH state and
other candidate states (e.g., charge density wave state) at fractional filling
factors in the multi-band model using techniques beyond the mean-field theory.

FQAH states are analogous to fractional quantum Hall states formed in Landau
levels. However, there is a crucial difference between Landau levels and
topological moiré bands in $t$MoTe2, since the latter respects time-reversal
symmetry. The FQAH states realized in $t$MoTe2 spontaneously break the time-
reversal symmetry. Time-reversal symmetric fractional quantum spin Hall
insulators [62, 63], as another distinct type of fractionalized states, may
also emerge in $t$MoTe2 and $t$WSe2.

The obtained quantum phases can have interesting responses to external fields.
For example, an out-of-plane electric displacement field can drive the QAHI
state at $\nu=1$ into a topologically trivial layer polarized state. An out-
of-plane magnetic field turns the $\nu=1$ $O-120^{\circ}$AF± state into a
canted antiferromagnetic state, and finally to a ferromagnetic state for
magnetic field above a critical value, where the ferromagnetic state can be in
the QAHI phase, as shown in Fig. 9. The exploration of the rich quantum states
in $t$MoTe2 is just at its beginning stage.

##  V ACKNOWLEDGMENTS

This work is supported by National Key Research and Development Program of
China (Grants No. 2021YFA1401300 and No. 2022YFA1402401), National Natural
Science Foundation of China (Grant No. 12274333), and start-up funding of
Wuhan University. W.-X. Q. is also supported by Postdoctoral Innovation
Research Position of Hubei Province (Grant No. 211000128). The numerical
calculations in this paper have been done on the supercomputing system in the
Supercomputing Center of Wuhan University.

Note added. Fractionally quantized anomalous Hall effect has recently been
observed in $t$MoTe2 through transport measurement [64, 65].

##  Appendix A $C_{3z}$ symmetry eigenvalues

The single-particle wave function can be viewed as a direct product of the
spatial envelope wave function $\psi_{\bm{k}}^{n\tau}(\bm{r})$, the atomic
state $|d_{\tau}\rangle$, and the spin state $|s_{\tau}\rangle$, where
$|d_{\tau}\rangle=|d_{x^{2}-y^{2}}\rangle+i\tau|d_{xy}\rangle$ stands for the
dominant atomic orbital, and $|s_{\tau}\rangle$ represents spin up and down,
respectively, for $\tau=+$ and $-$. The $C_{3z}$ symmetry acts separately on
the three parts. For the internal degrees of freedom, the $C_{3z}$ symmetry
eigenvalue is given by,

$C_{3z}|d_{\tau}\rangle\otimes|s_{\tau}\rangle=e^{i2\pi
l_{\tau}/3}|d_{\tau}\rangle\otimes|s_{\tau}\rangle,$ (12)

where $l_{\tau}=-\tau/2$ is the angular momentum contributed by the spin and
atomic orbital.

We now study the $C_{3z}$ symmetry property of the spatial wave function using
the approach developed in Ref. 44. We first apply the following unitary
transformation to $\mathcal{H}_{\tau}$ to make the $C_{3z}$ symmetry
transparent,

$\displaystyle\tilde{\mathcal{H}}_{\tau}(\bm{r})\equiv\Lambda_{\tau}(\bm{r})\mathcal{H}_{\tau}(\bm{r})\Lambda_{\tau}^{-1}(\bm{r}),$
(13)
$\displaystyle\Lambda_{\tau}(\bm{r})=\begin{pmatrix}e^{-i\tau\bm{\kappa}_{+}\bm{\cdot}\bm{r}}&0\\\
0&e^{-i\tau\bm{\kappa}_{-}\bm{\cdot}\bm{r}}\end{pmatrix},$
$\displaystyle\tilde{\mathcal{H}}_{\tau}(\bm{r})=\begin{pmatrix}-\frac{\hbar^{2}\bm{k}^{2}}{2m^{*}}+\Delta_{+}(\bm{r})&\tilde{\Delta}_{\mathrm{T,\tau}}(\bm{r})\\\
\tilde{\Delta}_{\mathrm{T,\tau}}^{\dagger}(\bm{r})&-\frac{\hbar^{2}\bm{k}^{2}}{2m^{*}}+\Delta_{-}(\bm{r})\end{pmatrix},$

where $\tilde{\Delta}_{\mathrm{T,\tau}}(\bm{r})=w(e^{i\tau\bm{q_{1}\cdot
r}}+e^{i\tau\bm{q_{2}}\bm{\cdot}\bm{r}}+e^{i\tau\bm{q}_{3}\bm{\cdot}\bm{r}})$.
Here
$\bm{q}_{1}=\bm{\kappa}_{-}-\bm{\kappa}_{+},\bm{q}_{2}=\hat{R}_{3}\bm{q}_{1}$,
$\bm{q}_{3}=\hat{R}_{3}\bm{q}_{2}$, and $\hat{R}_{3}$ is the real-space
anticlockwise rotation by $2\pi/3$. The new Hamiltonian
$\tilde{\mathcal{H}}_{\tau}(\bm{r})$ is clearly invariant under the threefold
rotation of $\bm{r}$ since
$\tilde{\mathcal{H}}_{\tau}(\hat{R}_{3}\bm{r})=\tilde{\mathcal{H}}_{\tau}(\bm{r})$.

The $C_{3z}$ symmetry of the original Hamiltonian $\mathcal{H}_{\tau}$ can be
represented as
$\hat{C}_{3z,\tau}=\Lambda_{\tau}^{-1}(\bm{r})\hat{R}_{3}\Lambda_{\tau}(\bm{r})$,
which leads to the symmetry identity
$\hat{C}_{3z,\tau}\mathcal{H}_{\tau}(\bm{r})\hat{C}_{3z,\tau}^{-1}=\mathcal{H}_{\tau}(\bm{r})$.
The $C_{3z}$ symmetry acts on the wave function in the following way,

$\displaystyle\hat{C}_{3z,\tau}\psi_{\bm{k}}^{n\tau}(\bm{r})=\Lambda_{\tau}^{-1}(\bm{r})\hat{R}_{3}\tilde{\psi}_{\bm{k}}^{n\tau}(\bm{r}),$
(14)

where
$\tilde{\psi}_{\bm{k}}^{n\tau}(\bm{r})=\Lambda_{\tau}(\bm{r})\psi_{\bm{k}}^{n\tau}(\bm{r})$.

At the three high-symmetry momenta $\bm{\gamma}$ and $\bm{\kappa}_{\pm}$, the
Bloch wave function is an eigenstate of the $C_{3z}$ symmetry,

$\hat{C}_{3z,\tau}{\psi}_{\bm{k}}^{n\tau}(\bm{r})=e^{i2\pi
L_{\bm{k}}^{n\tau}/3}{\psi}_{\bm{k}}^{n\tau}(\bm{r}),$ (15)

where $L_{\bm{k}}^{n\tau}$ is the $C_{3z}$ angular momentum of the spatial
wave function and defined modulo 3. Using Eqs. (14) and (15), we find that

$\displaystyle\tilde{\psi}_{\bm{k}}^{n\tau}(\hat{R}_{3}\bm{r})=e^{i2\pi
L_{\bm{k}}^{n\tau}/3}\tilde{\psi}_{\bm{k}}^{n\tau}(\bm{r}).$ (16)

Therefore, $L_{\bm{k}}^{n\tau}$ can be determined by numerically comparing
$\tilde{\psi}_{\bm{k}}^{n\tau}(\hat{R}_{3}\bm{r})$ and
$\tilde{\psi}_{\bm{k}}^{n\tau}(\bm{r})$. The values of $L_{\bm{k}}^{n\tau}$
for the first three bands at the three high-symmetry momenta are labeled in
Figs. 1(b) and 1(c).

In a system with $C_{3z}$ symmetry [66], the Chern number of a band is
determined by the sum of the $C_{3z}$ angular momentum up to modulo 3,
$[(L_{\bm{\gamma}}^{n\tau}+L_{\bm{\kappa_{+}}}^{n\tau}+L_{\bm{\kappa_{-}}}^{n\tau})-C_{\tau}^{(n)}]$
mod $3=0$. The calculated Chern numbers and $C_{3z}$ angular momentum are
consistent with this condition.

##  Appendix B Physical understanding

In this Appendix, we present analyses that help to deepen understanding of the
$\nu=1$ phase diagram. As explained in the following, we show that (1) the
QAHI state is captured by the Kane-Mele-Hubbard model, (2) the transition from
the LP-FMz to the QAHI state is phenomenologically described by the
Haldane-$V$ model (i.e., spinless Haldane model with nearest-neighbor
repulsion $V$), and (3) the $O-120^{\circ}$AF± state can arise from Fermi
surface instability. We emphasize that these analyses should be taken to be
qualitative instead of quantitative. The quantitative energy competition
between different phases depends crucially on microscopic details and can only
be obtained from detailed numerical calculations.

###  B.1 Kane-Mele-Hubbard model

We first consider the Kane-Mele-Hubbard model, which can be constructed by
retaining only the $A$ and $B$ orbitals and neglecting remote hoppings and
repulsions. The model is given by,

$\displaystyle\hat{\mathcal{H}}_{\textrm{KMH}}$
$\displaystyle=\hat{\mathcal{H}}_{\textrm{KM}}+\hat{\mathcal{H}}_{U},$ (17)
$\displaystyle\hat{\mathcal{H}}_{\textrm{KM}}$ $\displaystyle=\sum_{\langle
ij\rangle s}t_{0}b_{is}^{+}b_{js}+\sum_{\langle\langle ij\rangle\rangle
s}t_{ij}^{s}b_{is}^{+}b_{js}$
$\displaystyle=\sum_{\bm{k}}b_{\bm{k}}^{+}\left[F_{0}\sigma_{0}s_{0}+F_{x}\sigma_{x}s_{0}+F_{y}\sigma_{y}s_{0}+F_{z}\sigma_{z}s_{z}\right]b_{\bm{k}},$
$\displaystyle\hat{\mathcal{H}}_{U}$
$\displaystyle=U\sum_{i}\hat{n}_{i\uparrow}\hat{n}_{i\downarrow},$

where $\hat{\mathcal{H}}_{\textrm{KM}}$ is the Kane-Mele model defined on a
honeycomb lattice, the fermion operators $b_{is}^{+}$ and $b_{is}$ are defined
in the hole basis, the subscript $s$ is the spin index (locked to the valley
index in $t$MoTe2), $t_{0}$ is the nearest neighbor hopping parameter, and
$t_{ij}^{s}$ describes the second nearest neighbor hopping with spin and
sublattice-dependent flux. Here
$b_{\bm{k}}=\\{b_{\bm{k}A\uparrow},b_{\bm{k}A\downarrow},b_{\bm{k}B\uparrow},b_{\bm{k}B\downarrow}\\}^{\mathrm{T}}$,
where the subscripts $A$ and $B$ denote the sublattices in the honeycomb
lattice. $\sigma_{x,y,z}$ and $s_{x,y,z}$ are, respectively, Pauli matrices in
the sublattice and spin spaces, and $F_{0,x,y,z}$ are real functions of the
momentum $\bm{k}$.

Using Hartree-Fock decomposition, the Hubbard term $\hat{\mathcal{H}}_{U}$ can
be approximated as

$\hat{\mathcal{H}}_{U}\approx-\frac{NU}{4}\sum_{\ell}\left[n_{\ell}^{2}-\bm{m}_{\ell}^{2}\right]+\frac{U}{2}\sum_{\bm{k}\ell}b_{\bm{k}\ell}^{+}\left[n_{\ell}s_{0}-\bm{m}_{\ell}\cdot\bm{s}\right]b_{\bm{k}\ell},$
(18)

where the lattice translational symmetry is assumed to be preserved, $N$ is
the number of unit cells,
$n_{\ell}=\left\langle{b}_{\ell}^{+}{b}_{\ell}\right\rangle$,
$\bm{m}_{\ell}=\left\langle b_{\ell}^{+}\bm{s}b_{\ell}\right\rangle$,
$b_{\ell}=\\{b_{\ell\uparrow},b_{\ell\downarrow}\\}^{\mathrm{T}}$, and $\ell$
is the sublattice index.

By diagonalizing the mean-field Hamiltonian, we obtain the total energy at
$\nu=1$ as follows,

$\displaystyle
E_{tot}=\frac{NU}{8}+\frac{2NM^{2}}{U}+\sum_{\bm{k}}F_{0}-\sum_{\bm{k}}\sqrt{F^{2}+M^{2}+\Phi},$
(19)
$\displaystyle\Phi=F_{z}M\left(\cos\theta_{A}-\cos\theta_{B}\right)+\sqrt{2M^{2}\left|F_{\|}\right|^{2}\Phi_{1}+M^{2}F_{z}^{2}\Phi_{2}},$
$\displaystyle\Phi_{1}=1+\cos\theta_{A}\cos\theta_{B}+\sin\theta_{A}\sin\theta_{B}\cos\left(\phi_{A}-\phi_{B}\right),$
$\displaystyle\Phi_{2}=\left(\cos\theta_{A}+\cos\theta_{B}\right)^{2},$

where $\bm{M}_{\ell}=-U\bm{m}_{\ell}/2$, $F^{2}=|F_{\|}|^{2}+F^{2}_{z}$,
$F_{\|}=F_{x}-iF_{y}$, and $\bm{M}_{\ell}$ is parameterized as
$M_{\ell}(\sin\theta_{\ell}\cos\phi_{\ell},\sin\theta_{\ell}\sin\phi_{\ell},\cos\theta_{\ell})$.
In deriving the total energy, we assume that $n_{A}=n_{B}$ and
$M_{A}=M_{B}=M$. It is obvious that $E_{tot}$ is minimized when
$\phi_{A}=\phi_{B}$. By further taking $\partial
E_{tot}/\partial\theta_{\ell}=0$, we find two types of solutions, (1)
$\theta_{A}=\theta_{B}=0$ and (2) $\theta_{A}=0$, $\theta_{B}=\pi$. It can be
shown analytically that
$E_{tot}(\theta_{A}=\theta_{B}=0)<E_{tot}(\theta_{A}=0,\theta_{B}=\pi)$.
Therefore, the Kane-Mele-Hubbard model supports the out-of-plane spin
polarized state at $\nu=1$, which exactly gives rise to the valley polarized
QAHI state in $t$MoTe2. From the expression of $E_{tot}$, we see that the
spin-dependent term $F_{z}\sigma_{z}s_{z}$ in the Kane-Mele model leads to
Ising anisotropy and stabilizes the QAHI state.

Figure 10: Phase diagram of the Haldane-$V$ model. Phase i is the QAHI phase
with $\rho_{z}=0$ and quantized Chern number $|C|=1$. Phase iii is sublattice
polarized with $\rho_{z}\neq 0$ and topologically trivial with $|C|=0$. There
is an intermediate phase ii with $\rho_{z}\neq 0$ but $|C|=1$. Figure 11:
Noninteracting Fermi surface (white dashed line) at half-filling of the
topmost moiré valence bands. The color map shows the energy dispersion in the
first moiré Brillouin zone. The vector $\mathcal{\bm{Q}}$ indicates an
approximate nesting.

###  B.2 Haldane-$V$ model

The spontaneous sublattice (layer) polarization in the LP-FMz phase is driven
primarily by the repulsion between $A$ and $B$ orbitals. To capture this
physics, we focus on states with out-of-plane spin polarization. In this spin
sector, we can study the spinless Haldane model with nearest-neighbor
repulsion $V$, which is expressed as,

$\displaystyle\hat{\mathcal{H}}$
$\displaystyle=\hat{\mathcal{H}}_{\textrm{H}}+\hat{\mathcal{H}}_{V},$ (20)
$\displaystyle\hat{\mathcal{H}}_{\textrm{H}}$ $\displaystyle=\sum_{\langle
ij\rangle}t_{0}b_{i}^{+}b_{j}+\sum_{\langle\langle
ij\rangle\rangle}t_{ij}b_{i}^{+}b_{j}$
$\displaystyle=\sum_{\bm{k}}b_{\bm{k}}^{+}\left[F_{0}\sigma_{0}+F_{x}\sigma_{x}+F_{y}\sigma_{y}+F_{z}\sigma_{z}\right]b_{\bm{k}},$
$\displaystyle\hat{\mathcal{H}}_{V}$ $\displaystyle=V\sum_{\langle
ij\rangle}\hat{n}_{i}\hat{n}_{j},$

where $\hat{\mathcal{H}}_{\textrm{H}}$ is the Haldane model on the honeycomb
lattice and $\hat{\mathcal{H}}_{V}$ describes the nearest-neighbor repulsion.
We note that $\hat{\mathcal{H}}_{\textrm{KM}}$ can be understood as two copies
of $\hat{\mathcal{H}}_{\textrm{H}}$ that are time-reversal partners.

In the Hartree-Fock theory, $\hat{\mathcal{H}}$ can be approximated as,

$\displaystyle\hat{\mathcal{H}}\approx
3NV\left[\rho^{2}-\frac{\nu^{2}}{4}\right]+\sum_{\bm{k}}\left[F_{0}+\frac{3V\nu}{2}\right]b_{\bm{k}}^{+}\sigma_{0}b_{\bm{k}}$
(21)
$\displaystyle+\sum_{\bm{k}}b_{\bm{k}}^{+}\left(\begin{array}[]{cc}F_{z}-3V\rho_{z}&\left(t_{0}-V\rho_{\|}\right)f_{\|}\\\
\left(t_{0}-V\rho_{\|}^{*}\right)f_{\|}^{*}&-F_{z}+3V\rho_{z}\end{array}\right)b_{\bm{k}},$

where $b_{\bm{k}}=\left\\{b_{\bm{k}A},b_{\bm{k}B}\right\\}^{\mathrm{T}}$ and
$f_{\|}=F_{\|}/t_{0}$. We define a vector $\bm{\rho}$ as $\left\langle
b^{+}\bm{\sigma}b\right\rangle/2$, where
$b=\left\\{b_{A},b_{B}\right\\}^{\mathrm{T}}$ for a pair of nearest neighbors.
$\rho_{z}$ is the $z$ component of $\bm{\rho}$ and quantifies the sublattice
polarization. $\rho_{\|}$ is defined as $\rho_{x}-i\rho_{y}$ and quantifies
the inter-sublattice coherence. The nearest-neighbor hopping parameter is
renormalized from the bare value $t_{0}$ to be $t_{0}-V\rho_{\|}$ by the
interaction.

Figure 12: Energy per moiré unit cell of competing states relative to the LUP-
VP states at $\nu=1$, $\epsilon=15$ and $d=20$ nm. The results are obtained
using (a) one-band and (b) two-band models, respectively. Other parameter
values are the same as those used in Fig. 3(b).

The total mean-field energy at $\nu=1$ is,

$\displaystyle E_{tot}\left(\rho_{z},\rho_{\|}\right)$
$\displaystyle=\frac{3}{4}VN+\sum_{\bm{k}}F_{0}+3VN\rho^{2}$ (22)
$\displaystyle-\sum_{\bm{k}}\sqrt{\left(F_{z}-3V\rho_{z}\right)^{2}+\left|\left(t_{0}-V\rho_{\|}\right)f_{\|}\right|^{2}},$

where $\rho^{2}=\left|\rho_{\|}\right|^{2}+\rho_{z}^{2}$. We obtain the
following two self-consistent equations by, respectfully, taking
$\frac{\partial E_{tot}}{\partial\rho_{z}}=0$ and $\frac{\partial
E_{tot}}{\partial\rho_{\|}}=0$,

$\displaystyle\rho_{z}=\frac{1}{2N}\sum_{\bm{k}}\frac{3V\rho_{z}-F_{z}}{\sqrt{\left(F_{z}-3V\rho_{z}\right)^{2}+\left|\left(t_{0}-V\rho_{\|}\right)f_{\|}\right|^{2}}},$
(23)
$\displaystyle\rho_{\|}=\frac{1}{6N}\sum_{\bm{k}}\frac{\left(V\rho_{\|}-t_{0}\right)\left|f_{\|}\right|^{2}}{\sqrt{\left(F_{z}-3V\rho_{z}\right)^{2}+\left|\left(t_{0}-V\rho_{\|}\right)f_{\|}\right|^{2}}}.$

We note that the self-consistent equations always have a symmetric solution
with no sublattice polarization ($\rho_{z}=0$), but it can also have a
symmetry-breaking solution with spontaneous sublattice polarization
($\rho_{z}\neq 0$). By minimizing the total energy, we obtain the phase
diagram [Fig. 10] of the Haldane-$V$ model as a function of $V/t_{0}$ and
$t^{\prime}_{1}/t_{0}$, where $t^{\prime}_{1}$ is the imaginary part of one of
the second nearest neighbor hopping parameters. For a fixed
$t^{\prime}_{1}/t_{0}$, $\rho_{z}$ is 0 (nonzero) for $V$ below (above) a
critical value. Therefore, spontaneous sublattice polarization can be driven
by $V$ above a critical value. This qualitatively explains why the LP-FMz
becomes the ground state at small twist angles, as $V/t_{0}$ increases with
decreasing twist angle. The state with a spontaneous finite $\rho_{z}$ can
also be described as a charge density wave state with imbalanced charge
density at $A$ and $B$ sites. In $t$MoTe2, $A$ and $B$ orbitals have opposite
layer polarization, and therefore, spontaneous sublattice polarization also
leads to spontaneous layer polarization.

###  B.3 Fermi surface instability

At large twist angles, the Coulomb interaction can become less dominant
compared to the single-particle kinetic energy, and the fermiology associated
with the Fermi surface can become important. In Fig. 11, we plot the
noninteracting Fermi surface at half-filling (i.e., $\nu=1$) of the topmost
moiré valence bands for $\theta=4.5^{\circ}$. There is an approximate Fermi
surface nesting, and the nesting vector $\mathcal{\bm{Q}}$ is approximately
equal to $\bm{\kappa}_{-}-\bm{\kappa}_{+}$, which can drive Fermi surface
instability with a $\sqrt{3}\times\sqrt{3}$ real-space texture. This
instability exactly explains the formation of the $O-120^{\circ}$AF± state at
large twist angles.

Figure 13: (a)-(c) The HF band structure for the $\nu=1$ QAHI phase at
$\theta=2.5^{\circ}$ calculated with one-band, two-band, and three-orbital
model, respectively. The solid and dotted lines, respectively, plot bands from
$+K$ and $-K$ valleys. (d) The charge gap of the QAHI phase obtained from one-
band (red), two-band (green) and three-orbital (blue) model for the twist
angle range from $2^{\circ}$ to $2.9^{\circ}$, where the QAHI phase is the
same ground state in all three models. Parameter values are the same as those
used in Fig. 12.

##  Appendix C Comparison of different models

In this Appendix, we present a comparison of results obtained, respectively,
from one-band, two-band, and three-orbital models, as summarized in Fig. 12.
First, in the one-band model [Fig. 12(a)], we project the interacting
Hamiltonian onto the topmost moiré valence bands, where each valley
contributes one band. As shown in Fig. 12(a), the valley polarized state at
$\nu=1$ is always the QAHI in the one-band model. This model fails to capture
the LP-FMz state, the topologically trivial FMz state, and the
$O-120^{\circ}$AF± state. (2) In the two-band model, we project the
interacting Hamiltonian onto the first two moiré valence bands, where each
valley contributes two bands. As shown in Fig. 12(b), the two-band model
captures the LP-FMz state for small twist angles and its transition to the
QAHI state above a critical angle, but fails to describe the topologically
trivial FMz and $O-120^{\circ}$AF± states. Quantitatively, the transition
angle between the LP-FMz and QAHI states is underestimated in the two-band
model compared to the three-orbital model shown in Fig. 3(b). The additional
$O$ orbital in the three-orbital model is crucial in correctly finding all
competing states such the topologically trivial FMz and the $O-120^{\circ}$AF±
states.

The QAHI state is the same ground state for one-band, two-band model, and
three-orbital models for a certain range of twist angles. However, there is a
crucial difference in the size of the charge gap. As shown in Fig. 13, the
one-band and two-band models generally overestimate the charge gap compared to
the three-orbital model. In the three-orbital model, the charge gap has a
charge-transfer nature, since it is determined by electron and hole located at
different real-space orbitals but within the same valley. The one-band model
completely fails to capture this charge-transfer gap, and therefore,
overestimates the charge gap.

We also perform the Hartree-Fock calculation in the plane-wave basis without
projecting onto the first three bands, which leads to quantum phase diagrams
that quantitatively agree with that from the three-orbital model, as we report
in a follow-up work [67]. Therefore, the three-orbital model can be viewed as
a minimal model that provides a unified description of competing phases over a
large range of twist angles.

## References

  * Wu  _et al._ [2018] F. Wu, T. Lovorn, E. Tutuc, and A. H. MacDonald, Hubbard Model Physics in Transition Metal Dichalcogenide Moiré Bands, Phys. Rev. Lett. 121, 026402 (2018).
  * Wu  _et al._ [2019] F. Wu, T. Lovorn, E. Tutuc, I. Martin, and A. H. MacDonald, Topological Insulators in Twisted Transition Metal Dichalcogenide Homobilayers, Phys. Rev. Lett. 122, 086402 (2019).
  * Angeli and MacDonald [2021] M. Angeli and A. H. MacDonald, $\Gamma$ valley transition metal dichalcogenide moiré bands, Proc. Natl. Acad. Sci. U.S.A 118, e2021826118 (2021).
  * Zhang  _et al._ [2021a] Y. Zhang, T. Liu, and L. Fu, Electronic structures, charge transfer, and charge order in twisted transition metal dichalcogenide bilayers, Phys. Rev. B 103, 155142 (2021a).
  * Xian  _et al._ [2021] L. Xian, M. Claassen, D. Kiese, M. M. Scherer, S. Trebst, D. M. Kennes, and A. Rubio, Realization of nearly dispersionless bands with strong orbital anisotropy from destructive interference in twisted bilayer ${\mathrm{MoS}}_{2}$, Nat. Commun. 12, 5644 (2021).
  * Naik and Jain [2018] M. H. Naik and M. Jain, Ultraflatbands and Shear Solitons in Moiré Patterns of Twisted Bilayer Transition Metal Dichalcogenides, Phys. Rev. Lett. 121, 266401 (2018).
  * Xiao  _et al._ [2012] D. Xiao, G.-B. Liu, W. Feng, X. Xu, and W. Yao, Coupled Spin and Valley Physics in Monolayers of ${\mathrm{MoS}}_{2}$ and Other Group-VI Dichalcogenides, Phys. Rev. Lett. 108, 196802 (2012).
  * Yu  _et al._ [2019] H. Yu, M. Chen, and W. Yao, Giant magnetic field from moiré induced Berry phase in homobilayer semiconductors, Nat. Sci. Rev. 7, 12 (2019).
  * Pan  _et al._ [2020a] H. Pan, F. Wu, and S. Das Sarma, Band topology, Hubbard model, Heisenberg model, and Dzyaloshinskii-Moriya interaction in twisted bilayer WSe2, Phys. Rev. Res. 2, 033087 (2020a).
  * Pan  _et al._ [2020b] H. Pan, F. Wu, and S. Das Sarma, Quantum phase diagram of a Moiré-Hubbard model, Phys. Rev. B 102, 201104 (2020b).
  * Devakul  _et al._ [2021] T. Devakul, V. Crépel, Y. Zhang, and L. Fu, Magic in twisted transition metal dichalcogenide bilayers, Nat. Commun. 12, 6730 (2021).
  * Li  _et al._ [2021a] H. Li, U. Kumar, K. Sun, and S.-Z. Lin, Spontaneous fractional Chern insulators in transition metal dichalcogenide moiré superlattices, Phys. Rev. Res. 3, L032070 (2021a).
  * Kaushal  _et al._ [2022] N. Kaushal, N. Morales-Dur$\acute{\textrm{a}}$n, A. H. MacDonald, and E. Dagotto, Magnetic ground states of honeycomb lattice Wigner crystals, Commun. Phys. 5, 289 (2022).
  * Haavisto  _et al._ [2022] M. Haavisto, J. L. Lado, and A. O. Fumega, Topological multiferroic order in twisted transition metal dichalcogenide bilayers, SciPost Phys. 13, 052 (2022).
  * Crépel and Fu [2023] V. Crépel and L. Fu, Anomalous hall metal and fractional chern insulator in twisted transition metal dichalcogenides, Phys. Rev. B 107, L201109 (2023).
  * Abouelkomsan  _et al._ [2022] A. Abouelkomsan, E. J. Bergholtz, and S. Chatterjee, Multiferroicity and Topology in Twisted Transition Metal Dichalcogenides, arXiv:2210.14918 (2022).
  * Kaushal and Dagotto [2023] N. Kaushal and E. Dagotto, Moiré kanamori-hubbard model for transition metal dichalcogenide homobilayers, Phys. Rev. B 107, L201118 (2023).
  * Wang  _et al._ [2020] L. Wang, E.-M. Shih, A. Ghiotto, L. Xian, D. A. Rhodes, C. Tan, M. Claassen, D. M. Kennes, Y. Bai, B. Kim, K. Watanabe, T. Taniguchi, X. Zhu, J. Hone, A. Rubio, A. N. Pasupathy, and C. R. Dean, Correlated electronic phases in twisted bilayer transition metal dichalcogenides, Nat. Mater 19, 861 (2020).
  * Ghiotto  _et al._ [2021] A. Ghiotto, E.-M. Shih, G. S. S. G. Pereira, D. A. Rhodes, B. Kim, J. Zang, A. J. Millis, K. Watanabe, T. Taniguchi, J. C. Hone, L. Wang, C. R. Dean, and A. N. Pasupathy, Quantum criticality in twisted transition metal dichalcogenides, Nature 597, 345 (2021).
  * Foutty  _et al._ [2023] B. A. Foutty, C. R. Kometter, T. Devakul, A. P. Reddy, K. Watanabe, T. Taniguchi, L. Fu, and B. E. Feldman, Mapping twist-tuned multi-band topology in bilayer WSe2, arXiv:2304.09808 (2023).
  * Anderson  _et al._ [2023] E. Anderson, F.-R. Fan, J. Cai, W. Holtzmann, T. Taniguchi, K. Watanabe, D. Xiao, W. Yao, and X. Xu, Programming correlated magnetic states with gate-controlled moiré geometry, Science 381, 325 (2023).
  * Cai  _et al._ [2023] J. Cai, E. Anderson, C. Wang, X. Zhang, X. Liu, W. Holtzmann, Y. Zhang, F. Fan, T. Taniguchi, K. Watanabe, Y. Ran, T. Cao, L. Fu, D. Xiao, W. Yao, and X. Xu, Signatures of fractional quantum anomalous Hall states in twisted ${\text{MoTe}}_{2}$, Nature 622, 63 (2023).
  * Zeng  _et al._ [2023] Y. Zeng, Z. Xia, K. Kang, J. Zhu, P. Kn$\ddot{\textrm{u}}$ppel, C. Vaswani, K. Watanabe, T. Taniguchi, K. F. Mak, and J. Shan, Thermodynamic evidence of fractional Chern insulator in moiré MoTe2, Nature 622, 69 (2023).
  * Tang  _et al._ [2020] Y. Tang, L. Li, T. Li, Y. Xu, S. Liu, K. Barmak, K. Watanabe, T. Taniguchi, A. H. MacDonald, J. Shan, and K. F. Mak, Simulation of Hubbard model physics in WSe2/WS2 moiré superlattices, Nature 579, 353 (2020).
  * Regan  _et al._ [2020] E. C. Regan, D. Wang, C. Jin, M. I. Bakti Utama, B. Gao, X. Wei, S. Zhao, W. Zhao, Z. Zhang, K. Yumigeta, M. Blei, J. D. Carlström, K. Watanabe, T. Taniguchi, S. Tongay, M. Crommie, A. Zettl, and F. Wang, Mott and generalized Wigner crystal states in WSe2/WS2 moiré superlattices, Nature 579, 359 (2020).
  * Xu  _et al._ [2020] Y. Xu, S. Liu, D. A. Rhodes, K. Watanabe, T. Taniguchi, J. Hone, V. Elser, K. F. Mak, and J. Shan, Correlated insulating states at fractional fillings of moiré superlattices, Nature 587, 214 (2020).
  * Li  _et al._ [2021b] H. Li, S. Li, E. C. Regan, D. Wang, W. Zhao, S. Kahn, K. Yumigeta, M. Blei, T. Taniguchi, K. Watanabe, S. Tongay, A. Zettl, M. F. Crommie, and F. Wang, Imaging two-dimensional generalized Wigner crystals, Nature 597, 650 (2021b).
  * Li  _et al._ [2021c] T. Li, S. Jiang, L. Li, Y. Zhang, K. Kang, J. Zhu, K. Watanabe, T. Taniguchi, D. Chowdhury, L. Fu, J. Shan, and K. F. Mak, Continuous Mott transition in semiconductor moiré superlattices, Nature 597, 350 (2021c).
  * Zhang  _et al._ [2022] M. Zhang, X. Zhao, K. Watanabe, T. Taniguchi, Z. Zhu, F. Wu, Y. Li, and Y. Xu, Tuning Quantum Phase Transitions at Half Filling in $3\mathrm{L}\text{$-$}{\mathrm{MoTe}}_{2}/{\mathrm{WSe}}_{2}$ Moiré Superlattices, Phys. Rev. X 12, 041015 (2022).
  * Zhao  _et al._ [2023] W. Zhao, B. Shen, Z. Tao, Z. Han, K. Kang, K. Watanabe, T. Taniguchi, K. F. Mak, and J. Shan, Gate-tunable heavy fermions in a moiré Kondo lattice, Nature 616, 61 (2023).
  * Zhang  _et al._ [2021b] Y. Zhang, T. Devakul, and L. Fu, Spin-textured Chern bands in AB-stacked transition metal dichalcogenide bilayers, Proc. Natl. Acad. Sci. U.S.A. 118, e2112673118 (2021b).
  * Li  _et al._ [2021d] T. Li, S. Jiang, B. Shen, Y. Zhang, L. Li, Z. Tao, T. Devakul, K. Watanabe, T. Taniguchi, L. Fu, J. Shan, and K. F. Mak, Quantum anomalous Hall effect from intertwined moiré bands, Nature 600, 641 (2021d).
  * Zhao  _et al._ [2022] W. Zhao, K. Kang, L. Li, C. Tschirhart, E. Redekop, K. Watanabe, T. Taniguchi, A. Young, J. Shan, and K. F. Mak, Realization of the Haldane Chern insulator in a moiré lattice, arXiv:2207.02312 (2022).
  * Tao  _et al._ [2022] Z. Tao, B. Shen, S. Jiang, T. Li, L. Li, L. Ma, W. Zhao, J. Hu, K. Pistunova, K. Watanabe, T. Taniguchi, T. F. Heinz, K. F. Mak, and J. Shan, Valley-coherent quantum anomalous Hall state in AB-stacked MoTe2/WSe2 bilayers, arXiv:2208.07452 (2022).
  * Pan  _et al._ [2022] H. Pan, M. Xie, F. Wu, and S. Das Sarma, Topological Phases in AB-Stacked MoTe2/WSe2 : $\mathbb{Z}_{2}$ Topological Insulators, Chern Insulators, and Topological Charge Density Waves, Phys. Rev. Lett. 129, 056804 (2022).
  * Xie  _et al._ [2022] Y.-M. Xie, C.-P. Zhang, J.-X. Hu, K. F. Mak, and K. T. Law, Valley-Polarized Quantum Anomalous Hall State in Moiré ${\text{MoTe}}_{2}/{\text{WSe}}_{2}$ Heterobilayers, Phys. Rev. Lett. 128, 026402 (2022).
  * Devakul and Fu [2022] T. Devakul and L. Fu, Quantum Anomalous Hall Effect from Inverted Charge Transfer Gap, Phys. Rev. X 12, 021031 (2022).
  * Chang and Chang [2022] Y.-W. Chang and Y.-C. Chang, Quantum anomalous Hall effect and electric-field-induced topological phase transition in AB-stacked ${\mathrm{MoTe}}_{2}/{\mathrm{WSe}}_{2}$ moiré heterobilayers, Phys. Rev. B 106, 245412 (2022).
  * Dong and Zhang [2023] Z. Dong and Y.-H. Zhang, Excitonic Chern insulator and kinetic ferromagnetism in a ${\mathrm{MoTe}}_{2}/{\mathrm{WSe}}_{2}$ moiré bilayer, Phys. Rev. B 107, L081101 (2023).
  * Rademaker [2022] L. Rademaker, Spin-orbit coupling in transition metal dichalcogenide heterobilayer flat bands, Phys. Rev. B 105, 195428 (2022).
  * Guerci  _et al._ [2023] D. Guerci, J. Wang, J. Zang, J. Cano, J. H. Pixley, and A. Millis, Chiral Kondo lattice in doped MoTe2/WSe2 bilayers, Sci. Adv 9, eade7701 (2023).
  * Xie  _et al._ [2023] M. Xie, H. Pan, F. Wu, and S. Das Sarma, Nematic excitonic insulator in transition metal dichalcogenide moiré heterobilayers, Phys. Rev. Lett. 131, 046402 (2023).
  * Xie  _et al._ [2022] Y.-M. Xie, C.-P. Zhang, and K. T. Law, Topological $p_{x}+ip_{y}$ inter-valley coherent state in Moiré MoTe2/WSe2 heterobilayers, arXiv:2206.11666 (2022).
  * Luo  _et al._ [2023] X.-J. Luo, M. Wang, and F. Wu, Symmetric Wannier states and tight-binding model for quantum spin Hall bands in $AB$-stacked ${\text{MoTe}}_{2}/{\text{WSe}}_{2}$, Phys. Rev. B 107, 235127 (2023).
  * Haldane [1988] F. D. M. Haldane, Model for a Quantum Hall Effect without Landau Levels: Condensed-Matter Realization of the “Parity Anomaly”, Phys. Rev. Lett. 61, 2015 (1988).
  * Kane and Mele [2005a] C. L. Kane and E. J. Mele, ${Z}_{2}$ Topological Order and the Quantum Spin Hall Effect, Phys. Rev. Lett. 95, 146802 (2005a).
  * Kane and Mele [2005b] C. L. Kane and E. J. Mele, Quantum Spin Hall Effect in Graphene, Phys. Rev. Lett. 95, 226801 (2005b).
  * Zhang and Senthil [2019] Y.-H. Zhang and T. Senthil, Bridging Hubbard model physics and quantum Hall physics in trilayer $\text{graphene}/h-\mathrm{BN}$ moiré superlattice, Phys. Rev. B 99, 205150 (2019).
  * Hu and MacDonald [2021] N. C. Hu and A. H. MacDonald, Competing magnetic states in transition metal dichalcogenide moiré materials, Phys. Rev. B 104, 214403 (2021).
  * Morales-Durán  _et al._ [2022] N. Morales-Durán, N. C. Hu, P. Potasz, and A. H. MacDonald, Nonlocal Interactions in Moiré Hubbard Systems, Phys. Rev. Lett. 128, 217202 (2022).
  * Bultinck  _et al._ [2020] N. Bultinck, E. Khalaf, S. Liu, S. Chatterjee, A. Vishwanath, and M. P. Zaletel, Ground state and hidden symmetry of magic-angle graphene at even integer filling, Phys. Rev. X 10, 031034 (2020).
  * Cao  _et al._ [2018a] Y. Cao, V. Fatemi, S. Fang, K. Watanabe, T. Taniguchi, E. Kaxiras, and P. Jarillo-Herrero, Unconventional superconductivity in magic-angle graphene superlattices, Nature 556, 43 (2018a).
  * Cao  _et al._ [2018b] Y. Cao, V. Fatemi, A. Demir, S. Fang, S. L. Tomarken, J. Y. Luo, J. D. Sanchez-Yamagishi, K. Watanabe, T. Taniguchi, E. Kaxiras, R. C. Ashoori, and P. Jarillo-Herrero, Correlated insulator behaviour at half-filling in magic-angle graphene superlattices, Nature 556, 80 (2018b).
  * Zhang  _et al._ [2011] F. Zhang, J. Jung, G. A. Fiete, Q. Niu, and A. H. MacDonald, Spontaneous quantum hall states in chirally stacked few-layer graphene systems, Phys. Rev. Lett. 106, 156801 (2011).
  * Geisenhof  _et al._ [2021] F. R. Geisenhof, F. Winterer, A. M. Seiler, J. Lenz, T. Xu, F. Zhang, and R. T. Weitz, Quantum anomalous hall octet driven by orbital magnetism in bilayer graphene, Nature 598, 53 (2021).
  * Hohenadler  _et al._ [2011] M. Hohenadler, T. C. Lang, and F. F. Assaad, Correlation Effects in Quantum Spin-Hall Insulators: A Quantum Monte Carlo Study, Phys. Rev. Lett. 106, 100403 (2011).
  * Hohenadler  _et al._ [2012] M. Hohenadler, Z. Y. Meng, T. C. Lang, S. Wessel, A. Muramatsu, and F. F. Assaad, Quantum phase transitions in the Kane-Mele-Hubbard model, Phys. Rev. B 85, 115132 (2012).
  * Yankowitz  _et al._ [2019] M. Yankowitz, S. Chen, H. Polshyn, Y. Zhang, K. Watanabe, T. Taniguchi, D. Graf, A. F. Young, and C. R. Dean, Tuning superconductivity in twisted bilayer graphene, Science 363, 1059 (2019).
  * Morales-Durán  _et al._ [2023] N. Morales-Durán, J. Wang, G. R. Schleder, M. Angeli, Z. Zhu, E. Kaxiras, C. Repellin, and J. Cano, Pressure-enhanced fractional chern insulators along a magic line in moiré transition metal dichalcogenides, Phys. Rev. Res. 5, L032022 (2023).
  * Wang  _et al._ [2023] C. Wang, X.-W. Zhang, X. Liu, Y. He, X. Xu, Y. Ran, T. Cao, and D. Xiao, Fractional Chern Insulator in Twisted Bilayer MoTe2, arXiv:2304.11864 (2023).
  * Reddy  _et al._ [2023] A. P. Reddy, F. Alsallom, Y. Zhang, T. Devakul, and L. Fu, Fractional quantum anomalous hall states in twisted bilayer ${\mathrm{mote}}_{2}$ and ${\mathrm{wse}}_{2}$, Phys. Rev. B 108, 085117 (2023).
  * Levin and Stern [2009] M. Levin and A. Stern, Fractional Topological Insulators, Phys. Rev. Lett. 103, 196803 (2009).
  * Qi [2011] X.-L. Qi, Generic wave-function description of fractional quantum anomalous hall states and fractional topological insulators, Phys. Rev. Lett. 107, 126803 (2011).
  * Park  _et al._ [2023] H. Park, J. Cai, E. Anderson, Y. Zhang, J. Zhu, X. Liu, C. Wang, W. Holtzmann, C. Hu, Z. Liu, T. Taniguchi, K. Watanabe, J.-H. Chu, T. Cao, L. Fu, W. Yao, C.-Z. Chang, D. Cobden, D. Xiao, and X. Xu, Observation of fractionally quantized anomalous Hall effect, Nature 622, 74 (2023).
  * Xu  _et al._ [2023] F. Xu, Z. Sun, T. Jia, C. Liu, C. Xu, C. Li, Y. Gu, K. Watanabe, T. Taniguchi, B. Tong, J. Jia, Z. Shi, S. Jiang, Y. Zhang, X. Liu, and T. Li, Observation of Integer and Fractional Quantum Anomalous Hall Effects in Twisted Bilayer ${\mathrm{MoTe}}_{2}$, Phys. Rev. X 13, 031037 (2023).
  * Fang  _et al._ [2012] C. Fang, M. J. Gilbert, and B. A. Bernevig, Bulk topological invariants in noninteracting point group symmetric insulators, Phys. Rev. B 86, 115112 (2012).
  * Li  _et al._ [2023] B. Li, W.-X. Qiu, and F. Wu, Electrically tuned topology and magnetism in twisted bilayer MoTe2 at $\nu_{h}=1$, arXiv:2310.02217 (2023).