The following observation is crucial for the proof of the lemma.

Let $\constraint = ((x_1, \ldots, x_k)), R)$ be a constraint in $\instance$
and $t\in R$. If for at least one $(x_i,x_j) \in (\Var(\constraint))^2$ we have
$(x_i, x_j) \in (\Var(\maximal))^2$ and
$\Pi_{i,j} t \in \orbitA$, then for all $(x_i,x_j) \in (\Var(\constraint))^2 \cap (\Var(\maximal))^2$ we have $\Pi_{i,j} t \in \orbitA$.

Suppose there is $\constraint = ((x_1, \ldots, x_k), R)$ with $R \subseteq A^k$,
$(x_{i_1}, x_{j_1}), (x_{i_2}, x_{j_2}) \in (\Var(\maximal))^2$
and $t$ in $R$ satisfying $(t[i_1], t[i_2]) \in \orbitC$ and
$(t[i_1], t[i_2]) \in \orbitA$ with $\orbitA \neq \orbitC$.
It induces a forbidden arc $((x_{i_1}, x_{j_1}),*, \orbitC), (x_{i_2}, x_{j_2}, *, \orbitA))$
in $\graphinstancemaximal$, contradicts the assumption of the lemma and completes the proof of the observation.

Suppose  now that the first item does not hold, i.e.,
there exists a constraint $\constraint' = ((x_1, \ldots, x_k), R')$ in $\instanceJ$, $(x_i,x_j) \in (\Var(\maximal))^2$ and $\orbitC \subseteq \instance_{x_i,x_j} \setminus \orbitA$ such that there is no $t' \in R'$ satisfying $(t'[i], t'[j]) \in \orbitC$.
By the construction of $\instanceJ$, the constraint $\constraint'$
originates from some $\constraint = ((x_1, \ldots, x_k), R)$ in $\instance$.
Since $\instance$ is $(2, \maxboundA)$-minimal and $\instance_{x_i, x_j}$
contains $\orbitA$, the relation $R$ has
a tuple $t$ satisfying $(t[i], t[j]) \in \orbitC$.
Since $t$ is not in $R'$ it satisfies
$(t[m], t[l]) \in \orbitA$ for some other $(x_l,x_m) \in (\Var(\maximal))^2$.
But it contradicts Observation <ref> and completes the proof of the first item in the formulation of the lemma.

Finally, suppose that there is some
$\constraint' = ((x_1, \ldots, x_k), R')$ in $\instanceJ$, $(x_i,x_j) \notin (\Var(\maximal))^2$ and $\orbitC \subseteq \instance_{x_i,x_j}$ such that there is no $t' \in R'$ satisfying $(t'[i], t'[j]) \in \orbitC$.
Again, there is some $\constraint = ((x_1, \ldots, x_k), R)$ in $\instance$ from which $\constraint'$ originates and some $(x_l,x_m) \in (\Var(\maximal))^2 \cap (\Var(\constraint))^2$. In particular, we have that
$((x_l,x_m), \subsetC) \in \maximal$ for some $\subsetC \subsetneq \instance_{x_l, x_m}$. Observe that if
$\subsetC + \Pi_{l,m,i,j}(R) \supseteq \orbitC$
then there is a tuple $t$ in $R$ satisfying $(t[i], t[j]) \in \orbitC$
and $(t[l], t[m]) \notin \orbitA$. Since $t \notin R'$ there must be
$(t[l'], t[m']) \in \orbitA$ for some other $(x_{l'},x_{m'}) \in (\Var(\maximal))^2 \cap (\Var(\constraint))^2$.
But it contradicts Observation <ref> and implies that
$\subsetC + \Pi_{l,m,i,j}(R) \supsetneq \orbitC$. Thus, we have
$(x_i,x_j) \in (\Var(\maximal))^2$ which contradicts the assumption
and completes the proof of the lemma.

We are now ready to prove the main result of the paper.

Proof of Theorem <ref>:
Let the size of an instance of $\csp(\structB)$ be the number of all orbits of tuples of relations in all constraints.
Suppose there is a non-trivial $(2,\maxboundA)$-minimal  instance $\instance$ of $\csp(\structB)$ that has no solution. Assume without loss of generality  that $\instance$ is of minimal size. If there are any $x,y \in \Var(\instance)$
with $\instance_{x,y}$ consisting of at least two orbitals, then
$\graphinstance$ is non-empty, and thereby there is some maximal component $\maximal$ of $\graphinstance$ and by Observation <ref>, a minimal component $\componentmin$ of $\graphinstancemaximal$.
By Lemma <ref> and Lemmas <ref>–<ref>, we have that every vertex in $\componentmin$ is of the form $(*, *, \orbitA)$ for the very same $\orbitA$.
By Observation <ref> for every $((x,y), \subsetA)$ in $\maximal$ the component $\componentmin$ contains a vertex $((x,y), \subsetA, \orbitA)$. Thus, by Lemma <ref>
there is a smaller non-trivial $(2, \maxboundA)$-minimal instance of $\csp(\structB)$ that has no solution.

From now on we assume that for all $(x,y) \in \Var(\instance)$ we have that $\instance_{x,y}$
contains only one orbit $\orbitO_{x,y}$. Observe first that

$x \sim y$ iff $\orbitO_{x,y}$ is $=$

is an equivalence relation. It is clearly reflexive and symmetric. It is also transitive since $\instance$ is $(2, \maxboundA)$-minimal and $\left| \maxboundA \right| \geq 3$.
Also because  $\instance$ is $(2, \maxboundA)$-minimal, there  is an embedding of a finite structure $\structC$ over $\Var(\instance)/\sim$
with $([x]_{\sim}, [y]_{\sim}) \in \orbitA^{\structC}$ iff $\orbitO_{x,y}$ is $\orbitA$ for all $\orbitA$ in $\structA$.$\square$

[1]
Barto, L.
Finitely related algebras in congruence distributive varieties have
near unanimity terms.
Canadian Journal of Mathematics 65, 1 (2013), 3–21.

[2]
Barto, L.
The collapse of the bounded width hierarchy.
J. Log. Comput. 26, 3 (2016), 923–943.

[3]
Barto, L., and Kozik, M.
Congruence distributivity implies bounded width.
SIAM J. Comput. 39, 4 (2009), 1531–1542.

[4]
Barto, L., and Kozik, M.
Constraint satisfaction problems solvable by local consistency
J. ACM 61, 1 (2014), 3:1–3:19.

[5]
Barto, L., and Pinsker, M.
The algebraic dichotomy conjecture for infinite domain constraint
satisfaction problems.
In Proceedings of the 31st Annual ACM/IEEE Symposium on Logic
in Computer Science, LICS '16, New York, NY, USA, July 5-8, 2016 (2016),
M. Grohe, E. Koskinen, and N. Shankar, Eds., ACM, pp. 615–622.

[6]
Bodirsky, M.
Complexity of Infinite-Domain Constraint Satisfaction.
Lecture Notes in Logic. Cambridge University Press, 2021.

[7]
Bodirsky, M., and Dalmau, V.
Datalog and constraint satisfaction with infinite templates.
J. Comput. Syst. Sci. 79, 1 (2013), 79–100.

[8]
Bodirsky, M., and Kára, J.
The complexity of temporal constraint satisfaction problems.
J. ACM 57, 2 (2010), 9:1–9:41.

[9]
Bodirsky, M., Martin, B., Pinsker, M., and Pongrácz, A.
Constraint satisfaction problems for reducts of homogeneous graphs.
SIAM J. Comput. 48, 4 (2019), 1224–1264.

[10]
Bodirsky, M., and Mottet, A.
A dichotomy for first-order reducts of unary structures.
Log. Methods Comput. Sci. 14, 2 (2018).

[11]
Bodirsky, M., and Nesetril, J.
Constraint satisfaction with countable homogeneous templates.
J. Log. Comput. 16, 3 (2006), 359–373.

[12]
Bodirsky, M., and Pinsker, M.
Schaefer's theorem for graphs.
J. ACM 62, 3 (2015), 19:1–19:52.

[13]
Bodirsky, M., and Pinsker, M.
Canonical Functions: a Proof via Topological Dynamics.
Homogeneous Structures, A Workshop in Honour of Norbert Sauer's
70th Birthday, Contributions to Discrete Mathematics 16, 2 (2021), 36–45.

[14]
Bulatov, A. A.
A dichotomy theorem for nonuniform csps.
In 58th IEEE Annual Symposium on Foundations of Computer
Science, FOCS 2017, Berkeley, CA, USA, October 15-17, 2017 (2017),
C. Umans, Ed., IEEE Computer Society, pp. 319–330.

[15]
Bulatov, A. A., Jeavons, P., and Krokhin, A. A.
Classifying the complexity of constraints using finite algebras.
SIAM J. Comput. 34, 3 (2005), 720–742.

[16]
Cherlin, G.
Homogeneous Ordered Graphs, Metrically Homogeneous Graphs, and
Beyond, vol. 2 of Lecture Notes in Logic.
Cambridge University Press, 2022.

[17]
Feder, T., and Vardi, M. Y.
The computational structure of monotone monadic SNP and constraint
satisfaction: A study through datalog and group theory.
SIAM J. Comput. 28, 1 (1998), 57–104.

[18]
Hell, P., and Nešetřil, J.
On the complexity of H-coloring.
Journal of Combinatorial Theory, Series B 48 (1990), 92–110.

[19]
Idziak, P. M., Markovic, P., McKenzie, R., Valeriote, M., and Willard, R.
Tractability and learnability arising from algebras with few
SIAM J. Comput. 39, 7 (2010), 3023–3037.

[20]
Kazda, A., Kozik, M., McKenzie, R., and Moore, M.
Absorption and directed Jónsson terms.
Springer International Publishing, Cham, 2018, pp. 203–220.

[21]
Lachlan, A. H., and Woodrow, R.
Countable ultrahomogeneous undirected graphs.
TAMS 262, 1 (1980), 51–94.

[22]
Mottet, A., Nagy, T., Pinsker, M., and Wrona, M.
Smooth approximations and relational width collapses.
In 48th International Colloquium on Automata, Languages, and
Programming, ICALP 2021, July 12-16, 2021, Glasgow, Scotland (Virtual
Conference) (2021), N. Bansal, E. Merelli, and J. Worrell, Eds., vol. 198
of LIPIcs, Schloss Dagstuhl - Leibniz-Zentrum für Informatik,
pp. 138:1–138:20.

[23]
Mottet, A., and Pinsker, M.
Smooth approximations and csps over finitely bounded homogeneous
In LICS '22: 37th Annual ACM/IEEE Symposium on Logic in
Computer Science, Haifa, Israel, August 2 - 5, 2022 (2022), C. Baier and
D. Fisman, Eds., ACM, pp. 36:1–36:13.

[24]
Mottet, A., Pinsker, M., and Nagy, T.
An order out of nowhere: a new algorithm for infinite-domain csps.
CoRR abs/2301.12977 (2023).

[25]
Pinsker, M.
Current challenges in infinite-domain constraint satisfaction:
Dilemmas of the infinite sheep.
CoRR abs/2203.17182 (2022).

[26]
Schaefer, T. J.
The complexity of satisfiability problems.
In Proceedings of the 10th Annual ACM Symposium on Theory of
Computing, May 1-3, 1978, San Diego, California, USA (1978), R. J.
Lipton, W. A. Burkhard, W. J. Savitch, E. P. Friedman, and A. V. Aho, Eds.,
ACM, pp. 216–226.

[27]
Siggers, M. H.
A strong Mal'cev condition for varieties omitting the unary type.
Algebra Universalis 64, 1 (2010), 15–20.

[28]
Wrona, M.
On the relational width of first-order expansions of finitely bounded
homogeneous binary cores with bounded strict width.
In LICS '20: 35th Annual ACM/IEEE Symposium on Logic in
Computer Science, Saarbrücken, Germany, July 8-11, 2020 (2020),
H. Hermanns, L. Zhang, N. Kobayashi, and D. Miller, Eds., ACM,
pp. 958–971.

[29]
Wrona, M.
Relational width of first-order expansions of homogeneous graphs with
bounded strict width.
In 37th International Symposium on Theoretical Aspects of
Computer Science, STACS 2020, March 10-13, 2020, Montpellier, France
(2020), C. Paul and M. Bläser, Eds., vol. 154 of LIPIcs, Schloss
Dagstuhl - Leibniz-Zentrum für Informatik, pp. 39:1–39:16.

[30]
Zhuk, D.
A proof of the CSP dichotomy conjecture.
J. ACM 67, 5 (2020), 30:1–30:78.

A structure $\structA$ is homogeneous if every isomorphism between two its finite substructures may be extended to an automorphism, it is finitely bounded if there exists a finite set of obstructions so that in order to test if a finite structure $\structC$ embeds into $\structA$ it is enough to check if none of the structures from the obstruction set
embeds into $\structA$. (For precise definitions of all the notions we kindly refer the reader to Section <ref>.)
Furthermore, all finite-signature homogeneous structures are $\omega$-categorical which in particular implies that every relation is a finite set of orbits of tuples wrt. the automorphism group of $\structA$. Thus every such relations has a finite natural  representation. It happens that one can easily construct an infinite structure having all these properties out of a class of finite structures.
A class $\mathscr{C}$ of finite $\tau$-structures is an amalgamation class if for all $\structC_1, \structC_2$ in $\mathscr{C}$ not necessarily over disjoint domains there is $\structC \in \mathscr{C}$, called an amalgam, that embeds both $\structC_1, \structC_2$. The Fraïssé Theorem says that for an amalgamation class $\mathscr{C}$ there exists a unique up to isomorphism structure $\structA$ whose
finite substructures are exactly the structures in $\mathscr{C}$. The structure
$\structA$ is called the Fraïssé limit of $\mathscr{C}$.
In particular, $(\mathbb{Q};<)$  is the Fraïsseé limit of the class of all partial orders. That structure and all its finite-signature reducts a.k.a. temporal languages are among infinite languages of interest in this paper.


# DeeLM: Dependency-enhanced Large Language Model
for Sentence Embeddings

Xianming Li, Jing Li
Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
Corresponding author

###### Abstract

Recent studies have proposed using large language models (LLMs) for sentence
embeddings. However, most existing LLMs are built with an autoregressive
architecture that primarily captures forward dependencies while neglecting
backward dependencies. Previous work has highlighted the importance of
backward dependencies in improving sentence embeddings. To address this issue,
in this paper, we first present quantitative evidence demonstrating the
limited learning of backward dependencies in LLMs. Then, we propose a novel
approach called Dependency-Enhanced Large Language Model (DeeLM) to improve
sentence embeddings. Specifically, we found a turning point in LLMs, where
surpassing specific LLM layers leads to a significant performance drop in the
semantic textual similarity (STS) task. STS is a crucial task for evaluating
sentence embeddings. We then extract the layers after the turning point to
make them bidirectional, allowing for the learning of backward dependencies.
Extensive experiments demonstrate that DeeLM outperforms baselines and
achieves state-of-the-art performance across various STS tasks.

DeeLM: Dependency-enhanced Large Language Model
for Sentence Embeddings

Xianming Li, Jing Li††thanks:  Corresponding author Department of Computing,
The Hong Kong Polytechnic University, Hong Kong SAR
<EMAIL_ADDRESS><EMAIL_ADDRESS>

##  1 Introduction

Sentence embedding learning is a challenging task in the NLP community. It
learns crucial semantic information within texts and can affect the
performance of various downstream tasks, such as text classification (Li et
al., 2021), sentiment analysis (Zhang et al., 2022), semantic matching (Grill
et al., 2020; Lu et al., 2020), clustering (Reimers and Gurevych, 2019), and
question answering (Yue et al., 2021).

Figure 1: Box plot of the sentence-level Spearman correlation on the STS-B
test set. The sentence-level Spearman correlation is calculated by averaging
the Spearman correlations between the pivot token and the remaining tokens in
the sentence.

The field of natural language processing (NLP) research has recently undergone
a significant transformation with the emergence of large language models
(LLMs) such as ChatGPT OpenAI (2022, 2023) and LLaMA Touvron et al. (2023).
These remarkable LLMs have been applied for sentence embeddings (Li and Li,
2023; Jiang et al., 2023). In the study by Jiang et al. (2023), they proposed
a “one word” limitation on LLMs that uses the embedding of a representative
word to represent the entire sentence. Li and Li (2023) proposes angle
optimization to improve the representation of LLMs. However, these works only
consider the use of forward dependencies in mainstream autoregressive LLMs,
neglecting the importance of backward dependencies. Many widely used prior
works (Reimers and Gurevych, 2019; Gao et al., 2021) have employed
bidirectional networks and achieved remarkable performance in STS tasks,
indicating the importance of backward dependencies. Learning sentence
embeddings does not raise concerns about information leakage, making it
possible to incorporate backward dependencies to improve sentence embeddings.
Backward dependencies are important for improving sentence embeddings. For
instance, consider the sentence, I love dogs, but I’m allergic to them. If
only forward dependencies are considered, the word embedding for “dogs” might
have a positive sentiment because its previous word, “love”, is very positive.
However, if the model also considers backward dependencies, it will recognize
the relationship between “dogs” and “allergic”. As a result, the final word
embedding for “dogs” may reflect a relatively neutral sentiment.

How to introduce backward dependencies into autoregressive LLMs for sentence
embeddings? Before delving into this research question, it is important to
provide quantitative evidence, rather than just qualitative explanations, to
demonstrate the lack of backward dependencies in autoregressive LLMs. Previous
studies (Ethayarajh, 2019) have found an intriguing phenomenon: ELMo (Peters
et al., 2018) and BERT (Devlin et al., 2019) exhibit high intra-sentence
similarity, indicating strong word similarities within a sentence. On the
other hand, GPT-2 (Radford et al., 2019), which is based on an autoregressive
architecture, shows low intra-sentence similarity. This phenomenon was
observed, but no explanation was provided. Instead, it was marked as “unclear”
in (Ethayarajh, 2019). In this paper, we first provide a quantitative
explanation for this phenomenon from a dependency perspective. Firstly, we
select the representation of the “CLS” token (pivot token) as the sentence
representation for BERT. For autoregressive LLMs, we choose the representation
of the last token (pivot token) for LLaMA (Touvron et al., 2023) and ChatGLM
(Zeng et al., 2022). We then compute the Spearman correlation between the
pivot token and other tokens within a sentence from the STS-B test set to
analyze the dependencies capturing ability of different models. The sentence-
level Spearman correlation from different models is depicted in Figure 1. It
shows that BERT can learn more dependencies (with higher Spearman correlation)
between the pivot token and the remaining tokens within a sentence than LLaMA
and ChatGLM. Additionally, it is noted that the average sentence-level
Spearman correlation for LLaMA, ChatGLM, and BERT are about $0.17$, $0.15$,
and $0.35$, respectively. Particularly, BERT has approximately twice average
score compared to LLaMA and ChatGLM. This quantitative evidence suggests that
BERT can capture forward and backward dependencies through its bidirectional
attention mechanism. Conversely, autoregressive models like LLaMA and ChatGLM
predominantly capture forward dependencies while hardly learning backward
dependencies.

Here, we will introduce the approaches to incorporate backward dependencies in
LLMs. Li et al. (2023) recently proposed a straightforward approach that
eliminates all causal masks in LLaMA to allow bidirectional encoding and does
not increase model parameters. However, this method is inconsistent because it
involves pretraining with a causal mask and fine-tuning without it. As a
result, this inconsistency impairs the generation ability of LLMs. To address
this issue, we propose a novel dependency-enhanced large language model
(DeeLM) for sentence embeddings. We first conducted a degradation experiment
investigating the relationship between LLM layers (associated with generation
ability) and STS performance. We discovered a turning point where exceeding
specific LLM layers leads to dramatically decreased STS performance. Based on
this finding, we extract the layers after the turning point and make them
bidirectional by removing the causal mask, allowing all tokens to attend to
each other during attention computation. These bidirectional layers can learn
both forward and backward dependencies. Then, the bidirectional layers and the
slimmed LLMs are combined to improve sentence embeddings. Figure 2 provides an
intuitive illustration of the difference in dependencies capturing between
BERT, LLMs, and DeeLM.

Figure 2: The ability to capture dependencies of different models. The
dependency distribution depicts the dependencies distribution between the
pivot token ($\mathbf{p}$ in the figure) and other tokens in texts. BERT and
LLMs pivot tokens are the “CLS” and the last tokens, respectively.

We conducted extensive STS experiments. The experimental results indicate that
DeeLM can achieve state-of-the-art performance on both the standard semantic
textual similarity tasks and the conditional semantic textual similarity task.
This evidence suggests that DeeLM can produce high-quality sentence embeddings
by introducing backward dependencies into LLMs. Furthermore, our transfer
experiments suggest that the sentence embeddings produced by DeeLM are
expressive and can generalize well to other tasks.

In summary, the contributions of this paper are listed as follows:

$\bullet$ We investigate the dependencies within LLMs and provide quantitative
evidence that autoregressive LLMs hardly learn backward dependencies.

$\bullet$ We propose a novel dependency-enhanced large language model, DeeLM,
for improving sentence embeddings.

$\bullet$ Extensive experiments suggest that the proposed model DeeLM can
achieve state-of-the-art performance on various STS tasks. In particular,
there is a $2.24\%$ improvement on the conditional STS task.

##  2 Related Work

There are two main approaches for learning sentence embeddings: unsupervised
and supervised.

#### Unsupervised Approaches

Early studies (Hill et al., 2016; Pagliardini et al., 2018) have indicated the
effectiveness of augmenting word2vec (Mikolov et al., 2013) with n-gram. More
recently, BERT-flow (Li et al., 2020) introduced a flow-based method that maps
BERT embeddings to a standard Gaussian latent space. To improve sentence
embeddings, BERT-whitening (Su et al., 2021) applies the whitening operation
to BERT embeddings. Additionally, many contrastive learning-based methods
(Zhang et al., 2020; Gao et al., 2021; Zhuo et al., 2023) have been proposed
to improve the sentence embedding quality.

#### Supervised Approaches

Many prior studies have proposed the use of supervised datasets to improve
sentence embeddings. Conneau et al. (2017) introduced a method that applies
supervised Natural Language Inference (NLI) data for improving sentence
embeddings. USE (Cer et al., 2018) builds upon the transformer and
incorporates the SNLI (Bowman et al., 2015) dataset to augment unsupervised
training, therefore enhancing sentence embedding. Additionally, SBERT (Reimers
and Gurevych, 2019) enhances sentence embedding by using BERT in a siamese
manner. Jiang et al. (2022, 2023) used contrastive learning with prompt
engineering to improve sentence embeddings. Li and Li (2023) proposed an angle
optimization approach to mitigate the negative effects of the saturation zone
of cosine, thereby improving sentence embeddings.

##  3 Methodology

The inclusion of additional parameters in existing large-scale LLMs, which
already have massive parameter sizes (OpenAI, 2022; Touvron et al., 2023) can
lead to challenges such as GPU memory limitations or the need for smaller
training batch sizes. These factors can significantly complicate and model
fine-tuning and hyperparameter settings. To overcome this issue, we propose a
novel method called DeeLM, which enhances dependencies without introducing new
parameters to the LLMs. DeeLM achieves this by modifying specific layers
within the model instead of increasing new layers. More specifically, it
transforms these layers into bidirectional ones by removing the causal mask.
This modification allows all tokens to attend to each other during attention
computation. The procedure is as follows:

$\mathbf{o}=\mathrm{LLM}^{1:t}(\mathbf{x})+\mathrm{BiLLM^{t+1:n}}(\mathbf{x}),$
(1)

where $\mathbf{x}$ is the input token indices obtained from corresponding LLM
tokenizers. $t$ represents the attention layer of LLMs at the turning point.
The turning point can be determined by the degradation experiment. We will
delineate the degradation experiment in Section 4.3.

The $\mathrm{LLM}^{1:t}(\mathbf{x})$ component of DeeLM retains the generation
ability of LLMs. It is still the autoregressive architecture. The generation
ability is crucial for DeeLM as it utilizes prompt engineering to summarize
the representative word of the sentence and subsequently learn its embedding.
The embedding of the representative word is used as the sentence embeddings.
This approach can help to mitigate the anisotropy problem (Ethayarajh, 2019)
caused by word frequency imbalance Li et al. (2020). This is because the
representative word is usually not a high-frequency word. For example, a
representative word for the sentence I am unhappy because it is raining.
obtained from ChatGPT is “dismal”. It is obviously not a high-frequency word.
Specifically, we adopt the prompt Summarize the sentence sentence in one word:
following Li and Li (2023) to produce the representative word. The one word
limitation in LLMs was introduced by Jiang et al. (2023).

On the other hand, the bidirectional layers $\mathrm{BiLLM}^{t+1:n}$ can
enhance the model’s dependency capturing capability, allowing the model to
learn backward dependencies, which benefits sentence embedding learning. The
overall framework of DeeLM is illustrated in Figure 3.

Figure 3: The framework of DeeLM. The architecture of DeeLM is derived from
LLMs. It begins by identifying the turning point and extracting LLMs’
attention layers beyond this point. These extracted layers are then
transformed into bidirectional layers, allowing all tokens to attend to each
other. The final output of DeeLM is obtained by combining the outputs of the
slimmed LLMs with the extracted bidirectional layers.

##  4 Experiment

###  4.1 Dataset and Evaluation Metrics

We mainly evaluate the quality of sentence embeddings on STS tasks following
baselines. The adopted STS tasks include the standard STS and the Conditional
STS.

#### Standard STS

The Standard STS consists of seven STS tasks, namely: STS 2012-2016 Agirre et
al. (2012, 2013, 2014, 2015, 2016), SICK-R Marelli et al. (2014), and STS-B
Cer et al. (2017). The test sets of these tasks are commonly used for
evaluating sentence embeddings. To ensure a fair comparison, we follow
previous works and train our model using the NLI datasets MNLI (Williams et
al., 2018) and SNLI (Bowman et al., 2015).

#### Conditional STS

A recent study by Deshpande et al. (2023) introduced a challenging task called
Conditional Semantic Textual Similarity (C-STS). The C-STS dataset follows a
supervised format, including training, validation, and test sets. However, it
does not provide ground-truth labels for the test set. To obtain test results,
one must use their official test API. Table 1 illustrates an example of the
C-STS dataset.

Sentence 1 | Sentence 2 | Condition and Similarity
---|---|---
An older man holding a glass of wine while standing between two beautiful ladies. |  A group of people gather around a table with bottles and glasses of wine. | The people’s demeanor: $5$
|  | The number of bottles: $1$
Table 1: An example from the C-STS validation set. Under different conditions,
the same pair of sentences can be classified into high and low similarity. The
similarity scale ranges from 1 (dissimilar) to 5 (similar).

#### Evaluation Metrics

For a fair comparison, we follow previous studies and use Spearman’s
correlation to evaluate the standard STS. We use SentEval Conneau and Kiela
(2018) toolkit to compute Spearman’s correlation and report the results in the
“all” setting following baselines. For the C-STS, we report the Spearman’s
correlation returned by the official C-STS evaluation API.

###  4.2 Training Details

This paper uses the widely used LLaMA2 model Touvron et al. (2023) as the
backbone, specifically the seven billion parameter variant, i.e., LLaMA2-7B.
DeeLM-7B is our proposed model. It stands for the dependency-enhanced
LLaMA2-7B. Very recently, the supervised angle optimization technique (Li and
Li, 2023) has shown impressive results on STS tasks. Since the chosen datasets
are supervised, we use the angle optimization (Li and Li, 2023) to learn
sentence embeddings and follow it to set hyperparameters. Due to the large-
scale parameters of LLMs and our limited resources on GPU, it is difficult to
fine-tune the LLMs in full rank. Thus, we use the LoRA technique (Hu et al.,
2021) to fine-tune the DeeLM-based models.

###  4.3 Degradation Experiment

We first conducted a degradation experiment to evaluate the STS performance of
different attention layers in LLaMA2-7B, which has 32 attention layers, and
LLaMA2-13B, which has 40 attention layers. Specifically, we systematically
removed hidden layers, starting from the last attention layer to the first,
and evaluated the model on the standard STS benchmark. The degradation results
are shown in Figure 4. Since we use prompt engineering to obtain the
representative word of the sentence, the generation ability plays a crucial
role. Typically, generation ability is associated with the number of layers in
LLMs. LLMs with more layers tend to have a stronger generation ability than
those with fewer layers (Kaplan et al., 2020). Interestingly, the results show
a S-shaped curve. This indicates that using too deep or too shallow layers
does not result in good STS performance. In other words, overly strong and
weak generation abilities fail to produce satisfactory STS performance. There
is a turning point for generation ability in STS tasks. Once this turning
point is exceeded, generation ability no longer positively impacts STS
performance; instead, it becomes a negative factor, leading to a dramatic
performance drop in the STS tasks. The figure shows a significant turning
point at layer $31$ in LLaMA2-7B, where the STS result reaches $69.21$.
However, at layer $32$, the result drops to $64.15$. In LLaMA2-13B, we have
identified two distinct turning points. The first turning point occurs at
layer $1$, while the second turning point is observed at layer $39$. Notably,
the performance at layer $1$ is significantly lower compared to layer $39$,
indicating a potential instability. Therefore, the proper turning point should
be considered as layer $39$. LLaMA2-13B tends to have a stronger generation
ability than LLaMA2-7B because of its larger model scale. However, we can see
from the figure that the performance of LLaAM2-7B is even better than that of
LLaMA2-13B. This finding reinforces that there exists a limit to generation
ability in STS tasks. Too strong generation abilities could lead to
unsatisfactory STS performance. Supported by these findings, in this study, we
make the attention layers of LLMs after the turning point bidirectional to
introduce backward dependencies for the STS task.

Figure 4: Degradation results on the standard STS Benchmark. The average
Spearman’s correlations are computed using SentEval (Conneau and Kiela, 2018).

###  4.4 Main Results

#### Standard STS

We compare our proposed model with widely adopted unsupervised and supervised
baselines as follows: 1) The adopted unsupervised models are GloVe (Pennington
et al., 2014) with average pooling, BERT-flow (Li et al., 2020), BERT-
whitening (Su et al., 2021), LLaMA2 (Touvron et al., 2023), and contrastive
learning models including IS-BERT (Zhang et al., 2020), CT-BERT (Carlsson et
al., 2020), SimCSE (Gao et al., 2021), ConSERT (Yan et al., 2021), and DiffCSE
(Chuang et al., 2022); 2) the chosen supervised models include InferSent
(Conneau et al., 2017), USE (Cer et al., 2018), SBERT (Reimers and Gurevych,
2019), PromptCSE (Jiang et al., 2023), AnglE Li and Li (2023), as well as
supervised versions of SimCSE and ConSERT.

The standard STS results are presented in Table 2. We can see that most
supervised models perform better than unsupervised models. We can also see
that the LLM-based models achieve better performance than the BERT-based
models. This might contribute to the larger model size of LLMs. It is noted
that the AnglE-based models outperform the contrastive learning (CSE) based
models. Thus, we choose the AnglE framework for learning sentence embeddings.
It is evident from the table that DeeLM outperforms baseline models, achieving
the best results in 4 out of 7 metrics. Additionally, DeeLM demonstrates
state-of-the-art performance on the standard STS benchmark for the same scale
LLMs, further highlighting the effectiveness of enhanced dependency in
improving sentence embeddings. This evidence provides compelling evidence for
the efficacy of our approach.

Model | STS12 | STS13 | STS14 | STS15 | STS16 | STS-B | SICR-R | Avg.
---|---|---|---|---|---|---|---|---
Unsupervised Models
GloVe (avg.) $\dagger$ | $55.14$ | $70.66$ | $59.73$ | $68.25$ | $63.66$ | $58.02$ | $53.76$ | $61.32$
BERT-flow $\ddagger$ | $58.40$ | $67.10$ | $60.85$ | $75.16$ | $71.22$ | $68.66$ | $64.47$ | $66.55$
BERT-whitening $\ddagger$ | $57.83$ | $66.90$ | $60.90$ | $75.08$ | $71.31$ | $68.24$ | $63.73$ | $66.28$
IS-BERT $\ddagger$ | $56.77$ | $69.24$ | $61.21$ | $75.23$ | $70.16$ | $69.21$ | $64.25$ | $66.58$
CT-BERT $\ddagger$ | $61.63$ | $76.80$ | $68.47$ | $77.50$ | $76.48$ | $74.31$ | $69.19$ | $72.05$
ConSERT-BERT | $64.64$ | $78.49$ | $69.07$ | $79.72$ | $75.95$ | $73.97$ | $67.31$ | $72.74$
DiffCSE-BERT | $72.28$ | $84.43$ | $76.47$ | $83.90$ | $80.54$ | $80.59$ | $71.23$ | $78.49$
SimCSE-BERT | $68.40$ | $82.41$ | $74.38$ | $80.91$ | $78.56$ | $76.85$ | $72.23$ | $76.25$
LLaMA2-7B $\star$ | $50.66$ | $73.32$ | $62.76$ | $67.00$ | $70.98$ | $63.28$ | $67.40$ | $65.06$
Supervised Models
InferSent-GloVe $\dagger$ | $52.86$ | $66.75$ | $62.15$ | $72.77$ | $66.87$ | $68.03$ | $65.65$ | $65.01$
USE $\dagger$ | $64.49$ | $67.80$ | $64.61$ | $76.83$ | $73.18$ | $74.92$ | $76.69$ | $71.22$
ConSERT-BERT | $74.07$ | $83.93$ | $77.05$ | $83.66$ | $78.76$ | $81.36$ | $76.77$ | $79.37$
SBERT $\dagger$ | $70.97$ | $76.53$ | $73.19$ | $79.09$ | $74.30$ | $77.03$ | $72.91$ | $74.89$
SimCSE-BERT | $75.30$ | $84.67$ | $80.19$ | $85.40$ | $80.82$ | $84.25$ | $80.39$ | $81.57$
SimCSE-LLaMA-7B $\star$ | $78.39$ | $89.95$ | $84.80$ | $88.50$ | $86.04$ | $87.86$ | $81.11$ | $85.24$
PromptCSE-LLaMA-7B | $\mathbf{79.16}$ | $90.22$ | $85.40$ | $88.99$ | $86.25$ | $88.37$ | $\mathbf{81.51}$ | $85.70$
AnglE-LLaMA-7B | $79.00$ | $\mathbf{90.56}$ | $85.79$ | $89.43$ | $87.00$ | $88.97$ | $80.94$ | $85.96$
DeeLM-7BAnglE (ours) | $79.01$ | $90.32$ | $\mathbf{85.84}$ | $\mathbf{89.47}$ | $\mathbf{87.18}$ | $\mathbf{89.15}$ | $81.08$ | $\mathbf{86.01}$
Table 2: Sentence embedding performance on STS tasks. We report the Spearman’s
correlation $\rho\times 100$ of the “all” setting computed by SentEval
(Conneau and Kiela, 2018). Results marked with $\dagger$ are obtained from
(Reimers and Gurevych, 2019), while results marked with $\ddagger$ are
retrieved from (Gao et al., 2021). Additionally, results marked with $\star$
are retrieved from (Li and Li, 2023), and AnglE-LLaMA-7B is the previous SOTA
at 7B scale. We refer to the corresponding original papers for the remaining
baselines to obtain their results.

#### Conditional STS

The standard STS results in table 2 show that the improvement achieved using
LLMs is marginal to each other. This suggests that the performance of the
standard STS has probably reached its limit using LLMs. As a result, we also
evaluate our model on other recently introduced challenging datasets for a
comprehensive comparison. In this paper, we explore using the conditional STS
dataset to evaluate the capabilities of our model further. For the conditional
STS task, we designed a prompt to obtain the representative word of the
sentence given condition, as follows: Given the context {condition}, summarize
the sentence {sentence} in one word.", where {condition} and {sentence} are
placeholders for the actual input condition and sentence. The results of our
model, along with the baselines, are shown in Table 3. From the table, we can
see that LLMs such as GPT-3.5 and GPT-4 produce lower results in few-shot
settings than SimCSE with fine-tuning. This illustrates that LLMs need to be
fine-tuned to achieve better results on STS tasks. Note that our model, DeeLM,
outperforms the baselines. Specifically, DeeLM achieves a $1.10\%$ improvement
over LLaMA without enhanced dependency and a $2.24\%$ gain over the previous
SOTA model SimCSE. This evidence suggests that the dependency-enhanced large
language model can improve sentence embeddings.

Model | Spearman’s
---|---
Baselines
Flan-T5XL $\dagger$ | $24.80$
Flan-T5XXL $\dagger$ | $29.20$
Flan-UL2 $\dagger$ | $23.20$
Tk-Instruct3B $\dagger$ | $4.90$
Tk-Instruct11B $\dagger$ | $17.10$
GPT-3.5 $\dagger$ | $15.50$
GPT-4 $\dagger$ | $43.60$
SimCSElarge (previous SOTA) $\dagger$ | $47.50$
Ours
LLaMA-7BAnglE | $48.64$
DeeLM-7BAnglE | $\mathbf{49.74}$
Table 3: Results on the C-STS test set. The metrics are Spearman’s
correlation, which is computed using the official C-STS test API. $\dagger$
denotes results are retrieved from Deshpande et al. (2023). LLM baselines are
evaluated in few-shot settings.

#### Transfer Tasks

We also evaluate our models on the following seven transfer tasks: MR (Pang
and Lee, 2005), CR (Hu and Liu, 2004), SUBJ (Pang and Lee, 2004), MPQA (Wiebe
et al., 2005), SST2 (Socher et al., 2013), TREC (Voorhees and Tice, 2000), and
MRPC (Dolan and Brockett, 2005). Specifically, we obtain the sentence
embeddings of our proposed model DeeLM-7B from the standard STS (in Section
4.1). Then, we use the SentEval (Conneau and Kiela, 2018) toolkit to evaluate
the model performance on the transfer tasks. Following baselines, a logistic
regression classifier is trained on top of sentence embeddings obtained by
different methods. For a fair comparison, we follow default configurations
from SentEval. The results are reported in Table 4. From the table, we can see
that our proposed model DeeLM achieves better results than baselines, 4 out of
7 best results. The average result of DeeLM outperforms baselines. In
comparison to AnglE-LLaMA without dependency enhancement, DeeLM shows a
$0.14\%$ improvement in the average score. This indicates that incorporating
backward dependencies can improve sentence embeddings and ultimately help to
improve the performance of downstream tasks.

Model | MR | CR | SUBJ | MPQA | SST2 | TREC | MRPC | Avg.
---|---|---|---|---|---|---|---|---
GloVe (avg.) $\dagger$ | $77.25$ | $78.30$ | $91.17$ | $87.85$ | $80.18$ | $83.00$ | $72.87$ | $81.52$
Skip-thought (avg.) $\ddagger$ | $76.50$ | $80.10$ | $93.60$ | $87.10$ | $82.00$ | $92.20$ | $73.00$ | $83.50$
Avg. BERT $\dagger$ | $78.66$ | $86.25$ | $94.37$ | $88.66$ | $84.40$ | $92.80$ | $69.54$ | $84.94$
BERT-CLS $\dagger$ | $78.68$ | $84.85$ | $94.21$ | $88.23$ | $84.13$ | $91.40$ | $71.13$ | $84.66$
IS-BERT $\ddagger$ | $81.09$ | $87.18$ | $94.96$ | $88.75$ | $85.96$ | $88.64$ | $74.24$ | $85.83$
SimCSE-RoBERTa $\star$ | $83.37$ | $87.76$ | $95.05$ | $87.16$ | $89.02$ | $90.80$ | $75.13$ | $86.90$
DiffCSE-RoBERTa $\diamondsuit$ | $82.82$ | $88.61$ | $94.32$ | $87.71$ | $88.63$ | $90.40$ | $\mathbf{76.81}$ | $87.04$
PromptEOL-LLaMA-7B $\clubsuit$ | $90.40$ | $92.90$ | $\mathbf{96.88}$ | $91.57$ | $95.11$ | $95.40$ | $75.13$ | $91.06$
AnglE-LLaMA-7B $\spadesuit$ | $91.09$ | $93.54$ | $96.47$ | $91.50$ | $95.28$ | $\mathbf{96.40}$ | $75.36$ | $91.38$
DeeLM-7BAnglE | $\mathbf{91.17}$ | $\mathbf{93.78}$ | $96.37$ | $\mathbf{91.58}$ | $\mathbf{95.99}$ | $95.00$ | $76.75$ | $\mathbf{91.52}$
Table 4: Transfer task results of different sentence embedding models
(measured as accuracy). $\dagger$: results from Reimers and Gurevych (2019);
$\ddagger$: results from Zhang et al. (2020); $\star$: results from Gao et al.
(2021). $\diamondsuit$: results from Chuang et al. (2022); $\clubsuit$:
results from Jiang et al. (2023); $\spadesuit$: results from Li and Li (2023).

###  4.5 Ablation Study

To gain a deeper understanding of DeeLM, we conducted an ablation study
focusing on different turning points. The default turning point, set at
$t=31$, was chosen based on the significant performance drop observed between
layer $31$ and $32$ for LLaMA-7B, as illustrated in Section 4. We tested the
performance when the turning point was set to $32$, which means the
dependencies were not enhanced since LLaMA-7B only has $32$ attention layers.
The results clearly indicate that LLMs with enhanced dependencies perform
better than those without. Additionally, we tested the performance with a
turning point at $27$ since it shows the best results in Figure 4. However,
its performance was lower after fine-tuning than the default turning point.
This discrepancy could be attributed to insufficient generation ability during
DeeLM fine-tuning, or it might be a fluctuating value in Figure 4.
Furthermore, we evaluated the performance with a turning point at $0$. All
attention layers are bidirectional at this point. We can see that its results
are lower than others. This suggests that the generation ability is important
for DeeLM because it uses prompt engineering to produce the representative
word of the sentence. The ablation study verifies the importance of enhancing
dependencies for improving sentence embeddings. It also confirms the guideline
of choosing the turning point. The turning point should be the layer where
exceeding it will lead to a significant performance drop.

Model | Spearman’s Correlation
---|---
DeeLM-7BAnglE | $\mathbf{86.01}$
DeeLM-7B${}^{t=32}_{AnglE}$ | $85.96$
DeeLM-7B${}^{t=27}_{AnglE}$ | $85.86$
DeeLM-7B${}^{t=1}_{AnglE}$ | $85.52$
Table 5: Ablation study of DeeLM. The results are the average Spearman’s
correlation of the standard STS benchmark.

###  4.6 Discussion of Enhanced Dependency

To better understand the changes in dependencies, we visually draw the
sentence-level Spearman correlation box plot for LLaMA-7BAnglE and
DeeLM-7BAnglE in Figure 5. The figure shows that the DeeLM-7BAnglE has a
higher sentence-level Spearman correlation compared to the LLaMA-7BAnglE, with
an average gain of about $0.08$. This suggests that the bidirectional layers
in DeeLM can effectively incorporate backward dependencies.

To further demonstrate the ability of bidirectional layers to introduce
backward dependencies, we plot the average sentence-level Spearman correlation
for different turning points in DeeLM in Figure 6. As observed, a decrease in
the turning point (resulting in an increase in bidirectional layers) leads to
a higher average sentence-level Spearman correlation. This implies that the
model can more effectively capture backward dependencies. However, it is
important to note that the generation ability of DeeLM is also crucial because
it uses prompt engineering, as discussed above. Therefore, a shallower turning
point does not guarantee better performance due to insufficient generation
ability. This is evident in the ablation study, where the results for the
turning point at $31$ are better than for the turning point at $27$.

Figure 5: Sentence-level Spearman correlation box plot of LLaMA-7BAnglE and
DeeLM-7BAnglE on STS-B test set. Figure 6: Average Spearman correlation plot
at sentence level for different turning points of DeeLM.

##  5 Conclusion and Future Work

In this paper, we have presented a novel dependency-enhanced large language
model, called DeeLM, for sentence embeddings. Our research reveals a turning
point in generation ability for STS tasks. Once this turning point is
surpassed, further improvements in generation ability do not positively impact
STS performance. To address this, DeeLM makes the attention layers of LLMs
beyond the turning point bidirectional to learn backward dependencies.
Extensive experiments demonstrate that DeeLM outperforms baselines in various
semantic textual similarity tasks, indicating its effectiveness in improving
sentence embeddings.

While DeeLM has shown competitive performance compared to existing works, its
large model size hinders efficiency. In future work, we aim to explore more
efficient solutions to facilitate the practical application of LLMs-based
models.

##  6 Limitation

DeeLM has two main limitations that need to be addressed. Firstly, the
identification of the turning point currently requires an additional
degradation experiment. In the future, we aim to develop a mechanism that
dynamically learns the turning point during network training, eliminating the
need for such experiments. Secondly, DeeLM suffers from a large number of
parameters, resulting in inefficiency. This poses challenges when applying the
model to real-world applications. To overcome these limitations, in future
work, we will focus on dynamically developing methods to learn the turning
point within the network training process. Additionally, we will explore
strategies to optimize the model’s parameter size, making it more practical
and efficient for real-world applications.

## References

  * Agirre et al. (2015) Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Iñigo Lopez-Gazpio, Montse Maritxalar, Rada Mihalcea, German Rigau, Larraitz Uria, and Janyce Wiebe. 2015.  SemEval-2015 task 2: Semantic textual similarity, English, Spanish and pilot on interpretability.  In _Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015)_ , pages 252–263, Denver, Colorado. Association for Computational Linguistics.
  * Agirre et al. (2014) Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2014.  SemEval-2014 task 10: Multilingual semantic textual similarity.  In _Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014)_ , pages 81–91, Dublin, Ireland. Association for Computational Linguistics.
  * Agirre et al. (2016) Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2016.  SemEval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation.  In _Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)_ , pages 497–511, San Diego, California. Association for Computational Linguistics.
  * Agirre et al. (2012) Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. 2012.  SemEval-2012 task 6: A pilot on semantic textual similarity.  In _*SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012)_ , pages 385–393, Montréal, Canada. Association for Computational Linguistics.
  * Agirre et al. (2013) Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, and Weiwei Guo. 2013.  *SEM 2013 shared task: Semantic textual similarity.  In _Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task: Semantic Textual Similarity_ , pages 32–43, Atlanta, Georgia, USA. Association for Computational Linguistics.
  * Bowman et al. (2015) Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015.  A large annotated corpus for learning natural language inference.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 632–642, Lisbon, Portugal. Association for Computational Linguistics.
  * Carlsson et al. (2020) Fredrik Carlsson, Amaru Cuba Gyllensten, Evangelia Gogoulou, Erik Ylipää Hellqvist, and Magnus Sahlgren. 2020.  Semantic re-tuning with contrastive tension.  In _International conference on learning representations_.
  * Cer et al. (2017) Daniel Cer, Mona Diab, Eneko Agirre, Iñigo Lopez-Gazpio, and Lucia Specia. 2017.  SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation.  In _Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)_ , pages 1–14, Vancouver, Canada. Association for Computational Linguistics.
  * Cer et al. (2018) Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Brian Strope, and Ray Kurzweil. 2018.  Universal sentence encoder for English.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 169–174, Brussels, Belgium. Association for Computational Linguistics.
  * Chuang et al. (2022) Yung-Sung Chuang, Rumen Dangovski, Hongyin Luo, Yang Zhang, Shiyu Chang, Marin Soljacic, Shang-Wen Li, Scott Yih, Yoon Kim, and James Glass. 2022.  DiffCSE: Difference-based contrastive learning for sentence embeddings.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4207–4218, Seattle, United States. Association for Computational Linguistics.
  * Conneau and Kiela (2018) Alexis Conneau and Douwe Kiela. 2018.  SentEval: An evaluation toolkit for universal sentence representations.  In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)_ , Miyazaki, Japan. European Language Resources Association (ELRA).
  * Conneau et al. (2017) Alexis Conneau, Douwe Kiela, Holger Schwenk, Loïc Barrault, and Antoine Bordes. 2017.  Supervised learning of universal sentence representations from natural language inference data.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 670–680, Copenhagen, Denmark. Association for Computational Linguistics.
  * Deshpande et al. (2023) Ameet Deshpande, Carlos E. Jimenez, Howard Chen, Vishvak Murahari, Victoria Graf, Tanmay Rajpurohit, Ashwin Kalyan, Danqi Chen, and Karthik Narasimhan. 2023.  Csts: Conditional semantic textual similarity.  In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Dolan and Brockett (2005) William B. Dolan and Chris Brockett. 2005.  Automatically constructing a corpus of sentential paraphrases.  In _Proceedings of the Third International Workshop on Paraphrasing (IWP2005)_.
  * Ethayarajh (2019) Kawin Ethayarajh. 2019.  How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 55–65, Hong Kong, China. Association for Computational Linguistics.
  * Gao et al. (2021) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021.  Simcse: Simple contrastive learning of sentence embeddings.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 6894–6910. Association for Computational Linguistics.
  * Grill et al. (2020) Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. 2020.  Bootstrap your own latent-a new approach to self-supervised learning.  _Advances in neural information processing systems_ , 33:21271–21284.
  * Hill et al. (2016) Felix Hill, Kyunghyun Cho, and Anna Korhonen. 2016.  Learning distributed representations of sentences from unlabelled data.  In _NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1367–1377. The Association for Computational Linguistics.
  * Hu et al. (2021) Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021.  Lora: Low-rank adaptation of large language models.  _arXiv preprint arXiv:2106.09685_.
  * Hu and Liu (2004) Minqing Hu and Bing Liu. 2004.  Mining and summarizing customer reviews.  In _Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining_ , pages 168–177.
  * Jiang et al. (2023) Ting Jiang, Shaohan Huang, Zhongzhi Luan, Deqing Wang, and Fuzhen Zhuang. 2023.  Scaling sentence embeddings with large language models.  _arXiv preprint arXiv:2307.16645_.
  * Jiang et al. (2022) Ting Jiang, Jian Jiao, Shaohan Huang, Zihan Zhang, Deqing Wang, Fuzhen Zhuang, Furu Wei, Haizhen Huang, Denvy Deng, and Qi Zhang. 2022.  PromptBERT: Improving BERT sentence embeddings with prompts.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 8826–8837, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Kaplan et al. (2020) Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020.  Scaling laws for neural language models.  _arXiv preprint arXiv:2001.08361_.
  * Li et al. (2020) Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020.  On the sentence embeddings from pre-trained language models.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 9119–9130, Online. Association for Computational Linguistics.
  * Li and Li (2023) Xianming Li and Jing Li. 2023.  Angle-optimized text embeddings.  _arXiv preprint arXiv:2309.12871_.
  * Li et al. (2021) Xianming Li, Zongxi Li, Haoran Xie, and Qing Li. 2021.  Merging statistical feature via adaptive gate for improved text classification.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 35, pages 13288–13296.
  * Li et al. (2023) Zongxi Li, Xianming Li, Yuzhang Liu, Haoran Xie, Jing Li, Fu-lee Wang, Qing Li, and Xiaoqin Zhong. 2023.  Label supervised llama finetuning.  _arXiv preprint arXiv:2310.01208_.
  * Lu et al. (2020) Wenpeng Lu, Xu Zhang, Huimin Lu, and Fangfang Li. 2020.  Deep hierarchical encoding model for sentence semantic matching.  _Journal of Visual Communication and Image Representation_ , 71:102794.
  * Marelli et al. (2014) Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014.  A SICK cure for the evaluation of compositional distributional semantic models.  In _Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14)_ , pages 216–223, Reykjavik, Iceland. European Language Resources Association (ELRA).
  * Mikolov et al. (2013) Tomás Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013.  Distributed representations of words and phrases and their compositionality.  In _27th Annual Conference on Neural Information Processing Systems 2013._ , pages 3111–3119.
  * OpenAI (2022) OpenAI. 2022.  Introducing chatgpt.
  * OpenAI (2023) OpenAI. 2023.  GPT-4 technical report.  _CoRR_ , abs/2303.08774.
  * Pagliardini et al. (2018) Matteo Pagliardini, Prakhar Gupta, and Martin Jaggi. 2018.  Unsupervised learning of sentence embeddings using compositional n-gram features.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 528–540, New Orleans, Louisiana. Association for Computational Linguistics.
  * Pang and Lee (2004) Bo Pang and Lillian Lee. 2004.  A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts.  In _Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)_ , pages 271–278, Barcelona, Spain.
  * Pang and Lee (2005) Bo Pang and Lillian Lee. 2005.  Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales.  In _Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05)_ , pages 115–124, Ann Arbor, Michigan. Association for Computational Linguistics.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014.  GloVe: Global vectors for word representation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1532–1543, Doha, Qatar. Association for Computational Linguistics.
  * Peters et al. (2018) Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018.  Deep contextualized word representations.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 2227–2237, New Orleans, Louisiana. Association for Computational Linguistics.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-bert: Sentence embeddings using siamese bert-networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing_ , pages 3980–3990. Association for Computational Linguistics.
  * Socher et al. (2013) Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013.  Recursive deep models for semantic compositionality over a sentiment treebank.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_ , pages 1631–1642, Seattle, Washington, USA. Association for Computational Linguistics.
  * Su et al. (2021) Jianlin Su, Jiarun Cao, Weijie Liu, and Yangyiwen Ou. 2021.  Whitening sentence representations for better semantics and faster retrieval.  _arXiv preprint arXiv:2103.15316_.
  * Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023.  Llama 2: Open foundation and fine-tuned chat models.  _arXiv preprint arXiv:2307.09288_.
  * Voorhees and Tice (2000) Ellen M Voorhees and Dawn M Tice. 2000.  Building a question answering test collection.  In _Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval_ , pages 200–207.
  * Wiebe et al. (2005) Janyce Wiebe, Theresa Wilson, and Claire Cardie. 2005.  Annotating expressions of opinions and emotions in language.  _Language resources and evaluation_ , 39:165–210.
  * Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018.  A broad-coverage challenge corpus for sentence understanding through inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics.
  * Yan et al. (2021) Yuanmeng Yan, Rumei Li, Sirui Wang, Fuzheng Zhang, Wei Wu, and Weiran Xu. 2021.  Consert: A contrastive framework for self-supervised sentence representation transfer.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing_ , pages 5065–5075. Association for Computational Linguistics.
  * Yue et al. (2021) Zhenrui Yue, Bernhard Kratzwald, and Stefan Feuerriegel. 2021.  Contrastive domain adaptation for question answering using limited text corpora.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 9575–9593, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Zeng et al. (2022) Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al. 2022.  Glm-130b: An open bilingual pre-trained model.  _arXiv preprint arXiv:2210.02414_.
  * Zhang et al. (2022) Han Zhang, Zongxi Li, Haoran Xie, Raymond YK Lau, Gary Cheng, Qing Li, and Dian Zhang. 2022.  Leveraging statistical information in fine-grained financial sentiment analysis.  _World Wide Web_ , 25(2):513–531.
  * Zhang et al. (2020) Yan Zhang, Ruidan He, Zuozhu Liu, Kwan Hui Lim, and Lidong Bing. 2020.  An unsupervised sentence embedding method by mutual information maximization.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1601–1610, Online. Association for Computational Linguistics.
  * Zhuo et al. (2023) Wenjie Zhuo, Yifan Sun, Xiaohan Wang, Linchao Zhu, and Yi Yang. 2023.  WhitenedCSE: Whitening-based contrastive learning of sentence embeddings.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 12135–12148, Toronto, Canada. Association for Computational Linguistics.

##  Appendix A Example Appendix

This is an appendix.


††thanks<EMAIL_ADDRESS>
# Reexamining the doped two-legged Hubbard ladders

Yang Shen  Key Laboratory of Artificial Structures and Quantum Control
(Ministry of Education), School of Physics and Astronomy, Shanghai Jiao Tong
University, Shanghai 200240, China  Guang-Ming Zhang  State Key Laboratory of
Low-Dimensional Quantum Physics and Department of Physics, Tsinghua
University, Beijing 100084, China  Frontier Science Center for Quantum
Information, Beijing 100084, China  Mingpu Qin  Key Laboratory of Artificial
Structures and Quantum Control (Ministry of Education), School of Physics and
Astronomy, Shanghai Jiao Tong University, Shanghai 200240, China

###### Abstract

We revisit the ground state of the Hubbard model on 2-legged ladders in this
work. We perform DMRG calculation on large system sizes with large kept states
and perform extrapolation of DMRG results with truncation errors in the
converged region. We find the superconducting correlation exponent $K_{sc}$
extracted from the pair-pair correlation is very sensitive to the position of
the reference bond, reflecting a huge boundary effect on it. By systematically
removing the effects from boundary conditions, finite sizes, and truncation
errors in DMRG, we obtain the most accurate value of $K_{sc}$ and $K_{\rho}$
so far with DMRG. With these exponents, we confirm that the 2-legged Hubbard
model is in the Luther-Emery liquid phase with $K_{sc}\cdot K_{\rho}=1$ from
tiny doping near half-filling to $1/8$ hole doping. When the doping is
increased to $\delta\gtrapprox 1/6$, the behaviors of charge, pairing, and
spin correlations don’t change qualitatively, but the relationship
$K_{sc}\cdot K_{\rho}=1$ is likely to be violated. With the further increase
of the doping to $\delta=1/3$, the quasi long-ranged charge correlation turns
to a true long-ranged charge order and the spin gap is closed, while the pair-
pair correlation still decays algebraically.

##  I Introduction

The Hubbard model Hubbard (1963); Arovas et al. (2022); Qin et al. (2022) is
the prototype lattice model of interacting Fermions and plays a paradigmatic
role in correlated electron physics Dagotto (1994). The connection of the two-
dimensional Hubbard model and the related t-J model to cuprate superconductors
has been debated Himeda et al. (2002); Gull et al. (2013); Darmawan et al.
(2018); Ponsioen et al. (2019); Qin et al. (2020); Chung et al. (2020); Jiang
et al. (2021); Gong et al. (2021); Jiang and Kivelson (2021); Xu et al. (2023)
ever since the discovery of the first cuprate superconductor Bednorz and
Müller (1986). Because there is no rigorous analytic solution to the Hubbard
model beyond one dimension Lieb and Wu (1968), most studies of it rely on
numerical methods LeBlanc et al. (2015).

Quasi-one-dimensional systems, such as 2-legged Hubbard ladders, were
extensively studied in the past Troyer et al. (1996); Balents and Fisher
(1996); Schollwöck (2005). On one hand, the correlations in the 2-legged
Hubbard model can be taken as the precursor to the possible superconducting
and charge density wave (CDW) instabilities in the two-dimensional systems. On
the other hand, 2-legged systems can be accurately solved by Density Matrix
Renormalization Group (DMRG) White (1992, 1993); Schollwöck (2011). Early
analytic and numerical works Troyer et al. (1996); Balents and Fisher (1996);
Schollwöck (2005) demonstrate that the slightly doped ladders can be
categorized into the Luther-Emery liquid phase Luther and Emery (1974);
Haldane (1980), which promises a single gapless charge mode and a gapped spin
mode (labeled as C1S0 in the literature Balents and Fisher (1996); Dolfi et
al. (2015); Lu et al. (2023)).

The Luther-Emery liquid phase is characterized by the algebraic decay of both
charge (with exponent $K_{\rho}$) and pair-pair correlations (with exponent
$K_{sc}$). Moreover, the exponents are related and satisfy the relationship
$K_{sc}\cdot K_{\rho}=1$ Luther and Emery (1974); Dolfi et al. (2015); Lu et
al. (2023). Numerically, whether the relationship $K_{sc}\cdot K_{\rho}=1$
holds was debated with both positive Dolfi et al. (2015) and negative Noack et
al. (1997, 1996) results. $K_{\rho}$ can be usually extracted from the Friedel
oscillations White et al. (2002) in a system with open boundaries. The
difficulty to obtain a reliable $K_{sc}$ lies in the subtlety in the
extraction of the exponent $K_{sc}$ from the pair-pair correlation function,
which usually oscillates with distance under the influence of the modulation
of charge density Noack et al. (1997, 1996); Dolfi et al. (2015). In the
strong-coupling limit of the $t-J$ model with hole doping
$\delta\xrightarrow{}0$, bosonization calculation yields $K_{sc}=1/2$ Schulz
(1999); Siller et al. (2001). Previous DMRG calculations found the decay of
the superconducting correlation function to be much faster than $1/K_{\rho}$
at small doping, hence violates the relationship $K_{sc}\cdot K_{\rho}=1$
Noack et al. (1997, 1996). But later results Dolfi et al. (2015) found the
relationship of the exponents holds. Therefore, it is natural to ask to what
degree, $e.g.$, doping levels, the relationship of the exponents holds in the
2-legged Hubbard model.

Table 1: Summary of the extracted exponents and correlation lengths.
$K_{\rho}$ and $K_{sc}$ are the exponents for the algebraic decay of charge
density and pair-pair correlations. $\xi_{s}$ and $\xi_{G}$ are the
correlation lengths for spin correlation (local magnetization) and single-
particle Green’s function. Quantities not listed in the table are ill-defined.

In this work, we revisit the ground state of the 2-legged Hubbard model. We
study dopings ranging from $1/16$ to as large as $1/3$, trying to figure out
the boundary of the Luther-Emery liquid phase by examining the behaviors of
charge, spin, and pair-pair correlations. We focus on the issue whether the
relationship $K_{sc}\cdot K_{\rho}=1$ holds. We employ DMRG in this work which
can provide extremely accurate results for ladder systems nowadays with the
increase of computational power. We push the bond dimension to as large as
$9500$ (with truncation error at the order of $10^{-7}$) to achieve an
unprecedented accuracy for the Hubbard model on ladders Noack et al. (1997,
1996); Dolfi et al. (2015). More importantly, we study large system sizes and
make sure the extracted exponents are free from finite size effect. We also
vary the position of the reference bond in the calculation of pair-pair
correlation functions to get rid of the effect of open boundaries, which turn
out to have a huge impact on the extracted exponent $K_{sc}$ (the error caused
by the open boundaries could be as large as $50\%$ as will be shown in the
discussion of results). With these careful treatment of effects of boundary
conditions, finite sizes and truncation error in DMRG calculations, we give
the most accurate values of $K_{\rho}$ and $K_{sc}$ so far with which we can
resolve the previous controversy on whether the relationship $K_{sc}\cdot
K_{\rho}=1$ holds and determine the precise phase boundary of the Luther-Emery
liquid phase.

Figure 1: Density profile at $1/8$ doping. The local rung density $n(x)$ for
different kept states $m$ and the extrapolated with truncated errors results
are shown. The length of the system is $L_{x}=192$. The solid line is the
fitting curve using Eq. (2) to extract $K_{\rho}$. The dashed horizontal line
represents the averaged electron density. The inset is the finite-size scaling
of $\delta n(L_{x}/2)$ as a function of the system size $L_{x}$ using Eq. (3).

The main results are summarized in Tab. 1. We focus on the strongly correlated
region with $U/t=8$. From tiny doping near half-filling to $\delta=1/8$,
2-legged Hubbard model is found to be in the Luther-Emery liquid phase with
algebraic decay of pair-pair and charge correlations and exponential decay of
spin correlation. The exponents of charge and pairing correlations also
satisfy the relationship $K_{sc}\cdot K_{\rho}=1$. In this region,
$K_{sc}<K_{\rho}$ which means superconducting is the dominant correlation. The
increase of the hole doping tends to suppress the pairing correlations, and
consequently enhances the charge correlation. By increasing the doping to
$1/6$, the characterization of charge, pairing, and spin correlations don’t
change qualitatively, but the exponents of superconducting and charge
correlations seem to violate the $K_{sc}\cdot K_{\rho}=1$ relationship. The
system also switches to a charge correlation dominating phase with
$K_{\rho}<K_{sc}$. By further increasing doping level, $i.e.$, for
$\delta=1/4$ and $1/3$, $K_{\rho}$ is ill-defined and the charge correlation
is likely to be long-ranged. The spin gap is closed at $1/3$ doping but the
algebraic decay of pair-pair correlation functions remains.

We notice that $K_{\rho}$ in Tab. 1 are consistent with the recently results
obtained from infinite system matrix product states Eberharter et al. (2023).

Table 2: The dependence of the extracted parameters $K_{\rho}$ and $n_{0}$
(using Eq. (2)) on the range of sites used ($\delta=1/8$, $L_{x}$ = 192).

The remaining of the paper is organized as follows. In Sec. II we introduce
the Hubbard model and discuss the calculation details. We then provide results
of different doping levels in Sec. III. We show the long-distance behaviors of
the pairing, charge, spin and single-particle correlations. We extract the
exponents and correlation lengths from them by carefully analyzing the effects
from boundary conditions, finite sizes, and truncation error in DMRG
calculations. We finally summarize this work in Sec. IV.

##  II MODEL AND METHODOLOGY

The Hamiltonian of the Hubbard model is

$\displaystyle\hat{H}=-\sum_{\langle
i,j\rangle,\sigma}t_{ij}\left(\hat{c}_{i\sigma}^{\dagger}\hat{c}_{j\sigma}+h.c.\right)+U\sum_{i}\hat{n}_{i\uparrow}\hat{n}_{i\downarrow}$
(1)

where $\hat{c}_{i\sigma}^{\dagger}(\hat{c}_{j\sigma})$ creates (annihilates)
an electron on site $i=(x_{i},y_{i})$ with spin $\sigma$, and $U$ represents
the on-site Coulomb interaction. We only consider nearest hopping $t$ and set
$U=8t$.
$\hat{n}_{i}=\sum_{\sigma}\hat{c}_{i\sigma}^{\dagger}\hat{c}_{i\sigma}$ is the
electron number operator.

We focus on ladders with width $L_{y}\equiv 2$ and length $L_{x}$, so there
are totally $N=L_{x}\times L_{y}$ lattice sites. The average hole
concentration away from half-filling is defined as $\delta=N_{h}/N$ with
$N_{h}=\sum_{i}(1-\hat{n}_{i})$. We apply a one-site pinning field (an
addition term $h_{m}\cdot\hat{S}_{0z}$ with $h_{m}=0.5$ is added to the
Hamiltonian in Eq. (1)) at the left edge of the ladder, which allows us to
detect the local magnetization $\langle\hat{S}_{i}^{z}\rangle$ instead of the
more demanding correlation functions. We also measure the $d$-wave pair-pair
correlation function defined as
$D(r)=\langle\hat{\Delta}_{i}^{\dagger}\hat{\Delta}_{i+r}\rangle$ with
$\hat{\Delta}_{i}^{\dagger}=\hat{c}_{(i,1),\uparrow}^{\dagger}\hat{c}_{(i,2),\downarrow}^{\dagger}-\hat{c}_{(i,1),\downarrow}^{\dagger}\hat{c}_{(i,2),\uparrow}^{\dagger}$
creating a singlet on the rungs Dolfi et al. (2015); Shen et al. (2022). Other
physical observables are defined wherein they are firstly mentioned.

We employ DMRG method which can provide extremely accurate results for ladder
systems. We push the bond dimension in DMRG calculation up to $m=9500$ with
typical truncation errors $\epsilon$ at the order of $10^{-7}$. Extrapolations
with truncation errors $\epsilon$ are also performed to remove the tiny
residual truncation errors. We study the system as long as $192$ sites to get
rid of the finite size effect. More importantly, when calculating the pair-
pair correlation function, we vary the position of reference bond to ensure
the effect of boundary conditions is absent. It turns out the boundary
conditions have a huge effect (could cause an error as large as $50\%$) on the
extracted value of $K_{sc}$. With the careful treatment of the possible
errors, we provide reliable results on the charge density, pair-pair
correlations and local magnetization.

##  III RESULTS AND DISCUSSION

###  III.1 $1/8$ doping level

Figure 2: Pair-pair correlation and the extracted exponents at $1/8$ doping.
(a) Pair-pair correlation function $D(r)$ for different numbers of kept states
$m$ and the extrapolated result. The length of the system is $L_{x}=192$. The
reference bond is set at the 49th vertical bond. The red and blue lines denote
linear and cosine fits defined in the main text, respectively. Only peaked
values are used in the linear fit. (b) $K_{sc}$ as a function of the reference
bond. Here we only show the results of $K_{sc1}$ from linear fits ($K_{sc2}$
has similar value). The shaded area represents the value of $1/K_{\rho}$ with
error bar.

Figure 3: Local magnetization and single-particle Green’s function at $1/8$
doping. (a) Absolute values of the local magnetization with different lengths.
Only the results from the extrapolation with truncation errors are shown. The
solid line denotes the exponential fit using
$\left|\mathbf{\textit{S}}_{\mathbf{\textit{z}}}\right|\propto\mathrm{e}^{-x_{i}/\xi_{\mathrm{s}}}$
with $\xi_{\mathrm{s}}=45.3(3)$ for $L_{x}=192$. (b) Single-particle Green’s
function with different numbers of kept states $m$ and the extrapolated result
for $L_{x}$ = 192. The reference site is set at (49, 2). Only peaked values
are used in the fits for both the local magnetization and the single-particle
Green’s function.

We start with the 1/8 doping case, which draws dramatic interests because of
the intertwined spin, charge, and pairing correlation near $1/8$ doping in
cuprates Tranquada et al. (1995); Emery et al. (1999). Fig. 1 shows the charge
density profiles of a ladder with length $192$, defined as the rung density
$n(x_{i})=\sum_{y=1}^{L_{y}}\left\langle\hat{n}_{i}(x,y)\right\rangle/L_{y}$
on the ladders. The charge density from DMRG with kept states $m=7000$ to
$m=8500$ and the result from an extrapolation with truncation error are all on
top of each other, indicating the DMRG results are well converged. For the 1/8
hole doping case in Fig. 1, the charge distribution $n(x)$ forms a CDW pattern
with a wavelength $\lambda_{c}=1/\delta=8$. The spatial decay of the CDW
correlation at long distances can be described by a power law with Luttinger
exponent $K_{\rho}$. Previous works White et al. (2002); Dolfi et al. (2015)
show that $K_{\rho}$ can be obtained by fitting the Friedel oscillations
induced by the boundaries of the ladder

$n(x)\approx A\frac{\cos\left(2\pi
N_{h}x/L_{x}+\phi_{1}\right)}{\left[L_{x}\sin\left(\pi
x/L_{x}+\phi_{2}\right)\right]^{K_{\rho}/2}}+n_{0}$ (2)

where $A$ is an amplitude, $\phi_{1}$ and $\phi_{2}$ are phase shifts, and
$n_{0}=1-\delta$ is the averaged charge density (we set $n_{0}$ as a parameter
to be determined in the fit Dolfi et al. (2015) ). To get rid of the boundary
effect, we fit the density by gradually excluding data near the boundaries,
and the results are listed in Tab. 2. We can find an obvious boundary effect
on the extracted exponent $K_{\rho}$. After excluding about a dozen of data
near the boundary, the extracted $K_{\rho}$ converges fast to a value about
$1.02(4)$ and the $R^{2}$ of the fit is very close to 1 (with a deviation in
the order of 0.0001). We also study ladders with lengths 48, 64, 96, and 128
and find the extracted values of $K_{\rho}$ is converged. For comparison, we
can alternatively obtain the parameter $K_{\rho}$ from the finite-size
scaling, because the density at the center of the system scales as Dolfi et
al. (2015)

$\delta n(L_{x}/2)=n(L_{x}/2)-n_{0}\sim{L_{x}}^{-K_{\rho}/2}$ (3)

The advantage of this scheme is that $n(L_{x}/2)$ is least affected by the
boundary effect. The finite-size fit is shown in the inset of Fig. 1. To
calculate $\delta n(L_{x}/2)$, we use the obtained $n_{0}$ from the least-
square fit of the charge density profiles using Eq. (2) Dolfi et al. (2015).
Considering the values from the two fitting procedures, we give an estimation
of $K_{\rho}$ at $1/8$ as $K_{\rho}\approx 1.02(4)$. The values of $K_{\rho}$
for other dopings are estimated in the same way.

Figure 4: Density profiles at (a) $1/12$ and (b) $1/16$ dopings. The local
rung density $n(x)$ for different kept states $m$ and the extrapolated with
truncated errors results are shown. The lengths of the systems are $L_{x}=120$
for $1/12$ doping and $L_{x}=192$ for $1/16$ doping, respectively. The solid
lines are the fitting curves using Eq. (2) to extract $K_{\rho}$. The dashed
horizontal lines represent the averaged electron density. The insets of (a)
and (b) are finite-size scaling of $\delta n(L_{x}/2)$ as a function of the
system size $L_{x}$ using Eq. (3).

The singlet pair-pair correlation function $D(r)$ for the $L_{x}=192$ ladder
is displayed in Fig. 2. The numerical results are carefully extrapolated with
truncation errors in DMRG calculations to remove the finite bond dimension
effect. We find $D(r)$ decays algebraically with oscillations in the period of
CDW’s wavelength $\lambda_{c}$, which makes the extraction of the exponent
$K_{sc}$ hard. Apart from fitting the peaks using $D(r)\propto r^{-K_{sc1}}$
as was typically done before Dolfi et al. (2015); Lu et al. (2023), we also
fit the whole data by including an extra cosine term as
$D(r)\propto\mathrm{A}_{0}r^{-K_{sc2}}+\mathrm{A}_{1}\cos\left(2\pi
N_{h}/L_{x}+\phi\right)r^{-K_{sc2}}$, which can be viewed as a first-order
approximation to the oscillated pair-pair correlation function. We have
checked both fits give the consistent results within error bar. We also
checked the extracted value of $K_{sc}$ is converged with the size studied.

As can be seen from Fig. 2(a), the pair-pair correlations clearly deviate from
the algebraically decay behavior at the long distance near the boundary even
after an extrapolation with truncation error. So to account for this effect,
we exclude the very long-distance data for all the fits of pair-pair
correlations. At the same time, we study long ladders which provide enough
data for the fit process.

In the calculation of the pair-pair correlations, a more important factor is
the choice of the position of the reference bond. Fig. 2(b) shows the
extracted $K_{sc}$ for different reference bonds (only correlations between
vertical bonds are shown, the horizontal bonds have similar results). We find
the boundary has a very large effect on $K_{sc}$. When moving the reference
bond away from the boundary, $K_{sc}$ converges and the converged value
satisfy the relationship $K_{sc}\cdot K_{\rho}=1$ as predicted in the Luther
Emery liquid theory.

Table 3: The dependence of the extracted parameters $K_{\rho}$ and $n_{0}$
(using Eq. (2)) on the range of sites used ($\delta=1/4$, $L_{x}$ = 128).

Figure 5: Results for $1/12$ (upper pane) and $1/16$ (lower pane) dopings. (a)
and (d) show $K_{sc}$ as a function of the reference bond. The lengths of the
systems are $L_{x}=120$ for $\delta=1/12$ and $L_{x}=192$ for $\delta=1/16$.
The shaded area represents the value of $1/K_{\rho}$ with error bar. (b) and
(e) show the absolute values of the local magnetization at $1/12$ and $1/16$
dopings with different lengths. Only the results from the extrapolation with
truncation errors are shown. The solid lines denote exponential fits using
$\left|\mathbf{\textit{S}}_{\mathbf{\textit{z}}}\right|\propto\mathrm{e}^{-x_{i}/\xi_{\mathrm{s}}}$
with $\xi_{\mathrm{s}}=28.9(2)$ for $\delta=1/12$($L_{x}=120$) and
$\xi_{\mathrm{s}}=22.26(7)$ for $\delta=1/16$($L_{x}=192$). (c) and (f) show
the single-particle Green’s function for different numbers of kept states $m$
and the extrapolated result. The reference site is set at (31, 2) for
$\delta=1/12$ ($L_{x}=120$) and (49, 2) for $\delta=1/16$ ($L_{x}=192$). Only
peaked values are used in the fits for both the local magnetization and the
single-particle Green’s function.

Fig. 3(a) shows the absolute value of local magnetization for systems with
different lengths. We find $\left|S_{z}\right|$ decays exponentially as
$\left|S_{z}\right|\propto\mathrm{e}^{-x_{i}/\xi_{\mathrm{s}}}$ at long
distances with a finite correlation length $\xi_{\mathrm{s}}=45.3(3)$,
indicating the existence of a finite spin gap in the system which is
consistent with the prediction of the Luther Emery liquid phase. Moreover, the
staggered spin density $(-1)^{x_{i}}\langle\hat{S}_{i}^{z}\rangle$ (not shown)
has a spatial modulation with a wavelength twice that of the hole density and
shows a $\pi$ phase flip at the hole concentrated position. The spin and hole
modulations are consistent with the stripe phase Tranquada et al. (1995);
Emery et al. (1999).

The single-particle Green’s function is defined as
$G_{\sigma}(r)=\langle\hat{c}_{(x_{0},y),\sigma}^{\dagger}\hat{c}_{(x_{0}+r,y),\sigma}\rangle$
with $\sigma$ the spin index. We find $G_{\sigma}(r)$ is also sensitive to the
position of the reference bond similar as the pair-pair correlations. We only
show the converged results in this work. The results for $1/8$ doping is shown
in Fig. 3(b). $G_{\sigma}(r)$ decays exponentially as
$\left|G_{\sigma}(r)\right|\propto\mathrm{e}^{-r/\xi_{\mathrm{G}}}$ with the
correlation length $\xi_{\mathrm{G}}=26.8(2)$.

From the above analysis of charge, pairing correlation, local magnetization,
and single-particle Green’s function, we conclude that the two-legged ladder
at $1/8$ doping with $U=8$ belongs to the Luther-Emery liquid phase and the
exponents of charge and superconducting correlations satisfy the relationship
$K_{sc}\cdot K_{\rho}=1$.

###  III.2 $1/12$ and $1/16$ dopings

We perform similar calculations and analyses for $1/12$ and $1/16$ dopings in
this subsection. The charge density profiles for 1/12 and 1/16 dopings are
shown in Figs. 4(a) and (b), which also display the Friedel oscillations with
$\lambda_{c}=1/\delta$. We find the amplitude of charge density oscillations
is reduced with the decrease of doping level. The exponents increases from
$K_{\rho}=1.13(3)$ to $K_{\rho}=1.28(3)$ from $\delta=1/12$ to $\delta=1/16$
(see the insets of Fig. 4 and Tab. 1). This behavior is consistent of the
prediction that $K_{\rho}\rightarrow 2$ with $\delta\rightarrow 0$ Schulz
(1999); Siller et al. (2001).

Following the fitting procedures in the 1/8 doping case, we also extract
$K_{sc}$ form the pair-pair correlations by varying the position of reference
bond as shown in Figs. 5(a) and 5(d). The converged values for $K_{sc}$ are
$0.95(3)$ and $0.85(5)$ for $1/12$ and $1/16$ dopings respectively. For these
dopings, $K_{sc}<K_{\rho}$ indicating superconducting correlation is dominant
at low dopings Dolfi et al. (2015). Similar as the $1/8$ doping case, the
relationship $K_{sc}\cdot K_{\rho}=1$ is also likely to be satisfied, which is
consistent with the prediction from Luther-Emery liquid theory.

The absolute value of local magnetization and single-particle Green’s function
for 1/12 and 1/16 dopings are shown in Figs. 5(b), 5(c) and Figs. 5(e), 5(f).
Both of them decay exponentially with distance and the correlation lengths are
smaller than $1/8$ cases, indicating the spin and single-particle gap increase
with the decrease of doping.

Table 4: The dependence of the extracted parameters $K_{\rho}$ and $n_{0}$
(using Eq. (2) on the range of sites used ($\delta=1/3$, $L_{x}$ = 96)).
Figure 6: Density profile at $1/6$ doping. The local rung density $n(x)$ for
different kept states $m$ and the extrapolated with truncated errors results
are shown. The length of the system is $L_{x}=120$ at $1/6$ doping. The solid
line is the fitting curve using Eq. (2) to extract $K_{\rho}$. The dashed
horizontal line represents the average electron density. The inset is the
finite-size scaling of $\delta n(L_{x}/2)$ as a function of the system size
$L_{x}$ using Eq. (3)

###  III.3 $1/6$ doping level

Figure 7: Results for $1/6$ doping level. (a) shows $K_{sc}$ as a function of
the reference bond. The length of the system is $L_{x}=120$. The shaded area
represents the value of $1/K_{\rho}$ with error bar. (b) shows the absolute
values of the local magnetization at 1/6 doping for systems with different
lengths. Only the results from the extrapolation with truncation errors are
shown. The solid lines denote exponential fitting using
$\left|\mathbf{\textit{S}}_{\mathbf{\textit{z}}}\right|\propto\mathrm{e}^{-x_{i}/\xi_{\mathrm{s}}}$
with $\xi_{\mathrm{s}}=67.6(2)$ for $L_{x}=120$. (c) shows the single-particle
Green’s function for different numbers of kept states $m$ and the extrapolated
result. The reference site is set at (31, 2). Only peaked values are used in
the exponential fits for both the local magnetization and the single-particle
Green’s function.

In this subsection, we move to the $1/6$ doping case. Fig. 6 shows the charge
density profiles for a $L_{x}=120$ ladder, which is also characterized by an
algebraic quasi-long range order with period $\lambda_{c}=1/\delta=6$. As we
can see in the inset of Fig. 6, $K_{\rho}$ extracted from two approaches are
consistent with $K_{\rho}\approx 1.01(4)$. Following the procedure described
above, we show the variance of $K_{sc}$ with the position of reference bond in
Fig. 7(a). We find $K_{\rho}<K_{sc}$ (also see Tab. 1), suggesting the charge
correlation is dominating at 1/6 doping. We find that charge and
superconducting exponents are likely to violate the relationship $K_{sc}\cdot
K_{\rho}=1$. The local magnetization and single-particle Green’s function are
shown in Figs. 7(b) and 7(c). Both of them decay exponentially with
correlation lengths $\xi_{\mathrm{s}}=67.6(2)$ and $\xi_{G}=36(2)$.

###  III.4 $1/3$ and $1/4$ dopings

Figure 8: Density profiles at (a) $1/4$ and (b) $1/3$ dopings. The local rung
density $n(x)$ for different kept states $m$ and the extrapolated with
truncated errors results are shown. The lengths of the systems are $L_{x}=128$
for $1/4$ doping and $L_{x}=96$ for $1/3$ doping, respectively. The solid
lines are the fitting curves using a cosine function.

Fig. 8(a) shows the charge density profile at $1/4$ doping for a $L_{x}=128$
ladder. We can see that the charge modulation has a persistent amplitude with
period $\lambda_{c}=1/\delta$ which doesn’t decay in the bulk. The extracted
$K_{\rho}$ using Eq. (2) are listed in Tab. 3. The value of $K_{\rho}$ varies
dramatically with the data range used in the fit. The fact that $K_{\rho}$ has
very large error bars and oscillates across zero indicates the failure of the
fit using Eq. (2), which suggests the charge order is likely long-ranged in
the system.

The pair-pair correlation for $\delta=1/4$ are shown in Figs. 9(a) and 9(b).
Following the same procedure above, we find $K_{sc}=1.05(6)$ at $\delta=1/4$.
We show the local magnetization in both semi-logarithmic and double-
logarithmic scale in Fig. 9(c) and (d). We find the data is better described
by a exponential decay fit with a very large correlation length
$\xi_{\mathrm{s}}=107(2)$ instead of a power-law decay. The single-particle
Green’s function is found to decay exponentially with a correlation length
$\xi_{\mathrm{G}}=50(2)$.

The results for $1/3$ doping are similar as $1/4$ doping. The charge order is
likely to be a long-ranged one as shown in Fig. 8(b). In Tab. 4 , we find the
extracted value of $K_{\rho}$ also varies dramatically with the data range.
The extracted $K_{sc}=1.10(8)$ as shown in Figs. 10(a) and (b). The spatial
decay of local magnetization can be well fitted using either an exponential
law or a power law but with correlation length equal to the size of the system
(see Figs. 10(c) and 10(d), indicating the spin gap is likely to be closed at
$1/3$ doping. The decay of single-particle Green’s function is found to be
power-law like, indicating the closing of the gap at $1/3$ doping.

##  IV Conclusions

We revisit the ground state of the Hubbard model on 2-legged ladders with
DMRG. We find $K_{sc}$ extracted from the algebraic decay of pair-pair
correlation depends strongly on the position of reference bond. We obtain the
most accurate exponent $K_{\rho}$ and $K_{sc}$ with DMRG after systematically
treating the effects from boundary conditions, finite sizes, and truncation
errors in DMRG. We confirm that the 2-legged Hubbard model is in the Luther-
Emery liquid phase with $K_{\rho}\cdot K_{sc}=1$ from tiny doping near half-
filling to 1/8 hole doping, resolving the long-time controversy. When the
doping is increases to $\delta\gtrapprox 1/6$, the behaviors of charge,
pairing, and spin correlations don’t change qualitatively, but the
relationship $K_{sc}\cdot K_{\rho}=1$ is likely to be violated. With the
further increase of the doping to $\delta=1/3$, the quasi long-ranged charge
correlation turns to a long-ranged charge order and the spin gap is closed,
while the pair-pair correlation still decays algebraically. In Shen et al.
(2022), the Hubbard ladder at $1/3$ doping with next-nearest-neighbouring
hopping $t^{\prime}$ and a larger $U=12$ was studied Shen et al. (2022). The
charge correlation was found to decay algebraically, while the spin and
single-particle excitations are gapped Shen et al. (2022). The comparison of
the results in Shen et al. (2022) and in this work indicates the boundary of
different phases depend on the details of the parameters in the model. Our
work provides a standard way to analyze the correlation functions when
studying systems with open boundaries.

Figure 9: Pair-pair correlation and local magnetization at $1/4$ doping. (a)
$K_{sc}$ as a function of the reference bond. The length of the system is
$L_{x}=128$. (b) A typical fit of pair-pair correlation function. The
reference bond is set at the 65th vertical bond. (c) and (d) Absolute values
of the local magnetization. The results from finite kept state and the
extrapolated with truncation errors results are shown. The solid line in (c)
denotes exponential fitting using
$\left|\mathbf{\textit{S}}_{\mathbf{\textit{z}}}\right|\propto\mathrm{e}^{-x_{i}/\xi_{\mathrm{s}}}$
with $\xi_{\mathrm{s}}=107(2)$. The plots in (c) and (d) are shown in the
semi- and double-logarithmic axes, respectively. By comparing (c) and (d), we
find the decay of the local magnetization is likely to be exponential, even
though the extracted correlation length is comparable to the length of the
system. Only peaked values are used in the fits for both pair-pair correlation
and the local magnetization.

Figure 10: Pair-pair correlation and local magnetization at $1/3$ doping. (a)
$K_{sc}$ as a function of the reference bond. The length of the system is
$L_{x}=96$. (b) A typical fit of pair-pair correlation function. The reference
bond is set at the 49th vertical bond. (c) and (d) Absolute values of spin
density. The results from finite kept state and the extrapolated with
truncation errors results are shown. The solid lines in (c) and (d) denote
exponential and power-law fits, respectively. We find the data can be fitted
with either exponential or power law. But the correlation length extracted
from the power law fit is larger than the system size which indicates the
decay of the local magnetization is likely to be power law. Only peaked values
are used in the fits for both pair-pair correlation and the local
magnetization.

###### Acknowledgements.

We thank useful discussions with Xingjie Han. Y. Shen and M. P. Qin thank
Weidong Luo for his generosity to provide computational resources for this
work. M. P. Qin acknowledges the support from the National Key Research and
Development Program of MOST of China (2022YFA1405400), the National Natural
Science Foundation of China (Grant No. 12274290) and the sponsorship from
Yangyang Development Fund. All the DMRG calculations are carried out with
iTensor libraryFishman et al. (2022).

## References

  * Hubbard (1963) J. Hubbard, Proc. R. Soc. Lond. Ser. A 276, 238 (1963).
  * Arovas et al. (2022) D. P. Arovas, E. Berg, S. A. Kivelson, and S. Raghu, Annu. Rev. Mater. Res. 13, 239 (2022).
  * Qin et al. (2022) M. Qin, T. Schäfer, S. Andergassen, P. Corboz, and E. Gull, Annu. Rev. Mater. Res. 13, 275 (2022).
  * Dagotto (1994) E. Dagotto, Rev. Mod. Phys. 66, 763 (1994).
  * Himeda et al. (2002) A. Himeda, T. Kato, and M. Ogata, Phys. Rev. Lett. 88, 117001 (2002), URL https://link.aps.org/doi/10.1103/PhysRevLett.88.117001.
  * Gull et al. (2013) E. Gull, O. Parcollet, and A. J. Millis, Phys. Rev. Lett. 110, 216405 (2013), URL https://link.aps.org/doi/10.1103/PhysRevLett.110.216405.
  * Darmawan et al. (2018) A. S. Darmawan, Y. Nomura, Y. Yamaji, and M. Imada, Phys. Rev. B 98, 205132 (2018), URL https://link.aps.org/doi/10.1103/PhysRevB.98.205132.
  * Ponsioen et al. (2019) B. Ponsioen, S. S. Chung, and P. Corboz, Phys. Rev. B 100, 195141 (2019), URL https://link.aps.org/doi/10.1103/PhysRevB.100.195141.
  * Qin et al. (2020) M. Qin, C.-M. Chung, H. Shi, E. Vitali, C. Hubig, U. Schollwöck, S. R. White, and S. Zhang (Simons Collaboration on the Many-Electron Problem), Phys. Rev. X 10, 031016 (2020).
  * Chung et al. (2020) C.-M. Chung, M. Qin, S. Zhang, U. Schollwöck, and S. R. White (The Simons Collaboration on the Many-Electron Problem), Phys. Rev. B 102, 041106 (2020).
  * Jiang et al. (2021) S. Jiang, D. J. Scalapino, and S. R. White, Proceedings of the National Academy of Sciences 118, e2109978118 (2021), eprint https://www.pnas.org/doi/pdf/10.1073/pnas.2109978118, URL https://www.pnas.org/doi/abs/10.1073/pnas.2109978118.
  * Gong et al. (2021) S. Gong, W. Zhu, and D. N. Sheng, Phys. Rev. Lett. 127, 097003 (2021).
  * Jiang and Kivelson (2021) H.-C. Jiang and S. A. Kivelson, Phys. Rev. Lett. 127, 097002 (2021), URL https://link.aps.org/doi/10.1103/PhysRevLett.127.097002.
  * Xu et al. (2023) H. Xu, C.-M. Chung, M. Qin, U. Schollwöck, S. R. White, and S. Zhang, arXiv e-prints arXiv:2303.08376 (2023), eprint 2303.08376.
  * Bednorz and Müller (1986) J. G. Bednorz and K. A. Müller, Z. Physik B - Condensed Matter 64, 189 (1986).
  * Lieb and Wu (1968) E. H. Lieb and F. Y. Wu, Phys. Rev. Lett. 20, 1445 (1968).
  * LeBlanc et al. (2015) J. P. F. LeBlanc, A. E. Antipov, F. Becca, I. W. Bulik, G. K.-L. Chan, C.-M. Chung, Y. Deng, M. Ferrero, T. M. Henderson, C. A. Jiménez-Hoyos, et al. (Simons Collaboration on the Many-Electron Problem), Phys. Rev. X 5, 041041 (2015), URL https://link.aps.org/doi/10.1103/PhysRevX.5.041041.
  * Troyer et al. (1996) M. Troyer, H. Tsunetsugu, and T. M. Rice, Phys. Rev. B 53, 251 (1996).
  * Balents and Fisher (1996) L. Balents and M. P. A. Fisher, Phys. Rev. B 53, 12133 (1996).
  * Schollwöck (2005) U. Schollwöck, Rev. Mod. Phys. 77, 259 (2005).
  * White (1992) S. R. White, Phys. Rev. Lett. 69, 2863 (1992).
  * White (1993) S. R. White, Phys. Rev. B 48, 10345 (1993).
  * Schollwöck (2011) U. Schollwöck, Ann. Phys. 326, 96 (2011), ISSN 0003-4916, january 2011 Special Issue.
  * Luther and Emery (1974) A. Luther and V. J. Emery, Phys. Rev. Lett. 33, 589 (1974).
  * Haldane (1980) F. D. M. Haldane, Phys. Rev. Lett. 45, 1358 (1980).
  * Dolfi et al. (2015) M. Dolfi, B. Bauer, S. Keller, and M. Troyer, Phys. Rev. B 92, 195139 (2015).
  * Lu et al. (2023) X. Lu, D.-W. Qu, Y. Qi, W. Li, and S.-S. Gong, Phys. Rev. B 107, 125114 (2023).
  * Noack et al. (1997) R. M. Noack, N. Bulut, D. J. Scalapino, and M. G. Zacher, Phys. Rev. B 56, 7162 (1997).
  * Noack et al. (1996) R. Noack, S. White, and D. Scalapino, Physica C 270, 281 (1996), ISSN 0921-4534.
  * White et al. (2002) S. R. White, I. Affleck, and D. J. Scalapino, Phys. Rev. B 65, 165122 (2002).
  * Schulz (1999) H. J. Schulz, Phys. Rev. B 59, R2471 (1999).
  * Siller et al. (2001) T. Siller, M. Troyer, T. M. Rice, and S. R. White, Phys. Rev. B 63, 195106 (2001).
  * Eberharter et al. (2023) A. A. Eberharter, L. Vanderstraeten, F. Verstraete, and A. M. Läuchli, arXiv e-prints arXiv:2303.00663 (2023), eprint 2303.00663.
  * Shen et al. (2022) Y. Shen, M. Qin, and G.-M. Zhang, arXiv preprint arXiv:2207.00266 (2022).
  * Tranquada et al. (1995) J. Tranquada, B. Sternlieb, J. Axe, Y. Nakamura, and S.-i. Uchida, Nature 375, 561 (1995).
  * Emery et al. (1999) V. J. Emery, S. A. Kivelson, and J. M. Tranquada, Proc. Natl. Acad. Sci. 96, 8814 (1999).
  * Fishman et al. (2022) M. Fishman, S. R. White, and E. M. Stoudenmire, SciPost Phys. Codebases pp. 4–r0.3 (2022).