# Parameter-Efficient Instruction Tuning of Large Language Models For Extreme
Financial Numeral Labelling

Subhendu Khatuya1 Rajdeep Mukherjee1 Akash Ghosh1 Manjunath Hegde2
Koustuv Dasgupta2 Niloy Ganguly1 Saptarshi Ghosh1 Pawan Goyal1
1Indian Institute of Technology Kharagpur, India
2Language Modeling, Goldman Sachs
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

We study the problem of automatically annotating relevant numerals (GAAP
metrics) occurring in the financial documents with their corresponding XBRL
tags. Different from prior works, we investigate the feasibility of solving
this extreme classification problem using a generative paradigm through
instruction tuning of Large Language Models (LLMs). To this end, we leverage
metric metadata information to frame our target outputs while proposing a
parameter efficient solution for the task using LoRA. We perform experiments
on two recently released financial numeric labeling datasets. Our proposed
model, FLAN-FinXC, achieves new state-of-the-art performances on both the
datasets, outperforming several strong baselines. We explain the better scores
of our proposed model by demonstrating its capability for zero-shot as well as
the least frequently occurring tags. Also, even when we fail to predict the
XBRL tags correctly, our generated output has substantial overlap with the
ground-truth in majority of the cases.

Parameter-Efficient Instruction Tuning of Large Language Models For Extreme
Financial Numeral Labelling

Subhendu Khatuya1 Rajdeep Mukherjee1 Akash Ghosh1 Manjunath Hegde2 Koustuv
Dasgupta2 Niloy Ganguly1 Saptarshi Ghosh1 Pawan Goyal1 1Indian Institute of
Technology Kharagpur, India 2Language Modeling, Goldman Sachs
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

##  1 Introduction

The U.S. Securities and Exchange Commission (SEC) mandates publicly traded
companies to disclose periodic filings such as quarterly 10-Q & annual 10-K
reports. These documents are important to finance professionals and investors
who rely on SEC filings to make informed investment decisions. Each company is
directed to follow the Generally Accepted Accounting Principles (GAAP) to
report the metrics appearing in these documents and tag them using the
eXtensive Business Reporting Language (XBRL) according to a well-defined
taxonomy consisting of thousands of labels. In a recently released FNXL
dataset Sharma et al. (2023), such numerals are tagged from a large set of
2,794 labels. Implementing XBRL tagging therefore requires advanced accounting
skills to map financial data to the correct XBRL concepts. This requires
hiring experts to meticulously review each document and assign appropriate
labels which is neither a cost-effective nor a scalable solution. Fig. 1 shows
various challenges involved with the task.

Figure 1: Demonstrating the challenges in the Extreme Financial Numeral
Labelling (XFNL) task. Within a financial statement, there are scenarios where
every numeral is associated with a distinct XBRL tag, such as in Example 2 (6
distinct tags). Then, there are cases where a mixture of both relevant and
irrelevant numerals (tagged ‘Others’) coexist in the same statement, often
within a very limited context, such as in Examples 1 & 3.

Tag |  Documentation
---|---
common stocks shares issued |  Total number of common shares of an entity that have been sold or granted to shareholders (includes common shares that were issued, repurchased and remain in the treasury). These shares represent capital invested by the firm’s shareholders and owners, and may be all or only a portion of the number of shares authorized.
common stocks shares authorized |  The maximum number of common shares permitted to be issued by an entity’s charter and bylaws.

Table 1: Examples of (XBRL tag, documentation) pairs. We observe that the two
tags differ by a single word only whereas their corresponding documentations
vary significantly. We take advantage of this distinction while training our
FLAN-FinXC variants. More such examples given in the Appendix A.1

Prior works and their limitations: FiNER Loukas et al. (2022) formulated the
task of identifying relevant numerals from financial texts and labelling them
with XBRL tags as an NER problem, and used a BERT-based sequence labelling
approach. However, the label-set in FiNER consists of only 139 most frequently
occurring XBRL tags. For practical purposes, a much larger number of XBRL
tags/labels are needed to effectively annotate the diverse types of numerals
present in these documents. More recently, the authors of FNXL dataset Sharma
et al. (2023), demonstrated the poor performance of FiNER when extended to
thousands of labels. They also explored an extreme classification methodology,
called AttentionXML. None of these methods, however, exploit the rich metadata
information available with XBRL tags, to improve the classification
performance. Table 1 provides example of XBRL tag documentations that can help
with the labelling task. Among recent methods that utilize label metadata for
better results, GalaXC Saini et al. (2021), is a GNN-based extreme
classification approach that embeds label metadata information in its
document-label graph nodes. Label Semantics Ma et al. (2022) is another
generic approach that leverages entity descriptions to solve the standard NER
task. However, none of these methods have been applied in the financial
domain. We, therefore, adapt these models to use as additional baselines for
our task.

Additionally, all the methods stated above lack the capacity to identify
unseen labels during inference as they follow a discriminative paradigm.
Generative models, on the other hand, display intrinsic zero-shot capabilities
if trained effectively. In this space, LLMs have achieved impressive
performances for a wide range of NLP tasks across several domains Zhao et al.
(2023), including finance Wu et al. (2023); Yang et al. (2023).

FLAN-FinXC Framework: In this work, we show for the first time that generative
models (LLMs) can achieve impressive results for the XFNL task. We
systematically explore and propose FLAN-FinXC, a framework of Parameter-
Efficient Instruction Tuning for Extreme Classification.

Our FLAN-FinXC framework consists of FLAN-T5 Chung et al. (2022) models
instruction-tuned with carefully-curated task-specific instructions, as shown
in Fig. 2, to generate the appropriate XBRL tag documentations. We then make
use of an unsupervised Tag Matcher module to predict the final XBRL tag for
this generated documentation. We perform extensive experiments to devise a
total of five different model variants as part of our proposed FLAN-FinXC
framework, ranging from T5-Base to FLAN-T5-Large, and with varying training
strategies. We observe that FLAN-T5-Large (instruction-tuned) achieves 9.4%
Macro-F1 gains and 3.5% Hits@1 gains, over T5-Large (fine-tuned) for FNXL
dataset, both models being architecturally same with 780M parameters. The same
trend is also observed in FiNER data. This highlights the advantages of
Instruction Tuning similar-sized LLMs over old-fashioned fine-tuning. Given
that training larger models is costly, next we experiment with parameter-
efficient (PEFT) techniques, specifically Prefix Tuning Li and Liang (2021)
and LoRA Hu et al. (2021), to instruction-tune our FLAN-T5-Large models.

Among these, we observe that FLAN-T5-Large with LoRA outperforms the Prefix-
Tuned version with 2.4% Macro-F1 gains and 5% gains in Hits@1, giving the best
performance for both the datasets.111Experimenting with 100 NLP tasks, Ding et
al. (2023) had also made similar observations that LoRA can sometime even
outperform full fine-tuning on certain tasks.

Taking advantage of the generative paradigm, parameter-efficient instruction
tuning of LLMs, as well as our financial domain-specific novelty (through the
use of XBRL tag documentations to improve extreme classification performance),
our best model, FLAN-T5-Large with LoRA, outperforms the state-of-the-art
AttentionXML model with 39.3% Macro-F1 gains and 17.2% Hits@1 gains, thereby
achieving new state-of-the-art results for the XFNL task. We then present
several interesting analyses to investigate the reasons for its considerably
better performances. We find that our model achieves impressive zero-shot
Macro-F1 scores of 58.89% for the 67 XBRL tags that were unseen during
training. Even for tags that appear fewer than 5 times in the training data,
our model is able to achieve 41% Macro-F1 gains and 23% Hits@1 gains compared
to AttentionXML. Qualitatively, among the instances where we fail to predict
the correct XBRL tags, in around 60% of the cases, our generated tag
documentations are very close to the ground truth documentations with Jaccard
Similarity scores ranging between 0.6 and 0.85. The proposed model also
achieves superior performance (15.22% Macro-F1 gain and 3.3% Hits@1 gains) in
case of FiNER data containing only most frequent 139 labels.

Our contributions can be summarized as follows:

  * •

We propose FLAN-FinXC, a generative framework, consisting of a suite of
instruction-tuned FLAN-T5 models, varying in model sizes and training
strategies, to tackle the XFNL task, which is a practically important problem
in the finance domain. Different from the prior methods for the task, our
models utilize the XBRL tag documentations, instead of considering the tags as
just class labels.

  * •

In addition to comparing with the state-of-the-art methods, we adopt several
prior works (Label Semantics and GalaXC) for the task, along with devising our
own competitive generative baselines (T5-Base and T5-Large).

  * •

For both FiNER as well as FNXL datasets, our best model, FLAN-T5-Large with
LoRA, achieves huge improvements over all the baselines. Additional advantages
of our best model include substantial performance over rare labels, and its
zero-shot capability to tag numerals with labels unseen during training. Our
dataset and codes are publicly available at
https://github.com/subhendukhatuya/FLAN-FinXC.

##  2 Related Works

One of the (re)emerging applications of NLP is in the field of Named Entity
Recognition (NER), particularly for identifying and categorizing various
entities within text. The categorization of different entities into labels can
also be considered as an extreme classification task Dahiya et al. (2021). In
recent years, XBRL tagging has gained a special importance in financial
domain, which involves tagging of numeric values. Datasets created for such a
task are the FNXL dataset Sharma et al. (2023) and FiNER Loukas et al. (2022)
which have a very large number of entity types compared to standard NER tasks
and thus present challenges to the state-of-the-art NER models. Sharma et al.
(2023) reformulated this task as an extreme classification You et al. (2019)
problem with a pipeline approach. Financial domain-specific pre-trained
language models Shah et al. (2022) have been developed, which can be
repurposed on financial tasks. Among LLMs pre-trained on financial text
(FinLLMs), BloombergGPT Wu et al. (2023) is a proprietary model that suffers
from limited accessibility and lack of transparency in their data collections
and training protocols. FinGPT Yang et al. (2023) is an open-source FinLLM but
only suitable for financial sentiment analysis, trading, forecasting and fraud
detection tasks.

##  3 Problem Formulation

Figure 2: FLAN-FinXC Architecture. FLAN-T5 takes as input a task-specific
instruction, the financial statement, and a question with a designated target
numeral. FLAN-T5-generated tag documentation subsequently flows into the Tag
Matcher that predicts the final tag for the given numeral.

We break the task of XFNL into two stages (with only the first stage requiring
supervised training of models), as illustrated in Fig. 2, in order to take
advantage of more elaborate XBRL tag documentations. We present a set of
diverse annotated examples in Fig. 1. Some tag with documentation examples are
shown in Table 1.

In the first stage, we formulate the problem as a generative task using LLMs,
where given a financial statement, and a question targeted towards a specific
numeral occurring in the statement, the task is to accurately generate its
appropriate XBRL tag documentation (not the tag) if the numeral is relevant
and ‘Other’ if the numeral is irrelevant. Let
$S_{i}=(w_{i}^{1},...,w_{i}^{a},...w_{i}^{b},...,w_{i}^{n})$ be the $i^{th}$
statement consisting of $n$ tokens, where $w_{i}^{a}$ and $w_{i}^{b}$ be two
different numerals with $tag_{i}^{a}$ and $tag_{i}^{b}$ being their respective
XBRL tag documentations. We prepend an instruction prompt $IP$, containing a
natural language description of the task, to the statement $S_{i}$, as shown
in Fig. 2. A question $Q_{i}^{a}$ is then appended to $S_{i}$ asking for the
tag to be determined for a specific numeral, say $w_{i}^{a}$. The modified
input $S_{i}^{a}$ therefore takes the shape $IP||S_{i}||Q_{i}^{a}$, where $||$
is a text concatenation operation. The target answer
$genTag_{i}^{a}=LLM(S_{i}^{a})$ for the LLM (FLAN-T5 in our case) therefore
becomes: $tag_{i}^{a}$.

In the second stage, we obtain the final XBRL tag through a separate Tag
Matcher module, since the entire documentation may not be generated exactly.
Specifically, we obtain the embedding for $genTag_{i}^{a}$ using a pre-trained
state-of-the-art sentence encoder. We obtain the same for the documentations
corresponding to all the available tags. The one having the highest cosine
similarity with the $genTag_{i}^{a}$ embedding is declared to be the predicted
XBRL tag documentation, $predTag_{i}^{a}$.

##  4 Methodology

Our proposed framework FLAN-FinXC for the XFNL task is divided into two
phases, as depicted in Fig. 2: a supervised generative phase, and an
unsupervised documentation-to-tag matching phase. In the first phase, given a
financial statement $S_{i}$, and a question $Q_{i}^{a}$ asking for the XBRL
tag (documentation) to be determined for a numeral $w_{i}^{a}$ appearing in
the sentence $S_{i}$, we instruction tune FLAN-T5 Chung et al. (2022); Longpre
et al. (2023) with carefully-curated task-specific instructions (see Fig. 2)
to generate the tag documentation $genTag_{i}^{a}$. The model is trained to
condition on the modified input $S_{i}^{a}$, as described in the previous
section, to generate the target answer one token at a time using auto-
regressive decoding. Cross-entropy loss between the generated and true tokens
is minimized in the process. In our case, the entire output generated by the
LLM corresponds to the tag description, hence no additional parsing is
required.

Our choice for FLAN-T5 is based on the observation that FLAN-T5 models are
pre-trained (using instruction tuning) on more than 1.8K tasks, and hence can
significantly reduce the amount of fine-tuning steps required if adopted as
starting checkpoints for learning new tasks. Additionally, they achieve strong
zero-shot and few-shot performances in comparison to T5 Raffel et al. (2020),
their non-instruction-tuned counterpart.

Note that we train FLAN-T5 models to generate the XBRL tag documentations
instead of the tag themselves (which is the final target) since the more
elaborate documentations allow for a better distinction than their
corresponding tags (see Table 1), thereby aiding in the extremely challenging
XFNL task (as demonstrated in Fig. 1). Generating the lengthy tag
documentations exactly is however difficult, and hence the generated
documentation $genTag_{i}^{a}$ may not exactly match the ground truth. In the
second phase of our proposed framework, we therefore leverage a pre-trained
state-of-the-art sentence encoder, as the backbone of our Tag Matcher module,
as depicted in Fig. 2. More specifically, we use Sentence-T5-XXL Ni et al.
(2021), pre-trained using contrastive loss, to generate 786-dim embeddings for
$genTag_{i}^{a}$ as well as for each of the ground truth tag documentations.
The ground truth tag documentation with the highest cosine similarity (between
embeddings) with $genTag_{i}^{a}$ is considered to be the predicted tag
documentation $predTag_{i}^{a}$. Since there exists a 1:1 mapping between XBRL
tags and their corresponding documentations, the final predicted tag can be
easily obtained from $predTag_{i}^{a}$.

In order to investigate the suitability of a generative paradigm for solving
the task, we perform a systematic evaluation of several model variants,
varying both model sizes and training strategies.

Non-FLAN Model Variants: First, we compare the fine-tuned performances of
T5-Base (220M parameters) with T5-Large (780M parameters). Note that for fine-
tuning, given a financial statement $S_{i}$, the modified input now becomes
$S_{i}^{a}=S_{i}||Q_{i}^{a}$, where $Q_{i}^{a}$ refers to the question
targeted towards the numeral $w_{i}^{a}$ appearing in $S_{i}$. The target (tag
documentation) however remains the same.

FLAN-FinXC Model Variants: Next, we compare the instruction-tuned performance
of FLAN-T5-Large with fine-tuned T5-Large, both models with identical
architectures. Next, we instruction tune FLAN-T5-Large with PEFT techniques,
namely Prefix Tuning and LoRA, respectively requiring only 0.13% and 0.08% of
model parameters to be updated. Finally, we instruction-tune FLAN-T5-Large
with LoRA to achieve our best results.

##  5 Baselines

We compare the performance of our proposed models with the following
baselines:

Baselines already applied for XBRL tagging: (1) FiNER: Loukas et al. (2022)
solved the task as a NER task, but only for top 139 frequent XBRL tags. We
adapt this method for our use case (large number of labels). (2) AttentionXML
Pipeline: Sharma et al. (2023) tackled this problem using a BERT-based
sequence-to-sequence tagger.

New baselines that we adopt for XBRL tagging: Additionally, we adopt two
diverse methods for the task for XBRL tagging, which can utilize the label
semantics of XBRL tags. (3) GalaXC: Saini et al. (2021) applies collaborative
learning over document-label graphs that allows additional information like
label metadata to be incorporated. (4) Label Semantics Ma et al. (2022)
leverages the entity description to solve a standard NER task. Lastly, with
the emergence of ChatGPTOpenAI (2022), we were curious to check ChatGPT’s
performance for this task for 500 random samples using gpt-3.5-turbo
222https://platform.openai.com/docs/models/gpt-3-5 API.

##  6 Dataset & Evaluation Metrics

Dataset: We perform our experiments on the recently released FNXL dataset
Sharma et al. (2023) containing 10-K documents333https://tinyurl.com/t43mwd5m
for 2,339 companies and FiNER Loukas et al. (2022) dataset having most
frequent (at least 1000 appearances) 139 labels. FNXL dataset contains a total
of 79,088 sentences containing 142,922 annotated numerals, with a tag set of
2,794 distinct tags that follows a heavy tail distribution. Further, the test
set contains 67 XBRL tags that are not part of the train or validation sets.
Fig. 1 shows a few annotated examples from this dataset. Each tag is
associated with a pre-defined textual description known as its ‘tag
documentation’ (see Table 1). The tag documentations have an average length of
28 words and a maximum length of 273 words.

Metrics: To ensure a comprehensive evaluation of all models, we employ the
following metrics: 1) Macro-Precision, 2) Macro-Recall, 3) Macro-F1, 4)
Hits@1. In financial numeral labelling, where equal importance is given to all
tags, macro based metrics (precision, recall, F1) are a good choice as it
treats all classes equally, irrespective of their frequency. We report Hits@1
metric to showcase the usability of this system from business perspective,
recommending the top tag to domain experts.

##  7 Experimental Setup

For all our model variants (experiments performed on Tesla V100 32GB GPUs), we
obtain the pre-trained checkpoints from the Huggingface
Library444https://huggingface.co/. For instruction tuning FLAN-T5-Large with
Prefix Tuning, the prefix length was set to 20. For training the models with
LoRA, the rank for the trainable decomposition matrices was set to 2.
FLAN-T5-Large models were instruction-tuned for 10 epochs with a learning rate
(lr) of $1e-4$ (training time: 1hr 22 minutes/epoch, inference time: 2
minutes/sample); with an lr of $1e-2$ with Prefix Tuning (training time: 29
minutes/epoch, inference time: 2 minutes/sample); and with an lr of $5e-4$
with LoRA (training time: 56 minutes/epoch, inference time: 2 minutes/sample).
These hyperparameters were selected based on the best Macro-F1 results on the
validation set.

Note that the tag information is not included in the input prompt. Rather, it
is generated during the decoding phase based on the question asked. The input
prompt containing the question gets encoded and it is much smaller than the
maximum input context length. Hence, there is no truncation during encoding.
The average tag information length is 28 tokens and we set the length of
output to be generated to 30 tokens. Hence, in a few cases, truncation may
occur during the decoding phase. However, the crucial tag information is
typically present in the initial tokens, and our tag matcher ensures correct
labels by comparing generated tags with the descriptions for all available
tags in the dataset. Thus any potential truncation in a few cases does not
pose any risk to our model’s performance.

Instruction Selection: While experimenting, we gradually improved our
instructions. First, we tried out a prompt without any task-specific
instruction, thereby consisting of the input financial statement and the
question for a given numeral. Then, we included a simple task description and
repeated our experiments. Finally, we made the task description/instruction
more elaborate which led us to obtain the best results. On the target side, we
tried out two variants, generating labels vs. generating the label
descriptions. The latter resulted in significant performance improvement
(refer Table 4). This iterative process of instruction selection was guided by
the performance on the validation set.

Model | Dataset
---|---
| FNXL Sharma et al. (2023) | FiNER Loukas et al. (2022)
| M-P | M-R | M-F1 | Hits@1 | M-P | M-R | M-F1 | Hits@1
FiNER (bert-base) | 49.17 | 49.71 | 47.13 | 75.34 | 72.60 | 81.10 | 76.61 | 81.50
FiNER (sec-base) | 47.76 | 48.87 | 46.20 | 74.67 | 81.11 | 83.20 | 82.14 | 82.30
Label Semantics | 46.35 | 45.12 | 45.72 | 71.25 | 71.50 | 80.15 | 75.57 | 80.25
GalaXC | 46.91 | 44.81 | 45.81 | 72.97 | 72.20 | 80.95 | 76.32 | 81.10
AttentionXML Pipeline | 50.69 | 48.51 | 47.54 | 76.76 | 82.15 | 82.30 | 82.22 | 83.25
ChatGPT (500 samples) | 11.13 | 7.68 | 9.08 | 19.6 | 20.12 | 15.67 | 17.61 | 22.35
T5-Base | 59.94 | 49.48 | 54.21 | 79.21 | 86.92 | 84.35 | 85.61 | 83.45
T5-Large | 61.87 | 58.46 | 60.11 | 83.26 | 88.12 | 85.10 | 86.58 | 84.12
FLAN-T5-Large | 66.21 | 65.34 | 65.77 | 86.21 | 92.10 | 96.35 | 94.17 | 85.89
FLAN-T5-Large with Prefix-Tuning | 65.10 | 64.21 | 64.65 | 85.69 | 90.18 | 94.35 | 92.21 | 85.12
FLAN-T5-Large with LORA | 65.14 | 67.36 | 66.23 | 89.98 | 91.84 | 97.85 | 94.74 | 86.03

Table 2: Performance evaluation based on Macro & Hits@1 metrics for FNXL
dataset and FINER dataset. The best performance is highlighted in bold, and
the strongest baseline result is underlined.

##  8 Main Results

We report the results of our proposed model variants and various baselines in
Table 2 for both FNXL and FiNER dataset. Among the baselines, FiNER does not
perform well for large number of entity labels. On the other hand, while the
‘Label Semantics’ method leverages tag words within an NER framework, its
scalability is constrained when dealing with a large number of entity labels,
ultimately leading to poor performance. The AttentionXML pipeline performs
better than all other baselines. GalaXC and ‘Label Semantics’ have very
similar performance, with GalaXC giving slight edge. ChatGPT’s performance for
this complex task is not satisfactory.

We also demonstrate the advantage of training bigger models for extreme
financial numeral labelling for FNXL dataset, by comparing the results of
T5-Large (780M parameters, Macro-F1 60.11) with T5-Base (220M parameters,
Macro-F1 54.21). Note that these baselines that we devise for the task,
already outperform the existing state-of-the-art.

Next, we turn to different variations of our FLAN-FinXC framework (listed in
the lower part of Table 2). First, we demonstrate the advantage of instruction
tuning over fine-tuning by the fact that FLAN-T5-Large (Macro-F1 65.77)
substantially outperforms T5-Large (Macro-F1 60.11) for FNXL dataset. Given
that training larger models is costly, next we experiment with parameter-
efficient (PEFT) techniques, specifically Prefix Tuning Li and Liang (2021)
and LoRA Hu et al. (2021), to instruction tune our FLAN-T5-Large models. Both
techniques require only a small fraction of model parameters to be fine-tuned
(0.13% with Prefix Tuning, and 0.08% with LoRA), thereby greatly reducing the
computation cost and training times. Instruction tuning FLAN-T5-large with
LoRA (Macro-F1 66.23) outperforms Prefix-Tuned version of the same model
(Macro-F1 64.65) and has a slight edge over the model variant instruction-
tuned without PEFT (Macro-F1 65.77). Our best results are obtained by
instruction tuning FLAN-T5-Large with LoRA (only 0.08% of 780M parameters to
be finetuned). This model outperforms the state-of-the-art AttentionXML model
with 39.3% Macro-F1 gains and 17.2% Hits@1 gains, thereby achieving new state-
of-the-art results for FNXL dataset. For FiNER dataset, we achieve 15.22%
Macro-F1 gain and 3.3% Hits@1 gains.

##  9 Analysis

We now present different analyses of our best model (FLAN-T5-Large with LoRA),
and the closest baseline (AttentionXML Pipeline). The following analysis is
specific to the FNXL dataset, chosen for its more extensive range of labels
compared to other datasets, emphasizing its practical relevance.

###  9.1 Performance on least frequent labels

Accurate tagging of infrequent labels in XBRL is crucial for reliable
financial reporting, yet it presents challenges stemming from imbalanced data,
scarce training instances, and sparse data distribution. We observe our FLAN-
FinXC performs substantially better than AttentionXML for the rare labels. To
demonstrate this, we group the tags into various buckets based on their
frequency of occurrence in the training set.

Figure 3: Relative improvement in performance achieved by FLAN-T5-Large with
LoRA over AttentionXML Pipeline, for the least frequent labels under various
frequency buckets

Fig. 3 shows the percentage improvement in Hits@1 and Macro-F1 that is
achieved by FLAN-FinXC over AttentionXML Pipeline for the tags in every
bucket. We see that our model can effectively identify and tag rare financial
concepts. Notably, even for tags that appear fewer than 10 times in the
training data, our model is able to achieve 35.3% improvement in Macro-F1
score and 25% in Hits@1 over the closest baseline.

###  9.2 Zero-Shot Capability

One of the key strengths of our proposed model is its zero-shot capability,
i.e., its ability to generate tags/labels for which it has not been explicitly
trained. While SOTA discriminative models in this domain often require
specific fine-tuning or retraining to handle new tags, our generative model
transcends these limitations.

Tag | F1-score
---|---
foreign currency transaction gain before tax | 0.85
commercial paper at carrying value | 0.85
accounts payable other current | 0.80
recognition of deferred revenue | 0.76
available for sale debt securities gross unrealized gain | 0.66
Table 3: Zero-Shot Performance of FLAN-T5-Large with LoRA, for a few XBRL tags
absent in the training set of FNXL dataset.

Recall that the test set included 67 new labels that were not present in the
train set. We observe that our best model achieves a Macro-F1 of 58.89 over
these 67 unseen labels, which is a commendable performance. Table 3 shows the
performance (F1-score) of our best model for few such tags.

Model | Macro-F1 | Hits@1
---|---|---
FLAN-T5-Large with LoRA | 66.23 | 89.98
w/ S-BERT-L12 as Tag Matcher | 63.11 | 88.13
w/ S-BERT-L6 as Tag Matcher | 62.87 | 87.72
w/o instruction prompt | 56.46 | 76.55
w/o tag metadata | 53.12 | 73.14

Table 4: Results for our ablation studies of FNXL dataset

###  9.3 Ablation Study

We now try out various ablations over our best model to understand the
significance of different modules. First, as ablations of our Tag Matcher
module, we replace Sentence-T5-XXL (used in our best model) with Sentence-BERT
Reimers and Gurevych (2019) to generate the embeddings for the XBRL tag
documentations. We experiment with two versions of Sentence-BERT, one with 6
encoder layers and other with 12 encoder layers, and report our findings in
Table 4. We observe a drop in performance (compared to our best model FLAN-
FinXC with LoRA) in both cases, thereby showing the better efficacy of
Sentence-T5-XXL over Sentence-BERT as sentence encoder.

Next, we instruction-tune FLAN-T5-Large (with LoRA) without the instruction
prompt containing specific instructions for the XFNL task. Given a financial
statement $S_{i}$, the modified input now becomes
$S_{i}^{a}=S_{i}||Q_{i}^{a}$, where $Q_{i}^{a}$ refers to the question
targeted towards the numeral $w_{i}^{a}$ appearing in $S_{i}$, and $||$ is a
text concatenation operation. From the fourth row in Table 4, we observe a
huge drop in performance (e.g., Macro-F1 drops from 66.23 to 56.46), thereby
demonstrating the importance of aligning FLAN-T5 fine-tuning with task-
specific instructions to tackle the challenging XFNL task.

Next, we experiment with setting the XBRL tags, instead of their
documentations, as the target for FLAN-T5-Large. In other words, we no longer
use the tag metadata. Accordingly, the Tag Matcher module now compares the
embeddings of the generated and ground truth tags (and not documentations).
From the last row in Table 4, we again observe a drop in scores with 19.8%
$\downarrow$ in Macro-F1 and 18.7% $\downarrow$ in Hits@1 compared to our best
results. This confirms our hypothesis that the more elaborate tag
documentations allow for a clearer distinction than the corresponding tags
(possibly differing only in few words) while tackling the challenging extreme
classification task.

Sentence |  Ground truth tag |  AttentionXML Pipeline prediction |  FLAN-T5-Large with LoRA
---|---|---|---
As of December 31, 2019 and 2018, we had a cumulative translation loss, net of tax of $ 617 million and $ 466 million, respectively. |  accumulated other comprehensive income loss foreign currency translation adjustment net of tax |  accumulated other comprehensive income loss defined benefit pension and other postretirement plans net of tax ✗ |  accumulated other comprehensive income loss foreign currency translation adjustment net of tax ✓
We also have $4.54 billion of non - cancelable contractual commitments as of December 31, 2019 related to network infrastructure |  contractual obligation |  unrecorded unconditional purchase obligation balance sheet amount ✗ |  contractual obligation ✓
At April 24, 2020, plan participants had approximately $ 14 million withheld to purchase the company’s ordinary shares |  others |  share based compensation arrangement by share based payment award number of shares available for grant ✗ |  others ✓

Table 5: Challenging examples where AttentionXML predicts a wrong tag, while
FLAN-T5-Large with LoRA predicts correctly.  Sentence: Our Board of Directors
declared quarterly dividends per share of $1.45, $1.32 and $1.15, which were
paid in each of the four quarters of 2019, 2018, and 2017.
---
GT Tag Doc: aggregate  dividends paid during the period for each share of
common stock outstanding. GT Tag: common stock dividends per share cash paid
FLAN-T5-Large with LoRA (Generated, No Tag Matcher): Aggregate dividends
declared during the period for each share of common stock outstanding.
Predicted Tag: common stock dividends per share declared
Sentence: In fiscal year 2016, Bard paid the Company $121 million towards the
settlement of 11,000 of these claims.
GT Tag Doc: amount awarded from other party in judgment or settlement of
litigation.
GT Tag: litigation settlement amount awarded from other party
FLAN-T5-Large with LoRA (Generated, No Tag Matcher): Amount awarded to other
party in judgment or settlement of litigation
Predicted Tag: litigation settlement amount awarded to other party
Table 6: Examples to show that subtle differences between ground truth tag
(GT) and the predicted tag. The few-word differences between Ground Truth
(green) and predicted text (red) are highlighted. More such examples are in
the Appendix A.3

Qualitative Analysis of predicted labels: Table 5 shows the predictions made
by our best model and the closest baseline (AttentionXML) for a few
challenging instances. Our proposed method classifies relevant numerals (first
two examples) correctly and is able to find irrelevant numerals (the last
example) and correctly tag that as ‘Others‘, whereas the baseline struggles
for each cases.

Error Analysis: To characterize the errors committed by our model, we ask –
when a model generates a wrong tag, how similar is the generated tag with the
ground truth tag? We quantify the similarity between a generated tag and the
ground truth (GT) tag by the Jaccard similarity between the tag documentation
words. Fig. 4 compares the errors by our best model and errors by the closest
baseline. For a majority (60%) of errors by our best model, the wrongly
predicted tag is very similar to the ground truth tag (Jaccard similarity
between tags $\geq 0.6$). Whereas, most of the tags wrongly predicted by
AttentionXML are very different from the ground truth tags (Jaccard similarity
between tags $\leq 0.4$).

Figure 4: Comparing errors by best proposed model and those by the closest
baseline. Even when our model generates incorrect tags, most of them are
semantically very similar to the ground truth tags. But AttentionXML often
generates completely unrelated tags.

Finally, Table 6 demonstrates instances where the predicted tag documentation
closely resembled the ground truth (GT) tag documentation, but even minor
variations led to a wrong final tag prediction. As illustrated in Table 6,
subtle differences between ‘from other party’ (in the GT tag, highlighted in
green) and ‘to other party’ (in the prediction, highlighted in red), or
between ‘dividends paid’ (in the GT tag, highlighted in green) and ‘dividends
declared’ (in the prediction, highlighted in red) can lead to a wrong final
tag prediction. To address these complexities, in future we plan to
incorporate external financial knowledge.

###  9.4 Comparison with ChatGPT

We used ChatGPT (GPT-3.5 turbo) under a few-shot setting. For a fair
comparison, we used the same prompt (shown in Fig. 2) that is used to
instruction tune our proposed models. The ChatGPT prompt structure was:
<Instruction, Task statement, Question for a specific numeral>. In order to
guide ChatGPT what to generate, we augment the prompt with 5 exemplars of
(input, desired output) pairs, thereby making it a 5-shot setting. Based on
experimental observations, we set “desired outputs” to be the labels instead
of label descriptions. Hence, in our final setting, no label descriptions were
fed. On the same 500 test set samples used to report ChatGPT’s performance
(macro-f1 of 9.08) in Table 2, we obtain macro-f1 score of 58.25 of our best
model when trained to generate labels (note that Table 4 last row reports
scores on the full test set). We believe that ChatGPT’s lower performance is
owing to two main factors: 1) It is not fine-tuned; rather, we use it in a
5-shot (few-shot) setting. 2) ChatGPT was not specifically pretrained on
financial data and the challenging nature of the FNXL dataset further makes it
difficult. Further analysis can be found in Appendix A.2.

###  9.5 Experimental comparison among models

As stated previously, given the unavailability of generative baselines for the
task, we had to first train and compare a suite of generative baselines as
reported in Table 2. This initial step allowed us to establish a fair
comparison among models of similar sizes before showcasing the incremental
performance gains with larger models. Table 7 presents a comparison of the
number of trainable parameters in the different baselines, including the ones
(generative) we trained, and our best-performing model. Also, a comparison of
training times per epoch of the different models is shown in Table 8. T5-Base
(220M), the weakest generative baseline trained by us, is comparable with
Label Semantics, the largest existing non-generative baseline (218M). From
Table 2, T5-Base already outperforms Label Semantics with 18.56% Macro-F1 and
11.17% Hits@1 gains. T5-Base also outperforms AttentionXML (112M)
substantially as reported above. Also, while FLAN-T5-Large (780M) is larger
than AttentionXML, our best results are obtained when we instruction-tune
FLAN-T5-Large with LoRA, thereby drastically reducing the number of trainable
parameters (0.59M), substantially lesser than AttentionXML (112M). Yet, we
significantly outperform AttentionXML with 39.3% Macro-F1 and 17.2% hits@1
gains.

Model | Trainable parameters
---|---
FINER |  109M
Label Semantics |  219M
GalaXC |  41M
AttentionXML |  112M
T5-Base |  220M
T5-Large |  739M
Flan-T5-Large |  780M
Flan-T5-Large w/ LoRA |  0.59M
Table 7: Comparison of number of trainable parameters between the baselines and FLAN-T5-Large with LoRA Model | Training time per epoch
---|---
FINER | 12 hours
Label Semantics | 9 hours
GalaXC | 1.2 hours
AttentionXML | 4 hours
Flan-T5-Large w/ LoRA | 56 minutes
Table 8: Comparison of training time per epoch between the baselines and
FLAN-T5-Large with LoRA

##  10 Conclusion

This work proposes a generative approach to solve the financial numeric
labelling task. We propose a novel FLAN-FinXC framework, that makes use of
parameter-efficient instruction tuning of LLMs for this extreme labelling
task. While comparing with the state-of-the-art models and various competitive
baselines that we devise for the task, we find that our best model,
Flan-T5-Large with LoRA, achieves huge improvements, providing a Macro-F1 of
66.23% as compared to the previously reported best numbers of 47.54% for FNXL
dataset. For adapting to other tasks, our approach requires only minor task-
specific modifications to the input prompt, and the target output. Moreover,
our approach offers flexibility in using either the class labels or label
descriptions, depending on the available data. As potential future directions,
we believe the scope to include more financial knowledge and integrate a
human-AI feedback loop would be the way forward to improve performance of this
challenging task.

##  11 Limitations

In this work, we have not integrated external financial knowledge to address
the subtle differences between tags as identified in our error analysis. Also,
we have observed that labeling numerals solely based on sentence-level text
(as done in this work) can be challenging, since the context depends on the
surrounding paragraph, associated tables, and other elements which are not
used in this work. Incorporating such elements as well as external financial
domain knowledge into a financial numeral labeling model would be interesting
future works.

## References

  * Chung et al. (2022) Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022.  Scaling instruction-finetuned language models.  _arXiv preprint arXiv:2210.11416_.
  * Dahiya et al. (2021) Kunal Dahiya, Deepak Saini, Anshul Mittal, Ankush Shaw, Kushal Dave, Akshay Soni, Himanshu Jain, Sumeet Agarwal, and Manik Varma. 2021.  Deepxml: A deep extreme multi-label learning framework applied to short text documents.  In _Proceedings of the 14th ACM International Conference on Web Search and Data Mining_ , pages 31–39.
  * Ding et al. (2023) Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. 2023.  Parameter-efficient fine-tuning of large-scale pre-trained language models.  _Nature Machine Intelligence_ , 5(3):220–235.
  * Hu et al. (2021) Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021.  Lora: Low-rank adaptation of large language models.  _arXiv preprint arXiv:2106.09685_.
  * Li and Liang (2021) Xiang Lisa Li and Percy Liang. 2021.  Prefix-tuning: Optimizing continuous prompts for generation.  _arXiv preprint arXiv:2101.00190_.
  * Longpre et al. (2023) Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, and Adam Roberts. 2023.  The flan collection: Designing data and methods for effective instruction tuning.
  * Loukas et al. (2022) Lefteris Loukas, Manos Fergadiotis, Ilias Chalkidis, Eirini Spyropoulou, Prodromos Malakasiotis, Ion Androutsopoulos, and Georgios Paliouras. 2022.  Finer: Financial numeric entity recognition for xbrl tagging.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 4419–4431.
  * Ma et al. (2022) Jie Ma, Miguel Ballesteros, Srikanth Doss, Rishita Anubhai, Sunil Mallya, Yaser Al-Onaizan, and Dan Roth. 2022.  Label semantics for few shot named entity recognition.  _arXiv preprint arXiv:2203.08985_.
  * Ni et al. (2021) Jianmo Ni, Gustavo Hernández Ábrego, Noah Constant, Ji Ma, Keith B Hall, Daniel Cer, and Yinfei Yang. 2021.  Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models.  _arXiv preprint arXiv:2108.08877_.
  * OpenAI (2022) TB OpenAI. 2022.  Chatgpt: Optimizing language models for dialogue. openai.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _J. Mach. Learn. Res._ , 21(1).
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-bert: Sentence embeddings using siamese bert-networks.  _arXiv preprint arXiv:1908.10084_.
  * Saini et al. (2021) Deepak Saini, Arnav Kumar Jain, Kushal Dave, Jian Jiao, Amit Singh, Ruofei Zhang, and Manik Varma. 2021.  Galaxc: Graph neural networks with labelwise attention for extreme classification.  In _Proceedings of the Web Conference 2021_ , pages 3733–3744.
  * Shah et al. (2022) Raj Sanjay Shah, Kunal Chawla, Dheeraj Eidnani, Agam Shah, Wendi Du, Sudheer Chava, Natraj Raman, Charese Smiley, Jiaao Chen, and Diyi Yang. 2022.  When flue meets flang: Benchmarks and large pre-trained language model for financial domain.  _arXiv preprint arXiv:2211.00083_.
  * Sharma et al. (2023) Soumya Sharma, Subhendu Khatuya, Manjunath Hegde, Afreen Shaikh, Koustuv Dasgupta, Pawan Goyal, and Niloy Ganguly. 2023.  Financial numeric extreme labelling: A dataset and benchmarking.  In _Findings of the Association for Computational Linguistics: ACL 2023_ , pages 3550–3561.
  * Wu et al. (2023) Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. 2023.  Bloomberggpt: A large language model for finance.  _arXiv preprint arXiv:2303.17564_.
  * Yang et al. (2023) Hongyang Yang, Xiao-Yang Liu, and Christina Dan Wang. 2023.  Fingpt: Open-source financial large language models.
  * You et al. (2019) Ronghui You, Zihan Zhang, Ziye Wang, Suyang Dai, Hiroshi Mamitsuka, and Shanfeng Zhu. 2019.  Attentionxml: Label tree-based attention-aware deep model for high-performance extreme multi-label text classification.  _Advances in Neural Information Processing Systems_ , 32.
  * Zhao et al. (2023) Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023.  A survey of large language models.

##  Appendix A Appendix

In this section, we provide some supplementary materials that enhance the
content presented in the main paper.

###  A.1 XBRL Tag Documentation

Each XBRL tag is associated with a pre-defined textual description known as
its ‘tag documentation’. Table 9 shows some more examples of XBRL tags and
their documentations. Note that, while certain tag-pairs may exhibit subtle
distinctions, their accompanying documentations vary significantly. Domain
experts leverage the information provided in the documents during manual
annotation. Our proposed model is also designed to utilize these
documentations.

Tag |  Documentation
---|---
common stocks shares issued |  Total number of common shares of an entity that have been sold or granted to shareholders (includes common shares that were issued, repurchased and remain in the treasury). These shares represent capital invested by the firm’s shareholders and owners, and may be all or only a portion of the number of shares authorized. Shares issued include shares outstanding and shares held in the treasury.
common stock shares authorized |  The maximum number of common shares permitted to be issued by an entity’s charter and bylaws.
tax credit carry forward amount |  The amount of the tax credit carryforward, before tax effects, available to reduce future taxable income under enacted tax laws.
tax credit carry forward valuation allowance |  Amount of valuation allowance pertaining to the deferred tax asset representing potential future taxable deductions from tax credit carryforwards for which it is more likely than not that a tax benefit will not be realized.
due to affiliate noncurrent |  Amount of receivables owed to an entity that is affiliated with the reporting entity by means of direct or indirect ownership, which are usually due after one year (or one business cycle, if longer).
due to affiliate current and non current |  Amount of payable due to an entity that is affiliated with the reporting entity by means of direct or indirect ownership.
notes payable related parties classified current |  The amount for notes payable (written promise to pay), due to related parties. Used to reflect the current portion of the liabilities (due within one year or within the normal operating cycle if longer).
notes payable related parties current and non current |  The amount for notes payable (written promise to pay), due to related parties.

Table 9: Examples of XBRL tag documentations. While some tag-pairs exhibit
very subtle distinctions, their accompanying documentations vary
significantly. Our model takes advantage of the distinctions between the tag
documentations.

###  A.2 Comparison with ChatGPT

With the emergence of ChatGPT, we were curious to check its performance on
this task using the same instruction prompt for a few samples. As can be seen
from the examples in Table 10, ChatGPT struggles to correctly identify the
appropriate tags in the majority of cases, and in some cases, it
misunderstands the context entirely. This indicates that ChatGPT has still
very limited performance in financial numerical tagging.

Instruction: First, read the task description. There could be multiple
numerals reported …. Sentence: Now read the following financial statement. At
April24, 2020 the estimated fair value was $27.1 billion compared to a
principal value of$24.5 billion.
---
Question: What is the tag associated with the numeral 27.1?
GT Tag: long term debt fair value
ChatGPT: estimated fair value
FLAN-FinXC: long term debt fair value
Sentence: Ordinary shares - par value $0.0001, 2.6 billion shares authorized,
1,345,400,671 and 1,341,074,724 shares issued and outstanding, respectively
Question: What is the tag associated with the numeral 2.6?
GT Tag: common stock shares authorized
ChatGPT tag: ordinary shares authorized
FLAN-FinXC: common stock shares authorized
Sentence: Share Capital Medtronic plc is authorized to issue 2.6 billion
Ordinary Shares, $0.0001 par value; 40 thousand Euro Deferred Shares, €1.00
par value; 127.5 million Preferred Shares, $0.20 par value; and 500 thousand A
Preferred Shares, $1.00 par value.
Question: What is the tag associated with the numeral 500?
GT Tag: preferred stock shares authorized
ChatGPT: a preferred shares authorized
FLAN-FinXC: preferred stock shares authorized
Instruction: …… Sentence: At April26, 2019, $764 million of rebates were
classified as other accrued expenses and $432 million of rebates were
classified as a reduction of accounts receivable in the consolidated balance
sheets.
Question: What is the tag associated with the numeral 432?
GT Tag: contract with customer refund liability current
ChatGPT: rebates reduction of accounts receivable
FLAN-FinXC: contract with customer refund liability
Sentence: As of December 31, 2018, we expect to receive total future rental
income of $203 million related to noncancelable subleases for abandoned
facilities.
Question: What is the tag associated with the numeral 203?
GT Tag: operating leases future minimum payments receivable
ChatGPT tag: future rental income from noncancelable subleases
FLAN-FinXC: operating leases rent expense sublease rentals 1
Table 10: Comparison between ChatGPT and our best FLAN-FinXC model variant’s
prediction

###  A.3 Error Analysis

Table 11 demonstrates instances where the generated tag documentation (by our
best model) closely resembled the ground truth (GT) tag documentation, but
even minor variations led to a wrong final tag prediction. In the financial
domain, subtle word changes can drastically alter the context. As illustrated
in Table 6, subtle differences between ‘from other party’ (in the GT tag,
highlighted in green) and ‘to other party’ (in the prediction, highlighted in
red), or between ‘dividends paid’ (in the GT tag, highlighted in green) vs
‘dividends declared’ (in the prediction, highlighted in red), or between ‘cash
outflow’ and ‘cash inflow’, ‘loss’ vs ‘damages’ can lead to a wrong final tag
prediction.

Instruction: First, read the task description. There could be multiple
numerals reported in a financial statement……Sentence: On January 15, 2020, the
parties agreed to a settlement in principle to resolve the lawsuit, which will
require a payment of $550 million by us and is subject to approval by the
court. Question: What is the tag associated with the numeral 550?
---
GT Tag Doc: amount of loss contingency liability. GT Tag: loss contingency
accrual at carrying value
Flan-Large Generated: Amount awarded damages contingency liability.
Flan-Large +Tag Matcher: amount of damages awarded to the plaintiff in the
legal matter.
Pred Tag-Words: loss contingency damages awarded value
Sentence: In fiscal year 2016, Bard paid the Company $121 million towards the
settlement of 11,000 of these claims. Question: What is the tag associated
with the numeral 121?
GT Tag Doc: amount awarded from other party in judgment or settlement of
litigation. GT Tag: litigation settlement amount awarded from other party
Flan-Large Generated: Amount awarded to other party in judgment or settlement
of litigation
Flan-Large +Tag Matcher: amount awarded to other party in judgment or
settlement of litigation.
Pred Tag-Words: litigation settlement amount awarded to other party
Sentence: During the year ended December 31, 2017, we issued and repaid an
aggregate of $12.3 billion of commercial paper and had a maximum outstanding
balance of $1.5 billion under our commercial paper program. Question: What is
the tag associated with the numeral 12.3?
GT Tag Doc: the cash outflow due to repaying amounts borrowed by issuing
commercial paper.GT Tag: repayments of commercial paper
Flan-Large Generated: The cash inflow during to theing short borrowed under
issuing commercial paper.
Flan-Large +Tag Matcher: the cash inflow from borrowing by issuing commercial
paper.
Pred Tag-Words: proceeds from issuance of commercial paper
Sentence: Our Board of Directors declared quarterly dividends per share of
$1.45, $1.32 and $1.15, which were paid in each of the four quarters of 2019,
2018, and 2017, respectively. Question: What is the tag associated with the
numeral 1.45?
GT Tag Doc: aggregate  dividends paid during the period for each share of
common stock outstanding. GT Tag: common stock dividends per share cash paid
Flan-Large Generated: Aggregate dividends declared during the period for each
share of common stock outstanding.
Flan-Large +Tag Matcher: aggregate dividends declared during the period for
each share of common stock outstanding.
Pred Tag-Words: common stock dividends per share declared
Instruction: …..Sentence: As of December 31, 2020, Duong met the held-for-sale
criteria and loan receivable balance of $1.3 billion, net of CECL reserve of
$32 million was reclassified Question: What is the tag associated with the
numeral 32?
GT Tag Doc: amount of allowance for credit loss on accounts receivable. GT Tag
Words: allowance for doubtful accounts receivable
Flan-Large Generated: othersmount of allowance for credit loss on financing
receivable, A
Flan-Large +Tag Matcher: amount of allowance for credit loss on financing
receivable, classified as noncurrent.
Pred Tag-Words: allowance for notes and loans receivable noncurrent
Table 11: Error cases. Examples to show the subtle difference between ground
truth (shown in green color) and generated tag docs (shown in red color) and
predicted tag.


# Line emission from filaments in molecular clouds

F. D. Priestley1, D. Arzoumanian2 & A. P. Whitworth1
1School of Physics and Astronomy, Cardiff University, Queen’s Buildings, The
Parade, Cardiff CF24 3AA, UK
2Division of Science, National Astronomical Observatory of Japan, 2-21-1
Osawa, Mitaka, Tokyo 181-8588, Japan
Email<EMAIL_ADDRESS>
(Accepted XXX. Received YYY; in original form ZZZ)

###### Abstract

Filamentary structures are often identified in column density maps of
molecular clouds, and appear to be important for both low- and high-mass star
formation. Theoretically, these structures are expected to form in regions
where the supersonic cloud-scale turbulent velocity field converges. While
this model of filament formation successfully reproduces several of their
properties derived from column densities, it is unclear whether it can also
reproduce their kinematic features. We use a combination of hydrodynamical,
chemical and radiative transfer modelling to predict the emission properties
of these dynamically-forming filaments in the 13CO, HCN and N2H+ $J=1-0$
rotational lines. The results are largely in agreement with observations; in
particular, line widths are typically subsonic to transonic, even for
filaments which have formed from highly supersonic inflows. If the observed
filaments are formed dynamically, as our results suggest, no equilibrium
analysis is possible, and simulations which presuppose the existence of a
filament are likely to produce unrealistic results.

###### keywords:

astrochemistry – stars: formation – ISM: molecules – ISM: clouds – ISM:
structure

††pubyear: 2023††pagerange: Line emission from filaments in molecular clouds–A

##  1 Introduction

Molecular clouds, the birthplaces of stars, are typically filamentary in
nature (see Pineda et al. (2022) and Hacar et al. (2022) for recent reviews).
The majority of prestellar cores, the objects expected to eventually form a
star or stellar system, are observed to be associated with filaments (Könyves
et al., 2015, 2020), suggesting that these filaments may play a significant
role in star formation. Filaments may fragment and thus subsequently form
these cores (e.g. Clarke et al., 2017), and channel accreting material onto
them (Peretto et al., 2013; Anderson et al., 2021). They may be formed by the
interaction between gas dynamics and magnetic fields (Pattle et al., 2021;
Arzoumanian et al., 2021). Understanding the formation of these structures is
therefore necessary in order to understand star formation.

In hydrodynamical simulations of molecular clouds, filamentary structures form
naturally as a result of supersonic turbulence (Kirk et al., 2015; Federrath,
2016; Priestley & Whitworth, 2020), as regions where the turbulent velocity
field converges tend to produce elongated overdensities which manifest as
filaments in projected maps of the column density. This formation mechanism
reproduces many of the observationally-derived properties of filaments in
molecular clouds (Priestley & Whitworth, 2022): the typical width of $\sim
0.1\,{\rm pc}$ (Arzoumanian et al., 2011, 2019); the detailed shape of the
width distribution (Panopoulou et al., 2017; Arzoumanian et al., 2019); the
relatively flat density profiles (compared to an isothermal filament in
hydrostatic equilibrium; Palmeirim et al. 2013; Whitworth et al. 2021); and
the lack of correlation between peak surface density and filament width
(Arzoumanian et al., 2011, 2019).

The agreements between theory and observation outlined above all rely on
column densities derived from submillimetre dust continuum observations. These
column densities rely on assumptions such as a uniform line-of-sight grain
temperature, which may not be the case in reality, and could substantially
distort the results (Howard et al., 2019, 2021). Confusion between overlapping
line-of-sight structures can also affect the inferred filament properties
(e.g. Juvela et al., 2012). Dynamical properties of filaments derived from
molecular line emission, such as sub- to transonic molecular line widths
(Arzoumanian et al., 2013; Hacar et al., 2013, 2018; Suri et al., 2019), are
not obviously consistent with a picture of filaments forming via supersonic
turbulent flows. In this paper, we investigate the predicted line emission
from such a model, and thereby establish the viability of this picture as a
description of real filaments.

##  2 Method

We consider a hydrodynamical model of filament formation in a cylindrically-
symmetric converging flow. A full description is given in Priestley &
Whitworth (2022); we briefly summarise the key features here. The initial
state of the model is a uniform-density cylinder of isothermal gas, described
by two dimensionless parameters: ${\cal G}$, the ratio of the cylinder’s line
density to the critical value for stability against gravitational collapse,
$\mu_{\rm CRIT}=2a_{0}^{2}/G$, where $a_{0}$ is the isothermal sound speed and
$G$ is the gravitational constant (Stodólkiewicz, 1963; Ostriker, 1964); and
${\cal M}$, the Mach number of the radial inflow. For an initial gas density
$\rho_{0}$, inflow velocity $v_{0}$, and cylinder radius $W_{0}$, ${\cal
G}=\pi G\rho_{0}W_{0}^{2}/2a_{0}^{2}$ and ${\cal M}=v_{0}/a_{0}$. We
investigate model filaments with ${\cal G}=1.2$, which are marginally unstable
to collapse, and with ${\cal M}=1$ and $3$, corresponding to transonic and
supersonic flows. Radial distance from the axis of symmetry is given by the
symbol $w$, and the projected distance (i.e. for surface density profiles) by
$b$.

We set the isothermal sound speed to $a_{0}=0.187\,{\rm km\,s^{-1}}$
(appropriate for molecular gas at $10\,{\rm K}$) and the cylinder radius to
$W_{0}=1\,{\rm pc}$, giving an initial gas density $\rho_{0}=4.2\times
10^{-22}\,{\rm g\,cm}^{-3}$, and a hydrogen nuclei density $n_{\rm
H,0}=180\,{\rm cm}^{-3}$. The inflow velocities are then $v_{0}=0.187\,{\rm
km\,s^{-1}}$ and $0.561\,{\rm km\,s^{-1}}$ for the Mach 1 and 3 models
respectively. The initial cylinder is surrounded by material with a density
ten times lower and a temperature ten times higher than that within $W_{0}$,
to ensure pressure balance across the boundary. We simulate the evolution of
these models using the phantom smoothed-particle hydrodynamics (SPH) code
(Price et al., 2018), with $5\times 10^{6}$ particles per model. Our SPH
simulations are of a $5\,{\rm pc}$ long cylinder, rather than one of infinite
length, but this is a sufficient length to avoid any significant edge effects
(e.g. Clarke & Whitworth, 2015); the inflow dominates the filament evolution,
rather than gravitational contraction along the longitudinal axis.

We randomly select $10^{5}$ SPH particles within a $1\,{\rm pc}$ radius of the
simulation origin, and track their chemical evolution using the time-dependent
uclchem code (Holdship et al., 2017) with the UMIST12 reaction network
(McElroy et al., 2013). Elemental abundances are taken from Sembach et al.
(2000), and we assume a cosmic ray ionisation rate of $1.3\times
10^{-17}\,{\rm s^{-1}}$ and an external ultraviolet (UV) radiation field of
$1.7$ Habing units (Habing, 1968). This UV field strength is the value
typically assumed for the Solar neighbourhood (Jura, 1974; Draine, 1978);
filaments located near sites of active star formation, such as the Orion
filaments in Suri et al. (2019), may be exposed to significantly stronger
external UV fields. The column density shielding each particle from the UV
field, $N_{\rm H}$, is calculated by integrating the filament density profile
radially outwards from the particle position, with a conversion factor $A_{\rm
V}=6\times 10^{-22}N_{\rm H}\,{\rm cm^{2}\,mag}$ from Bohlin et al. (1978).
This is certain to underestimate the true level of shielding; any sightline
other than radially outward will have a higher line-of-sight column density.
We therefore also consider models with no external UV field, representing the
extreme limit of total shielding, although the UV field generated by cosmic
ray interactions is still modelled.

Molecular line intensity position-position-velocity (PPV) cubes are created
using the lime radiative transfer code (Brinch & Hogerheijde, 2010), with
collisional data taken from the LAMDA database (Schöier et al., 2005) and dust
properties from Ossenkopf & Henning (1994) with thin ice mantles. We use
$10^{5}$ sampling points, each of which is assigned the properties of the
closest post-processed SPH particle. The spatial and velocity resolutions of
the PPV cubes are $0.01\,{\rm pc}$ and $0.015\,{\rm km\,s^{-1}}$ respectively.
We focus on the $J-1-0$ rotational transitions of 13CO, HCN and N2H+. The 13CO
line is excited in relatively low-density gas ($\sim 100\,{\rm cm}^{-3}$;
Clark et al. 2019) compared to the latter two, so should trace all of the
filament material to some extent. The HCN and N2H+ emission should
preferentially come from the densest regions, as these lines have similarly
high ($\gtrsim 10^{4}\,{\rm cm}^{-3}$) critical densities (Shirley, 2015),
with the HCN line typically being optically thick and so potentially showing
infall signatures (Myers et al., 1996). Our reaction network does not include
isotopic chemistry, so we assume the 12CO/13CO abundance ratio reflects an
underlying 12C/13C ratio of $77$ (Wilson & Rood, 1994). HCN and N2H+ have
hyperfine structure, which we neglect in the radiative transfer calculations
due to the significantly higher computational cost of including it, but we
argue in Appendix A that this has little impact on our main results.

Figure 1: Density (left) and radial velocity (right) profiles for the ${\cal
M}=1$ model (red) at $1.94$ (dotted line), $2.09$ (dashed line) and
$2.18\,{\rm Myr}$ (solid line), and the ${\cal M}=3$ model (black) at $0.80$
(dotted line), $0.99$ (dashed line) and $1.09\,{\rm Myr}$ (solid line). The
radial distance from the axis of symmetry is denoted by $w$.

Figure 2: Line profiles of the $J=1-0$ rotational transitions of 13CO (blue),
HCN (orange) and N2H+ (green), for ${\cal M}=1$ (top row) and $3$ (bottom
row), evolving with time/central density from left to right.

Figure 3: Peak line intensities (left) and FWHMs (right) of the $J=1-0$
rotational transitions of 13CO (blue), HCN (orange) and N2H+ (green) versus
filament central density, for ${\cal M}=1$ (circles) and $3$ (squares). HCN
widths are not shown, because the asymmetric self-absorbed line profile
(Figure 2) makes the FWHM a poor representation of the underlying velocity
dispersion. The isothermal sound speed for molecular gas at $10\,{\rm K}$,
$a_{0}=0.187\,{\rm km\,s^{-1}}$, is marked in the right panel by a dashed
black line. Figure 4: Line profiles of the $J=1-0$ rotational transitions of
HCN (solid line) and H13CN (dashed line), which has been scaled up by the
assumed HCN/H13CN abundance ratio of 77 so that for optically thin emission,
the line profiles would be identical.

Figure 5: Line profiles of the $J=1-0$ rotational transitions of 13CO (blue),
HCN (orange) and N2H+ (green), for ${\cal M}=1$ (left) and $3$ (right), and an
external UV field of $1.7$ (solid lines) or $0$ (dashed lines) Habing (1968)
units.

Figure 6: Abundance profiles of CO (left), N2H+ (centre) and e- (right) for
the ${\cal M}=3$ model at $0.99\,{\rm Myr}$, for an external UV field of $1.7$
(solid lines) or $0$ (dashed lines) Habing (1968) units. The radial distance
from the axis of symmetry is denoted by $w$.

Figure 7: Integrated intensity profiles perpendicular to the filament spine,
normalised to the peak value, of the $J=1-0$ rotational transitions of 13CO
(blue), HCN (orange) and N2H+ (green), and of the $850\,{\rm\mu m}$ thermal
dust emission (black), for ${\cal M}=1$ (left) and $3$ (right). The projected
distance from the axis of symmetry is denoted by $b$.

##  3 Results

Figure 1 shows the density and radial velocity profiles111These profiles are
averaged over the $z$ axis between $\pm 1\,{\rm pc}$, but differences between
the average profiles and individual cuts are minor (Priestley & Whitworth,
2022).  for the ${\cal M}=1$ and ${\cal M}=3$ models at three epochs, chosen
so that the corresponding model with a different ${\cal M}$ value has a
comparable central density. As discussed in Priestley & Whitworth (2022), the
gradient of the density profiles undergoes an abrupt steepening at the
location of the accretion shock, clearly visible in the velocity profiles as a
sharp transition from supersonic to subsonic inflow. This serves as a useful
distinction between the central filament and the ambient material out of which
the filament has formed. The location of the filament boundary moves inwards
with time, while the peak inflow velocity of the ambient material increases
due to the gravitational attraction of the central filament. Filaments with
${\cal M}=1$ are broader than their ${\cal M}=3$ counterparts at equal central
density due to the lower ram pressure of the accretion flow; the density
profiles are not solely set by self-gravity, resulting in shallower slopes
than the hydrostatic equilibrium solution (Priestley & Whitworth, 2022), as is
observed (Palmeirim et al., 2013).

Figure 2 shows the 13CO, HCN and N2H+ line profiles222We average the line
profiles within a radius of $0.05\,{\rm pc}$ of the origin, approximating the
effect of typical single-dish beam sizes for nearby ($\sim 100\,{\rm pc}$)
filaments. As the line emission is dominated by the central filament,
increasing the averaging radius has a negligible effect on the shape of the
profiles, although the averaged intensities are lower. at different
evolutionary stages. The 13CO line is stronger and broader than the others,
and is typically symmetric, although in some cases ‘wings’ around the central
line corresponding to the inflowing material are visible. HCN profiles are
asymmetric, showing the characteristic infall signature of redshifted self-
absorption. However, this self-absorption is not due to the inflowing
material, which can also be seen as extended wings in some cases. The velocity
of the self-absorption is much lower than that of the inflow, and arises
instead from the gravitational contraction of the dense central filament. N2H+
profiles are symmetric, showing no self-absorption, and have negligible
emission from the inflowing material.

Figure 3 shows the evolution of line peak intensities and full-widths at half
maximum (FWHMs) with central density. Line properties are similar between the
two values of ${\cal M}$ when the central densities are comparable, making
this a much better proxy for evolutionary stage than the age; the ${\cal M}=3$
model terminates (defined as the point at which it begins to fragment along
its longitudinal axis; Priestley & Whitworth 2022) before the ${\cal M}=1$
model shows any detectable emission from lines other than 13CO. The peak
intensity of the 13CO line decreases as the filament evolves, due to depletion
of the molecule onto grain surfaces as the density rises. Peak intensities of
both the HCN and N2H+ lines increase monotonically with central density. This
suggests that the ratio of the peak intensity of lines tracing denser gas to
those of CO isotopologues may serve as an evolutionary indicator, with higher
values indicating relatively more evolved filaments, although the
normalisation of this indicator is likely to be too sensitive to the local
conditions to accurately diagnose the central filament volume density. The
peak intensities of the HCN and N2H+ lines will also depend on which hyperfine
component is being measured (Appendix A), but in all cases should still
increase relative to 13CO with evolutionary stage.

Line FWHMs, by contrast, show relatively little evolution, being sub- to
transonic throughout the filament lifetime, as observed for filaments with
${\cal G}\sim 1$ (Arzoumanian et al., 2013; Hacar et al., 2013).333Arzoumanian
et al. (2013) find that filaments with line densities significantly in excess
of $\mu_{\rm CRIT}$ tend to have higher velocity dispersions, indicating
supersonic non-thermal motions. This may be a sign of rapid infall, caused by
the much greater importance of self-gravity in these objects. Notably, 13CO
FWHMs from the ${\cal M}=1$ model are generally higher than those of the
${\cal M}=3$ model, despite the lower inflow velocity. This is most apparent
at the lowest central densities ($\sim 6\times 10^{4}\,{\rm cm}^{-3}$), where
the ${\cal M}=3$ 13CO FWHM is $\sim 0.2\,{\rm km\,s^{-1}}$ below the
equivalent ${\cal M}=1$ value. This appears to be due to blending of the
emission from the central filament and the inflowing material, as the ${\cal
M}=1$ inflow velocities only shift the line centroids by approximately one
thermal linewidth. This effect, combined with additional opacity broadening
(Hacar et al., 2016b), makes the 13CO linewidth a poor tracer of the velocity
dispersion within the filament, which it will inevitably overestimate. The
N2H+ line, with a much lower opacity and a negligible contribution from the
inflowing material, is more representative of the conditions in the central
filament, which from Figure 1 are clearly subsonic at all times. Smith et al.
(2012) obtained similar results for the N2H+ line from studying filaments
formed in a larger-scale simulation of a turbulent molecular cloud, suggesting
that subsonic line widths occur generically, even in scenarios less idealised
than our models.

We do not show HCN FWHMs in Figure 3 because the self-absorbed line profile
makes this an unrealiable tracer of the intrinsic line width. Figure 4
compares the HCN line profile to that of H13CN, an optically-thin
isotopologue.444Collisional data for H13CN are not available, so we assume the
same properties as HCN, with the only difference being the molecular abundance
(reduced by a factor of 77 for H13CN). As with N2H+, the intrinsic velocity
dispersion of the line, as represented by that of the symmetric H13CN profile,
is clearly subsonic.

As noted above, we have assumed what is effectively a lower limit for the
column density shielding the filament material from any background UV field.
Figure 5 compares these results with line profiles resulting from the extreme
case of no background UV field whatsoever. This results in stronger N2H+
emission, particularly for the ${\cal M}=1$ model, but line widths are not
greatly affected, although this is difficult to quantify due to the asymmetry
of the no-UV profiles. This increase in intensity is due to the main
destruction mechanism outside the filament centre changing from dissociative
recombination with free electrons to reactions with CO. Figure 6 shows the
abundance profiles of the relevant species for the ${\cal M}=3$ model. With no
external UV field, almost all carbon is in the form of CO at all radii,
whereas a non-zero UV flux produces an extensive region of photoionised C+
with a much higher electron density. The dissociative recombination rate for
N2H+ is significantly higher than that of reactions with CO (McElroy et al.,
2013), so even a moderate electron density causes a sharp drop in the N2H+
abundance. With no UV field, the abundance of the main source of destruction
is almost unchanged ($\sim 10^{-4}$), but the reaction rate is much lower, and
N2H+ maintains a roughly constant abundance throughout the filament, although
the densities at large radii are too low to excite detectable line emission.

The 13CO line profiles, on the other hand, are significantly altered by the
removal of an external UV field; without photodissociation of the molecule in
the lower-density accreting material, the greatly increased abundance (Figure
6) causes additional peaks to appear at velocities corresponding to the
initial inflow, which are blended with the central filament emission for the
${\cal M}=1$ model. These predicted line properties (multiple peaks and/or
substantially broadened emission) are quite unlike those observed for CO
isotopologues (Arzoumanian et al., 2013; Panopoulou et al., 2014; Hacar et
al., 2016a; Suri et al., 2019). If our model is applicable to real filaments,
this suggests that they cannot be so deeply embedded in ambient material as to
be substantially shielded from the background UV field.

The sample of filaments in Arzoumanian et al. (2019) has a range of background
column densities spanning $1-10\times 10^{21}\,{\rm cm}^{-2}$, which
corresponds to extinctions associated with the parent cloud material of
$0.6-6\,{\rm mag}$ (Bohlin et al., 1978). However, the relevant quantity for
shielding is the effective $A_{\rm V}$, a weighted average over all lines of
sight from the filament itself (Glover & Mac Low, 2007). This quantity is
typically lower than the line-of-sight $A_{\rm V}$, because low-$A_{\rm V}$
sightlines contribute disproportionately to the total photodissociation rate.
Clark & Glover (2014) find that observed extinctions of up to $20-30$ mag only
correspond to $\sim 2-3$ mag of effective shielding, so even the highest
background values from Arzoumanian et al. (2019) are still consistent with the
filaments being at most moderately shielded ($A_{\rm V}\lesssim 1\,{\rm mag}$)
from the external UV field, as we require. In regions of active star
formation, where the UV strength may be significantly higher than the Solar
neighbourhood value adopted here, an even higher effective $A_{\rm V}$ would
be required to reach a situation comparable to the no-UV case in Figure 5.

##  4 Discussion

###  4.1 Signatures of filament accretion

Our results suggest that in a scenario where filaments form via converging
turbulent flows, these accretion flows would be difficult to detect
observationally, even for supersonic velocities. The accreting material is
only apparent as relatively faint wings around the central line profile, even
for low-density tracers such as 13CO, unless this material is well-shielded
from the ambient UV field. Line widths are at most transonic, and do not
correspond to the original inflow velocity. Where lines show self-absorption
features commonly associated with inflows, such as the HCN profiles in Figure
2, these are associated with the central filament itself and not the material
accreting onto it. Distinct velocity components in lines other than those from
CO isotopologues, such as those in HC5N identified by Smith et al. (2023), are
likely to represent distinct dense filamentary structures, rather than
accretion onto a main filament. However, we note that this only holds when the
accreting material has a relatively low density, rather than itself being
shock-compressed (see Section 4.4).

###  4.2 Fragmentation and substructure

Our simulations are not designed to study substructure; they have cylindrical
symmetry, and are terminated before any longitudinal fragmentation can occur.
Nonetheless, the fact that the N2H+ $J=1-0$ line almost exclusively traces the
dense filament material may have some implications for the ‘fibres’ that are
identified in PPV space within observed filaments (Hacar et al., 2013, 2018;
Shimajiri et al., 2019b). Previous theoretical studies of filaments with
substructure have focused on transitions from CO isotopologues (Seifried et
al., 2017; Clarke et al., 2018), which likely includes emission from material
which has not yet been incorporated into a filament (or fibre). This may
account for the lack of correspondance between PPV structures, identified via
CO lines, and actual physical density structures (Clarke et al., 2018); lower-
density material has a disproportionate contribution to the observed line
intensity relative to its mass. N2H+ has the opposite behaviour (Tafalla et
al., 2021), preferentially highlighting the densest material. Substructures
identified via this line, and others with similar behaviour, are more likely
to represent genuinely separate physical entities.

###  4.3 Line widths in filaments and cores

Despite forming from supersonic turbulent motions, even our ${\cal M}=3$ model
displays at most transonic line widths, in agreement with observations of
filaments with comparable (${\cal G}\sim 1$) line densities. Similarly to
filaments, prestellar cores are generally observed to have sub- to transonic
line widths (e.g. Lee et al., 1999; Tafalla et al., 2002; Li et al., 2023),
but models of isolated cores almost inevitably develop supersonic infall
motions (Larson, 1969) which are predicted to be readily apparent in the
molecular line profiles (Yin et al., 2021; Priestley et al., 2022).555Models
with high magnetic field strengths predict line profiles in good agreement
with observations, but magnetic fields in prestellar cores are typically found
to be much weaker than the required values (Crutcher, 2012; Pattle et al.,
2022) This conflict is often resolved via highly extended initial density
profiles (Keto et al., 2015; Sipilä & Caselli, 2018; Sipilä et al., 2022),
which do not reach supersonic infall velocities before the formation of a
protostar, but these structures are well beyond the limit for gravitational
stability, and so represent quite implausible initial conditions to find in a
turbulent molecular cloud (Whitworth et al., 1996). Our results, like those of
Smith et al. (2012), suggest that structures formed from supersonic flows tend
to efficiently dissipate the original velocity, and that the subsequent
gravitational contraction has a much lower velocity than would be expected if
the object were initially at rest. This may provide an alternative explanation
for the narrow observed line widths in prestellar cores.

###  4.4 Limitations of cylindrical symmetry

Filaments are often found to have transverse velocity gradients (Arzoumanian
et al., 2018; Shimajiri et al., 2019a; Watkins et al., 2019; Chen et al.,
2022; Kim et al., 2022). While this is sometimes attributed to rotation around
the long axis (e.g. Álvarez-Gutiérrez et al., 2021), a more common
interpretation is that the gradient represents material being accreted from a
sheet-like structure at an inclination to the line of sight (Pineda et al.,
2022), so that lines are redshifted on one side of the filament and
blueshifted on the other. The sheet itself is often suggested to have formed
in the interface between two colliding molecular clouds (e.g. Balfour et al.,
2015; Balfour et al., 2017; Inoue et al., 2018; Abe et al., 2021). As our
models are cylindrically symmetric by construction, they are incapable of
reproducing these gradients, which would require angular variation in the
properties of the accreting gas. However, synthetic line observations of
turbulent molecular clouds (Priestley & Whitworth, 2020) find that the
resulting filamentary structures do in fact tend to have velocity gradients
perpendicular to their spines, even in the case of isolated clouds with no
collision-induced layer. This suggests that supersonic turbulence by itself is
capable of producing the sheet-like structures inferred observationally,
particularly as some degree of compression is necessary in order for the
accreting material to be detectable in line emission.

###  4.5 Filament widths in different tracers

It is expected theoretically (Priestley & Whitworth, 2020) and found
observationally (Shimajiri et al., 2023) that the same filament will appear to
have different widths when observed in different tracers. This is due to the
complex combination of excitation, abundance and optical depth effects which
relate the column density along the line of sight to the observed line
intensity. Figure 7 shows the spatial variation of the integrated line
intensities perpendicular to the filament spine. As the spatial resolution of
our radiative transfer models ($0.01\,{\rm pc}$) is much worse than that of
the underlying SPH simulation, we compare the line data to $850\,{\rm\mu m}$
dust continuum emission images produced with lime. These have identical
spatial resolution to the line PPV cubes, and should trace the column density
almost exactly, as we assume a constant dust temperature of $10\,{\rm K}$ and
the optical depth at this wavelength is extremely low ($\tau_{850}<10^{-4}$).
The filaments appear to be broader than the underlying density profile when
observed in 13CO, and narrower when observed in N2H+. This likely explains the
different width distributions found by studies in these two tracers
(Panopoulou et al., 2014; Hacar et al., 2018) when compared to those using
dust continuum emission (Arzoumanian et al., 2011, 2019).

The different widths can be understood as a consequence of the relationship
between intensity and column density of the two lines (Tafalla et al., 2021);
the intensity of CO isotopologues saturates at a relatively low hydrogen
column density of $N_{\rm H_{2}}\sim 2\times 10^{21}\,{\rm cm}^{-2}$, whereas
the N2H+ line only becomes visible above $10^{23}\,{\rm cm}^{-2}$. The 13CO
intensity therefore overrepresents the lower-density outer parts of the
filament, whereas the N2H+ line mainly traces the densest central regions.
Interestingly, the HCN line, which Tafalla et al. (2021) find to have a linear
relationship between intensity and column density, does in fact appear to
accurately trace the shape of the underlying density profile, at least in the
central part of the filament.

###  4.6 Preexisting versus dynamical filaments

It is common practice in theoretical studies to assume that a filament already
exists at the outset of the model, in that the initial radial density profile
is quite centrally condensed (e.g. Seifried & Walch, 2015, 2016; Gutiérrez-
Vera et al., 2023; Misugi et al., 2023). If filaments are actually structures
formed via turbulent accretion, this is inappropriate. Even if the initial
filament also has supersonic converging motions (e.g. Clarke et al., 2016),
there is no guarantee that the resulting density and velocity profiles will
resemble those in the more realistic scenario where no filament exists at the
outset, particularly given that the definition of what counts as a filament
seems to be strongly related to the accretion shock (Priestley & Whitworth,
2022). We suggest that future studies should focus on forming the objects of
interest self-consistently (e.g. Smith et al., 2012, 2013), in keeping with
their presumed origins from cloud-scale turbulent motions (Mac Low & Klessen,
2004).

##  5 Conclusions

The filaments identified in molecular clouds are thought to form due to
supersonic turbulent motions within these larger objects. We investigate the
observational consequences of this scenario, specifically the predicted
molecular line emission properties of filaments formed via supersonic flows.
These are largely consistent with data; in particular, line widths are trans-
to subsonic, even for highly supersonic inflows. The central filament
dominates the line emission, making it difficult to identify signs of the
accreting material. Signatures of infall, such as redshifted self-absorption
in the HCN $J=1-0$ line, are due to the contraction of the filament itself,
rather than the accretion flows it formed out of. Additional velocity
components would only be visible in the limit of almost total shielding from
the external UV field, and even then only for CO isotopologues. Our results
support the idea that filaments in molecular clouds are dynamically-forming
objects, which cannot be modelled as equilibrium structures, or considered in
isolation from their environment.

## Acknowledgements

We are grateful to Michael Anderson and Nicolas Peretto for their advice on
measuring velocity dispersions, and to the anonymous referee for a useful
report. FDP and APW acknowledge the support of a consolidated grant
(ST/K00926/1) from the UK Science and Technology Facilities Council (STFC).

## Data Availability

The data underlying this article will be shared on request.

## References

  * Abe et al. (2021) Abe D., Inoue T., Inutsuka S.-i., Matsumoto T., 2021, ApJ, 916, 83
  * Álvarez-Gutiérrez et al. (2021) Álvarez-Gutiérrez R. H., Stutz A. M., Law C. Y., Reissl S., Klessen R. S., Leigh N. W. C., Liu H. L., Reeves R. A., 2021, ApJ, 908, 86
  * Anderson et al. (2021) Anderson M., et al., 2021, MNRAS, 508, 2964
  * Arzoumanian et al. (2011) Arzoumanian D., et al., 2011, A&A, 529, L6
  * Arzoumanian et al. (2013) Arzoumanian D., André P., Peretto N., Könyves V., 2013, A&A, 553, A119
  * Arzoumanian et al. (2018) Arzoumanian D., Shimajiri Y., Inutsuka S.-i., Inoue T., Tachihara K., 2018, PASJ, 70, 96
  * Arzoumanian et al. (2019) Arzoumanian D., et al., 2019, A&A, 621, A42
  * Arzoumanian et al. (2021) Arzoumanian D., et al., 2021, A&A, 647, A78
  * Balfour et al. (2015) Balfour S. K., Whitworth A. P., Hubber D. A., Jaffa S. E., 2015, MNRAS, 453, 2471
  * Balfour et al. (2017) Balfour S. K., Whitworth A. P., Hubber D. A., 2017, MNRAS, 465, 3483
  * Barnes et al. (2021) Barnes A. T., et al., 2021, MNRAS, 503, 4601
  * Bohlin et al. (1978) Bohlin R. C., Savage B. D., Drake J. F., 1978, ApJ, 224, 132
  * Brinch & Hogerheijde (2010) Brinch C., Hogerheijde M. R., 2010, A&A, 523, A25
  * Chen et al. (2022) Chen M. C.-Y., Di Francesco J., Pineda J. E., Offner S. S. R., Friesen R. K., 2022, ApJ, 935, 57
  * Clark & Glover (2014) Clark P. C., Glover S. C. O., 2014, MNRAS, 444, 2396
  * Clark et al. (2019) Clark P. C., Glover S. C. O., Ragan S. E., Duarte-Cabral A., 2019, MNRAS, 486, 4622
  * Clarke & Whitworth (2015) Clarke S. D., Whitworth A. P., 2015, MNRAS, 449, 1819
  * Clarke et al. (2016) Clarke S. D., Whitworth A. P., Hubber D. A., 2016, MNRAS, 458, 319
  * Clarke et al. (2017) Clarke S. D., Whitworth A. P., Duarte-Cabral A., Hubber D. A., 2017, MNRAS, 468, 2489
  * Clarke et al. (2018) Clarke S. D., Whitworth A. P., Spowage R. L., Duarte-Cabral A., Suri S. T., Jaffa S. E., Walch S., Clark P. C., 2018, MNRAS, 479, 1722
  * Crutcher (2012) Crutcher R. M., 2012, ARA&A, 50, 29
  * Draine (1978) Draine B. T., 1978, ApJS, 36, 595
  * Federrath (2016) Federrath C., 2016, MNRAS, 457, 375
  * Glover & Mac Low (2007) Glover S. C. O., Mac Low M.-M., 2007, ApJS, 169, 239
  * Gutiérrez-Vera et al. (2023) Gutiérrez-Vera N., Grassi T., Bovino S., Lupi A., Galli D., Schleicher D. R. G., 2023, A&A, 670, A38
  * Habing (1968) Habing H. J., 1968, Bull. Astron. Inst. Netherlands, 19, 421
  * Hacar et al. (2013) Hacar A., Tafalla M., Kauffmann J., Kovács A., 2013, A&A, 554, A55
  * Hacar et al. (2016a) Hacar A., Kainulainen J., Tafalla M., Beuther H., Alves J., 2016a, A&A, 587, A97
  * Hacar et al. (2016b) Hacar A., Alves J., Burkert A., Goldsmith P., 2016b, A&A, 591, A104
  * Hacar et al. (2018) Hacar A., Tafalla M., Forbrich J., Alves J., Meingast S., Grossschedl J., Teixeira P. S., 2018, A&A, 610, A77
  * Hacar et al. (2022) Hacar A., Clark S., Heitsch F., Kainulainen J., Panopoulou G., Seifried D., Smith R., 2022, arXiv e-prints, p. arXiv:2203.09562
  * Holdship et al. (2017) Holdship J., Viti S., Jiménez-Serra I., Makrymallis A., Priestley F., 2017, AJ, 154, 38
  * Howard et al. (2019) Howard A. D. P., Whitworth A. P., Marsh K. A., Clarke S. D., Griffin M. J., Smith M. W. L., Lomax O. D., 2019, MNRAS, 489, 962
  * Howard et al. (2021) Howard A. D. P., Whitworth A. P., Griffin M. J., Marsh K. A., Smith M. W. L., 2021, MNRAS, 504, 6157
  * Inoue et al. (2018) Inoue T., Hennebelle P., Fukui Y., Matsumoto T., Iwasaki K., Inutsuka S.-i., 2018, PASJ, 70, S53
  * Jura (1974) Jura M., 1974, ApJ, 191, 375
  * Juvela et al. (2012) Juvela M., Malinen J., Lunttila T., 2012, A&A, 544, A141
  * Keto et al. (2015) Keto E., Caselli P., Rawlings J., 2015, MNRAS, 446, 3731
  * Kim et al. (2022) Kim S., Lee C. W., Tafalla M., Gophinathan M., Caselli P., Myers P. C., Chung E. J., Li S., 2022, ApJ, 940, 112
  * Kirk et al. (2015) Kirk H., Klassen M., Pudritz R., Pillsworth S., 2015, ApJ, 802, 75
  * Könyves et al. (2015) Könyves V., et al., 2015, A&A, 584, A91
  * Könyves et al. (2020) Könyves V., et al., 2020, A&A, 635, A34
  * Larson (1969) Larson R. B., 1969, MNRAS, 145, 271
  * Lee et al. (1999) Lee C. W., Myers P. C., Tafalla M., 1999, ApJ, 526, 788
  * Li et al. (2023) Li S., et al., 2023, arXiv e-prints, p. arXiv:2304.01718
  * Mac Low & Klessen (2004) Mac Low M.-M., Klessen R. S., 2004, Reviews of Modern Physics, 76, 125
  * McElroy et al. (2013) McElroy D., Walsh C., Markwick A. J., Cordiner M. A., Smith K., Millar T. J., 2013, A&A, 550, A36
  * Misugi et al. (2023) Misugi Y., Inutsuka S.-i., Arzoumanian D., 2023, ApJ, 943, 76
  * Myers et al. (1996) Myers P. C., Mardones D., Tafalla M., Williams J. P., Wilner D. J., 1996, ApJ, 465, L133
  * Ossenkopf & Henning (1994) Ossenkopf V., Henning T., 1994, A&A, 291, 943
  * Ostriker (1964) Ostriker J., 1964, ApJ, 140, 1056
  * Palmeirim et al. (2013) Palmeirim P., et al., 2013, A&A, 550, A38
  * Panopoulou et al. (2014) Panopoulou G. V., Tassis K., Goldsmith P. F., Heyer M. H., 2014, MNRAS, 444, 2507
  * Panopoulou et al. (2017) Panopoulou G. V., Psaradaki I., Skalidis R., Tassis K., Andrews J. J., 2017, MNRAS, 466, 2529
  * Pattle et al. (2021) Pattle K., et al., 2021, ApJ, 907, 88
  * Pattle et al. (2022) Pattle K., Fissel L., Tahani M., Liu T., Ntormousi E., 2022, arXiv e-prints, p. arXiv:2203.11179
  * Peretto et al. (2013) Peretto N., et al., 2013, A&A, 555, A112
  * Pineda et al. (2022) Pineda J. E., et al., 2022, arXiv e-prints, p. arXiv:2205.03935
  * Price et al. (2018) Price D. J., et al., 2018, Publ. Astron. Soc. Australia, 35, e031
  * Priestley & Whitworth (2020) Priestley F. D., Whitworth A. P., 2020, MNRAS, 499, 3728
  * Priestley & Whitworth (2022) Priestley F. D., Whitworth A. P., 2022, MNRAS, 509, 1494
  * Priestley et al. (2022) Priestley F. D., Yin C., Wurster J., 2022, MNRAS, 515, 5689
  * Schöier et al. (2005) Schöier F. L., van der Tak F. F. S., van Dishoeck E. F., Black J. H., 2005, A&A, 432, 369
  * Seifried & Walch (2015) Seifried D., Walch S., 2015, MNRAS, 452, 2410
  * Seifried & Walch (2016) Seifried D., Walch S., 2016, MNRAS, 459, L11
  * Seifried et al. (2017) Seifried D., Sánchez-Monge Á., Suri S., Walch S., 2017, MNRAS, 467, 4467
  * Sembach et al. (2000) Sembach K. R., Howk J. C., Ryans R. S. I., Keenan F. P., 2000, ApJ, 528, 310
  * Shimajiri et al. (2019a) Shimajiri Y., André P., Palmeirim P., Arzoumanian D., Bracco A., Könyves V., Ntormousi E., Ladjelate B., 2019a, A&A, 623, A16
  * Shimajiri et al. (2019b) Shimajiri Y., André P., Ntormousi E., Men’shchikov A., Arzoumanian D., Palmeirim P., 2019b, A&A, 632, A83
  * Shimajiri et al. (2023) Shimajiri Y., André P., Peretto N., Arzoumanian D., Ntormousi E., Könyves V., 2023, arXiv e-prints, p. arXiv:2301.10560
  * Shirley (2015) Shirley Y. L., 2015, PASP, 127, 299
  * Sipilä & Caselli (2018) Sipilä O., Caselli P., 2018, A&A, 615, A15
  * Sipilä et al. (2022) Sipilä O., Caselli P., Redaelli E., Spezzano S., 2022, A&A, 668, A131
  * Smith et al. (2012) Smith R. J., Shetty R., Stutz A. M., Klessen R. S., 2012, ApJ, 750, 64
  * Smith et al. (2013) Smith R. J., Shetty R., Beuther H., Klessen R. S., Bonnell I. A., 2013, ApJ, 771, 24
  * Smith et al. (2023) Smith S. E. T., et al., 2023, MNRAS, 519, 285
  * Stodólkiewicz (1963) Stodólkiewicz J. S., 1963, Acta Astron., 13, 30
  * Suri et al. (2019) Suri S., et al., 2019, A&A, 623, A142
  * Tafalla et al. (2002) Tafalla M., Myers P. C., Caselli P., Walmsley C. M., Comito C., 2002, ApJ, 569, 815
  * Tafalla et al. (2021) Tafalla M., Usero A., Hacar A., 2021, A&A, 646, A97
  * Watkins et al. (2019) Watkins E. J., Peretto N., Marsh K., Fuller G. A., 2019, A&A, 628, A21
  * Whitworth et al. (1996) Whitworth A. P., Bhattal A. S., Francis N., Watkins S. J., 1996, MNRAS, 283, 1061
  * Whitworth et al. (2021) Whitworth A. P., Priestley F. D., Arzoumanian D., 2021, MNRAS, 508, 2736
  * Wilson & Rood (1994) Wilson T. L., Rood R., 1994, ARA&A, 32, 191
  * Yin et al. (2021) Yin C., Priestley F. D., Wurster J., 2021, MNRAS, 504, 2381

##  Appendix A Hyperfine structure

Figure 8: HCN (left) and N2H+ (right) $J=1-0$ line profiles for the ${\cal
M}=3$ model at $0.99\,{\rm Myr}$, with (dashed lines) and without (solid
lines) hyperfine structure.

Figure 9: As Figure 8, but zoomed-in on the central HCN (left) and isolated
N2H+ (right) hyperfine components (dashed lines). The non-hyperfine profiles
(solid lines) have been shifted to the same central frequency for comparison.

Figure 8 shows the effect of hyperfine structure on the HCN and N2H+ $J=1-0$
lines for the ${\cal M}=3$ model after $0.99\,{\rm Myr}$. For HCN, the
individual hyperfine components have lower peak intensities than the non-
hyperfine treatment, and display less pronounced self-absorption features, as
the column density (and thus optical depth) of the molecule in each hyperfine
state is lower than the sum of all equal-$J$ states. The situation for N2H+ is
more complex, and individual hyperfine components can be either weaker or
stronger than the non-hyperfine approximation. However, the dynamical
information encoded in the line profiles is essentially unchanged in both
cases.

Figure 9 compares the non-hyperfine profiles to the central HCN component and
the isolated N2H+ component (commonly used to trace dense gas dynamics; Hacar
et al. 2018; Barnes et al. 2021). Line shapes, in particular the line widths
(and therefore velocity dispersions) and the velocity of the HCN self-
absorption feature, are almost identical. Similar conclusions regarding the
dynamics of the filament would be reached regardless of which hyperfine
component were considered, particularly for the N2H+ line due to its
optically-thin nature. As we do not base any of our conclusions on the
absolute intensities of either line but their relative variations, we consider
it acceptable to neglect the effects of hyperfine structure.