fax Hermaphrodite

# Leveraging Users’ Social Network Embeddings for Fake News Detection on
Twitter

Ting Su, Craig Macdonald, Iadh Ounis
University of Glasgow
Glasgow, UK
<EMAIL_ADDRESS>{craig.macdunald<EMAIL_ADDRESS>

###### Abstract

Social networks (SNs) are increasingly important sources of news for many
people. The online connections made by users allows information to spread more
easily than traditional news media (e.g., newspaper, television). However,
they also make the spread of fake news easier than in traditional media,
especially through the users’ social network connections. In this paper, we
focus on investigating if the SNs’ users connection structure can aid fake
news detection on Twitter. In particular, we propose to embed users based on
their follower or friendship networks on the Twitter platform, so as to
identify the groups that users form. Indeed, by applying unsupervised graph
embedding methods on the graphs from the Twitter users’ social network
connections, we observe that users engaged with fake news are more tightly
clustered together than users only engaged in factual news. Thus, we
hypothesise that the embedded user’s network can help detect fake news
effectively. Through extensive experiments using a publicly available Twitter
dataset, our results show that applying graph embedding methods on SNs, using
the user connections as network information, can indeed classify fake news
more effectively than most language-based approaches. Specifically, we observe
a significant improvement over using only the textual information (i.e.,
TF.IDF or a BERT language model), as well as over models that deploy both
advanced textual features (i.e., stance detection) and complex network
features (e.g., users network, publishers cross citations). We conclude that
the Twitter users’ friendship and followers network information can
significantly outperform language-based approaches, as well as the existing
state-of-the-art fake news detection models that use a more sophisticated
network structure, in classifying fake news on Twitter.

_K_ eywords Fake news detection $\cdot$ Social Network Embedding $\cdot$
Neural Language models $\cdot$ Clustering

##  1 Introduction

In recent years, the term “fake news" has been used abundantly, by both
politicians and journalists. The existence of large amounts of information
that members of the public can’t or won’t trust, or that can easily confuse
and mislead the general public, can cause public distrust in journalism. Such
a reality is already unfolding, as the reputation of news outlets and news
media has been in doubt amid a global distrust crisis [1].

Figure 1: An example showing that users are connected on the Twitter platform,
while only a subset of users engage with a specific news article.

Similarly, polarisation in political and scientific debates are observed more
frequently [2, 3], since fake news 111In this paper, we use “news” to describe
all types of news format - articles, tweets, claims, etc.  can amplify and
reinforce false information through repeated exposure [4, 5].

Cho et al. [6] showed that one possible reason for the increasing polarisation
is that the selected contents delivered by search engines (e.g., Google,
YouTube) and social media platforms (e.g., Facebook, Twitter) are individually
tailored to the users’ specific interests, thus creating filter bubbles that
reinforce their existing beliefs. Such an observation echoes the findings of
Ling [7], who showed that individuals’ false beliefs are emphasised by
repeated exposure and a selective focus because of the confirmation bias
effect. Indeed, the confirmation bias states that individuals are more
inclined to interact with information that aligns or confirms their existing
beliefs, while avoid confrontational information that challenge their existing
beliefs. Furthermore, Yoo [8] identified echo chamber effect in social media
platforms and showed that this facilitates rumours and fake news being created
and circulating within small and specific groups, before being spread further.
Echo chamber effect in social media can be described as a subset of like-
minded users grouping together, with little interaction with dissimilarly-
minded users, to preserve their beliefs and avoid confrontation. Figure 1
illustrates that a news article might only reach a subset of users, with it
being circulated and discussed by like-minded group members.

However, Lewandowsky at al. [9] argued that the misinformation crisis should
be considered and evaluated as a public concern, especially with the
popularity of social media. Indeed, increasing efforts [10, 11, 12, 13] are
made in recent years to develop methods that can help scholars and journalists
detect fake news. These methods generally deploy machine learning methods
using text features (i.e., TF.IDF [14], word2vec [15]), information cascade
features [16, 17], metadata extracted from tweets, as well as entity and
knowledge base methods [13, 18].

However, the above mentioned methods largely focus on the textual features,
statistics of tweets and users engaged in the topic, without paying much
attention to the SN structure around users – indeed, users on social media
form their unique society and are largely distinguishable based on their
interests and beliefs. One recent related study [19] that investigated the
user network’s role in detecting fake news deployed complex network structures
(e.g., citation network, publication network, stances) and achieved
significant improvements in accuracy at classifying a news article as fake or
not, compared to the existing state-of-the-art models. Inspired by this, we
propose a user network embedding method to represent each Twitter user in a
lower dimensional space, based on their connections with other users within
the platform, and projecting the social network structure as a linked graph.
We argue that this graph structure can aid in detecting clusters of users
engaging in fake news, and can assist the fake news detection task on Twitter.
Thus, the main contributions of this paper are as follows:

  * •

We propose to incorporate the idea of echo chamber effect into automatic fake
news detection task. Specifically, we hypothesise that training user network
embeddings without the prior knowledge of users’ engagements with fake news
can help identifying user communities that frequently engage in and spread
fake news, and facilitate the fake news detection in social media.

  * •

We propose a User Network Embedding Structure (UNES) model, which performs
fake news classification on Twitter through the use of graph embeddings to
represent Twitter user’s social network structure. Compared to the approach of
[19], UNES does not require any pre-annotated data (e.g., user type
(individual users or publishers), users stance, and if they have engaged with
fake news before).

  * •

We observe that the user embeddings generated by UNES exhibit a clustering
effect between users who engage with fake news and users who solely engage
with factual news, despite not having knowledge if the users have engaged with
fake news before.

  * •

We also show that using the social network’s user connections alone to build
network embeddings, and using only users that engaged with the news when
representing such news, can significantly outperform the existing state-of-
the-art fake news detection approaches that use both textual features and
complex social network features.

This paper is structured as follows: Section 2 describes the fake news
detection methods proposed in the literature, and highlights the limitations
in these approaches, while also providing a background on graph embeddings,
particularly in the social networks domain; Section 3 formally states the task
to be tackled and and describes our proposed UNES model for fake news
detection; The used experimental setup and obtained results for fake news
detection using the UNES model are provided in Sections 4 and 5, respectively;
Concluding remarks follow in Section 6.

##  2 Related Work

There is an extensive body of research on fake news detection in social media.
In this section, we focus on commonly used fake detection approaches, as well
as survey recent advances in graph embedding models. Then, we identify some
limitations of the existing methods, and position our study in relation to
these methods.

###  2.1 Fake News Detection

Research in fake news – sometimes also called rumours, misinformation, or
disinformation – has attracted increasing attention in recent years.
Identifying fake news is an important step in informing the general public
about the widespread of false news. Thus, we first focus on existing methods
that aim to tackle the task of identifying fake news in social media.

Fake news detection using textual information. Text representations are widely
used in fake news detection. There are several methods used to represent text.
The most commonly used methods in fake news detection include the bag-of-words
(BoW) method (e.g. TF.IDF [20]), parts-of-speech (POS tagging [21]), and word
embedding (e.g. Word2Vec [22]) representations. For example, Ma et al. [14]
used a typical BoW representation (TF.IDF) to represent tweets. Deeper NNs,
such as long short term memory (LSTM) models and bidirectional LSTMs
(BiLSTMs), can capture the semantic meaning of the text based on nearby
tokens, and is used in fake news detection tasks [23]. Kochkina et al. [24]
proposed BranchLSTM to analyse tweet threads (tweets, retweets, comments)
sequentially. BranchLSTM yielded more accurate results in identifying fake
news, than those without the sequential information of tweet threads.
Moreover, recent research has shown that attention-based neural network
language models (e.g. ELMO [25], BERT [26]) can benefit from both the complex
neural network structure and the large pre-training corpus. As the state-of-
the-art pre-trained language model, the BERT model can be fine-tuned to tailor
to the needs of a specific task, and has been shown to consistently outperform
other models in many tasks [27, 28], as well as in fake news detection tasks
[29].

Many scholars also used other handcrafted textual features (e.g. Number of @,
#, exclamation marks, first-person pronouns [30, 31, 10]). Deep syntactical
analysis approaches [11] are also useful in detecting fake news [12].

Network Analysis. Networks, usually represented as a linked graph, have data
points linking to one or more other data points through information or
relationships. Knowledge graphs and social networks are the two main types of
networks that are particularly rich in conveying additional information that
textual features may not capture, when detecting fake news online [32, 33].

Knowledge Graphs use a linked structure that consists of entities and their
relationships (e.g. $\langle$Beethoven, composer_of, Moonlight
Sonata$\rangle$, where Beethoven and Moonlight Sonata are the two entities and
composer_of is the relationship between the two entities). By extracting
entities and relationships between entities from the text, and comparing this
information to well-established knowledge graphs, it is possible to estimate
the likelihood of the news being factual or fake. For example, Ciampaglia at
a. [13] found that by calculating the shortest path of the subject and object
in a knowledge base, where the subject and the object are from the triplet
($\langle subject,predicate,object\rangle$) extracted from the article, they
can reach an accuracy as high as 90% in classifying fake and non-fake news.

Social networks (such as Twitter) are platforms where people can connect with
each other online, and share information and news with their followers and
friends. The links between people on Twitter can be explicitly categorised
into two types: followers or friends. Researchers have developed a range of
features that use the rich information that social networks provide, to help
identify fake news. Some of the examples are as follows: simple user features
(e.g, # of followers, # of friends, verified or not, description) [16],
relations between users (e.g., in the same region, engaged with same
tweet/URL) [17], relations between tweets (e.g., replies, retweets, likes,
viewpoints conflicts) [34].

However, all the above mentioned methods that include users information as one
of the features, only consider statistical user information, but ignore the
potential network information that may help in identifying the social group a
particular user belongs to, and the stance that this group has on the
particular issue. Recently, researchers started analysing the user network
structure of social media, such as the user network on Twitter or the channel
network on YouTube. For example, Röchert et al. [35] studied the user network
structure of the YouTube channels. They found that the channels and
individuals propagating fake news on YouTube are usually integrated in
heterogeneous discussion networks that involve factual content more than
misinformation. Similarly, Sosnkowsk et al. [36] showed that changes in the
users’ network structure on Twitter can help detect the change of political
opinions among users. Nguyen et al. [19] proposed the FActual News Graph
(FANG) model, and showed that network information can be beneficial in
detecting fake news in social media. For example, FANG considers the news
outlet that published the article, other news outlets that cited such a
publication as citation relationship, and uses different types of stances
(i.e., support, deny, comment (neutral) and comment (negative)) that users
took when engaging with the news articles as additional information to the
engagement graph.

###  2.2 Graph Embeddings

Recent advances in learning graph structures most commonly use the hyper
dimensional space to represent a graph, and represent each node and/or vertex
using a vector, based on their neighbouring nodes and the vertices that
connect them. Typically, graph embeddings are achieved by embedding only the
topological structure of the graph [37, 38], or using both the topological
structure as well as the auxiliary information of the graph, such as the
content of nodes [39, 40].

For instance, Node2Vec [37] and DeepWalk [38] were the first two neural
network approaches developed to represent the graph in a hyper dimensional
space. These two methods are similar, as they both use a Skip-Gram
architecture with negative sampling to learn the embeddings of each node,
based on the portion of graph generated by random walks/edge sampling.
However, such methods only consider the local context (i.e., closest
neighbouring nodes), and cannot obtain a global optimum in representing the
complexity of a graph. To better represent a node in a complete graph
structure, graph convolutional networks [41] (GCN) were proposed to capture
the global structure of a given graph. Specifically, GCN uses several layers
of graph convolution operations, where each layer is built to capture the
information of each node’s neighbours, which is then fed to the next layer,
therefore achieving convolutional learning of the graph structure. However,
GCN has a very high computational cost [39]. Thus, various methods have been
proposed to reduce the computational time and space consumption, such as mini-
batch stochastic gradient descent with variance reduction GCN [42], and
Cluster-GCN [39]. The GraphSAGE [40] method was developed to perform
parameterised random walks and uses recurrent aggregators. It can be used for
both unsupervised or supervised representation learning with a proximity loss
between nodes. Moreover, the model adopts a dynamic inductive algorithm, so
that it can generate embeddings for unseen nodes and edges at inference time.

Graph embedding methods have been shown to be effective in many tasks, such as
node classification [38], link prediction [43, 44], and social networks
alignment [45]. For example, Monti et al. [46] showed that the user
embeddings, trained from a user network with a credibility score for each
user, can more accurately identify fake news on Twitter than when the user
network information was not used; Wu and Liu [47] aimed to leverage the
community information that users may have encapsulated in their embeddings to
identify fake news on Twitter platform. The aforementioned Factual News Graph
model (FANG) of Nguyen et al. [19] also proposed to use inductive learning for
representing social structure, and combined the graphical social network
information with sophisticated textual features. Specifically, Nguyen et al.
combined citation network among publishers, with user engagement and follower
relationship, to build a multi-source graph model for fake news detection on
Twitter. Similarly, Rath et al. [48] proposed to use the network structure
information to identify the potential super spreader of fake news, and showed
the proposed model can identify potential spreaders with the retweet network,
the follower-following network, and the timeline data.

###  2.3 Summary

Recent works that aim at detecting fake news most focus on using either
textual features and/or static and numerical social network features. In this
work, we view the users’ network structure as a directed graph (i.e., if one
user follows/is followed by another user), where users are the nodes in the
graph, and their relationships with other users are the edges, thus forms a
dynamically incremental graph network. In order to represent such a directed
graph of users, we propose to use graph embedding methods to embed the graph
in a hyper-dimensional space, thereby obtaining embedded users representation
w.r.t. the graph structure.

Compared to one of the most recent works that adopted graph representation for
fake news detection by Nguyen et al. [19], our work also uses inductive
representation learning of social network structures in a fake news detection
task. However, instead of the complex network information (i.e., the
publishers’ publication history, citation network among publishers, and the
users’ followers), we aim to use the readily available data that can be
obtained directly through Twitter - without further processing - to construct
a user network embedding that is able to accurately cluster users into
different groups, based on their friendships with other users and their
followers. Building on the users’ network embeddings, we then aim to determine
if the network structure of users engaging with a news story on social media
can effectively represent a news story’s truthfulness, and can thus help to
detect fake news circulating on social media. Indeed, our later experiments
demonstrate that the users’ network embeddings do improve the accuracy in
detecting fake news in social media. In the next section, we formally define
the research problem to be tackled, and present our proposed model, namely the
User Network Embedding Structure (UNES) model.

(a) Our proposed UNES model for fake news detection.

(b) Language model (BERT) baseline for fake news detection.

Figure 2: Comparison between our proposed UNES model and a text-based (BERT)
baseline classifier.

##  3 Using Social Network Embedding for Fake News Detection

In this section, we first define the task we aim to tackle, and describe the
notations we will be using in this paper. Thereafter, we introduce our
proposed UNES model in detail.

###  3.1 Twitter Users-based News Article Classification

In this paper, we aim to develop a model allowing to accurately classify news
articles as fake or not, based on the tweets and retweets with comments that
have engaged with the news articles (i.e., included a news link in the
tweets), and the corresponding users that posted those tweets on the Twitter
SN. In particular, we define the following concepts:

  1. 1.

$X$ = $\\{x_{1},x_{2},...\\}$ is the list of news articles that need to be
fact-checked, where each article has a modelled vector representation of each
news article $x_{i}$, denoted by $\overrightarrow{x_{i}}$.

  2. 2.

$U$ = $\\{u_{1},u_{2},...\\}$ is the set of users engaged with all news
articles in the dataset. $U_{x_{i}}\in U$ is the set of users engaged with the
specific news article $x_{i}$. $\overrightarrow{u_{n}}$ is a modelled vector
representation of each user $u_{n}$.

  3. 3.

Specifically, let us define $x=\langle a_{x},U_{x},T_{x}\rangle$ where for
each news article $x$, $a_{x}$ is the text (i.e., title and body) of the news
article, $T_{x}$ is the set of the tweets that engaged with the news article
$x$; and $U_{x}$ is the set of users that tweeted $T_{x}$.

  4. 4.

Finally, let $G$ be a representation of the social network graph that is
formed by all users $U$.

Therefore, our news article classification task is defined as follows:

$\widetilde{Y}_{x}=f(x)=f(a_{x},U_{x},T_{x})$ (1)

In particular, for each news article $x$, we aim to predict whether the
article is fake or not, $\widetilde{Y}_{x}$, based on the text of the article
$a_{x}$, the tweets $T_{x}$ that engaged (e.g. tweeted/retweeted the news link
with comment) with the news article $x$, as well as the users $U_{x}$ that
tweeted $T_{x}$, where $u_{n}$ posted the tweet $t_{n}$ ($t_{n}\in T_{x}$).
Hence, the objective of this study is to identify the best $f()$ for fake news
detection – next, we describe our proposed UNES model, which identifies fake
news on Twitter based on the network structure, $G$, of the Twitter users.

###  3.2 Proposed Model - UNES

We now describe our proposed User Network Embedding Structure (UNES) model for
fake news detection on Twitter, which applies user embeddings obtained from
network embedding methods. Figure 2(a) presents the structure of our proposed
UNES model, while Figure 2(b) shows a BERT-based model as an example of the
state-of-the-art language model for fake news detection.

To obtain a prediction for a given news article $x$ based on the text of the
article, the engaged tweets, and the engaged users, i.e.
$f(a_{x},U_{x},T_{x})$, we propose to make use of the social networks of each
of the engaged users $u\in U_{x}$, as illustrated earlier by Figure 1 in
Section 1. We consider each user as a node in a directed graph, and their
relationships (i.e. following, friendship) as vertices connecting with other
users. To this end, we introduce $\operatorname{follows}(u_{i},u_{j})$ as a
binary function that returns 1 if user $u_{i}$ follows $u_{j}$, and 0
otherwise. Thus user $u_{i}$ follows user $u_{j}$ is denoted as
$\operatorname{follows}(u_{i},u_{j})$. Moreover, Twitter users can be friends
– i.e. follow each other – for which we use two edges that have different
directions but connecting the same two nodes:
$\operatorname{follows}(u_{i},u_{j})\land\operatorname{follows}(u_{i},u_{j})$.

For all users $U$, the social network graph can be expressed as an adjacency
matrix $G$. Specifically, let $G^{\text{fo}}$ denote the followers graph, and
$G^{\text{fr}}$ the friends graph. Entries in these adjacency matrices for any
pair of users $u_{i},u_{j}\in U$ are defined as $\operatorname{follows}()$:

$\begin{split}G^{\text{fo}}_{i,j}&=\operatorname{follows}(u_{i},u_{j})\\\
G^{\text{fr}}_{i,j}&=\operatorname{follows}(u_{i},u_{j})\land\operatorname{follows}(u_{j},u_{i})\end{split}$
(2)

In the following, we use $G$ to represent either $G^{\text{fo}}$ or
$G^{\text{fr}}$.

To make use of the large social network graphs within a classifier, we convert
the sparse network structure into dense graph embeddings. In particular, the
connections of $u_{i}$ is defined as all users that user $u_{i}$ is connected
to (i.e., as either a follower or a friend). Thus, we represent each user
$u_{i}$ by their connections to other users through the application of a graph
embedding function222In the remainder of the paper, we will use graph
embedding and network embedding interchangeably, unless it is required
otherwise by the context of the section, as we apply graph embedding to the
user network structure. $f_{u}()$, to obtain embeddings for each user $u$, as
follows:

$\overrightarrow{u_{n}}=f_{u}(G_{u_{n}})\\\ $ (3)

Note that we employ unsupervised graph embedding models, i.e., users are not
labelled with their engagement with fake news or factual news. This is a
realistic setup, since a large number of users may not have engaged with any
news articles on Twitter, fake or factual.

Table 1: Statistics of the SD dataset. Note that the avg., max., and min.
numbers are per news article.

| Overall | Fake | Factual
---|---|---|---
# of news | 1054 | 448 | 606
# w/o tweets | 11 | 4 | 7
avg. # tweets | 46.30 | 52.31 | 41.89
max. # tweets | 1054 | 750 | 1054
avg. # users | 34.10 | 47.48 | 35.95
max. # users | 1028 | 645 | 1027

Table 2: Statistics of the edges users have in our friendships and followers
networks, for the SD dataset44footnotemark: 4. $\\#>5000$ denotes the users
with more than 5000 friends/followers shown in their profile.
| # edges of $G^{\text{fr}}$ | # edges of $G^{\text{fo}}$
---|---|---
Avg | 1388.37 | 1565.16
Max | 14012 | 18255
Std. | 1614.43 | 1891.05
25%l | 181 | 147
50%l | 674 | 621
75%l | 2017 | 2498
$\\#>5000$ | 3808 | 6263

We represent each news article as the element-wise sum of all the engaged
users’ embeddings (i.e., users who tweeted about the news article $x$, as also
used in the FANG model [19]). Thus, Equation (1) can be instantiated as
follows:

$\widetilde{Y}_{x}=f(x)=f(\sum_{u_{n}\in U_{x}}f_{x}({G_{u_{n}}}))$ (4)

22footnotetext: 10 users missing, as 2 accounts are set as private, and 8
accounts were deleted.

In particular, we use three types of graph embedding methods for the fake news
classification task, representing the basic graph embedding models (i.e.,
DeepWalk), graph cluster-based graph embedding models (i.e., Cluster-GCN), and
the existing state-of-the-art graph embedding models (i.e., GraphSAGE),
namely:

  1. 1.

DeepWalk [38]. It was the first deep learning-based method to address a graph
embedding task, in a manner inspired by word2vec. DeepWalk takes truncated
random walks on a graph to learn embedded representations of nodes.

  2. 2.

Cluster-GCN [41]. Given a graph, Cluster-GCN uses a graph convolution
operation (GCN) [41] to obtain node embeddings, by aggregating the
neighbouring nodes’ embeddings of each node, applying CNN layers for each
aggregation. Moreover, Cluster-GCN identifies a subgraph with a graph
clustering algorithm, and restricts the neighbourhood search within this
subgraph, thus presenting a more efficient and effective graph embedding model
than a plain GCN.

  3. 3.

GraphSAGE [40]. This method performs parameterised random walks and uses
recurrent aggregators. It can be used for both unsupervised or supervised
representation learning with an algorithm within the model that allows it to
generate embeddings for unseen nodes and edges at inference time.

Thus, using the graph embeddings from these three methods, we can represent a
news article in a hyper-dimensional space based on the social network
structure of its engaged users (aggregated as per Equation (4)), without
needing textual information, or a more sophisticated analysis of each users’
account (e.g., account type, stance on the topic). Furthermore, in order to
compare the effectiveness of our UNES model with widely-used textual features
for detecting fake news on Twitter, in our experiments, we use classifier
models learned using social network graph embedding features (as in Figure
2(a)), as well as those using textual features (as in Figure 2(b)) – such as
the state-of-the-art language model BERT. Next, we detail our research
questions and the setup for our experiments.

##  4 Experimental setup

We aim to address three research questions as follows:

  * •

RQ1: Can our proposed UNES model allow to identify clusters of users who
engage with fake news on Twitter?

  * •

RQ2: How effective is UNES in identifying fake news on Twitter? Which graph
embedding method is the most effective?

  * •

RQ3: Which type of social network structure (i.e., followers or friendship
networks) provides the most effective information in allowing to accurately
identify fake news on Twitter?

In the following, we describe the used dataset, the approaches we use to
represent both tweets and news articles, the user network embedding models,
the baselines, and evaluation metrics we use in reporting our results.

###  4.1 Dataset

In our conducted experiments, we use the stance detection (SD) dataset
provided by [19]555https://github.com/nguyenvanhoang7398/FANG. The SD dataset
consists of news article links and human judgements labels denoting if they
are fake or not, as well as engaged tweets, the stance of such tweets, the
publisher of the news article, and article citations by other news outlets on
Twitter. We download all the available tweets in the dataset, along with the
tweet authors’ friends and followers. We limit this to a maximum of 5000 for
friends and 5000 for followers, which is the maximum number we can download as
per Twitter API limit. We also remove those users that have less than 2 edges
in the graph (provided they have not engaged with any news article) in order
to reduce the size of the graph and allow for tractable experiments. In Table
2 we provide the statistics of the dataset in terms of news articles, tweets
and users; Table 2 details statistics of the friendship and follower networks
of the users. One can argue that information such as the likes, replies, and
retweet relations can also be viewed as possible types of relationships on
Twitter. However, due to the difficulties in obtaining data concerning the
likes, replies, and retweet relationships from the Twitter API, we do not use
these types of relationships in our work.

###  4.2 Semantic Representation

In order to investigate the effectiveness of social network structure in
detecting fake news, we also deploy textual-based classifiers as baselines. In
particular, we employ two language processing methods, namely, a TF.IDF
representation and a BERT representation for both tweets and news articles.
The experimental setup for these two language processing methods are as
follows:

  * •

TF.IDF: We use NLTK’s
TweetTokenizer666https://www.nltk.org/api/nltk.tokenize.html to tokenise
tweets. Scikit-Learn’s TfidfVectorizer777https://scikit-learn.org/ is used to
extract the TF.IDF representation for both articles and tweets. We limit the
maximum number of tokens per text entry to 10k, to include all the tweets and
news article tokens.

  * •

BERT: We fine-tune the BERT model with a BERT-base English model (uncased,
12-layer, 768-hidden, 12-heads, 110M parameters). We maintain the suggested
learning rate [26], with a drop out rate of 0.05, the maximum token length for
the news article and tweets combination is 512 (i.e., as the maximum number of
tokens that BERT can encode is 512 [26]).

###  4.3 User Network Embedding Methods

As mentioned earlier, we instantiate the UNES model with three graph embedding
methods, namely DeepWalk, Cluster-GCN, and GraphSAGE. Each of these graph
embedding methods allows two sets of features per node: node network
structures and additional node features. In order to address RQ2, we deploy
our models without textual node features. The only information we pass to the
graph embedding methods are the network connection features. Moreover, the two
types of relationships we use to initiate the graph embedding methods are
friendship and followers. The detailed setup of each graph embedding method is
as follows:

  * •

DeepWalk. We train DeepWalk with 50 hidden units, a window unit of 10. Each
node has a maximum of 10 walks, with a maximum of 80 steps per walk, and
results in a 64 dimension vector to represent each user, as per the original
DeepWalk paper [38].

  * •

Cluster-GCN. We train Cluster-GCN with 1000 epochs each, with 16 hidden units,
and results in 100 dimension vector to represent each user, as per [19].

  * •

GraphSAGE. We train GraphSAGE with 30 epochs, 16 hidden units, and 2 layers.
It results in 100 dimension vector for each user, as per [19].

Note that we represent the users who do not have any followers or friends
using an embedding vector of [-1, …, -1].

###  4.4 Classifiers

We use SVM for the baseline model, along with a TD.IDF representation. We use
the Scikit-Learn’s implementation of SVM, with the standard parameters (i.e.,
RBF kernel, a C penalty of 10, and a $\gamma$ of 0.1). For the deep learning
models, we use a fully connected dense layer to classify a news article as
fake or factual, which is trained end-to-end with the user embedding vectors.
For instance, for the BERT model, we fine-tune the BERT model with a fully
connected dense layer; for our proposed UNES model, we train the fully
connected dense layer as $f_{n}()$, as per Equation (1).

###  4.5 Baselines

We report two types of baselines, 5 baselines in total, in order to test the
effectiveness of our UNES model.

The first group of baselines consists of models that only use textual
features. The second group of baselines are the current state-of-the-art
models proposed by [19], namely the GCN model and FANG model. These five
baselines are as follows:

  1. 1.

Textual features only:

    1. 1.1.

SVM classifier with a TF.IDF representation of tweet and news article, denoted
as SVM TF.IDF($\boldsymbol{a_{n}}$ and $\boldsymbol{t_{n}}$);

    2. 1.2.

Fine-tuned BERT model, using the news article content $a_{n}$, denoted as
BERT($\boldsymbol{a_{n}}$);

    3. 1.3.

Fine-tuned BERT model, using the engaged tweets content $T_{n}$, denoted as
BERT($\boldsymbol{T_{n}}$);

  2. 2.

Models proposed by [19]:

    1. 2.1.

GCN with FANG network information, as well as TF.IDF representation for the
tweets and news, denoted as GCN ($\boldsymbol{G^{\text{FANG}}}$);

    2. 2.2.

GraphSage model using FANG network information, as well as TF.IDF
representation for the tweets and news, denoted as FANG.

###  4.6 Evaluation Metrics

We evaluate our methods using the existing training and testing splits (i.e.,
trained on 10%, 30%, 50%, 70%, and 90% of all data) provided by [19]. Hence,
we test all the models’ performances using the same testing sets, and the
performances are therefore comparable those numbers reported in [19].

As evaluation metrics, we report macro Accuracy, Precision, Recall, and F1, as
well as Area under the ROC Curve (AUC) – indeed, we go further than previous
work on the SD dataset, which focused solely upon AUC. Moreover, in order to
visualise the effectiveness of the users’ embeddings in identifying groups of
people engaged with fake news versus factual news, we apply the PCA dimension
reduction technique on the users’ embeddings. For the PCA technique, we also
report the cumulative explained variation (CEV), which analyses the variation
for individual components, and sums up the variation of the $k$ ($k=50$ in our
experiments) most varied principle components. This metric shows the variation
between two groups of users (i.e., users that have engaged with fake news vs.
users that have never engaged with fake news) that are embedded using the
friendship/followers networks.

(a) Friendship network.

(b) Follower network.

(c) Random user network embeddings.

Figure 3: Unsupervised embedded users shown in PCA mapping. (a) and (b) are
trained using DeepWalk, while (c) uses randomly generated user network
embeddings.

##  5 Results and Analysis

In this section, we present the results from our experiments to answer our
three research questions. Furthermore, we conduct a case study and discuss the
implications of the obtained results.

###  5.1 RQ1: Clustering Effect of Users’ Network Embeddings

To gauge the effectiveness of unsupervised user network embeddings in
identifying clusters where users engage with fake news, we visualise the
distributions of users in our embedded user networks, using the PCA
dimensionality reduction technique888Plots using the tSNE dimensionality
reduction technique produced similar observations, and hence are omitted for
reasons of brevity., and measure the cumulative explained variation (CEV)
between users who engage with fake news and users who engage with factual
news. Specifically, we visualise the users’ embeddings trained with the
DeepWalk graph embedding method, since DeepWalk provides a lower bound in user
embeddings performance, due to its simplicity compared to the Cluster-GCN and
the GraphSAGE methods.

Figures 3(a) & 3(b) illustrate the distribution of user embeddings (learned
from the friendship network and the followers network with the DeepWalk graph
embedding method) that are reduced to 2 dimensions using a PCA. The red dots
denote users that have engaged with at least one fake news article, and the
teal dots denote users that only engaged with factual news articles. Note that
as mentioned earlier, during the training session, we did not label users as
engaged with factual news or engaged with fake news, hence the embeddings are
learned in an unsupervised fashion. In Figure 3(c), we also show randomly
generated user embeddings plotted using PCA, where, unlike in Figures 3a and
3b, the distributions of users engaging with fake news and with factual news
are more uniform.

Specifically, from Figure 3a, it can be observed that for the user embeddings
learned from the friendship network, users who are engaged with fake news are
more tightly clustered together. That is, the PCA mapping shows that users who
engaged with fake news are tightly gathered around the top left corner, which
suggests that the users who engage in fake news are more tightly grouped into
smaller echo chambers than the users who do not engage in fake news, as
observed by Yoo [8]. Moreover, the CEV analysis shows that the top 50
principle components achieved a cumulative explained variation of 0.8472,
indicates that the two groups of users have a variance of 84.74% calculated
from the embeddings, learned from the friendship network. In Figure 3(b), we
observe similar trends from the user embeddings learned from the followers
network, with the CEV for the 50 principle components being 0.8472, indicates
that there are valid differences between the two user groups, in comparison to
the randomly generated user embeddings, where the CEV is 0.5863. Our results
echo the findings in [49], namely that users tend to form a more tightly
grouped community when they are more heavily influenced by fake news, and show
that echo chamber effects indeed exist in social media.

Thus, in response to RQ1, we conclude that the unsupervised Twitter user
network embeddings can indeed cluster users into different groups (i.e., users
who have engaged with fake news, versus users who only engaged with factual
news), with a cumulative explained variation of around 85%, using either the
follower network or the friendship network.

Table 3: Performance comparison among the models using 90% training data. Numbers in the significance column indicates that the model is significantly better than the numbered model (McNemar’s Test, $p$$<$$0.01$).  # | Model | Accuracy | P | R | F1 | AUC | Significance
---|---|---|---|---|---|---|---
Textual Baselines – Textual features only
1 | Random | 0.4737 | 0.3205 | 0.5000 | 0.3906 | 0.5000 | -
2 | SVM TF.IDF($a_{n}$ and $T_{n}$) | 0.6068 | 0.6010 | 0.6095 | 0.5962 | 0.6095 | 1, 4
3 | BERT($a_{n}$) | 0.5897 | 0.5584 | 0.5595 | 0.5588 | 0.5595 | 1, 4
4 | BERT($T_{n}$) | 0.5299 | 0.5410 | 0.5443 | 0.5249 | 0.5443 | 1
Network baselines from FANG – Complex network features and texutal features
(publisher, citation, follower network, stance and TF.IDF features)
5 | GCN($G^{\text{FANG}}$) | 0.6458 | 0.6328 | 0.6250 | 0.6262 | 0.7125 | 1-4, 7
6 | FANG | 0.6875 | 0.6799 | 0.6821 | 0.6807 | 0.7518 | 1-5, 7,9
UNES variants – Unsupervised network features only
7 | DeepWalk($G^{\text{fr}}$) | 0.6410 | 0.5717 | 0.5052 | 0.4122 | 0.5052 | 1-4
8 | Cluster-GCN($G^{\text{fr}}$) | 0.7083 | 0.7083 | 0.7142 | 0.7062 | 0.7071 | 1-7,9
9 | Cluster-GCN($G^{\text{fo}}$) | 0.6667 | 0.6556 | 0.6500 | 0.6515 | 0.7000 | 1-5,7
10 | GraphSAGE($G^{\text{fr}}$) | 0.7708 | 0.7650 | 0.7607 | 0.7625 | 0.7498 | 1-9,11
11 | GraphSAGE($G^{\text{fo}}$) | 0.7292 | 0.7222 | 0.7250 | 0.7233 | 0.7365 | 1-9

(a) Accuracy.

(b) F1 score.

(c) ROC AUC

Figure 4: Performances of GraphSAGE($G^{\text{fr}}$),
GraphSAGE($G^{\text{fo}}$), Cluster-GCN($G^{\text{fr}}$), Cluster-
GCN($G^{\text{fo}}$), GCN($G^{\text{FANG}}$), and the FANG model, on accuracy,
F1, and AUC. § in the legend denotes that our corresponding model variant
significantly outperforms FANG, on all training percentages.

###  5.2 RQ2: UNES Model for Fake News Classification

To evaluate whether our proposed UNES model is more effective in detecting
fake news on the SD dataset w.r.t. the baseline models, as well as to identify
the most effective graph embedding method for fake news detection on Twitter,
we compare our various UNES instantiations with the baselines described
earlier in terms of classification performances. Table 3 shows how the models
perform when trained with 90% of all data, validated with 5% of the data, and
tested on 5% of the data. Indeed, we use the pre-partitioned training-testing
sets provided by [19], thus the results shown in Table 3 are comparable to
those reported in [19]. To analyse the effect of the size of training data on
the results, we also evaluate the various models across different training
data size percentages, and present the results in Figure 4, in terms of
Accuracy, F1 and ROC AUC.

On analysing Table 3, we firstly observe that the text-based baselines (i.e.,
the TF.IDF and BERT models, lines 2-4 in Table 3) can identify fake news
significantly better the random baseline. However, a simple SVM(TF.IDF) model
outperforms both BERT models, indicating the difficulties for contextual
features to analyse news and tweets on Twitter, with the SD dataset. Moreover,
all models that use network features (lines 5-11) significantly outperform the
textual features only models (lines 1-4). This observation suggests that
network features are more successful at identifying fake news on Twitter than
textual features alone.

Secondly, we observe that with only the network information, our UNES model
with the simpler DeepWalk graph embedding (line 7 in Table 3) does not
outperform either of the network baselines (lines 5 & 6). On the other hand,
contrary to DeepWalk, the use of the UNES model along the advanced graph
embedding models (i.e., Cluster-GCN and GraphSAGE) can achieve better accuracy
and F1 performances, compared to the GCN($G^{\text{FANG}}$) and FANG,
respectively. Indeed, Cluster-GCN with friendship/followers networks (lines 8
& 9) can significantly outperform GCN($G^{\text{FANG}}$) (line 5), while
GraphSAGE with the friendship/followers networks (lines 10 & 11) significantly
outperforms FANG (line 6). However, FANG (line 6) obtained the highest ROC AUC
performance among all the tested models. A further inspection of all the
evaluated UNES variants (i.e., lines 7-11 in Table 3) shows that while they
are accurate, they are generally less certain in their predictions.
Specifically, the dense neural network layer outputs posterior probabilities
for each class closer to 0.5 rather than 0 or 1 for our binary classification
task (i.e., classifying a given news as fake or not), while the FANG baseline
tends to produce probability outputs closer to 0 and 1 rather than 0.5. We
argue that this is because the UNES model uses a network structure data with
no textual information on the content of the news or tweets, unlike FANG,
which uses both the textual data (the stance and TF.IDF representations of the
news articles and the tweets) along with the network data. We leave to future
work, the investigation of how best to integrate the textual data into the
UNES model.

Furthermore, Figure 4 shows the performances of the UNES variants and the FANG
baselines when tested with all possible training data size percentages. From
the figure, we observe that the UNES variant using the
GraphSAGE($G^{\text{fr}}$) graph embeddings consistently outperforms FANG in
terms of the accuracy and F1 metrics, regardless of the used training data
size. However, similar to the observation that FANG achieved the highest ROC
scores in Table 3, we observe that all the variants of our UNES model (i.e.,
Cluster-GCN($G^{\text{fr}}$), Cluster-GCN($G^{\text{fo}}$),
GraphSAGE($G^{\text{fr}}$), and GraphSAGE($G^{\text{fo}}$)) do not outperform
FANG consistently, due to the aforementioned issue, namely that model
GraphSAGE($G^{\text{fr}}$) tends to predict posterior probabilities closer to
0.5 rather than 0 or 1.

Moreover, from Figure 4 we observe that all the GCN-based models (i.e.,
Cluster-GCN($G^{\text{fr}}$), Cluster-GCN($G^{\text{fo}}$), and
GCN($G^{\text{FANG}}$)) suffer from instability when trained using different
percentages of training data. Specifically, the Cluster-GCN($G^{\text{fr}}$)
and Cluster-GCN($G^{\text{fo}}$) approaches both exhibit performance drops
when trained with 30% and 70% of the data. One of the baseline models,
GCN($G^{\text{FANG}}$), shows a drop in the Accuracy and F1 performances when
trained with 50% of the data, and a ROC AUC drop when trained with 70% of the
data. Indeed, the GCN-based models have unstable performances when tested on
the different training percentages. This suggests that the GCN graph embedding
model might represent nodes (i.e., users in our experiments) inaccurately as
embeddings, as GCN and Cluster-GCN both identify subgraphs before computing
the embeddings of each node, while the subgraphs are not updated throughout
the training session, contrary to the GraphSAGE model, which aims to perform
graph embeddings without any subgraph partitioning.

Overall, in response to RQ2, we conclude that the models that use the users’
network embeddings alone significantly outperform the language model baselines
in classifying fake news on the SD dataset. Moreover, our models that use
unsupervised users’ network embeddings can identify fake news on the SD
dataset more accurately than the FANG baseline, which uses complex users’
network embeddings that include additional relations such as the publisher
network and the citation network, as well as the textual information from the
tweets and news articles. Among the variants of our proposed UNES model, the
variant using the GraphSAGE graph embeddings is the most effective.

###  5.3 RQ3: Followers or Friends?

Recall that users can have two types of relations with other users, namely
through the following relation or through the friendship relation, where the
latter requires both users to follow each other. As discussed before, we
instantiate the UNES model using either the follower or the friendship
relationships for both the GraphSAGE and Cluster-GCN graph embedding models.
Their respective performances are included in Table 3 and Figure 4. On
analysing Table 3, we observe that with 90% of the data as training data,
GraphSAGE($G^{\text{fr}}$) outperforms GraphSAGE($G^{\text{fo}}$) on all
metrics, while Cluster-GCN($G^{\text{fr}}$) outperforms Cluster-
GCN($G^{\text{fo}}$) on all metrics. Figure 4 shows that
GraphSAGE($G^{\text{fr}}$) outperforms GraphSAGE($G^{\text{fo}}$) consistently
on all metrics, regardless of the portion of training data used.

To understand this result, we analyse the statistical differences between the
friendship network and the follower network in the SD dataset to investigate
the dissimilarity of their embeddings. Recall Table 2, where it can be seen
that on average, users have a higher number of edges in the follower network
than in the friendship network (1565.16 vs. 1388.37), showing that on average
users have more followers than friends, echoing the fact that the friendship
network on Twitter forms a more sparse network than the follower network.
Although, intuitively speaking, the denser the network, the more information
we can collect, we argue that the denser followers network may introduce more
noise due to the lack of shared interests between followers and followees,
compared to friends that share more interests and similar opinions, as
represented by the friendship network.

Furthermore, the higher accuracy of the friends graph is advantageous from a
data point of view: as noted that we are limited in terms of possible Twitter
API calls, which reduces the observable portion of the users’ friends or
follower networks. Indeed, from Table 2 it can be observed that the friendship
network has a smaller proportion of users with more than 5000 friends (i.e.,
10.64% of all news-engaging users in the SD dataset do not have all their
friends downloaded); in contrast, the follower network has 6263 users with
more than 5000 followers (17.50% of all news-engaging users). Overall, in
response to RQ3, we conclude that in our experiments, with a limited number of
Twitter API calls available, the users’ friendship network is the most
effective type of social network relationships, when used to construct the
users’ graph embeddings and for detecting fake news circulating on Twitter.

Table 4: Case study examples. The Fake? column indicates if the news article is fake news (Y) or not (N); # $T_{n}$ describes the number of tweets are associated with the news article. Case | Fake? | News article title | # $T_{n}$
---|---|---|---
UNES correct, BERT incorrect |
1 | N | At Trump hotel site, immigrant workers wary | 462
2 | N | Charlie Hebdo : Le témoignage de la dessinatrice Coco. | 132
3 | N | Coldsip.com | News | 155
4 | Y | Illegal Immigrant Deported 6 Times Charged in Felony Hit-and-Run of Family that Injured Children. | 588
5 | Y | Sasha Obama Just Crashed Her Expensive New Car Into A Lake. | 246
6 | Y | Hundreds of people died after eating the Patti LaBelle brand Patti Sweet Potato Pie. | 227
UNES incorrect, BERT correct |
7 | N | UCLA Students protest after partygoers wear blackface at fraternity party. | 246
8 | N | Killer to face firing squad. | 539
9 | N | Caitlyn Jenner to receive courage award at ESPY’s | 797
10 | Y | Obama Orders Chicago School to Let ‘Transgender’ Boy Use Girls’ Locker Room. | 398
11 | Y | Peanut Butter and Jelly Deemed Racist. | 835
12 | Y | UPDATE: Second Roy Moore Accuser Works For Michelle Obama Right NOW. | 3913
Figure 5: PCA mapping of the users engaged with 2 news articles related to
immigration issues, on Friendship network. Red dots represent users who
engaged with fake news “Illegal Immigrant Deported 6 Times Charged in Felony
Hit-and-Run of Family that Injured Children”, while teal dots represent users
that engaged in factual news “At Trump hotel site, immigrant workers wary".

###  5.4 Case Study

In order to better demonstrate the successes and failures of our proposed UNES
model, we present in Table 4 a case study. In the table, in cases 1-6 UNES
makes a correct prediction where the BERT model does not, while in cases 7-12
are for the opposite condition. We also report the number of tweets discussing
each news article.

Indeed, from cases 1-3 in the table, we observe that our UNES model can
classify the news correctly regardless of the language (case 2), or if the
title is redacted (case 3). These cases show that our proposed model can be
used universally across different language, and when the news content is
corrupted. Cases 4-6 in Table 4 shows that the language used in the news title
can mislead the language models into misclassifying the news as genuine, while
our proposed UNES model can correctly classify the news as fake with
additional information from the users engaged in such news.

On the other hand, cases 7-12 show difficult cases for the UNES model. These
cases are mostly related to significant and controversial topics (such as
racism, politician scandals, transgender right, immigrants), where people with
different views can be easily drawn to such news, and voice their opinions. We
believe these cases are indeed difficult to identify with a cluster-based
model.

Furthermore, case 1 (factual) and case 4 (fake) are two news articles related
toimmigration issues in the US, that the UNES model correctly classified.
Figure 5 shows the PCA mapping of the users engaged in these two cases. We
observe two embedding clusters from Figure 5, one among users who commented on
the fake news (case 4) and one among users who commented on the factual news
(case 1). These two user clusters illustrate a clearer echo chamber effect on
the immigration issue, than the lack of separation among the clusters observed
in Figure 3(a). We believe these two clearly separated user clusters explain
why UNES accurately classified these two cases. In order to further improve
accuracy of identifying fake news, in the future we aim to combine UNES with
language models to enhance the classification accuracy in such difficult
examples.

##  6 Conclusions and Limitations

In this paper, we have demonstrated that:

  * •

User network embedding methods can cluster the users who engaged in fake news
more closely than users who only engaged with factual news, observing the echo
chamber effects, with $CEV\approx 0.84$;

  * •

Our proposed UNES model, using unsupervised network embedding model trained
with the users’ relationships, can identify the users clusters that engage
with fake news, when tested on a large recent Twitter dataset;

  * •

With the readily available information (i.e., only the followers/friendship
networks), the UNES model can identify fake news more accurately than the
existing SOTA models such as FANG (uses complex network information that
requires additional labelling) and text-based models;

  * •

The friendship network can more accurately help identify fake news than the
follower network, within the limited access afforded to the Twitter social
networks given the available Twitter API rate.

However, we also identify several limitations:

  * •

The current UNES model does not perform well when combined with language
representations, thus it’s important to identify better mechanisms that can
leverage both types of information (i.e., user network embeddings and language
embeddings);

  * •

User networks are dynamic and are constantly changing, how to adapt to the
changing user network representation is worth studying, thus help identifying
new types of fake news clusters and their associated fake news;

  * •

Develop methods to better handle cold-start users (i.e., the new accounts
without social connections and/or tweets).

## References

  * [1] Carl A Latkin, Lauren Dayton, Justin C Strickland, Brian Colon, Rajiv Rimal, and Basmattee Boodram.  An assessment of the rapid decline of trust in US sources of public information about COVID-19.  Journal of Health Communication, 25(10):764–773, 2020.
  * [2] Markus Prior.  Media and political polarization.  Annual Review of Political Science, 16:101–127, 2013.
  * [3] Paul Pierson and Eric Schickler.  Madison’s constitution under stress: A developmental analysis of political polarization.  Annual Review of Political Science, 23:37–58, 2020.
  * [4] Danielle Polage.  Making up history: False memories of fake news stories.  Europe’s Journal of Psychology, 8(2), 2012.
  * [5] Gordon Pennycook, Tyrone D Cannon, and David G Rand.  Prior exposure increases perceived accuracy of fake news.  Journal of Experimental Psychology: general, 147(12):1865, 2018\.
  * [6] Jaeho Cho, Saifuddin Ahmed, Martin Hilbert, Billy Liu, and Jonathan Luu.  Do search algorithms endanger democracy? an experimental investigation of algorithm effects on political polarization.  Journal of Broadcasting & Electronic Media, 64(2):150–172, 2020\.
  * [7] Rich Ling.  Confirmation bias in the era of mobile news consumption: the social and psychological dimensions.  Digital Journalism, 8(5):596–604, 2020.
  * [8] Jieun Yoo.  Ideological homophily and echo chamber effect in internet and social media.  Student International Journal of Research, 4(1):1–7, 2007.
  * [9] Stephan Lewandowsky, Ullrich KH Ecker, Colleen M Seifert, Norbert Schwarz, and John Cook.  Misinformation and its correction: Continued influence and successful debiasing.  Psychological Science in the Public Interest, 13(3):106–131, 2012\.
  * [10] Sardar Hamidian and Mona Diab.  Rumor identification and belief investigation on Twitter.  In WASSA@ACL, pages 3–8, 2016.
  * [11] Tadao Kasami.  An efficient recognition and syntax-analysis algorithm for context-free languages.  Coordinated Science Laboratory Report no. R-257, 1966.
  * [12] Song Feng, Ritwik Banerjee, and Yejin Choi.  Syntactic stylometry for deception detection.  In ACL, pages 171–175, 2012.
  * [13] Giovanni Luca Ciampaglia, Prashant Shiralkar, Luis M Rocha, Johan Bollen, Filippo Menczer, and Alessandro Flammini.  Computational fact checking from knowledge networks.  PloS one, 10(6):e0128193, 2015.
  * [14] Jing Ma, Wei Gao, Prasenjit Mitra, Sejeong Kwon, Bernard J Jansen, Kam-Fai Wong, and Meeyoung Cha.  Detecting rumors from microblogs with recurrent neural networks.  In IJCAI, pages 3818–3824, 2016.
  * [15] Beakcheol Jang, Inhwan Kim, and Jong Wook Kim.  Word2vec convolutional neural networks for classification of news articles and tweets.  PloS one, 14(8):e0220976, 2019.
  * [16] Gang Liang, Wenbo He, Chun Xu, Liangyin Chen, and Jinquan Zeng.  Rumor identification in microblogging systems based on users’ behavior.  TCSS, 2(3):99–108, 2015.
  * [17] Natali Ruchansky, Sungyong Seo, and Yan Liu.  CSI: A hybrid deep model for fake news detection.  In CIKM, pages 797–806, 2017.
  * [18] Ting Su, Craig Macdonald, and Iadh Ounis.  Entity detection for check-worthiness prediction: Glasgow Terrier at CLEF CheckThat! 2019.  In CLEF, 2019.
  * [19] Van-Hoang Nguyen, Kazunari Sugiyama, Preslav Nakov, and Min-Yen Kan.  Fang: Leveraging social context for fake news detection using graph representation.  In KDD, pages 1165–1174, 2020.
  * [20] Karen Sparck Jones.  A statistical interpretation of term specificity and its application in retrieval.  Journal of Documentation, 28(1), 1972.
  * [21] Eric Brill.  A simple rule-based part of speech tagger.  In CANLP, pages 152–155, 1992.
  * [22] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean.  Efficient estimation of word representations in vector space.  arXiv preprint arXiv:1301.3781, 2013.
  * [23] Luca Favano, M Carman, and Jakob Grue Simonsen.  Theearthisflat’s submission to CLEF’19CheckThat! challenge.  In CLEF, pages 1–11, 2019.
  * [24] Elena Kochkina, Maria Liakata, and Isabelle Augenstein.  Turing at SemEval-2017 task 8: Sequential approach to rumour stance classification with Branch-LSTM.  In SemEval-2017, pages 475–480, 2017.
  * [25] Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer.  Deep contextualized word representations.  In HLT/NAACL, pages 2227–2237, 2018.
  * [26] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In NAACL, pages 4171–4186, 2019.
  * [27] Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang.  BioBERT: a pre-trained biomedical language representation model for biomedical text mining.  Bioinformatics, 36(4):1234–1240, 2020.
  * [28] Zeynep Akkalyoncu Yilmaz, Shengjin Wang, Wei Yang, Haotian Zhang, and Jimmy Lin.  Applying BERT to document retrieval with Birch.  In EMNLP, pages 19–24, 2019.
  * [29] Ting Su, Craig Macdonald, and Iadh Ounis.  Ensembles of recurrent networks for classifying the relationship of fake news titles.  In SIGIR, pages 893–896, 2019.
  * [30] Vahed Qazvinian, Emily Rosengren, Dragomir Radev, and Qiaozhu Mei.  Rumor has it: Identifying misinformation in microblogs.  In EMNLP, pages 1589–1599, 2011.
  * [31] Zhe Zhao, Paul Resnick, and Qiaozhu Mei.  Enquiring minds: Early detection of rumors in social media from enquiry posts.  In WWW, pages 1395–1405, 2015.
  * [32] Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu.  Fake news detection on social media: A data mining perspective.  In KDD, pages 22–36, 2017.
  * [33] Xinyi Zhou and Reza Zafarani.  Network-based fake news detection: A pattern-driven approach.  In KDD, pages 48–60, 2019.
  * [34] Zhiwei Jin, Juan Cao, Yongdong Zhang, and Jiebo Luo.  News verification by exploiting conflicting social viewpoints in microblogs.  In AAAI, pages 2972–2978, 2016.
  * [35] Daniel Röchert, Gautam Kishore Shahi, German Neubaum, Björn Ross, and Stefan Stieglitz.  The networked context of COVID-19 misinformation: Informational homogeneity on YouTube at the beginning of the pandemic.  Online Social Networks and Media, 26:100164, 2021.
  * [36] Alexandra Sosnkowski, Carol J Fung, and Shivram Ramkumar.  An analysis of twitter users’ long term political view migration using cross-account data mining.  Online Social Networks and Media, 26:100177, 2021.
  * [37] Aditya Grover and Jure Leskovec.  node2vec: Scalable feature learning for networks.  In KDD, pages 855–864, 2016.
  * [38] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena.  Deepwalk: Online learning of social representations.  In KDD, pages 701–710, 2014.
  * [39] Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, and Cho-Jui Hsieh.  Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks.  In KDD, pages 257–266, 2019.
  * [40] William L Hamilton, Rex Ying, and Jure Leskovec.  Inductive representation learning on large graphs.  In NIPS, pages 1025–1035, 2017.
  * [41] Thomas Kipf and Max Welling.  Semi-supervised classification with graph convolutional networks.  arXiv preprint arXiv:1609.02907, 2016.
  * [42] Jianfei Chen, Jun Zhu, and Le Song.  Stochastic training of graph convolutional networks with variance reduction.  In ICML, pages 942–950, 2018.
  * [43] Andrea Rossi, Denilson Barbosa, Donatella Firmani, Antonio Matinata, and Paolo Merialdo.  Knowledge graph embedding for link prediction: A comparative analysis.  ACM Transactions on Knowledge Discovery from Data, 15(2):1–49, 2021\.
  * [44] Seyed Mehran Kazemi and David Poole.  Simple embedding for link prediction in knowledge graphs.  In NIPS, pages 4289–4300, 2018.
  * [45] Xingbo Du, Junchi Yan, and Hongyuan Zha.  Joint link prediction and network alignment via cross-graph embedding.  In IJCAI, pages 2251–2257, 2019.
  * [46] Federico Monti, Fabrizio Frasca, Davide Eynard, Damon Mannion, and Michael M Bronstein.  Fake news detection on social media using geometric deep learning.  arXiv preprint arXiv:1902.06673, 2019.
  * [47] Liang Wu and Huan Liu.  Tracing fake-news footprints: Characterizing social media messages by how they propagate.  In WSDM, pages 637–645, 2018.
  * [48] Bhavtosh Rath, Aadesh Salecha, and Jaideep Srivastava.  Detecting fake news spreaders in social networks using inductive representation learning.  arXiv preprint arXiv:2011.10817, 2020.
  * [49] Petter Törnberg.  Echo chambers and viral misinformation: Modeling fake news as complex contagion.  PloS one, 13(9):e0203958, 2018.


* Tibshirani and Saunders (2005) Tibshirani, S. and Saunders, M. and Rosset, S. and Zhu, J. and Knight, K. (2005). Sparsity and smoothness via the fused lasso. Journal of Royal Statistical Society, Series B, 67, 91 108.
  * Tibshirani (1996) Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of Royal Statistical Society, Series B, 58, 267-288.
  * Tseng (2001) Tseng, P. (2001). Convergence of a block coordinate descent method for nondifferentiable minimization. Journal of Optimization Theory and Applications, 109, 475-494.
  * Tsiatis et al. (2007) Tsiatis, A. A. and Davidian, M. and Zhang, M. and Lu, X. (2007). Covariate adjustment for two-sample treatment comparisons in randomized clinical trials: a principled yet flexible approach. Statistics in Medicine, 27, 4658-4677.
  * van de Geer and Bühlmann (2009) van de Geer, S. A. and Bühlmann, P. (2007). On the conditions used to prove oracle results for the Lasso. Electronic Journal of Statistics, 3, 1360-1392.
  * Wang et al. (2007) Wang, H. and Li, R. and Tsai, C. L. (2007). Tuning parameter selectors for the smoothly clipped absolute deviation method. Biometrika, 94, 553-568.
  * Wei and Huang (2010) Wei, F. and Huang, J. (2010). Consistent group selection in high-dimensional linear regression. Bernoulli, 16, 1369-1384.
  * Yuan and Lin (2006) Yuan, M. and Lin, Y. (2006). Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B, 68, 49 67.
  * Zhang (2010) Zhang, C. (2010). Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 38, 894-942.


* Irnich and Steiner, (2018) Irnich, S. and Steiner, K. (2018).  Schedule-based integrated intercity bus line planning via branch-and-cut.  Transportation Science, 52(4):882–897.
  * Jäger and Schöbel, (2020) Jäger, S. and Schöbel, A. (2020).  The blockwise coordinate descent method for integer programs.  Mathematical Methods of Operations Research, 91:357–381.
  * Jansson, (1980) Jansson, J. O. (1980).  A simple bus line model for optimisation of service frequency and bus size.  Journal of Transport Economics and Policy, pages 53–80.
  * (89) Jara-Díaz, S. and Gschwender, A. (2003a).  Towards a general microeconomic model for the operation of public transport.  Transport Reviews, 23(4):453–469.
  * (90) Jara-Díaz, S. R. and Gschwender, A. (2003b).  From the single line model to the spatial structure of transit services: corridors or direct?  Journal of Transport Economics and Policy (JTEP), 37(2):261–277.
  * Jara-Díaz and Gschwender, (2009) Jara-Díaz, S. R. and Gschwender, A. (2009).  The effect of financial constraints on the optimal design of public transport services.  Transportation, 36:65–75.
  * Jha et al., (2019) Jha, S. B., Jha, J. K., and Tiwari, M. K. (2019).  A multi-objective meta-heuristic approach for transit network design and frequency setting problem in a bus transit system.  Computers & Industrial Engineering, 130:166–186.
  * Jiang et al., (2017) Jiang, F., Cacchiani, V., and Toth, P. (2017).  Train timetabling by skip-stop planning in highly congested lines.  Transportation Research B, 104:149–174.
  * Kaspi and Raviv, (2013) Kaspi, M. and Raviv, T. (2013).  Service-oriented line planning and timetabling for passenger trains.  Transportation Science, 47(3):295–311.
  * Kepaptsoglou and Karlaftis, (2009) Kepaptsoglou, K. and Karlaftis, M. (2009).  Transit route network design problem: Review.  Journal of transportation engineering, 135(8):491–505.
  * Kepaptsoglou et al., (2010) Kepaptsoglou, K., Karlaftis, M. G., and Li, Z. (2010).  The bus bridging problem in metro operations: models and algorithms.  Technical report.
  * Klier and Haase, (2015) Klier, M. J. and Haase, K. (2015).  Urban public transit network optimization with flexible demand.  Or Spectrum, 37:195–215.
  * Kontogiannis and Zaroliagis, (2008) Kontogiannis, S. and Zaroliagis, C. (2008).  Robust line planning under unknown incentives and elasticity of frequencies.  In Fischetti, M. and Widmayer, P., editors, ATMOS 2008 - 8th Workshop on Algorithmic Approaches for Transportation Modeling, Optimization, and Systems, Dagstuhl, Germany.
  * Kouvelis and Yu, (1997) Kouvelis, P. and Yu, G. (1997).  Robust Discrete Optimization and Its Applications.  Kluwer Academic Publishers.
  * Laporte et al., (2011) Laporte, G., Marin, A., Mesa, J. A., and Ortega, F. (2011).  Designing robust rapid transit networks with alternative routes.  Journal of Advanced Transportation, 45:54–65.
  * Laporte et al., (2005) Laporte, G., Mesa, J., and Ortega, F. (2005).  Maximizing trip coverage in the location of a single rapid transit alignment.  Annals of Operations Research, 136:49–63.
  * Laporte et al., (2010) Laporte, G., Mesa, J., and Perea, F. (2010).  A game theoretic framework for the robust railway transit network design problem.  Transportation Research B, 44(4):447–459.
  * Laporte et al., (2007) Laporte, G., n, A. M., Mesa, J., and Ortega, F. (2007).  An integrated methodology for rapid transit network design.  In Algorithmic Methods for Railway Optimization, number 4359 in Lecture Notes on Computer Science, pages 187–199. Springer.
  * Lee and Nair, (2021) Lee, C. and Nair, R. (2021).  Robust transit line planning based on demand estimates obtained from mobile phones.  EURO JOurnal on TRansportation and Logistics, 10:100034.
  * Lee et al., (2014) Lee, Y.-J., Shariat, S., and Choi, K. (2014).  Optimizing skip-stop rail transit stopping strategy using a genetic algorithm.  Journal of Public Transportation, 17(2).
  * Li et al., (2023) Li, C., Tang, J., Zhang, J., Zhao, Q., Wang, L., and Li, J. (2023).  Integrated optimization of urban rail transit line planning, timetabling and rolling stock scheduling.  PLoS ONE, 18(5):e0285932.
  * Liebchen et al., (2009) Liebchen, C., Lübbecke, M., Möhring, R. H., and Stiller, S. (2009).  The concept of recoverable robustness, linear programming recovery, and railway applications.  In Ahuja, R. K., Möhring, R., and Zaroliagis, C., editors, Robust and online large-scale optimization, volume 5868 of Lecture Note on Computer Science, pages 1–27. Springer.
  * Liebchen and Möhring, (2007) Liebchen, C. and Möhring, R. (2007).  The modeling power of the periodic event scheduling problem: Railway timetabless and beyond.  In Algorithmic Methods for Railway Optimization, volume 4359 of Lecture Notes in Computer Science, pages 3–40. Springer.
  * Limsawasd et al., (2022) Limsawasd, C., Athigakunagorn, N., Khathawatcharakun, P., and Boonmee, A. (2022).  Skip-stop strategy patterns optimization to enhance mass transit operation under physical distancing policy due to covid 19 pandemic outbreak.  Transport Policy, 126:225–238.
  * Liu et al., (2013) Liu, Z., Yan, Y., Qu, X., and Zhang, Y. (2013).  Bus stop-skipping scheme with random travel time.  Transportation Research C, 35:46–56.
  * Lumiplan, (2023) Lumiplan (2023).  Heurès - La solution de graphicage-habillage-commande de services.  https://www.lumiplan.com/produit/heures/.  retrieved 2023-12, in French.
  * Luo et al., (2019) Luo, D., Cats, O., van Lint, H., and Currie, G. (2019).  Integrating network science and public transport accessibility analysis for comparative assessment.  Journal of Transport Geography, 80:102505.
  * Lusby et al., (2018) Lusby, R., Larsen, J., and Bull, S. (2018).  A survey on robustness in railway planning.  European Journal of Operational Research, 266:1–15.
  * Marín et al., (2009) Marín, A., Mesa, J., and Perea, F. (2009).  Integrating robust railway network design and line planning under failures.  In Robust and Online Large-Scale Optimization, volume 5868 of Lecture Notes in Computer Science, pages 273–292. Springer.
  * Martínez et al., (2014) Martínez, H., Mauttone, A., and Urquhart, M. E. (2014).  Frequency optimization in public transportation systems: Formulation and metaheuristic approach.  European Journal of Operational Research, 236(1):27–36.
  * Masing et al., (2022) Masing, B., Lindner, N., and Borndörfer, R. (2022).  The price of symmetric line plans in the parametric city.  Transportation Research Part B: Methodological, 166:419–443.
  * Mauttone and Urquharta, (2009) Mauttone, A. and Urquharta, M. E. (2009).  A route set construction algorithm for the transit network design problem.  Computers & Operations Research, 36(8):2440–2449.
  * Michaelis and Schöbel, (2009) Michaelis, M. and Schöbel, A. (2009).  Integrating line planning, timetabling, and vehicle scheduling: A customer-oriented approach.  Public Transport, 1(3):211–232.
  * Migl et al., (2024) Migl, A., Friedrich, M., Schiewe, A., and Schöbel, A. (2024).  Die Lösbarkeit von algorithmischen Linienplänen mit Systemrouten verbessern.  In Proc. Heureka 2024.  (in German).
  * Mohring, (1972) Mohring, H. (1972).  Optimization and scale economies in urban bus transportation.  The American Economic Review, 62(4):591–604.
  * Mou et al., (2020) Mou, Z., Zhang, H., and Liang, S. (2020).  Reliability optimization model of stop-skipping bus operation with capacity constraints.  Hindawi Journal of Advanced Transportation, page 4317402.
  * Müller-Hannemann et al., (2021) Müller-Hannemann, M., Rückert, R., Schiewe, A., and Schöbel, A. (2021).  Towards Improved Robustness of Public Transport by a Machine-Learned Oracle.  In Müller-Hannemann, M. and Perea, F., editors, 21st Symposium on Algorithmic Approaches for Transportation Modelling, Optimization, and Systems (ATMOS 2021), volume 96 of Open Access Series in Informatics (OASIcs), pages 3:1–3:20. Schloss Dagstuhl – Leibniz-Zentrum für Informatik.
  * Müller-Hannemann et al., (2022) Müller-Hannemann, M., Rückert, R., Schiewe, A., and Schöbel, A. (2022).  Estimating the robustness of public transport schedules using machine learning.  Transportation Research Part C: Emerging Technologies, 137:103566.
  * Nachtigall and Jerosch, (2008) Nachtigall, K. and Jerosch, K. (2008).  Simultaneous network line planning and traffic assignment.  In Fischetti, M. and Widmayer, P., editors, ATMOS 2008 - 8th Workshop on Algorithmic Approaches for Transportation Modeling, Optimization, and Systems, Dagstuhl, Germany.  http://drops.dagstuhl.de/opus/volltexte/2008/1589.
  * Nguyen and Pallottino, (1988) Nguyen, S. and Pallottino, S. (1988).  Equilibrium traffic assignment for large scale transit networks.  European journal of operational research, 37(2):176–186.
  * Nöllenburg and Wolff, (2011) Nöllenburg, M. and Wolff, A. (2011).  Drawing and labeling high-quality metro maps by mixed-integer programming.  IEEE Transactions on Visualization and Computer Graphics, 17(5):626–641.
  * Oltrogge, (1994) Oltrogge, C. (1994).  Linienplanung für mehrstufige Bedienungssysteme im öffentlichen Personenverkehr.  PhD thesis, TU Braunschweig.  (in German).
  * Otto and Boysen, (2017) Otto, B. and Boysen, N. (2017).  Zone-based tariff design in public transportation networks.  Networks, 69(4):349–366.
  * Park et al., (2013) Park, B. H., Seo, Y.-I., Hong, S.-P., and Rho, H.-L. (2013).  Column generation approach to line planning with various halting patterns—application to the korean high-speed railway.  Asia-Pacific Journal of Operational Research, 30(04):1350006.
  * Patz, (1925) Patz, A. (1925).  Die richtige Auswahl von Verkehrslinien bei großen Strassenbahnnetzen.  Verkehrstechnik, 50/51.  (in German).
  * Pätzold et al., (2017) Pätzold, J., Schiewe, A., Schiewe, P., and Schöbel, A. (2017).  Look-Ahead Approaches for Integrated Planning in Public Transportation.  In D’Angelo, G. and Dollevoet, T., editors, 17th Workshop on Algorithmic Approaches for Transportation Modelling, Optimization, and Systems (ATMOS 2017), volume 59 of OpenAccess Series in Informatics (OASIcs), pages 1–16, Dagstuhl, Germany. Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik.
  * Pternea et al., (2015) Pternea, M., Kepaptsoglou, K., and Karlaftis, M. G. (2015).  Sustainable urban transit network design.  Transportation Research Part A: Policy and Practice, 77:276–291.
  * Pu and Zhan, (2021) Pu, S. and Zhan, S. (2021).  Two-stage robust railway line-planning approach with passenger demand uncertainty.  Transportation Research Part E: Logistics and Transportation Review, 152:102372.
  * Rajabighamchi et al., (2019) Rajabighamchi, F., Hajlou, E. M. H., and Hassannayebi, E. (2019).  A multi-objective optimization model for robust skip-stopscheduling with earliness and tardiness penalties.  Urban Rail Transit, 5(3):172–185.
  * Rezazada et al., (2022) Rezazada, M., Nassir, N., and Tanin, E. (2022).  Public transport bunching: A critical review with focus on methods and findings for implications for policy and future research.  In Australasian Transport Research Forum (ATRF), 43rd, 2022, Adelaide, South Australia, Australia.
  * Ropke and Cordeau, (2009) Ropke, S. and Cordeau, J.-F. (2009).  Branch and cut and price for the pickup and delivery problem with time windows.  Transportation Science, 43(3):267–286.
  * Sadrani et al., (2022) Sadrani, M., Tirachini, A., and Antoniou, C. (2022).  Optimization of service frequency and vehicle size for automated bus systems with crowding externalities and travel time stochasticity.  Transportation Research Part C: Emerging Technologies, 143:103793.
  * Şahin et al., (2020) Şahin, G., Digehsara, A. A., Borndörfer, R., and Schlechte, T. (2020).  Multi-period line planning with resource transfers.  Transportation Research Part C: Emerging Technologies, 119:102726.
  * Schiewe et al., (2019) Schiewe, A., Schiewe, P., and Schmidt, M. (2019).  The line planning routing game.  European Journal of Operational Research, 274(2):560 – 573.
  * Schiewe et al., (2023) Schiewe, A., Schöbel, A., and Sieber, L. (2023).  Line planning for different demand periods.  Operations Research Forum, page 28 pages.  published online.
  * Schiewe, (2020) Schiewe, P. (2020).  Integrated Optimization in Public Transport Planning, volume 160 of Optimization and Its Applications.  Springer.
  * Schiewe et al., (2024) Schiewe, P., Jäger, S., and Schöbel, A. (2024).  LinTim - Integrated Optimization in Public Transportation. Homepage.  https://www.lintim.net/.  open source.
  * Schiewe and Schöbel, (2022) Schiewe, P. and Schöbel, A. (2022).  Integrated optimization of sequential processes: General analysis and application to public transport.  EURO Journal on Transportation and Logistics, 11:100073.
  * Schiewe and Stinzendörfer, (2022) Schiewe, P. and Stinzendörfer, M. (2022).  Integrated line planning and vehicle scheduling for public transport.  In 10th International Network Optimization Conference (INOC).
  * Schmidt, (2014) Schmidt, M. (2014).  Integrating Routing Decisions in Public Transportation Problems, volume 89 of Optimization and Its Applications.  Springer.
  * Schöbel, (2012) Schöbel, A. (2012).  Line planning in public transportation: models and methods.  OR Spectrum, 34(3):491–510.
  * Schöbel, (2017) Schöbel, A. (2017).  An eigenmodel for iterative line planning, timetabling and vehicle scheduling in public transportation.  Transportation Research C, 74:348–365.
  * Schöbel and Scholl, (2006) Schöbel, A. and Scholl, S. (2006).  Line planning with minimal travel time.  In 5th Workshop on Algorithmic Methods and Models for Optimization of Railways, number 06901 in Dagstuhl Seminar Proceedings.
  * Schöbel and Schwarze, (2013) Schöbel, A. and Schwarze, S. (2013).  Finding delay-resistant line concepts using a game-theoretic approach.  Netnomics, 14(3):95–117.
  * Shimamoto et al., (2012) Shimamoto, H., Schmöcker, J.-D., and Kurauchi, F. (2012).  Optimisation of a bus network configuration and frequency considering the common lines problem.  Journal of Transportation Technologies, 2(03):220.
  * Silman et al., (1974) Silman, L. A., Barzily, Z., and Passy, U. (1974).  Planning the route system for urban buses.  Computers & operations research, 1(2):201–211.
  * Spiess and Florian, (1989) Spiess, H. and Florian, M. (1989).  Optimal strategies: a new assignment model for transit networks.  Transportation research part b: methodological, 23(2):83–102.
  * Suman and Bolia, (2019) Suman, H. K. and Bolia, N. B. (2019).  Improvement in direct bus services through route planning.  Transport Policy, 81:263–274.
  * Szeto and Jiang, (2014) Szeto, W. Y. and Jiang, Y. (2014).  Transit route and frequency design: Bi-level modeling and hybrid artificial bee colony algorithm approach.  Transportation Research Part B: Methodological, 67:235–263.
  * Tian et al., (2022) Tian, S., Li, X., Liu, J., Ma, H., and Yu, H. (2022).  A short-turning strategy to alleviate bus bunching.  Journal of Ambient Intelligence and Humanized Computing, pages 1–12.
  * Tirachini et al., (2014) Tirachini, A., Hensher, D. A., and Rose, J. M. (2014).  Multimodal pricing and optimal design of urban public transport: The interplay between traffic congestion and bus crowding.  Transportation research part b: methodological, 61:33–54.
  * Torres et al., (2008) Torres, L. M., Torres, R., Borndörfer, R., and Pfetsch, M. E. (2008).  Line planning on paths and tree networks with applications to the quito trolebús system.  In Fischetti, M. and Widmayer, P., editors, ATMOS 2008 - 8th Workshop on Algorithmic Approaches for Transportation Modeling, Optimization, and Systems, Dagstuhl, Germany.  http://drops.dagstuhl.de/opus/volltexte/2008/1583.
  * Van der Hurk et al., (2016) Van der Hurk, E., Koutsopoulos, H. N., Wilson, N., Kroon, L. G., and Maróti, G. (2016).  Shuttle planning for link closures in urban public transport networks.  Transportation Science, 50(3):947–965.
  * Vansteenwegen et al., (2022) Vansteenwegen, P., Melis, L., Aktaş, D., Montenegro, B. D. G., Vieira, F. S., and Sörensen, K. (2022).  A survey on demand-responsive public bus systems.  Transportation Research Part C: Emerging Technologies, 137:103573.
  * Vermeir et al., (2021) Vermeir, E., Engelen, W., Philips, J., and Vansteenwegen, P. (2021).  An exact approach for the bus line planning problem with integrated passenger routing.  Journal of Advaced Transportation, pages 1–18.
  * Vuchic, (1976) Vuchic, V. (1976).  Skip-stop operation: high speed with good area coverage.  UITO Revue, 2:114–120.
  * Wang et al., (2024) Wang, L., Jin, J. G., Sun, L., and Lee, D.-H. (2024).  Urban rail transit disruption management: Research progress and future directions.  Frontiers of Engineering Management, pages 1–13.
  * Wardman, (2004) Wardman, M. (2004).  Public transport values of time.  Transport policy, 11(4):363–377.
  * Wolff, (2007) Wolff, A. (2007).  Drawing subway maps: A survey.  Informatik – Forschung & Entwicklung, 22(1):23–44.
  * Wu et al., (2020) Wu, H., Niedermann, B., Takahashi, S., Roberts, M. J., and Nöllenburg, M. (2020).  A survey on transit map layout – from design, machine, and human perspectives.  Comput. Graph. Forum, 39(3):619–646.
  * Xu et al., (2021) Xu, J., Liang, Q., Huang, X., and Wang, L. (2021).  Optimization of stop plan for skip-stop operation on suburban railway line.  Applied Science, 11:9519.
  * Yan and Goverde, (2019) Yan, F. and Goverde, R. M. (2019).  Combined line planning and train timetabling for strongly heterogeneous railway lines with direct connections.  Transportation Research Part B: Methodological, 127:20–46.
  * Yan et al., (2013) Yan, Y., Liu, Z., Meng, Q., and Jiang, Y. (2013).  Robust optimization model of bus transit network design with stochastic travel time.  Journal of Transportation Engineering, 139(6):625–634.
  * Yao et al., (2014) Yao, B., Hu, P., Lu, X., Gao, J., and Zhang, M. (2014).  Transit network design based on travel time reliability.  Transportation Research C, 43:233–248.
  * Yin et al., (2016) Yin, J., Tang, T., Yang, L., Gao, Z., and Ran, B. (2016).  Energy-efficient metro train rescheduling with uncertain time-variant passenger demands: An approximate dynamic programming approach.  Transportation Research B, 91:178–210.
  * Yue et al., (2016) Yue, Y., Wang, S., Thou, L., Tong, L., and Saat, M. (2016).  Optimizing train stopping patterns and schedules for high-speed passenger raill corridors.  Transportation Research c, 63:126–146.
  * Zhang et al., (2022) Zhang, Y., Peng, Q., Lu, G., Zhong, Q., Yan, X., and Zhou, X. (2022).  Integrated line planning and train timetabling through price-based cross-resolution feedback mechanism.  Transportation Research Part B: Methodological, 155:240–277.
  * Zhou et al., (2023) Zhou, W., Li, X., and Shi, X. (2023).  Joint optimization of time-dependent line planning and differential pricing with passenger train choice in high-speed railway networks.  Mathematics, 11(6):1290.
  * Zhu et al., (2022) Zhu, Z., Xu, R., Wang, F., and Vansteenwegen, P. (2022).  Scheduling method for pairing night-shift and morning-shift duties on metri lines with complex structure.  Transportmetrica A: Transport Science, pages 1–25.