# SHGNN: Structure-Aware Heterogeneous Graph Neural Network

Wentao Xu1*, Yingce Xia2, Weiqing Liu2, Jiang Bian2, Jian Yin1, Tie-Yan Liu2
1Sun Yat-sen University
2Microsoft Research Asia
xuwt6<EMAIL_ADDRESS>yingce.xia, weiqing.liu, jiang.bian,
<EMAIL_ADDRESS>
###### Abstract.

Many real-world graphs (networks) are heterogeneous with different types of
nodes and edges. Heterogeneous graph embedding, aiming at learning the low-
dimensional node representations of a heterogeneous graph, is vital for
various downstream applications. Many meta-path based embedding methods have
been proposed to learn the semantic information of heterogeneous graphs in
recent years. However, most of the existing techniques overlook the graph
structure information when learning the heterogeneous graph embeddings. This
paper proposes a novel Structure-Aware Heterogeneous Graph Neural Network
(SHGNN) to address the above limitations. In detail, we first utilize a
feature propagation module to capture the local structure information of
intermediate nodes in the meta-path. Next, we use a tree-attention aggregator
to incorporate the graph structure information into the aggregation module on
the meta-path. Finally, we leverage a meta-path aggregator to fuse the
information aggregated from different meta-paths. We conducted experiments on
node classification and clustering tasks and achieved state-of-the-art results
on the benchmark datasets, which shows the effectiveness of our proposed
method.

Heterogeneous graph; Graph neural network; Graph embedding

*Work done while Wentao Xu was an intern at Microsoft Research

††copyright: none††ccs: Mathematics of computing Graph algorithms††ccs:
Computing methodologies Neural networks; Learning latent representations††ccs:
Information systems Social networks

##  1\. Introduction

Lots of real-world data naturally exists in the graph structure, such as the
social networks (Wang et al., 2016a; Hamilton et al., 2017), the traffic
networks (Li et al., 2018; Zhang et al., 2018), the citation networks (Atwood
and Towsley, 2016; Kipf and Welling, 2017; Hamilton et al., 2017), the
physical systems (Battaglia et al., 2016; Fout et al., 2017), and the
knowledge graphs (Bordes et al., 2013; Xu et al., 2020). The underlying data
structure of graph data is non-Euclidean. We cannot directly use the
algorithms designed for data in the Euclidean space. Learning good
embeddings/representations of graph structure data benefits many downstream
applications, such as the recommendation systems (Fan et al., 2019; Zhao et
al., 2019; Liu et al., 2020; Jin et al., 2020) and the question answering
(Teney et al., 2017; Huang et al., 2019; Shangwen Lv and Hu, 2020; Zhao et
al., 2020). Researchers had conducted extensive study on graph embeddings,
ranging from classic machine learning methods (e.g., LINE (Tang et al., 2015),
DeepWalk (Perozzi et al., 2014), node2vec (Grover and Leskovec, 2016), TADW
(Yang et al., 2015), and Struc2vec (Ribeiro et al., 2017)) to deep learning
based methods, i.e., graph neural networks (briefly, GNN). Typical GNN
networks include ChebNet (Defferrard et al., 2016), GCN (Kipf and Welling,
2017), GraphSAGE (Hamilton et al., 2017), GAT (Velickovic et al., 2018), GGNN
(Li et al., 2016) and GaAN (Zhang et al., 2018).

(a) The movie nodes with different node centralities.

(b) Information aggregation on different graph structures.

Figure 1. The heterogeneous graph in different structures.

Most graph embedding methods assume that the input is a homogeneous graph,
where all nodes are of the same type, and so are edges. For example, in drug-
drug interaction prediction, all nodes are drugs, and all edges are binary
labels $0/1$, indicating whether taking two drugs together can lead to a
significant difference. However, many real-world graphs are heterogeneous,
where both nodes and edges have different types. As shown in Figure 1, if
there is a network about actors and movies, there are two different types of
nodes (i.e., actors, movies). A heterogeneous graph usually contains richer
information than the corresponding homogeneous graph, making it harder to
encode. Previously, there are some unsupervised methods (Shang et al., 2016;
Dong et al., 2017; Fu et al., 2017; Shi et al., 2018) and semi-supervised
methods (Wang et al., 2019; Yun et al., 2019; Hu et al., 2020; Fu et al.,
2020) to learning the embeddings of heterogeneous graphs. A common part of the
above work is that they are built upon meta-paths. A meta-path is an ordered
sequence of node types and edge types defined on the network schema, which
describes a composite relation between the nodes types involved. For example,
in the IMDB dataset (Fu et al., 2020), which consists of a heterogeneous graph
with three types of nodes: movies, actors, and directors, Movie-Actor-Movie
(M-A-M) and Movie-Director-Movie (M-D-M) are two different meta-paths. The
M-A-M meta-path describes the co-actor relation between two movies, and the
M-D-M meta-path associates two movies directed by the same director. Depending
on the meta-paths, the relation between nodes in the heterogeneous graph can
have different semantics. Most of the existing heterogeneous graph neural
networks utilize the meta-paths to sample meta-path based neighbor nodes and
aggregate the node information to learn the heterogeneous graph’s
representations.

Motivation. However, most previous heterogeneous graph embedding methods
overlook the graph structure information when learning the embeddings.
Firstly, previous methods overlook the node centrality information determined
by the graph structure, which measures how important a node is in the graph.
For instance, in Figure 1(a), both of the Movie nodes $m_{1}$ and $m_{3}$
connect with one Actor node only, which are $a_{1}$ and $a_{2}$, respectively.
The actor $a_{1}$ is more famous since he performs many other movies, so the
movie $m_{1}$ performed by $a_{1}$ should be more popular than the movie
$m_{2}$ performed by actor $a_{2}$. However, existing heterogeneous graph
embedding methods cannot model this type of structural information that
measures a node’s importance. Secondly, most of the existing meta-path based
methods assume that each meta-path instance is isolate. They do not consider
the graph structure when they aggregate information on the meta-path. For
instance, there are three meta-path instances $m_{1}-a_{1}-m_{i}$,
$m_{2}-a_{1}-m_{i}$ and $m_{3}-a_{2}-m_{i}$ in Figure 1(b). Most of the
previous methods (Wang et al., 2019; Fu et al., 2020) assume these three
instances are isolate, and combine them via some techniques like attention
mechanism to aggregate the information of $m_{1}$, $m_{2}$ and $m_{3}$ to node
$m_{i}$. However, the meta-path instances $m_{1}-a_{1}-m_{i}$ and
$m_{2}-a_{1}-m_{i}$ share the intermediate node $a_{1}$, and the nodes $m_{1}$
and $m_{2}$ should have a closer connection than the node $m_{3}$. Therefore,
we should consider these differences caused by the graph structure when
aggregating information to node $m_{i}$.

Our method. To incorporate the structure information into the graph
embeddings, we propose a new model, Structure-Aware Heterogeneous Graph Neural
Network (SHGNN), for heterogeneous graph embedding. SHGNN first uses an
aggregation module to provide initial embedding of all nodes to aggregate the
features (Section 4.1). To encode the node centrality, we utilize the coverage
times of meta-path instances for representing the node centrality (Section
4.2). To incorporate the graph structure information into the information
aggregation on meta-path, SHGNN applies a tree-attention aggregator to
aggregate information for every meta-path (Section 4.3). In this way, SHGNN
can capture the overall graph structure information instead of processing each
meta-path instance individually. Finally, we utilize a meta-paths aggregator
to fuse the information aggregated from different meta-paths. Thus we can
learn the comprehensive semantics ingrained in the heterogeneous graph. We
conduct downstream node classification and node clustering tasks on several
benchmark datasets to evaluate our proposed structure-aware heterogeneous
graph neural network. The experimental results show that our SHGNN outperforms
existing heterogeneous graph embedding methods.

Contributions. Our contributions are summarized as follows:

  1. (1)

We explicitly model the importance of nodes using node centrality. Compared
with previous work on homogeneous graphs, which simply uses the degrees of a
node, we leverage the coverage times of meta-path instances to differentiate
the importance of nodes in different graph structures.

  2. (2)

We use a tree-based attention module to aggregate information on a specific
meta-path and meanwhile consider the graph structure in multiple meta-path
instances. Next, we use another attention module to aggregate the information
across different types of meta-paths. In this way, the structural information
in the meta-paths is fully leveraged.

  3. (3)

We conduct experiments on three IMDB, DBLP and ACM datasets. Our method
achieved state-of-the-art results on node classification and clustering tasks
on all above datasets. We also provide extensive studies to show how our
method works.

##  2\. Related Work

(a) Heterogeneous Graph

(b) Meta-paths

(c) Meta-path Instances

(d) Meta-path based Neighbors

Figure 2. The illustrations of some examples in Section 3.

###  2.1. Graph Embedding

Graph embedding aims to learn the low dimensional representations of graph
nodes while preserving the graph structure and property so that the graph
embeddings can apply to the downstream tasks. We can further divide graph
embedding into two types: homogeneous and heterogeneous graph embedding.

Among the homogeneous graph embedding methods, the LINE (Tang et al., 2015)
exploits the first-order and second-order proximity between nodes to learn
node embeddings. The DeepWalk (Perozzi et al., 2014), node2vec (Grover and
Leskovec, 2016), TADW (Yang et al., 2015), and Struc2vec (Ribeiro et al.,
2017) are random walk based methods, which utilize random walk to generate
node sequence and feed the sequence to a skip-gram (Mikolov et al., 2013a)
model learn node embeddings. SDNE (Wang et al., 2016b) implies multiple layers
of non-linear functions to capture highly non-linear network structure.

Heterogeneous graph embedding needs to further operate on nodes and edges that
have different types. Metapath2vec (Dong et al., 2017) generates random walks
guided by a single meta-path, and feeds it to a skip-gram model (Mikolov et
al., 2013b) to generate node embeddings. ESim (Shang et al., 2016) uses the
user-defined meta-paths to learn the node embeddings from sampled positive and
negative meta-path instances. HIN2vec (Fu et al., 2017) leverages multiple
prediction training tasks to learn representations of nodes and meta-paths of
a heterogeneous graph. HERec (Shi et al., 2018) translates a heterogeneous
graph into a homogeneous graph based on meta-path based neighbors and uses the
DeepWalk model to learn the node embeddings of the specific type.

###  2.2. Graph Neural Network

The graph neural network (GNN) utilizes the deep neural network to learn the
node embeddings via message passing. Based on graph signal processing,
spectral-based GNNs were first proposed to perform graph convolution in the
Fourier domain of a graph. ChebNet (Defferrard et al., 2016) uses Chebyshev
polynomials to filter graph signals (node features) in the graph Fourier
domain. The graph convolutional network (GCN) (Kipf and Welling, 2017)
constrains and simplifies the parameters of ChebNet to address the overfitting
problem and improve the performance. On the other hand, spatial-based GNNs
utilize the aggregator to aggregate information from neighbors for each node,
imitating the convolution operations of convolutional neural networks for
image data. GraphSAGE (Hamilton et al., 2017) proposes the general format of
aggregator functions for efficient learning of node embeddings. Motivated by
the Transformer (Vaswani et al., 2017), GAT (Velickovic et al., 2018) leverage
the self-attention mechanism to calculate the relative importance of each
neighbor’s information in the aggregator. GGNN (Li et al., 2016) adds a gated
recurrent unit (GRU) (Cho et al., 2014) to the aggregator by treating the
aggregated neighborhood information as the input to the GRU of the current
time step. GaAN (Zhang et al., 2018) combines GRU with the gated multi-head
attention mechanism for spatiotemporal graphs.

The GNNs mentioned above operate on node features in the same embedding space,
designing for homogeneous graphs. Therefore, they can not naturally apply to
heterogeneous graphs with different nodes and edges. To process the
heterogeneous graph, some heterogeneous GNNs were proposed. HAN (Wang et al.,
2019) converts a heterogeneous graph into multiple meta-path connected
homogeneous graphs and uses a graph attention network architecture to
aggregate information from the meta-path based neighbors and leverages the
attention mechanism to combine various meta-paths. MAGNN (Fu et al., 2020)
utilizes the intra-meta-path aggregation to incorporate the word embedding of
intermediate nodes and the inter-meta-path aggregation to combine messages
from multiple meta-paths. MAGNN-AC-(Jin et al., 2021) use topological
relationship between nodes as guidance to complete attributes for no-attribute
nodes.

However, the heterogeneous GNNs introduced above have the limitations of
ignoring the node centrality information determined by the graph structure, or
overlooking the graph structure when aggregating information on the meta-path.
Although they might have improved the performance of homogeneous graph
embedding methods for some heterogeneous graph datasets, there is still room
for improvement by incorporating the graph structure information into the
heterogeneous graph embeddings.

##  3\. PRELIMINARY

In this section, we will introduce some definitions and the corresponding
examples in the heterogeneous graph.

###### Definition 1.

Heterogeneous Graph (Sun and Han, 2013). A heterogeneous graph is a graph
$\mathcal{G}=\left(\mathcal{V},\mathcal{E}\right)$, where $\mathcal{V}$ is the
set of nodes and $\mathcal{E}$ is the set of edges. It associates with a node
type mapping function $\phi:\mathcal{V}\rightarrow\mathcal{A}$, and an edge
type mapping function $\psi:\mathcal{E}\rightarrow\mathcal{R}$, where
$|\mathcal{A}|+|\mathcal{R}|>2$. The $\mathcal{A}$ and $\mathcal{R}$ are the
predefined sets of node types and edge types in the heterogeneous graph,
respectively.

###### Example 1.

Figure 2(a) is a example of heterogeneous graph with three node types (Actors,
Movies, and Directors) and two edge types (Actor-Movie edge and Movie-Director
edge).

###### Definition 2.

Meta-path (Sun et al., 2011). A meta-path $P$ is a path in the form of
$A_{1}\stackrel{{\scriptstyle
R_{1}}}{{\longrightarrow}}A_{2}\stackrel{{\scriptstyle
R_{2}}}{{\longrightarrow}}\cdots\stackrel{{\scriptstyle
R_{l}}}{{\longrightarrow}}A_{l+1}$ (abbreviated as $A_{1}-A_{2}-\cdots
A_{l+1}$), representing a composite relation $R=R_{1}\circ
R_{2}\circ\cdots\circ R_{l}$ between the node types $A_{1}$ and $A_{l+1}$. The
symbol $\circ$ is the composition operator on relations.

###### Example 2.

Figure 2(b) shows the Movie-Actor-Movie (M-A-M) and Movie-Director-Movie
(M-D-M) meta-paths in the heterogeneous graph given in Figure 2(a). Different
meta-paths represent different semantics. For instance, meta-path M-A-M
represents the co-actor relation, while M-D-M means they are directed by the
same director.

###### Definition 3.

Meta-path Instance (Fu et al., 2020). A meta-path instance $p$ of a
corresponding meta-path $P$ is a node sequence in the graph that follows the
schema defined by $P$.

###### Example 3.

Figure 2(c) shows examples of some meta-path instances in Figure 2(a). The
meta-path instances $m_{1}-a_{1}-m_{2}$ and $m_{2}-d_{3}-m_{3}$ are
corresponding to the meta-paths Movie-Actor-Movie and Movie-Director-Movie,
respectively.

###### Definition 4.

Meta-path based Neighbor (Wang et al., 2019). The meta-path based neighbors
$\mathcal{N}^{P}_{v}$ of a node $v$ under the meta-path $P$ are the set of
nodes that connect with node $v$ via the meta-path instances of $P$. A
neighbor connected by two different meta-path instances is regarded as two
different nodes in $\mathcal{N}^{P}_{v}$. When the meta-path $P$ is symmetric,
the set $\mathcal{N}^{P}_{v}$ also includes node $v$.

###### Example 4.

Figure 2(d) shows some meta-path based neighbors of Movie $m_{2}$. Given the
meta-path Movie-Actor-Movie, the meta-path based neighbors of $m_{2}$ are the
$m_{1}$ and $m_{3}$. Similarly, the neighbor of $m_{2}$ based on meta-path
Movie-Director-Movie is $m_{3}$. We can get meta-path based neighbors by the
multiplication of a sequences of adjacency matrices.

###### Definition 5.

Heterogeneous Graph Embedding (Fu et al., 2020). Given a heterogeneous graph
$\mathcal{G}=\left(\mathcal{V},\mathcal{E}\right)$, the heterogeneous graph
embedding task is to learn the $d$-dimensional node representations
$\mathbf{h}_{v}\in\mathbb{R}^{d}$ for all $v\in\mathcal{V}$ with
$d\ll|\mathcal{V}|$ that are able to capture rich structural and semantic
information involved in $\mathcal{G}$.

##  4\. Approach

In this section, we elaborate on our structure-aware heterogeneous graph
neural network (SHGNN). Figure 3 is the illustration of our SHGNN on a
heterogeneous graph example of IMDB. The key insights of our SHGNN are to
consider the graph structural information when learning the embeddings,
including the node structural features in Section 4.1, the node centrality
encoding in Section 4.2 and the meta-path structural information encoder in
Section 4.3.

Figure 3. The illustration of our structure-aware graph neural network on the
heterogeneous graph of IMDB.

###  4.1. Node Structural Features

Feature Propagation. As pointed out in (Jin et al., 2021), many types of nodes
in heterogeneous graphs are not adequately modeled, where their features are
the corresponding word embeddings of the descriptions (i.e., whom the actor
is, a brief description of a director, etc.). Such features cannot correctly
model the graph due to the lack of structural information. To represent the
graph semantic information in the node features, we use a feature propagation
module to learn the semantic information of nodes. We first choose a type of
nodes as the basic type (e.g., the Movie nodes in IMDB, the Author nodes in
DBLP, and the Paper nodes in ACM), and initialize the nodes in the basic type
with bag-of-words embeddings. The embeddings of the basic type’s nodes are
also the nodes’ features. Then as shown in Figure 4, we propagate the features
from the basic type’s nodes to the nodes in other types on the network schema
of a heterogeneous graph. For a node $v\in\mathcal{V}_{A_{i}}$ of type
$A_{i}\in\mathcal{A}$, and the nodes in type $A_{i}$ do not have features, we
obtain the node $v$’s features by aggregating the features from the nodes in
type $A_{j}$:

(1)
$\mathbf{x}_{v}^{A_{i}}=\frac{1}{|\mathcal{N}_{v}^{ij}|}\sum_{k\in\mathcal{N}_{v}^{ij}}\mathbf{x}_{k}^{A_{j}},$

where the nodes in the type $A_{j}$ has features and the type $A_{j}$ is the
nearest type to type $A_{i}$ on the network schema. $\mathcal{N}_{v}^{ij}$ are
the neighbors of node $v$ that in type $A_{j}$, and $\mathbf{x}_{k}^{A_{j}}$
is the feature of the node $k$, which is one of the node $v$’s neighbors in
type $A_{j}$. Taking the node $a_{1}$ (type Actor) in Figure 2(a) as an
example, the type Movie is the basic type, and the node $a_{1}$ would
aggregate its feature from the node $m_{1}$ and $m_{2}$. After the feature
propagation, each node in the heterogeneous graph has the local structural
embedding, which involves semantic information. The embeddings of an Actor
node can reflect all the participated movies. Therefore, the feature
propagation module can incorporate a node’s graph structural information into
the node feature.

(a) IMDB

(b) DBLP

Figure 4. Feature propagation on the network schema.

Type-specific Transformation. Due to the heterogeneity of nodes, different
types of nodes have different feature spaces. Hence, we apply a type-specific
linear transformation for each type of nodes by projecting their features into
the same latent factor space. For a node $v\in\mathcal{V}_{A_{i}}$ of type
$A_{i}\in\mathcal{A}$, we have:

(2) $\mathbf{\hat{x}}_{v}^{A_{i}}=W^{x}_{A_{i}}\mathbf{x}_{v}^{A_{i}},$

where $\mathbf{\hat{x}}_{v}^{A_{i}}\in\mathbb{R}^{d_{1}}$ is the projected
latent vector of node $v$. $W_{A_{i}}\in\mathbb{R}^{d_{1}\times d_{0}}$ is the
parametric weight matrix for type $A_{i}$’s nodes. The type-specific
transformation addresses the heterogeneity of a graph.

###  4.2. Node Centrality Encoding

The node centrality, measuring how important a node is in the graph, is
usually significant structural information for graph understanding. For
instance, the celebrities who have massive followers are vital factors in
predicting the social network trend (Marshall, 2010). In the homogeneous
graph, some existing methods (Ying et al., 2021) would utilize the degree
centrality, including the indegree and outdegree of a node to measure a node’s
centrality. However, we found that the degree centrality is not
distinguishable for the heterogeneous graph’s nodes since many nodes have the
same degree. For example, in the heterogeneous graph of IMDB dataset (Fu et
al., 2020), 99.91% of the Movie nodes have the same degree 4.

To properly model the node centrality in the heterogeneous graph, we propose
utilizing the coverage times of meta-path instances to represent the node
centrality. Specifically, we count the number of times $c(v)$ that a node $v$
covered by all the meta-path instances in the heterogeneous graph. When more
meta-path instances cover a node, we assume that this node is more important
in the graph and has a higher $c(v)$ value. For example, in the heterogeneous
graph of IMDB, even two Movie nodes have the same number of degrees (actors),
the Movie node covered by more meta-path instances (Movie-Actor-Movie)
indicates that this movie is performed by the actors performing more other
movies, so this movie is more important than the other movie covered by fewer
meta-path instances. After that, we assign each node a one hot embedding
vector $\mathbf{z}_{c(v)}$ according to the ${\tiny}c(v)$ value.

Moreover, to capture more node centrality information, we further count the
number of times $c^{+}(v)$ that a node $v$ as the direct neighbor of any one
of the nodes in a meta-path instance. We can simply calculate the $c^{+}(v)$
with the following equation:

(3) $c^{+}(v)=\sum_{k\in\mathcal{N}_{v}}c(k),$

where $\mathcal{N}_{v}$ is the set of all direct neighbors of node $v$, and
$c(k)$ is the number of times that meta-path instances cover node $k$. We
assign another one hot embedding vector $\mathbf{z}^{\prime}_{c^{+}(v)}$
according to the $c^{+}(v)$ value. As the centrality encodings
$\mathbf{z}_{c(v)}$ and $\mathbf{z}^{\prime}_{c^{+}(v)}$ are applied to each
node, we concatenate them and apply another type-specific transformation on
them:

(4)
$\mathbf{\hat{z}}_{v}^{A_{i}}=W^{z}_{A_{i}}\left(\mathbf{z}_{c(v)}||\mathbf{z}^{\prime}_{c^{+}(v)}\right),$

where $\mathbf{z}_{c(v)}\in\mathbb{R}^{d_{1}}$ and
$\mathbf{z}^{\prime}_{c^{+}(v)}\in\mathbb{R}^{d_{1}}$ are one hot embedding
vectors specific by the counts $c(v)$ and $c^{+}(v)$, respectively. The
$W^{z}_{A_{i}}\in\mathbb{R}^{d_{1}\times 2d_{1}}$ is the weighted transform
matrix for type $A_{i}$, and $\mathbf{\hat{z}}_{v}^{A_{i}}$ is the transformed
centrality embedding of node $v$.

We concatenate the node structural features $\mathbf{\hat{x}}_{v}^{A_{i}}$ and
node centrality embedding $\mathbf{\hat{z}}_{v}^{A_{i}}$ as the node $v$’s
latent vector $\mathbf{\hat{h}}_{v}$
($\mathbf{\hat{h}}_{v}\in\mathbb{R}^{2d_{1}}$).

(5)
$\mathbf{\hat{h}}_{v}=\left(\mathbf{\hat{x}}_{v}^{A_{i}}||\mathbf{\hat{z}}_{v}^{A_{i}}\right).$

###  4.3. Meta-path Structural Information Encoder

After obtaining the latent vector of each node, we utilize a meta-path
structural information encoder to aggregate the latent vectors of meta-path
based neighbors. The proposed encoder includes a tree-attention aggregator for
incorporating the graph structure into the information aggregation on the
meta-path, and a meta-path aggregator to fuse the information aggregated by
various meta-paths in the heterogeneous graph.

Tree-attention Aggregator. To incorporate the graph structure into the
information aggregation on the meta-path, we propose a tree-attention
aggregator to aggregate the information for a meta-path. Given a meta-path
$P_{j}$ and a target node $t$ whose type is the end of $P_{j}$, we expand all
the $P_{j}$’s meta-path instances that end with the node $t$ as a rooted tree.
This tree’s root node is the target node $t$, and its leaf nodes are the meta-
path based neighbors of the target node $t$. Then we perform a tree-attention
to aggregate the information from the tree’s leaf nodes to the root node. For
a parent node $v$, since different child nodes would differently contribute to
the parent node, we compute the cosine similarity between the node $v$’s
latent vector $\mathbf{\hat{h}}_{v}$ and its child nodes’ latent vectors as
the attention values from the child nodes to the parent node:

(6)
$e^{P_{j}}_{vu}=\mathrm{Cosine}(\mathbf{\hat{h}}_{v},\mathbf{\hat{h}}_{k})=\dfrac{\mathbf{\hat{h}}_{v}\cdot\mathbf{\hat{h}}_{k}}{||\mathbf{\hat{h}}_{v}||\cdot||\mathbf{\hat{h}}_{k}||},k\in\mathcal{C}_{v}^{P_{j}},$

where $\mathcal{C}_{v}^{P_{j}}$ is the child node set of the node $v$ on the
tree generated by meta-path the $P_{j}$. Hence, when the child node’s latent
vector $\mathbf{\hat{h}}_{k}$ has a higher similarity with parent node’s
vector $\mathbf{\hat{h}}_{v}$, the child node $k$ would have a higher
attention weight to the parent node $v$. We normalize the attention value
$e^{P_{j}}_{vu}$ across all child nodes of node $v$ using the softmax
function, then we can obtain the aggregated weight $\alpha_{vk}^{P_{j}}$ from
the child node $k$ to the parent node $v$. We use the aggregated weights to
aggregate the child nodes’ information $\mathbf{h}_{k}^{P_{j}}$ to the parent
node:

(7) $\displaystyle\alpha_{vk}^{P_{j}}$
$\displaystyle=\frac{\exp\left(e_{vk}^{P_{j}}\right)}{\sum_{s\in\mathcal{C}_{v}^{P_{j}}}\exp\left(e_{vs}^{P_{j}}\right)},$
$\displaystyle\mathbf{h}_{v}^{P_{j}}$
$\displaystyle=\sum_{k\in\mathcal{C}_{v}^{P_{j}}}\alpha_{vk}^{P_{j}}\cdot\mathbf{h}_{k}^{P_{j}}.$

Note that each parent node $v$ aggregate the information
$\mathbf{h}_{v}^{P_{j}}$ from its child nodes, and if node $k$ is a leaf node
in the tree, then $\mathbf{h}_{k}^{P_{j}}$ is equal to the node $k$’s latent
vector $\mathbf{\hat{h}}_{k}$. As shown in Figure 3c, from leaf nodes to root
node, the tree-attention aggregator step-by-step aggregates each parent node’s
information from its child nodes. Since the different child nodes and
different graph structures would result in different aggregated results on the
parent node, we can incorporate the graph structure into the information
aggregation on the meta-path. Moreover, we do not introduce other parameters
in the tree-attention aggregator, so we can model the effect of different
graph structures without increasing parameters.

Meta-paths Aggregator. After aggregating the node information on each meta-
path’s corresponding tree, we combine the aggregated information of each meta-
path using an meta-paths aggregator. For a node type $A_{i}$, we have
$|\mathcal{V}_{A_{i}}|$ sets of latent vectors:
$\left\\{\mathbf{h}_{v}^{P_{1}},\mathbf{h}_{v}^{P_{2}},\ldots,\mathbf{h}_{v}^{P_{M}}\right\\}$
for $v\in\mathcal{V}_{A_{i}}$, where $M$ is the number of meta-paths for type
$A_{i}$. Since different meta-paths are not equally important in a
heterogeneous graph, we leverage the attention mechanism to assign different
weights to different meta-paths. We firstly summarize each meta-path
$P_{j}\in\mathcal{P}_{A_{i}}$ by averaging the transformed meta-path specific
node vectors for all nodes $v\in\mathcal{V}_{A_{i}}$:

(8)
$\mathbf{s}_{P_{j}}=\frac{1}{|\mathcal{V}_{A_{i}}|}\sum_{v\in\mathcal{V}_{A_{i}}}\tanh\left(W^{h}_{A_{i}}\mathbf{h}_{v}^{P_{j}}+\mathbf{b}_{A_{i}}\right),$

where $W^{h}_{A_{i}}\in\mathbb{R}^{d_{1}\times 2d_{1}}$ and
$\mathbf{b}_{A_{i}}\in\mathbb{R}^{d_{1}}$ are learnable parameters. Then we
use the attention mechanism to fuse the meta-path specific vectors of node $v$
as follows:

(9) $\displaystyle u_{P_{j}}$
$\displaystyle=\mathbf{q}_{A_{i}}^{\intercal}\mathbf{s}_{P_{j}},$
$\displaystyle\beta_{P_{j}}$
$\displaystyle=\frac{\exp\left(u_{P_{j}}\right)}{\sum_{P_{k}\in\mathcal{P}_{A_{i}}}\exp\left(u_{P_{k}}\right)},$
$\displaystyle\mathbf{h}_{v}^{\mathcal{P}_{A_{i}}}$
$\displaystyle=\sum_{P_{j}\in\mathcal{P}_{A_{i}}}\beta_{P_{j}}\cdot\mathbf{h}_{v}^{P_{j}},$

where $\mathbf{q}_{A_{i}}\in\mathbb{R}^{d_{1}}$ is the parameterized vector
for node type $A_{i}$. $\beta_{P_{j}}$ is the importance weight of meta-path
$P_{j}$ to type $A_{i}$’s nodes, and $\mathbf{h}_{v}^{\mathcal{P}_{A_{i}}}$ is
aggregated information of all meta-paths on node $v$.

Input: The heterogeneous graph
$\mathcal{G}=\left(\mathcal{V},\mathcal{E}\right)$, node types set
$\mathcal{A}$, meta-paths set $\mathcal{P}$, features of basic type’s node,
and the number of layers $L$.

Output: The node embeddings $\left\\{\mathbf{h}_{v},\forall
v\in\mathcal{V}\right\\}$.

1

2for  _node type $A_{i}\in\mathcal{A}$_ do

3 for  _node $v\in\mathcal{V}_{A_{i}}$_ do

4 Feature Propagation
$\mathbf{x}_{v}^{A_{i}}\leftarrow\frac{1}{|\mathcal{N}_{v}^{ij}|}\sum_{k\in\mathcal{N}_{v}^{ij}}\mathbf{x}_{k}^{A_{j}}$;

5 Type-specific transformation $\mathbf{\hat{x}}_{v}\leftarrow
W^{x}_{A_{i}}\cdot\mathbf{x}_{v}^{A_{i}}$;

6 Count the times $c(v)$ and $c^{+}(v)$ for node $v$;

7 Node centrality encoding: $\mathbf{\hat{z}}_{v}\leftarrow
W^{z}_{A_{i}}(\mathbf{z}_{c(v)}||\mathbf{z}^{\prime}_{c^{+}(v)})$;

8 Obtain the node latent vector:
$\mathbf{\hat{h}}_{v}\leftarrow\left(\mathbf{\hat{x}}_{v}||\mathbf{\hat{z}}_{v}\right)$;

9

10 end for

11

12 end for

13for  _$l=1\ldots L$_ do

14 for  _node type $A_{i}\in\mathcal{A}$_ do

15 for  _meta-path $P_{j}\in\mathcal{P}_{A_{i}}$_ do

16 for  _$v\in\mathcal{V}_{A_{i}}$_ do

17 Generate a rooted tree according to the target node $v$ and $P_{j}$’s meta-
path instances;

18 Perform tree-attention
$\left[\mathbf{h}_{v}^{P_{j}}\right]^{l}\leftarrow\sigma\left(\sum_{k\in\mathcal{C}_{v}^{P_{j}}}\alpha_{vk}^{P_{j}}\cdot\left[\mathbf{h}_{k}^{P_{j}}\right]^{l}\right)$;

19

20 end for

21

22 end for

23 Calculate the weight $\beta_{P}$ for meta-path
$P_{j}\in\mathcal{P}_{A_{i}}$;

24 Fuse the embeddings from different meta-paths
$\left[\mathbf{h}_{v}^{\mathcal{P}_{A_{i}}}\right]^{l}\leftarrow\sum_{P_{j}\in\mathcal{P}_{A_{i}}}\beta_{P_{j}}\cdot\left[\mathbf{h}_{v}^{P_{j}}\right]^{l},\forall
v\in\mathcal{V}_{A_{i}}$;

25

26 end for

27 Layer output projection:
$\mathbf{h}_{v}^{l}=\sigma\left(W_{o}^{l}\cdot\left[\mathbf{h}_{v}^{\mathcal{P}_{A_{i}}}\right]^{l}\right),\forall
v\in\mathcal{V}_{A_{i}}$;

28

29 end for

Algorithm 1 The overall training process of our SHGNN.

###  4.4. Optimization Objective

Finally, we use a linear layer with a ELU activation function to project the
node embeddings $\mathbf{h}_{v}^{\mathcal{P}_{A_{i}}}$ to the desired output
vector:

(10)
$\mathbf{h}_{v}=\mathrm{ELU}\left(W_{o}\mathbf{h}_{v}^{\mathcal{P}_{A_{i}}}\right),$

where $W_{o}\in\mathbb{R}^{d\times 2d_{1}}$ is a weight matrix, and
$\mathbf{h}_{v}$ is the structure-aware node representation in heterogeneous
graph. The projection is task-specific, which can be a linear classifier for
the node classification task or be a projection to the node similarity
measuring space for the node clustering task.

We use the stochastic gradient descent algorithm (SGD) to train our framework
and leverage the Adam (Kingma and Ba, 2014) for tuning the learning rate. We
optimize our SHGNN by minimizing the cross entropy loss function
$\mathcal{L}$:

(11)
$\mathcal{L}=-\sum_{v\in\mathcal{V}_{L}}\sum_{c=1}^{C}\mathbf{y}_{v}[c]\cdot\log\mathbf{h}_{v}[c],$

where $\mathcal{V}_{L}$ is the set of nodes with labels, $C$ is the number of
classes, $\mathbf{y}_{v}$ is the one-hot label vector of node $v$, and
$\mathbf{h}_{v}$ is the predicted probability vector of node $v$. The
Algorithm 1 shows the overall training process of our SHGNN.

##  5\. Experiments

In this section, we study the efficacy of the proposed structure-aware
heterogeneous graph neural network with experiments. We aim to answer the
following research questions via the experiments:

  * •

RQ1: How does SHGNN perform in node classification task?

  * •

RQ2: How does SHGNN perform in node clustering task?

  * •

RQ3: What are the effect of different components in SHGNN ?

  * •

RQ4: How do we understand the representation capability of different graph
embedding methods?

###  5.1. Datasets

We evaluate the performance of our proposed SHGNN framework on three benchmark
heterogeneous graph datasets: IMDB, DBLP, and ACM. Table 1 summarizes the
statistics of the datasets. The detailed datasets’ introductions are as
follows:

  * •

IMDB111https://www.imdb.com/. is an online database of movies and television
programs. We use a subset of IMDB released by (Fu et al., 2020), containing
4278 movies (M), 2081 directors (D), and 5257 actors (A) after extraction.
Movies have one of three classes of labels (Action, Comedy, and Drama) based
on their genre information. The feature of each movie node is its plot
keywords’ bag-of-words representation. Following (Fu et al., 2020; Jin et al.,
2021), we divide the movie nodes into training, validation, and testing sets
of 400 (9.35%), 400 (9.35%), and 3478 (81.30%) nodes, respectively.

  * •

DBLP222https://dblp.uni-trier.de/. is a computer science bibliography website.
We use a subset of DBLP that also used in (Fu et al., 2020; Jin et al., 2021),
containing 4057 authors (A), 14328 papers (P), 7723 terms (T), and 20
publication venues (V). The authors are divided into four research areas
(Database, Data Mining, Artificial Intelligence, and Information Retrieval).
The feature of each author node is the bag-of-words representation of their
paper keywords. Following (Fu et al., 2020; Jin et al., 2021), we divide the
author nodes into training, validation, and testing sets of 400 (9.86%), 400
(9.86%), and 3257 (80.28%) nodes, respectively.

  * •

ACM333http://dl.acm.org/. is a collection of all ACM publications, including
journals, conference proceedings, and books. We use a subset of ACM extracted
by (Jin et al., 2021), which comprises 4019 papers (P), 7167 authors (A), and
60 subjects (S). The papers are divided into three classes (Database, Wireless
Communication, Data Mining). In this dataset, papers’ features are the bag-of-
words representation of their keywords. Following (Jin et al., 2021), we
divide the paper nodes into training, validation, and testing sets of 400
(9.95%), 400 (9.95%), and 3219 (80.10%) nodes, respectively.

Table 1. Statistics of datasets.

Datasets | Nodes | Edges | Meta-paths
---|---|---|---
IMDB |  | # Movie (M): 4278
---
# Director (D): 2081
# Actor (A): 5257
| # M-D: 4278
---
# M-A: 12828
| M-D-M
---
M-A-M
DBLP |  | # Author (A): 4057
---
# Paper (P): 14328
# Term (T): 7723
Venue (V): 20
| # A-P: 19645
---
# P-T: 85810
# P-V: 14328
| A-P-A
---
A-P-T-P-A
A-P-V-P-A
ACM |  | # Paper (P): 4019
---
# Author (A): 7167
# Subject (S): 60
| # P-A: 13407
---
# P-S: 4019
| P-A-P
---
P-S-P

Table 2. The node classification results (%) on the IMDB, DBLP, and ACM
datasets.

Dataset | Metrics | Train % | metapath | GCN | GAT | HetGNN | HAN | MAGNN | MAGNN | SHGNN
---|---|---|---|---|---|---|---|---|---|---
2vec | -AC
IMDB | Macro-F1 | 20% | 46.42 | 44.75 | 54.81 | 48.92 | 57.61 | 58.11 | 59.67 | 60.69
40% | 47.79 | 45.26 | 55.09 | 51.61 | 57.75 | 59.39 | 60.18 | 62.36
60% | 48.25 | 46.72 | 55.71 | 53.00 | 57.66 | 59.97 | 60.60 | 63.23
80% | 48.73 | 47.13 | 55.40 | 53.24 | 57.23 | 60.02 | 60.75 | 63.67
Micro-F1 | 20% | 48.08 | 47.44 | 55.02 | 49.70 | 57.82 | 58.16 | 59.84 | 60.80
40% | 49.55 | 47.62 | 55.56 | 55.29 | 57.98 | 59.46 | 60.38 | 62.48
| 60% | 50.06 | 48.49 | 55.91 | 53.91 | 57.87 | 60.05 | 60.79 | 63.34
| 80% | 50.68 | 48.73 | 55.67 | 54.25 | 57.46 | 60.15 | 60.98 | 63.85
DBLP | Macro-F1 | 20% | 91.50 | 90.06 | 66.92 | 91.72 | 91.69 | 92.53 | 94.20 | 94.80
40% | 92.55 | 90.37 | 73.23 | 92.03 | 91.84 | 92.97 | 94.35 | 94.81
60% | 93.25 | 90.57 | 77.17 | 92.26 | 92.01 | 93.30 | 94.37 | 94.84
80% | 93.48 | 90.74 | 78.20 | 92.39 | 92.15 | 93.77 | 94.63 | 95.17
Micro-F1 | 20% | 92.14 | 90.53 | 76.98 | 92.23 | 92.30 | 93.08 | 94.59 | 95.19
40% | 93.09 | 90.83 | 79.61 | 92.55 | 92.46 | 93.50 | 94.72 | 95.20
| 60% | 93.76 | 91.01 | 81.62 | 92.79 | 92.65 | 93.83 | 94.75 | 95.23
| 80% | 93.94 | 91.15 | 82.22 | 92.92 | 92.78 | 94.27 | 94.98 | 95.52
ACM | Macro-F1 | 20% | 69.95 | 70.41 | 89.59 | 89.16 | 90.01 | 88.01 | 90.00 | 92.06
40% | 71.15 | 70.82 | 89.77 | 90.14 | 90.82 | 89.42 | 91.49 | 92.33
60% | 71.74 | 69.67 | 89.72 | 90.71 | 91.51 | 90.39 | 92.27 | 92.59
80% | 72.18 | 67.23 | 89.42 | 91.01 | 91.71 | 90.79 | 92.70 | 93.08
Micro-F1 | 20% | 72.12 | 74.02 | 89.47 | 89.12 | 89.89 | 88.08 | 90.03 | 91.94
40% | 73.17 | 74.57 | 89.65 | 90.11 | 90.73 | 89.48 | 91.57 | 92.27
| 60% | 73.65 | 74.10 | 89.60 | 90.64 | 91.37 | 90.42 | 92.32 | 92.52
| 80% | 74.14 | 72.69 | 89.29 | 90.93 | 91.56 | 90.80 | 92.73 | 92.97

###  5.2. Experimental Settings

####  5.2.1. Compared Methods

We compare our SHGNN with following homogeneous and heterogeneous graph
embedding methods.

  * •

metapath2vec (Dong et al., 2017): A heterogeneous graph embedding method that
performs a random walk on meta-paths and utilizes skip-gram to generate
embedding. This model relies on a single user-specified meta-path, so we test
all meta-paths separately and report the best results.

  * •

GCN (Kipf and Welling, 2017): A homogeneous graph convolutional network (GCN)
performs on the meta-path connected homogeneous graphs generated by different
meta-paths. We test GCN on different meta-paths and report the results from
the best meta-path.

  * •

GAT (Velickovic et al., 2018): A homogeneous GNN, which utilizes the masked
attention mechanism on the homogeneous graphs. Similarly, we test GAT on meta-
path based homogeneous graphs and report the results from the best meta-path.

  * •

HetGNN (Zhang et al., 2019): A heterogeneous GNN, which jointly considers
heterogeneous node contents encoding, type-based neighbors aggregation, and
heterogeneous types combination.

  * •

HAN (Wang et al., 2019): A heterogeneous GNN that learns meta-path specific
node embeddings from different meta-paths first and then leverages the
attention mechanism to combine them into one vector representation for each
node.

  * •

MAGNN (Fu et al., 2020): A heterogeneous GNN that boosts the graph embedding
performance by encapsulating input node attributes, incorporating intermediate
semantic nodes, and combining messages from multiple meta-paths.

  * •

MAGNN-AC (Jin et al., 2021): A state-of-the-art heterogeneous GNN, which uses
topological relationship between nodes as guidance to complete attributes for
no-attribute nodes via attention mechanism. It can solve missing attributes
problem in heterogeneous graphs.

####  5.2.2. Implementation Details

We implement our framework using the PyTorch library444Our source code and
datasets are available at: https://github.com/Wentao-Xu/SHGNN. (Paszke et al.,
2019), and run on all experiments on a single NVIDIA Tesla V100 GPU. To
eliminate the fluctuations caused by different initialization, we repeat the
training and testing procedure 10 times for all the experimental results and
report the average values. For the hyper-parameters, we mainly tune the
learning rate and the hyper-parameter $d_{1}$, which is the dimension of many
vectors in the SHGNN. The best learning rate is 0.005; the best $d_{1}$ is 512
for IMDB, 128 for DBLP and ACM.

###  5.3. Node Classification Task (RQ1)

We first compare the performance of different methods on the node
classification task, which is a frequently-used task to evaluate the
heterogeneous graph embeddings. We feed the embeddings of labeled nodes
(movies in IMDB, authors in DBLP, and papers in ACM) learned by each embedding
model to a linear support vector machine (SVM) classifier. The proportions of
nodes used for training the SVM are range from 20% to 80%. Following the
setting of previous work (Wang et al., 2019; Fu et al., 2020; Jin et al.,
2021), only the nodes in the testing set are fed to the linear SVM because the
labels of nodes in the training and validation sets have participated in the
model training process.

Table 2 shows the results of Macro-F1 and Micro-F1 on IMDB, DBLP and ACM
datasets. We reproduce the results of
MAGNN555https://github.com/cynricfu/MAGNN and MAGNN-
AC666https://github.com/liangchundong/HGNN-AC with their open-source codes,
and we take the results of other baselines from (Jin et al., 2021). From Table
2 we can find that our SHGNN outperforms other baselines across all training
proportions and datasets. On the IMDB dataset, our SHGNN gains the 3% to 5%
improvements on Macro-F1 and Micro-F1 compare with the current state-of-the-
art MAGNN-AC. On DBLP and ACM datasets, even the existing methods have
obtained high scores over 90%; our SHGNN still achieves better performance.
The improvements on the IMDB, DBLP and ACM datasets verify that incorporating
the graph structural information in the node’s feature and the information
aggregation on meta-paths can boost the performance of heterogeneous graph
embeddings.

Table 3. The results (%) of node clustering.

Dataset | IMDB | DBLP | ACM
---|---|---|---
Metrics | NMI | ARI | NMI | ARI | NMI | ARI
metapath2vec | 0.89 | 0.22 | 74.18 | 78.11 | - | -
GCM | 7.46 | 7.69 | 73.45 | 77.50 | - | -
GAT | 7.84 | 8.87 | 70.73 | 76.04 | - | -
HAN | 10.79 | 11.11 | 77.49 | 82.95 | - | -
MAGNN | 14.03 | 13.25 | 79.63 | 84.58 | 66.48 | 71.32
MAGNN-AC | 13.82 | 13.78 | 80.29 | 85.47 | 67.52 | 72.06
SHGNN | 14.75 | 16.55 | 82.50 | 86.83 | 72.94 | 77.41

###  5.4. Node Clustering Task (RQ2)

We further evaluate the performance of our SHGNN on another widely-used task:
the node clustering task, to evaluate the heterogeneous graph embeddings. We
feed the embeddings of labeled nodes (movies in IMDB, authors in DBLP, and
papers in ACM) generated by each method to the K-Means algorithm. We set the
number of clusters in K-Means as the number of each dataset’s classes (3 for
IMDB and ACM, and 4 for DBLP). Following the previous work (Wang et al., 2019;
Fu et al., 2020), we adopt the normalized mutual information (NMI) and
adjusted rand index (ARI) as evaluation metrics.

Table 3 shows the results of NMI and ARI on IMDB, DBLP, and ACM datasets. We
reproduce the MAGNN and MAGNN-AC with their open-source codes, and we take the
results of other compared methods from (Fu et al., 2020). From Table 3, we can
see that SHGNN is consistently better than other baselines in node clustering.
Note that all models have much poorer performance on IMDB than on DBLP and
ACM. This is probably because every movie node in the original IMDB database
has multiple genres, but we only select the first one as its class label. Our
SHGNN achieves a 5% relative improvement on NMI and 20% on ARI compared with
the current best results on the IMDB dataset. On the ACM dataset, our SHGNN
obtains a nearly 8% relative improvement on NMI and 7% on ARI compared with
the state-of-the-art MAGNN-AC model. These improvements achieved by SHGNN on
the node clustering task further demonstrate the effectiveness of our method.

###  5.5. Ablation Study (RQ3)

To study the effect of some key designs in our framework, we apply an ablation
study on our SHGNN model. To be specific, we study the effect of node
centrality encoding in Section 4.2 and tree-attention aggregator in Section
4.3. We remove some components in SHGNN and observe the new experimental
results of SHGNN’s variants. For the variants without node centrality
encoding, we directly use the node structural features in Section 4.1 as the
node’s latent vector. For the variants without a tree-attention aggregator, we
calculate the attention values between two end nodes of the meta-path and
directly aggregate information from meta-path based neighbors to the target
node with the attention values. Table 4 shows the results of ablation study,
and we have the following observations:

  1. (1)

Adding the node centrality encoding to differentiate the importance of nodes
in different structural information can boost the performance of heterogeneous
graph embeddings.

  2. (2)

Both node centrality indicators $c(v)$ and $c^{+}(v)$ are important and
helpful to embeddings.

  3. (3)

The tree-attention aggregator can improve the performance, especially on the
DBLP dataset. Therefore, incorporating the graph structural information in
information aggregation on meta-paths is effective for heterogeneous graph
embeddings.

Table 4. The node classification results (%) of ablation study. The ✓and - indicate having or not having the component. The Macro and Micro are the average Macro-F1 and Micro-F1 of different training proportions (20%, 40%, 60% and 80%). Centrality | Tree- | IMDB | DBLP
---|---|---|---
$c(v)$ | $c^{+}(v)$ | attention | Macro | Micro | Macro | Micro
- | - | - | 59.99 | 60.12 | 91.96 | 92.53
✓ | - | - | 60.93 | 61.14 | 92.49 | 93.09
✓ | ✓ | - | 61.80 | 61.95 | 93.22 | 93.83
- | - | ✓ | 61.75 | 61.86 | 94.35 | 94.80
✓ | ✓ | ✓ | 62.49 | 62.62 | 94.91 | 95.29

(a) MAGNN

(b) MAGNN-AC

(c) SHGNN

Figure 5. Visualization of the Author nodes’ embeddings in the test set of
DBLP dataset. Each point represents an author and its color indicates the
research area of author.

###  5.6. Visualization (RQ4)

We visualize the same-dimensional node embeddings of different methods to show
a more intuitive comparison. We use the t-SNE (Van der Maaten and Hinton,
2008) to visualize the embeddings of Author nodes in the test set of the DBLP
dataset. Figure 5 shows the results of visualization. The MAGNN, MAGNN-AC, and
SHGNN methods can separate the nodes of different classes, but our SHGNN
incorporates the structural information into the node embeddings; thus, our
SHGNN can relatively better separate the nodes of various labels.

##  6\. Conclusion

We propose a new structure-aware heterogeneous graph neural network (SHGNN)
that can incorporate structural information into graph embeddings. The key
insights of our SHGNN include: 1) encoding the node centrality, which measures
the importance of nodes; 2) incorporating the graph structure information into
the aggregation module on meta-path. Experimental results of node
classification and node clustering tasks on benchmark datasets demonstrate the
effectiveness of our SHGNN.

## References

  * (1)
  * Atwood and Towsley (2016) James Atwood and Don Towsley. 2016.  Diffusion-Convolutional Neural Networks.  In _NIPS_. 1993–2001.
  * Battaglia et al. (2016) Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, and koray kavukcuoglu. 2016\.  Interaction Networks for Learning about Objects, Relations and Physics.  In _NIPS_. 4502–4510.
  * Bordes et al. (2013) Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013\.  Translating embeddings for modeling multi-relational data. In _Advances in neural information processing systems_. 2787–2795.
  * Cho et al. (2014) Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014.  Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation.  _CoRR_ abs/1406.1078 (2014).  arXiv:1406.1078
  * Defferrard et al. (2016) Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In _NIPS_. 3844–3852.
  * Dong et al. (2017) Yuxiao Dong, Nitesh V. Chawla, and Ananthram Swami. 2017\.  Metapath2Vec: Scalable Representation Learning for Heterogeneous Networks. In _SIGKDD_. 135–144.
  * Fan et al. (2019) Shaohua Fan, Junxiong Zhu, Xiaotian Han, Chuan Shi, Linmei Hu, Biyu Ma, and Yongliang Li. 2019.  Metapath-guided heterogeneous graph neural network for intent recommendation. In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 2478–2486.
  * Fout et al. (2017) Alex Fout, Jonathon Byrd, Basir Shariat, and Asa Ben-Hur. 2017\.  Protein Interface Prediction using Graph Convolutional Networks.  In _NIPS_. 6530–6539.
  * Fu et al. (2017) Tao-yang Fu, Wang-Chien Lee, and Zhen Lei. 2017.  HIN2Vec: Explore Meta-paths in Heterogeneous Information Networks for Representation Learning. In _CIKM_. 1797–1806.
  * Fu et al. (2020) Xinyu Fu, Jiani Zhang, Ziqiao Meng, and Irwin King. 2020\.  MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding. In _Proceedings of The Web Conference 2020_. 2331–2341.
  * Grover and Leskovec (2016) Aditya Grover and Jure Leskovec. 2016.  Node2Vec: Scalable Feature Learning for Networks. In _SIGKDD_. 855–864.
  * Hamilton et al. (2017) William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017\.  Inductive Representation Learning on Large Graphs. In _NIPS_. 1024–1034.
  * Hu et al. (2020) Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. 2020\.  Heterogeneous graph transformer. In _Proceedings of The Web Conference 2020_. 2704–2710.
  * Huang et al. (2019) Xiao Huang, Jingyuan Zhang, Dingcheng Li, and Ping Li. 2019\.  Knowledge graph embedding based question answering. In _Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining_. 105–113.
  * Jin et al. (2021) Di Jin, Cuiying Huo, Chundong Liang, and Liang Yang. 2021\.  Heterogeneous Graph Neural Network via Attribute Completion. In _Proceedings of the Web Conference 2021_. 391–400.
  * Jin et al. (2020) Jiarui Jin, Jiarui Qin, Yuchen Fang, Kounianhua Du, Weinan Zhang, Yong Yu, Zheng Zhang, and Alexander J Smola. 2020\.  An Efficient Neighborhood-based Interaction Model for Recommendation on Heterogeneous Graph. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 75–84.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ (2014).
  * Kipf and Welling (2017) Thomas N. Kipf and Max Welling. 2017.  Semi-Supervised Classification with Graph Convolutional Networks. In _ICLR_.
  * Li et al. (2016) Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard S. Zemel. 2016.  Gated Graph Sequence Neural Networks. In _ICLR_.
  * Li et al. (2018) Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2018\.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. In _ICLR_.
  * Liu et al. (2020) Siwei Liu, Iadh Ounis, Craig Macdonald, and Zaiqiao Meng. 2020\.  A Heterogeneous Graph Neural Model for Cold-Start Recommendation. In _Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval_. 2029–2032.
  * Marshall (2010) P David Marshall. 2010\.  The promotion and presentation of the self: celebrity as marker of presentational media.  _Celebrity studies_ 1, 1 (2010), 35–48.
  * Mikolov et al. (2013a) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a.  Efficient estimation of word representations in vector space.  _arXiv preprint arXiv:1301.3781_ (2013).
  * Mikolov et al. (2013b) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013b.  Efficient Estimation of Word Representations in Vector Space. In _ICLR_.
  * Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019\.  Pytorch: An imperative style, high-performance deep learning library.  _arXiv preprint arXiv:1912.01703_ (2019).
  * Perozzi et al. (2014) Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014\.  DeepWalk: Online Learning of Social Representations. In _SIGKDD_. 701–710.
  * Ribeiro et al. (2017) Leonardo FR Ribeiro, Pedro HP Saverese, and Daniel R Figueiredo. 2017.  struc2vec: Learning node representations from structural identity. In _Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining_. 385–394.
  * Shang et al. (2016) Jingbo Shang, Meng Qu, Jialu Liu, Lance M. Kaplan, Jiawei Han, and Jian Peng. 2016\.  Meta-Path Guided Embedding for Similarity Search in Large-Scale Heterogeneous Information Networks.  _CoRR_ abs/1610.09769 (2016).  arXiv:1610.09769
  * Shangwen Lv and Hu (2020) Jingjing Xu Duyu Tang Nan Duan Ming Gong Linjun Shou Daxin Jiang Guihong Cao Shangwen Lv, Daya Guo and Songlin Hu. 2020.  Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering. In _The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, New York, USA_. AAAI Press, 8449–8456.
  * Shi et al. (2018) Chuan Shi, Binbin Hu, Wayne Xin Zhao, and S Yu Philip. 2018\.  Heterogeneous information network embedding for recommendation.  _IEEE Transactions on Knowledge and Data Engineering_ 31, 2 (2018), 357–370.
  * Sun and Han (2013) Yizhou Sun and Jiawei Han. 2013.  Mining heterogeneous information networks: a structural analysis approach.  _Acm Sigkdd Explorations Newsletter_ 14, 2 (2013), 20–28.
  * Sun et al. (2011) Yizhou Sun, Jiawei Han, Xifeng Yan, Philip S Yu, and Tianyi Wu. 2011.  Pathsim: Meta path-based top-k similarity search in heterogeneous information networks.  _Proceedings of the VLDB Endowment_ 4, 11 (2011), 992–1003.
  * Tang et al. (2015) Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015\.  LINE: Large-scale Information Network Embedding. In _WWW_. 1067–1077.
  * Teney et al. (2017) Damien Teney, Lingqiao Liu, and Anton van Den Hengel. 2017\.  Graph-structured representations for visual question answering. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 1–9.
  * Van der Maaten and Hinton (2008) Laurens Van der Maaten and Geoffrey Hinton. 2008.  Visualizing data using t-SNE.  _Journal of machine learning research_ 9, 11 (2008).
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017\.  Attention is All you Need. In _NIPS_. 5998–6008.
  * Velickovic et al. (2018) Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018.  Graph Attention Networks. In _ICLR_.
  * Wang et al. (2016a) Daixin Wang, Peng Cui, and Wenwu Zhu. 2016a.  Structural Deep Network Embedding. In _SIGKDD_. 1225–1234.
  * Wang et al. (2016b) Daixin Wang, Peng Cui, and Wenwu Zhu. 2016b.  Structural deep network embedding. In _Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining_. 1225–1234.
  * Wang et al. (2019) Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu. 2019.  Heterogeneous Graph Attention Network. In _WWW_. 2022–2032.
  * Xu et al. (2020) Wentao Xu, Shun Zheng, Liang He, Bin Shao, Jian Yin, and Tie-Yan Liu. 2020.  SEEK: Segmented Embedding of Knowledge Graphs. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Online, 3888–3897.
  * Yang et al. (2015) Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y. Chang. 2015.  Network Representation Learning with Rich Text Information. In _IJCAI_. 2111–2117.
  * Ying et al. (2021) Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. 2021\.  Do Transformers Really Perform Bad for Graph Representation?  _arXiv preprint arXiv:2106.05234_ (2021).
  * Yun et al. (2019) Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo J Kim. 2019.  Graph transformer networks. In _Advances in Neural Information Processing Systems_. 11983–11993.
  * Zhang et al. (2019) Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V Chawla. 2019.  Heterogeneous graph neural network. In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 793–803.
  * Zhang et al. (2018) Jiani Zhang, Xingjian Shi, Junyuan Xie, Hao Ma, Irwin King, and Dit-Yan Yeung. 2018\.  GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs. In _UAI_. 339–349.
  * Zhao et al. (2020) Chen Zhao, Chenyan Xiong, Xin Qian, and Jordan Boyd-Graber. 2020\.  Complex Factoid Question Answering with a Free-Text Knowledge Graph. In _Proceedings of The Web Conference 2020_. 1205–1216.
  * Zhao et al. (2019) Jun Zhao, Zhou Zhou, Ziyu Guan, Wei Zhao, Wei Ning, Guang Qiu, and Xiaofei He. 2019\.  Intentgc: a scalable graph convolution framework fusing heterogeneous information for recommendation. In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 2347–2357.