2023

Equal contribution.

Equal contribution.

[2]Quanming Yao

1]Baidu Research, Baidu Inc., Beijing, China

2]Department of Electronic Engineering, Tsinghua University, Beijing, China

3]Institute of Computing Technology, Chinese Academy of Sciences, Beijing,
China

# Accurate and interpretable drug-drug interaction prediction enabled by
knowledge subgraph learning

Yaqing Wang<EMAIL_ADDRESS>Zaifei Yang<EMAIL_ADDRESS><EMAIL_ADDRESS>[  [  [

###### Abstract

Background: Discovering potential drug-drug interactions (DDIs) is a long-
standing challenge in clinical treatments and drug developments. Recently,
deep learning techniques have been developed for DDI prediction. However, they
generally require a huge number of samples, while known DDIs are rare.

Methods: In this work, we present KnowDDI, a graph neural network-based method
that addresses the above challenge. KnowDDI enhances drug representations by
adaptively leveraging rich neighborhood information from large biomedical
knowledge graphs. Then, it learns a knowledge subgraph for each drug-pair to
interpret the predicted DDI, where each of the edges is associated with a
connection strength indicating the importance of a known DDI or resembling
strength between a drug-pair whose connection is unknown. Thus, the lack of
DDIs is implicitly compensated by the enriched drug representations and
propagated drug similarities.

Results: We evaluate KnowDDI on two benchmark DDI datasets. Results show that
KnowDDI obtains the state-of-the-art prediction performance with better
interpretability. We also find that KnowDDI suffers less than existing works
given a sparser knowledge graph. This indicates that the propagated drug
similarities play a more important role in compensating for the lack of DDIs
when the drug representations are less enriched.

Conclusions: KnowDDI nicely combines the efficiency of deep learning
techniques and the rich prior knowledge in biomedical knowledge graphs. As an
original open-source tool, KnowDDI can help detect possible interactions in a
broad range of relevant interaction prediction tasks, such as protein-protein
interactions, drug-target interactions and disease-gene interactions,
eventually promoting the development of biomedicine and healthcare.

###### keywords:

Drug Development, Drug-Drug Interaction, Biomedical Network, Knowledge Graph,
Deep Learning, Graph Neural Network, Bioinformatics

##  1 Introduction

Accurately predicting drug-drug interaction (DDI) can play an important role
in the field of biomedicine and healthcare. On the one hand, combination
therapies, where multiple drugs are used together, can be used to treat
complex disease and comorbidities, such as human immunodeficiency virus (HIV)
juurlink2003drug ; bangalore2007fixed . Recent study also shows that
combination therapies, such as a combination of lopinavir and ritonavir, may
treat coronavirus disease (COVID-19) scavone2020current ; chakraborty2021drug
; akinbolade2022combination , the infectious disease which causes global
pandemic in the past three years. On the other hand, DDI is an important cause
of adverse drug reactions, which accounts for 1% hospitalizations in the
general population and 2-5% hospital admissions in the elderly
percha2013informatics ; letinier2019risk ; jiang2022adverse . A concrete
example is that if warfarin and aspirin enter the body together, they will
compete for binding to plasma proteins. Then, the remained warfarin that
cannot be bounded to plasma proteins will remain in the blood, which results
in acute bleeding in patients marijon2013causes .

Identifying DDIs by clinical evidence such as laboratory studies is extremely
costly and time-consuming percha2013informatics ; jiang2022adverse . In recent
years, computational techniques especially deep learning approaches are
developed to speed up the discovery of potential DDIs. Naturally, DDI fact
triplets can be represented as a graph where each node corresponds to a drug,
and each edge represents an interaction between two drugs. Provided with DDI
fact triplets, a number of graph learning methods have been developed to
identify unknown interactions between drug-pairs. Graph neural networks (GNNs)
zitnik2018modeling ; huang2020skipgnn , which can obtain expressive node
embeddings by end-to-end learning from the topological structure and
associated node features, have also been applied for DDI prediction problem.
However, known DDI fact triplets are rare due to the high experimental cost
and continually emerging new drugs derry2001incomplete . For example, the
latest DrugBank database with 14,931 drug entries only contains 365,984 known
DDI fact triplets wishart2018drugbank , the quantity of which is less than 1%
of the total potential DDIs. This makes over-parameterized deep learning
models fail to give full play to its expressive ability and may perform even
worse than traditional two-stage embedding methods perozzi2014deepwalk ;
yao2022effective .

In biomedicine and healthcare, many international level agencies such as
National Center for Biotechnology Information and European Bioinformatics
Institute are endeavored to regularly maintain rich publicly available
biomedical data resources bonner2021review . Researchers then integrate these
disparate and heterogeneous data resources into knowledge graphs (KGs) to
facilitate an organized use of information. Examples are Hetionet
himmelstein2015heterogeneous , PharmKG zheng2021pharmkg  and PrimeKG
Chandak2023 . These KGs contain rich prior knowledge discovered in biomedicine
and healthcare. A proper usage of them may compensate for the lack of samples
for DDI prediction. The pioneer work KGNN lin2020kgnn  firstly leverages
external KGs to provide topological information for each drug in target drug-
pair. In particular, it uniformly samples a fixed size set of neighbors around
each drug, then aggregates drug features and messages from the sampled
neighbors into the drug representation without considering which drug to
interact. Later works merge the DDI network with external KGs as a combined
network, extract enclosing subgraphs for different drug-pairs to encode the
drug-pair specific information, and then predict DDI for the target drug-pair
using the concatenation of nodes embeddings of drugs and subgraph embedding of
enclosing subgraphs teru2020inductive ; yu2021sumgnn ; Hong2022 . However, as
these KGs integrate diverse data resources by automated process or experts,
existing methods fail to filter out noise or inconsistent information. As a
result, properly leveraging external KGs is still a challenging problem.

In this paper, we propose KnowDDI, an accurate and interpretable method for
DDI prediction. First, we merge the provided DDI graph and an external KG into
a combined network, upon which generic representations for all nodes are
learned to encode the generic knowledge. Next, we extract a drug-flow subgraph
for each drug-pair from the combined network. We then learn a knowledge
subgraph from generic representations and the drug-flow subgraph. After
optimization, the representations of drugs are transformed to be more
predictive of the DDI types between the target drug-pair. In addition, the
returned knowledge subgraph contains explaining paths to interpret the
prediction result for the drug-pair, where the explaining paths consist of
only edges of important known DDIs or newly-added edges connecting highly
similar drugs. In other words, the learned knowledge subgraph helps filter out
irrelevant information and adds in resembling relationships between drugs
whose interactions are unknown. This allows the lack of DDIs to be implicitly
compensated by the enriched drug representations and propagated drug
similarities. We perform extensive experimental results on benchmark datasets,
and observe that KnowDDI consistently outperforms existing works. We also
conduct a series of case studies which further show that KnowDDI can discover
convincing explaining paths which help interpret the DDI prediction results.
KnowDDI has the potential to be used in a broad range of relevant interaction
prediction tasks, such as protein-protein interactions, drug-target
interactions and disease-gene interactions to help detect potential
interactions, eventually advancing the development of biomedicine and
healthcare.

Figure 1: Overview of KnowDDI. On a combined network which merges the DDI
graph with an external KG, generic embeddings of all nodes are firstly learned
to capture generic knowledge. Then for each target drug-pair, a drug-flow
subgraph is extracted from the combined network, whose node embeddings are
initialized as the generic embeddings. Via propagating drug resembling
relationships, the generic embeddings are transformed to be more predictive of
the DDI types between the drug-pair, and the drug-flow subgraph is adapted as
knowledge subgraph which contains explaining paths to interpret the prediction
result.

##  2 Overview of KnowDDI

Our KnowDDI (Figure1) learns to predict DDIs between a drug-pair, i.e., head
drug $h$ and tail drug $t$ in the DDI graph, by learning with knowledge
subgraph, i.e., denoted as $\mathcal{S}_{h,t}$. The provided DDI graph and an
external KG are merged into a combined network as the start. Every node of the
combined network is associated with a unique generic embedding which is
learned to encode the generic knowledge. Given a target drug-pair $(h,t)$, a
drug-flow subgraph $\bar{\mathcal{S}}_{h,t}$ which captures local context
relevant to $(h,t)$ is extracted from the combined network. As directly
leveraging external KG (and hence $\bar{\mathcal{S}}_{h,t}$) may bring in
irrelevant information, the graph structure and node embeddings of
$\bar{\mathcal{S}}_{h,t}$ are further iteratively optimized. During this
process, the generic embeddings are transformed to be more predictive of the
DDI types between the target drug-pair. In addition, KnowDDI estimates a
connection strength for every drug-pair in the subgraph, representing the
importance of a given edge between connected nodes in the drug-flow subgraph
or similarity between two nodes whose connection is unknown. Accordingly, a
new edge of type “resemble” is added between two nodes if their node
embeddings are highly similar, and existing edges can be dropped if the
importance is estimated as low. Thus, only useful information flows between
nodes are kept. The final optimized subgraph becomes our knowledge subgraph
$\mathcal{S}_{h,t}$ which consists of explaining paths. The average connection
strength over all consecutive node-pairs along each explaining path indicates
its ability of explaining the current prediction result in the perspective of
KnowDDI. Supplementary Table 1 shows a summary of characteristics comparing
KnowDDI with existing works.

##  3 Results

###  3.1 Data

In this study, we perform experiments on two publicly available benchmark DDI
datasets: (i) Drugbank wishart2018drugbank  is a multiclass DDI prediction
dataset consisting of 86 types of pharmacological relations occurred between
drugs; and (ii) TWOSIDES tatonetti2012data  is a multilabel DDI prediction
dataset recording multiple DDI side effects between drugs. We adopt Hetionet
himmelstein2015heterogeneous ; bonner2021review , which is a benchmark
biomedical KG for various tasks within drug discovery, as the external KG in
this paper. Other recent developed biomedical KGs such as ogbl-biokg
hu2020open , OpenBioLink breit2020openbiolink , and PharmKG zheng2021pharmkg
can also be used.

#### Data preprocessing

We preprocess the two benchmark DDI datasets DrugBank wishart2018drugbank  and
TWOSIDES tatonetti2012data  following the same procedure adopted by SumGNN
yu2021sumgnn . In DrugBank, relations are skewed. Each drug-pair is filtered
to have one relation only yu2021sumgnn . In TWOSIDES, 200 commonly occurring
relations are selected. In particular, relations are ranked by decreasing
number of associating fact triplets, and the 200 relations ranked between 600
to 800 are kept such that each relation is associated with at least 900 fact
triplets zitnik2018modeling . Thus, relations in TWOSIDES are associated with
comparable number of fact triplets. We formulate the benchmark DDI datasets as
DDI graphs separately, whose statistics are summarized in Table 1. The fact
triplets in DDI datasets are split into training, validation, and testing sets
with a ratio of 7:1:2 following SumGNN yu2021sumgnn  for fair comparison. We
remove from external KG the drug-drug edges contained in DDI graph to avoid
information leakage, then merge the resultant external KG and DDI graph into a
large combined network. Eventually, the DDI graph of DrugBank is merged with a
graph of 33765 nodes and 1690693 edges extracted from Hetionet, and the DDI
graph of TWOSIDES is merged with a graph of 28132 nodes and 1666632 edges
extracted from Hetionet, respectively. During training, the drug-drug edges in
validation and testing sets are unseen. After tuning hyperparameters on fact
triplets in validation set, the model performance is evaluated on fact
triplets in testing set.

#### Evaluation metric

We evaluate the multiclass DDI prediction performance by three metrics: (i)
Macro-averaged F1 which is averaged over class-wise F1 scores, (ii) Accuracy
(ACC) which is the micro-averaged F1 score calculated using all testing fact
triplets, and (iii) Cohen’s $\kappa$ which measures the inter-annotator
agreement. As for multilabel DDI prediction, we report the results averaged
over all relation types. The performance is evaluated by (i) AUROC which is
the average area under the receiver operating characteristics (ROC) curve,
(ii) AUPRC which is the average area under the precision-recall (PR) curve,
and (iii) AP@50 which is the average precision at 50.

###  3.2 Comparison with the state-of-the-art

Table 1: Performance (%) is evaluated on two benchmark DDI datasets DrugBank
and TWOSIDES. Average (Avg.) rank of each method is reported in the last
column, which is averaged over the six columns of performance. The best and
comparable results (according to the pairwise t-test with 95% confidence) are
highlighted in bold, and the second-best results are underlined. $|\cdot|$
counts the number of elements in a set. $\mathcal{V}$ is the set of drug
nodes, $\mathcal{E}$ is the set of fact triplets, and $\mathcal{R}$ is the set
of relation types.

| Dataset | DrugBank (multiclass) | TWOSIDES (multilabel) |
---|---|---|---|---
| Statistics | $|\mathcal{V}|=1710,|\mathcal{R}|=86,|\mathcal{E}|=192284$ | $|\mathcal{V}|=604,|\mathcal{R}|=200,|\mathcal{E}|=41270$ | Avg.
| Metric | F1 | ACC |  Cohen’s $\kappa$ | AUROC | AUPRC | AP@50 | Rank
| TransE |  18.32$\pm$0.16 |  64.60$\pm$0.11 |  57.19$\pm$1.22 |  77.53$\pm$0.02 |  70.16$\pm$0.02 |  77.54$\pm$0.03 | 15
Traditional | KG-DDI |  37.21$\pm$0.36 |  82.74$\pm$0.10 |  78.71$\pm$0.33 |  90.64$\pm$0.09 |  87.99$\pm$0.11 |  83.50$\pm$0.06 | 10
Two-stage | MSTE |  53.96$\pm$0.07 |  77.76$\pm$0.11 |  73.35$\pm$0.05 |  89.40$\pm$0.04 |  83.06$\pm$0.06 |  79.38$\pm$0.03 | 12
(w/o external | node2vec |  50.00$\pm$1.67 |  62.39$\pm$0.99 |  56.27$\pm$0.89 |  90.62$\pm$0.43 |  89.42$\pm$0.47 |  82.21$\pm$0.54 | 12
KG) | DeepWalk |  49.17$\pm$1.38 |  62.90$\pm$0.37 |  56.77$\pm$0.44 |  91.77$\pm$0.26 |  90.56$\pm$0.23 |  84.13$\pm$0.35 | 9
| LINE |  48.89$\pm$1.38 |  59.87$\pm$0.92 |  52.15$\pm$1.51 |  88.63$\pm$0.20 |  87.02$\pm$0.22 |  80.80$\pm$0.20 | 14
GNN-based | GAT |  35.05$\pm$0.41 |  78.02$\pm$0.14 |  74.68$\pm$0.21 |  91.22$\pm$0.13 |  89.79$\pm$0.10 |  83.05$\pm$0.18 | 11
(w/o external | Decagon |  56.24$\pm$0.27 |  86.97$\pm$0.31 |  86.12$\pm$0.09 |  91.83$\pm$0.14 |  90.79$\pm$0.18 |  82.49$\pm$0.36 | 7
KG) | SkipGNN |  62.36$\pm$0.96 |  88.04$\pm$0.66 |  85.71$\pm$0.81 |  92.31$\pm$0.15 |  90.84$\pm$0.03 |  84.23$\pm$0.19 | 6
| GraIL |  75.92$\pm$0.69 |  89.63$\pm$0.39 |  87.63$\pm$0.47 |  93.73$\pm$0.10 |  92.26$\pm$0.07 |  86.89$\pm$0.11 | 3
GNN-based | KGNN |  74.08$\pm$0.92 |  88.30$\pm$0.08 |  86.09$\pm$0.10 |  92.93$\pm$0.10 |  90.11$\pm$0.14 |  87.43$\pm$0.09 | 5
(w/ external | DDKG |  75.84$\pm$0.22 |  88.70$\pm$0.39 |  87.53$\pm$0.21 |  93.15$\pm$0.18 |  91.09$\pm$0.39 |  87.50$\pm$0.43 | 4
KG) | SumGNN | 86.88$\pm$0.63 | 91.86$\pm$0.23 | 90.34$\pm$0.28 | 94.61$\pm$0.06 | 93.13$\pm$0.15 | 88.38$\pm$0.07 | 2
| LaGAT |  83.69$\pm$0.74 |  88.86$\pm$0.12 |  87.33$\pm$0.14 |  88.72$\pm$0.22 |  84.03$\pm$0.43 |  82.46$\pm$0.41 | 8
| KnowDDI | 91.53$\pm$0.24 | 93.17$\pm$0.09 | 91.89$\pm$0.11 | 95.43$\pm$0.02 | 94.14$\pm$0.03 | 89.54$\pm$0.03 | 1

We consider multi-typed DDI prediction problem where interactions between
drugs can have multiple relation types. For example, a drug-pair (drug A, drug
B) can have two relation types “the matabolism of drug A can be decreased when
combined with drug B” and “the therapeutic efficacy of drug A can be increased
when combined with drug B”. In particular, we compare the proposed KnowDDI
with the following models:

  * •

Traditional two-stage methods (w/o external KG). (i) KG embedding-based
methods use shallow linear models to encode drug entities and their associated
relations into low-dimentional embeddings, then feed the drug embeddings into
a separately learned classifier for DDI prediction. Exemplar methods are
TransE bordes2013translating , KG-DDI karim2019drug , and MSTE
yao2022effective . (ii) Network embedding-based methods which use neural
networks to encode structural information into node embeddings of drugs, and
predict the relation types by a linear layer. Exemplar methods are DeepWalk
perozzi2014deepwalk , node2vec grover2016node2vec , and LINE tang2015line .

  * •

GNN-based methods (w/o external KG) formulate the existing DDI fact triplets
into the form of a graph where each node corresponds to a drug and each edge
between two drugs represents one relation type, then solve the resultant link
prediction problem on the DDI graph using GNNs kipf2017semi , including GAT
velivckovic2018graph , Decagon zitnik2018modeling , and SkipGNN
huang2020skipgnn .

  * •

GNN-based methods (w/ external KG) leverage an external KG to provide rich
organized biomedical knowledge, and aggregate the messages from neighboring
nodes of drugs by GNNs. Existing methods include GraIL teru2020inductive ,
KGNN lin2020kgnn , DDKG su2022attention , SumGNN yu2021sumgnn , and LaGAT
Hong2022 .

We implement the baselines using public codes of the respective authors, and
provide the links in Supplementary Table 2.

#### Overall performance

Table 1 shows the results obtained on two benchmark DDI datasets. Overall, we
can see that GNN-based methods (w/ external KG) generally perform the best and
traditional two-stage methods generally perform the worst. Comparing with
traditional two-stage methods, GNN-based methods (w/o external KG) can better
propagate information among connected nodes (i.e., drugs) by modeling the fact
triplets integrally as a graph and jointly learning all model parameters
w.r.t. the objective in an end-to-end manner. However, due to the lack of DDI
fact triplets and the over-parameterization of GNN, they may not consistently
be better than traditional two-stage methods. This can be supported by the
observation that DeepWalk and KG-DDI perform better than the deep GAT.

Next, GNN-based methods ((w/ external KG)) leverage rich biomedical knowledge
to alleviate the data scarcity problem. Among these methods, KGNN performs the
worst. In contrast to pure GNN, KGNN uniformly samples $N$ nodes as neighbors
of each node during message passing to reduce computational overhead. DDKG
improves KGNN by assigning attention weights to the $N$ nodes during message
passing, where the attention weights are obtained by calculating the
similarity between initial node embeddings constructed from SMILES. In the
end, each drug obtains its representation without considering which drug to
interact in KGNN and DDKG. While GraIL, SumGNN, LaGAT and our KnowDDI merge
the DDI graph with an external KG as a large combined network, then learn to
encode more local semantic information from the combined network by extracting
subgraphs w.r.t. drug-pairs. A drug can be represented differently in
different subgraphs. Thus, these methods can obtain drug-pair-aware
representations that can be beneficial to predict DDI types. In particular,
GraIL directly propagates messages on the extracted subgraphs, LaGAT
aggregates messages with attention weights calculated using node embeddings,
and SumGNN only prunes edges based on node features that are randomly
initialized or pretrained on other tasks and then fixes the subgraphs. They
all cannot adaptively adjust the structure of subgraphs during learning.

Finally, KnowDDI learns to remove irrelevant edges and add new edge of type
“resemble” based on learned node embeddings. Upon the purified subgraph (i.e.,
knowledge subgraph) of target drug-pair, KnowDDI transforms generic node
embeddings to be more predictive of DDI types. The performance gain of KnowDDI
over existing methods validates its effectiveness.

#### Relation-wise performance

Next, we take a closer look at the performance gain w.r.t. different relations
grouped by frequency. Figure2(a) shows the relation-wise F1 score (%) grouped
into bins according to the number of fact triplets associated with the
relation. We compare the proposed KnowDDI with SkipGNN which performs the best
among GNN-based methods (w/o external KG), and SumGNN which obtains the
second-best among GNN-based methods (w/ external KG). In addition, we compare
with KnowDDI (w/o resemble), a variant of KnowDDI which does not add new edges
of type “resemble” between nodes with highly similar node embeddings.

As shown, by comparing the performance of SkipGNN and the other methods, we
can see that external KG plays an important role. In general, KnowDDI and
KnowDDI (w/o resemble) consistently obtain better performance than SumGNN, and
the performance gap is larger on relations with fewer known fact triplets.
This shows that enriched drug representations and adjusted subgraphs can be
helpful to compensate for the lack of known DDI fact triplets. KnowDDI
performs the best, which further shows the contribution of learning to
propagate resembling relationships between highly similar nodes. Additionally,
Supplementary Figure1 shows the relation frequency and Supplementary Figure2
shows the relation-wise performance improvement of KnowDDI over SumGNN on
DrugBank, TWOSIDES and a larger version of TWOSIDES with more relations.
Results consistently shows KnowDDI obtains better performance.

Figure 2:  The proposed KnowDDI is compared with (i) SumGNN which is a GNN-
based method (w/ external KG) and obtains the second-best results in Table 1,
(ii) SkipGNN which is the best among the GNN-based methods (w/o external KG)
in Table 1, and (iii) KnowDDI (w/o resemble) which does not propagate
resembling relationships between nodes whose interactions are unknown. a, F1
(%) obtained for relations with different number of DDI fact triplets on
DrugBank. b, F1 (%) obtained with different portion (%) of fact triplets
sampled from the external KG on DrugBank. SkipGNN, which does not use external
KG, is plotted just for reference.

#### Compensating unknown DDIs

Recall that the lack of DDI fact triplets is compensated by both enriched drug
representations and propagated drug similarities in KnowDDI. Here, we pay a
closer look to the effectiveness of these two designs. To achieve this goal,
we examine the performance of proposed KnowDDI and KnowDDI (w/o resemble) with
different amount of fact triplets introduced from external KG to the combined
network. We compare them with SumGNN which obtains the second-best among GNN-
based methods (w/ external KG), and take SkipGNN from GNN-based methods (w/o
external KG) as a reference.

Figure2(b) plots the performance changes w.r.t varying portion (%) of fact
triplets sampled from the external KG. First, SumGNN, KnowDDI and KnowDDI (w/o
resemble) all perform worse given fewer fact triplets from the external KG.
This is because a sparser external KG means less information introduced into
DDI datasets, which reduces the information gap between GNN-based methods w/
or w/o external KG. Then, KnowDDI (w/o resemble) consistently outperforms the
other two methods as it keeps information that is more relevant to predicting
DDI for the drug-pair at hand both during subgraph construction and learning.
Besides, KnowDDI is the best, as it further learns drug similarities by
propagating resembling relationships between drugs with highly similar
representations. As a result, KnowDDI suffers the least from a sparser KG.
However, the performance gap between KnowDDI (w/o resemble) and KnowDDI gets
larger with fewer triples. This means removing irrelevant edges and
propagating drug similarities have a larger influence on compensatingq for the
lack of DDIs when drug representations are less enriched. Finally, let us pay
special attention to the case of given 0% triples, which means external KG is
not used. Still, we can observe that both KnowDDI (w/o resemble) and SumGNN
still outperform SkipGNN. This can be attributed to the different subgraph
extraction strategies adopted in these three methods, which will be carefully
examined in Section 3.3.

Figure 3: Comparing different subgraphs extraction strategy. a, F1 (%)
obtained with different portion (%) of fact triplets sampled from the external
KG on DrugBank. b-e, four exemplar drug-pairs $(h,t)$s in DrugBank and their
enclosing subgraphs (first column), drug-flow subgraphs (second column),
knowledge subgraphs without propagating resembling relationships (third
column), and knowledge subgraphs (fourth column). Drugs from DDI graph are
colored in blue, drugs from external KG are colored in gray, while the other
entities are colored in green. In particular, we draw a red dashed circle
encompassing the target drug-pair, and mark the head drug $h$ by filled plus
and the tail drug $t$ by star. We mark edges from DDI graph and external KG in
solid blue lines and solid gray lines respectively. We specially use
undirected edges to represent relation type“resemble” as the semantics of
resemble are not directive, and plot new edges of relation type“resemble”
estimated by KnowDDI with dotted gray lines. The thickness of an edge is used
to reveal the magnitude of the connection strength learned by KnowDDI.

###  3.3 Effectiveness of knowledge subgraph

Here, we pay a closer look at knowledge subgraph $\mathcal{S}_{h,t}$ designed
in KnowDDI, and compare it with other choices of subgraphs in terms of
performance and interpretability.

#### Subgraph extraction strategy

To empirically validate the effectiveness of the proposed knowledge subgraph,
we consider the following subgraphs: (i) Random subgraph consists of a fixed-
size set of nodes uniformly sampled from the neighborhoods of $h$ and $t$ in
$\mathcal{G}$, which is adopted in KGNN lin2020kgnn  to reduce the computation
overhead; (ii) Enclosing subgraph is the interaction of $K$-hop neighborhoods
of $h$ and $t$ in $\mathcal{G}$, which is adopted in GraIL teru2020inductive ,
and SumGNN yu2021sumgnn ; (iii) Drug-flow subgraph $\bar{\mathcal{S}}_{h,t}$
consists of relational paths pointing from $h$ to $t$ in $\mathcal{G}$ with
length at least $P$; and (iv) Knowledge subgraph ${\mathcal{S}}_{h,t}$
consists of explaining paths from $h$ to $t$, which is obtained by iteratively
refining the graph structure and node embeddings of $\bar{\mathcal{S}}_{h,t}$.
Apart from them, we further compare with the knowledge subgraphs obtained by
KnowDDI (w/o resemble), and denote the results as knowledge subgraph (w/o
resemble).

Table 2: Explaining paths with the largest average connection strengths assigned by KnowDDI for four drug-pairs in DrugBank, possible explanations are discovered from Hetionet and DrugBank. Drug-Pair |  $(309,610)$
---|---
DDI Type |  The metabolism of Atomoxetine (610) can be decreased when combined with Reboxetine (309).
Explaining Path |  $309\xrightarrow[]{\text{resembles}}\underline{16879}\xrightarrow[]{\text{resembles}}610$
Explanation |  Reboxetine (309) resembles Diphenylpyraline (16879), while Diphenylpyraline (16879) resembles Atomoxetine (610). We can deduce that Reboxetine (309) resembles Atomoxetine (610). When taking two similar drugs, the body may absorb less of the Atomoxetine (610).
Drug-Pair |  $(103,1127)$
DDI Type |  The serum concentration of Betamethasone (103) can be increased if combined with Estriol (1127).
Explaining Path |  $103\xrightarrow[]{\text{resembles}}121\xrightarrow[]{\text{resembles (new edge)}}622\xrightarrow[]{\text{resembles}}1127$
Explanation |  Betamethasone (103) bears resemblance to Desonide (121). Although the interaction between Desonide (121) and Mestranol (622) is not provided, our search within DrugBank reveals that combining Mestranol (622) with Budesonide can elevate the serum concentration of Budesonide. Furthermore, Budesonide is similar to Desonide (121), and Mestranol (622) is similar to Estriol (1127). Given the tendency for similar drugs to exhibit akin properties, it is plausible that the serum concentration of Betamethasone (103) could rise when used alongside Estriol (1127).
Drug-Pair |  $(284,882)$
DDI Type |  The therapeutic efficacy of Sulfadiazine (284) can be increased when used in combination with Gatifloxacin (882).
Explaining Path |  $284\xrightarrow[]{\text{therapeutic efficacy increased}}852\xrightarrow[]{\text{resembles}}882$
Explanation |  The combined use of Sulfadiazine (282) and Pefloxacin (852) can improve the efficacy. Meanwhile, Pefloxacin (852) resembles Gatifloxacin (882). It can be deduced that the therapeutic efficacy of Sulfadiazine (282) can be increased when used in combination with Gatifloxacin (882).
Drug-Pair |  $(47,51)$
DDI Type |  The risk or severity of adverse effects can be increased when Atropine (47) is combined with Scopolamine (51).
Explaining Path |  $47\xrightarrow[]{\text{resembles}}16892\xrightarrow[]{\text{resembles}}51$
Explanation |  Atropine (47) resembles Homatropine methylbromide (16892), while Homatropine methylbromide (16892) resembles Scopolamine (51). We can deduce that Atropine (47) resembles Scopolamine (51). The similarities between the two drugs can also be seen through the chemical structures of the drugs. As the two drugs are similar in structure, the effects of using both drugs at the same time should be similar to the side effects of overdose of either drug, which can cause serious side effects. Based on DrugBank, Scopolamine (51) overdose may manifest as lethargy, somnolence, coma, confusion, agitation, hallucinations, convulsion, visual disturbance, dry flushed skin, dry mouth, decreased bowel sounds, urinary retention, tachycardia, hypertension and supraventricular arrhythmias, while Atropine (47) overdose may cause palpitation, dilated pupils, difficulty swallowing, hot dry skin, thirst, dizziness, restlessness, tremor, fatigue and ataxia.

Figure3(a) shows the results obtained by KnowDDI on DrugBank with different
subgraphs and different percentages of fact triplets from external KG.
Enlarged enclosing subgraphs are provided in Supplementary Figure3. As shown,
leveraging subgraphs consistently leads to performance gain, regardless of the
subgraph type. This shows the necessity of modeling local contexts of target
drug-pairs. Among these subgraphs, learning with knowledge subgraph obtains
the best performance. As random subgraph consists of uniformly sampled nodes
without considering the node importance, the selected nodes may not contribute
to recognize the relationships between head and tail drugs. Enclosing subgraph
keeps the local neighborhood of head and tail drugs intact, thus it does not
lose information. However, directly learning on these subgraphs may lead to
bad performance, if irrelevant edges exist. In contrast, drug-flow subgraph
focuses on relational paths pointing from head drug to tail drug, and
knowledge subgraph further only keeps explaining paths. They all remove
irrelevant nodes which do not appear in any paths. Besides, by comparing the
performance of drug-flow subgraph and knowledge subgraph under different
percentages of fact triplets, we can see that both enriched drug
representations and propagated drug similarities contribute to the performance
improvements. However, the performance gain is larger when fewer fact triplets
are used. This means removing irrelevant edges and propagating drug
similarities play a stronger influence on compensating for the lack of DDIs
when drug representations are less enriched. In summary, learning knowledge
subgraph is effective.

#### Interpretability

As discussed, being able to understand the DDI between drug-pairs helps drug
discovery. Here, we show that KnowDDI can explain why two drugs associate with
each other by leveraging explaining paths in knowledge subgraphs
$\mathcal{S}_{h,t}$.

Figure3 shows the subgraphs of four drug-pairs. Random subgraphs are not
plotted, as they naturally loses semantic information. As can be observed,
drug-flow subgraphs contain fewer nodes in comparison to the enclosing
subgraphs. Particularly, as we take Hetionet as the external KG, where only
drugs have incoming edges with drugs and the relation type is
“Compound–resembles–Compound”, drug-flow subgraphs only contain drugs.
Knowledge subgraphs further adjust the graph structure. In particular, KnowDDI
assigns a connection strength between each node-pair from both direction. It
represents the importance of a given edge between two connected nodes in the
drug-flow subgraph or similarity between two nodes whose interactions are
unknown. Even if two nodes are connected in the $\bar{\mathcal{S}}_{h,t}$,
KnowDDI can delete an existing edge if the estimated connection strength is
too small, such as the edge pointing from 575 to 284 in Figure3(d). Likewise,
two originally disconnected nodes can be connected after learning, if the
estimated connection strength is large. This reveals that KnowDDI thinks the
connected drugs are highly similar and can contribute to explaining the DDI
type between two drugs, such as the edge pointing from 121 to 622 in
Figure3(c). Supplementary Figure6 shows the node-pair whose connection
strength is the largest on each of knowledge graph, including their molecular
graphs and drug efficacy.

Further, Table 2 shows the explaining paths with the largest average
connection strengths assigned by KnowDDI (see Section 5.2.5) for the four
drug-pairs in Figure3. We use Hetionet KG and DrugBank database to help
interpret these explaining paths. As can be seen, these explaining paths
indeed discover reasonable explanations. Moreover, note that in the second
drug-pair, without the newly added edge of relation type “resemble” pointing
from 121 to 622, the discovered explaining path no longer exists. This
validates the necessity of learning with knowledge subgraphs.

Figure 4: T-SNE visualization of drug-pair embeddings. The four graphs share
the same relations and drug-pairs. a, T-SNE of drug-pair embeddings obtained
by SumGNN. b, T-SNE of drug-pair embeddings which are composed of concatenated
generic node embeddings. c, T-SNE of drug-pair embeddings obtained by KnowDDI
(w/o resemble). d, T-SNE of drug-pair embeddings obtained by KnowDDI.

#### Embedding visualization

Finally, we wish to show that KnowDDI helps better shape the embeddings of
drug-pairs and relations to be more predictive of DDI types between target
drug-pairs. From the DDI dataset, we randomly sample ten DDI relations, then
randomly sample fifty fact triplets per relation. Given a drug-pair $(h,t)$,
it is represented as the concatenation of drug embeddings which correspond to
the node embeddings of $h$ and $t$. For methods operating on subgraphs, the
drug-pair embedding is obtained by concatenating drug embeddings with subgraph
embedding which is obtained by mean pooling over node embeddings of all nodes
in the subgraph teru2020inductive ; yu2021sumgnn . In this way, local context
of target drug-pair is leveraged to obtain better prediction results. We
compare KnowDDI with SumGNN and KnowDDI (w/o resemble). In addition, we also
show the drug-pair embeddings obtained by simply learning generic node
embeddings without refining them on subgraphs.

Figure4 shows the t-SNE visualization (VanderMaaten2008, ) obtained on
DrugBank. First, we can see from generic embeddings in Figure4(b), to KnowDDI
(w/o resemble) in Figure4(c), to KnowDDI in Figure4(d), drug-pair with the
same relation type are getting closer while drug-pair with different relation
types are moving farther apart. Also, as can be seen, the clusters are more
obvious in KnowDDI (Figure4(c) and Figure4(d)) than that of SumGNN
(Figure4(a)). This means that learning knowledge subgraphs is beneficial to
obtain more distinctive drug-pair embeddings.

##  4 Discussion

In this study, we are motivated to develop an effective solution to accurately
construct a DDI predictor from the rare DDI fact triplets. The proposed
KnowDDI achieves the goal by taking advantage of rich knowledge in biomedicine
and healthcare and the plasticity of deep learning approaches. In KnowDDI, the
enriched drug representations and propagated drug similarities together
implicitly compensate for the lack of known DDIs. We first combine the
provided DDI graph and an external KG into a combined network, and manage to
encode the rich knowledge recorded in KG into the generic node
representations. Then, we extract a drug-flow subgraph for each drug-pair from
the combined network, and learn a knowledge subgraph from generic
representations and the drug-flow subgraph. During learning, the knowledge
subgraph is optimized, where irrelevant edges are removed and new edges are
added if two disconnected nodes have highly similar node representations.
Finally, the representations of drugs are transformed to be more predictive of
the DDI types between the target drug-pair while knowledge subgraph contains
explaining paths to interpret the prediction result. The performance gap
between KnowDDI and other approaches gets larger for relation types given a
smaller number of known DDI fact triplets, which validates the effectiveness
of KnowDDI.

Due to the popularity of GNN for learning from graphs, existing works have
applied it to solve link prediction problem. Early works, like GAT
velivckovic2018graph  and R-GCN schlichtkrull2018modeling , usually obtain the
representation for each node by running message passing on the whole graph,
then feed the representation of two target nodes into a predictor to estimate
the existence of a link between two target nodes. Decagon zitnik2018modeling ,
SkipGNN huang2020skipgnn , KGNN lin2020kgnn  and DDKG su2022attention
compared in Table 1 also follow this routine. In particular, KGNN uniformly
samples a fixed number of nodes as neighbors of each node during message
passing to reduce the computation overhead. DDKG improves the message passing
part of KGNN by assigning attention weights to the uniformly sampled
neighboring nodes, where the attention weights are obtained by calculating the
similarities between initial node embeddings constructed from SMILES. These
works treat all nodes equally and ignore pair-wise information when
propagating messages. Each drug will get a representation without considering
which drug to interact. Thus, the performance of these methods can be worse
than classical hand-designed heuristics, which count common neighbors or
connected paths between a node-pair zhang2018link . As a result, recent work
GraIL teru2020inductive  proposes a pipeline to learn with subgraphs, i.e.,
first extracting a subgraph containing the two target nodes, then obtaining
the node representation from the subgraph, finally estimating the link between
two target nodes using the node-pair representation which consists of the node
embeddings of two target nodes and the subgraph embedding. KnowDDI, SumGNN
yu2021sumgnn  and LaGAT Hong2022  compared in Table 1 follow this pipeline.
They adopt different strategies to learn with subgraphs. In particular, LaGAT
extracts a subgraph consisting of a fixed number of nodes around the head and
tail drugs, updates node embeddings by aggregating neighboring nodes based on
attention weights calculated using node embeddings, and leaves the subgraph
unchanged. While SumGNN extracts the enclosing subgraph of each drug-pair,
then prunes edges based on the node features. By encoding local context within
subgraphs, these methods obtain node-pair-aware representations, i.e., a drug
can be represented different depending on which drug to interact. Our KnowDDI
also learns with subgraphs, while two major design differences makes it obtain
better and more interpretable results. The first difference is that KnowDDI
learns generic node embeddings on the combined network to enrich the drug
representations, then transforms them on knowledge subgraphs to incorporate
with the local context of drug-pairs. The second difference is the adjustment
of subgraphs where existing edges can be dropped if their estimated importance
are low, and new edges of type “resemble” can be added between disconnected
nodes if their node embeddings are highly similar. This allows KnowDDI to
capture explaining paths pointing from head drug to tail drug. While SumGNN
directly learns drug-pair-aware representations from the extracted subgraphs.
With the two differences, KnowDDI achieves the balance of generic information
and drug-pair-aware local context during learning.

The architecture of KnowDDI can be further improved. For instance, pretraining
GNN from other large datasets which may provide better initialized parameters
and therefore reduce the training time. Besides, we do not use any molecular
features of drugs in order to test the ability of KnowDDI learning solely from
the combination of external KG and DDI fact triplets. Taking these predefined
node features may improve the predictive performance of KnowDDI in the future.
Although we implement KnowDDI to handle DDI prediction in this paper, KnowDDI
is a general approach which can be applied to other relevant applications, to
help detect possible protein–protein interactions, drug–target interactions,
and disease-gene interactions. Relevant practitioners can easily leverage the
rich biomedical knowledge existing in large KGs to obtain good and explainable
prediction results. We believe our open-source KnowDDI can act as an original
algorithm and unique deep learning tool to promote the development of
biomedicine and healthcare. For example, it can help detect possible
interactions of new drugs, accelerating the speed of drug design. Given drug
profiles of patients, KnowDDI can be used to identify possible adverse
reactions. These results have the potential to serve as a valuable resource
for alerting clinicians and healthcare providers when devising management
plans for polypharmacy, as well as for guiding the inclusion criteria of
participants in clinical trials. Beyond biomedicine and healthcare, similar
approaches can be developed to adaptively leverage domain-specific large KGs
to help solve downstream applications in low-data regimes.

##  5 Method

###  5.1 Problem setup

A Drug-Drug Interaction (DDI) graph is denoted as $\mathcal{G}_{\text{DDI}}$
$=$ $\\{\mathcal{V}_{\text{DDI}}$, $\mathcal{E}_{\text{DDI}}$,
$\mathcal{R}_{\text{DDI}}\\}$, where $\mathcal{V}_{\text{DDI}}$ is a set of
drug nodes, $\mathcal{E}_{\text{DDI}}$ is a set of edges, and
$\mathcal{R}_{\text{DDI}}$ is a set of DDI relation types associated with the
edges. In particular, each edge $(u,r,v)\in\mathcal{E}_{\text{DDI}}$
corresponds to an observed fact triplet, which records the DDI relation type
$r\in\mathcal{R}_{\text{DDI}}$ associated with $(u,v)$.

The external Knowledge Graph (KG) is denoted as
$\mathcal{G}_{\text{KG}}=\\{\mathcal{V}_{\text{KG}},\mathcal{E}_{\text{KG}},\mathcal{R}_{\text{KG}}\\}$,
which contains rich biomedical knowledge of various kinds of biomedical
entities. Particularly, $\mathcal{V}_{\text{KG}}$ consists of
$|\mathcal{V}_{\text{KG}}|$ entities ranging from drugs, genes to proteins,
$\mathcal{R}_{\text{KG}}$ consists of $|\mathcal{R}_{\text{KG}}|$ types of
interactions occurred in $\mathcal{G}_{\text{KG}}$, while
$\mathcal{E}_{\text{KG}}=\\{(u,r,v)|u,v\in\mathcal{V}_{\text{KG}},r\in\mathcal{R}_{\text{KG}}\\}$
consists of observed fact triplets in $\mathcal{G}_{\text{KG}}$. Usually,
$\mathcal{G}_{\text{KG}}$ is much larger than $\mathcal{G}_{\text{DDI}}$, such
that $\mathcal{V}_{\text{DDI}}\subseteq\mathcal{V}_{\text{KG}}$ and
$\mathcal{R}_{\text{DDI}}\subseteq\mathcal{R}_{\text{KG}}$ hold.

The combination of $\mathcal{G}_{\text{DDI}}$ and $\mathcal{G}_{\text{KG}}$
then forms a large combined network
$\mathcal{G}=\\{\mathcal{V},\mathcal{E},\mathcal{R}\\}=\\{\mathcal{V}_{\text{DDI}}\cup\mathcal{V}_{\text{KG}}$,
$\mathcal{E}_{\text{DDI}}\cup\mathcal{E}_{\text{KG}}$,
$\mathcal{R}_{\text{DDI}}\cup\mathcal{R}_{\text{KG}}\\}$.

The target of this paper is to learn a mapping function from the combined
network $\mathcal{G}$, which can predict the relation type between new drug-
pairs. For multiclass DDI prediction, each drug-pair only has one specific
relation type $r\in\mathcal{R}_{\text{DDI}}$. As for multilabel DDI
prediction, multiple relation types $r_{1}$, $r_{2}$, $\dots$ $\in$
$\mathcal{R}_{\text{DDI}}$ can co-occur between a drug-pair.

###  5.2 Architecture of KnowDDI

The overall architecture of KnowDDI is shown in Figure1. Here, we provide the
details of how drug representations are enriched by an external KG and how
similarities are propagated in knowledge subgraphs. The complete algorithms of
training and testing KnowDDI are summarized in Supplementary Section E.

####  5.2.1 Generic embedding generation

To encode the generic knowledge of various other type of entities in
$\mathcal{G}$, which can help enrich representation for drug nodes, we run a
GNN on the combined network $\mathcal{G}$ to obtain generic embedding of each
node.

Let $\mathbf{e}^{(0)}_{v}$ denote the feature of node $v\in\mathcal{V}$. In
KnowDDI, we follow GraphSAGE hamilton2017inductive  and update the embedding
$\mathbf{e}_{v}^{(l)}$ for node $v$ at the $l$th layer as:

$\displaystyle\\!\\!\\!\\!{\mathbf{a}}_{v}^{(l)}$
$\displaystyle\\!=\\!\text{MEAN}\big{(}\\{\text{ReLU}(\mathbf{W}^{(l)}_{a}\mathbf{e}_{u}^{(l)})\\!:\\!(u,r,v)\\!\in\\!\mathcal{E}\\}\big{)},\\!\\!$
(1) $\displaystyle\\!\\!\\!\\!\mathbf{e}_{v}^{(l)}$
$\displaystyle\\!=\\!\mathbf{W}^{(l)}_{c}\\!\cdot\\!\big{[}\mathbf{a}_{v}^{(l)}\parallel\mathbf{e}_{v}^{(l-1)}\big{]},$
(2)

where $\text{MEAN}(\cdot)$ is element-wise mean pooling,
$\mathbf{W}^{(l)}_{a}$ and $\mathbf{W}^{(l)}_{c}$ are learnable parameters,
and $\left[\cdot\\!\\!\parallel\\!\\!\cdot\right]$ concatenates vectors along
the last dimension. After $L$ layers of message passing, the final node
embedding ${\mathbf{e}}_{v}^{(L)}$ is taken as the generic embedding of
$v\in\mathcal{V}$.

####  5.2.2 Drug-flow subgraph construction

As each drug-pair can depend on different local contexts, i.e., entities and
relations, we construct a drug-flow subgraph $\bar{\mathcal{S}}_{h,t}$
specific to $(h,t)$ from the combined network $\mathcal{G}$ , which transforms
drug representations obtained in Section 5.2.1 to be drug-pair-aware.

For each $(h,t)$, we define its drug-flow subgraph
$\bar{\mathcal{S}}_{h,t}=\\{\bar{\mathcal{V}}_{h,t},\bar{\mathcal{E}}_{h,t},\bar{\mathcal{R}}_{h,t}\\}$
as a directed subgraph of graph $\mathcal{G}$ consisting of relational paths
$\\{\bar{\mathbf{p}}_{h,t}\\}$ with length at most $P$ pointing from $h$ to
$t$ in $\mathcal{G}$, where a relational path

$\displaystyle\bar{\mathbf{p}}_{h,t}=h\overset{r_{1}}{\rightarrow}v_{1}\overset{r_{2}}{\rightarrow}v_{2}\cdots\overset{r_{P}}{\rightarrow}t,$

is a sequence of nodes connected by relations. Here, $\bar{\mathcal{V}}_{h,t}$
is a set of nodes appearing in $\bar{\mathcal{S}}_{h,t}$,
$\bar{\mathcal{R}}_{h,t}$ is a set of relation types occurred between nodes in
$\bar{\mathcal{V}}_{h,t}$ and
$\bar{\mathcal{E}}_{h,t}=\\{(u,r,v)|u,v\in\mathcal{V}_{h,t}~{}\text{and}~{}(u,r,v)\in\mathcal{E}\\}$
is a set of edges connecting nodes in $\bar{\mathcal{V}}_{h,t}$.

The procedure of extracting $\bar{\mathcal{S}}_{h,t}$ is summarized in
Supplementary Algorithm 1. Given a drug-pair $(h,t)$, we first extract the
interaction of local neighborhoods of $h$ and $t$ (i.e. $K$-hop enclosing
subgraph teru2020inductive , where $K$ is a hyperparameter) from
$\mathcal{G}$. For computational simplicity, we make all the relational paths
$\\{\bar{\mathbf{p}}_{h,t}\\}$ between $h$ and $t$ have length $P$. This is
done by augmenting relational paths with length less than $P$ by identity
relations vashishth2020composition ; sadeghian2019drum , i.e.,
$(t,r_{\text{identity}},t)$. If there is no relational path connecting $h$ and
$t$, we return $\bar{\mathcal{S}}_{h,t}=\\{\\{h,t\\},\emptyset,\emptyset\\}$.
As these $K$-hop enclosing subgraphs neglect directional information, we need
to conduct directional pruning to remove all nodes and corresponding edges
which are not on any relational path pointing from $h$ to $t$. Thus, after
directional pruning, the resultant drug-flow subgraph
$\bar{\mathcal{S}}_{h,t}$ only contains nodes which supports learning the
information flow from $h$ to $t$.

####  5.2.3 Knowledge subgraph generation

Further, we learn a knowledge subgraph ${\mathcal{S}}_{h,t}$ from generic
embeddings and the drug-flow subgraph $\bar{\mathcal{S}}_{h,t}$. During this
process, irrelevant edges are removed and new edges of type “resemble” are
added between nodes with highly similar node embeddings.

Let $\bar{\mathbf{A}}_{h,t}$ be a binary third-order tensor with size
$|\bar{\mathcal{V}}_{h,t}|\times|\bar{\mathcal{V}}_{h,t}|\times|\bar{\mathcal{R}}_{h,t}|$.
Its $(u,v,r)$th entry is computed as

$\displaystyle\bar{\mathbf{A}}_{h,t}(u,v,r)=\begin{cases}1&\text{if\;}(u,r,v)\in\bar{\mathcal{E}}_{h,t}\\\
0&\text{otherwise}\end{cases},$

which records whether drug-pair $(u,v)$ is connected by relation type
$r\in\bar{\mathcal{R}}_{h,t}$ in $\bar{\mathcal{S}}_{h,t}$.

In addition, we estimate another third-order tensor $\mathbf{A}_{h,t}$ from
$\bar{\mathbf{A}}_{h,t}$ with elements $\mathbf{A}_{h,t}(u,v,r)\in[0,1]$ and
size
$|\bar{\mathcal{V}}_{h,t}|\times|\bar{\mathcal{V}}_{h,t}|\times(|\bar{\mathcal{R}}_{h,t}|+1)$
to record the connection strength between nodes
$u,v\in\bar{\mathcal{V}}_{h,t}$ w.r.t. relation $r$. Specifically, if
$\bar{\mathbf{A}}_{h,t}(u,v,r)=1$ but $\mathbf{A}_{h,t}(u,v,r)=0$, this means
the existing edge $(u,r,v)$ is not useful and should be removed. Besides, if
$\bar{\mathbf{A}}_{h,t}(u,v,r)=0$ for all $r\in\bar{\mathcal{R}}_{h,t}$, we
add an edge of relation type “resemble” $r_{\text{sim}}\in\mathcal{R}$ to
connect $u$ and $v$. $\mathbf{A}_{h,t}(u,v,r_{\text{sim}})>0$ then represents
the similarity between $u$ and $v$. Corresponding to $\mathbf{A}_{h,t}$ and
$\bar{\mathcal{S}}_{h,t}$, the knowledge subgraph $\mathcal{S}_{h,t}$ is
generated as

$\displaystyle\mathcal{S}_{h,t}=\left\\{\bar{\mathcal{V}}_{h,t},\mathcal{E}_{h,t},\mathcal{R}_{h,t}\right\\},$

where $\mathcal{R}_{h,t}=\\{r_{\text{sim}}\\}\cup\bar{\mathcal{R}}_{h,t}$, and
$\mathcal{E}_{h,t}=\\{(u,r,v)\\}$ with each $(u,r,v)$ constructed as
$(u,r,v)\in\bar{\mathcal{E}}_{h,t}$ if
$\bar{\mathbf{A}}_{h,t}(u,v,r)=1\land\mathbf{A}_{h,t}(u,v,r)>0$, or
$(u,r_{\text{sim}},v)$ if
$\bar{\mathbf{A}}_{h,t}(u,v,r)=0\land\mathbf{A}_{h,t}(u,v,r_{\text{sim}})>0$.

To learn such a $\mathbf{A}_{h,t}$, we conduct graph structure learning to
alternate the following two steps for $T$ times:

  * •

estimate connection strengths between every pair of nodes in
$\bar{\mathcal{V}}_{h,t}$, and

  * •

refine node embeddings on the updated subgraph.

First, we initialize the node embedding of each $v\in\bar{\mathcal{V}}_{h,t}$
as $\mathbf{h}_{u}^{(0)}=\mathbf{e}_{u}^{(L)}$ to encode the global topology
of $\mathcal{G}$. Let $\mathbf{A}^{(\tau)}_{h,t}$ be the estimation of
$\mathbf{A}_{h,t}$ at the $\tau$th iteration. We initialize
$\mathbf{A}^{(0)}_{h,t}=\bar{\mathbf{A}}_{h,t}$. Next, we estimate relevance
score $\mathbf{C}_{h,t}^{(\tau)}(u,v,r)$ for each relation
$r\in\mathcal{R}_{h,t}$ between every node-pair $(u,v)$ in
${\mathcal{S}}_{h,t}$ as

$\\!\\!\\!\mathbf{C}_{h,t}^{(\tau)}(u,v,r)\\!=\\!\text{MLP}\Big{(}\left[\mathbf{h}_{uv}^{\tau-1}\parallel\mathbf{h}_{r}\right]\Big{)},\\!$
(3)

where
$\mathbf{h}_{uv}^{\tau-1}=\exp(-|\mathbf{h}_{u}^{(\tau\\!-\\!1)}-\mathbf{h}_{v}^{(\tau-1)}|)$,
$|\cdot|$ returns the element-wise absolute value, MLP is multi-layer
perception, and $\mathbf{h}_{r}$ is the learnable relation embedding of
relation $r$. We set $\mathbf{C}_{h,t}^{(\tau)}(v,v,r)=1$ for all
$v\in\bar{\mathcal{V}}_{h,t}$ and $r\in\mathcal{R}_{h,t}$.

This learned $\mathbf{C}_{h,t}^{(\tau)}$ reveals how the model understands the
connection between different node-pairs at the $\tau$th iteration. It helps
filter out irrelevant information and captures resembling relationships
between drugs whose interactions are unknown. In the early stage of
optimization, $\mathbf{C}_{h,t}^{(\tau)}$ can be less trustworthy. Hence, we
merge this learned subgraph with $\bar{\mathbf{A}}_{h,t}$ to obtain
${\mathbf{A}}_{h,t}^{(\tau)}$, i.e.,

${\mathbf{A}}_{h,t}^{(\tau)}\\!=\\!\text{ReLU}\big{(}\delta\big{(}\alpha\mathbf{A}^{(0)}_{h,t}\\!+\\!(1\\!-\\!\alpha)\mathbf{C}_{h,t}^{(\tau)}\big{)}\\!-\\!\gamma\big{)},$
(4)

where hyperparameter $\alpha$ is used to balance their contribution in the
final prediction, the threshold $\gamma\geq 0$ is used to screen out those
less informative edges, and $[\text{ReLU}(x)]=\max(x,0)$. Considering that
nodes are connected by different numbers of neighbors, we use function
$\delta(\cdot)$ to ensure that the relevance scores of incoming edges for $v$
sum into 1, i.e.,

$\displaystyle\sum\nolimits_{i\in\bar{\mathcal{V}}_{h,t}}\sum\nolimits_{r\in\mathcal{R}_{h,t}}\mathbf{A}_{h,t}^{(\tau)}(i,v,r)=1.$

Here, we instantiate $\delta(\cdot)$ as edge softmax function which computes
softmax over attention weights of incoming edges regardless of their relation
types for every node, i.e.,
$\text{softmax}(x_{i})=\exp(x_{i})/\sum_{j\in\mathcal{N}(i)}\exp(x_{j})$ where
$\mathcal{N}(i)=\\{j:(j,r,i)\in\mathcal{E}_{h,t}\\}$. Let
$\mathbf{H}^{(\tau)}_{h,t}$ be embeddings of all nodes in
$\bar{\mathcal{V}}_{h,t}$ where the $v$th row corresponds to node embedding
$\mathbf{h}^{(\tau)}_{v}$ of $v\in\bar{\mathcal{V}}_{h,t}$, and

$\displaystyle\mathbf{Q}^{(\tau)}_{h,t,r}$
$\displaystyle=\text{ReLU}\left({\mathbf{A}}^{(\tau)}_{h,t}(:,:,r)\mathbf{H}^{(\tau-1)}_{h,t}\mathbf{W}_{r}\right),$

where ${\mathbf{A}}^{(\tau)}_{h,t}(:,:,r)$ is the $r$th slice of
${\mathbf{A}}^{(\tau)}_{h,t}$ and $\mathbf{W}_{r}$ is a learnable parameter.
Then, $\mathbf{H}^{(\tau)}_{h,t}$ is updated as

$\displaystyle\mathbf{H}^{(\tau)}_{h,t}$
$\displaystyle=\text{MEAN}(\\{\mathbf{Q}^{(\tau)}_{h,t,r}:r\in\mathcal{R}_{h,t}\\}).$
(5)

After $T$ iterations, the representations of drugs are transformed to be more
predictive of the DDI types between the target drug-pair, and
$\mathcal{S}_{h,t}$ only keeps edges of important known DDIs or newly-added
edges connecting highly similar drugs. We set
${\mathbf{h}}_{v}={\mathbf{h}}_{v}^{(T)}$ as the final node embedding, and
return $\mathbf{A}_{h,t}=\mathbf{A}^{(T)}_{h,t}$ which records the updated
graph structure of $\mathcal{S}_{h,t}$. Learning subgraph embedding of
$\mathcal{S}_{h,t}$ is commonly adopted to encode the subgraph topology
hamilton2017inductive ; teru2020inductive ; yu2021sumgnn . Hence, we follow
this routine and obtain the subgraph embedding
${\mathbf{h}}_{\mathcal{S}_{h,t}}$ of $\mathcal{S}_{h,t}$ as

$\displaystyle{\mathbf{h}}_{\mathcal{S}_{h,t}}=\text{MEAN}\Big{(}\\{\mathbf{h}_{v}|v\in\bar{\mathcal{V}}_{h,t}\\}\Big{)}.$
(6)

Finally, we predict the relation for $(h,t)$ as

$\displaystyle\hat{\mathbf{y}}_{h,t}=\delta\Big{(}\mathbf{W}_{c}\cdot\left[{\mathbf{h}}_{\mathcal{S}_{h,t}}\parallel\mathbf{h}_{h}\parallel\mathbf{h}_{t}\right]\Big{)},$
(7)

where $\mathbf{W}_{c}$ is the classifier parameter.

More choices of knowledge subgraph generation strategy and their empirical
effect are analyzed in Supplementary Figure4.

####  5.2.4 Learning and inference

Let $\bm{\theta}_{g}$ and $\bm{\theta}_{k}$ denote the collection of
parameters associated with generic embedding generation and knowledge subgraph
generation respectively. Further, let $\mathbf{y}_{h,t}=[\mathbf{y}_{h,t}(i)]$
be a vector where the $i$th element $\mathbf{y}_{h,t}(i)=1$ if relation
$i\in\mathcal{R}_{\text{DDI}}$ occurs in $(h,t)$ and 0 otherwise.

For multiclass DDI prediction, we optimize KnowDDI w.r.t. the cross entropy
loss:

$\displaystyle\min_{\bm{\theta}_{g},\bm{\theta}_{k}}\ell_{\text{CE}}\equiv\sum\nolimits_{(h,r,t)\in\mathcal{E}_{\text{DDI}}}-\mathbf{y}_{h,t}^{\top}\cdot\log\left(\hat{\mathbf{y}}_{h,t}\right).$
(8)

As for multilabel DDI prediction, drug-pairs are associated with varying
number of relations. We further use a loss function with negative sampling.
Following related works teru2020inductive ; yu2021sumgnn , we construct
negative triplets to prevent KnowDDI from selecting those unknown relations.
For each $(h,r,t)\in\mathcal{E}_{\text{DDI}}$, we replace $t$ by a randomly
sampled drug $w\in\mathcal{V}_{\text{DDI}}$ to form $(h,r,w)$ whose label
vector $\mathbf{y}_{h,w}=[0,\dots,0]$ contains zeros only. Let
$\mathcal{E}_{\text{neg}}=\\{(h,r,w)|(h,r,t)\in\mathcal{E}_{\text{DDI}}~{}\text{and}~{}(h,r,w)\notin\mathcal{E}_{\text{DDI}}\\}$
collectively contains the negative triplets. We optimize KnowDDI w.r.t. the
following loss for multilabel DDI prediction:

$\min_{\bm{\theta}_{g},\bm{\theta}_{k}}\ell_{\text{CE}}+\sum\nolimits_{(h,r,w)\in\mathcal{E}_{\text{neg}}}\\!\\!\\!\\!-\mathbf{1}^{\top}\\!\cdot\log\left(\mathbf{1}-\hat{\mathbf{y}}_{h,w}\right),$
(9)

where $\mathbf{1}$ is a vector of all 1s. Note that equation (9) only
penalizes wrong prediction of known relations between drug-pairs. In other
words, for triplets that are not observed in $\mathcal{E}_{\text{DDI}}$, we
regard them as unknown.

During inference, given a new drug-pair $(h^{\prime},t^{\prime})$ where
$h^{\prime},t^{\prime}\in\mathcal{V}$, we directly use KnowDDI with optimized
$\bm{\theta}_{g}$, $\bm{\theta}_{k}$ to obtain the class prediction vector
$\hat{\mathbf{y}}_{h^{\prime},t^{\prime}}$. For multiclass prediction, the
class is predicted as the relation which obtains the highest possibility in
$\hat{\mathbf{y}}_{h^{\prime},t^{\prime}}$. As for multilabel prediction, the
complete $\hat{\mathbf{y}}_{h^{\prime},t^{\prime}}$ is returned. Please refer
to Supplementary Algorithm 3 for details.

####  5.2.5 Identifying explaining paths

To explain the predicted DDI for $(h,t)$, we take out the explaining paths
from $\mathcal{S}_{h,t}$. In particular, an explaining path

$\displaystyle\mathbf{p}_{h,t}=h\xrightarrow{\mathbf{A}_{h,t}(h,v_{1},r_{1})}$
$\displaystyle v_{1}\xrightarrow{\mathbf{A}_{h,t}(v_{1},v_{2},r_{2})}v_{2}$
$\displaystyle\cdots\xrightarrow{\mathbf{A}_{h,t}(v_{P-1},t,r_{P})}t,$

is a sequence of nodes, where node $v_{i}$ and node $v_{i+1}$ are connected by
relation $r_{i+1}\in\mathcal{R}$ with a connection strength indicated by
$\mathbf{A}_{h,t}(v_{i},v_{j},r_{j})$. We then obtain the average connection
strength of $\mathbf{p}_{h,t}$ by averaging over
$\mathbf{A}_{h,t}(v_{i},v_{j},r_{j})$ of consecutive pairs of nodes in
$\mathbf{p}_{h,t}$. This average connection strength reflects the ability of
the explaining path to interpret the prediction result from the perspective of
KnowDDI.

###  5.3 Training details

In KnowDDI, we use a two-layer GraphSAGE hamilton2017inductive  to obtain the
generic node embedding $\mathbf{e}_{v}^{(l)}$ whose dimension is set as 32.
For drug-flow subgraph extraction, we extract 2-hop neighborhood and then
extract relational paths with length at most 4 pointing from $h$ to $t$. The
dimension of edge embedding $\mathbf{h}_{r}$ in equation (3) is set as 32. We
alternate between estimating connection strengths and refining node embeddings
for 3 times ($T$ in Algorithm 2). We select $\gamma$ in equation (4) from
$[0.05,0.2]$ and $\alpha$ in equation (5) from $[0.3,0.7]$. We train the model
for a maximum number of 50 epochs using Adam kingma2015adam  with learning
rate $5*10^{-3}$ and weight decay rate $10^{-5}$. We early stop training if
the validation loss does not decrease for 10 consecutive epochs. We set
dropout rate as 0.2 and batch size as 256. All results are averaged over five
runs and are obtained on a 32GB NVIDIA Tesla V100 GPU. A summary of
hyperparameters used by KnowDDI is provided in Supplementary Table 4. In
addition, sensitivity analysis of key hyperparameters is provided in
Supplementary Figure5.

## Data availability

The data used in this study, including the two benchmark DDI datasets and
external KG Hetionet, is publicly available. DrugBank dataset
wishart2018drugbank  can be downloaded from
https://bitbucket.org/kaistsystemsbiology/deepddi/src/master/data/. TWOSIDES
dataset tatonetti2012data  can be downloaded from
https://tatonettilab.org/resources/nsides/. Hetionet is obtained from
https://het.io. The processed data analyzed in this paper is available in
GitHub repository at https://github.com/LARS-research/KnowDDI/raw_data.
Combined networks which separately merges the two DDI graph with external KG
are provided at https://github.com/LARS-research/KnowDDI/data.

## Code availability

The code implementing KnowDDI is deposited in public available GitHub
repository at https://github.com/LARS-research/KnowDDI.

## Author contributions

For this manuscript, Y. W. contributes to the idea development, experiment
design, paper presentation and writing; Z. Y. contributes to the code
implementations, obtaining and analysis results; Q. Y. contributes to the idea
development, result analysis, paper presentation and writing.

## References

  *   * [1] Juurlink, D. N., Mamdani, M., Kopp, A., Laupacis, A. & Redelmeier, D. A.  Drug-drug interactions among elderly patients hospitalized for drug toxicity.  _JAMA_ 289 (13), 1652–1658 (2003) .
  * [2] Bangalore, S., Kamalakkannan, G., Parkar, S. & Messerli, F. H.  Fixed-dose combinations improve medication compliance: A meta-analysis.  _The American Journal of Medicine_ 120 (8), 713–719 (2007) .
  * [3] Scavone, C. _et al._ Current pharmacological treatments for COVID-19: What’s next?  _British Journal of Pharmacology_ 177 (21), 4813–4824 (2020) .
  * [4] Chakraborty, C., Sharma, A. R., Bhattacharya, M., Agoramoorthy, G. & Lee, S.-S.  The drug repurposing for COVID-19 clinical trials provide very effective therapeutic combinations: Lessons learned from major clinical studies.  _Frontiers in Pharmacology_ 12 (2021) .
  * [5] Akinbolade, S. _et al._ Combination therapies for COVID-19: An overview of the clinical trials landscape.  _British Journal of Clinical Pharmacology_ 88 (4), 1590–1597 (2022) .
  * [6] Percha, B. & Altman, R. B.  Informatics confronts drug–drug interactions.  _Trends in Pharmacological Sciences_ 34 (3), 178–184 (2013) .
  * [7] Letinier, L. _et al._ Risk of drug-drug interactions in out-hospital drug dispensings in france: Results from the drug-drug interaction prevalence study.  _Frontiers in Pharmacology_ 10, 265 (2019) .
  * [8] Jiang, H. _et al._ Adverse drug reactions and correlations with drug–drug interactions: A retrospective study of reports from 2011 to 2020.  _Frontiers in Pharmacology_ 13 (2022) .
  * [9] Marijon, E. _et al._ Causes of death and influencing factors in patients with atrial fibrillation: A competing-risk analysis from the randomized evaluation of long-term anticoagulant therapy study.  _Circulation_ 128 (20), 2192–2201 (2013) .
  * [10] Zitnik, M., Agrawal, M. & Leskovec, J.  Modeling polypharmacy side effects with graph convolutional networks.  _Bioinformatics_ 34 (13), i457–i466 (2018) .
  * [11] Huang, K., Xiao, C., Glass, L. M., Zitnik, M. & Sun, J.  SkipGNN: Predicting molecular interactions with skip-graph networks.  _Scientific Reports_ 10 (1), 1–16 (2020) .
  * [12] Derry, S., Kong Loke, Y. & Aronson, J. K.  Incomplete evidence: The inadequacy of databases in tracing published adverse drug reactions in clinical trials.  _BMC Medical Research Methodology_ 1 (1), 1–6 (2001) .
  * [13] Wishart, D. S. _et al._ DrugBank 5.0: A major update to the DrugBank database for 2018.  _Nucleic Acids Research_ 46 (D1), D1074–D1082 (2018) .
  * [14] Perozzi, B., Al-Rfou, R. & Skiena, S.  Macskassy, S. A., Perlich, C., Leskovec, J., Wang, W. & Ghani, R. (eds) _DeepWalk: Online learning of social representations_.  (eds Macskassy, S. A., Perlich, C., Leskovec, J., Wang, W. & Ghani, R.) _The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , 701–710 (2014).
  * [15] Yao, J., Sun, W., Jian, Z., Wu, Q. & Wang, X.  Effective knowledge graph embeddings based on multidirectional semantics relations for polypharmacy side effects prediction.  _Bioinformatics_ 38 (8), 2315–2322 (2022) .
  * [16] Bonner, S. _et al._ A review of biomedical datasets relating to drug discovery: A knowledge graph perspective.  _arXiv preprint arXiv:2102.10062_ (2021) .
  * [17] Himmelstein, D. S. & Baranzini, S. E.  Heterogeneous network edge prediction: A data integration approach to prioritize disease-associated genes.  _PLoS Computational Biology_ 11 (7), e1004259 (2015) .
  * [18] Zheng, S. _et al._ PharmKG: A dedicated knowledge graph benchmark for bomedical data mining.  _Briefings in Bioinformatics_ 22 (4), bbaa344 (2021) .
  * [19] Chandak, P., Huang, K. & Zitnik, M.  Building a knowledge graph to enable precision medicine.  _Scientific Data_ 10 (1), 67 (2023) .
  * [20] Lin, X., Quan, Z., Wang, Z.-J., Ma, T. & Zeng, X.  Bessiere, C. (ed.) _KGNN: Knowledge graph neural network for drug-drug interaction prediction._ (ed.Bessiere, C.) _International Joint Conference on Artificial Intelligence_ , Vol. 380, 2739–2745 (2020).
  * [21] Teru, K., Denis, E. & Hamilton, W.  III, H. D. & Singh, A. (eds) _Inductive relation prediction by subgraph reasoning_.  (eds III, H. D. & Singh, A.) _International Conference on Machine Learning_ , 9448–9457 (2020).
  * [22] Yu, Y. _et al._ SumGNN: Multi-typed drug interaction prediction via efficient knowledge graph summarization.  _Bioinformatics_ 37 (18), 2988–2995 (2021) .
  * [23] Hong, Y., Luo, P., Jin, S. & Liu, X.  LaGAT: link-aware graph attention network for drug–drug interaction prediction.  _Bioinformatics_ 38 (24), 5406–5412 (2022) .
  * [24] Tatonetti, N. P., Patrick, P. Y., Daneshjou, R. & Altman, R. B.  Data-driven prediction of drug effects and interactions.  _Science Translational Medicine_ 4 (125), 125ra31 (2012) .
  * [25] Hu, W. _et al._ Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. & Lin, H. (eds) _Open graph benchmark: Datasets for machine learning on graphs_.  (eds Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. & Lin, H.) _Advances in Neural Information Processing Systems_ , Vol. 33, 22118–22133 (2020).
  * [26] Breit, A., Ott, S., Agibetov, A. & Samwald, M.  OpenBioLink: A benchmarking framework for large-scale biomedical link prediction.  _Bioinformatics_ 36 (13), 4097–4098 (2020) .
  * [27] Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J. & Yakhnenko, O.  _Translating embeddings for modeling multi-relational data_ , 2787–2795 (2013).
  * [28] Karim, M. R. _et al._ Shi, X. M., Buck, M., Ma, J. & Veltri, P. (eds) _Drug-drug interaction prediction based on knowledge graph embeddings and convolutional-LSTM network_.  (eds Shi, X. M., Buck, M., Ma, J. & Veltri, P.) _ACM International Conference on Bioinformatics, Computational Biology and Health Informatics_ , 113–123 (2019).
  * [29] Grover, A. & Leskovec, J.  Krishnapuram, B. _et al._ (eds) _node2vec: Scalable feature learning for networks_.  (eds Krishnapuram, B. _et al._) _ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , 855–864 (2016).
  * [30] Tang, J. _et al._ Gangemi, A., Leonardi, S. & Panconesi, A. (eds) _LINE: Large-scale information network embedding_.  (eds Gangemi, A., Leonardi, S. & Panconesi, A.) _International Conference on World Wide Web_ , 1067–1077 (2015).
  * [31] Kipf, T. & Welling, M.  Semi-supervised classification with graph convolutional networks (2017).  Paper presented at the 5th International Conference on Learning Representations.
  * [32] Veličković, P. _et al._ Graph attention networks (2018).  Paper presented at the 6th International Conference on Learning Representations.
  * [33] Su, X., Hu, L., You, Z., Hu, P. & Zhao, B.  Attention-based knowledge graph representation learning for predicting drug-drug interactions.  _Briefings in Bioinformatics_ 23 (3) (2022) .
  * [34] Van der Maaten, L. & Hinton, G.  Visualizing data using t-SNE.  _Journal of Machine Learning Research_ 9 (11) (2008) .
  * [35] Schlichtkrull, M. _et al._ Gangemi, A. _et al._ (eds) _Modeling relational data with graph convolutional networks_.  (eds Gangemi, A. _et al._) _The Semantic Web: 15th International Conference_ , 593–607 (2018).
  * [36] Zhang, M. & Chen, Y.  Bengio, S. _et al._ (eds) _Link prediction based on graph neural networks_.  (eds Bengio, S. _et al._) _Advances in Neural Information Processing Systems_ , 5171–5181 (2018).
  * [37] Hamilton, W., Ying, Z. & Leskovec, J.  Guyon, I. _et al._ (eds) _Inductive representation learning on large graphs_.  (eds Guyon, I. _et al._) _Advances in Neural Information Processing Systems_ , 1024–1034 (2017).
  * [38] Vashishth, S., Sanyal, S., Nitin, V. & Talukdar, P.  Composition-based multi-relational graph convolutional networks (2020).  Paper presented at the 8th International Conference on Learning Representations.
  * [39] Sadeghian, A., Armandpour, M., Ding, P. & Wang, D.  Wallach, H. M. _et al._ (eds) _DRUM: End-to-end differentiable rule mining on knowledge graphs_.  (eds Wallach, H. M. _et al._) _Advances in Neural Information Processing Systems_ , 15347–15357 (2019).
  * [40] Kingma, D. P. & Ba, J.  Adam: A method for stochastic optimization (2015).  Paper presented at the 3rd International Conference on Learning Representations.

##  Appendix A Summary of characteristics comparing KnowDDI with existing
works

Supplementary Table 1 shows a summary of characteristics comparing KnowDDI
with existing works on solving DDI prediction tasks.

Table 1: Comparing the proposed KnowDDI with existing works.  Method |  External KG |  Drug-pair-aware Representation |  Adding New Edges |  Removing Irrelevant Edges |  Interpretable Paths
---|---|---|---|---|---
GAT [32] |  ✗ |  ✗ |  ✗ |  ✗ |  ✗
Decagon [10] |  ✗ |  ✗ |  ✗ |  ✗ |  ✗
SkipGNN [11] |  ✗ |  ✗ |  ✗ |  ✗ |  ✗
GraIL [21] |  ✓ |  ✗ |  ✗ |  ✗ |  ✗
KGNN [20] |  ✓ |  ✗ |  ✗ |  ✗ |  ✗
DDKG [33] |  ✓ |  ✗ |  ✗ |  ✗ |  ✗
SumGNN [22] |  ✓ |  ✓ |  ✗ |  ✓ |  ✓
LaGAT [23] |  ✓ |  ✓ |  ✗ |  ✗ |  ✓
KnowDDI |  ✓ |  ✓ |  ✓ |  ✓ |  ✓

##  Appendix B Baselines and their codes

For DDI models compared in the paper, we use the public codes of the
respective authors if they are available. TransE [27] is implemented by us.
The links to download the codes are provided in Supplementary Table 2 below.

Table 2: Baselines with URLs to download the codes provided by the respective authors. Baseline | URL
---|---
KG-DDI [28] | https://github.com/rezacsedu/Drug-Drug-Interaction-Prediction
MSTE [15] | https://github.com/galaxysunwen/MSTE-master
node2vec [29] | https://github.com/shenweichen/GraphEmbedding
DeepWalk [14] | https://github.com/phanein/deepwalk
LINE [30] | https://github.com/tangjianpku/LINE
GAT [32] | https://github.com/PetarV-/GAT
Decagon [10] | https://github.com/mims-harvard/decagon
SkipGNN [11] | https://github.com/kexinhuang12345/SkipGNN
GraIL [21] | https://github.com/kkteru/grail
KGNN [20] | https://github.com/xzenglab/KGNN
DDKG [33] | https://github.com/Blair1213/DDKG
SumGNN [22] | https://github.com/yueyu1030/SumGNN
LaGAT [23] | https://github.com/Azra3lzz/LaGAT

##  Appendix C Relation-wise performance improvement

Here, we first report the relation frequency. Supplementary Figure 1(a) and
Supplementary Figure 1(b) show statistics of relation frequency for DrugBank
and TWOSIDES. As can be seen, relations in DrugBank follow long-tail
distribution, while relations in TWOSIDES are associated with more comparable
fact triples. Recall that TWOSIDES used in this study contains 200 commonly
occurring relations with more than 900 fact triplets following SumGNN [22].
This explains why it is easier to predict DDI on TWOSIDES, as can be seen in
Table 1. To validate the general effectiveness of KnowDDI, we use the original
TWOSIDES [24] which contains 4649441 fact triplets for 645 drugs and 1317
relations, and remove relations with less than 3 fact triplets. Afterwards, we
obtain a new dataset denoted as TWOSIDES-Large, which contains 4649430 fact
triplets for 645 drugs and 1311 relations. We preprocess it as described in
Data preprocessing section. Supplementary Figure 1(c) shows statistics of
relation frequency for TWOSIDES-Large. As shown, relations in TWOSIDES-Large
contain varying number of fact triplets.

Figure 1: Statistics of relation frequency for DrugBank (a), TWOSIDES (b), and
TWOSIDES-Large (c). Each bin represents the performance improvement of one
relation. From leftside to rightside of x-axis, relations are ordered by
descreasing number of associated fact triplets.

Next, we provide relation-wise performance improvement of our KnowDDI over
SumGNN which obtains the second-best performance in Table 1. Supplementary
Figure2 shows the results obtained on DrugBank, TWOSIDES, and TWOSIDES-Large.
As can be seen, KnowDDI obtains better performance than SumGNN on most
relations, including those rare relations. As expected, KnowDDI demonstrates a
larger performance gain over SumGNN on TWOSIDES-Large, which contains both
sample-sufficient and rare relations, compared to TWOSIDES, which is more
balanced in its composition.

Figure 2:  Performance improvement of KnowDDI over SumGNN (the second-best
method) on DrugBank (a), TWOSIDES (b), and TWOSIDES-Large (c). Each bin
represents the performance improvement of one relation. From leftside to
rightside of x-axis, relations are ordered by descreasing number of associated
fact triplets. A higher (lower) bin suggests a larger performance improvement
(decrease) for KnowDDI compared to SumGNN.

Finally, Supplementary Table 3 shows the performance of KnowDDI and the
second-best method SumGNN obtained on some important and commonly studied
adverse drug reactions (ADRs) which exist in TWOSIDES-Large. Results show that
KnowDDI consistently performs better on these relations.

Table 3: Performance comparison between KnowDDI and SumGNN on the
representative ADRs from the complete TWOSIDES dataset. The metric is AUROC
(%).

Adverse Drug Reaction | Acute Liver Failure | Acute Myocardial Infarction | Acute Renal Failure | Upper GI Bleeding
---|---|---|---|---
KnowDDI | 92.23 | 93.15 | 90.74 | 94.78
SumGNN | 90.83 | 91.95 | 88.90 | 92.87
Improvement | 1.40 | 1.20 | 1.84 | 1.91

In conclusion, KnowDDI not only excels in terms of overall metrics but also
adeptly manages most relations, including those commonly studied. With our
open-source approach, we anticipate that KnowDDI can contribute to predicting
DDIs for other intriguing relations as well.

##  Appendix D Enlarged enclosing subgraphs in Figure3

Supplementary Figure3 below provides the enlarged enclosing subgraphs of the
four drug-pairs plotted in Figure3(b-e).

Figure 3: Enlarged enclosing subgraphs plotted in Figure3. $\mathbf{b-e}$,
Enlarged version of enclosing subgraphs plotted in the first column of
Figure3(b-e).

##  Appendix E Complete algorithm of KnowDDI

We first provide the procedure of extracting drug-flow subgraph
$\bar{S}_{h,t}$ for drug-pair $(h,t)$ in Algorithm 1. Then, we present the
training and testing procedures of KnowDDI in Algorithm 2 and Algorithm 3
respectively.

Algorithm 1 Drug-flow Subgraph Extraction: DSextract($\mathcal{G},h,t,K,P$).

1:a combined network $\mathcal{G}=\\{\mathcal{V},\mathcal{E},\mathcal{R}\\}$
which combines the DDI graph and an external KG, target drug-pair $(h,t)$,
hyperparameters $K,P$;

2:$\mathcal{V}_{h}^{(0)}=\\{h\\}$,$\mathcal{V}_{h\rightarrow
t}^{(P)}=\\{t\\}$; $\ignorespaces\triangleright$ initialize node sets

3:$\mathcal{N}_{K}(h)=\\{s|d(s,h)\leq K,s\in\mathcal{V}\\}$;
$\ignorespaces\triangleright$ extract $K$-hop (undirected) neighborhood of $h$

4:$\mathcal{N}_{K}(t)=\\{s|d(s,t)\leq K,s\in\mathcal{V}\\}$;
$\ignorespaces\triangleright$ extract $K$-hop (undirected) neighborhood of $t$

5:$\bar{\mathcal{E}}_{h,t}=\\{(u,r,v)|u,v\in\mathcal{N}_{K}(h)\cap\mathcal{N}_{K}(t),(u,r,v)\in\mathcal{E}\\}$;
$\ignorespaces\triangleright$ extract facts in the interaction of
$\mathcal{N}_{K}(h)$ and $\mathcal{N}_{K}(t)$

6:$\hat{\mathcal{E}}_{h,t}=\bar{\mathcal{E}}_{h,t}\cup\\{(t,r_{\text{identity}},t)\\}$;
$\ignorespaces\triangleright$ add identity relation $r_{\text{identity}}$ for
$t$ such that all relation paths between $h$ and $t$ will have length $P$

7:for $p=1,2,\dots,P$ do

8:
${\mathcal{E}}_{h}^{(p)}=\\{(u,r,v)\in\hat{\mathcal{E}}_{h,t}|u\in\mathcal{V}_{h}^{(p-1)}\\}$;
$\ignorespaces\triangleright$ record the $p$-hop outgoing edges of $h$

9: $\mathcal{V}_{h}^{(p)}=\\{v|(u,r,v)\in{\mathcal{E}}_{h}^{(p)}\\}$;
$\ignorespaces\triangleright$ record tail entity nodes of edges in
$\hat{\mathcal{E}}_{h}^{(p)}$

10:end for

11:if $\mathcal{V}_{h}^{(P)}\cup\mathcal{V}_{h\rightarrow t}^{(P)}=\emptyset$
return $\mathcal{S}_{h,t}=\\{\\{h,t\\},\emptyset,\emptyset\\}$;

12:for $p=P,P-1\dots,1$ do

13: $\mathcal{E}_{h\rightarrow
t}^{(p)}=\\{(u,r,v)\in{\mathcal{E}}_{h}^{(p)}|v\in\mathcal{V}_{h\rightarrow
t}^{(p)}\\}$; $\ignorespaces\triangleright$ record the $p$-hop incoming edges
of $t$

14: $\mathcal{V}_{h\rightarrow
t}^{(p-1)}=\\{u|(u,r,v)\in{\mathcal{E}}_{h\rightarrow t}^{(p)}\\}$;
$\ignorespaces\triangleright$ record head entity nodes of edges in
$\mathcal{E}_{h\rightarrow t}^{(p)}$

15: $\mathcal{R}_{h\rightarrow
t}^{(p-1)}=\\{r|(u,r,v)\in{\mathcal{E}}_{h\rightarrow t}^{(p)}\\}$;
$\ignorespaces\triangleright$ record relation types of edges in
$\mathcal{E}_{h\rightarrow t}^{(p)}$

16:end for

17:$\bar{\mathcal{V}}_{h,t}=\mathcal{V}_{h\rightarrow
t}^{(1)}\cup\mathcal{V}_{h\rightarrow
t}^{(2)}\cup\cdots\cup\mathcal{V}_{h\rightarrow t}^{(P)}\cup\\{h,t\\}$;
$\ignorespaces\triangleright$ obtain the set of nodes in $\mathcal{S}_{h,t}$

18:$\bar{\mathcal{R}}_{h,t}=\mathcal{R}_{h\rightarrow
t}^{(1)}\cup\mathcal{R}_{h\rightarrow
t}^{(2)}\cup\cdots\cup\mathcal{R}_{h\rightarrow t}^{(P)}$;
$\ignorespaces\triangleright$ obtain the set of relations in
$\mathcal{S}_{h,t}$

19:$\bar{\mathcal{E}}_{h,t}=\mathcal{E}_{h\rightarrow
t}^{(1)}\cup\mathcal{E}_{h\rightarrow
t}^{(2)}\cup\cdots\cup\mathcal{E}_{h\rightarrow t}^{(P)}$;
$\ignorespaces\triangleright$ obtain the set of facts in $\mathcal{S}_{h,t}$

20:return
$\bar{\mathcal{S}}_{h,t}=\\{\bar{\mathcal{V}}_{h,t},\bar{\mathcal{E}}_{h,t},\bar{\mathcal{R}}_{h,t}\\}$
specific to $(h,t)$.

Algorithm 2 Training procedure of KnowDDI.

1:a DDI graph
$\mathcal{G}_{\text{DDI}}\\!=\\!\\{\mathcal{V}_{\text{DDI}},\mathcal{E}_{\text{DDI}},\mathcal{R}_{\text{DDI}}\\}$
which contains the known DDI fact triplets, an external KG
$\mathcal{G}_{\text{KG}}\\!=\\!\\{\mathcal{V}_{\text{KG}},\mathcal{E}_{\text{KG}},\mathcal{R}_{\text{KG}}\\}$
which carries abundant knowledge in biomedicine and healthcare, and
hyperparameters $L,T,P,\alpha,\gamma$.

2:initialize $\bm{\theta}_{g},\bm{\theta}_{k}$ randomly;

3:merge $\mathcal{G}_{\text{DDI}}$ and $\mathcal{G}_{\text{KG}}$ into the
large combined network
$\mathcal{G}=\\{\mathcal{V},\mathcal{E},\mathcal{R}\\}$;

4:initialize $\mathbf{e}_{v}^{(0)}$ for $v\in\mathcal{V}$ by random Gaussian
or pretrained KG embeddings;

5:while not done do

6: for $l=1,\dots,L$ do

7: obtain generic embedding $\mathbf{e}_{v}^{(l)}$ by equation (1) and
equation (2);

8: end for

9: for all $(h,r,t)\in\mathcal{E}_{\text{DDI}}$ do

10: extract its drug-flow subgraph $\bar{\mathcal{S}}_{h,t}$ by
DSextract($\mathcal{G},h,t,K,P$) (Algorithm 1);

11: initialize node embedding $\mathbf{h}_{v}^{(0)}$ as $\mathbf{e}_{v}^{(L)}$
and graph structure ${\mathbf{A}}^{(0)}_{h,t}$ as $\bar{\mathbf{A}}_{h,t}$;

12: for $\tau=1,\dots,T$ do

13: estimate graph structure $\mathbf{A}^{(\tau)}_{h,t}$ by equation (3) and
equation (4);

14: refine node embedding $\mathbf{h}_{v}^{(\tau)}$ by equation (5);

15: end for

16: obtain knowledge subgraph $\mathcal{S}_{h,t}$ corresponding to graph
structure recorded in ${\mathbf{A}}^{(T)}_{h,t}$;

17: obtain subgraph embedding $\mathbf{h}_{\mathcal{S}_{h,t}}$ of
$\mathcal{S}_{h,t}$ by equation (6);

18: obtain class prediction $\hat{\mathbf{y}}_{h,t}$ by equation (7);

19: end for

20: if multiclass DDI prediction then optimize $\bm{\theta}_{g}$ and
$\bm{\theta}_{k}$ w.r.t. equation (8).

21: else if multilabel DDI prediction then optimize $\bm{\theta}_{g}$ and
$\bm{\theta}_{k}$ w.r.t. equation (9).

22: end if

23:end while

24:return optimized $\bm{\theta}_{g}$ and $\bm{\theta}_{k}$.

Algorithm 3 Testing procedure of KnowDDI.

1:a testing drug-pair $(h^{\prime},t^{\prime})$, a trained KnowDDI with
optimized $\bm{\theta}_{g}$ and $\bm{\theta}_{k}$, a combined network
$\mathcal{G}=\\{\mathcal{V},\mathcal{E},\mathcal{R}\\}$ which combines the DDI
graph and external KG, and hyperparameters $L,T,P,\alpha,\gamma$.

2:initialize $\mathbf{e}_{v}^{(0)}$ for $v\in\mathcal{V}$ by random Gaussian
or pretrained KG embeddings;

3:for $l=1,\dots,L$ do

4: obtain generic embedding $\mathbf{e}_{v}^{(l)}$ by equation (1) and
equation (2);

5:end for

6:extract its drug-flow subgraph $\bar{\mathcal{S}}_{h^{\prime},t^{\prime}}$
by DSextract($\mathcal{G},h^{\prime},t^{\prime},K,P$) (Algorithm 1);

7:initialize node embedding $\mathbf{h}_{v}^{(0)}$ as $\mathbf{e}_{v}^{(L)}$
and graph structure ${\mathbf{A}}^{(0)}_{h^{\prime},t^{\prime}}$ as
$\bar{\mathbf{A}}_{h^{\prime},t^{\prime}}$;

8:for $\tau=1,\dots,T$ do

9: estimate graph structure $\mathbf{A}^{(\tau)}_{h^{\prime},t^{\prime}}$ by
equation (3) and equation (4);

10: refine node embedding $\mathbf{h}_{v}^{(\tau)}$ by equation (5);

11:end for

12:obtain knowledge subgraph $\mathcal{S}_{h^{\prime},t^{\prime}}$
corresponding to graph structure recorded in
${\mathbf{A}}^{(T)}_{h^{\prime},t^{\prime}}$;

13:obtain subgraph embedding
$\mathbf{h}_{\mathcal{S}_{h^{\prime},t^{\prime}}}$ of
$\mathcal{S}_{h^{\prime},t^{\prime}}$ by equation (6);

14:obtain class prediction $\hat{\mathbf{y}}_{h^{\prime},t^{\prime}}$ by
equation (7);

15:if multiclass DDI prediction then return
$\arg\max_{i}[\hat{\mathbf{y}}_{h^{\prime},t^{\prime}}]_{i}$ which is the
relation with the highest possibility,

16:else if multilabel DDI prediction then return
$\hat{\mathbf{y}}_{h^{\prime},t^{\prime}}$.

17:end if

##  Appendix F Model analysis

Here, we provide more experimental results of model analysis.

###  F.1 Different knowledge subgraph generation strategies

We compare the KnowDDI with variants changing the knowledge subgraph
generation strategy of KnowDDI: (i) w/o relation emb: remove relation
embedding $\mathbf{h}_{r}$ in equation (3); (ii) w/ cosine sim: replace
equation (3) by cosine similarity between $\mathbf{h}_{u}^{\tau-1}$ and
$\mathbf{h}_{v}^{\tau-1}$; (iii) w/o threshold: remove threshold $\gamma$ in
equation (4); (iv) w/o learned graph: use the graph structure
$\mathbf{A}^{(0)}_{h,t}$ of drug-flow subgraph directly, which can be achieved
by always setting ${\mathbf{A}}^{(\tau)}_{h,t}=\mathbf{A}^{(0)}_{h,t}$ in
equation (5); (v) w/o edge update: calculate $\mathbf{C}^{(\tau)}_{h,t}$ in
equation (3) using generic embedding $\mathbf{h}_{u}^{(0)}$ for all layers;
and (vi) w/o subgraph emb: remove subgraph embedding
$\mathbf{h}_{\mathcal{S}_{h,t}}$ in equation (7).

Supplementary Figure4 plots the results. As shown, KnowDDI defeats all these
variants. Both“w/o relation emb” and “w/ cosine sim” do not use the relation
embedding $\mathbf{h}_{r}$, which leads to performance drop in comparison to
KnowDDI. “w/o relation emb” is slightly better than “w/ cosine sim”, which
validates the benefit of estimating relevance scores from the datasets. “w/o
threshold” suffers from a too dense knowledge subgraph, which has also been
observed in Supplementary Figure5(d) below. The performance gain of KnowDDI
over “w/o learned graph” shows that simply counting on the drug-flow subgraph
cannot obtain satisfactory results. “w/o edge update” directly learns a fixed
graph structure using generic embeddings which do not encode pair-specific
information, while KnowDDI alternatively estimates graph structure and refines
both the graph structure and node embeddings to be more predictive of DDI
types. “w/o subgraph emb” performs worse, which validates the effectiveness of
leveraging subgraph embedding to explicitly encode the local context of the
knowledge subgraph.

Figure 4: Different knowledge subgraph generation strategies for KnowDDI on
DrugBank.

###  F.2 Hyperparameters

####  F.2.1 Summary of hyperparameters

We summarize the hyperparameters used by KnowDDI in Supplementary Table 4.

Table 4: Hyperparameters used by KnowDDI. For each hyperparameter, we provide its meaning, range and values chosen on DrugBank and TWOSIDES respectively. Hyperparameter |  Meaning |  Range |  DrugBank |  TWOSIDES
---|---|---|---|---
$lr$ |  learning rate |  [0.001,0.005,0.01] |  0.005 |  0.005
$lr\\_decay\\_rate$ |  decay rate of learning rate |  [0.9,0.93,0.95] |  0.93 |  0.93
$weight\\_decay\\_rate$ |  weight decay rate in Adam |  [5e-6,1e-5,5e-5] |  1e-5 |  1e-5
$K$ |  the hop $K$ of neighborhood |  [2,3,4] |  2 |  2
$emb\\_dim$ |  dimension of node embedding |  [24,32,64] |  32 |  32
$rel\\_emb\\_dim$ |  dimension of relation embedding |  [16,24,32] |  32 |  24
$L$ |  number of layers in GraphSAGE |  [1,2,3] |  2 |  2
$gcn\\_dropout$ |  dropout rate in GraphSAGE |  [0.1,0.2,0.3] |  0.2 |  0.2
$T$ |  the times of alternating steps in graph structure learning |  [1,2,3] |  3 |  1
$P$ |  the length of relational path |  [3,4,5] |  3 |  3
$MLP\\_hidden\\_dim$ |  embedding dimension of MLP in equation (3) |  [16,24,32] |  16 |  24
$MLP\\_num\\_layers$ |  number of MLP layer in equation (3) |  [1,2,3] |  2 |  3
$MLP\\_dropout$ |  MLP dropout rate in equation (3) |  [0.1,0.2,0.3] |  0.2 |  0.2
$\gamma$ |  the threshold used to screen out the edges |  [0.05,0.1,0.15,0.2] |  0.05 |  0.1
$\alpha$ |  a hyperparameter to balance the drug-flow subgraph and updated subgraph |  [0.3,0.5,0.7] |  0.7 |  0.5

####  F.2.2 Sensitivity analysis

Here, we analyze the effect of varying key hyperparameters used in KnowDDI.
Supplementary Figure5 shows the results. In particular, Supplementary
Figure5(a) and Supplementary Figure5(b) plot the effect of $P$ and $K$ which
control the size of drug-flow subgraph and also knowledge subgraph. Larger $P$
and $K$ will result in larger subgraphs. As can be observed, neither too small
nor too large a subgraph obtains satisfactory performance. Recall that KnowDDI
targets at selectively aggregating global topology information from the
combined network and pair-specific information encoded in the local subgraph.
A larger subgraph may bring in extra but useless information for the drug-
pair. Therefore, one should start with a drug-flow subgraph with proper size
by tuning $P$ and $K$. Supplementary Figure5(c) plots the effect of $\alpha$
which balances the contribution of the updated subgraph and the drug-flow
subgraph in equation (4). As shown, performance first increases with larger
$\alpha$ then decreases, and reaches the best performances at $\alpha=0.5$.
This shows that the learned subgraph can help correct information in the drug-
flow subgraph and consequently leads to better performance. Supplementary
Figure5(d) plots the effect of $\gamma$ which is the threshold to screen out
less informative edges. A larger $\gamma$ means more edges will be removed and
the subgraph becomes more sparse, while a smaller $\gamma$ keeps a denser
graph. As can be observed, $\gamma$ should be properly selected to only keep
those informative edges.

Figure 5: Sensitivity analysis of KnowDDI on DrugBank. a, Varying the length
$P$ of relational path. b, Varying the hop $K$ of neighborhood. c, Varying
$\alpha$ in equation (4). d, Varying $\gamma$ in equation (4).

##  Appendix G Case study: Resembling drugs

For each of the four exemplar drug-pairs in Figure3(b-e), we select the node-
pair with the largest connection strength in each knowledge subgraph, and list
their molecular graphs and drug efficacy below.

Diphenylpyraline (16879) is an antihistamine used for the treatment of
allergic rhinitis, hay fever, and allergic skin disorders.
---
Atomoxetine (610) is a selective norepinephrine reuptake inhibitor (SNRI) used
in the management of Attention Deficit Hyperactivity Disorder (ADHD).

Mestranol (622) is a synthetic estradiol found in oral contraceptive pills for
contraception and the treatment of other conditions in the female reproductive
system, such as dysmenorrhea and dysfunctional uterine bleeding.
---
Estriol (1127) is a weak estrogen used to treat vaginal dryness and estrogen
deficiency conditions, such as vaginitis and vulvar itching.

Pefloxacin (852) is an antibiotic used to treat a variety of bacterial
infections.
---
Gatifloxacin (882) is a fourth generation fluoroquinolone used to treat a wide
variety of infections in the body.

Homatropine (16892) methylbromide is a muscarinic receptor antagonist used to
treat peptic ulcers, gastric ulcers, and duodenal ulcers.
---
Scopolamine (51) is a belladonna alkaloid with anticholinergic effects
indicated for the treatment of nausea and vomiting associated with motion
sickness and postoperative nausea and vomiting (PONV).

Figure 6: Case study on resembling drugs. $\mathbf{b-e}$, The molecular graphs
and drug efficacy of the node-pair with the largest connection strength in
four knowledge subgraphs in Figure3(b-e).