# Product Manifold Representations for Learning on Biological Pathways

Daniel McNeela  Frederic Sala* Anthony Gitter*

###### Abstract

Machine learning models that embed graphs in non-Euclidean spaces have shown
substantial benefits in a variety of contexts, but their application has not
been studied extensively in the biological domain, particularly with respect
to biological pathway graphs. Such graphs exhibit a variety of complex network
structures, presenting challenges to existing embedding approaches. Learning
high-quality embeddings for biological pathway graphs is important for
researchers looking to understand the underpinnings of disease and train high-
quality predictive models on these networks. In this work, we investigate the
effects of embedding pathway graphs in non-Euclidean mixed-curvature spaces
and compare against traditional Euclidean graph representation learning
models. We then train a supervised model using the learned node embeddings to
predict missing protein-protein interactions in pathway graphs. We find large
reductions in distortion and boosts on in-distribution edge prediction
performance as a result of using mixed-curvature embeddings and their
corresponding graph neural network models. However, we find that mixed-
curvature representations underperform existing baselines on out-of-
distribution edge prediction performance suggesting that these representations
may overfit to the training graph topology. We provide our mixed-curvature
Product GCN code at https://github.com/mcneela/Mixed-Curvature-GCN and our
pathway analysis code at https://github.com/mcneela/Mixed-Curvature-Pathways.

protein-protein interaction, edge prediction, graph representation learning,
non-Euclidean, graph neural network

##  1 Introduction

Machine learning methods for embedding graphs (Wu et al., 2021) enable
learning on data ranging from social media networks, to proteins and
molecules, to phylogenies and knowledge graphs. These embeddings then enable
useful node classification and edge prediction models, which can perform tasks
as diverse as predicting whether a molecule is active against a given drug
target or whether two users are likely to share a preference for a given
product.

While traditional graph learning methods employ Euclidean representations
(Grover & Leskovec, 2016), it is known that for certain graphs Euclidean
representations are unable to perfectly preserve graph structure, regardless
of what algorithm is used (Bourgain, 1985). As a result, recent works have
studied whether lower graph distortion can be achieved by embedding in non-
Euclidean spaces, such as hyperbolic space (Sala et al., 2018). Generally
speaking, lower distortion correlates with better downstream task performance.

The works of Gu et al. (2018) and Giovanni et al. (2022) have examined
embeddings into mixed-curvature products of spaces and heterogeneous
manifolds, respectively. By “mixed-curvature products of spaces”, we mean a
Cartesian product of embedding spaces having constant positive, negative, or
zero curvature. These correspond to the model spherical, hyperbolic, and
Euclidean spaces, respectively (Lee, 2018). While these methods have been
evaluated on standard graph benchmarking datasets, their hypotheses have not
been validated for specialized graphs found in biological pathways and
networks, which may have properties and topologies that differ from general
graphs in other domains. The product space approach is appealing because
deciding a priori which space to embed into can be challenging.

Biological pathways are graphs that encode cellular processes. Typically, the
nodes in such graphs are entities such as genes, proteins, or metabolites, and
the edges designate relationships between them. For example, an edge
connecting nodes A and B might indicate that the presence of protein A
controls the transcription of gene B. Pathways are an important object of
study in network biology as they can be used to infer subcellular
relationships and understand the mechanisms underlying disease.

Embedding biological pathways is difficult because no canonical methods exist.
Pathways exhibit a high degree of complexity. Some, due to a lack of study,
are sparse while others exhibit high inter-connectivity. Their complex network
structures suggest that non-Euclidean representations might provide
significant benefits. However, no systematic study of embedding methods
applied to biological pathway graphs has been undertaken. Only Euclidean
embedding methods have been applied to pathway graphs (M A Basher & Hallam,
2020; Pershad et al., 2020), and because pathway graphs differ from standard
graphs used to benchmark non-Euclidean embedding models, it is unknown to what
extent these models would work for pathway graphs.

In this work, we study non-Euclidean embeddings of biological pathway graphs
and their performance relative to standard Euclidean embeddings.111This work
extends our workshop paper McNeela et al. (2023). We perform a large-scale
test of a variety of embedding methods to pathway graphs taken from PathBank,
Reactome, HumanCyc, the NCI Pathway Interaction Database, and KEGG and embed
into a number of different combinations of spaces. For each pathway graph, we
determine a best embedding space, as measured by the lowest graph distortion,
and learn the node embeddings for that graph. Although biological pathway
databases are of high quality, they are incomplete and only capture a fraction
of the knowledge about the relevant biological processes (Hanspers et al.,
2020). Therefore, we investigate the downstream performance of our graph
embeddings by predicting potentially missing pathway edges. Instead of only
predicting held out pathway edges, we also examine the considerably more
challenging task of predicting protein-protein interactions (PPIs) from an
independent database that may participate in the pathway. We find that mixed-
curvature representations outperform, as measured by distortion, Euclidean
representations in all cases. Furthermore, the positive impact of improved
embeddings seems to generalize to downstream tasks such as edge prediction,
where we find improvements in area under the receiver operating characteristic
curve (AUROC) and average precision (AP) for held out in-distribution edges.
However, for out-of-distribution STRING PPI edges, we find that the mixed-
curvature representations underperform baselines, perhaps suggesting that
these representations overfit to the in-distribution graph topology.

##  2 Background and Related Work

###  2.1 Non-Euclidean Embeddings and Machine Learning

Much of the research into non-Euclidean embeddings in machine learning
originated in studies of graphs and networks, where they were originally used
to embed concept ontologies. For example, Nickel & Kiela (2017) developed a
method for embedding into the Poincaré model of hyperbolic space and used it
to generate node embeddings for the WORDNET ontology. Sala et al. (2018)
expanded on this work by determining the precision-dimensionality tradeoffs
inherent in hyperbolic embeddings. Ganea et al. (2018) and Chami et al. (2019)
then produced generalized Hyperbolic Neural Networks and Hyperbolic Graph
Convolutional Networks (GCNs) to perform prediction directly in hyperbolic
space on data of various types. Gu et al. (2018) extended graph representation
learning to a Cartesian product of hyperbolic, spherical, and Euclidean
spaces, while Giovanni et al. (2022) further generalized this to a product of
manifolds of heterogeneous curvature. Our approach follows that of Gu et al.
(2018). More recently, hyperbolic and non-Euclidean geometries have been
applied to contrastive learning (Desai et al., 2023) and attention blocks
(Tseng et al., 2023).

Hyperbolic modeling approaches have been applied to embed biological and
chemical data exhibiting hierarchical structure. Many methods focus on drug
discovery, specifically drug repurposing (Lau et al., 2023) or drug target
prediction (Yue et al., 2023; Poleksic, 2023; Zahra et al., 2023; Pogány &
Antal, 2023b). The hierarchical relationships in the Gene Ontology make it
appealing for learning hyperbolic representations of its terms and their
associated genes (Kim et al., 2021; Li et al., 2023). In transcriptomics,
methods have been developed to visualize and estimate the curvature of gene
expression samples (Zhou & Sharpee, 2021), embed single-cell RNA-seq data
(Klimovskaia et al., 2020; Ding & Regev, 2021), and model differential
expression signatures (Pogány & Antal, 2023a). Finally, embedding biological
sequences has enabled visualizing members of a protein family (Susmelj et al.,
2023) and Bayesian phylogenetic inference (Macaulay et al., 2023).

###  2.2 Pathway Graphs and Embeddings

Pathway graphs have been well-studied from a biological perspective, but
embedding them to facilitate downstream prediction tasks is relatively new.
For example, M A Basher & Hallam (2020) developed a method called pathway2vec,
which combines five different Euclidean embedding methods to learn embeddings
of biological pathways. Similarly, Pershad et al. (2020) used node2vec to
generate embeddings of PPI networks and used the resulting embeddings as one
component of a method to predict response to psychiatric drugs. Pathway
embeddings are a crucial input to models that operate on pathway network
structures to make predictions. Euclidean GCNs have been broadly applied to
biological pathways to predict cancer subtypes (Lee et al., 2020; Hayakawa et
al., 2022), synthetic lethality (Lai et al., 2021), PPIs (Pham & Dang, 2021),
cancer survival (Liang et al., 2022), textual pathway descriptions (Yang et
al., 2022), and subcellular localization (Magnano & Gitter, 2023). However,
there has been no systematic study investigating the use of non-Euclidean and
mixed-curvature embedding models for pathway graphs.

###  2.3 PPI Prediction

PPIs can be predicted based on combinations of proteins’ sequence, expression,
functional, evolutionary, or 3D structural features (Durham et al., 2023). One
PPI prediction formulation is as an edge or link prediction task in a network
of known PPIs (Li et al., 2022). This can be accomplished using features from
the original graph or by learning node embeddings as features for the edge
prediction task. For example, Feng et al. (2020) predict signaling cascades
from PPI graphs by integrating transcriptomics and copy-number data into a
GCN. Jiang et al. (2020) use graph embeddings to predict links that indicate
an enzymatic reaction between pairs of molecules from the KEGG database.
Finally, Zhang & Kabuka (2019) and Liu et al. (2020) leverage a combination of
sequence and network information to predict PPIs.

| Pathbank | Reactome | KEGG | HumanCyc | NCI
---|---|---|---|---|---
Number of graphs | 875 | 1723 | 82 | 242 | 212
Number of graphs with single best space | 411 | 884 | 47 | 210 | 144
Number of graphs with multiple best spaces | 464 | 839 | 35 | 32 | 68
Mean number of nodes | 63.23 | 54.36 | 159.73 | 104.83 | 100.95
Mean number of edges | 283.56 | 1091.79 | 657.39 | 289.62 | 531.98
Median number of nodes | 53 | 24 | 152 | 73 | 73
Median number of edges | 162 | 65 | 532 | 133 | 194

Table 1: General statistics summarizing the five pathway databases.

##  3 Methods

###  3.1 Data Processing

We downloaded five pathway datasets from Pathway Commons (Rodchenkov et al.,
2020) v12, namely, PathBank (Wishart et al., 2020), Reactome (Gillespie et
al., 2022), HumanCyc (Romero et al., 2004), KEGG (Kanehisa et al., 2012), and
the NCI Pathway Interaction Database (Schaefer et al., 2009). We used the .txt
files containing interaction participants, edge types, and associated
metadata. For each pathway in each dataset, we created a NetworkX (Hagberg et
al., 2008) graph object and ignored Pathway Commons edge types, treating each
edge as undirected with no additional annotations. We then generated edge
lists for the undirected graphs and learned embeddings using the mixed-
curvature embedding Python library (Gu et al., 2018). All subsequent
experiments were performed only for those pathway graphs for which a unique
“best” embedding space could be determined, defined as the space exhibiting
minimum distortion. Those pathways which did not exhibit a unique embedding
space of minimum distortion were discarded as there exists no canonical way to
select a representative product space to compare embedding and edge prediction
methods against. We also discarded one pathway from the Reactome dataset
having 36,293 edges because its runtime on the edge prediction task was
prohibitive and it is an extreme outlier in terms of pathway size (Table 1).

For edge prediction, we downloaded Homo sapiens PPIs from STRING v12.0
(Szklarczyk et al., 2023). In STRING, PPIs are listed as pairs of ENSEMBL
protein identifiers. For each gene or protein node name in the given pathway
dataset, we map it to its UniProt SwissProt identifier using the MyGene API
(Xin et al., 2016). For each ENSEMBL protein identifier, we map it to the
Ensembl_HGNC_uniprot_id listed in the STRING aliases file. We then iterate
over all STRING PPI edges. For each pathway, if there exists a STRING PPI
corresponding to two UniProt identifiers in the given pathway, we add an edge
to the potential test set for that graph. We only included STRING edges with
experimental evidence and discarded all edges with a score of less than 500
out of 1000. This yields a dataset of reasonable size for edge prediction,
although the score used to filter edges can be tuned. Filtering using higher
experimental evidence scores would likely yield less false positives in the
list of candidate edges for each pathway graph; however, the relationship here
is not strictly linear.

###  3.2 Learning Embeddings

Let $\mathbb{S}^{d}_{K},\mathbb{H}^{d}_{K}$ be the spherical and hyperbolic
spaces of dimension $d$ and curvature $K,-K$, respectively, and
$\mathbb{E}^{d}$ the Euclidean space of dimension $d$. 222We write
$\mathbb{E}$ for our Euclidean embedding space component to distinguish it
from $\mathbb{R}$, since our models of hyperbolic and spherical geometry also
use $\mathbb{R}$ as an ambient space. We describe our main embedding space:
for sequences of dimensions $s_{1},\ldots,s_{m}$, $h_{1},\ldots,h_{n}$, and
$e_{1},\ldots,e_{p}$, we write

$\mathcal{P}=\mathbb{S}^{s_{1}}\times\cdots\times\mathbb{S}^{s_{m}}\times\mathbb{H}^{h_{1}}\times\cdots\times\mathbb{H}^{h_{n}}\times\mathbb{E}^{e_{1}}\times\cdots\times\mathbb{E}^{e_{p}},$

for a product manifold with $m+n+p$ component spaces and total dimension
$\sum_{i}s_{i}+\sum_{j}h_{j}+\sum_{k}e_{k}$. We refer to each
$\mathbb{S}^{s_{i}},\mathbb{H}^{h_{i}},\mathbb{E}^{e_{i}}$ as _components_ or
_factors_. We refer to the decomposition, e.g.,
$(\mathbb{H}^{2})^{2}=\mathbb{H}^{2}\times\mathbb{H}^{2}$, as the _signature_.
For convenience, let ${M}_{1},\dots,{M}_{m+n+p}$ refer to the factors in the
product.

We learn an embedding function
$f:\mathcal{G}\to\mathcal{P}=M_{1}\times\cdots\times M_{m+n+p}$ where
$\mathcal{G}$ is the space of pathway graphs and $\mathcal{P}$ is the
Cartesian product of the factors. Squared distances in the product space
decompose as sums of squared distances in the factor spaces (Lee, 2018), i.e.

$d_{\mathcal{P}}(u,v)^{2}=\sum_{i}d_{M_{i}}(\pi_{i}(u),\pi_{i}(v))^{2}$

for $u,v\in\mathcal{P}$, where $\pi_{i}$ denotes projection onto the $i$th
factor. The learning takes place according to the method given in Gu et al.
(2018), namely embeddings are learned in a product manifold $\mathcal{P}$ via
optimization of the following loss function

$\mathcal{L}(x)=\sum_{1\leq i\leq j\leq
n}\left|\left(\frac{d_{\mathcal{P}}(x_{i},x_{j})}{d_{G}(X_{i},X_{j})}\right)^{2}-1\right|$

where $d_{G}(X_{i},X_{j})$ denotes the graph distance between nodes $X_{i}$
and $X_{j}$ and $d_{\mathcal{P}}(x_{i},x_{j})$ denotes the product manifold
geodesic distance between their learned embeddings. The main metric used to
evaluate our embeddings is the average graph distance distortion. We define
the distortion, $\mathcal{D}$, of $f$ relative to a graph $G=(V,E)$ to be

$\mathcal{D}(f)=\frac{1}{|V|^{2}}\sum_{u,v\in
V}\frac{|d_{\mathcal{P}}(f(u),f(v))-d_{G}(u,v)|}{d_{G}(u,v)}$

where $d_{G}(u,v)$ denotes the graph distance between nodes $u$ and $v$,
defined as the length of the shortest path connecting them in $G$.

###  3.3 Hyperparameters

We create 252 embeddings for each pathway graph corresponding to different
combinations of the number of hyperbolic, spherical, and Euclidean spaces; the
learning rate; and the dimensionalities of each space (Appendix). We test
having different numbers of copies of each space, where the number of copies
of each type ranges from 0 to 3. We also constrain the total dimensionality of
all spaces to sum to 100. This is to keep the comparison across different
combinations fair by ensuring they each have the same representational
capacity as governed by the number of dimensions.

(a) Node Distributions

(b) Edge Distributions

Figure 1: Histograms of node and edge distributions for the pathway databases
studied. A few outliers were excluded.

###  3.4 Mixed-Curvature Product GCN

Similar to Bachmann et al. (2020), we extend the Hyperbolic GCN of Chami et
al. (2019) to products of constant curvature manifolds. The following facts of
Riemannian geometry allow us to extend GCNs from single manifolds to product
manifolds.

######  Definition 3.1.

Given Riemannian manifolds, $(M_{1},g_{1}),\ldots,(M_{n},g_{n})$, we can
construct a Riemannian product manifold $(M_{1}\times M_{2}\times\cdots\times
M_{n},g)$ with the product metric, $g=g_{1}\oplus\cdots\oplus g_{n}$, where
the distance decomposes as

$d_{M}(x,y)^{2}=d_{M_{1}}(x_{1},y_{1})^{2}+\cdots+d_{M_{n}}(x_{n},y_{n})^{2}$

for any $x=(x_{1},\ldots,x_{n}),y=(y_{1},\ldots,y_{n})\in M$.

Thus, operations in the tangent spaces, $T_{p}M$, of the product manifold $M$
decompose as direct sums of operations over the constituent tangent spaces
$T_{p_{i}}M_{i}$, where $M=M_{1}\times\cdots\times M_{n}$ and
$p=(p_{1},\ldots,p_{n})$.

On each pathway graph for which we perform edge prediction, we match the
signature333We define the signature of a product manifold to be the number of
copies and dimensionality of each $\mathbb{H}$, $\mathbb{S}$, and $\mathbb{E}$
space, e.g.
$\mathbb{H}_{1}\times\mathbb{H}_{2}\times\mathbb{E}_{1}\times\mathbb{E}_{2}\times\mathbb{E}_{3}\times\mathbb{S}_{1}$
where $\dim(\mathbb{H}_{i})=15,\dim(\mathbb{E}_{i})=20,\dim(S_{i})=10$. of the
product manifold in the GCN to the one used to learn the pathway embedding.
The operations in the Product GCN are defined analogously to the ones in the
hyperbolic GCN paper (Chami et al., 2019). Briefly put, pathway nodes are
represented as a Cartesian product of points on some combination of copies of
$\mathbb{H}^{n}$, $\mathbb{S}^{n}$, and $\mathbb{E}^{n}$ manifolds. At each
layer in the Product GCN, the points are projected into a tangent space to the
manifold via the $\log$ map of the product space and a GCN operates in this
Euclidean tangent space using the standard message-passing updates. The points
are then projected back onto the manifold after the layer operation via the
product space $\exp$ map. Both the $\log$ map and the $\exp$ map decompose as
sums over the constituent manifolds in the product space. The same procedure
is used for attention and activation layers, with projection into the tangent
space, followed by a standard layer operation, followed by projection back
onto the manifold.

The Product GCN is trained using the Riemannian Adam optimization algorithm
(Becigneul & Ganea, 2019) on the edge prediction task described in Section
4.2.

##  4 Results

###  4.1 Pathway Embeddings

We first summarize each of the pathway databases in Table 1 and provide node
and edge histograms in Figure 1. There exist uniquely beneficial product
spaces, as measured by distortion, for only roughly half of the pathways. Most
pathways are small by graph machine learning standards with the majority
having less than 100 nodes and less than 200 edges. Thus, pathways exhibit a
unique opportunity to study the effects of popular graph learning techniques,
including GCNs, on small graphs, as most graph machine learning benchmarks
possess thousands, or even millions and billions, of nodes and edges.

Next, for each pathway graph, we determine the best combination of hyperbolic,
Euclidean, and spherical components as determined by lowest distortion. Figure
2 demonstrates the reductions in distortion gained from learning the non-
Euclidean embeddings over all graphs in each of the five pathway datasets we
studied. The red diagonal indicates the position at which the distortions of
the Euclidean embeddings and mixed-curvature product embeddings would be
equal. Points lying below this line indicate graphs for which a product
representation yielded a reduction in distortion over a Euclidean
representation. Because a fully Euclidean embedding is a special case of the
mixed-curvature product embedding, the best mixed-curvature product embedding
should always have better distortion than the best Euclidean embedding. The
question is whether the improvement is marginal or substantial on biological
pathways. We observe that mixed-curvature product spaces provide marked
reductions in distortion relative to the standard Euclidean embedding, with
many graphs achieving a greater than 50% reduction in distortion.

(a) PathBank dataset

(b) NCI dataset

(c) KEGG dataset

(d) Reactome dataset

(e) HumanCyc

(f) All datasets

Figure 2: Scatterplots of distortion in the Euclidean embedding versus
distortion in the mixed-curvature embedding for pathway datasets. Points are
colored by local density, with yellow indicating the highest density.  Figure
3: A violin plot of distortions across all datasets for the graph Laplacian,
node2vec, and mixed-curvature embedding methods.

We further compare our mixed-curvature embeddings against two common graph
embedding baselines. The first is node2vec (Grover & Leskovec, 2016), a
popular graph embedding method which uses an algorithm analogous to word2vec
(Mikolov et al., 2013) trained on random walks taken from the graph. We also
compare against a standard embedding method formed by taking eigenvectors of
the graph Laplacian matrix (Bonald, 2019). For both methods, we embed the
pathway graphs from all datasets and compute distortions for each pathway
embedding in each dataset. Because node2vec and the graph Laplacian embedding
do not directly minimize distortion in their loss functions, they can
arbitrarily scale embedding vectors, causing them to perform poorly on a naive
calculation of the distortion metric. Therefore, we perform a scaling
optimization for the node2vec and graph Laplacian embeddings, finding for each
dataset and embedding type the constant $c$ which minimizes the below
objective

$\min_{c\in\mathbb{R}^{+}}\sum_{i<j}\left(\frac{c\cdot
d(x_{i},x_{j})}{d(n_{i},n_{j})}-1\right)$

where $d(x_{i},x_{j})$ is the distance between the embeddings of nodes $i$ and
$j$, and $d(n_{i},n_{j})$ is the shortest-path graph distance between nodes
$i$ and $j$. After calculating the optimal scaling factor, $c$, we scale the
embeddings by this factor and recalculate the distortions for all pathways in
the dataset. We summarize the distortions in Figure 3. The mixed-curvature
embeddings exhibit significantly lower mean distortion and variance than the
node2vec and Laplacian baselines.

###  4.2 Edge Prediction

####  4.2.1 Overview

We train on individual pathway graphs to predict a set of held-out edges, then
test our prediction models on the test set of experimentally validated edges
from STRING. For each pathway, we determine its optimal embedding space using
the results of our hyperparameter sweep for the distortion task; i.e., for
each pathway graph, we select the signature for the embedding space (number of
copies of hyperbolic, euclidean, and spherical space, as well as their
respective dimensionalities) that minimizes the distortion across the set of
embeddings for that pathway graph. We then perform edge prediction on two
different datasets:

##### In-Distribution

An in-distribution validation set containing held-out edges from the original
pathway graph. These edges are not seen by the model during training, but the
validation set performance metrics, such as validation accuracy and validation
loss, are used to guide model training and hyperparameter selection. 95% of
the edges from each pathway were used for training, and 5% were used for
validation.

##### Out-of-Distribution

An out-of-distribution test set that includes edges from the STRING PPI
database. This dataset is not seen by the model during training and is not
used to guide model training or hyperparameter selection. These edges are
general PPIs and do not necessarily have the same biological context as the
pathway. However, pathway edge predictions supported by a general PPI are more
plausible than those that are not.

We expect that the Product GCN, initialized with the min-distortion product
space embeddings, has a better inductive bias in its pretrained
representations that more preferentially match the topology of each graph than
any of the other baselines. Thus, we expect that the Product GCN should more
accurately predict in-distribution edges held out from the original graph.
Although we are less certain what the effect of introducing non-Euclidean
geometry should be to the prediction of out-of-distribution edges from an
external database, we would hope that the Product GCN would still outperform
the other baselines on this task.

####  4.2.2 Baselines and Product GCN

We compare four different methods on the edge prediction task:

Figure 4: Box plots giving comparison of edge prediction performance on all
five pathway datasets for all models as given by four metrics: Validation Set
AP, Validation Set AUROC, Test Set AP, and Test Set AUROC.

  1. 1.

A Euclidean GCN initialized with pretrained node2vec embeddings.

  2. 2.

A Euclidean GCN initialized with pretrained Laplacian embeddings.

  3. 3.

A Euclidean GCN initialized with pretrained Euclidean embeddings designed to
minimize distortion in $\mathbb{R}^{n}$.

  4. 4.

A Product GCN architecture that matches the signature of the optimal non-
Euclidean pathway embedding as measured by minimum distortion and is
initialized with the pretrained mixed-curvature embeddings from the first task
described in Section 4.1.

The pretrained Euclidean embeddings in method 3 are learned using the same
Python package as is used for learning the mixed-curvature embeddings (Gu et
al., 2018) but use a Euclidean manifold as the target output space. For all
methods, we keep the total dimensionality of the embedding space fixed at 100
dimensions, with the exception of the Laplacian embedding method for which the
embedding dimension was set to be the number of nodes in the graph, $|V|$.
Since most pathway graphs have $|V|\leq 100$ (Figure 1(a)) the reduced
dimensionality of the Laplacian embedding would be expected to be harmful,
though it could provide an outsized benefit in representational capacity for
pathway graphs with a large number of nodes.

For the Product GCN, we match the signature of the best mixed-curvature
embedding. For example, if the best embedding for pathway 21 of the PathBank
dataset has the signature
$\mathbb{H}^{2}\times\mathbb{S}^{3}\times\mathbb{E}^{2}$, then we use a
Product GCN architecture with two hyperbolic manifold layers, three spherical
manifold layers, and two Euclidean manifold layers. We furthermore split each
node embedding into its constituent pieces as determined by the product
manifold signature. In this example, the embedding would be split into
$2+3+2=7$ slices, each of dimension 14. Each slice of the embedding is fed
into the GCN for its corresponding manifold. Scores are then generated by each
of the component GCNs and averaged to produce a final output score that is
used to predict the presence of an edge between a given pair of nodes.

####  4.2.3 Model Training

For each graph on which we perform edge prediction with the Product GCN, we
perform a sweep over 324 hyperparameter combinations (Appendix) and choose the
one with the highest average of validation set AUROC and AP for use in
downstream analyses. For the baseline methods, we perform a sweep over 108
hyperparameter configurations (Appendix). The difference in the two sweeps
arises from the fact that the Product GCN has one additional hyperparameter,
the curvature of the hyperbolic and spherical embedding spaces. We sweep over
three values for this parameter, yielding three times as many hyperparameter
configurations.

####  4.2.4 Summary of Results

We find that mixed-curvature product manifold representations yield
substantial benefits in representational capacity for boosting downstream edge
performance on the in-distribution validation edges (Figure 4). The Product
GCN initialized with pretrained product manifold representations outperforms,
on average, all baselines for prediction of in-distribution validation set
edges. Among the baseline methods, the graph Laplacian-based Euclidean GCN
outperforms the two other Euclidean GCNs. For pathways with more than 100
nodes, this advantage is likely due to its larger embedding dimension. In the
Appendix, we provide detailed paired comparisons of the AP and AUROC for the
Product GCN and each Euclidean GCN on individual graphs in each pathway
database.

The Product GCN does not have the same advantage over the baselines for the
prediction of out-of-distribution PPI edges from STRING. The Euclidean GCN
with pretrained graph Laplacian embeddings is the clear best model, and all
others have near random AUROC. We surmise that the introduction of these edges
induces a substantial distribution shift in the graph topologies, which makes
the initially learned product space embeddings no longer a good fit. This
causes the Product GCN to underperform relative to the Euclidean GCNs with
their respective pretrained embeddings. One conclusion we can draw is that
common Euclidean-style embeddings are more robust to distribution shifts of
graph topology while product space embeddings, via their learned curvatures,
impose a strong inductive bias to tree-like and ring-like structures in the
graphs.

##  5 Discussion and Conclusion

We find that performing representation learning in non-Euclidean and mixed-
curvature spaces yields notable improvements in distortion and downstream in-
distribution edge prediction performance. In all cases, a mixed-curvature
representation yields an embedding with lower distortion than a simple
Euclidean embedding. However, the exact decomposition of the product space
into its mixture of component factors is highly dependent on the graph
topology. Thus, it is beneficial to perform a hyperparameter sweep over the
number and types of factors, as well as their dimensionalities, when learning
a representation for a biological pathway graph. Such a sweep need not be
highly time intensive as biological pathway graphs are generally of a modest
size.

Interestingly, for the out-of-distribution edge prediction task, the full
structure of the graph is not known when the node embeddings are learned
because the test set edges from STRING are not included in the training set.
This leads to a number of questions about how to further generalize our
approach. For example, we may ask how close the full graph structure must be
to the observed structure in order to produce node embeddings that are usable
for edge prediction via our method. We can also explore whether there are
alternative hyperparameter tuning strategies that produce slightly worse
distortions or in-distribution edge prediction performance but generalized
better to out-of-distribution edge prediction.

Another future direction would be to consider additional downstream tasks, as
edge prediction is not the only useful task for biological pathways that can
be improved by non-Euclidean representation learning. We plan to investigate
how our pathway embeddings benefit other problems such as node classification,
for example, predicting the type (gene, protein, small molecule, metabolite,
etc.) of a pathway node.

The lower performance of all models on the out-of-distribution edge prediction
setting relative to the in-distribution setting may be partially attributable
to biological context. Edges in a pathway reflect a pair of proteins that
interact in the context of conducting some specific biological process,
potentially only in certain cells or tissue types. The trained GCNs predict
edges with that same context. However, the STRING-based evaluation lacks that
context. There may be false negative edges in which there is a real PPI that
is not relevant to the pathway. Alternatively, there may be false positive
edges that are cell- or tissue-specific and not yet identified by the
experiments aggregated into STRING.

Non-Euclidean embedding models have not been applied to pathway graphs,
perhaps due to a lack of awareness among network biology researchers of their
utility in reducing distortion of graph distances and improving downstream
predictive performance. We demonstrate that pathway graphs benefit from the
incorporation of non-Euclidean geometries into embeddings and prediction
models. We encourage researchers to consider making use of these non-standard
geometries when learning embeddings and making downstream predictions. To this
end, we provide code at https://github.com/mcneela/Mixed-Curvature-Pathways
and https://github.com/mcneela/Mixed-Curvature-GCN.

## Acknowledgements

We thank David Merrell, Christopher Magnano, and Sam Gelman for their help in
processing Pathway Commons graphs and generating visualizations. This research
was supported by NIH award R01GM135631, the Wisconsin Alumni Research
Foundation, and computing resources provided by the Center for High Throughput
Computing (2006).

## References

  * Bachmann et al. (2020) Bachmann, G., Becigneul, G., and Ganea, O.  Constant curvature graph convolutional networks.  In III, H. D. and Singh, A. (eds.), _Proceedings of the 37th International Conference on Machine Learning_ , volume 119 of _Proceedings of Machine Learning Research_ , pp. 486–496. PMLR, 13–18 Jul 2020.  URL https://proceedings.mlr.press/v119/bachmann20a.html.
  * Becigneul & Ganea (2019) Becigneul, G. and Ganea, O.-E.  Riemannian adaptive optimization methods.  In _International Conference on Learning Representations_ , 2019.  URL https://openreview.net/forum?id=r1eiqi09K7.
  * Bonald (2019) Bonald, T.  Spectral graph embedding - telecom paris, 2019.  URL https://perso.telecom-paristech.fr/bonald/documents/spectral.pdf.
  * Bourgain (1985) Bourgain, J.  On lipschitz embedding of finite metric spaces in Hilbert space.  _Israel Journal of Mathematics_ , 52(1):46–52, 1985.  doi: 10.1007/BF02776078.
  * Center for High Throughput Computing (2006) Center for High Throughput Computing.  Center for high throughput computing.  2006\.  doi: 10.21231/GNT1-HW21.
  * Chami et al. (2019) Chami, I., Ying, Z., Ré, C., and Leskovec, J.  Hyperbolic graph convolutional neural networks.  In _Advances in Neural Information Processing Systems_ , pp. 4869–4880, 2019.
  * Desai et al. (2023) Desai, K., Nickel, M., Rajpurohit, T., Johnson, J., and Vedantam, S. R.  Hyperbolic image-text representations.  In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), _Proceedings of the 40th International Conference on Machine Learning_ , volume 202 of _Proceedings of Machine Learning Research_ , pp. 7694–7731. PMLR, 23–29 Jul 2023.  URL https://proceedings.mlr.press/v202/desai23a.html.
  * Ding & Regev (2021) Ding, J. and Regev, A.  Deep generative model embedding of single-cell RNA-Seq profiles on hyperspheres and hyperbolic spaces.  _Nature Communications_ , 12(1):2554, May 2021\.  doi: 10.1038/s41467-021-22851-4.
  * Durham et al. (2023) Durham, J., Zhang, J., Humphreys, I. R., Pei, J., and Cong, Q.  Recent advances in predicting and modeling protein–protein interactions.  _Trends in Biochemical Sciences_ , 48(6):527–538, June 2023.  doi: 10.1016/j.tibs.2023.03.003.
  * Feng et al. (2020) Feng, J., Zeng, A., Chen, Y., Payne, P., and Li, F.  Signaling interaction link prediction using deep graph neural networks integrating protein-protein interactions and omics data.  _bioRxiv_ , pp. 2020.12.23.424230, December 2020.  doi: 10.1101/2020.12.23.424230.
  * Ganea et al. (2018) Ganea, O.-E., Bécigneul, G., and Hofmann, T.  Hyperbolic Neural Networks.  _arXiv:1805.09112_ , June 2018.  doi: 10.48550/arXiv.1805.09112.
  * Gillespie et al. (2022) Gillespie, M., Jassal, B., Stephan, R., Milacic, M., Rothfels, K., Senff-Ribeiro, A., Griss, J., Sevilla, C., Matthews, L., Gong, C., Deng, C., Varusai, T., Ragueneau, E., Haider, Y., May, B., Shamovsky, V., Weiser, J., Brunson, T., Sanati, N., Beckman, L., Shao, X., Fabregat, A., Sidiropoulos, K., Murillo, J., Viteri, G., Cook, J., Shorser, S., Bader, G., Demir, E., Sander, C., Haw, R., Wu, G., Stein, L., Hermjakob, H., and D’Eustachio, P.  The Reactome pathway knowledgebase 2022.  _Nucleic Acids Research_ , 50(D1):D687–D692, January 2022.  doi: 10.1093/nar/gkab1028.
  * Giovanni et al. (2022) Giovanni, F. D., Luise, G., and Bronstein, M. M.  Heterogeneous manifolds for curvature-aware graph embedding.  In _ICLR 2022 Workshop on Geometrical and Topological Representation Learning_ , 2022.  URL https://openreview.net/forum?id=rtUxsN-kaxc.
  * Grover & Leskovec (2016) Grover, A. and Leskovec, J.  node2vec: Scalable feature learning for networks.  In _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , 2016.
  * Gu et al. (2018) Gu, A., Sala, F., Gunel, B., and Ré, C.  Learning mixed-curvature representations in product spaces.  In _International Conference on Learning Representations_ , 2018.
  * Hagberg et al. (2008) Hagberg, A. A., Schult, D. A., and Swart, P. J.  Exploring Network Structure, Dynamics, and Function using NetworkX.  In _Proceedings of the 7th Python in Science Conference_ , pp. 11–15, Pasadena, CA USA, 2008.
  * Hanspers et al. (2020) Hanspers, K., Riutta, A., Summer-Kutmon, M., and Pico, A. R.  Pathway information extracted from 25 years of pathway figures.  _Genome Biology_ , 21(1):273, November 2020.  doi: 10.1186/s13059-020-02181-2.
  * Hayakawa et al. (2022) Hayakawa, J., Seki, T., Kawazoe, Y., and Ohe, K.  Pathway importance by graph convolutional network and Shapley additive explanations in gene expression phenotype of diffuse large B-cell lymphoma.  _PLOS ONE_ , 17(6):e0269570, June 2022.  doi: 10.1371/journal.pone.0269570.
  * Jiang et al. (2020) Jiang, J., Liu, L.-P., and Hassoun, S.  Learning graph representations of biochemical networks and its application to enzymatic link prediction.  _Bioinformatics_ , 37(6):793–799, 10 2020.  doi: 10.1093/bioinformatics/btaa881.
  * Kanehisa et al. (2012) Kanehisa, M., Goto, S., Sato, Y., Furumichi, M., and Tanabe, M.  KEGG for integration and interpretation of large-scale molecular data sets.  _Nucleic Acids Research_ , 40(D1):D109–D114, January 2012.  doi: 10.1093/nar/gkr988.
  * Kim et al. (2021) Kim, J., Kim, D., and Sohn, K.-A.  HiG2Vec: hierarchical representations of Gene Ontology and genes in the Poincaré ball.  _Bioinformatics_ , 37(18):2971–2980, September 2021.  doi: 10.1093/bioinformatics/btab193.
  * Klimovskaia et al. (2020) Klimovskaia, A., Lopez-Paz, D., Bottou, L., and Nickel, M.  Poincaré maps for analyzing complex hierarchies in single-cell data.  _Nature Communications_ , 11(1):2966, June 2020\.  doi: 10.1038/s41467-020-16822-4.
  * Lai et al. (2021) Lai, M., Chen, G., Yang, H., Yang, J., Jiang, Z., Wu, M., and Zheng, J.  Predicting Synthetic Lethality in Human Cancers via Multi-Graph Ensemble Neural Network.  In _43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)_, pp. 1731–1734, November 2021.  doi: 10.1109/EMBC46164.2021.9630716.
  * Lau et al. (2023) Lau, Y., Gutierrez, J. M., Volkovs, M., and Zuberi, S.  Drug repurposing for leishmaniasis with hyperbolic graph neural networks.  _bioRxiv_ , pp. 2023.02.11.528117, 2023.  doi: 10.1101/2023.02.11.528117.
  * Lee (2012) Lee, J. M.  _Introduction to Smooth Manifolds_ , volume 218 of _Graduate Texts in Mathematics_.  Springer, New York, NY, 2012.  ISBN 978-1-4419-9981-8 978-1-4419-9982-5.  doi: 10.1007/978-1-4419-9982-5.  URL https://link.springer.com/10.1007/978-1-4419-9982-5.
  * Lee (2018) Lee, J. M.  _Introduction to Riemannian Manifolds_ , volume 176 of _Graduate Texts in Mathematics_.  Springer International Publishing, Cham, 2018.  ISBN 978-3-319-91754-2 978-3-319-91755-9.  doi: 10.1007/978-3-319-91755-9.  URL http://link.springer.com/10.1007/978-3-319-91755-9.
  * Lee et al. (2020) Lee, S., Lim, S., Lee, T., Sung, I., and Kim, S.  Cancer subtype classification and modeling by pathway attention and propagation.  _Bioinformatics_ , 36(12):3818–3824, June 2020\.  doi: 10.1093/bioinformatics/btaa203.
  * Li et al. (2022) Li, M. M., Huang, K., and Zitnik, M.  Graph representation learning in biomedicine and healthcare.  _Nature Biomedical Engineering_ , 6(12):1353–1369, December 2022.  doi: 10.1038/s41551-022-00942-x.
  * Li et al. (2023) Li, N., Yang, Z., Yang, Y., Wang, J., and Lin, H.  Hyperbolic hierarchical knowledge graph embeddings for biological entities.  _Journal of Biomedical Informatics_ , 147:104503, November 2023.  doi: 10.1016/j.jbi.2023.104503.
  * Liang et al. (2022) Liang, B., Gong, H., Lu, L., and Xu, J.  Risk stratification and pathway analysis based on graph neural network and interpretable algorithm.  _BMC Bioinformatics_ , 23(1):394, September 2022\.  doi: 10.1186/s12859-022-04950-1.
  * Liu et al. (2020) Liu, L., Zhu, X., Ma, Y., Piao, H., Yang, Y., Hao, X., Fu, Y., Wang, L., and Peng, J.  Combining sequence and network information to enhance protein–protein interaction prediction.  _BMC Bioinformatics_ , 21(16):537, 2020.  doi: 10.1186/s12859-020-03896-6.
  * M A Basher & Hallam (2020) M A Basher, A. R. and Hallam, S. J.  Leveraging heterogeneous network embedding for metabolic pathway prediction.  _Bioinformatics_ , 37(6):822–829, 10 2020.  doi: 10.1093/bioinformatics/btaa906.
  * Macaulay et al. (2023) Macaulay, M., Darling, A., and Fourment, M.  Fidelity of hyperbolic space for Bayesian phylogenetic inference.  _PLOS Computational Biology_ , 19(4):e1011084, April 2023.  doi: 10.1371/journal.pcbi.1011084.
  * Magnano & Gitter (2023) Magnano, C. S. and Gitter, A.  Graph algorithms for predicting subcellular localization at the pathway level.  In _Pacific Symposium on Biocomputing_ , pp. 145–156. World Scientific, 2023.  ISBN 9789811270604.  doi: 10.1142/9789811270611˙0014.  URL https://www.worldscientific.com/doi/10.1142/9789811270611_0014.
  * McNeela et al. (2023) McNeela, D., Sala, F., and Gitter, A.  Mixed-curvature representation learning for biological pathway graphs.  In _2023 ICML Workshop on Computational Biology_ , Honolulu, Hawaii, USA, July 2023.  URL https://icml-compbio.github.io/2023/papers/WCBICML2023_paper117.pdf.
  * Mikolov et al. (2013) Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J.  Distributed representations of words and phrases and their compositionality.  In Burges, C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger, K. (eds.), _Advances in Neural Information Processing Systems_ , volume 26. Curran Associates, Inc., 2013.  URL https://proceedings.neurips.cc/paper_files/paper/2013/file/9aa42b31882ec039965f3c4923ce901b-Paper.pdf.
  * Nickel & Kiela (2017) Nickel, M. and Kiela, D.  Poincaré embeddings for learning hierarchical representations.  In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), _Advances in Neural Information Processing Systems 30_ , pp. 6338–6347. Curran Associates, Inc., 2017.
  * Paszke et al. (2019) Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S.  PyTorch: An Imperative Style, High-Performance Deep Learning Library.  In _Advances in Neural Information Processing Systems_ , volume 32, pp. 8024–8035. Curran Associates, Inc., 2019.
  * Pershad et al. (2020) Pershad, Y., Guo, M., and Altman, R. B.  Pathway and network embedding methods for prioritizing psychiatric drugs.  _Pacific Symposium on Biocomputing_ , 25:671–682, 2020\.
  * Pham & Dang (2021) Pham, C. and Dang, T.  Link Prediction for Biomedical Network.  In _The 12th International Conference on Advances in Information Technology_ , IAIT2021, pp. 1–5, New York, NY, USA, July 2021\. Association for Computing Machinery.  ISBN 978-1-4503-9012-5.  doi: 10.1145/3468784.3471608.  URL https://dl.acm.org/doi/10.1145/3468784.3471608.
  * Pogány & Antal (2023a) Pogány, D. and Antal, P.  Hyperbolic Manifold Learning on Differential Expression Signatures.  _TechRxiv_ , December 2023a.  doi: 10.36227/techrxiv.24630747.v1.
  * Pogány & Antal (2023b) Pogány, D. and Antal, P.  Towards explainable interaction prediction: Embedding biological hierarchies into hyperbolic interaction space.  _bioRxiv_ , pp. 2023.12.05.568518, December 2023b.  doi: 10.1101/2023.12.05.568518.
  * Poleksic (2023) Poleksic, A.  Hyperbolic matrix factorization improves prediction of drug-target associations.  _Scientific Reports_ , 13(1):959, January 2023\.  doi: 10.1038/s41598-023-27995-5.
  * Rodchenkov et al. (2020) Rodchenkov, I., Babur, O., Luna, A., Aksoy, B. A., Wong, J. V., Fong, D., Franz, M., Siper, M. C., Cheung, M., Wrana, M., Mistry, H., Mosier, L., Dlin, J., Wen, Q., O’Callaghan, C., Li, W., Elder, G., Smith, P. T., Dallago, C., Cerami, E., Gross, B., Dogrusoz, U., Demir, E., Bader, G. D., and Sander, C.  Pathway Commons 2019 Update: integration, analysis and exploration of pathway data.  _Nucleic Acids Research_ , 48(D1):D489–D497, January 2020.  doi: 10.1093/nar/gkz946.
  * Romero et al. (2004) Romero, P., Wagg, J., Green, M. L., Kaiser, D., Krummenacker, M., and Karp, P. D.  Computational prediction of human metabolic pathways from the complete human genome.  _Genome Biology_ , 6(1):R2, December 2004.  doi: 10.1186/gb-2004-6-1-r2.
  * Sala et al. (2018) Sala, F., De Sa, C., Gu, A., and Re, C.  Representation tradeoffs for hyperbolic embeddings.  In Dy, J. and Krause, A. (eds.), _Proceedings of the 35th International Conference on Machine Learning_ , volume 80 of _Proceedings of Machine Learning Research_ , pp. 4460–4469. PMLR, July 2018.  URL https://proceedings.mlr.press/v80/sala18a.html.
  * Schaefer et al. (2009) Schaefer, C. F., Anthony, K., Krupa, S., Buchoff, J., Day, M., Hannay, T., and Buetow, K. H.  PID: the Pathway Interaction Database.  _Nucleic Acids Research_ , 37(suppl 1):D674–D679, January 2009.  doi: 10.1093/nar/gkn653.
  * Susmelj et al. (2023) Susmelj, A. K., Ren, Y., Vander Meersche, Y., Gelly, J.-C., and Galochkina, T.  Poincaré maps for visualization of large protein families.  _Briefings in Bioinformatics_ , 24(3):bbad103, May 2023.  doi: 10.1093/bib/bbad103.
  * Szklarczyk et al. (2023) Szklarczyk, D., Kirsch, R., Koutrouli, M., Nastou, K., Mehryary, F., Hachilif, R., Gable, A. L., Fang, T., Doncheva, N. T., Pyysalo, S., Bork, P., Jensen, L. J., and von Mering, C.  The STRING database in 2023: protein–protein association networks and functional enrichment analyses for any sequenced genome of interest.  _Nucleic Acids Research_ , 51(D1):D638–D646, January 2023.  doi: 10.1093/nar/gkac1000.
  * Tseng et al. (2023) Tseng, A., Yu, T., Liu, T. J., and Sa, C. D.  Coneheads: Hierarchy aware attention.  In _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023.  URL https://openreview.net/forum?id=CzAFnfwbGd.
  * Tu (2008) Tu, L. W.  _An Introduction to Manifolds_.  Universitext. Springer, New York, NY, 1 edition, 2008.  ISBN 978-0-387-48098-5 978-0-387-48101-2.  doi: 10.1007/978-0-387-48101-2.  URL http://link.springer.com/10.1007/978-0-387-48101-2.
  * Wishart et al. (2020) Wishart, D. S., Li, C., Marcu, A., Badran, H., Pon, A., Budinski, Z., Patron, J., Lipton, D., Cao, X., Oler, E., Li, K., Paccoud, M., Hong, C., Guo, A. C., Chan, C., Wei, W., and Ramirez-Gaona, M.  PathBank: a comprehensive pathway database for model organisms.  _Nucleic Acids Research_ , 48(D1):D470–D478, January 2020.  doi: 10.1093/nar/gkz861.
  * Wu et al. (2021) Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., and Yu, P. S.  A Comprehensive Survey on Graph Neural Networks.  _IEEE Transactions on Neural Networks and Learning Systems_ , 32(1):4–24, January 2021.  doi: 10.1109/TNNLS.2020.2978386.
  * Xin et al. (2016) Xin, J., Mark, A., Afrasiabi, C., Tsueng, G., Juchler, M., Gopal, N., Stupp, G. S., Putman, T. E., Ainscough, B. J., Griffith, O. L., Torkamani, A., Whetzel, P. L., Mungall, C. J., Mooney, S. D., Su, A. I., and Wu, C.  High-performance web services for querying gene and variant annotation.  _Genome Biology_ , 17(1):91, 2016.  doi: 10.1186/s13059-016-0953-9.
  * Yang et al. (2022) Yang, J., Liu, Z., Zhang, M., and Wang, S.  Pathway2Text: Dataset and Method for Biomedical Pathway Description Generation.  In _Findings of the Association for Computational Linguistics: NAACL 2022_ , pp. 1441–1454, Seattle, United States, July 2022\. Association for Computational Linguistics.  doi: 10.18653/v1/2022.findings-naacl.108.  URL https://aclanthology.org/2022.findings-naacl.108.
  * Yue et al. (2023) Yue, Y., McDonald, D., Hao, L., Lei, H., Butler, M. S., and He, S.  FLONE: fully Lorentz network embedding for inferring novel drug targets.  _Bioinformatics Advances_ , 3(1):vbad066, May 2023\.  doi: 10.1093/bioadv/vbad066.
  * Zahra et al. (2023) Zahra, N. u. A., Vagiona, A.-C., Uddin, R., and Andrade-Navarro, M. A.  Selection of Multi-Drug Targets against Drug-Resistant Mycobacterium tuberculosis XDR1219 Using the Hyperbolic Mapping of the Protein Interaction Network.  _International Journal of Molecular Sciences_ , 24(18):14050, January 2023.  doi: 10.3390/ijms241814050.
  * Zhang & Kabuka (2019) Zhang, D. and Kabuka, M.  Multimodal deep representation learning for protein interaction identification and protein family classification.  _BMC Bioinformatics_ , 20(16):531, 2019.  doi: 10.1186/s12859-019-3084-y.
  * Zhou & Sharpee (2021) Zhou, Y. and Sharpee, T. O.  Hyperbolic geometry of gene expression.  _iScience_ , 24(3), March 2021.  doi: 10.1016/j.isci.2021.102225.

Appendix

##  Appendix A Supplementary Methods

###  A.1 Pathway Commons Data

We use the Pathway Commons v12 PathBank pathways provided in this file:
https://www.pathwaycommons.org/archives/PC2/v12/PathwayCommons12.pathbank.hgnc.txt.gz

Pathway Commons provides a number of different data formats, including text,
SIF, JSON, and BioPAX. We use the text format as it provides the simplest
graph representation of the most important interactions in PathBank pathways.

###  A.2 Initializing GCNs with Embeddings

We save the embeddings learned by the embedding model in a PyTorch .pt file
(Paszke et al., 2019). We then load these embeddings and use them to
initialize the weights of the embedding layer for the models in the edge
prediction task, namely the Euclidean GCN and Product GCN. These embeddings
are then further trained via backpropagation during the edge prediction task.

##  Appendix B Supplementary Tables

learning rate | hyperbolic copies | Euclidean copies | spherical copies
---|---|---|---
1e-3 | 0 | 0 | 0
1e-2 | 1 | 1 | 1
1e-1 | 2 | 2 | 2
1 | 3 | 3 | 3
Table 2: Range of values for the pathway embedding hyperparameter sweep. Space dimensions were calculated automatically (to sum to 100) based on number of copies of each space. For example, if there were 2 hyperbolic copies, 1 Euclidean copy, and 3 spherical copies, then there would be floor(100/6) = 16 dimensions assigned to each copy. learning rate | hidden dim | # of layers | dropout | use bias | curvature (for Product GCN)
---|---|---|---|---|---
1e-4 | 16 | 2 | 0.1 | true | 0.5
1e-3 | 32 | 3 | 0.2 | false | 1
1e-2 | 64 | 4 |  |  | 2
Table 3: Hyperparameters used in the sweep for training the edge prediction
GCNs.

##  Appendix C Supplementary Figures

###  C.1 PathBank

Figure 5: Comparison of Euclidean GCN initialized with pretrained Euclidean
embeddings and Product GCN performance on in-distribution validation set and
out-of-distribution test set. Each density plot shows one of either AP or
AUROC metrics taken across all graphs in the PathBank dataset.

Figure 6: Comparison of Euclidean GCN initialized with pretrained Laplacian
embeddings and Product GCN performance on in-distribution validation set and
out-of-distribution test set. Each density plot shows one of either AP or
AUROC metrics taken across all graphs in the PathBank dataset.

Figure 7: Comparison of Euclidean GCN initialized with pretrained node2vec
embeddings and Product GCN performance on in-distribution validation set and
out-of-distribution test set. Each density plot shows one of either AP or
AUROC metrics taken across all graphs in the PathBank dataset.

###  C.2 Reactome

Figure 8: Comparison of Euclidean GCN initialized with pretrained Euclidean
embeddings and Product GCN performance on in-distribution validation set and
out-of-distribution test set. Each density plot shows one of either AP or
AUROC metrics taken across all graphs in the Reactome dataset.

Figure 9: Comparison of Euclidean GCN initialized with pretrained Laplacian
embeddings and Product GCN performance on in-distribution validation set and
out-of-distribution test set. Each density plot shows one of either AP or
AUROC metrics taken across all graphs in the Reactome dataset.

Figure 10: Comparison of Euclidean GCN initialized with pretrained node2vec
embeddings and Product GCN performance on in-distribution validation set and
out-of-distribution test set. Each density plot shows one of either AP or
AUROC metrics taken across all graphs in the Reactome dataset.

###  C.3 HumanCyc

Figure 11: Comparison of Euclidean GCN initialized with pretrained Euclidean
embeddings and Product GCN performance on in-distribution validation set and
out-of-distribution test set. Each density plot shows one of either AP or
AUROC metrics taken across all graphs in the HumanCyc dataset.

Figure 12: Comparison of Euclidean GCN initialized with pretrained Laplacian
embeddings and Product GCN performance on in-distribution validation set and
out-of-distribution test set. Each density plot shows one of either AP or
AUROC metrics taken across all graphs in the HumanCyc dataset.

Figure 13: Comparison of Euclidean GCN initialized with pretrained node2vec
embeddings and Product GCN performance on in-distribution validation set and
out-of-distribution test set. Each density plot shows one of either AP or
AUROC metrics taken across all graphs in the HumanCyc dataset.

###  C.4 NCI

Figure 14: Comparison of Euclidean GCN initialized with pretrained Euclidean
embeddings and Product GCN performance on in-distribution validation set and
out-of-distribution test set. Each density plot shows one of either AP or
AUROC metrics taken across all graphs in the NCI dataset.

Figure 15: Comparison of Euclidean GCN initialized with pretrained Laplacian
embeddings and Product GCN performance on in-distribution validation set and
out-of-distribution test set. Each density plot shows one of either AP or
AUROC metrics taken across all graphs in the NCI dataset.

Figure 16: Comparison of Euclidean GCN initialized with pretrained node2vec
embeddings and Product GCN performance on in-distribution validation set and
out-of-distribution test set. Each density plot shows one of either AP or
AUROC metrics taken across all graphs in the NCI dataset.

###  C.5 KEGG

Figure 17: Comparison of Euclidean GCN initialized with pretrained Euclidean
embeddings and Product GCN performance on in-distribution validation set and
out-of-distribution test set. Each density plot shows one of either AP or
AUROC metrics taken across all graphs in the KEGG dataset.

Figure 18: Comparison of Euclidean GCN initialized with pretrained Laplacian
embeddings and Product GCN performance on in-distribution validation set and
out-of-distribution test set. Each density plot shows one of either AP or
AUROC metrics taken across all graphs in the KEGG dataset.

Figure 19: Comparison of Euclidean GCN initialized with pretrained node2vec
embeddings and Product GCN performance on in-distribution validation set and
out-of-distribution test set. Each density plot shows one of either AP or
AUROC metrics taken across all graphs in the KEGG dataset.