# The Intelligible and Effective Graph Neural Additive Networks

Maya Bechler-Speicher
Blavatnik School of Computer Science
Tel-Aviv University
&Amir Globerson
Blavatnik School of Computer Science
Tel-Aviv University
&Ran Gilad-Bachrach
Department of Bio-Medical Engineering
Edmond J. Safra Center for Bioinformatics
Tel-Aviv University

###### Abstract

Graph Neural Networks (GNNs) have emerged as the predominant approach for
learning over graph-structured data. However, most GNNs operate as black-box
models and require post-hoc explanations, which may not suffice in high-stakes
scenarios where transparency is crucial. In this paper, we present a GNN that
is interpretable by design. Our model, Graph Neural Additive Network (GNAN),
is a novel extension of the interpretable class of Generalized Additive
Models, and can be visualized and fully understood by humans. GNAN is designed
to be fully interpretable, allowing both global and local explanations at the
feature and graph levels through direct visualization of the model. These
visualizations describe the exact way the model uses the relationships between
the target variable, the features, and the graph. We demonstrate the
intelligibility of GNANs in a series of examples on different tasks and
datasets. In addition, we show that the accuracy of GNAN is on par with black-
box GNNs, making it suitable for critical applications where transparency is
essential, alongside high accuracy.

##  1 Introduction

In many domains, ranging from biology to fraud detection, Artificial
Intelligence (AI) is applied to data with graph structure. Neural Networks,
and specifically Graph Neural Networks (GNNs), have emerged as the predominant
approach in these applications (see, for example, Zhou et al. [1]). While GNNs
demonstrate high accuracy, in terms of the correctness of their predictions,
they often function as black-box models; thus, their decision-making processes
are opaque. Transparency is vital for assessing potential biases or safety
risks and is particularly critical in high-stakes areas such as criminal
justice, healthcare, and finance, where decisions significantly impact
individuals’ lives. In such contexts, interpretable models, despite sometimes
being less accurate, may be preferred over complex black-box models [2].
Furthermore, the transparency of automated decision making processes is
increasingly becoming a legal mandate. While there is ongoing debate over
whether the European Union’s General Data Protection Regulation (GDPR) implies
a _“right to explanation”_ [3, 4], the proposed European AI Act explicitly
addresses this issue, stating that _“To address concerns related to opacity
and complexity of certain AI systems and help deployers to fulfill their
obligations under this regulation, transparency should be required for high-
risk AI systems before they are placed on the market or put into service”_
[5].

In this context, interpretability refers to the ease with which a human can
understand the reasoning behind model decisions or the general logic of a
model’s operation. It is important to distinguish between interpretability and
explainability [2]. Interpretability relates to models that are inherently
comprehensible by design, while explainability pertains to post-hoc methods
that elucidate aspects of black-box models [6]. These explanations often come
without correctness guarantees [7, 8] and may not provide a complete
description of the model and its predictions, potentially failing to expose
hidden pitfalls [9, 10, 11].

Methods for model explainability or interpretability can be categorized into
local and global types. Local methods, such as SHAP [12] and LIME [13],
elucidate individual predictions made by a model, whereas global methods, such
as feature-importance [14] and partial dependence plots [15], provide holistic
insights about the model, i.e., explain the overarching logic of the model
decision making [16]. However, it has been noted that local explainability
methods may not consistently align with their global counterparts [17].
Moreover, local explanations may be inadequate for verifying fairness and
other risks [8].

In this work, we introduce the Graph Neural Additive Networks (GNAN), an
interpretable-by-design GNN that offers both transparency and accuracy. GNAN
is a glass-box model [18] that allows for both local and global
interpretability. GNAN extends the family of Generalized Additive Models
(GAMs) [19], to accommodate graph data. GAMs are known for their ability to
fit complex, nonlinear functions while remaining interpretable and have proven
effective across various domains [20, 21, 22, 23, 24]. They operate by
learning shape functions for each feature and then linearly combining these
functions, making it easy to interpret them, as the influence of each feature
on the prediction is independent of other features and can be visualized
through their corresponding shape functions. Similarly, GNAN’s
interpretability is achieved through an architecture that restricts the use of
cross-products of features and graphs’ topology, thereby reducing its
complexity compared to other GNNs. Nonetheless, we demonstrate that GNAN,
despite its limited capacity, matches the performance of more expressive GNNs
on several real-world datasets. Additionally, GNAN does not rely on iterative
local message-passing, avoiding the computational bottlenecks commonly
associated with such GNNs [25].

In Section 4, we showcase through a series of examples how users can interpret
GNAN and gain precise insights into the connections between the target and the
graph, the target and the features, and the interplay between features and
graph information. In some cases, an exact description of the model can be
visualized through only a few figures. We also demonstrate how the
interpretability of GNAN allows users to debug their model, a process that can
be used for ensuring consistency with prior knowledge and avoiding biases and
safety risks. In Section 5, we compare the performance of GNAN with other GNN
architectures. This comparison underscores that sacrificing performance for
intelligibility is not necessary, as the performance of GNAN is comparable to
that of commonly used black-box GNNs.

The main contributions of this work are:

  1. 1.

The extension of Generalized Additive Models (GAMs) to graph data.

  2. 2.

The introduction of a fully interpretable-by-design model for graph prediction
tasks, demonstrating that its explanations provide both global and local
insights, through visualizations of the model itself, and include debugging
capabilities.

  3. 3.

The demonstration that GNAN achieves good performance on common real-world
graph datasets, despite its limited capacity. This observation supports
previous findings that some real-world graph problems are simple and do not
require the capacity of other GNNs.

Thus we argue that GNAN is suitable for high-stakes applications due to its
interpretability and performance.

##  2 Related work

#### Generalized Additive Models

Generalized Additive Models (GAMs) are a class of statistical models that
build upon generalized linear models by incorporating non-linear functions for
each variable while maintaining additivity [19, 20, 21]. Essentially, GAMs
model the expected value of the target variable as a sum of univariate
functions of the features. Formally, in GAMs, a prediction for an input $x$ is
computed by $\sigma\left(\sum f_{k}\left(x_{k}\right)\right)$ where $\sigma$
is a predefined activation function, such as the sigmoid111In the context of
Generalized Linear Models (GLMs) $\sigma$ can be though of as the inverse of
the link-function., and the $f_{k}$’s are shape functions learned during the
training process. This approach extends generalized linear models, in which
predictions are computed by $\sigma\left(\sum w_{k}x_{k}\right)$ where $w$ is
a learned weight vector.

GAMs are more expressive than generalized linear models while remaining
interpretable, as the effect of each predictor is modeled separately. For
example, they can capture non-monotone effects of features, which generalized
linear models cannot achieve without feature engineering. Traditionally, GAMs
utilize splines or other smooth shape functions to model the non-linear
relationships between each feature and the target variable. However, other
methods, such as trees, have been proposed to fit the shape functions [24].
Recently, Agarwal et al. [26] suggested using neural networks to learn the
shape functions. This approach combines the representational power of deep
learning with the interpretability of additive models.

#### Graph Neural Networks

Graph Neural Networks (GNNs) [27, 28, 29, 30] have emerged as the leading
approach for learning over graph data. The fundamental idea behind GNNs is to
use neural-networks that combine node features with graph-structure. A
commonly used family of GNNs is message-passing GNNs, where the
representations of nodes are updated in iterations through neighborhood
aggregations. This aggregation is done, for example, through a convolution-
like operation or an attention mechanism. [31, 32, 33]

Various non-message-passing approaches have been explored to disentangle the
node features from the graph structure. Such approaches were shown to enhance
performance across diverse applications [34, 35, 36]. Disentanglement can also
reduce overfitting, as popular GNNs which do entangle features and graph-
structure, were shown to have the tendency to overfit non-informative graph
information [37] GNAN uses these concepts in order to achieve a model that is
both high-performing and fully interpretable.

There are different prediction tasks on graphs [38]. In _graph tasks_ , the
goal is to predict properties of entire graphs. For example, a graph could
represent a molecule, and the goal would be to predict its toxicity level. In
_node tasks_ , the goal is to predict a property of a node (vertex) within a
graph. An example of a node task is predicting whether a user in a social
network is a human or a bot. In _link prediction tasks_ , the goal is to
determine whether there is an edge between two nodes of a graph. In this work,
we focus on graph tasks and node tasks. Although link prediction tasks are not
within the scope of this work, it is possible to view these problems as node
tasks on the dual line graph [39].

#### GNNs explanations

The inherent complexity of graph-structured data poses unique challenges for
explainability. Most approaches for explaining black-box GNNs focus on
providing a sub-graph or a similar structure that can explain a certain
example. This is done either as a post-hoc explanation for GNNs [40, 41, 42]
or by adjusting the data a priory [43, 44, 45]. For example, the method
suggested in Ying et al. [41] identifies both important subgraph structures
and node features influencing the GNN’s predictions by maximizing the mutual
information between the prediction and the distribution of possible subgraph
structures and node features. Yin et al. [43] suggested a structural pattern
learning module that is learning through pre-training. GNAN, on the contrary
to these methods, does not aim to provide an explanation through a proxy
object like a subgraph, nor does it require modification to the data, or the
training process. Instead, GNAN is a interpretable by design, and its exact
description can be visualized through its learned shape functions. In
particular, the exact relation between the target, the features, and the graph
can be visualized and conveyed to users.

##  3 Graph Neural Additive Networks

In this section, we introduce the Graph Neural Additive Networks (GNAN). We
begin by defining some essential notation. A graph $G$ has a set of $N$
vertices, where each vertex is associated with a $d$-dimensional feature
vector. Specifically, $\mathbf{x}_{i}\in\mathbb{R}^{d}$ represents the feature
vector of the $i$’th node in $G$. We define the distance
$\text{dist}\left(j,i\right)$ between node $j$ and node $i$ within the graph
$G$ as the number of edges in the shortest path from $j$ to $i$. This
definition implies that the distance from a node to itself is zero. In cases
where no path exists from $j$ to $i$, we set
$\text{dist}\left(j,i\right)=\infty$. For enhanced readability, we denote
vectors in boldface, and an entry $k$ of a vector $\mathbf{x}$ is denoted by
$[\mathbf{x}]_{k}$. We begin by describing GNAN for applications such as
binary classification and regression where the model output is one-
dimensional. At the end of this section, we discuss extensions to scenarios
such as multi-class classification, where the model output is multi-
dimensional.

GNAN generates a representation $\mathbf{h}_{i}\in\mathbb{R}^{d}$ for each
node $i$ by learning a distance function
$\rho(x;\theta):\mathbb{R}\rightarrow\mathbb{R}$ and a set of feature shape
functions
$\\{f_{k}\\}_{k=1}^{d},f_{k}(x;\theta_{k}):\mathbb{R}\rightarrow\mathbb{R}$.
Each of these functions is a neural network, and is optimized through back-
propagation. For brevity, we omit the parameterization $\theta$ and
$\theta_{k}$ for the remainder of this section. In GNAN, the $k$’th entry of
the representation $\mathbf{h}_{i}$ for the $i$’th node is defined as follows:

$[\mathbf{h}_{i}]_{k}=\sum_{j=1}^{N}\frac{1}{\\#\text{dist}_{i}(j,i)}\cdot\rho\left(\frac{1}{1+\text{dist}\left(j,i\right)}\right)\cdot
f_{k}\left(\left[\mathbf{x}_{j}\right]_{k}\right)$

where $\\#\text{dist}_{i}(j,i)$ represents the number of nodes at distance
$\text{dist}\left(j,i\right)$ from node $i$. The underlying rationale for this
definition is as follows: each node’s $k$’th feature is transformed by a shape
function $f_{k}$, independently from other features. The effect the $k$’th
feature value of node $j$ has on node $i$’s representation is influenced by
their distance. Specifically, if $\text{dist}\left(j,i\right)=l$, then the
cumulative influence of all nodes at distance $l$ from node $i$ is captured by
$\rho(\nicefrac{{1}}{{(1+l)}})$. This is achieved by the normalization term
$\nicefrac{{1}}{{\\#\text{dist}_{i}(j,i)}}$. Here, $\rho$’s argument
$\nicefrac{{1}}{{(1+l)}}$ scales the distance such that a distance of $0$ (the
self-distance of a node) is mapped to $1$, and an infinite distance, which
implies no path exists, is scaled to $0$. Thus, $\rho$ spans the interval
$[0,1]$.

The representation of each node is dependent on the entire graph, yet the
function $\rho$ enables weighting the influence from nodes, based on their
distance. This enables, for example, diminishing the impact of distant nodes,
or close neighbors. For each node $i$, the weighted sum of the transformed
feature vectors of all other nodes is computed, with weights assigned
according to their distance from $i$. This weighted sum is computed after the
shape functions are applied to the distances and the features of each node.

Given the node representations, both node prediction tasks and graph
prediction tasks can be implemented. To predict for the $i$’th node, the
computation is as follows:

$\sigma\left(\sum_{k=1}^{d}[\mathbf{h}_{i}]_{k}\right)~{}~{}~{},$

where the entry-wise sum of the representation vector $\mathbf{h}_{i}$ is
computed and subsequently processed using an activation function such as the
sigmoid for classification and the identity for regression. For a prediction
over the entire graph, the collective node representation is computed via sum-
pooling:

$\mathbf{h}=\sum_{i=1}^{N}\mathbf{h}_{i}~{}~{}~{}.$

Following this, the entry-wise sum of the graph representation $\mathbf{h}$ is
computed and also processed using the activation function:

$\sigma\left(\sum_{k=1}^{d}[\mathbf{h}]_{k}\right)~{}~{}~{}.$

Once the model is trained, it can be fully described using its univariate
functions $\rho$ and $\\{f_{k}\\}_{k=1}^{d}$.

From the definitions provided above, it follows that the entire model can be
represented with just a few figures, thus providing global interpretability.
For local explanations, it is feasible to examine the contribution of each
feature and each node to the predictions. Furthermore, consider the following
graph representation:

$\displaystyle[\mathbf{h}]_{k}=\sum_{i=1}^{N}[\mathbf{h}_{i}]_{k}=$
$\displaystyle\sum_{i=1}^{N}\sum_{j=1}^{N}\frac{1}{\\#\text{dist}_{i}(j,i)}\cdot\rho\left(\frac{1}{1+\text{dist}\left(j,i\right)}\right)\cdot
f_{k}\left(\left[\mathbf{x}_{j}\right]_{k}\right)$
$\displaystyle=\sum_{j=1}^{N}f_{k}\left(\left[\mathbf{x}_{j}\right]_{k}\right)\sum_{i=1}^{N}\frac{1}{\\#\text{dist}_{i}(j,i)}\cdot\rho\left(\frac{1}{1+\text{dist}\left(j,i\right)}\right)~{}~{}~{}.$

This mathematical formulation reveals how each node and feature contribute to
the overall graph representation. The elements in the first equation convey
the effect of nodes, while the elements of the second equation convey the
influence of individual features across all nodes. Therefore, the model
facilitates a detailed understanding of local behavior from multiple
perspectives.

The functions $\rho$ and $\\{f_{k}\\}_{k=1}^{d}$ may be implemented using a
variety of neural network architectures. In our experiments, we employed
multi-layer perceptrons (MLPs) with ReLU activations to implement these
functions. Nonetheless, other alternatives are viable, such as employing
learning splines for activations to achieve smoother shape functions [46].
Additionally, it is feasible to develop a separate distance network for each
feature to enhance the model’s capacity. Specifically, rather than utilizing a
single function $\rho$, one can train a distinct function $\rho_{k}$ for each
feature $k$, which weights the contribution of each feature based on its
node’s distance. For graph-level tasks, additional feature networks may be
integrated prior to aggregating the graph’s representation vector, akin to a
readout layer in GNNs. These extensions, along with a discussion on a tensor
representation of the computation that facilitates efficient GPU utilization,
are further elaborated in the Appendix.

For multiclass classification involving $C$ classes, we configure the final
layers of the feature shape functions
$f_{k}(x;\theta_{k}):\mathbb{R}\rightarrow\mathbb{R}^{C\times 1}$ and the
distance function $\rho(x;\theta):\mathbb{R}\rightarrow\mathbb{R}^{C\times 1}$
to accommodate the required dimensionality. The transformed feature vectors
and the distance metrics are combined using an element-wise multiplication
denoted by $\odot$, as follows:

$[\mathbf{h}_{i}]_{k}=\sum_{j=1}^{N}\frac{1}{\\#\text{dist}_{i}(j,i)}\cdot\rho\left(\frac{1}{1+\text{dist}\left(j,i\right)}\right)\odot
f_{k}([\mathbf{x}_{j}]_{k})~{}~{}~{}.$

For prediction purposes, the sum operator is applied independently across the
dimensions corresponding to each class, and a softmax is employed as the
activation function.

##  4 Inteligibility

In this section, we demonstrate the intelligibility of GNAN through
visualizations. Each GNAN model is characterized by the univariate learned
shape functions $\rho$ and $\\{f_{k}\\}_{k=1}^{d}$, and can thus be depicted
as a set of illustrative figures. Below, we present examples of such figures
and explain their utility in generating insights. Our focus in this section is
on global interpretability, as local interpretability can utilize analogous
ways. We showcase GNAN’s application on two datasets, with additional examples
detailed in the Appendix.

Figure 1: Visualization of the distance and feature functions, learned on
Mutagenicity. As the features are binary, the feature functions are evaluated
only on the value $1$. These plots provide an exact description of the
functions’ signal processing and a global explanation of how the model uses
the distances and features.

Our initial examples focus on the task of detecting mutation-causing molecules
using the Mutagenicity dataset [47]. In this task, molecules are modeled as
graphs where nodes correspond to atoms and edges to connections between these
atoms. Each atom type is represented by a 14-dimensional one-hot encoding. A
GNAN model trained on this dataset is illustrated in Figure 1. On the left,
the function $\rho$ is presented, demonstrating how distance impacts
prediction, with a clear diminishing influence of more distant atoms. On the
right, the shape functions for the features are displayed. Given that the
features are binary, each shape function manifests only two values: one when
the feature is $0$ (indicating that the atom is not of the specified type),
and another when the feature is $1$ (indicating that the atom is of the
specified type). Defining $b=\sum_{k}f_{k}(0)$ as the bias term allows us to
set $f_{k}(0)=0$ for each $k$, thereby enabling the plotting of only
$f_{k}(1)$. The graphical representation reveals that atoms such as Ca, Na,
and Li are predicted to correlate with an increased mutagenicity effect,
whereas N and P atoms are predicted to be associated with a slight protective
effect.

It is essential to emphasize that Figure 1 displays the entire model
comprehensively. This means that combined with the value of the bias term,
which is $-5.6672$ in this case, every crucial detail needed to understand and
utilize this model for predictions is contained within this single figure.
This stands in stark contrast to methods like feature importance, which offer
a limited perspective on models. While the figure provides complete
information about the model, presenting additional views can sometimes be
helpful.

Figure 2 showcases the cross product of the shape functions and the distance
function as a heatmap. Each cell $(k,l)$ in the heatmap represents the value
$\rho\left(\nicefrac{{1}}{{(1+l)}}\right)\cdot f_{k}(1)$. This figure
illustrates the interplay the model has learned between the graph’s structure
and the attributes of its nodes. As the task involves binary classification,
positive values in the heatmap contribute to classifying a molecule as
mutagenic, whereas negative values indicate non-mutagenic properties.

This heatmap illustrates how atoms at specific distances influence the final
outcome. For instance, it shows that the model has learned that the presence
of a Ca atom (cell (Ca, 0)) or its proximity (cell (Ca, 1)) contributes to
mutagenicity. The visualizations can also be used for debugging purposes. This
can be crucial, for example, to ensure that the model is free from biases or
to identify any discrepancies with existing scientific knowledge. If it is
already known that Ca atoms actually have a negative effect on mutagenicity,
users could identify and correct this misalignment in the model’s learning.
Additionally, this detailed understanding allows users to select models that
not only achieve high accuracy on the given samples but also align with prior
knowledge, optimizing both performance and reliability.

Figure 2: Visualization of products of the outputs of the distance function
and the feature functions, trained on Mutagenicity. Each cell shows the exact
contribution, positive or negative, of features at a certain distance to the
prediction. Positive values (green) contribute to classifying a molecule as
mutagenic, and negative values (red) contribute to classifying a molecule as
non-mutagenic.

Interpreting multiclass prediction tasks poses significant challenges, as
noted by Zhang et al. [6]. In this context, we showcase the interpretability
of GNAN using the PubMed dataset [48]. This dataset comprises 19,717
scientific publications related to diabetes archived on PubMed and categorized
into three distinct classes (type-1 diabetes, type-2 diabetes, and gestational
diabetes). The dataset’s citation network includes 44,338 links. Each
publication, represented as a node, is characterized by a TF/IDF weighted word
vector derived from a dictionary containing 500 unique words.

As there are three classes, we trained the GNAN model such that the output of
the distance and feature functions are of dimension three. In this setting it
is interesting to compare the three functions, corresponding to the three
classes and therefore we draw them on the same figure [6]. Figure 3 shows that
the model uses only the local neighborhood of each node, and as nodes become
more distanced, the information between them is less used. We also observe a
difference between the classes; while for type 2 diabetes, the longer the
distance, the less their information is used (converges to 0), for type 1 and
gestational diabetes, nodes of long-distance have a negative effect.

In Figure 4, we display the feature shape functions for nine selected
features, demonstrating GNAN’s capability to learn complex, non-monotone
functions such as those seen in the ’diet’ and ’hepat’ features. Observing
these shape functions across the three classes simultaneously allows for an
understanding of how different feature values are utilized by the model to
distinguish among the classes. For instance, the shape function for the
’insulin’ feature reveals that the absence of this word in a document (i.e.,
feature values close to zero) does not significantly indicate the document’s
class. However, as the frequency of ’insulin’ increases within the document,
its impact on the prediction becomes more pronounced, though this effect
varies distinctly between type 1 & 2 diabetes and gestational diabetes.

To visualize the contribution of a feature value at a specific distance, we
employ a heatmap for each class, evaluating the products between the outputs
of the feature function over the input range ($[0,1]$) and the output of the
corresponding distance function. Figure 5 exemplifies this visualization
technique with the ’children’ feature. It is insightful to observe that the
presence of the word ’children’ influences the predictions differently across
the diabetes types. The model has learned that papers concerning type 1
diabetes seldom mention ’children’, nor do related papers. In contrast, the
term frequently appears in the context of gestational diabetes. Further
examples of GNAN visualizations are presented in the Appendix.

Figure 3: Visualization of the distance shape function learned on the PubMed
dataset. As the output of the function is of dimension three, we plot it as
three shape functions, one for each class. We plot them on the same figure to
compare them. The shape functions show that the model uses only the local
neighborhood of each node. It also shows a difference between the classes;
while for type 2 diabetes, the longer the distance, the less their information
is used (converges to 0), for type 1 and gestational diabetes, nodes of long-
distance have a negative effect.  Figure 4: Visualization of nine features’
shape functions, learned over the PubMed dataset. Figure 5: Visualization of
the products between the outputs of the ’children’ feature function over the
input range $[0,1]$ and the outputs of the distance function, learned over the
PubMed dataset.

##  5 Empirical evaluation

In this section, we evaluate GNAN on real-world graph and node labeling tasks,
including large-scale, long-range, and heterophily datasets.222The
implementation can be found at https://github.com/mayabechlerspeicher/Graph-
Neural-Additive-Networks---GNAN. We compare GNAN to multiple commonly used
black-box GNNs including GraphConv [49], GraphSAGE [30], Graph Isomorphism
Network (GIN) [33], the expressive version of the Graph Attention Network
(GATv2) [29, 50], the Graph Transformer (GTransformer) [51]. We also evaluate
the FSGNN model, which disentangles the node features from the graph structure
[35]. The information on the hyper-parameters tuned for each baseline can be
found in the Appendix. We used the following common benchmarks:

Node labeling tasks _Cora, Citeseer, PubMed, ogb-arxiv_ [52, 53] are paper
citation networks where the goal is to classify papers into one of several
topics. The ogb-arxiv dataset is a large-scale network.
_Cornell[54] & Tolokers [55]_ are heterophilious datasets. Cornell is a web-
link network with the task of classifying nodes into one of five categories.
Tolokers dataset is based on data from the Toloka crowdsourcing platform. The
nodes represent tolokers (workers) who have participated in at least one of 13
selected projects. An edge connects two tolokers if they have worked on the
same task. The goal is to predict which tolokers have been banned in one of
the projects. Node features are based on the worker’s profile information and
task performance statistics.
Graph labeling tasks _NCI1, Proteins, Mutagen & PTC_ [56] are datasets of
chemical compounds. In each dataset, the goal is to classify compounds
according to some property of interest.
Thr _ $\mu$ ,$\alpha$ ,$\alpha_{HOMO}$_ [57] datasets are long-range molecular
property prediction regression tasks, over the large-scale QM9 molecular
dataset.
Additional data information, including the data statistics, can be found in
the Appendix.

#### Protocol

For all tasks, we used existing splits, protocols, and metrics, as commonly
used in the literature for each dataset. The complete protocols for each
dataset are given in detail in the Appendix. The metrics we report are: For
Cornell, Cora, Citeseer, PubMed, ogb-arxiv, Mutagenicity, PTC, NCI, and
Proteins, we report accuracy. For $\mu$, $\alpha$, $\alpha_{HOMO}$ we report
MAE. For Tolokers we report ROC-AUC. For the node labeling tasks, we used the
pre-defined splits in the data and followed the common protocols for each
dataset. The results are an average of the test set using $5$ or $10$ random
seeds. For the Proteins and NCI1 tasks, we followed the splits and the nested-
cross-validation protocol from [58]. The final reported result on these
datasets is an average of 30 runs (10-folds and 3 random seeds). For NCI1 and
PTC we followed the splits and protocol from [39] and report the average
accuracy and std of a 10-fold nested cross-validation.

#### Results

The results are presented in Table 1. GNAN performed as the best or second-
best model in 9 out of the 13 tasks we evaluated. In GNAN, each node gathers
information from all others, ensuring complete information flow, while the
$\rho$ function modulates influence based on distance. Consequently, GNAN
avoids the computational bottlenecks encountered by some message-passing GNNs
[25]. Particularly in the long-range tasks $\mu$, $\alpha$, and
$\alpha_{HOMO}$, GNAN outperformed all other evaluated baselines, aligning
with findings by Alon and Yahav [25] that emphasize the benefits of capturing
long-range information. While intelligibility sometimes comes at the cost of
accuracy, our findings suggest that enhancing intelligibility does not
necessarily result in significant accuracy loss. It may appear surprising that
GNAN, despite its limited capacity, matches the accuracy of more expressive
GNNs. However, prior research indicates that even limited-capacity GNNs, such
as linear GNNs, can achieve high accuracy on various real-world datasets [59,
58, 60], suggesting that some real-world graph problems are simpler than
anticipated. Our results corroborate these observations.

Table 1: Evaluation of GNAN on node (top) and graph (bottom) tasks. The best
and second-best models are marked in cyan and violet colors, respectively. We
report accuracy and std for all tasks, except for the Tolokers dataset where
we report ROC-AUC and std, and the $\mu$, $\alpha$, $\alpha_{HOMO}$ datasets
where we report MAE and std.

Model | Cornell | Tolokers | Cora | Citeseer | PubMed | ogb-arxiv
---|---|---|---|---|---|---
GraphConv | 65.9 ± 0.5 | 83.5 ± 0.7 | 81.3 ± 1.1 | 75.9 ± 2.0 | 85.9 ± 0.5 | 72.4 ± 0.1
GraphSAGE | 75.9 ± 5.0 | 82.4 ± 0.4 | 81.4 ± 0.7 | 76.4 ± 0.8 | 88.4 ± 0.4 | 71.7 ± 0.2
GIN | 69.0 ± 1.3 | 81.0 ± 0.4 | 80.0 ± 1.2 | 77.1 ± 1.9 | 85.3 ± 0.9 | 73.8 ± 1.4
GATv2 | 72.5 ± 0.7 | 83.8 ± 1.1 | 83.1 ± 0.9 | 73.9 ± 1.5 | 84.4 ± 0.5 | 74.0 ± 2.1
GTransformer | 70.5 ± 1.7 | 83.3 ± 0.9 | 80.7 ± 0.5 | 76.0 ± 0.9 | 85.3 ± 1.6 | 73.1 ± 0.2
FSGNN | 86.0 ± 4.1 | 83.1 ± 0.6 | 83.0 ± 1.3 | 76.2 ± 1.3 | 85.0 ± 1.3 | 72.9 ± 1.7
GNAN | 85.7 ± 4.8 | 84.5 ± 0.9 | 81.1 ± 1.5 | 75.8 ± 0.6 | 86.9 ± 1.2 | 74.1 ± 1.5

Model | $\mu$ | $\alpha$ | $\alpha_{HOMO}$ | Proteins | Mutagen | PTC | NCI1
---|---|---|---|---|---|---|---
GraphConv | 2.91 ± 0.1 | 4.37 ± 0.5 | 1.46 ± 0.1 | 73.1 ± 1.6 | 64.3 ± 1.7 | 63.9 ± 5.0 | 76.5 ± 1.2
GraphSAGE | 3.55 ± 0.2 | 4.51 ± 0.7 | 1.44 ± 0.2 | 73.0 ± 4.5 | 64.1 ± 0.3 | 67.1±12.6 | 76.0 ± 1.8
GIN | 2.60 ± 0.1 | 4.67 ± 0.5 | 1.42 ± 0.1 | 73.3 ± 4.0 | 69.4 ± 1.2 | 55.6±11.1 | 80.0 ± 1.4
GATv2 | 2.72 ± 0.1 | 4.39 ± 0.6 | 1.41 ± 0.1 | 73.5 ± 2.8 | 72.0 ± 0.9 | 59.5 ± 2.1 | 80.4 ± 1.6
GTransformer | 2.90 ± 0.3 | 4.30 ± 0.5 | 1.41 ± 0.2 | 73.9 ± 1.5 | 73.1 ± 0.9 | 55.9 ± 3.5 | 80.5 ± 1.1
FSGNN | 3.57 ± 0.3 | 4.50 ± 0.4 | 1.44 ± 0.3 | 72.9 ± 2.1 | 66.9 ± 1.5 | 60.3 ± 7.2 | 79.7 ± 1.1
GNAN | 2.55 ± 0.1 | 4.28 ± 0.9 | 1.40 ± 0.1 | 73.2 ± 3.1 | 72.2 ± 1.0 | 64.9 ± 7.1 | 76.9 ± 1.2

##  6 Conclusion

In this work, we introduced the Graph Neural Additive Network (GNAN), a novel
extension of the interpretable class of Generalized Additive Models, to
accommodate graph data. GNAN is inherently interpretable, and provides both
global and local explanations directly from its architecture, eliminating the
need for post-hoc interpretations and thus enhancing transparency.
Furthermore, GNAN demonstrates competitive performance with popular GNNs,
showing that intelligibility does not necessarily entail a significant
degradation in accuracy.

It is possible to enhance GNAN in several ways. To generate smooth shape
functions, one could integrate techniques from the recently proposed
Kolmogorov–Arnold Networks [46]. Increasing the capacity of GNAN is feasible
by learning individual distance functions for each feature. Exploring reduced
capacity is also intriguing, particularly in scenarios with many features,
where it may be beneficial to employ regularization to limit the number of
shape functions used. Additionally, applying these techniques to biological
network datasets, such as protein interactions, could be a valuable tool to
support scientific discoveries. These and other directions are left for future
studies.

## Acknowledgements

This work was supported by the Tel Aviv University Center for AI and Data
Science (TAD) and the Israeli Science Foundation grants 1186/18 and 1437/22.

## References

  * Zhou et al. [2020] Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun.  Graph neural networks: A review of methods and applications.  _AI open_ , 1:57–81, 2020.
  * Rudin [2019] Cynthia Rudin.  Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, 2019.
  * Goodman and Flaxman [2017] Bryce Goodman and Seth Flaxman.  European union regulations on algorithmic decision-making and a “right to explanation”.  _AI magazine_ , 38(3):50–57, 2017.
  * Selbst and Powles [2018] Andrew Selbst and Julia Powles.  “meaningful information” and the right to explanation.  In _conference on fairness, accountability and transparency_ , pages 48–48. PMLR, 2018.
  * European Parliament [2024] European Parliament.  European parliament legislative resolution of 13 march 2024 on the proposal for a regulation of the european parliament and of the council on laying down harmonised rules on artificial intelligence (artificial intelligence act).  _OJ_ , 2024.  URL https://www.europarl.europa.eu/doceo/document/TA-9-2024-0138_EN.pdf.
  * Zhang et al. [2019] Xuezhou Zhang, Sarah Tan, Paul Koch, Yin Lou, Urszula Chajewska, and Rich Caruana.  Axiomatic interpretability for multiclass additive models.  In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, pages 226–234, 2019.
  * Bilodeau et al. [2024] Blair Bilodeau, Natasha Jaques, Pang Wei Koh, and Been Kim.  Impossibility theorems for feature attribution.  _Proceedings of the National Academy of Sciences_ , 121(2):e2304406120, 2024.
  * Vale et al. [2022] Daniel Vale, Ali El-Sharif, and Muhammed Ali.  Explainable artificial intelligence (xai) post-hoc explainability methods: Risks and limitations in non-discrimination law.  _AI and Ethics_ , 2(4):815–826, 2022.
  * Wexler [2017] Rebecca Wexler.  When a computer program keeps you in jail.  _The New York Times_ , 13:1, 2017.
  * McGough [2018] Michael McGough.  How bad is sacramento’s air, exactly? google results appear at odds with reality, some say.  _Sacramento Bee_ , 7, 2018.
  * Ghassemi et al. [2021] Marzyeh Ghassemi, Luke Oakden-Rayner, and Andrew L Beam.  The false hope of current approaches to explainable artificial intelligence in health care.  _The Lancet Digital Health_ , 3(11):e745–e750, 2021.
  * Lundberg and Lee [2017] Scott M Lundberg and Su-In Lee.  A unified approach to interpreting model predictions.  _Advances in neural information processing systems_ , 30, 2017.
  * Ribeiro et al. [2016] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  " why should i trust you?" explaining the predictions of any classifier.  In _Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining_ , pages 1135–1144, 2016.
  * Hooker et al. [2018] Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim.  Evaluating feature importance estimates.  _arXiv preprint arXiv:1806.10758_ , 2, 2018.
  * Friedman [2001] Jerome H Friedman.  Greedy function approximation: a gradient boosting machine.  _Annals of statistics_ , pages 1189–1232, 2001.
  * Saeed and Omlin [2023] Waddah Saeed and Christian Omlin.  Explainable ai (xai): A systematic meta-survey of current challenges and future opportunities.  _Knowledge-Based Systems_ , 263:110273, 2023.
  * Laberge et al. [2024] Gabriel Laberge, Yann Batiste Pequignot, Mario Marchand, and Foutse Khomh.  Tackling the xai disagreement problem with regional explanations.  In _International Conference on Artificial Intelligence and Statistics_ , pages 2017–2025. PMLR, 2024.
  * Franzoni [2023] Valentina Franzoni.  From black box to glass box: advancing transparency in artificial intelligence systems for ethical and trustworthy ai.  In _International Conference on Computational Science and Its Applications_ , pages 118–130. Springer, 2023.
  * Hastie and Tibshirani [1986] Trevor Hastie and Robert Tibshirani.  Generalized Additive Models.  _Statistical Science_ , 1(3):297 – 310, 1986.  doi: 10.1214/ss/1177013604.  URL https://doi.org/10.1214/ss/1177013604.
  * Hastie and Tibshirani [1995] Trevor Hastie and Robert Tibshirani.  Generalized additive models for medical research.  _Statistical methods in medical research_ , 4(3):187–196, 1995.
  * Hastie and Tibshirani [1987] Trevor Hastie and Robert Tibshirani.  Generalized additive models: Some applications.  _Journal of the American Statistical Association_ , 82(398):371–386, 1987.  doi: 10.1080/01621459.1987.10478440.
  * Yee and Mitchell [1991] Thomas W Yee and Neil D Mitchell.  Generalized additive models in plant ecology.  _Journal of vegetation science_ , 2(5):587–602, 1991.
  * Rigby and Stasinopoulos [2005] Robert A Rigby and D Mikis Stasinopoulos.  Generalized additive models for location, scale and shape.  _Journal of the Royal Statistical Society Series C: Applied Statistics_ , 54(3):507–554, 2005.
  * Caruana et al. [2015] Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad.  Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission.  In _Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining_ , pages 1721–1730, 2015.
  * Alon and Yahav [2021] Uri Alon and Eran Yahav.  On the bottleneck of graph neural networks and its practical implications, 2021.
  * Agarwal et al. [2021] Rishabh Agarwal, Levi Melnick, Nicholas Frosst, Xuezhou Zhang, Ben Lengerich, Rich Caruana, and Geoffrey Hinton.  Neural additive models: Interpretable machine learning with neural nets, 2021.
  * Kipf and Welling [2017a] Thomas N. Kipf and Max Welling.  Semi-supervised classification with graph convolutional networks.  In _International Conference on Learning Representations_ , 2017a.  URL https://openreview.net/forum?id=SJU4ayYgl.
  * Gilmer et al. [2017] Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl.  Neural message passing for quantum chemistry, 2017.
  * Veličković et al. [2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio.  Graph attention networks.  In _International Conference on Learning Representations_ , 2018.
  * Hamilton et al. [2018] William L. Hamilton, Rex Ying, and Jure Leskovec.  Inductive representation learning on large graphs, 2018.
  * Ying et al. [2021] Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu.  Do transformers really perform bad for graph representation?, 2021.
  * Wang et al. [2024] Chloe Wang, Oleksii Tsepa, Jun Ma, and Bo Wang.  Graph-mamba: Towards long-range graph sequence modeling with selective state spaces, 2024.
  * Xu et al. [2019] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka.  How powerful are graph neural networks?  In _International Conference on Learning Representations_ , 2019.  URL https://openreview.net/forum?id=ryGs6iA5Km.
  * Baranwal et al. [2023] Aseem Baranwal, Kimon Fountoulakis, and Aukosh Jagannath.  Optimality of message-passing architectures for sparse graphs.  In _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023.  URL https://openreview.net/forum?id=d1knqWjmNt.
  * Maurya et al. [2021] Sunil Kumar Maurya, Xin Liu, and Tsuyoshi Murata.  Improving graph neural networks with simple architecture design, 2021.
  * Frasca et al. [2020] Fabrizio Frasca, Emanuele Rossi, Davide Eynard, Ben Chamberlain, Michael Bronstein, and Federico Monti.  Sign: Scalable inception graph neural networks, 2020.
  * Bechler-Speicher et al. [2024a] Maya Bechler-Speicher, Ido Amos, Ran Gilad-Bachrach, and Amir Globerson.  Graph neural networks use graphs when they shouldn’t, 2024a.  URL https://arxiv.org/abs/2309.04332.
  * Chami et al. [2022] Ines Chami, Sami Abu-El-Haija, Bryan Perozzi, Christopher Ré, and Kevin Murphy.  Machine learning on graphs: A model and comprehensive taxonomy.  _Journal of Machine Learning Research_ , 23(89):1–64, 2022.
  * Bechler-Speicher et al. [2024b] Maya Bechler-Speicher, Amir Globerson, and Ran Gilad-Bachrach.  Tree-g: Decision trees contesting graph neural networks.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 38, pages 11032–11042, 2024b.
  * Luo et al. [2020] Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang.  Parameterized explainer for graph neural network, 2020.
  * Ying et al. [2019] Rex Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec.  Gnnexplainer: Generating explanations for graph neural networks, 2019.
  * Amara et al. [2022] Kenza Amara, Rex Ying, Zitao Zhang, Zhihao Han, Yinan Shan, Ulrik Brandes, Sebastian Schemm, and Ce Zhang.  Graphframex: Towards systematic evaluation of explainability methods for graph neural networks, 2022.
  * Yin et al. [2023] Jun Yin, Chaozhuo Li, Hao Yan, Jianxun Lian, and Senzhang Wang.  Train once and explain everywhere: Pre-training interpretable graph neural networks.  In _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023.  URL https://openreview.net/forum?id=enfx8HM4Rp.
  * Yu et al. [2020] Junchi Yu, Tingyang Xu, Yu Rong, Yatao Bian, Junzhou Huang, and Ran He.  Graph information bottleneck for subgraph recognition, 2020.
  * Wu et al. [2022] Ying-Xin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat-Seng Chua.  Discovering invariant rationales for graph neural networks, 2022.
  * Liu et al. [2024] Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y. Hou, and Max Tegmark.  Kan: Kolmogorov-arnold networks, 2024.
  * Kazius et al. [2005] Jeroen Kazius, Ross McGuire, and Roberta Bursi.  Derivation and validation of toxicophores for mutagenicity prediction.  _Journal of medicinal chemistry_ , 48(1):312–320, 2005.
  * Sen et al. [2008] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad.  Collective classification in network data.  _AI magazine_ , 29(3):93–93, 2008.
  * Morris et al. [2021] Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe.  Weisfeiler and leman go neural: Higher-order graph neural networks, 2021.
  * Brody et al. [2022] Shaked Brody, Uri Alon, and Eran Yahav.  How attentive are graph attention networks?, 2022.
  * Shi et al. [2021] Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjin Wang, and Yu Sun.  Masked label prediction: Unified message passing model for semi-supervised classification, 2021.
  * Yang et al. [2016] Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov.  Revisiting semi-supervised learning with graph embeddings, 2016.  URL https://arxiv.org/abs/1603.08861.
  * Hu et al. [2020] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec.  Open graph benchmark: Datasets for machine learning on graphs, 2020.  URL https://arxiv.org/abs/2005.00687.
  * Pei et al. [2020] Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang.  Geom-gcn: Geometric graph convolutional networks.  In _International Conference on Learning Representations_ , 2020.  URL https://openreview.net/forum?id=S1e2agrFvS.
  * Likhobaba et al. [2023] Daniil Likhobaba, Nikita Pavlichenko, and Dmitry Ustalov.  Toloker Graph: Interaction of Crowd Annotators, 2023.  URL https://github.com/Toloka/TolokerGraph.
  * Morris et al. [2020] Christopher Morris, Nils M. Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann.  Tudataset: A collection of benchmark datasets for learning with graphs.  In _ICML 2020 Workshop on Graph Representation Learning and Beyond (GRL+ 2020)_ , 2020.  URL www.graphlearning.io.
  * Ramakrishnan et al. [2014] Raghunathan Ramakrishnan, Pavlo Dral, Matthias Rupp, and Anatole von Lilienfeld.  Quantum chemistry structures and properties of 134 kilo molecules.  _Scientific Data_ , 1, 08 2014.  doi: 10.1038/sdata.2014.22.
  * Errica et al. [2022] Federico Errica, Marco Podda, Davide Bacciu, and Alessio Micheli.  A fair comparison of graph neural networks for graph classification, 2022.
  * Wu et al. [2019] Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger.  Simplifying graph convolutional networks.  In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, _Proceedings of the 36th International Conference on Machine Learning_ , volume 97 of _Proceedings of Machine Learning Research_ , pages 6861–6871. PMLR, 09–15 Jun 2019.  URL https://proceedings.mlr.press/v97/wu19e.html.
  * Yang et al. [2023] Chenxiao Yang, Qitian Wu, Jiahua Wang, and Junchi Yan.  Graph neural networks are inherently good generalizers: Insights by bridging gnns and mlps, 2023.
  * Paszke et al. [2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala.  Pytorch: An imperative style, high-performance deep learning library, 2019.
  * Fey and Lenssen [2019] Matthias Fey and Jan Eric Lenssen.  Fast graph representation learning with pytorch geometric, 2019.
  * Platonov et al. [2023] Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova.  A critical look at the evaluation of GNNs under heterophily: Are we really making progress?  In _The Eleventh International Conference on Learning Representations_ , 2023.  URL https://openreview.net/forum?id=tJbbQfw-5wv.
  * Kipf and Welling [2017b] Thomas N. Kipf and Max Welling.  Semi-supervised classification with graph convolutional networks, 2017b.

##  Appendix A Efficient GNAN implementation with tensor products

We formulate GNAM case of classification with $C$ classes. For regression, the
exact formulation holds with $C=1$. For the sake of notation, we assume that
every tensor that has its last dimension, $C$, is permuted to have the last
dimension as its first dimension, without stating it explicitly. This is
necessary to achieve a valid tensor multiplication.

GNAM linearly combines the outputs of at least $d+1$ neural networks $m$ and
$\\{f_{l}\\}_{l=1}^{d}$.

$f_{l}:\mathbb{R}\rightarrow\mathbb{R}^{1\times C}$ is a feature-
transformation function, and it acts on the $l^{\prime}th$ feature of some
given node $i$, i.e., $f_{l}(x^{i}_{l})$

$m:\mathbb{R}\rightarrow\mathbb{R}^{1\times C}$ is a distance-transformation
function, and it acts on distances between nodes, i.e., $m(dist(j,i))$.

Given $A$ , we build the distance matrix $D$ , where $D_{i,j}=dist(j,i)$

We denote with $M$ the matrix of the transformed distances that is outputted
by applying $m$ to each entry of $D$, i.e., $M_{i,j}=m(dist(j,i))$,
$M\in\mathbb{R}^{C\times N\times N}$.

We denote with $F$ the matrix of the transformed feature is outputted by
applying the corresponding $f_{l}$ for feature $l$ of each node in the graph,
i.e., $F_{il}=f_{l}(x_{l}^{i})$, $F\in\mathbb{R}^{C\times N\times d}$

Any GNAM, both for node and graph tasks, first computes the matrix $M\cdot
F\in\mathbb{R}^{C\times N\times d}$

Then, depending on the task.

###  A.1 Node Tasks

We aggregate the transformed features weighted by the transformed distances.
This is done by summing over the rows of $M\otimes F$ :

$\phi(i)=[M\otimes F\otimes\mathbbm{1}_{C\times d\times
1}]_{i}=\sum_{l=1}^{d}\sum_{j=1}^{N}m(dist(j,i))\otimes f_{l}(x_{l}^{j})$

###  A.2 Graph Tasks

For graph classification, we set $m$ and $f$ to output a scalar and aggregate
the transformed features over the nodes, i.e., the row of $M\cdot F$, to form
a fixed-size vector of size $d$.

$\bar{\Phi}(G)=\mathbbm{1}_{C\times 1\times N}\otimes M\otimes
F\in\mathbb{R}^{C\times 1\times d}$

Then, we can apply another readout NAM:

$\Phi(G)=\bar{F}(\bar{\Phi}(G))$

Such that $\bar{F}$ is the transformed features using the function
$\\{\bar{f}_{l}\\}_{l=1}^{d}$ such that
$\bar{f}_{l}:\mathbb{R}\rightarrow\mathbb{R}^{1\times C}$

We can also simply sum over the outputs. In that case, we will set $f$ and $m$
to output a vector of dimension $C$:

$\Phi(G)=\sum_{i=1}^{N}\phi(i)=\mathbbm{1}_{C\times 1\times N}\otimes M\otimes
F\otimes\mathbbm{1}_{C\times d\times 1}$

##  Appendix B Extensions and ablations

In Section 3 we mentioned several possible extensions for GNAN. Here, we
discuss them in detail.

#### Readout layer for graph tasks

In graph tasks, after aggregating the node representations, it is possible to
apply another transformation before aggregating over the entries of the graph
representations. There may be many ways to do so, and we did not explore all
of them. We did explore an application of another set of feature functions to
each feature or the graph representation vector. This approach increases the
capacity of the model in the cost of interpretability. This is because the set
of addition feature functions should be plotted separately, and the product
between the feature function and the distance functions does not affect the
final output directly but rather through another feature function.
Empirically, we observed this approach did not improve performance with
respect to the performance reported in Section 5.

#### Splines

It is possible to learn splines for the activations in each feature network to
achieve smoother shape functions [46]. We note that the Tolokers example
presented in Section C shows that the learned feature shape function is
smooth, although we use ReLU activations. Nonetheless, in other cases, such as
in the PubMed example presented in the main paper, many of the learned feature
functions are step functions. Therefore, it is likely that the model could
benefit from spline activations, to smooth its shape functions.

#### Normalization

In GNAN we normalize the weight of nodes of distance $l$ with the number of
nodes of distance $l$, so that the cumulative weight of nodes of distance $l$
will be $\rho(\nicefrac{{1}}{{(1+l)}})$. We examined the effect of removing
this normalization. We observed that without normalization, the loss scale is
drastically larger. Therefore, more epochs are required to fit the data. As a
result, for the fixed number of epochs we used in our experiments (1000),
without normalization, the accuracy decreases.

##  Appendix C Additional explanations examples

In the main paper, we presented two examples of explanations over two datasets
with different properties. In this section, we present additional explanations
and examples we could not fit into the main text due to space limitations.

###  C.1 Additional PubMed heatmaps

In the main text, we presented the heatmaps for the ’children’ feature. Here
we provide additional heatmaps for additional features: the ’fat’ feature and
the ’young’ feature, as presented in Figures 6 and 7.

Figure 6: Visualization of the products between the outputs of the ’fat’
feature function over the input range [0, 1] and the outputs of the distance
function, learned over the PubMed dataset Figure 7: Visualization of the
products between the outputs of the ’young’ feature function over the input
range [0, 1] and the outputs of the distance function, learned over the PubMed
dataset

###  C.2 Tolokers - Binary classification with binary and continuous features

Tolokers dataset is based on data from the Toloka crowdsourcing platform. The
nodes represent tolokers (workers) who have participated in at least one of 13
selected projects. An edge connects two tolokers if they have worked on the
same task. The goal is to predict which tolokers have been banned in one of
the projects. Node features are based on the worker’s profile information and
task performance statistics. Each node in the graph is associated with 9
features. There are 4 continuous features in the range $[0,1]$ and 5 are
binary features. Figure 8 shows the shape functions of the features learned by
GNAN. Figure 9 shows the distance shape function learned by GNAN. In Figures
11 and 10 we present the heatmaps of the cross product of the shape functions
and the distance function.

Figure 8: Visualization of the feature functions learned over the Tolokers
dataset. Figure 9: isualization of the distance shape function learned on the
Tolokers dataset. Figure 10: Visualization of the products between the outputs
of the continuous feature functions over the input range $[0,1]$ and the
outputs of the distance function, learned over the Tolokers dataset.  Figure
11: Visualization of products of the outputs of the distance function and the
feature functions, trained on Tolokers. Each cell shows the exact
contribution, positive or negative, of features at a certain distance to the
prediction. Positive values (green) contribute to classifying a toloker as
’banned’, and negative values (red) contribute to classifying a toloker as
’not banned’.

##  Appendix D Additional experimental details

All our baselines are implemented using PyTorch [61] and PyTorch-Geometric
[62].

###  D.1 Dataset information

Here we provide additional information about the datasets used in Section 5.
The data statistics are given in Table 2.

Proteins [56] is a dataset of chemical compounds consisting of $1113$ graphs,
respectively. The goal in the first two datasets is to predict whether a
compound is an enzyme or not, and the goal in the last datasets is to classify
the type of an enzyme among $6$ classes.

NCI1 [56] is a datasets consisting of $4110$ graphs, representing chemical
compounds. Vertices and edges represent atoms and the chemical bonds between
them. The graphs are divided into two classes according to their ability to
suppress or inhibit tumor growth.

Mutagenicity [56] is a dataset consisting of $4337$ chemical compounds of
drugs divided into two classes: mutagen and non-mutagen. A mutagen is a
compound that changes genetic material such as DNA, and increases mutation
frequency.

PTC [56] is a dataset consisting of $344$ chemical compounds divided into two
classes according to their carcinogenicity for rats.

Cornell [54] is a heterophilic webpage dataset collected from the computer
science department at Cornell University. Nodes represent web pages, and edges
are hyperlinks between them. The task is to classify the nodes into one of
five categories.

Table 2: Statistics of the real-world datasets used in our evaluation. Dataset | # Graphs | Avg # Nodes | Avg # Edges | # Node Features | # Classes
---|---|---|---|---|---
Proteins [56] | 1,113 | 39.06 | 72.82 | 3 | 2
NCI1 [56] | 4,110 | 29.87 | 32.3 | 37 | 2
Mutagenicity [56] | 4,337 | 30.32 | 30.37 | 7 | 2
PTC [56] | 344 | 14 | 14 | 19 | 2
QM9 [57] | 130,831 | 18 | 37.3 | 11 | -
Cora [52] | 1 | 2,708 | 10,556 | 1,433 | 7
Citeseer [52] | 1 | 3,327 | 9,104 | 3,703 | 6
PubMed [52] | 1 | 19,717 | 88,648 | 500 | 3
ogb-arxiv [53] | 1 | 169,343 | 1,166,243 | 128 | 40
Cornell [54] | 1 | 183 | 295 | 1,703 | 5
Tolokers [55] | 1 | 11758 | 519000 | 10 | 2

###  D.2 Protocols

#### ogb-arxive

The ogb-arxive datasets are large-scale datasets provided in the Open Graph
Benchmark (OGB) paper [53] with pre-defined train and test splits and
different metrics and protocols for each dataset. As common in the literature
when evaluating OGB datasets, we followed its pre-defined metric and protocol.
The metric used is accuracy. We ran GNAN $10$ times and reported the mean
accuracy and std over the runs.

#### Cornell

For the Cornell dataset we used the splits and protocol from [54] and report
the test accuracy averaged over $10$ runs, using the best hyper-paremeters
found on the validation set.

#### Tolokers

For the Tolokers dataset, we followed the protocol and pre-defined splits from
[55, 63]. The reported result is an average of a $10$-fold nested cross-
validation.

#### Core, Citeseer and PubMed

Following [64, 30, 29], for the Core, Citeseer and Pubmed datasets we tuned
the parameters on the Cora dataset using the pre-defined splits from [64]. For
all these datasets we report the test accuracies averaged over $5$ runs, using
the parameters obtained from the best accuracy on the validation set of Cora.

#### Proteins, NCI

We used $10$-fold nested cross validation with the splits and protocol of
Errica et al. [58]. The final reported result on these datasets is an average
of $30$ runs ($10$-folds and $3$ random seeds).

#### Mutagenicity, PTC

We use the splits and protocols from [39], and use a $10$-fold nested cross-
validation. The final reported test accuracies are averages over the $10$ test
sets of the outer $10$ folds.

###  D.3 Hyperparameters

All GNNs (excluded GNAN) use ReLU activations with $\\{3,5\\}$ layers and $64$
hidden channels. They were trained with Adam optimizer over $1000$ epochs and
early on the validation loss with a patient of $100$ steps, eight Decay of
$1e-4$, learning rate in $\\{1e-3,1e-4$}, dropout rate in $\\{0,0.5\\}$, and a
train batch size of $32$.

In GNAN, all the feature and distance networks use ReLU activations with
$\\{3,5\\}$ layers and $\\{64,32\\}$ hidden channels. They were trained with
Adam optimizer over $1000$ epochs weight decay of $0,5e-4$, learning rate in
$\\{1e-2,1e-3$}, dropout rate in $\\{0,0.6\\}$.

###  D.4 Compute resources

All experiments ran on an NVIDIA GeForce RTX 3090 GPU.


# OpenViDial: A Large-Scale, Open-Domain Dialogue Dataset
with Visual Contexts

Yuxian Meng♣, Shuhe Wang♠, Qinghong Han♣
Xiaofei Sun♣, Fei Wu◆, Rongbin Ouyang♠, Rui Yan★ and Jiwei Li◆♣
◆Zhejiang University, ♠Computer Center of Peking University
★Gaoling School of Artificial Intelligence, Renmin University of China
♣ Shannon.AI
{yuxian_meng, qinghong_han, xiaofei_sun<EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

When humans converse, what a speaker will say next significantly depends on
what he sees. Unfortunately, existing dialogue models generate dialogue
utterances only based on preceding textual contexts, and visual contexts are
rarely considered. This is due to a lack of a large-scale multi-module
dialogue dataset with utterances paired with visual contexts.

In this paper, we release OpenViDial, a large-scale multi-module dialogue
dataset. The dialogue turns and visual contexts are extracted from movies and
TV series, where each dialogue turn is paired with the corresponding visual
context in which it takes place. OpenViDial contains a total number of 1.1
million dialogue turns, and thus 1.1 million visual contexts stored in
images.111Dataset is found at https://github.com/ShannonAI/OpenViDial.

##  1 Introduction

Giving machines the ability to converse like humans in the open domain is a
key point towards passing the Turing test (Turing, 2009), and developing open-
domain dialogue agents is of growing interest (Li et al., 2017; Ghazvininejad
et al., 2017; Zhou et al., 2017; Gao et al., 2018; Asghar et al., 2018; Zhou
et al., 2020). Existing approaches towards developing open-domain dialogue
agents are mostly data-driven, for which a large-scale dataset is first
collected. The dataset usually consists of millions of turns of dialogue
utterances from real human conversations. A neural model is then trained on
the dataset, learning to predict the upcoming dialogue turn conditioned on the
previous textual contexts. Li et al. (2016b, a); Zhang et al. (2018); Huang et
al. (2020)

One important aspect that existing open-domain dialogue models miss is the
consideration of multi-modal features in dialogue, especially visual features.
When humans converse, what a speaker should say next significantly depends on
what he sees. The granularity of visual features could be as large as the
location that a conversation takes place in (e.g., a cafeteria or a theater),
or as small as his dialogue partner’s facial expressions. For example, in
Figure 1, we present two short conversations where visual contexts are
crucial. In both examples, if the model has no access to visual information,
it is hard to correctly generate dialogue utterances “see the picture” and
“moving to the attic” in response to the preceding contexts. Unfortunately,
existing dialogue models generate dialogue utterances only based on preceding
textual contexts and no visual contexts are considered. This is because of the
lack of a large-scale multi-modal dialogue dataset with utterances paired with
visual context.

Figure 1: Two examples drawn from OpenViDial showing the necessity of
considering visual contexts for dialogues.

In this paper, we collect and release OpenViDial, a large-scale open-domain
dialogue dataset with visual contexts. The dialogue turns and visual contexts
are extracted from movies and TV series, where each dialogue turn is paired
with the corresponding visual context in which it takes place. OpenViDial
contains a total number of 1.1 million dialogue turns, and thus 1.1 million of
visual contexts stored in images.

##  2 Related Work

###  2.1 Existing Dialog Datasets

#### Open Domain Dialog Datasets

Over the past few years, various open-domain dialog datasets have been
developed. The OpenSubtitle dataset (Tiedemann, 2009, 2012; Lison and
Tiedemann, 2016) consists of large-scale movie conversations extracted from
the OpenSubtitle website. It includes a total number of 1,782 bitexts with
3.35G sentence fragments. The Twitter Triple Corpus (Sordoni et al., 2015)
consists of 4,232 Twitter conversation triples evaluated from 33K candidate
triples by human raters, with 2,118 triples as tuning set and 2,114 as test
set. The Cornell Movie-Dialogs Corpus (Danescu-Niculescu-Mizil and Lee, 2011)
contains a collection of fictional conversations extracted from raw movie
scripts. Other plain-text dialog datasets include the Ubuntu Dialog Corpus
(Lowe et al., 2015), PersonaChat (Zhang et al., 2018), EmpatheticDialogues
(Rashkin et al., 2018), etc. The datasets described above only consist of
texts in the form of dialogues, with no visual information included.

#### Visual Dialog Datasets

The task of Visual Dialog is first introduced by Das et al. (2017a), where a
model is required to answer a series of questions grounded in an image, given
a dialog history and the image itself as contexts. Further, Das et al. (2017a)
released the VisDial v0.9 and v1.0 datasets as benchmarks. The v1.0 dataset
contains 120K images from MS COCO222http://mscoco.org/ and each image is
associated with 10 rounds of question-answer dialog, making up 1.2M examples
in total. The GuessWhat?! dataset (de Vries et al., 2017) focuses on high-
level image understanding and is more goal-oriented: models need to locate an
unknown object in an informative image scene by answering a sequence of “yes
or no” questions. The CLEVER-Dialog (Kottur et al., 2019) and MNIST-Dialog
(Seo et al., 2017) datasets are developed for diagnostic purposes. They are
crafted to test the reasoning capability of visual dialog models based on the
image and prior dialog turns. More recently, the Audio Visual Scene-Aware
Dialog (AVSD) dataset (Hori et al., 2018; Alamri et al., 2019) was introduced.
It contains more than 11,000 conversations paired with videos of human-
centered activities, serving as a benchmark for the scene-aware video dialog
task. The datasets described above mainly focus on answering questions
regarding an image or video, and thus are more concerned about question
answering rather than dialogue generation.

###  2.2 Dialogue Generation Models

#### Open Domain Dialog Generation

Building open-domain dialog systems that can converse with humans has a long
history in natural language processing (Weizenbaum, 1966; COLBY, 1975;
Wallace, 2009). Recent advances of neural networks have spurred great
interests in developing neural-based data-driven dialog models (Vinyals and
Le, 2015; Li et al., 2015; Dodge et al., 2016; Serban et al., 2016; Zhao et
al., 2017; Xie et al., 2017; Lee et al., 2019; Ghandeharioun et al., 2019; Li,
2020; Han et al., 2020; Zhang et al., 2019; Roller et al., 2020). Built on top
of sequence-to-sequence frameworks (Sutskever et al., 2014; Vaswani et al.,
2017), neural-based dialog models are able to generate coherent (Li et al.,
2016b, 2017; Tian et al., 2017; Bosselut et al., 2018; Adiwardana et al.,
2020), diverse (Xu et al., 2018; Baheti et al., 2018; Tao et al., 2018),
personalized (Li et al., 2016a; Luan et al., 2017; Zhang et al., 2018; Zheng
et al., 2019a, b; Madotto et al., 2019), informative (Shao et al., 2017; Lewis
et al., 2017; Ghazvininejad et al., 2017; Young et al., 2017; Zhao et al.,
2019) and knowledge-fused (Hua et al., 2020; Zhao et al., 2020; He et al.,
2020) responses, as well as bias toward different specific attributes or
topics (Xing et al., 2016; Zhou et al., 2017; Wang et al., 2017; Niu and
Bansal, 2018; See et al., 2019).

#### Visual Dialog Generation

Since natural utterances and visual images are in different modalities,
attention mechanisms to model the interplay between conversational utterances
and visual contents are widely used (Lu et al., 2017; Kottur et al., 2018;
Jiang et al., 2019; Yang et al., 2019; Guo et al., 2019; Niu et al., 2019;
Kang et al., 2019; Park et al., 2020; Jiang et al., 2020b). Seo et al. (2017)
employed memories to store (attention, key) pairs that can be used to retrieve
the most relevant attention maps for the current question in text. Schwartz et
al. (2019) designed the factor graph attention model to connect an arbitrary
number of modalities with attention flows. Gan et al. (2019) proposed ReDAN, a
recurrent dual attention network enhanced by a multi-step reasoning mechanism.
Techniques such as reinforcement learning (Das et al., 2017b; Wu et al.,
2018), variational auto-encoders (Massiceti et al., 2018) and graph networks
(Zheng et al., 2019c; Jiang et al., 2020a) have also been applied to deal with
the visual dialog task. Empowered by large-scale pretraining techniques,
pretraining based models have made promising progress (Lu et al., 2019; Tan
and Bansal, 2019; Su et al., 2019; Alberti et al., 2019; Li et al., 2019a, b;
Chen et al., 2019; Wang et al., 2020; Li et al., 2020), significantly boosting
the performances in terms of different metrics.

Number of turns | 1.1M
---|---
Number of images | 1.1M
Vocab size before BPE | 70K
Vocab size after BPE | 30K
Average length of each episode | 14
Average length of each turn | 7.6
Table 1: Detailed statistics for OpenViDial

##  3 Constructing OpenViDial

In this section, we describe the details for OpenViDial construction. The main
idea of dataset generation is to pair conversation scripts with images in
movies or TV series, and use these images as visual contexts for dialogue
learning.

We collect a raw dataset containing English movies and TV series with a total
length of roughly 8,000 hours. Each second of videos can be further divided
into 20$\sim$40 frames, where each frame is an image.

###  3.1 Subtitle Extraction based on OCR

Because of the fact that only a small proportion of movies readily come with
subtitle files, and that for most movies, subtitles are embedded in images, we
need to build models to extract conversation scripts from images. To build a
conversation dataset with millions of turns of image-text pairs, it is
prohibitively expensive and time-intensive to employ human labors to separate
each image frame with embedded scripts. We thus rely on the technique of
optical character recognition (OCR) for automatic extraction of conversation
subtitles from movie images.333An alternative is to extract scripts from
audios. We find extracting scripts using OCR from images obtains a much higher
accuracy than speech recognition from audios. We thus adopt the former
strategy.  We tailor the OCR model to the task of subtitle extraction, and
achieves an almost perfect accuracy.

Existing open-sourced OCR models are not fit for our purpose since they are
not tailored to subtitle extraction in the context of movies and TV series. We
thus need to train our own OCR model.

#### Training Data Generation

We first synthesize the OCR training dataset, where we embed texts into images
to form training examples. To achieve this goal, we first need to collect
text-free images from raw videos, to which texts will be later added. This is
done by running an existing open-sourced OCR
model444https://github.com/JaidedAI/EasyOCR on video images, and pick images
with no text character identified by the model. Since at this stage, our goal
of identifying whether an image contains text character is a relatively easy
task555This task can be make even easier by sacrificing recall (images without
characters) for precision, by making sure that all selected images do not
contain characters. , a super accurate OCR model is not necessary and the
open-sourced OCR model suffices to fulfill our need. With text-free images in
hand, we pair them with texts. Texts are randomly selected from the
CommonCrawl English corpus, then added to the images. Texts in images are
generated using different fonts666https://www.myfonts.com/WhatTheFont/ and
sizes. We generated a dataset containing about 10M images paired with texts.

Dataset | Genre | Multi-Modal? | # Sentences | # Images
---|---|---|---|---
OpenSubtitles 2016 (Lison and Tiedemann, 2016) | Plain-text Dialog | ✗ | 337M | –
Cornell Movie-Dialogs (Danescu-Niculescu-Mizil and Lee, 2011) | Plain-text Dialog | ✗ | 0.3M | –
VisDial v1.0 (Das et al., 2017a) | VQA | ✓ | 2.4M | 120K
Guess-What?! (de Vries et al., 2017) | VQA | ✓ | 0.8M | 66K
AVSD (Alamri et al., 2019) | VQA | ✓ | 152K | –
OpenViDial (this work) | Visual+Text Dialog | ✓ | 1.1M | 1.1M

Table 2: A comparison of different datasets. VQA: Visual Question Answering.

#### Model Training

Standard OCR training involves two stages, the detection of the bounding box
of texts, and the recognition of characters. For detection, we use the PSE
model as the backbone Wang et al. (2019), which is built upon the FPN model He
et al. (2016) with ResNet pre-trained on ImageNet dataset. For recognition, we
use the Convolutional Recurrent Neural Network (CRNN) model Shi et al. (2016)
as the backbone. We omit the details since the discussion on training OCR
models is beyond the scope of this paper. We use a held-out dataset for
evaluation, and the trained OCR model gets an accuracy higher than 99.98$\%$
at character level and 98.4$\%$ at the image/sentence level.

#### Post Processing

The trained OCR model is applied to videos and TV series for script
extraction. Since each second of the video consists of 20$\sim$40 frames, most
of which are nearly identical, we pick 3 frames for each second and discard
the rest. We also construct an English vocabulary with top 200,000 words by
frequency using a part of the CommonCrawl dataset, and remove images with
unknown word from the vocabulary. This further helps us remove the influence
from incorrect characters by the OCR model. In addition, the following
scenarios need to be handled: (1) there are cases where a consecutive number
of images are paired with the same texts. We only preserve the middle image
and abandon the rest; (2) There are cases where a full dialogue turn is
truncated into multiple consecutive images, with each image containing only
part of the text in that dialogue turn. We train a simple discriminative model
to identify whether a word in a context is the end of a sentence. Using this
model, we merge texts from multiple images into a single turn and pair the
text with the middle image.

###  3.2 Statistics for OpenViDial

We collect a final dataset of 1.1M turns, where each turn consists of a
sequence of words and an image. The size of the image is either $1280\times
720$ or $1920\times 1080$ based on different video resources. We employ the
BPE tokenizer (Sennrich et al., 2016) for text processing. The detailed
statistics for OpenViDial are shown in Table 1. We split the dataset into
1M/50K/50K for training, dev and test.

Table 2 shows the comparison between different datasets. Comparing against
OpenSubtitles (Lison and Tiedemann, 2016), OpenViDial has fewer sentences but
contains multi-modal features. Additionally, the OpenSubtitles dataset is an
extremely noisy dataset, where consecutive lines may not appear in the same
conversation or scene, and may not even be spoken by the same character.
Comparing with other datasets with visual features, i.e., VisDial, Guess-
What?! and AVSD, OpenViDial focuses more on dialogue learning rather than
question answering.

##  4 Conclusion

In this paper, we release OpenViDial, a large-scale open-domain dialogue
dataset with visual contexts. In OpenViDial, each dialogue turn is paired with
the corresponding visual context in which it takes place. Our work marks an
important step towards large-scale multi-modal dialogue learning.

## References

  * Adiwardana et al. (2020) Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, et al. 2020\.  Towards a human-like open-domain chatbot.  _arXiv preprint arXiv:2001.09977_.
  * Alamri et al. (2019) Huda Alamri, Vincent Cartillier, Abhishek Das, Jue Wang, Anoop Cherian, Irfan Essa, Dhruv Batra, Tim K Marks, Chiori Hori, Peter Anderson, et al. 2019.  Audio visual scene-aware dialog.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 7558–7567.
  * Alberti et al. (2019) Chris Alberti, Jeffrey Ling, Michael Collins, and David Reitter. 2019.  Fusion of detected objects in text for visual question answering.  _arXiv preprint arXiv:1908.05054_.
  * Asghar et al. (2018) Nabiha Asghar, Pascal Poupart, Jesse Hoey, Xin Jiang, and Lili Mou. 2018.  Affective neural response generation.  In _European Conference on Information Retrieval_ , pages 154–166. Springer.
  * Baheti et al. (2018) Ashutosh Baheti, Alan Ritter, Jiwei Li, and Bill Dolan. 2018.  Generating more interesting responses in neural conversation models with distributional constraints.  _arXiv preprint arXiv:1809.01215_.
  * Bosselut et al. (2018) Antoine Bosselut, Asli Celikyilmaz, Xiaodong He, Jianfeng Gao, Po-Sen Huang, and Yejin Choi. 2018.  Discourse-aware neural rewards for coherent text generation.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 173–184, New Orleans, Louisiana. Association for Computational Linguistics.
  * Chen et al. (2019) Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. 2019.  Uniter: Learning universal image-text representations.  _arXiv preprint arXiv:1909.11740_.
  * COLBY (1975) KENNETH MARK COLBY. 1975.  Chapter 4 - language-recognition processes for understanding dialogues in teletyped psychiatric interviews.  In KENNETH MARK COLBY, editor, _Artificial Paranoia_ , pages 37 – 49. Pergamon.
  * Danescu-Niculescu-Mizil and Lee (2011) Cristian Danescu-Niculescu-Mizil and Lillian Lee. 2011.  Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs.  In _Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, ACL 2011_.
  * Das et al. (2017a) Abhishek Das, Satwik Kottur, Khushi Gupta, Avi Singh, Deshraj Yadav, José M. F. Moura, Devi Parikh, and Dhruv Batra. 2017a.  Visual dialog.
  * Das et al. (2017b) Abhishek Das, Satwik Kottur, José MF Moura, Stefan Lee, and Dhruv Batra. 2017b.  Learning cooperative visual dialog agents with deep reinforcement learning.  In _Proceedings of the IEEE international conference on computer vision_ , pages 2951–2960.
  * Dodge et al. (2016) Jesse Dodge, Andreea Gane, Xiang Zhang, Antoine Bordes, Sumit Chopra, Alexander Miller, Arthur Szlam, and Jason Weston. 2016.  Evaluating prerequisite qualities for learning end-to-end dialog systems.
  * Gan et al. (2019) Zhe Gan, Yu Cheng, Ahmed El Kholy, Linjie Li, Jingjing Liu, and Jianfeng Gao. 2019\.  Multi-step reasoning via recurrent dual attention for visual dialog.  _arXiv preprint arXiv:1902.00579_.
  * Gao et al. (2018) Jianfeng Gao, Michel Galley, and Lihong Li. 2018.  Neural approaches to conversational ai.  In _The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval_, pages 1371–1374.
  * Ghandeharioun et al. (2019) Asma Ghandeharioun, Judy Hanwen Shen, Natasha Jaques, Craig Ferguson, Noah Jones, Agata Lapedriza, and Rosalind Picard. 2019.  Approximating interactive human evaluation with self-play for open-domain dialog systems.  In _Advances in Neural Information Processing Systems_ , pages 13658–13669.
  * Ghazvininejad et al. (2017) Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, and Michel Galley. 2017.  A knowledge-grounded neural conversation model.  _arXiv preprint arXiv:1702.01932_.
  * Guo et al. (2019) Dan Guo, Hui Wang, and Meng Wang. 2019.  Dual visual attention network for visual dialog.  In _IJCAI_ , pages 4989–4995.
  * Han et al. (2020) Qinghong Han, Yuxian Meng, Fei Wu, and Jiwei Li. 2020.  Non-autoregressive neural dialogue generation.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016.  Identity mappings in deep residual networks.  In _European conference on computer vision_ , pages 630–645. Springer.
  * He et al. (2020) Wanwei He, Min Yang, Rui Yan, Chengming Li, Ying Shen, and Ruifeng Xu. 2020.  Amalgamating knowledge from two teachers for task-oriented dialogue system with adversarial training.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 3498–3507, Online. Association for Computational Linguistics.
  * Hori et al. (2018) Chiori Hori, Huda Alamri, Jue Wang, Gordon Wichern, Takaaki Hori, Anoop Cherian, Tim K. Marks, Vincent Cartillier, Raphael Gontijo Lopes, Abhishek Das, Irfan Essa, Dhruv Batra, and Devi Parikh. 2018.  End-to-end audio visual scene-aware dialog using multimodal attention-based video features.
  * Hua et al. (2020) Kai Hua, Zhiyuan Feng, Chongyang Tao, Rui Yan, and Lu Zhang. 2020.  Learning to detect relevant contexts and knowledge for response selection in retrieval-based dialogue systems.  In _Proceedings of the 29th ACM International Conference on Information Knowledge Management_ , CIKM ’20, page 525–534, New York, NY, USA. Association for Computing Machinery.
  * Huang et al. (2020) Minlie Huang, Xiaoyan Zhu, and Jianfeng Gao. 2020.  Challenges in building intelligent open-domain dialog systems.  _ACM Transactions on Information Systems (TOIS)_ , 38(3):1–32.
  * Jiang et al. (2020a) Xiaoze Jiang, Siyi Du, Zengchang Qin, Yajing Sun, and Jing Yu. 2020a.  Kbgn: Knowledge-bridge graph network for adaptive vision-text reasoning in visual dialogue.  In _Proceedings of the 28th ACM International Conference on Multimedia_ , pages 1265–1273.
  * Jiang et al. (2019) Xiaoze Jiang, Jing Yu, Zengchang Qin, Yingying Zhuang, Xingxing Zhang, Yue Hu, and Qi Wu. 2019.  Dualvd: An adaptive dual encoding model for deep visual understanding in visual dialogue.
  * Jiang et al. (2020b) Xiaoze Jiang, Jing Yu, Yajing Sun, Zengchang Qin, Zihao Zhu, Yue Hu, and Qi Wu. 2020b.  Dam: Deliberation, abandon and memory networks for generating detailed and non-repetitive responses in visual dialogue.
  * Kang et al. (2019) Gi-Cheon Kang, Jaeseo Lim, and Byoung-Tak Zhang. 2019.  Dual attention networks for visual reference resolution in visual dialog.  _arXiv preprint arXiv:1902.09368_.
  * Kottur et al. (2018) Satwik Kottur, José MF Moura, Devi Parikh, Dhruv Batra, and Marcus Rohrbach. 2018.  Visual coreference resolution in visual dialog using neural module networks.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pages 153–169.
  * Kottur et al. (2019) Satwik Kottur, José M. F. Moura, Devi Parikh, Dhruv Batra, and Marcus Rohrbach. 2019.  Clevr-dialog: A diagnostic dataset for multi-round reasoning in visual dialog.
  * Lee et al. (2019) Sungjin Lee, Qi Zhu, Ryuichi Takanobu, Xiang Li, Yaoqin Zhang, Zheng Zhang, Jinchao Li, Baolin Peng, Xiujun Li, Minlie Huang, et al. 2019.  Convlab: Multi-domain end-to-end dialog system platform.  _arXiv preprint arXiv:1904.08637_.
  * Lewis et al. (2017) Mike Lewis, Denis Yarats, Yann Dauphin, Devi Parikh, and Dhruv Batra. 2017.  Deal or no deal? end-to-end learning of negotiation dialogues.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 2443–2453, Copenhagen, Denmark. Association for Computational Linguistics.
  * Li et al. (2019a) Gen Li, Nan Duan, Yuejian Fang, Ming Gong, Daxin Jiang, and Ming Zhou. 2019a.  Unicoder-vl: A universal encoder for vision and language by cross-modal pre-training.
  * Li (2020) Jiwei Li. 2020.  Teaching machines to converse.  _arXiv preprint arXiv:2001.11701_.
  * Li et al. (2015) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2015.  A diversity-promoting objective function for neural conversation models.  _arXiv preprint arXiv:1510.03055_.
  * Li et al. (2016a) Jiwei Li, Michel Galley, Chris Brockett, Georgios P Spithourakis, Jianfeng Gao, and Bill Dolan. 2016a.  A persona-based neural conversation model.  _arXiv preprint arXiv:1603.06155_.
  * Li et al. (2016b) Jiwei Li, Will Monroe, Alan Ritter, Michel Galley, Jianfeng Gao, and Dan Jurafsky. 2016b.  Deep reinforcement learning for dialogue generation.  _arXiv preprint arXiv:1606.01541_.
  * Li et al. (2017) Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. 2017.  Adversarial learning for neural dialogue generation.  _arXiv preprint arXiv:1701.06547_.
  * Li et al. (2019b) Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. 2019b.  Visualbert: A simple and performant baseline for vision and language.  _arXiv preprint arXiv:1908.03557_.
  * Li et al. (2020) Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, et al. 2020.  Oscar: Object-semantics aligned pre-training for vision-language tasks.  In _European Conference on Computer Vision_ , pages 121–137. Springer.
  * Lison and Tiedemann (2016) P. Lison and J. Tiedemann. 2016.  Opensubtitles2016: Extracting large parallel corpora from movie and tv subtitles.  In _LREC_.
  * Lowe et al. (2015) Ryan Lowe, Nissan Pow, Iulian Serban, and Joelle Pineau. 2015.  The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems.  _arXiv preprint arXiv:1506.08909_.
  * Lu et al. (2019) Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019.  Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks.  In _Advances in Neural Information Processing Systems_ , pages 13–23.
  * Lu et al. (2017) Jiasen Lu, Anitha Kannan, Jianwei Yang, Devi Parikh, and Dhruv Batra. 2017.  Best of both worlds: Transferring knowledge from discriminative learning to a generative visual dialog model.
  * Luan et al. (2017) Yi Luan, Chris Brockett, Bill Dolan, Jianfeng Gao, and Michel Galley. 2017.  Multi-task learning for speaker-role adaptation in neural conversation models.  _arXiv preprint arXiv:1710.07388_.
  * Madotto et al. (2019) Andrea Madotto, Zhaojiang Lin, Chien-Sheng Wu, and Pascale Fung. 2019.  Personalizing dialogue agents via meta-learning.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 5454–5459.
  * Massiceti et al. (2018) Daniela Massiceti, N Siddharth, Puneet K Dokania, and Philip HS Torr. 2018.  Flipdial: A generative model for two-way visual dialogue.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 6097–6105.
  * Niu and Bansal (2018) Tong Niu and Mohit Bansal. 2018.  Polite dialogue generation without parallel data.
  * Niu et al. (2019) Yulei Niu, Hanwang Zhang, Manli Zhang, Jianhong Zhang, Zhiwu Lu, and Ji-Rong Wen. 2019.  Recursive visual attention in visual dialog.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 6679–6688.
  * Park et al. (2020) Sungjin Park, Taesun Whang, Yeochan Yoon, and Hueiseok Lim. 2020.  Multi-view attention networks for visual dialog.  _arXiv preprint arXiv:2004.14025_.
  * Rashkin et al. (2018) Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2018.  Towards empathetic open-domain conversation models: A new benchmark and dataset.  _arXiv preprint arXiv:1811.00207_.
  * Roller et al. (2020) Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M Smith, et al. 2020.  Recipes for building an open-domain chatbot.  _arXiv preprint arXiv:2004.13637_.
  * Schwartz et al. (2019) Idan Schwartz, Seunghak Yu, Tamir Hazan, and Alexander G Schwing. 2019.  Factor graph attention.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 2039–2048.
  * See et al. (2019) Abigail See, Stephen Roller, Douwe Kiela, and Jason Weston. 2019.  What makes a good conversation? how controllable attributes affect human judgments.  _arXiv preprint arXiv:1902.08654_.
  * Sennrich et al. (2016) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016.  Neural machine translation of rare words with subword units.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1715–1725, Berlin, Germany. Association for Computational Linguistics.
  * Seo et al. (2017) Paul Hongsuck Seo, Andreas Lehrmann, Bohyung Han, and Leonid Sigal. 2017.  Visual reference resolution using attention memory for visual dialog.  In _Advances in neural information processing systems_ , pages 3719–3729.
  * Serban et al. (2016) Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2016.  A hierarchical latent variable encoder-decoder model for generating dialogues.  _arXiv preprint arXiv:1605.06069_.
  * Shao et al. (2017) Louis Shao, Stephan Gouws, Denny Britz, Anna Goldie, Brian Strope, and Ray Kurzweil. 2017.  Generating high-quality and informative conversation responses with sequence-to-sequence models.  _arXiv preprint arXiv:1701.03185_.
  * Shi et al. (2016) Baoguang Shi, Xiang Bai, and Cong Yao. 2016.  An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition.  _IEEE transactions on pattern analysis and machine intelligence_ , 39(11):2298–2304.
  * Sordoni et al. (2015) Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. 2015.  A neural network approach to context-sensitive generation of conversational responses.  _arXiv preprint arXiv:1506.06714_.
  * Su et al. (2019) Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. 2019\.  Vl-bert: Pre-training of generic visual-linguistic representations.  _arXiv preprint arXiv:1908.08530_.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014.  Sequence to sequence learning with neural networks.  In _Advances in neural information processing systems_ , pages 3104–3112.
  * Tan and Bansal (2019) Hao Tan and Mohit Bansal. 2019.  Lxmert: Learning cross-modality encoder representations from transformers.  _arXiv preprint arXiv:1908.07490_.
  * Tao et al. (2018) Chongyang Tao, Shen Gao, Mingyue Shang, Wei Wu, Dongyan Zhao, and Rui Yan. 2018\.  Get the point of my utterance! learning towards effective responses with multi-head attention mechanism.  In _Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18_ , pages 4418–4424. International Joint Conferences on Artificial Intelligence Organization.
  * Tian et al. (2017) Zhiliang Tian, Rui Yan, Lili Mou, Yiping Song, Yansong Feng, and Dongyan Zhao. 2017\.  How to make context more useful? an empirical study on context-aware neural conversational models.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 231–236, Vancouver, Canada. Association for Computational Linguistics.
  * Tiedemann (2009) J. Tiedemann. 2009.  News from opus — a collection of multilingual parallel corpora with tools and interfaces.
  * Tiedemann (2012) Jörg Tiedemann. 2012.  Parallel data, tools and interfaces in OPUS.  In _Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12)_ , pages 2214–2218, Istanbul, Turkey. European Language Resources Association (ELRA).
  * Turing (2009) Alan M Turing. 2009.  Computing machinery and intelligence.  In _Parsing the turing test_ , pages 23–65. Springer.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in neural information processing systems_ , pages 5998–6008.
  * Vinyals and Le (2015) Oriol Vinyals and Quoc Le. 2015.  A neural conversational model.  _arXiv preprint arXiv:1506.05869_.
  * de Vries et al. (2017) Harm de Vries, Florian Strub, Sarath Chandar, Olivier Pietquin, Hugo Larochelle, and Aaron Courville. 2017.  Guesswhat?! visual object discovery through multi-modal dialogue.
  * Wallace (2009) Richard S. Wallace. 2009.  _The Anatomy of A.L.I.C.E._ , pages 181–210. Springer Netherlands, Dordrecht.
  * Wang et al. (2017) Di Wang, Nebojsa Jojic, Chris Brockett, and Eric Nyberg. 2017.  Steering output style and topic in neural response generation.  _arXiv preprint arXiv:1709.03010_.
  * Wang et al. (2019) Wenhai Wang, Enze Xie, Xiang Li, Wenbo Hou, Tong Lu, Gang Yu, and Shuai Shao. 2019\.  Shape robust text detection with progressive scale expansion network.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 9336–9345.
  * Wang et al. (2020) Yue Wang, Shafiq Joty, Michael R Lyu, Irwin King, Caiming Xiong, and Steven CH Hoi. 2020.  Vd-bert: A unified vision and dialog transformer with bert.  _arXiv preprint arXiv:2004.13278_.
  * Weizenbaum (1966) Joseph Weizenbaum. 1966.  Eliza—a computer program for the study of natural language communication between man and machine.  _Communications of the ACM_ , 9(1):36–45.
  * Wu et al. (2018) Qi Wu, Peng Wang, Chunhua Shen, Ian Reid, and Anton Van Den Hengel. 2018.  Are you talking to me? reasoned visual dialog generation through adversarial learning.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 6106–6115.
  * Xie et al. (2017) Ziang Xie, Sida I Wang, Jiwei Li, Daniel Lévy, Aiming Nie, Dan Jurafsky, and Andrew Y Ng. 2017.  Data noising as smoothing in neural network language models.  _arXiv preprint arXiv:1703.02573_.
  * Xing et al. (2016) Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. 2016\.  Topic aware neural response generation.  _arXiv preprint arXiv:1606.08340_.
  * Xu et al. (2018) Jingjing Xu, Xuancheng Ren, Junyang Lin, and Xu Sun. 2018.  Dp-gan: diversity-promoting generative adversarial network for generating informative and diversified text.  _arXiv preprint arXiv:1802.01345_.
  * Yang et al. (2019) Tianhao Yang, Zheng-Jun Zha, and Hanwang Zhang. 2019.  Making history matter: History-advantage sequence training for visual dialog.
  * Young et al. (2017) Tom Young, Erik Cambria, Iti Chaturvedi, Minlie Huang, Hao Zhou, and Subham Biswas. 2017.  Augmenting end-to-end dialog systems with commonsense knowledge.  _arXiv preprint arXiv:1709.05453_.
  * Zhang et al. (2018) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018.  Personalizing dialogue agents: I have a dog, do you have pets too?  _arXiv preprint arXiv:1801.07243_.
  * Zhang et al. (2019) Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2019.  Dialogpt: Large-scale generative pre-training for conversational response generation.  _arXiv preprint arXiv:1911.00536_.
  * Zhao et al. (2019) Tiancheng Zhao, Kaige Xie, and Maxine Eskenazi. 2019.  Rethinking action spaces for reinforcement learning in end-to-end dialog agents with latent variable models.  _arXiv preprint arXiv:1902.08858_.
  * Zhao et al. (2017) Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017.  Learning discourse-level diversity for neural dialog models using conditional variational autoencoders.  _arXiv preprint arXiv:1703.10960_.
  * Zhao et al. (2020) Xueliang Zhao, Wei Wu, Can Xu, Chongyang Tao, Dongyan Zhao, and Rui Yan. 2020.  Knowledge-grounded dialogue generation with pre-trained language models.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 3377–3390, Online. Association for Computational Linguistics.
  * Zheng et al. (2019a) Yinhe Zheng, Guanyi Chen, Minlie Huang, Song Liu, and Xuan Zhu. 2019a.  Personalized dialogue generation with diversified traits.  _arXiv preprint arXiv:1901.09672_.
  * Zheng et al. (2019b) Yinhe Zheng, Rongsheng Zhang, Xiaoxi Mao, and Minlie Huang. 2019b.  A pre-training based personalized dialogue generation model with persona-sparse data.
  * Zheng et al. (2019c) Zilong Zheng, Wenguan Wang, Siyuan Qi, and Song-Chun Zhu. 2019c.  Reasoning visual dialogs with structural and partial observations.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 6669–6678.
  * Zhou et al. (2017) Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2017.  Emotional chatting machine: Emotional conversation generation with internal and external memory.  _arXiv preprint arXiv:1704.01074_.
  * Zhou et al. (2020) Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2020.  The design and implementation of xiaoice, an empathetic social chatbot.  _Computational Linguistics_ , 46(1):53–93.