# Toward a Comprehensive Simulation Framework for Hypergraphs: A Python-Based
Approach

Quoc Chuong Nguyen
Department of Mathematics
University at Buffalo
New York, NY 14260
<EMAIL_ADDRESS>&Trung Kien Le
Department of Applied Physics
Stanford University
Stanford, CA 94305
<EMAIL_ADDRESS>

###### Abstract

Hypergraphs, or generalization of graphs such that edges can contain more than
two nodes, have become increasingly prominent in understanding complex network
analysis. Unlike graphs, hypergraphs have relatively few supporting platforms,
and such dearth presents a barrier to more widespread adaptation of hypergraph
computational toolboxes that could enable further research in several areas.
Here, we introduce HyperRD, a Python package for hypergraph computation,
simulation, and interoperability with other powerful Python packages in graph
and hypergraph research. Then, we will introduce two models on hypergraph, the
general Schelling’s model and the SIR model, and simulate them with HyperRD.

##  1 Introduction

Hypergraphs, a natural extension of traditional graphs, have emerged as a
powerful mathematical tool for modeling complex relationships and structures
across various domains [1, 2, 3]. Unlike graphs, hypergraphs allow for edges
to connect more than two vertices, providing a more expressive representation
of intricate dependencies. Therefore, hypergraphs have a profound significance
in diverse disciplines, including computer science [4], biology [5], and
social networks [6]. The study of hypergraphs in computer science has far-
reaching implications for algorithm design, especially in addressing
optimization problems where traditional graph models may prove inadequate, as
noted in [7]. Hypergraphs also capture complexity in complex interacting
networks and provide a glimpse into qualitatively different emergent dynamics
[8]. Shifting the focus to the biological realm, hypergraphs find valuable
applications in modeling molecular interactions and regulatory networks. Their
unique capability to represent interactions among multiple biological
entities, such as genes or proteins, contributes to a more nuanced
understanding of the intricate relationships governing biological processes,
as highlighted in [9]. This dual application underscores the interdisciplinary
significance of hypergraphs in both computational and biological domains.

The generalization of hypergraph [10, 11] provides a more versatile framework
for capturing complex interactions that extend beyond binary relationships
[12]. Therefore, the pervasive integration of hypergraphs in modeling real-
world systems underscores the need for sophisticated simulation tools to
comprehensively examine and understand their dynamic behavior [13].
Simulation, a pivotal instrument in this context, facilitates an in-depth
exploration of the evolution of intricate relationships over time. It proves
essential for evaluating the influence of diverse factors on the dynamic
nature of hypergraph-based systems. By providing a virtual environment
conducive to experimentation, simulation tools play an indispensable role in
unraveling the complexities inherent in these relationships, offering insights
into emergent properties and behaviors that elude analytical approaches.

For this reason, we introduce a novel Python-based platform designed to
simulate and analyze hypergraphs, addressing a gap in current computational
resources available to researchers and practitioners in mathematics and
computer science. Our platform, distinguished by its near-comprehensive range
of features, stands out in its ability to represent, manipulate, visualize,
and analyze hypergraphs with a level of versatility and depth not commonly
found in existing tools [14, 15, 16, 17, 18, 19, 20]. A significant feature of
our platform is its interoperability, enabling integration with other
established platforms and tools. This feature is instrumental in fostering a
universal workspace, where users can leverage the strengths of various
platforms to conduct more intricate and expansive research on hypergraphs. By
bridging these tools, our platform facilitates a more efficient and cohesive
workflow and propels the exploration of hypergraphs into new realms, allowing
for more complex, interdisciplinary inquiries and applications.

The paper is organized as follows. In Sec. 2, we start by surveying some
recent platforms to simulate hypergraphs and compare their features with our
platform. In Sec. 3, we introduce the structure of HyperRD and discuss its
modules, which are used to analyze and generate random hypergraphs. Sec. 4
proposes some dynamical models on hypergraph and implements them by HyperRD.
Finally, we conclude in Sec. 5 with a discussion of the results and an outlook
on possible future research directions.

##  2 Literature Review

The development of platforms for simulating normal graphs in applied
mathematics and computer science has seen substantial progress over the past
decades. One of the most significant contributions has been the NetworkX
library in Python, which offers extensive tools for the creation,
manipulation, and study of the structure, dynamics, and functions of complex
networks [21]. NetworkX’s ease of use and flexibility have made it a popular
choice among researchers. Another notable platform is Gephi, an open-source
network visualization software that allows for the exploration and
understanding of graph structures through sophisticated graphical
representations [22]. These platforms have been instrumental in advancing the
field of graph theory by facilitating complex analyses and visualizations.

The rationale for developing platforms specifically for simulating hypergraphs
arises from the limitations of normal graph simulation tools in capturing
multi-way relationships. While normal graphs represent binary relationships
between entities, hypergraphs are capable of representing more complex,
higher-order interactions among multiple entities simultaneously. This
capability is crucial in fields such as computational biology, social network
analysis, and complex system modeling, where interactions often occur among
more than two entities. Therefore, the development of hypergraph-specific
platforms is essential to address the nuanced requirements of these advanced
applications, providing more accurate and comprehensive tools for researchers
and practitioners. Notwithstanding the commendable attributes of some existing
platforms such as HAT [14], HyperNetX [15], halp, [16], hMETIS [17], Phoenix
[18], HyperG [19], and XGI [20], each of them has their own limitations. These
limitations, in turn, accentuate the compelling need for a Python-based
simulation tool characterized by heightened comprehensiveness and versatility.
The choice of Python as the language of the proposed simulation platform stems
from its widespread adoption and versatility in scientific computing, ensuring
accessibility and adaptability for researchers and practitioners across
various disciplines. Python’s expressive syntax and rich ecosystem of
libraries make it an ideal candidate for developing a tool that caters
specifically to the nuanced requirements of hypergraph simulation.

Table [1] shows a more detailed comparison between our platform (HyperRD) with
the others.

Table 1: Comparison of HyperRD to other hypergraph libraries Features | HyperRD | HAT | HyperNetX | halp | hMETIS | Phoenix | HyperG | XGI
---|---|---|---|---|---|---|---|---
Clustering | ✓ |  | ✓ | ✓ |  | ✓ | ✓ | ✓
Visualization | ✓ | ✓ | ✓ |  |  |  | ✓ | ✓
Structure analysis | ✓ | ✓ | ✓ | ✓ | ✓ |  | ✓ | ✓
Random generator | ✓ |  | ✓ |  |  |  | ✓ | ✓
Multilayer hypergraph | ✓ |  |  |  |  |  |  | ✓
Integrate other platforms | ✓ |  |  |  |  |  |  |

##  3 Structure

Practically, HyperRD consists of two parts: two submodules to represent the
hypergraph, and the others are used for the analysis of the hypergraph as
shown in Fig. 1.

Figure 1: HyperRD consists of two classes of main submodules. Two are for the
representation of the hypergraph, and the others are for the analysis of
hypergraphs. A hypergraph is constructed through the Create class and its
containing features, as illustrated in this figure.

###  3.1 Hypergraph representation

In Hgraph, the class Create is designed to encapsulate the properties of an
undirected and non-multilayer hypergraph. It contains the following
properties:

  * •

vertices and edges: Python sets that ensure both vertices and edges are unique
and unordered, aligning with the standard mathematical definition of a
hypergraph. Sets are efficient for membership testing and eliminating
duplicates, key aspects in maintaining the integrity of hypergraph structures
[23].

  * •

vertices_dict and edges_dict: Python dictionaries that provide a mapping
mechanism to store additional attributes associated with vertices and edges,
respectively. This usage aligns with the need for extending hypergraph
representation to store labels, e.g. indexes of vertices and edges for
computation, thereby enhancing the model’s applicability to real-world
scenarios [24].

  * •

vertices_weight and edges_weight: the implementation of Python lists to store
weights for vertices and edges suggesting that the hypergraph is intended to
handle weighted scenarios, such as in network flow problems or to represent
the strength of connections. Lists provide an ordered and mutable collection
of items, which is suitable for scenarios where weights might need to be
updated or processed sequentially [25].

  * •

inci_matrix: suggest an intention to use an incidence matrix for
representation. In hypergraph theory, this matrix is particularly useful for
computational analysis of hypergraphs, such as determining vertex-edge
connectivity and implementing algorithms for traversal or search within the
hypergraph [12, 13].

Random hypergraphs are also important objects studied in hypergraph theory. In
essence, random hypergraphs are generalizations of random graphs [26, 27]. One
of the most renowned models in this domain is the Erdős–Rényi model [28],
which extends to hypergraphs by randomly adding hyperedges among a fixed set
of vertices. Another significant model is the random $k$-uniform hypergraph,
which is a hypergraph where each edge (or hyperedge) connects exactly $k$
vertices. Many studies have shown that random hypergraphs find applications in
computational biology and network science, providing insights into the
structural properties of complex biological networks [29, 30, 31].

Therefore, owing to the richness of hypergraphs, we present several algorithms
for generating random hypergraphs. Throughout the submodule Hrandom, a user
can generate hypergraphs via some functions:

  * •

simple_matrix: each element of an incidence matrix will be set to value 1
randomly with probability $p$.

  * •

simple_bipartite: create a random bipartite-featured hypergraph with
probability $p$.

  * •

simple_powersets: create a random hypergraph with probability $p$, based on
the random choice of all possible edges (the power set of vertices).

  * •

simple_order: similar to simple_powersets, but the degree of each hyperedge
has a bound (the cardinality of the hyperedge is less than or equal to a fixed
integer).

  * •

k_uniform: create a $k$-uniform hypergraph with probability $p$.

In addition to undirected hypergraph models, HyperRD provides the classes
Create_Direct and Multilayer that inherit from Create and new methods
appropriate for each type of hypergraphs.

  * •

Create_Direct: Construct directed hyperedges for a given hypergraph $H$ from
user-specified starting set and ending set. Directed hypergraph models are
useful in complex networks with non-reciprocal interactions.

  * •

Multilayer: Construct multilayer hypergraph by adding graphs into a list
self.layers, and list of inter-edges self.interlink between layers. Multilayer
hypergraph networks are important in modelling multiple types of relationships
between components, such as bio-chemical networks involving multiple pathways
or logistic networks comprised of multiple modes of transportations.

###  3.2 Analysis tools

There are four HyperRD submodules to analyze and visualize hypergraphs, named
as Halgorithm, Hdynamic, Hmetrix, Htranslate. The main function of each
submodule is:

  * •

Halgorithm: provide some basic algorithms on hypergraphs.

  * •

Hdynamic: used to study the dynamical systems on hypergraphs.

  * •

Hmetric: contains crucial metrics.

  * •

Htranslate: transform the Create object to other platforms’ object for deeper
analysis.

#### Hypergraph metrics

In Hmetric, one can analyze several relevant metrics of hypergraphs such as:

  * •

Density: is defined as the number of edges divided by the number of possible
edges that a hypergraph can have. That is

$\displaystyle\textrm{D(H)}=\frac{\textrm{no. of edges}}{2^{\textrm{no. of
vertices}}-1}$ (1)

  * •

Girth: the smallest non-empty hyperedge in the hypergraph. For example, if $H$
has edges of size $\\{2,\cdots,k\\}$ then the girth of $H$ is 2.

  * •

Average vertex degree: the vertex degree of a hypergraph is the number of
hyperedges connected to the vertex. The average degree is then defined as the
sum of all vertices’ degrees divided by the length of degree_vertex.

  * •

Average edge size: similar to the vertex degree, one can assign a notion of
average edge size by dividing the total length of all edges in $H$ by the
number of edges.

  * •

etc.

#### Hypergraph Algorithms

In HyperRD, the submodule Halgorithm contains algorithms for basic analysis of
hypergraph structures. Some of them are listed below:

  * •

components_connected: return a list of connected components in a hypergraph,
as shown in Algo. 1.

  * •

simple_reduction: reduce an arbitrary hypergraph to a simple form [12], as
shown in Algo. 2.

  * •

graph_expansion: expand a hypergraph to a normal graph via the clique
expansion or star expansion [13], as shown in Algo. 3

  * •

etc.

#### Connectivity

A major feature of HyperRD is its ability to integrate with and take advantage
of the rich Python ecosystem for graphs and hypergraphs via the submodule
Htranslate. Here, the instance of the Create can be translated into
HypernetX’s Hypergraph(), thereby combining the features of HyperRD with these
libraries. For example, visualization of hypergraphs can be performed with
ease through HypernetX method drawing.draw() after a translate application,
which enables quick visualization of a user’s instance of Create. At this
moment, Htranslate provides the connection to the following platforms:

  * •

NetworkX: a universal Python platform for normal graphs.

  * •

HypernetX: used for modularity and clustering on hypergraphs.

  * •

XGI: used for dynamics on hypergraphs.

#### Hypergraph dynamics

The higher-order structures in a hypergraph form a fertile ground for novel,
emergent dynamics that might not exist in a graph. To provide tools for
understanding hypergraph dynamics, HyperRD includes a submodule Hdynamic to
simulate random walk on hypergraphs via constructing a probability transition
matrix [32] between nodes $i$ and $j$

$\displaystyle T_{ij}=\frac{(e\hat{C}e^{T})_{ij}-A_{ij}}{\sum_{\ell\neq
i}(e\hat{C}e^{T})_{i\ell}-k_{i}^{H}}$ (2)

Where $e$ is the incidence matrix, $C=e^{T}e$ is the hyperedges matrix and
$\hat{C}$ is the diagonal matrix of $C$, $A=ee^{T}$ is the adjacency matrix
and $k_{i}^{H}=\sum_{\ell\neq i}A_{i\ell}$ is the hyperdegree of $i$-th node.
If the random walk is "lazy", one can compute the transition matrix by
reducing the numerator to $(e\hat{C}e^{T})_{ij}$.

##  4 Models

###  4.1 The general Schelling’s model of segregation

Thomas C. Schelling’s pioneering work on segregation models has been a
cornerstone in understanding social dynamics through mathematical simulations.
In his paper [33], Schelling introduced an agent-based model illustrating how
individual preferences, even when not overtly discriminatory, can lead to
significant segregation patterns. This model, employing a simple checkerboard
to simulate neighborhood dynamics, underscored the unintended consequences of
individual choices in societal structures. Further developments by Sakoda, who
extended Schelling’s concepts in his paper [34], and later researchers,
incorporated more complex variables and computational techniques, enriching
the model’s applicability to diverse social phenomena. Clark and Fossett’s
review in [35] and other works [36, 37] provided a comprehensive analysis of
the model’s evolution, highlighting its continued relevance in urban studies
and policy-making. However, this model does not capture the higher-order
relationship between individual actions and collective patterns since agents
in one society tend to interact in a group, rather than typical pairwise
interaction. To extend this model to higher-order interaction, we propose the
general Schelling’s model on a hypergraph as:

Given the hypergraph $H=(E,V)$, where $V=\\{v_{1},v_{2},...,v_{N}\\}$ and
$E=\\{e_{1},e_{2},...,e_{M}\\}\text{, }e_{j}\subset V\text{ }$ for all $j$.
Each node will be assigned one label $a_{i}$ (via function $F:V\rightarrow A$)
in the label set $A=\\{a_{1},a_{2},...\\}$ such that

$\displaystyle\bigcup_{i=1}^{|A|}V_{i}\subset V\text{ where }V_{i}=\\{v|v\in
V\text{ and }F(v)=a_{i}\\}\text{ and }V_{i}\cap V_{j}=\emptyset\text{,
}\forall i,j$ (3)

The original version of this model can be easily derived from this definition
whereas $|e_{j}|=2$. For the simplification, we only consider A to be finite
and there are some unlabeled vertices (i.e., $\sum_{i=1}^{|A|}|V_{i}|<|V|$)
and do not allow the one-vertex hyperedge (i.e., $|e_{j}|>1$).

For each iteration, the process of Schelling’s model is described below:

  1. 1.

Select one labeled vertex randomly.

  2. 2.

Calculate the neighborhood coefficient of this vertex via the formula:

$\displaystyle G(v)=\frac{1}{K_{v}}\sum_{j=1\text{, }v\in
e_{j}}^{|E|}\frac{|N_{j}(v)|}{|e_{j}|-1}$ (4)

where,

$\displaystyle N_{j}(v)=\\{v^{\prime}|v^{\prime}\neq v\text{,
}F(v^{\prime})=F(v)\text{, and }v^{\prime}\in e_{j}\\}$ $\displaystyle
K_{v}\text{ is the number of hyperedge that contains v}$

  3. 3.

If $G(v)<\tau\text{ (}0\leq\tau\leq 1)$, assign randomly the label of the
chosen vertex to one unlabeled vertex, and the chosen vertex in step 1 will
become unlabeled.

  4. 4.

Repeat step 1 to 3 until every node has $G(v)\geq\tau$ or after a fixed number
of iteration.

Observing Fig. 2, the simulation setup comprising 15 nodes, 3 labels, and a
randomly initialized state, a clustering phenomenon emerges after 100
iterations. This outcome aligns with the expectations set forth by classical
models, yet it introduces a novel characteristic: a convergence of similarly
labelled nodes into a single hyperedge. Detailed implementation of this model
can be found in the appendix of this paper. 1.

Figure 2: In the implementation of the general Schelling’s model, we consider
a system consisting of 15 nodes distributed into three distinct labels, with
each label assigned to 3 nodes. The initial configuration of these nodes is
randomized. We ran the simulation of our model for 100 iterations. The initial
state of the node configuration is depicted on the left side of the figure,
while the right side illustrates the final state after 100 iterations. It is
observed that nodes sharing identical labels tend to aggregate into a single
hyperedge, demonstrating the model’s inherent clustering dynamics.

###  4.2 The SIR model on hypergraph

The traditional Susceptible-Infected-Recovered (SIR) model on normal networks,
while a cornerstone in epidemiological modeling, exhibits limitations
particularly in representing complex multi-agent interactions, a gap that
hypergraph models aim to fill. For instance, the epidemic model on normal
networks, as elaborated by Pastor-Satorras et al. [38], explores the spread of
infectious diseases in various network topologies, emphasizing the critical
role of network structure on disease dynamics. However, these models often
overlook scenarios where contagion can occur via group interactions. To
address these limitations, researchers have proposed extending the SIR model
to hypergraphs. For example, Landry and Restrepo demonstrated the adaptation
of the Susceptible-Infected-Susceptible (SIS) model to hypergraphs [39],
showcasing how it can more accurately reflect the spread of diseases in
settings where group interactions are prevalent, such as social gatherings or
community clusters. The hypergraph-based SIR model, therefore, not only offers
a more realistic framework for simulating disease spread in complex social
systems but also provides insights that could be instrumental in shaping
effective public health policies and intervention strategies. Here we consider
the simple model where the update of the state of each node after one
iteration (which is S, I, or R) is given by the probabilities:

  * •

Infection probability: $P(S\rightarrow I)=1-(1-\beta)^{k_{i}}$ where $k_{i}$
is the number of infected individuals in the same hyperedge, and $\beta$ is
the transmission rate.

  * •

Recover probability: $P(I\rightarrow R)=\gamma$, where $\gamma$ is the recover
rate.

for all nodes $i$.

Fig. 3 presents the simulation results of the SIR model on a hypergraph,
illustrating two scenarios: one with a population of 20 individuals (shown on
the left) and the other with 30 individuals (displayed on the right). In these
simulations, the maximum order of the hyperedges is set to 3, with the
transmission rate parameter $beta$ at 0.4 and recovery rate parameter $\gamma$
at 0.1, over a total of 20 iterations. For an in-depth understanding of this
model’s Python implementation, refer to the appendix of this paper. 2.

Figure 3: The SIR model on a hypergraph, designed for a maximum interaction
order of 3 (3-hyperedges), is depicted for two distinct population sizes: 20
individuals (illustrated on the left) and 30 individuals (shown on the right).
In this model, the hyperedges are constrained to a maximum order of 3. The
parameters governing the model include a transmission rate $\beta$ set at 0.4
and a recovery rate $\gamma$ at 0.1. The simulation runs for 20
iterations/days.

##  5 Discussion

In conclusion, HyperRD is the platform designed to simulate hypergraphs, and a
cornerstone of this platform’s utility lies in its robust connectivity with
other platforms and its inherent versatility. This connectivity not only
facilitates integration with diverse software ecosystems but also promotes
interdisciplinary collaborations, thereby broadening the scope of potential
applications.

Moreover, the platform’s design philosophy emphasizes continual evolution,
ensuring that it remains at state-of-the-art. We will keep regularly upgrading
the algorithms and incorporating new features to enable the platform to adapt
to the ever-changing landscape of computational needs and user requirements.
These include leveraging network data or statistical tools for reconstructing
hypergraphs [40], as well as comprehensive topological [41, 42] and spectral
analysis [43] capabilities for fast and powerful structure analysis of large
hypergraph networks. Integration of our platform with existing Python
frameworks such as tensorflow, pytorch or CVXPY is also possible, furthering
our platform’s capabilities for interdisciplinary approaches.

Finally, the exploration of new applications remains a pivotal aspect of this
platform. By actively seeking novel use cases, particularly in areas that have
yet to fully leverage the power of hypergraph analysis, this platform can
significantly contribute to a variety of fields. Some of its applications we
will focus on after this paper are synchronization dynamics [8], directed and
non-reciprocal interactions [44], and multilayer dynamics [45]. Furthermore,
applications such as optimizing hypergraph structures for pre-determined goals
via tailored loss functions - as already done for graphs to design quantum
experiments [46] \- are also viable.

##  6 Acknowledgments

Q.C. and T.K. coordinated the project. Q.C. is the leading developer. All
authors contributed to the library and wrote the article.

##  7 Data availability statement

The source code that supports the findings of this study is openly available
at the following URL/DOI:

https://github.com/ChuongQuoc1413017/Hypergraph_RD

## References

  * [1] Federico Battiston and Giovanni Petri.  Higher-Order Systems.  Understanding Complex Systems. Springer Cham, 2023.
  * [2] Ginestra Bianconi.  Higher-Order Networks.  Elements in the Structure and Dynamics of Complex Networks. Cambridge University Press, 2021.
  * [3] Federico Battiston et al.  Networks beyond pairwise interactions: Structure and dynamics.  Physics Reports, 874:1–92, 2020.  Networks beyond pairwise interactions: Structure and dynamics.
  * [4] Andreaand Santoro et al.  Higher-order organization of multivariate time series.  Nature Physics, 19(2):221–229, Feb 2023.
  * [5] Steffen Klamt et al.  Hypergraphs and cellular networks.  PLOS Computational Biology, 5(5):1–6, 05 2009.
  * [6] Federico Musciotto et al.  Beyond the dyad: uncovering higher-order structure within cohesive animal groups.  bioRxiv, 2022.
  * [7] Egon Balas.  Disjunctive programming: Properties of the convex hull of feasible points.  Discrete Applied Mathematics, 89:3–44, Dec 1998.
  * [8] Yuanzhao Zhang, Maxime Lucas, and Federico Battiston.  Higher-order interactions shape collective dynamics differently in hypergraphs and simplicial complexes.  Nature Communications, 14(1), March 2023.
  * [9] Trey Ideker et al.  Integrated genomic and proteomic analyses of a systematically perturbed metabolic network.  Science, 292(5518):929–934, 2001.
  * [10] Claude Berge.  Two theorems in graph theory.  Proceedings of the National Academy of Sciences of the United States of America, 43:842–844, Sep 1957.
  * [11] Xavier Ouvrard.  Hypergraphs: an introduction and review.  arXiv, 2020.
  * [12] Alain Bretto.  Hypergraph Theory: An Introduction.  Springer, 2013.
  * [13] Qionghai Dai and Yue Gao.  Hypergraph Computation.  Springer, 2023.
  * [14] Joshua Pickard et al.  HAT: Hypergraph analysis toolbox.  PLOS Computational Biology, 2023.
  * [15] Brenda Praggastis et al.  HyperNetX.  Available from: https://pnnl.github.io/HyperNetX/, 2021.
  * [16] Brendan Avent et al.  halp.  Available from: https://murali-group.github.io/halp/, 2014\.
  * [17] George Karypis and Vipin Kumar.  hMETIS.  Available from: http://vlsicad.eecs.umich.edu/BK/Slots/cache/www-users.cs.umn.edu/~karypis/metis/hmetis/index.html, 1998\.
  * [18] Kuldeep Kurte et al.  Phoenix: A Scalable Streaming Hypergraph Analysis Framework.  Springer, 2021.
  * [19] David J. Marchette.  HyperG.  Available from: https://cran.r-project.org/web/packages/HyperG/index.html, 2021.
  * [20] Nicholas W. Landry et al.  XGI: A Python package for higher-order interaction networks.  Journal of Open Source Software, 8(85):5162, May 2023.
  * [21] Aric A. Hagberg, Daniel A. Schult, and Pieter J. Swart.  Exploring network structure, dynamics, and function using networkx.  In Gaël Varoquaux, Travis Vaught, and Jarrod Millman, editors, Proceedings of the 7th Python in Science Conference, pages 11–15, Pasadena, CA USA, 2008.
  * [22] Mathieu Bastian, Sebastien Heymann, and Mathieu Jacomy.  Gephi: An open source software for exploring and manipulating networks.  Proceedings of the International AAAI Conference on Web and Social Media, 3(1):361–362, Mar. 2009.
  * [23] Thomas H. Cormen et al.  Introduction to Algorithms, Third Edition.  The MIT Press, 3rd edition, 2009.
  * [24] Claude Berge.  Hypergraphs: Combinatorics of Finite Sets.  North-Holland Mathematical Library, 1989.
  * [25] Jon Kleinberg and Éva Tardos.  Algorithm Design.  Pearson Education, 2006.
  * [26] Marc Barthelemy.  Class of models for random hypergraphs.  Phys. Rev. E, 106:064310, 2022.
  * [27] Oliver Cooley et al.  The size of the giant component in random hypergraphs.  arXiv, 2015.
  * [28] P Erdös and A Rényi.  On random graphs i.  Publicationes Mathematicae Debrecen, 6:290–297, 1959.
  * [29] Florian Klimm, Charlotte M. Deane, and Gesine Reinert.  Hypergraphs for predicting essential genes using multiprotein complex data.  bioRxiv, 2020.
  * [30] Song Feng et al.  Hypergraph models of biological networks to identify genes critical to pathogenic viral response.  BMC Bioinformatics, 22(1):287, May 2021.
  * [31] Jose Lugo-Martinez et al.  Classification in biological networks with hypergraphlet kernels.  Bioinformatics, 37(7):1000–1007, 09 2020.
  * [32] Timoteo Carletti, Federico Battiston, Giulia Cencetti, and Duccio Fanelli.  Random walks on hypergraphs.  Phys. Rev. E, 101:022308, Feb 2020.
  * [33] Thomas C. Schelling.  Dynamic models of segregation.  The Journal of Mathematical Sociology, 1:143–186, 1971.
  * [34] James M. Sakoda.  The Checkerboard Model of Social Interaction.  Journal of Mathematical Sociology, 1:119–132, 1971.
  * [35] William A. V. Clark.  Understanding the social context of the Schelling segregation model.  Proceedings of the National Academy of Sciences, 105:4109–4114, 2008.
  * [36] Nicolás Goles Domic, Eric Goles, and Sergio Rica.  Dynamics and complexity of the schelling segregation model.  Phys. Rev. E, 83:056111, May 2011.
  * [37] Giorgio Fagiolo et al.  Segregation in networks.  Journal of Economic Behavior & Organization, 64:316–336, 2007\.
  * [38] Romualdo Pastor-Satorras, Claudio Castellano, Piet Van Mieghem, and Alessandro Vespignani.  Epidemic processes in complex networks.  Rev. Mod. Phys., 87:925–979, Aug 2015.
  * [39] Nicholas W. Landry and Juan G. Restrepo.  The effect of heterogeneity on hypergraph contagion models.  Chaos: An Interdisciplinary Journal of Nonlinear Science, 30(10):103117, 10 2020.
  * [40] Jean-Gabriel Young, Giovanni Petri, and Tiago P. Peixoto.  Hypergraph reconstruction from network data.  Communications Physics, 4(1), June 2021.
  * [41] Reza Ghorbanchian, Juan G. Restrepo, Joaquín J. Torres, and Ginestra Bianconi.  Higher-order simplicial synchronization of coupled topological signals.  Communications Physics, 4(1), June 2021.
  * [42] Sergio Barbarossa and Stefania Sardellitti.  Topological signal processing over simplicial complexes.  IEEE Transactions on Signal Processing, 68:2992–3007, 2020.
  * [43] Anirban Banerjee.  On the spectrum of hypergraphs.  Linear Algebra and its Applications, 614:82–110, April 2021.
  * [44] Luca Gallo, Riccardo Muolo, Lucia Valentina Gambuzza, Vito Latora, Mattia Frasca, and Timoteo Carletti.  Synchronization induced by directed higher-order interactions.  Communications Physics, 5(1), October 2022.
  * [45] Kalle Alaluusua, Konstantin Avrachenkov, B. R. Vinay Kumar, and Lasse Leskelä.  Multilayer Hypergraph Clustering Using the Aggregate Similarity Matrix, page 83–98.  Springer Nature Switzerland, 2023.
  * [46] Carlos Ruiz-Gonzalez, Sören Arlt, Jan Petermann, Sharareh Sayyad, Tareq Jaouni, Ebrahim Karimi, Nora Tischler, Xuemei Gu, and Mario Krenn.  Digital Discovery of 100 diverse Quantum Experiments with PyTheus.  Quantum, 7:1204, December 2023.

##  8 Appendix

###  8.1 Python codes

This is the code for the Schelling’s model above. 4.1

⬇

1from HyperRD.Hrandom import *

2from HyperRD.Halgorithm import *

3from HyperRD.Hgraph import *

4from HyperRD.Hdynamic import *

5from HyperRD.Htranslate import *

6import matplotlib.pyplot as plt

7

8class Schelling_Model(object):

9 # init function to declare class variables

10 def __init__(self, n_red, n_blue, n_yellow, graph):

11 """Random system initialisation.

12 RED = 0

13 BLUE = 1

14 YELLOW = 2

15 EMPTY = -1

16 """

17 array = np.zeros(len(graph.vertices))

18 array[n_red: n_red + n_blue] = 1

19 array[n_red + n_blue: n_red + n_blue + n_yellow] = 2

20 array[n_red + n_blue + n_yellow:] = -1

21 self.graph = graph

22 self.array = np.array(array)

23 np.random.shuffle(self.array)

24

25 def running(self, iteration, tol):

26 ’’’Running the model’’’

27 total = len(self.array)

28 for i in range(iteration):

29 citizen = np.where(self.array >= 0)[0]

30 start = np.random.choice(citizen, 1, replace = False)[0]

31 if self.neighbor_coefficient(start) < tol:

32 empty = np.where(self.array < 0)[0]

33 end = np.random.choice(empty, 1)[0]

34 if self.neighbor_coefficient(end) != 2:

35 self.array[start], self.array[end] = self.array[end], self.array[start]

36

37 def neighbor_coefficient(self, start):

38 ’’’calculate the neighbor coefficient’’’

39 edges = self.graph.edges

40 coeff = np.array([])

41 for edge in edges:

42 if start in edge and len(edge) > 1:

43 egde_coeff = self.array[list(edge)]

44 neighbor = egde_coeff == self.array[start]

45 coeff = np.append(coeff, (np.sum(neighbor) - 1) / (len(edge) - 1))

46 return np.mean(coeff) if len(coeff) > 0 else 2

47

48# Initialize hypergraph

49graph = simple_order(15, 3, 0.02)

50graph.edges

51

52# Initialize model

53model = Schelling_Model(3, 3, 3, graph)

54

55# Run the simulation

56model.running(100, 0.2)

57

58# Visualize

59colors = []

60for i in model.array:

61 if i == 0:

62 colors.append(’red’)

63 elif i == 1:

64 colors.append(’blue’)

65 elif i == 2:

66 colors.append(’yellow’)

67 else:

68 colors.append(’black’)

69graph_x = hyperrd_to_hypernetx(model.graph)

70hnx.drawing.draw(graph_x,

71 with_edge_labels = False,

72 layout_kwargs = {’seed’: 39}, edges_kwargs={

73 ’edgecolors’: ’black’

74 },

75 nodes_kwargs={

76 ’facecolors’: colors

77 })

Listing 1: The code for the Schelling’s model.

This is the code for the SIR model above. 4.2

⬇

1from HyperRD.Hrandom import *

2from HyperRD.Halgorithm import *

3from HyperRD.Hgraph import *

4from HyperRD.Hdynamic import *

5from HyperRD.Htranslate import *

6import random

7import numpy as np

8import matplotlib.pyplot as plt

9

10def compute_mutual_information(trajectory1, trajectory2, bins=10):

11 # Compute histograms

12 hist_1, _ = np.histogram(trajectory1, bins=bins, density=True)

13 hist_2, _ = np.histogram(trajectory2, bins=bins, density=True)

14

15 # Compute joint histogram

16 joint_hist, _, _ = np.histogram2d(trajectory1, trajectory2, bins=bins,
density=True)

17

18 # Compute mutual information

19 eps = np.finfo(float).eps

20 mutual_info = np.sum(joint_hist * np.log2((joint_hist + eps) /
(np.outer(hist_1, hist_2) + eps)))

21

22 return mutual_info

23

24# Parameters

25num_vertices = 20

26maximum_hyperedge_size = 3

27infection_probability = 0.4

28recovery_probability = 0.1

29num_steps = 20

30

31# Initialize hypergraph

32vertices = range(num_vertices)

33graph = simple_order(num_vertices, maximum_hyperedge_size, 0.02)

34

35# Initialize states

36states = [’S’] * num_vertices # S: Susceptible, I: Infected, R: Recovered

37initial_infected = random.sample(vertices, 1)

38for i in initial_infected:

39 states[i] = ’I’

40

41# Lists to store the trajectory

42susceptible_traj = [len([v for v in states if v == ’S’])]

43infected_traj = [len([v for v in states if v == ’I’])]

44recovered_traj = [len([v for v in states if v == ’R’])]

45

46# Simulation function

47def simulate_step():

48 new_states = states.copy()

49 for e in graph.edges:

50 infected = [v for v in e if states[v] == ’I’]

51 if infected:

52 for v in e:

53 if states[v] == ’S’ and random.random() < 1 - (1 -
infection_probability)**len(infected):

54 new_states[v] = ’I’

55 elif states[v] == ’I’ and random.random() < recovery_probability:

56 new_states[v] = ’R’

57 return new_states

58

59# Run the simulation

60for step in range(num_steps):

61 states = simulate_step()

62 susceptible_traj.append(len([v for v in states if v == ’S’]))

63 infected_traj.append(len([v for v in states if v == ’I’]))

64 recovered_traj.append(len([v for v in states if v == ’R’]))

65

66# Plot the trajectories

67plt.plot(susceptible_traj, label=’Susceptible’)

68plt.plot(infected_traj, label=’Infected’)

69plt.plot(recovered_traj, label=’Recovered’)

70plt.title(’SIR Model Trajectories’)

71plt.xlabel(’Days’)

72plt.ylabel(’Population’)

73plt.legend()

74plt.show()

75

76# Print the computed mutual information

77mutual_info = compute_mutual_information(susceptible_traj, infected_traj)

78print(f’Mutual Information between Susceptible and Infected: {mutual_info}’)

Listing 2: The code for the SIR model.

###  8.2 Algorithms

Algorithm 1 Find Connected Components in a Hypergraph

0: $H$ (a hypergraph)

0: components (list of connected components)

$V\leftarrow graph.vertices\\_dict$

$E\leftarrow graph.edges\\_dict$

$n_{V}\leftarrow|V|$

$n_{E}\leftarrow|E|$

$normal\\_graph\leftarrow$ Create_Normal$(n_{V}+n_{E})$ (from Hgraph)

for each $vertex$ in range($n_{V}$) do

for each $edge$ in range($n_{E}$) do

if $V[vertex]$ in $E[edge]$ then

$normal\\_graph$.add_edge$(vertex,edge+n_{V})$

end if

end for

end for

$cc\leftarrow normal\\_graph$.connected_components()

$components\leftarrow[]$ (list)

for each $cluster$ in $cc$ do

$component\leftarrow[]$ (list)

for each $i$ in $cluster$ do

if $i<n_{V}$ then

$component$.append($V[i]$)

end if

end for

$components$.append($component$)

end for

return $components$

Algorithm 2 Reduce Hypergraph to Simple Hypergraph

0: $H$ (hypergraph)

0: $H\\_new$ (reduced hypergraph)

$H\\_new\leftarrow H$.copy()

$edge\\_list\leftarrow$ list($H.edges$)

for $i$ in range(len($edge\\_list$)) do

for $j$ in range(len($edge\\_list$)) do

if $i\neq j$ and $edge\\_list[j]\subseteq edge\\_list[i]$ then

$edge\\_list\\_new\leftarrow$ list($H\\_new.edges$)

if $edge\\_list[j]$ in $edge\\_list\\_new$ then

$H\\_new$.remove_edge($edge\\_list[j]$)

end if

end if

end for

end for

return $H\\_new$

Algorithm 3 Hypergraph Expansion

0: $H$ (hypergraph), $mode$ (string)

0: $G$ (normal graph)

$vertices\leftarrow H.vertices\\_dict$

$vertices\\_swap\leftarrow\\{v:k\textbf{ for }k,v\in vertices.items()\\}$

$edges=graph.edges\\_dict$

$vertices\\_length=\textrm{len}(vertices)$

$edges\\_length=\textrm{len}(edges)$

if $mode==\textrm{`clique'}$ then

$G=\textrm{ Create\\_Normal}(vertices\\_length)$

for $edge\textrm{ in range}(edges\\_length)$ do

for $i\textrm{ in }Combinations(edges[edge],2)$ do

$G.add\\_edge(vertices\\_swap[i[0]],vertices\\_swap[i[1]])$

end for

end for

else if $mode==\textrm{`star'}$ then

$G=\textrm{ Create\\_Normal}(vertices\\_length)$

for $edge\textrm{ in range}(edges\\_length)$ do

for $i\textrm{ in }\textrm{Combinations}(edges[edge],2)$ do

$G.add\\_edge(edge+vertices\\_length,vertices\\_swap[i])$

end for

end for

end if

return $G$


# MuLMS-AZ: An Argumentative Zoning Dataset
for the Materials Science Domain

Timo Pierre Schrader1,6 Teresa Bürkle2 Sophie Henning1,3 Sherry Tan4 Matteo
Finco2
Stefan Grünewald1,5 Maira Indrikova2 Felix Hildebrand2 Annemarie Friedrich6
1Bosch Center for Artificial Intelligence, Renningen, Germany
2Robert Bosch GmbH, Stuttgart, Germany 3LMU Munich, Germany 4TU Darmstadt,
Germany
5University of Stuttgart, Germany 6University of Augsburg, Germany
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

Scientific publications follow conventionalized rhetorical structures.
Classifying the Argumentative Zone (AZ), e.g., identifying whether a sentence
states a Motivation, a Result or Background information, has been proposed to
improve processing of scholarly documents. In this work, we adapt and extend
this idea to the domain of materials science research. We present and release
a new dataset of 50 manually annotated research articles. The dataset spans
seven sub-topics and is annotated with a materials-science focused multi-label
annotation scheme for AZ. We detail corpus statistics and demonstrate high
inter-annotator agreement. Our computational experiments show that using
domain-specific pre-trained transformer-based text encoders is key to high
classification performance. We also find that AZ categories from existing
datasets in other domains are transferable to varying degrees.

MuLMS-AZ: An Argumentative Zoning Dataset
for the Materials Science Domain

Timo Pierre Schrader1,6 Teresa Bürkle2 Sophie Henning1,3 Sherry Tan4 Matteo
Finco2 Stefan Grünewald1,5 Maira Indrikova2 Felix Hildebrand2 Annemarie
Friedrich6 1Bosch Center for Artificial Intelligence, Renningen, Germany
2Robert Bosch GmbH, Stuttgart, Germany 3LMU Munich, Germany 4TU Darmstadt,
Germany 5University of Stuttgart, Germany 6University of Augsburg, Germany
<EMAIL_ADDRESS><EMAIL_ADDRESS>

##  1 Introduction

In academic writing, it is custom to adhere to a rhetorical argumentation
structure to convince readers of the relevance of the work to the field
(Swales, 1990). For example, authors typically first indicate a gap in prior
work before stating the goal of their own research. Argumentative Zoning (AZ)
is a natural language processing (NLP) task in which sentences are classified
according to their argumentative roles with varying granularity (Teufel et
al., 1999, 2009). AZ information can then be used for summarization (Teufel
and Moens, 2002; El-Ebshihy et al., 2020), improved citation indexing (Teufel,
2006), or writing assistance (Feltrim et al., 2006).

Label | Count | Label | Count
---|---|---|---
Motivation | 363 | Explanation | 603
Background | 3155 | Results | 2953
\- PriorWork | 1824 | Conclusion | 680
Experiment | 2579 | Heading | 702
\- Prep. | 962 | Caption | 485
\- Charact. | 1347 | Metadata | 210
Table 1: MuLMS-AZ label counts (multi-label).

Manually annotated AZ datasets (Teufel et al., 1999; Fisas et al., 2016;
Soldatova and Liakata, 2007) only exist for few domains and employ differing
annotation schemes. The resulting models are not directly applicable to our
domain of interest, materials science research, which presents a challenging
domain for current NLP methods (e.g., Mysore et al., 2019; Friedrich et al.,
2020; O’Gorman et al., 2021). In this paper, we present MuLMS-AZ, the first
dataset annotated for AZ in this domain. Working together with domain experts,
we derive a hierarchical multi-label annotation scheme (see Table 1). Our
scheme includes domain-specific labels such as descriptions of the materials’
Preparation and Characterization, which are crucial distinctions also for NLP
applications from the domain experts’ view.

This resource paper makes the following contributions:

  * •

We present a dataset of 50 scientific articles (more than 10,000 sentences) in
the domain of materials science manually annotated by domain experts with a
hierarchical fine-grained annotation scheme for AZ with high agreement. The
corpus will be publicly released.111https://github.com/boschresearch/mulms-az-
codi2023

  * •

We apply several neural models to our dataset that will serve as strong
baselines for future work using our new benchmark. We find (a) that using
domain-specific pre-trained transformers is key to a successful model, (b)
that multi-task learning with existing AZ datasets leads to small benefits,
and (c) that the effectiveness of transfer learning of materials science AZ
labels from other corpora differs by label.

##  2 Related Work

In this section, we describe related work on AZ.

AZ Datasets. Table 2 shows the statistics of several related datasets. Three
larger-scale datasets manually annotated with AZ information are the AZ-CL
corpus (Teufel et al., 1999; Teufel and Moens, 1999), consisting of
computational linguistics papers, the Dr. Inventor Multi-Layer Scientific
Corpus (DRI, Fisas et al., 2016, 2015), featuring computer graphics papers,
and, closest to our domain, the ART corpus (Soldatova and Liakata, 2007),
covering topics in physical chemistry and biochemistry. Appendix E explains
these datasets in more detail. Teufel et al. (2009) also apply and adapt the
annotation scheme of the AZ-CL corpus to the chemistry domain. Accuosto et al.
(2021) label sentences with argumentation-related categories (e.g., proposal,
means, observation). Making use of sentence-wise author-provided keywords, a
dataset of about 388k abstracts with silver standard rhetorical role
annotations has been derived from PubMed/MEDLINE (de Moura and Feltrim, 2018).

Modeling. AZ has been modeled as a sentence classification task using maximum
entropy models (Teufel and Kan, 2009), SVMs, and CRFs (Guo et al., 2011)
leveraging a variety of word, grammatical, heuristic, and discourse features
(Guo et al., 2013). Ensemble-based classifiers have also been shown to be
effective (Badie et al., 2018; Asadi et al., 2019). LSTM-based models relying
on word embeddings have been applied to AZ and to the fundamentally very
similar task of assigning zones to sentences in job ads (Liu, 2017; de Moura
and Feltrim, 2018; Gnehm and Clematide, 2020). BERT-style (Devlin et al.,
2019) models work well for AZ (Mo et al., 2020; Brack et al., 2022). Multi-
task training has been found to be beneficial for these models both in-domain
(Lauscher et al., 2018) as well as cross-domain (Brack et al., 2021).

Datasets in the Materials Science Domain. Several datasets targeting the
domain of materials science research have recently been released. Mysore et
al. (2019) annotate paragraphs describing synthesis procedures with graph
structures capturing relations and typed arguments. Friedrich et al. (2020)
mark similar graph structures corresponding to experiment information in 45
open-access publications. Several works and datasets address named entity
recognition in the domain (Yamaguchi et al., 2020; O’Gorman et al., 2021).

| AZ-CL | ART | DRI | MuLMS-AZ
---|---|---|---|---
# docs | 80 | 225 | 40 | 50
# sents | 12818 | 34995 | 10784 | 10186
# labels | 7 | 11 | 10 | 12
Table 2: Manually annotated AZ corpora. Label | Description | Example
---|---|---
Motivation | aims/motivation of the study | In this study, we perform a systematic analysis of …
Background | textbook-like technical background | The method is based on the Kelvin equation.
\- PriorWork | specific prior work relevant to current study | Irmawati et al. has concluded that …
Experiment | description of the experiment | We evaluate PtCo nanoparticle catalyst …
\- Preparation | steps describing the preparation of samples | The mixture was subjected to stirring for 60 minutes.
\- Charact. | characterizations and characterization | Ni foam surface coverage of the WO3 thin film and its
| techniques of the involved materials | homogeneity were analyzed by energy–dispersive X-ray
|  | spectroscopy (EDS).
Explanation | statements (hypotheses or assumptions) | In our calculation, all Pt loadings were considered
| relevant to results or experimental settings | to be electrochemically active.
Results | details on experimental results | The hydrogen adsorption/desorption peak is at about 0.2V.
Conclusion | conclusions and take-aways | This result indicated that …
Table 3: Content-based MuLMS-AZ Argumentative Zoning sentence labels.

##  3 Data Sources and Annotated Corpus

In this section, we present our new dataset.

Source of Texts and Preprocessing. We select 50 scientific articles licensed
under CC-BY from seven sub-areas of materials science: electrolysis, graphene,
polymer electrolyte fuel cell (PEMFC), solid oxide fuel cell (SOFC), polymers,
semiconductors, and steel. The four SOFC papers were selected from the SOFC-
Exp corpus Friedrich et al. (2020). 11 papers were selected from the OA-STM
corpus222https://github.com/elsevierlabs/OA-STM-Corpus and classified into the
above subject areas by a domain expert. The majority of the papers were found
via PubMed333https://pubmed.ncbi.nlm.nih.gov/ and DOAJ444https://doaj.org/
using queries prepared by a domain expert. For the OA-STM data, we use the
sentence segmentation provided with the corpus, which has been created using
GENIA tools (Tsuruoka and Tsujii, 2005). For the remaining texts, we rely on
the sentence segmentation provided by INCEpTION v21.0 (Klie et al., 2018) with
some manual fixes.

Annotation Scheme. AZs are functional sentence types, i.e., they capture the
rhetorical function of a sentence. Together with several domain experts, we
design a hierarchical scheme tailored to the materials science domain as shown
in Table 3. In addition, we provide Abstract, Heading, Metadata, Caption,
Figure/Table annotations for structural information. We assume a multi-label
setting in which annotators may assign any number of labels to a sentence. Our
detailed guidelines are available with our dataset.

Corpus Statistics. Documents are rather long (on average 203.7 sentences per
document with a standard deviation of $\pm$73.2). There is a tendency towards
long sentences (28.7 tokens per sentence on average), but with high variation
of $\pm$17.9 due to, e.g., short headings. Table 1 shows how often each AZ
label occurs. When ignoring tags for structural information 8133 sentences
have exactly one AZ label (or the AZ label and its supertype), 1056 sentences
have two labels, and 11 sentences have 3 labels. Labels are similarly
distributed across data splits (see Appendix D).

Inter-Annotator Agreement. Our entire dataset has been annotated by a single
annotator, a graduate student of materials science, who was also involved in
the design of the annotation scheme. We compare the annotations of this main
annotator to those of another annotator who holds a Master’s degree in
materials science and a PhD in engineering. The agreement study is performed
on 5 documents (357 sentences). Due to the multi-label scenario, following
Krippendorff (1980) we measure $\kappa$ (Cohen, 1960) for each label
separately, comparing whether each annotator used a particular label on an
instance or not (see Table 4). Our annotators achieve “substantial” agreement
(Landis and Koch, 1977) on most labels, “perfect” agreement on identifying
Headings (see also Appendix D). Lower, though still “moderate”, agreement on
Motivation, Explanation and Conclusion can in part be explained by their lower
frequency which makes it generally harder to obtain high $\kappa$-values.
Intuitively, they also have a more difficult nature compared to the other
tags, e.g., we observe disagreements regarding what constitutes a Motivation
or an Explanation versus what is purely reporting Background. The full
confusion matrix and a discussion of agreement on subtags are given in
Appendix D; a discussion of multi-label examples can be found in Appendix F.

Our scores are in the same ballpark as those reported by Teufel et al. (1999)
on a similar annotation task. For their 7-way task, they report $\kappa$
scores around 0.71-0.75, with differences between categories in one-vs-all
measurements ranging from about 0.49 to 0.78. In sum, we conclude that
agreement on AZ is satisfactory in our dataset.

AZ Label | $\kappa$ | AZ Label | $\kappa$
---|---|---|---
Heading | 0.89 | Metadata | 0.76
Motivation | 0.44 | Background | 0.75
Conclusion | 0.55 | Experiment | 0.78
Explanation | 0.39 | Results | 0.70
Table 4: IAA for AZ on 357 sentences.

##  4 Modeling

We model AZ as a multi-label classification problem, using BERT (Devlin et
al., 2019) as the underlying text encoder. We also test domain-specific pre-
trained variants of BERT. SciBERT (Beltagy et al., 2019) has been pre-trained
on articles in the scientific domain. MatSciBERT (Gupta et al., 2022) is a
version of SciBERT further pre-trained on materials science articles. We use
the CLS embedding as input to a linear layer, transform logits using a sigmoid
function and choose labels if their respective score exceeds 0.5. For multi-
task experiments with other datasets, we use a single shared language model
and one linear output layer per dataset. For hyperparameters, see Appendix A.

As shown in Table 1, the dataset suffers from strong class imbalance.
Classifiers tend to underperform on minority labels (Johnson and Khoshgoftaar,
2019). To address this problem, we apply the multi-label random oversampling
(ML-ROS, Charte et al., 2015) algorithm during training. The main idea behind
ML-ROS is to dynamically duplicate instances of minority classes while taking
the multi-label nature of the problem into account. In a nutshell, the
algorithm performs several oversampling iterations, keeping track of the
imbalance ratios associated with each label and choosing instances that carry
minority labels until a predefined number of additional samples have been
chosen. Details are given in Appendix B.

##  5 Experimental Results

We here detail our experimental results.

Settings. We split our corpus into train, dev, and test sets of 36, 7, and 7
documents. For all experiments and for hyperparameter tuning, we always train
five models. The training data is split into five folds. Similar to cross-
validation, we train on four folds and use the fifth fold for model selection
(cf. van der Goot, 2021), repeating this process five times (also for
hyperparameter tuning). The dev set is only used for tuning, and we report
scores for the five models on test. In this setting, deviations are naturally
higher than when reporting results for the same training data. For
hyperparameters and implementation details, see Appendix A. To evaluate our
experiments, we use hierarchical precision, recall, and F1 (Silla and Freitas,
2011). These scores operate on the sets of labels per instance, always
including the respective supertypes of gold or predicted labels.

Method | LM | mic.-F1 | mac.-F1
---|---|---|---
No Oversampling | BERT | 72.6±1.0 | 65.5±0.7
| MatSciBERT | 76.3±0.7 | 70.1±0.7
| SciBERT | 76.2±0.9 | 70.2±0.6
ML-ROS | SciBERT | 76.7±0.7 | 70.6±0.9
\+ MultiTask ART | SciBERT | 75.0±0.9 | 68.9±1.1
\+ MultiTask AZ-CL | SciBERT | 77.2±0.3 | 71.1±0.5
human agreement* |  | 78.7 | 74.9
Table 5: AZ classification results on MuLMS-AZ test set. *Not directly
comparable: computed on documents from agreement study (see Appendix D).

Results. Table 5 shows the performance of our neural models on MuLMS-AZ.
Overall, the categories can be learned well, approaching our estimate of human
agreement. SciBERT clearly outperforms BERT, i.e., using domain-specific
embeddings is a clear advantage. However, MatSciBERT does not add upon
SciBERT. We hence conduct the remaining experiments using SciBERT. Using ML-
ROS results in minor increases for most labels (see also Appendix G). When
multi-task learning with the AZ-CL dataset (using $40\%$ of its samples),
further increases are observed. It is worth noting that multi-task training
with ART does not result in increases although the chemistry domain should be
much closer to our domain. This might indicate that despite the domain
discrepancy, AZ annotations in AZ-CL are more compatible with ours.

Training data | PM Label | P | R
---|---|---|---
PM, AZ-CL, ART, DRI | Objective | 36.1 | 28.3
PM, AZ-CL, ART, DRI | Background | 84.2 | 40.0
PM, ART, DRI | Method | 58.1 | 74.7
PM, ART, DRI | Result | 82.4 | 30.9
PM, ART, DRI | Conclusion | 43.5 | 29.9
MuLMS-AZ | Objective | 56.8 | 54.3
MuLMS-AZ | Background | 82.1 | 78.8
MuLMS-AZ | Method | 79.9 | 78.2
MuLMS-AZ | Result | 82.1 | 83.2
MuLMS-AZ | Conclusion | 43.5* | 29.9*
Table 6: Results for transfer learning experiment. Precision and recall on
MuLMS-AZ test set. *not a typo.

As a first step to explaining what part of rhetorical information can be
induced based only on data from other corpora, we perform a transfer learning
experiment. We carefully manually map the AZ labels of the various datasets
(see Appendix E) to the coarse-grained categories used by PubMed (PM). Using
these mapped labels, we train binary classifiers that aim to detect the
presence of a particular PM label. As training data, we use ART, DRI, and a
selection of documents from the PM dataset by de Moura and Feltrim (2018) that
were published in materials science journals (see Appendix C). We add AZ-CL to
the training data only if an unambiguous mapping of its categories to the PM
label in question is possible. Here, we use the dev set of MuLMS-AZ for model
selection and hyperparameter tuning. Results for running the resulting
classifiers on MuLMS-AZ are reported in Table 6. For Background and Results,
we observe a high precision, which indicates that similar rhetorical elements
may be used. Objective and Method seem to differ most across datasets as they
are likely very domain- and problem-specific. When training with mapped labels
on the entire MuLMS-AZ, we observe much higher recall scores across all label
groups, again indicating the usefulness of our in-domain training data.

##  6 Conclusion and Outlook

We have presented a new AZ corpus in the field of materials science annotated
by domain experts with high agreement. Our experimental results demonstrate
that strong classifiers can be learned on the data and that AZ labels can be
transferred from related datasets only to a limited extent.

Our new dataset opens up new research opportunities on cross-domain AZ, class
imbalance scenarios, and integrating AZ information in information extraction
tasks in materials science.

## Limitations

This resource paper describes the dataset in detail, providing strong
baselines and first initial cross-domain experiments. It does not aim to
provide an extensive set of experiments on cross-domain argumentative zoning
yet.

The entire dataset is only singly-annotated. The agreement study was performed
on complete documents and hence has only limited data for several labels. Due
to the limited funding of the project, we could double-annotate the entire
dataset.

Finally, we only test one model class (BERT-based transformers). A potential
next step is to test a bigger variety of models and embeddings. Because AZ
labels are interdependent within a document, especially document-level models
or CRF-based models are promising methods to try. We have also tested only one
method (multi-label random oversampling) to deal with the strong class
imbalance in the dataset. We have not yet tested further such methods (Henning
et al., 2023) or data augmentation methods.

## Ethical Considerations

We took care of potential license issue of the data underlying our dataset by
exclusively selecting open-access articles published under CC BY.

The main annotator was paid above the minimum wage of our country in the
context of a full-time internship. The annotator was aware of the goal of the
study and consents to the public release of the data. The remaining domain
experts participated on a voluntary basis due to their interest in the topic.

## References

  * Accuosto et al. (2021) Pablo Accuosto, Mariana Neves, and Horacio Saggion. 2021.  Argumentation mining in scientific literature: From computational linguistics to biomedicine.  In _Frommholz I, Mayr P, Cabanac G, Verberne S, editors. BIR 2021: 11th International Workshop on Bibliometric-enhanced Information Retrieval; 2021 Apr 1; Lucca, Italy. Aachen: CEUR; 2021. p. 20-36_. CEUR Workshop Proceedings.
  * Asadi et al. (2019) Nasrin Asadi, Kambiz Badie, and Maryam Tayefeh Mahmoudi. 2019.  Automatic zone identification in scientific papers via fusion techniques.  _Scientometrics_ , 119(2):845–862.
  * Badie et al. (2018) Kambiz Badie, Nasrin Asadi, and Maryam Tayefeh Mahmoudi. 2018.  Zone identification based on features with high semantic richness and combining results of separate classifiers.  _Journal of Information and Telecommunication_ , 2(4):411–427.
  * Beltagy et al. (2019) Iz Beltagy, Kyle Lo, and Arman Cohan. 2019.  SciBERT: A pretrained language model for scientific text.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3615–3620, Hong Kong, China. Association for Computational Linguistics.
  * Brack et al. (2021) Arthur Brack, Anett Hoppe, Pascal Buschermöhle, and Ralph Ewerth. 2021.  Sequential sentence classification in research papers using cross-domain multi-task learning.  _CoRR_ , abs/2102.06008.
  * Brack et al. (2022) Arthur Brack, Anett Hoppe, Pascal Buschermöhle, and Ralph Ewerth. 2022.  Cross-domain multi-task learning for sequential sentence classification in research papers.  In _Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries_ , pages 1–13.
  * Charte et al. (2015) Francisco Charte, Antonio J. Rivera, María J. del Jesus, and Francisco Herrera. 2015.  Addressing imbalance in multilabel classification: Measures and random resampling algorithms.  _Neurocomputing_ , 163:3–16.  Recent Advancements in Hybrid Artificial Intelligence Systems and its Application to Real-World Problems Progress in Intelligent Systems Mining Humanistic Data.
  * Cohen (1960) Jacob Cohen. 1960.  A coefficient of agreement for nominal scales.  _Educational and Psychological Measurement_ , 20(1):37–46.
  * de Moura and Feltrim (2018) G. Bennemann de Moura and V. Delisandra Feltrim. 2018.  Using lstm encoder-decoder for rhetorical structure prediction.  In _2018 7th Brazilian Conference on Intelligent Systems (BRACIS)_ , pages 278–283, Los Alamitos, CA, USA. IEEE Computer Society.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * El-Ebshihy et al. (2020) Alaa El-Ebshihy, Annisa Maulida Ningtyas, Linda Andersson, Florina Piroi, and Andreas Rauber. 2020.  ARTU / TU Wien and artificial researcher@ LongSumm 20.  In _Proceedings of the First Workshop on Scholarly Document Processing_ , pages 310–317, Online. Association for Computational Linguistics.
  * Feltrim et al. (2006) Valéria D Feltrim, Simone Teufel, Maria Graças V das Nunes, and Sandra M Aluísio. 2006.  Argumentative zoning applied to critiquing novices’ scientific abstracts.  In _Computing Attitude and Affect in Text: Theory and Applications_ , pages 233–246. Springer.
  * Fisas et al. (2016) Beatriz Fisas, Francesco Ronzano, and Horacio Saggion. 2016.  A multi-layered annotated corpus of scientific papers.  In _Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16)_ , pages 3081–3088, Portorož, Slovenia. European Language Resources Association (ELRA).
  * Fisas et al. (2015) Beatriz Fisas, Horacio Saggion, and Francesco Ronzano. 2015.  On the discoursive structure of computer graphics research papers.  In _Proceedings of The 9th Linguistic Annotation Workshop_ , pages 42–51, Denver, Colorado, USA. Association for Computational Linguistics.
  * Friedrich et al. (2020) Annemarie Friedrich, Heike Adel, Federico Tomazic, Johannes Hingerl, Renou Benteau, Anika Marusczyk, and Lukas Lange. 2020.  The SOFC-exp corpus and neural approaches to information extraction in the materials science domain.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1255–1268, Online. Association for Computational Linguistics.
  * Gnehm and Clematide (2020) Ann-Sophie Gnehm and Simon Clematide. 2020.  Text zoning and classification for job advertisements in German, French and English.  In _Proceedings of the Fourth Workshop on Natural Language Processing and Computational Social Science_ , pages 83–93, Online. Association for Computational Linguistics.
  * Guo et al. (2011) Yufan Guo, Anna Korhonen, and Thierry Poibeau. 2011.  A weakly-supervised approach to argumentative zoning of scientific documents.  In _Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing_ , pages 273–283, Edinburgh, Scotland, UK. Association for Computational Linguistics.
  * Guo et al. (2013) Yufan Guo, Roi Reichart, and Anna Korhonen. 2013.  Improved information structure analysis of scientific documents through discourse and lexical constraints.  In _Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 928–937, Atlanta, Georgia. Association for Computational Linguistics.
  * Gupta et al. (2022) Tanishq Gupta, Mohd Zaki, N. M. Anoop Krishnan, and Mausam. 2022.  MatSciBERT: A materials domain language model for text mining and information extraction.  _npj Computational Materials_ , 8(1):102.
  * Henning et al. (2023) Sophie Henning, William Beluch, Alexander Fraser, and Annemarie Friedrich. 2023\.  A survey of methods for addressing class imbalance in deep-learning based natural language processing.  In _Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics_ , pages 523–540, Dubrovnik, Croatia. Association for Computational Linguistics.
  * Johnson and Khoshgoftaar (2019) Justin M Johnson and Taghi M Khoshgoftaar. 2019.  Survey on deep learning with class imbalance.  _Journal of Big Data_ , 6(1):1–54.
  * Klie et al. (2018) Jan-Christoph Klie, Michael Bugert, Beto Boullosa, Richard Eckart de Castilho, and Iryna Gurevych. 2018.  The INCEpTION platform: Machine-assisted and knowledge-oriented interactive annotation.  In _Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations_ , pages 5–9, Santa Fe, New Mexico. Association for Computational Linguistics.
  * Krippendorff (1980) Klaus Krippendorff. 1980.  _Krippendorff, Klaus, Content Analysis: An Introduction to its Methodology. Beverly Hills, CA: Sage, 1980._ Sage Publications, Inc.
  * Landis and Koch (1977) J Richard Landis and Gary G Koch. 1977.  An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers.  _Biometrics_ , pages 363–374.
  * Lauscher et al. (2018) Anne Lauscher, Goran Glavaš, Simone Paolo Ponzetto, and Kai Eckert. 2018.  Investigating the role of argumentation in the rhetorical analysis of scientific publications with neural multi-task learning models.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3326–3338, Brussels, Belgium. Association for Computational Linguistics.
  * Liakata and Soldatova (2008) Maria Liakata and Larisa Soldatova. 2008.  Guidelines for the annotation of general scientific concepts.  _Aberystwyth University, JISC Project Report http://ie-repository. jisc. ac. uk/88_.
  * Liu (2017) Haixia Liu. 2017.  Automatic argumentative-zoning using word2vec.  _CoRR_ , abs/1703.10152.
  * Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. 2019.  Decoupled weight decay regularization.
  * Mo et al. (2020) Wang Mo, Cui Yunpeng, Chen Li, and Li Huan. 2020.  A deep learning-based method of argumentative zoning for research articles.  _Data Analysis and Knowledge Discovery_ , 4(6):60–68.
  * Mysore et al. (2019) Sheshera Mysore, Zachary Jensen, Edward Kim, Kevin Huang, Haw-Shiuan Chang, Emma Strubell, Jeffrey Flanigan, Andrew McCallum, and Elsa Olivetti. 2019.  The materials science procedural text corpus: Annotating materials synthesis procedures with shallow semantic structures.  In _Proceedings of the 13th Linguistic Annotation Workshop_ , pages 56–64, Florence, Italy. Association for Computational Linguistics.
  * O’Gorman et al. (2021) Tim O’Gorman, Zach Jensen, Sheshera Mysore, Kevin Huang, Rubayyat Mahbub, Elsa Olivetti, and Andrew McCallum. 2021.  MS-mentions: Consistently annotating entity mentions in materials science procedural text.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 1337–1352, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Silla and Freitas (2011) Carlos N Silla and Alex A Freitas. 2011.  A survey of hierarchical classification across different application domains.  _Data Mining and Knowledge Discovery_ , 22(1):31–72.
  * Soldatova and Liakata (2007) Larisa Soldatova and Maria Liakata. 2007.  An ontology methodology and cisp-the proposed core information about scientific papers.  _JISC Project Report_.
  * Swales (1990) John M. Swales. 1990.  Discourse analysis in professional contexts.  _Annual Review of Applied Linguistics_ , 11:103–114.
  * Teufel (2006) Simone Teufel. 2006.  Argumentative zoning for improved citation indexing.  In _Computing attitude and affect in text: Theory and Applications_ , pages 159–169. Springer.
  * Teufel et al. (1999) Simone Teufel, Jean Carletta, and Marc Moens. 1999.  An annotation scheme for discourse-level argumentation in research articles.  In _Ninth Conference of the European Chapter of the Association for Computational Linguistics_ , pages 110–117, Bergen, Norway. Association for Computational Linguistics.
  * Teufel and Kan (2009) Simone Teufel and Min-Yen Kan. 2009.  Robust argumentative zoning for sensemaking in scholarly documents.  In _Advanced language technologies for digital libraries_ , pages 154–170. Springer.
  * Teufel and Moens (1999) Simone Teufel and Marc Moens. 1999.  Discourse-level argumentation in scientific articles: human and automatic annotation.  In _Towards Standards and Tools for Discourse Tagging_.
  * Teufel and Moens (2002) Simone Teufel and Marc Moens. 2002.  Articles summarizing scientific articles: Experiments with relevance and rhetorical status.  _Computational Linguistics_ , 28(4):409–445.
  * Teufel et al. (2009) Simone Teufel, Advaith Siddharthan, and Colin Batchelor. 2009.  Towards domain-independent argumentative zoning: Evidence from chemistry and computational linguistics.  In _Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing_ , pages 1493–1502, Singapore. Association for Computational Linguistics.
  * Tsuruoka and Tsujii (2005) Yoshimasa Tsuruoka and Jun’ichi Tsujii. 2005.  Bidirectional inference with the easiest-first strategy for tagging sequence data.  In _Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing_ , pages 467–474, Vancouver, British Columbia, Canada. Association for Computational Linguistics.
  * van der Goot (2021) Rob van der Goot. 2021.  We need to talk about train-dev-test splits.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 4485–4494, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Yamaguchi et al. (2020) Kyosuke Yamaguchi, Ryoji Asahi, and Yutaka Sasaki. 2020.  SC-CoMIcs: A superconductivity corpus for materials informatics.  In _Proceedings of the 12th Language Resources and Evaluation Conference_ , pages 6753–6760, Marseille, France. European Language Resources Association.

## Appendix

##  Appendix A Hyperparameters

We implement all our models using PyTorch. We use AdamW (Loshchilov and
Hutter, 2019) as the optimizer for all our models and set the batch size to
$16/32$ depending on what works best and GPU restrictions. The learning rate
stays constant after a linear warmup phase. We set a dropout rate to $0.1$ for
the linear layer that takes the contextualized embeddings that are produced by
BERT as input. Early stopping is applied if the micro-F1 score has not
improved for more than 3 epochs. Binary cross entropy is the loss function for
the MuLMS-AZ output layer, whereas cross entropy is the loss function used for
optimizing the multi-task output heads corresponding to the other AZ datasets.
Table 7 lists the various learning rates found during grid search. We tested
different learning rates between 1e-4 and 1e-7. A refinement of the grid was
done after an initial search, which almost always leads to a second search
area within the range of 1e-6 to 9e-6. When using ML-ROS, we oversample by
$20\%$. Training was performed on a single Nvidia A100 GPU or alternatively
V100 GPU.

Method | LM | Learning Rate
---|---|---
No Oversampling | BERT | 3e-6
| MatSciBERT | 8e-6
| SciBERT | 3e-6
ML-ROS | SciBERT | 2e-6
\+ MT (+PM) | SciBERT | 7e-6
\+ MT (+ART) | SciBERT | 2e-6
\+ MT (+AZ-CL) | SciBERT | 2e-6
\+ MT (+DRI) | SciBERT | 1e-6
\+ MT (+ART+AZ+DRI) | SciBERT | 8e-6
Data Augm. (+PM) | SciBERT | 8e-6
Table 7: Learning rates of the different model reported in Table 12

##  Appendix B Multi-Label Random Oversampling (ML-ROS) Algorithm

Figure 1 details our adaption of the multi-label random oversampling (ML-ROS)
algorithm originally proposed by Charte et al. (2015). In the initialization
(lines 3-7), for each label, all the instances that carry a particular label
are collected in what Charte et al. call a bag. The main part of the algorithm
(lines 10-24) does the following: For each label $y$, the Imbalance Ratio per
label ($\mathrm{IRLbl}$), which is the ratio between the count of the most
frequent label and the count of $y$, is calculated:

$\displaystyle\mathrm{IRLbl}(y)=\frac{\max_{y^{\prime}\in
L}\sum_{i=1}^{|D|}h(y^{\prime},Y_{i})}{\sum_{i=1}^{|D|}h(y,Y_{i})}$

$D$ is the dataset, $L$ is the label set, $Y_{i}$ is the set of labels
assigned to the $i$-th sample and $h$ is an indicator function evaluating if
$y\in Y_{i}$. Hence, the larger the value, the less frequently $y$ occurs
compared to the most frequent label.

The per-label values are then used to determine the mean imbalance ratio
($\mathrm{MeanIR}$):

$\displaystyle\mathrm{MeanIR}=\frac{1}{|L|}\sum_{y^{\prime}\in
L}\mathrm{IRLbl}(y^{\prime})$

For each of the labels with an imbalance ratio exceeding the current MeanIR, a
random instance of this label is duplicated.

The main part is repeated until the pre-specified size of the oversampled
dataset is reached. Our implementation differs from Charte et al. in that we
update $\mathrm{meanIR}$ in each iteration step and also oversample labels
originally not being a minority label when their $\mathrm{IRLbl}$ exceeds
$\mathrm{MeanIR}$ at the beginning of an iteration step.

⬇

1Inputs: <Dataset> D, <Percentage> P

2Outputs: Oversampled dataset

3$samplesToDuplicate$ <– $|D|/100*P$ # P %

4$L$ <– labelsInDataset($D$) # Obtain the full set of labels

5for each $label$ in $L$ do # Bags of samples for each label

6 $Bag_{label}$ <– getSamplesPerLabel($label$)

7end for

8

9while $samplesToDuplicate$ > 0 do # Loop duplicating instances

10 $MeanIR$ <– calculateMeanIR($D,L$)

11 # Gather minority bags (bag: all instances of a given label)

12 $minBags$ = []

13 for each $label$ in $L$ do

14 $IRLbl_{label}$ <– calculateIRperLabel($D,label$)

15 if $IRLbl_{label}$ > $MeanIR$ then

16 $minBags$ += $Bag_{label}$

17 end if

18 end for

19 # Duplicate a random sample from each minority bag

20 for each $minBag_{i}$ in $minBags$ do

21 $x$ <– random($1,|minBag_{i}|$)

22 duplicateSample($minBag_{i},x$)

23 $--samplesToDuplicate$

24 end for

25end while

Figure 1: Pseudocode for adapted (dynamic) ML-ROS algorithm.

##  Appendix C List of Materials Science Journals

We used the list of materials-science related journals collected on Wikipedia
to filter for abstracts in the PubMed Medline corpus published in journals.555
https://en.wikipedia.org/w/index.php?title=List˙of˙materials˙science˙journals&oldid=1078212543

##  Appendix D Further Corpus Statistics for MuLMS-AZ

Table 8 gives the counts of sentences carrying a particular AZ label.
Distributions are similar across data splits. Table 8 also lists counts for
Abstract, which we decide to exclude from our modeling experiments because
including it resulted in performance decreases due to confusion with other
labels. Locating the abstract in a document can usually be solved in rule-
based ways as abstracts of publications are commonly available in a machine-
readable format.

During annotation, we introduced two subtypes of Explanation, Hypothesis and
Assumption, distinguishing between scientific hypotheses and assumptions made
by the author in cases where often choices are possible. As the overall counts
and agreement were low, we decided to only use the supertype Explanation in
all experiments.

Figure 2(a) shows the label coincidence matrix between the two annotators in
the inter-annotator agreement study, i.e., how often each pair of labels co-
occurred on an instance. For all labels except Motivation, the majority of
coincidences occur on the diagonal. Results is the label most mixed up with
others, possibly because these sentences often are long and also contain
interpretative information of the other rhetorical types.

Figure 2(a) breaks this information down the level including subtypes.
Characterization and Preparation are rarely confused by the domain experts.
Similarly, Background and PriorWork are reliably distinguished.

#### Agreement on sub-labels.

Our agreement study contained only 12 Caption instances. Data inspection
showed that the additional (not the main) annotator neglected to use this tag
where appropriate, using only content-related tags on these instances. There
were also not enough instances of the subtypes Preparation and
Experiment_Characterization to measure agreement. On identifying the subtype
Background_PriorWork, annotators achieve a $\kappa$ of 0.8, with (minor)
disagreements mainly with regard to using Background or its subtype.

Label | total | train | dev | test
---|---|---|---|---
Motivation | 363 | 273 | 44 | 46
Background | 3155 | 2423 | 440 | 292
-PriorWork | 1824 | 1387 | 265 | 172
Experiment | 2579 | 1896 | 394 | 289
-Characterization | 1347 | 982 | 200 | 165
-Preparation | 962 | 705 | 146 | 111
Explanation | 603 | 430 | 91 | 82
Results | 2953 | 2146 | 440 | 367
Conclusion | 680 | 507 | 106 | 67
Abstract | 269 | 190 | 28 | 51
Caption | 485 | 309 | 91 | 85
Heading | 702 | 536 | 96 | 70
Metadata | 210 | 142 | 40 | 28
Table 8: Label counts on the complete dataset and on data split subsets.
Multi-label counts: Number of sentences in which the label is present. Due to
multi-labeling, the sum of these columns exceeds the total amount of
sentences. For hierarchical labels, the super-label count includes all sub-
label counts.

(a) Coincidence matrix for coarse AZ labels.

(b) Coincidence matrix for AZ labels with subtypes.

Figure 2: Coincidence matrices of inter-annotator agreement study for AZ
labels on 357 sentences.

#### Agreement on Heading.

As it should be straightforward to identify headings, we looked at the 6 cases
that one annotator labeled as Heading but not the other. We found 4 cases to
result from broken formatting. One Metadata sentence was wrongly labeled
Heading, and the remaining Heading sentence was missed by the other annotator.

#### Human “upper bound”.

In order to provide a rough estimate of how humans would perform on the
classification task, we use the data from the agreement study to compute
hierarchical precision, recall, and F1 scores. Due to insufficient data for
the remaining labels, we only compute the scores over the following labels:
Heading, Metadata, Motivation, Background, PriorWork, Experiment, Preparation,
Characterization, Results, and Conclusion. Table 9 reports detailed scores per
label. Scores have been computed using scikit-learn666https://scikit-
learn.org/stable.

| Precision | Recall | F1 | support
---|---|---|---|---
micro avg. | 77.3 | 80.0 | 78.7 |
macro avg. | 75.0 | 76.1 | 74.9 |
Heading | 100.0 | 81.2 | 89.7 | 32
Metadata | 75.0 | 81.8 | 78.3 | 11
Motivation | 50.0 | 44.4 | 47.1 | 18
Background | 77.0 | 89.5 | 82.8 | 105
-PriorWork | 77.9 | 90.9 | 83.9 | 66
Experiment | 80.0 | 84.5 | 82.2 | 71
-Preparation | 92.6 | 73.5 | 82.0 | 34
Characterization | 61.7 | 78.4 | 69.0 | 37
Results | 85.7 | 70.2 | 77.2 | 94
Conclusion | 50.0 | 66.7 | 57.1 | 18
Table 9: Human agreement computed in terms of hierarchical precision, recall,
and F1.

##  Appendix E Description and Comparison of AZ Datasets.

In this section, we provide a detailed description and comparison of existing
AZ datasets. The various corpora try to capture very similar information.
However, each corpus defines its set of labels in a slightly different way.
Table 10 lists the various labels used in the datasets and groups labels used
for the same or very similar purpose. Table 11 shows the label distributions
of the corpora.

PubMed | AZ-CL | ART | DRI | MuLMS-AZ |  Description
---|---|---|---|---|---
Objective | Aim | Hypothesis | Challenge | Motivation |  A sentence describing the research
|  | Motivation |  |  |  target, goal, aim or the motivation
|  | Goal |  |  |  for the research.
Background | Background | Background | Background | Background |  A statement concerning the knowledge
| Contrast |  |  | PriorWork |  domain or previous related work.
| Basis |  |  |  |
Method | Own |  Object, | Approach | Experiment |  A sentence describing the research
|  | Method |  | Preparation |  procedure, models used, or observations
|  | Model |  | Characteriz. |  made during the research.
|  | Experiment |  | Explanation |
|  | Observation |  |  |
Result | Own | Result | Outcome | Results |  A sentence describing the study findings,
|  |  |  | Explanation |  effects, consequences, and/or analysis of the results.
Conclusion | Own | Conclusion | Outcome | Conclusion |  A statement concerning the support or
|  |  | Futurework |  |  rejection of the hypothesis or suggestions of future research.
– | Text | – | Sentence | – |  Example sentences, broken sentences, etc.
| Other |  | Unspecified |  |
Table 10: AZ Corpus Zones Mapping and Descriptions. Compare to Table 3. Dataset | Label | Count
---|---|---
AZ-CL | Own | 8624
| Other | 2019
| Background | 789
| Contrast | 600
| Aim | 313
| Basis | 246
| Text | 227
ART | Result | 7373
| Background | 6657
| Observation | 4659
| Method | 3751
| Model | 3456
| Conclusion | 3083
| Experiment | 2841
| Object | 1190
| Hypothesis | 656
| Goal | 548
| Motivation | 466
DRI | Approach | 5038
| Background | 1760
| Sentence | 1247
| Outcome | 1175
| Unspecified | 759
| Challenge | 351
| Outcome_Contribution | 219
| FutureWork | 136
| Challenge_Hypothesis | 7
MatSci PubMed | Results | 1282
| Objective | 1264
| Methods | 1198
| Conclusion | 380
| Background | 60
Table 11: Label counts for the different AZ corpora.

#### AZ-CL corpus.

The Argumentative Zoning (AZ, Teufel et al., 1999; Teufel and Moens, 1999)
corpus777https://github.com/WING-NUS/RAZ consists of 80 manually annotated
open-access computational linguistics research articles. Sentences are marked
according to their argumentative zone or rhetorical function as one of the
following classes: Aim, Background, Basis, Contrast, Other, Own or Text.
Inter-annotator agreement is reported as substantial ($\kappa$ = 0.71). The
distribution of classes is quite skewed towards Other and Own.

#### ART corpus.

The ART corpus888https://www.aber.ac.uk/en/cs/research/cb/projects/art/art-
corpus/ (Soldatova and Liakata, 2007) covers topics in physical chemistry and
biochemistry. Articles are annotated according to the CISP/CoreSC annotation
scheme Liakata and Soldatova (2008). Sentences are labeled with one of the
categories Hypothesis, Motivation, Goal of investigation, Background, Object
of investigation, Research method, Model, Experiment, Observation, Result or
Conclusion. The annotation scheme also defines subcategories for some of
these. The corpus has been annotated by domain experts. In a preliminary
study, $\kappa$ was measured as 0.55, however, for the final corpus, only the
annotators that had the highest average agreement were selected. Hence, the
agreement in the final corpus is expected to be higher.

#### DRI corpus.

The Dr. Inventor Multi-Layer Scientific
Corpus999http://sempub.taln.upf.edu/dricorpus (DRI, Fisas et al., 2016, 2015),
contains 40 scientific articles taken from the domain of computer graphics.
Each of the 10,784 sentences was annotated with one of the rhetorical
categories: Challenge, Background, Approach, Outcome or Futurework. They have
also included two other categories Sentence for sentences that are
characterized by segmentation or character encoding errors and Unspecified for
sentences where identification is not possible. Also to note was the
possibility to annotate a combination of two different categories as seen in
the example of: Outcome_Contribution, Challenge_Goal and Challenge_Hypothesis.
Manual annotation reaches a $\kappa$ value of 0.66.

#### PubMed corpus.

The PubMed corpus101010https://github.com/dead/rhetorical-structure-pubmed-
abstracts (de Moura and Feltrim, 2018) contains abstracts of papers in the
biomedical domain extracted from PUBMED/MEDLINE. The collected abstracts were
written in English and annotated with predefined section names by their
authors; based on the mapping provided by the U.S. National Library of
Medicine (NLM), the section names were collapsed into five rhetorical roles:
Background, Objective, Methods, Results, and Conclusions. The abstracts that
did not contain the five mentioned rhetorical roles were removed from the
dataset with the resulting corpus containing close to 5 million sentences. The
dataset is not particularly challenging: a simple CRF model achieves an
F-score of 93.75, an LSTM-based model achieves 94.77 according to de Moura and
Feltrim (2018).

##  Appendix F Examples

In this section, we present and discuss several examples from our dataset.

###  F.1 Example Sentences

  * •

Motivation: Therefore, it is highly desirable to develop an innovative
technology to raise the mass activity of Ir-based OER catalysts to the
targeted level.

  * •

Background: For photocatalytic water splitting using photoelectrochemical
cells (PECs), the charge carriers are created from the photovoltaic effect
close to the catalytic site.

  * •

PriorWork: Proton exchange membrane (PEM) electrolysis, which occurs in acidic
electrolytes (pH 0–7), has better efficiency and enhanced ramping capability
over other types of electrolysis [7].

  * •

Experiment: In order to find an optimum efficiency of the PV–electrolysis,
different combinations of the electrolyzer with A-CIGS-based thin film solar
cell modules with different band gaps of the cell were examined.

  * •

Preparation: Pre-sputtering was performed for 5 min in argon plasma in order
to remove surface impurities.

  * •

Characterization: The current density-potential (j–V) characteristics of the
A-CIGS cells were recorded under simulated AM 1.5G sunlight in a set-up with a
halogen lamp (ELH).

  * •

Explanation: A possible explanation for the superior ECSA-specific activity in
the 3D WP-structured catalysts is efficient removal of oxygen bubbles from the
catalyst layer.

  * •

Results: The load curves were similar for the electrolyzers with different WO3
thin films and the lowest potential needed for 10 mA cm-2 in the overall
reaction was 1.77 V.

  * •

Conclusion: The Cu-N- rGO demonstrated superior catalytic activity to the
counterpart N-rGO, and enhanced durability compared to commercial Pt/C.

Structural tags are used, for example, in the following cases.

  * •

Heading: 4\. Discussion and concluding remarks

  * •

Metadata: This research was funded by Hubei Superior and Distinctive
Discipline Group of “Mechatronics and Automobiles” (No.XKQ2019009).

  * •

Caption: Figure 8. Enlarged view of the shaded portion of Figure 7.

###  F.2 Multi-Label Examples

In contrast to earlier works on AZ, our approach to labeling AZ in materials
science publications uses a multi-label approach. In this section, we discuss
some multi-label examples.

  * •

Background, PriorWork, Results: This indicates that the HER follows a rate-
determining Volmer or Heyrovsky step for different sputtering conditions
without any order [40,41]. In this example, a result obtained in the current
paper confirms a result known from prior work.

  * •

Experiment, Characterization, Results, Explanation: Attributing this enthalpy
release exclusively to the removal of grain boundaries in stage B, a specific
grain boundary energy(2)$\gamma$=H$\rho$3dini-1-dfin-1=0.85$\pm$0.08Jm-2is
estimated using the initial and final crystallite diameters of stage B, as
given above (dini=222nm, dfin=764nm), as well as the Cu bulk value of
8.92gcm-3 for the mass density $\rho$. The first subordinate clause of this
sentence (Attributing … stage B) is an Explanation. The remainder of the
sentence states a Characterization.

  * •

Background, PriorWork, Results, Conclusion: Furthermore, the fatigue life
decreased approximately by more than 12% when the pre-corroded time was
doubled, and the fatigue life decreased approximately by more than 11% when
the applied stress level was doubled, indicating that both pre-corroded time
and applied stress level can significantly affect the fatigue life of
specimens, which shows a good agreement with the previous works [37,38]. This
example illustrates a case where our simplification of labeling entire
sentences comes to its limits: The first part of the sentence (Furthermore …
was doubled) reports Results while the second part draws a Conclusion drawing
connections to specific PriorWork.

##  Appendix G Detailed Results

In this section, we provide detailed results for the experiments presented in
the main part of the paper.

Table 13 (no oversampling and ML-ROS) and Table 14 (multi-task AZ-CL) show the
results in terms of precision, recall and (hierarchical) F1 for each label
individually. We report the results on both dev and test of the specific model
that performed best on dev compared to all other models.

First, we compare the difference between no oversampling at all and when using
ML-ROS. As shown in Table 1, Motivation, Metadata, and Caption are the least
frequent labels in our dataset. Except for Metadata on the test set, there is
always an increase in terms of F1-score when applying ML-ROS on minority
labels during training. The biggest increase of $5.8$ happened for Motivation
on the test set. Furthermore, there is also an improvement of 1.2 points on
dev and 2.5 points on test in terms of F1-score for Explanation, which is
fourth in the list of rarest AZ labels.

During our experimentation, we observed that ML-ROS tends to be especially
helpful for models that show strong performance on majority labels, but not on
minority labels. Other models with different hyperparameters achieve even
better scores on minority labels without oversampling; however, they tend to
have worse overall performance.

Next, we describe the effects of multi-task training with the AZ-CL dataset.
We also apply ML-ROS to MuLMS-AZ in our multi-task experiments. Both micro-F1
and macro-F1 increase by 0.5 points in terms of micro- and macro-F1 when using
multi-tasking instead of ML-ROS only. Most of the per-label F1-scores
increased when using multi-tasking, interestingly with notable differences for
Characterization ($4.8$) and Metadata ($5.6$). We conclude that multi-tasking
with AZ-CL helps supporting common majority labels even though the domain of
this dataset is clearly different from ours.

In contrast, multi-task learning with the other datasets consistently resulted
in decreases of performance. The chemistry domain is intuitively closest to
that of materials science, hence, we would have expected ART to be a good
additional dataset in multi-task learning. Brack et al. (2022) provide some
insights into cross-domain learning of AZ categories using datasets from
biomedicine, chemistry, and computer graphics. Our MuLMS-AZ, alongside AZ-CL,
opens up new research opportunities.

Method | LM | mic.-F1 | mac.-F1
---|---|---|---
No Oversampling | BERT | 72.6±1.0 | 65.5±0.7
| MatSciBERT | 76.3±0.7 | 70.1±0.7
| SciBERT | 76.2±0.9 | 70.2±0.6
ML-ROS | SciBERT | 76.7±0.7 | 70.6±0.9
\+ MT (+PM) | SciBERT | 76.5±0.4 | 69.5±0.5
\+ MT (+ART) | SciBERT | 75.0±0.9 | 68.9±1.1
\+ MT (+AZ-CL) | SciBERT | 77.2±0.3 | 71.1±0.5
\+ MT (+DRI) | SciBERT | 76.6±0.3 | 70.5±0.4
\+ MT (+ART+AZ+DRI) | SciBERT | 76.4±0.6 | 70.2±0.5
Data Augm. (+PM) | SciBERT | 77.1±0.8 | 70.8±1.3
human agreement* |  | 78.7 | 74.9
Table 12: Results on MuLMS-AZ test set, hierarchical micro/macro F1: MT=Multi-
Task models, *not directly comparable.

In addition, we perform a data augmentation experiment using AZ data from
scientific abstracts of the PubMed Medline
corpus111111https://www.nlm.nih.gov/databases/download/pubmed˙medline.html,
filtering for abstracts that were published in journals related to the
materials science domain (see Appendix C). We map the four PubMed AZ labels
Background, Objective, Results, and Conclusions to our four AZ labels
Background, Motivation, Results and Conclusion. Augmenting with data from the
PubMed Medline dataset also helps to achieve better performance. However, the
micro-F1 score is 0.1 lower and the macro-F1 score is 0.3 lower compared to
the MT (+AZ-CL) model. On the other hand, training is much more time-efficient
since a low augmentation percentage of $10\%$ is sufficient to get good
results.

Label | dev |  | test |
---|---|---|---|---
P | R | H. F1 |  | P | R | H. F1 | Count
SciBERT, no oversampling |  |  |  |  |  |  |  |
Motivation | 65.5 | 46.8 | 54.4 |  | 68.5 | 36.5 | 47.6 | 363
Background | 89.2 | 80.0 | 84.3 |  | 85.0 | 76.6 | 80.6 | 3155
-PriorWork | 97.0 | 84.5 | 90.3 |  | 92.9 | 67.9 | 78.4 | 1824
Experiment | 82.1 | 85.8 | 83.9 |  | 80.6 | 82.6 | 81.6 | 2579
-Characterization | 72.0 | 68.9 | 70.3 |  | 75.8 | 67.3 | 71.1 | 962
-Preparation | 65.2 | 65.1 | 65.0 |  | 78.6 | 69.7 | 73.7 | 1347
Explanation | 46.3 | 33.0 | 38.4 |  | 55.0 | 35.9 | 43.4 | 603
Results | 75.0 | 84.6 | 79.5 |  | 79.9 | 85.9 | 82.8 | 2953
Conclusion | 56.7 | 55.3 | 56.0 |  | 42.4 | 43.0 | 42.6 | 680
Caption | 92.4 | 75.2 | 82.9 |  | 80.9 | 68.9 | 74.4 | 485
Heading | 84.8 | 97.9 | 90.9 |  | 87.4 | 96.6 | 91.7 | 702
Metadata | 93.1 | 68.0 | 78.6 |  | 78.6 | 72.9 | 75.2 | 210
Average | 76.6 | 70.4 | 72.9 |  | 75.5 | 67.0 | 70.2 |
SciBERT, ML-ROS |  |  |  |  |  |  |  |
Motivation | 56.3 | 55.9 | 55.9 |  | 72.9 | 43.0 | 53.4 | 363
Background | 82.2 | 84.8 | 83.5 |  | 79.7 | 84.2 | 81.9 | 3155
-PriorWork | 96.0 | 84.5 | 89.9 |  | 90.5 | 71.3 | 79.7 | 1824
Experiment | 85.1 | 83.2 | 84.1 |  | 81.1 | 81.7 | 81.4 | 2579
-Characterization | 73.3 | 67.3 | 70.1 |  | 73.2 | 67.5 | 70.2 | 962
-Preparation | 69.4 | 63.4 | 66.3 |  | 73.8 | 69.5 | 71.5 | 1347
Explanation | 45.7 | 35.2 | 39.6 |  | 53.4 | 40.2 | 45.9 | 603
Results | 77.6 | 83.4 | 80.4 |  | 83.6 | 83.8 | 83.7 | 2953
Conclusion | 60.6 | 44.5 | 51.3 |  | 46.8 | 35.2 | 40.1 | 680
Caption | 91.7 | 79.6 | 85.2 |  | 77.9 | 73.6 | 75.7 | 485
Heading | 85.4 | 97.5 | 91.1 |  | 90.6 | 96.3 | 93.4 | 702
Metadata | 89.3 | 70.5 | 78.8 |  | 61.9 | 80.0 | 69.8 | 210
Average | 76.1 | 70.8 | 73.0 |  | 73.8 | 68.9 | 70.6 |
Table 13: Per label scores on dev and test of MuLMS-AZ in terms of precision, recall, and hierarchical F1. Bold: best result for label. P, R, and F1 scores are averages over the P, R, F1 scores of 5 folds each. Label | dev |  | test
---|---|---|---
P | R | H. F1 |  | P | R | H. F1
Motivation | 62.7 | 54.1 | 58.0 |  | 71.2 | 43.9 | 54.3
Background | 85.6 | 82.1 | 83.8 |  | 80.9 | 81.6 | 81.2
-PriorWork | 95.4 | 84.2 | 89.4 |  | 93.7 | 68.8 | 79.3
Experiment | 83.6 | 82.8 | 83.2 |  | 83.1 | 83.0 | 83.0
-Characterization | 73.7 | 65.9 | 69.3 |  | 77.4 | 73.0 | 75.0
-Preparation | 69.4 | 55.6 | 61.7 |  | 79.4 | 67.2 | 72.8
Explanation | 42.6 | 35.8 | 38.8 |  | 51.2 | 35.9 | 41.7
Results | 76.6 | 84.4 | 80.3 |  | 81.5 | 85.1 | 83.2
Conclusion | 61.8 | 49.6 | 55.0 |  | 41.0 | 32.8 | 36.4
Caption | 90.5 | 77.6 | 83.5 |  | 79.2 | 76.2 | 77.7
Heading | 84.7 | 97.7 | 90.7 |  | 88.9 | 97.4 | 92.9
Metadata | 84.3 | 72.0 | 77.6 |  | 70.6 | 81.4 | 75.4
Table 14: Per label scores on dev and test in terms of precision, recall, and
hierarchical F1 using multi-task learning with the AZ-CL dataset, SciBERT, ML-
ROS.