# Iteratively Improving Biomedical Entity Linking and Event Extraction via
Hard Expectation-Maximization

Xiaochu Li∗, Minqian Liu∗, Zhiyang Xu∗, Lifu Huang
Computer Science Department
Virginia Tech
{xiaocli, minqianliu, zhiyangx<EMAIL_ADDRESS>
###### Abstract

Biomedical entity linking and event extraction are two crucial tasks to
support text understanding and retrieval in the biomedical domain. These two
tasks intrinsically benefit each other: entity linking disambiguates the
biomedical concepts by referring to external knowledge bases and the domain
knowledge further provides additional clues to understand and extract the
biological processes, while event extraction identifies a key trigger and
entities involved to describe each biological process which also captures the
structural context to better disambiguate the biomedical entities. However,
previous research typically solves these two tasks separately or in a
pipeline, leading to error propagation. What’s more, it’s even more
challenging to solve these two tasks together as there is no existing dataset
that contains annotations for both tasks. To solve these challenges, we
propose joint biomedical entity linking and event extraction by regarding the
event structures and entity references in knowledge bases as latent variables
and updating the two task-specific models in a hard Expectation–Maximization
(EM) fashion: (1) predicting the missing variables for each partially
annotated dataset based on the current two task-specific models, and (2)
updating the parameters of each model on the corresponding pseudo completed
dataset. Experimental results on two benchmark datasets: Genia 2011 for event
extraction and BC4GO for entity linking, show that our joint framework
significantly improves the model for each individual task and outperforms the
strong baselines for both tasks. We will make the code and model checkpoints
publicly available once the paper is accepted.

11footnotetext: Equal contribution.

##  1 Introduction

Figure 1:  Illustration of biomedical entity linking (lower half) and event
extraction (upper half) tasks given the same input. Below the input text, we
show the definitions of each entity retrieved from Gene Ontology (GO) after
running our entity linking model. We show the event types (in rounded boxes),
entity types (without rounded boxes), and argument roles above the text. We
highlight the event Regulation and its mention in blue, and the event Binding
and its mention in orange. We also highlight the keywords in GO that are
closely related to event extraction in corresponding colors.

As the volume of biomedical literature continues to grow, biomedical entity
linking and event extraction tasks have received increasingly more attention
as they are essential to aid domain experts in retrieving and organizing
critical information related to gene functions, bio-molecule relations, and
bio-molecule behaviors from the vast amount of unstructured texts Kim et al.
(2009); Leitner et al. (2010); Segura-Bedmar et al. (2013). Biomedical entity
linking (a.k.a. named-entity disambiguation) Zhu et al. (2020); Angell et al.
(2021); Bhowmik et al. (2021) aims to assign an entity mention in the text
with a biomedical concept or term in reference biomedical knowledge bases,
such as Gene Ontology (GO) Consortium (2012a); Van Auken et al. (2014),
Unified Medical Language System (UMLS) Bodenreider (2004), Universal Protein
Resource (UniProt) Bairoch et al. (2005), and the EMBL nucleotide sequence
database Kanz et al. (2005). Meanwhile, biomedical event extraction is the
task of identifying event triggers that most clearly convey the occurrence of
events (i.e., biological processes) and their arguments that participated in
those events. Figure 1 shows an example of biomedical entity linking and event
extraction.

Despite the recent progress achieved in biomedical entity linking and event
extraction, there are still several problems that remained unaddressed. In
biomedical entity linking, the entity mentions can be highly ambiguous as one
mention can be mapped to multiple distinct biomedical concepts, requiring the
model to have a good understanding of the context of the mention. For example,
CSC-1 in Figure 1 can refer to a centromeric protein or a DNA Vader et al.
(2006). Meanwhile, biomedical events usually have complex and nested
structures, and sufficient domain knowledge is required to capture biological
processes and their participants. While each task has its own challenges, we
find that these two tasks can be beneficial to each other: entity linking maps
the mentions in the text to biomedical concepts in external knowledge bases
and provides additional domain knowledge and semantic information (e.g., the
definitions in Gene Ontology) for extracting biological processes, while event
extraction identifies the key trigger and its associated arguments that can
provide more structural context to narrow down the pool of candidates and
better link the entities to the biomedical concepts in knowledge bases. As
shown in Figure 1, the GO definition of the protein CSC-1 clearly indicates
the function of this protein is related to the biological process binding,
which can help the event extraction model to infer the relationship between
CSC-1 and the binding event. On the other hand, given that CSC-1 is a Theme of
binding, the entity linking can leverage such structural and precise context
to better disambiguate the biological concept CSC-1.

While biomedical entity linking and event extraction intrinsically benefit
each other, most existing works in biomedical information extraction ignore
the close relationship between the two tasks and tackle them separately or in
a pipeline, leading to the error propagation issue. Besides, there is no
existing dataset that contains annotations for both tasks. For example, the
BC4GO dataset Van Auken et al. (2014) only contains the annotations for entity
linking, whereas the Genia 11 dataset Kim et al. (2011) only has the
annotations for event extraction. This makes it even more difficult to solve
these two tasks together.

To address these challenges, we propose a joint biomedical entity linking and
event extraction framework, where each task-specific model incorporates the
additional knowledge, i.e., the output from another model, to better perform
task-specific prediction. To iteratively improve the models specific to each
task, we model the entity references in knowledge bases and event structures
as latent variables and devise a hard-EM-style learning strategy that consists
of two steps: (1) E-step: estimating the missing variables for each partially
annotated dataset (e.g., event triggers and their argument roles in the entity
linking dataset) using the current two task-specific models, and; (2) M-step:
updating the parameters of each model on the corresponding dataset that has
been augmented by the pseudo labels in the complementary task.

We extensively evaluate our approach on a biomedical entity linking dataset
(i.e., BC4GO), and an event extraction dataset (i.e., Genia 11). The
experimental results and case study validate the effectiveness of our
approach. Our main contributions in this work are summarized as follows:

  * •

We propose a joint biomedical entity linking and event extraction framework,
namely Joint4E-EL and Joint4E-EE, where the two models can mutually improve
each other.

  * •

We design a collaborative training strategy to iteratively optimize two task-
specific models such that each model can learn to leverage the information
introduced by the other.

  * •

Our joint framework consistently achieves significant performance gain for
each individual task on four public benchmarks across different domains.

##  2 Related Work

Biomedical Entity Linking. Most recent state-of-the-art methods for biomedical
entity linking are based on pre-trained BERT and consist of two steps: (1)
candidate retrieval, which retrieves a small set of candidate references from
a particular knowledge base; and (2) mention disambiguation and candidate
ranking, which resolves the ambiguity of the mention based on the local
context and refines the likelihood of each candidate reference with the fine-
grained matching between mention and candidate Wu et al. (2020); Bhowmik et
al. (2021); Varma et al. (2021). These methods are not efficient enough as it
requires two pipelined models (a retrieval and a ranking model) and have shown
to not be able to generalize well on rare entities Wu et al. (2020); Bhowmik
et al. (2021). Some recent studies have demonstrated that incorporating
external information from biomedical knowledge bases, such as the latent type
or semantic type information about mentions Zhu et al. (2020); Xu et al.
(2020), or infusing the domain-specific knowledge into the encoders with
knowledge-aware pre-training tasks and objectives He et al. (2020) can help
improve the model performance on biomedical entity linking task Varma et al.
(2021). While these studies mainly leverage the knowledge from external
knowledge bases to improve biomedical entity linking, related tasks such as
biomedical event extraction can also provide meaningful clues to disambiguate
the meaning of the mentions in the local context, however, it has not been
previously studied, especially in the biomedical domain.

Biomedical Event Extraction Current approaches for biomedical event extraction
mainly focus on extracting triggers and arguments in a pipeline Han et al.
(2019); Espinosa et al. (2019); Trieu et al. (2020); Ramponi et al. (2020);
Wang et al. (2020). Some studies also explore state-of-the-art neural methods
with multiple classification layers to identify triggers, event types,
arguments, and argument roles, respectively Han et al. (2019); Li et al.
(2019); Huang et al. (2020); Trieu et al. (2020). Recently, Ramponi et al.
(2020) propose a sequence labeling framework by converting the extraction of
event structures into a sequence labeling task by taking advantage of a multi-
label aware encoding strategy. In addition, to improve the generalizability of
event extraction, Wang et al. (2020) establish a multi-turn question answering
framework for event extraction by iteratively predicting answers for the
template-based questions designed for event triggers, event arguments, and
nested events. Several recent studies have also proposed to leverage external
knowledge bases to disambiguate the biomedical terms in the local context and
incorporate the knowledge, such as the definition or properties of the terms,
into the event extraction process. Despite the success of these methods, they
still suffer from error propagation in the pipeline frameworks, e.g., linking
errors of biomedical terms in the local context will inform incorrect clues to
the event extraction model and lead to a negative effect on the event
predictions. Compared with all these studies, our Joint4E-EE framework
iteratively improves both biomedical entity linking and event extraction by
leveraging the outputs from each other as additional input features.

##  3 Problem Formulation

Biomedical Entity Linking. Given a text
$\mathbf{x}^{L}=[x_{1}^{L},x_{2}^{L},...,x_{n}^{L}]$ and a set of spans for
all the entity mentions $\mathcal{M}=\\{m_{1},m_{2},...,m_{p}\\}$ in
$\mathbf{x}^{L}$, where $n$ indicates the number of tokens and $p$ indicates
the number of mentions, biomedical entity linking maps each entity mention
$m_{i}$ to a particular entity concept $\hat{c}_{i}$ from a biomedical
knowledge base. Taking the sentence in Figure 1 as an example, for each entity
mention, such as CSC-1, a biomedical entity linking model will link it to a
reference entity such as GO:0043237 in the external knowledge base of Gene
Ontology. Each entity in the knowledge base is represented with a unique GO ID
and definition which is annotated by experts and Gene Ontology annotation
toolsConsortium (2012b); Balakrishnan et al. (2013).

Biomedical Event Extraction. Biomedical event extraction consists of two
subtasks: event detection and argument extraction. Given the input text
$\mathbf{x}^{E}=[x_{1}^{E},x_{2}^{E},...,x_{n}^{E}]$, the goal of event
detection is to assign each token $x_{i}^{E}$ in $\mathbf{x}^{E}$ with an
event type $\tau_{i}$ that indicates a biological process in a predefined set
of event types $\mathcal{T}$ or label it as Other if the token is not an event
trigger. For each identified event trigger, argument extraction needs to
assign each entity mention $m_{i}$ in $\mathcal{M}$ with an argument role
$\alpha_{j}$ or Other that indicates how the entity participates in the
biological process $\tau_{i}$, where $\alpha_{j}$ belongs to a predefined set
of argument role types $\mathcal{A}$. A mention is labeled as Other if it does
not participate in the particular biological processes triggered by
$\tau_{i}$. As shown in Figure 1, given the sentence as input, biomedical
event extraction aims to detect all the candidate triggers and their types,
such as associates as a Regulation event mention and binds as a Binding event
mention, and extract the arguments with arguments roles for each trigger,
e.g., ICP-1 is the Theme of the associates event while BIR-1 is the Site of
the binds event. Note that, each event mention can also be an argument in
another event, for example, associates event is the Cause of the binds event.
Thus, given a particular event trigger, we also predict an argument role
$\alpha_{j}$ or Other for each of the other triggers.

##  4 Approach

Figure 2:  Illustration for our Joint4E-EE for biomedical event extraction.
Joint4E-EE leverages the encoded GO definitions for each entity from the
entity linking model Joint4E-EL such that it has more domain knowledge to
extract biological processes such as Gene Expression and its participant Id1.

In this section, we present our joint event extraction and entity linking
framework that consists of (1) an entity-aware event extraction module, named
Joint4E-EE, that leverages the additional knowledge from knowledge bases, such
as GO, UMLS, UniProt, and the EMBL nucleotide sequence database Consortium
(2012a); Van Auken et al. (2014); Bodenreider (2004); Bairoch et al. (2005);
Kanz et al. (2005), to disambiguate the meaning of the biological terms in the
input sentence so as to benefit the learning of the context and event
extraction structures; and (2) an event-aware entity linking module, named
Joint4E-EL, which utilizes event structures to characterize the biological
processes that each entity mention is involved and disambiguate its
meaningful, so that we can better link each entity mention to the correct
reference entity in the knowledge base. Since both Joint4E-EL and Joint4E-EE
requires the output from the other task as additional input while there is no
existing benchmark dataset containing annotations for both tasks, we further
design a join training framework in an expectation-maximization (EM) fashion
to iteratively estimate the missing variables (i.e., event structures or
entity references from external knowledge base) and optimize both Joint4E-EL
and Joint4E-EE simultaneously. In the following, we first introduce the
details of Joint4E-EL and Joint4E-EE in Section 4.2 and 4.1, and then
elaborate on how we iteratively improve both task-specific models via an
iterative learning schema in Section 4.3.

###  4.1 Entity-aware Biomedical Event Extraction (Joint4E-EE)

Base Event Extraction Model. The base event extraction model takes a text
$\mathbf{x}^{E}=[x_{1}^{E},x_{2}^{E},...,x_{n}^{E}]$ and the set of all entity
mentions $\mathcal{M}$ in $\mathbf{x}^{E}$ as inputs. We first encode
$\mathbf{x}^{E}$ with a PLM encoder Devlin et al. (2019); Beltagy et al.
(2019) to obtain the contextualized representations
$\mathbf{H}_{w}=[\mathbf{h}_{1},\mathbf{h}_{2},...,\mathbf{h}_{n}]$ for the
text, where each token’s representation $\mathbf{h}_{j}$ is the average of the
representations of their corresponding subtokens. For each token $j$, we feed
its representation $\mathbf{h}_{j}$ into an event-type classification layer to
classify the token into a positive event type or Other if it is not an event
trigger. Note that all event triggers are single-token.

For argument extraction, we concatenate the contextualized representation of
each identified event trigger $\mathbf{h}_{\tau_{j}}$ with the representation
of each argument candidate (i.e., entity mention) $\mathbf{h}_{m_{i}}$ in
$\mathcal{M}$ and feed the concatenated representations into an argument role
classification layer to compute the probabilities for argument role types.
Both event detection and argument role classification are optimized with
multi-class cross entropy.

Joint4E-EE. For event extraction, we propose Joint4E-EE, a dual encoder
framework that incorporates the external domain knowledge of the given
entities by the base entity linking model such that it can better extract
biological processes from unstructured texts. Given the input text
$\mathbf{x}^{E}$ and an entity mention $m_{i}$ from the set of all entity
mentions $\mathcal{M}$, first we leverage the search engine of the QuickGO
API111https://www.ebi.ac.uk/QuickGO/ for GO knowledge base to retrieve a set
of candidate biomedical concepts $\mathcal{C}_{i}$ from the GO knowledge base.
We type in the tokens of the entity mention to the search engine and the
QuickGO API returns the set of all possible candidates. If there are more than
30 candidates returned, we only take the first 30 candidates returned by the
QuickGO API. For the rest of the section, we use the term retrieve candidate
concepts to refer to the same process mentioned above. Table 2 shows the
fraction of mentions that can be found with at least one positive candidate.
When we take more than 30 candidates, the fraction doesn’t increase. Second,
we apply the base entity linking model to select a biomedical concept from the
candidate set of concepts $\mathcal{C}_{i}$ retrieved from the GO knowledge
base. Third, we obtain the definition of the corresponding biomedical concept
from GO and use it as part of the input for Joint4E-EE. In particular, we
apply an additional PLM-based knowledge encoder that specifically takes in the
selected biomedical definition $\mathbf{d}_{i}=[d_{i,1},d_{i,2},...d_{i,q}]$
for $m_{i}$ and encodes it into contextualized representations. We take the
contextualized representation of the [CLS] token as the knowledge encoding for
$m_{i}$, denoted as $\mathbf{v}_{i}$. We adopt the same process for all the
entities in $\mathcal{M}$, which yields a set of knowledge encodings
$\mathcal{V}=\\{\mathbf{v}_{1},\mathbf{v}_{2},...,\mathbf{v}_{p}\\}$.
Meanwhile, similarly to the base event extraction model, we also encode the
input text $\mathbf{x}^{E}$ into contextualized representations
$\mathbf{H}_{w}=[\mathbf{h}_{1},\mathbf{h}_{2},...,\mathbf{h}_{n}]$ with a
text encoder. Forth, we integrate the external knowledge by applying element-
wise addition between the contextualized representation of each mention
$m_{i}$ and its corresponding knowledge encoding $\mathbf{v}_{i}$ such that we
obtain a knowledge-enhanced entity representation via
$\tilde{\mathbf{h}}_{m_{i}}=\mathbf{h}_{m_{i}}+\mathbf{v}_{i}$. Finally, we
concatenate the representation of each identified event trigger
$\mathbf{h}_{\tau_{j}}$ with the enhanced entity representation
$\tilde{\mathbf{h}}_{m_{i}}$ and feed it into the classification layer to
perform argument extraction.

###  4.2 Event-aware Biomedical Entity Linking (Joint4E-EL)

Base Entity Linking Model. The base entity linking model (Base-EL) takes in
$\mathbf{x}^{L}=[x_{1}^{L},x_{2}^{L},...,x_{n}^{L}]$ and the set of spans for
all entity mentions $\mathcal{M}=\\{m_{1},m_{2},...,m_{p}\\}$ in
$\mathbf{x}^{L}$, and maps each entity mention $m_{i}\in\mathcal{M}$ to a
concept in the external knowledge base, i.e., Gene Ontology (GO). We retrieve
a set of candidate concepts $\mathcal{C}_{i}$ from GO for entity mention
$m_{i}$. For each candidate $c_{k}$ from the candidate set $\mathcal{C}_{i}$,
we obtain its definition in GO which is also a text sequence, denoted as
$\mathbf{d}_{k}=[d_{k,1},d_{k,2},...d_{k,q}]$. We append the definition
$\mathbf{d}_{k}$ at the end of $\mathbf{x}^{L}$ separated by a special token
[SEP], which yields the whole input sequence for the model:

$\text{[CLS]}[x_{1}^{L},x_{2}^{L},...,x_{n}^{L}]\text{[SEP]}[d_{k,1},d_{k,2},...d_{k,q}].$
(1)

We encode the entire sequence with a pretrained language model (PLM) encoder
Devlin et al. (2019); Beltagy et al. (2019) and then take the contextualized
representation of the [CLS] token output from the encoder to compute the
probability

$\mathbb{P}(c_{k}|m_{i},\mathbf{x}^{L},\mathbf{d}_{k};\theta_{L})$

with a binary classification layer, where $\theta_{L}$ denotes the parameters
of the entity linking model. The model is optimized by the binary cross
entropy loss. Joint4E-EL We introduce Joint4E-EL, a framework that utilizes
the output of a base event extraction model (see Section 4.1 for details) for
biomedical entity linking. Joint4E-EL consists of a PLM encoder Devlin et al.
(2019); Beltagy et al. (2019) that computes the contextualized representations
for the input sequence and a binary classification layer that computes the
probability of the mapping between a given entity mention $m_{i}$ in the input
text and a biomedical concept from a candidate set $\mathcal{C}_{i}$ in the
knowledge base. Based on the base entity linking model, we incorporate the
event information into the entity linking model to provide more structural
context for better entity disambiguation.

Specifically, given the input text $\mathbf{x}^{L}$, first, we apply the base
event extraction model to obtain the pseudo trigger and argument role labels.
Second, we take one entity mention $m_{i}$ and retrieve candidate concepts
$\mathcal{C}_{i}$ for $m_{i}$. Third, we inject event information into the
input sequence $\mathbf{x}^{L}$ of the entity linking model. Each entity only
participates in a single event $\tau_{i}$ with a unique argument role
$\alpha_{i}$ (if any). We insert the name of the argument role $\alpha_{i}$
after the tokens of $m_{i}$ in the original text $\mathbf{x}^{L}$ and append
the name of the event type $\tau_{i}$ at the end of the sentence
$\mathbf{x}^{L}$. Note that we set the name of the argument role as "Other" if
the entity does not participate in any biological process. Similarly to the
base entity linking model, we also append the definition $\mathbf{d}_{k}$
w.r.t. the candidate concept $c_{k}$ after the original input. The event-
enhanced input sequence for our Joint4E-EL model is structured as:

$\text{[CLS]}[x_{1}^{L},x_{2}^{L},...,m_{i},\alpha_{i},...,x_{n}^{L},\tau_{i}]\text{[SEP]}[d_{k,1},d_{k,2},...d_{k,q}]$.

(2)

Forth, We encode the input sequence with the PLM encoder and feed the
contextualized representation of the [CLS] token into the binary
classification layer.

Figure 3:  Illustration of our Joint4E-EL for biomedical entity linking. Given
an entity mention (e.g., BIR-1), Joint4E-EL combines the original text,
mention definition from Gene Ontology, and the predicted event structure from
Joint4E-EE as the event-enhanced input and outputs a probability to indicate
its confidence on the candidate concept, e.g., $c_{1}$. We select the
candidate with the highest probability as the predicted concept.

1

Input: Entity linking dataset $\mathcal{D}_{L}$, event extraction dataset
$\mathcal{D}_{E}$, external knowledge base $\mathcal{B}$, learning rates
$\eta_{L}$ and $\eta_{E}$.

2

3for  _each entity set $\mathcal{M}$ in $\mathcal{D}_{L}$_ do

4 for  _$m_{i}\in\mathcal{M}$_ do

5 Retrieve the candidate set $\mathcal{C}_{i}$ for $m_{i}$ from $\mathcal{B}$;

6

7

8Initialize the entity linking model’s parameters $\theta_{L}$ and the event
extraction model’s parameters $\theta_{E}$;

9 Train $\theta_{L}$ on $\mathcal{D}_{L}$ and $\theta_{E}$ on
$\mathcal{D}_{E}$;

10

11while  _not converged_ do

// Initialize augmented datasets

12 $\mathcal{U}_{L}=\\{\\}$, $\mathcal{U}_{E}=\\{\\}$ ;

// E-step

13 for  _each $(\mathbf{x}^{L}_{i},\mathbf{y}^{L}_{i})\in\mathcal{D}_{L}$_ do

14
$\tilde{\mathbf{z}}^{E}_{i}=\text{argmax}_{\mathbf{z}_{j}^{E}\in\mathcal{Z}^{E}}\mathbb{P}(\mathbf{z}_{j}^{E}|\mathbf{x}^{L}_{i};\theta_{E})$;

15
$\mathcal{U}_{L}\leftarrow\mathcal{U}_{L}\bigcup\\{(\mathbf{x}^{L}_{i},\mathbf{y}^{L}_{i},\tilde{\mathbf{z}}^{E}_{i})\\}$;

16

17 for  _each $(\mathbf{x}^{E}_{i},\mathbf{y}^{E}_{i})\in\mathcal{D}_{E}$_ do

18
$\tilde{\mathbf{z}}^{L}_{i}=\text{argmax}_{\mathbf{z}_{j}^{L}\in\mathcal{Z}^{L}}\mathbb{P}(z_{j}^{L}|x^{E}_{i};\theta_{L})$;

19
$\mathcal{U}_{E}\leftarrow\mathcal{U}_{E}\bigcup\\{(\mathbf{x}^{E}_{i},\mathbf{y}^{E}_{i},\tilde{\mathbf{z}}^{L}_{i})\\}$;

20

// M-step

21 for  _each epoch_ do

22 Sample
$(\mathbf{x}^{L}_{i},\mathbf{y}^{L}_{i},\tilde{\mathbf{z}}^{E}_{i})\sim\mathcal{U}_{L}$;

23
$\theta_{L}\leftarrow\theta_{L}-\eta_{L}\nabla_{\theta_{L}}J_{L}(\theta_{L}|\mathbf{x}^{L}_{i},\tilde{\mathbf{z}}^{E}_{i})$;

24

25 for  _each epoch_ do

26 Sample
$(\mathbf{x}^{E}_{i},\mathbf{y}^{E}_{i},\tilde{\mathbf{z}}^{L}_{i})\sim\mathcal{U}_{E}$;

27
$\theta_{E}\leftarrow\theta_{E}-\eta_{E}\nabla_{\theta_{E}}J_{E}(\theta_{E}|\mathbf{x}^{E}_{i},\tilde{\mathbf{z}}^{L}_{i})$;

28

29

Algorithm 1 Iterative Training for Joint4E

###  4.3 Iterative Training via Hard-EM

In this section, we formulate our Hard EM style iterative training algorithm
which is shown in Algorithm 1. For the event extraction task, we denote a
training instance as $(\mathbf{x}^{E}_{i},\mathbf{y}^{E}_{i})$, where
$\mathbf{x}^{E}_{i}$ is a sentence and $\mathbf{y}^{E}_{i}$ is the event
annotation on $\mathbf{x}^{E}_{i}$, in the event extraction dataset
$\mathcal{D}_{E}$. We denote $\mathcal{Z}_{all}^{L}$ as the finite set of all
possible entity linking labels on the text $\mathbf{x}^{E}_{i}$. We further
define
$\mathcal{Z}^{L}=\\{\mathbf{z}^{L}\in\mathcal{Z}_{all}^{L}:f_{\theta_{E}}(\mathbf{x}^{E}_{i},\mathbf{z}^{L})=\mathbf{y}^{E}_{i}\\}$
as the set of entity linking labels that leads to the correct event extraction
prediction on the sentence $\mathbf{x}^{E}_{i}$, where $f_{\theta_{E}}$ is the
event extraction model and $\theta_{E}$ denotes its parameters. In our
setting, the (pseudo) entity linking labels become discrete latent variables
for the event extraction task.

For the entity linking task, given an instance
$(\mathbf{x}^{L}_{i},\mathbf{y}^{L}_{i})$ where $\mathbf{x}^{L}_{i}$ is a
sentence and $\mathbf{y}^{L}_{i}$ is the entity linking annotation on
$\mathbf{x}^{L}_{i}$, in the entity linking dataset $\mathcal{D}_{L}$, we
denote $\mathcal{Z}_{all}^{E}$ as the finite set of all possible event
extraction labels on the text $\mathbf{x}^{L}_{i}$. We further define
$\mathcal{Z}^{E}=\\{\mathbf{z}^{E}\in\mathcal{Z}_{all}^{E}:f_{\theta_{L}}(\mathbf{x}^{L}_{i},\mathbf{z}^{E})=\mathbf{y}^{L}_{i}\\}$
as the set of event extraction labels that leads to the correct entity linking
predictions on the sentence $\mathbf{x}^{L}_{i}$, where $f_{\theta_{L}}$ is
the entity linking model and $\theta_{L}$ denotes its parameters. In the above
setting, the (pseudo) event extraction labels become discrete latent variables
for the entity linking task.

Given a dataset $\mathcal{D}_{L}$ with entity linking annotations and a
dataset $\mathcal{D}_{E}$ with event extraction annotations, we first perform
the following prerequisite steps: First, We prepare the candidate biomedical
concepts for both entity linking dataset $\mathcal{D}_{L}$ and event
extraction dataset $\mathcal{D}_{E}$. Second, we randomly initialize the
parameter $\theta_{L}$ for Joint4E-EE and the parameter $\theta_{E}$ for
Joint4E-EL. To first obtain a well-initialized base model for each task, we
individually train Joint4E-EL on the labeled entity linking dataset
$\mathcal{D}_{L}$ and train Joint4E-EE on the labeled entity linking dataset
$\mathcal{D}_{L}$ until the model converges on the development sets,
respectively. After we obtain a base model individually trained on each task,
we start our Hard-EM style iterative training process that repeatedly performs
the following two steps: (1) the E-step that aims to estimate the latent
variables (i.e., predict pseudo labels) for each partially annotated dataset,
and; (2) M-step where it updates the parameters of each model given the
original inputs and the estimated latent variables.

E-Step. At the beginning of each round of the iterative training, we first
initialize two empty sets $\mathcal{U}_{L}=\\{\\}$ and
$\mathcal{U}_{E}=\\{\\}$ for collecting pseudo labeled instances. We run the
entity linking model Joint4E-EL on the event extraction dataset
$\mathcal{D}_{E}$ to generate pseudo entity linking annotations. Specifically,
for each instance in the event extraction dataset
$(\mathbf{x}^{E}_{i},\mathbf{y}^{E}_{i})\in\mathcal{D}_{E}$, we run the
Joint4E-EL model and predict the pseudo entity linking labels
$\mathcal{Z}^{L}$. Following hard EM, for the event extraction task, we take
the latent variable $\tilde{\mathbf{z}}^{L}_{i}\in\mathcal{Z}^{L}$ that has
the highest likelihood, i.e.,
$\tilde{\mathbf{z}}^{L}_{i}=\text{argmax}_{\mathbf{z}_{j}^{L}\in\mathcal{Z}^{L}}\mathbb{P}(z_{j}^{L}|x^{E}_{i};\theta_{L})$.
The estimated latent variable $\tilde{\mathbf{z}}^{L}_{i}$ together with
$\mathbf{x}^{E}_{i}$ and $\mathbf{y}^{E}_{i}$ form a new instance and is added
into $\mathcal{U}_{E}$. We also run the event extraction model Joint4E-EE on
the entity linking dataset $\mathcal{D}_{E}$ to generate pseudo event
extraction annotations. Specifically, for each instance in the entity linking
dataset $(\mathbf{x}^{L}_{i},\mathbf{y}^{L}_{i})\in\mathcal{D}_{L}$, we run
the Joint4E-EE model and predict the pseudo event labels $\mathcal{Z}^{E}$.
Following hard EM, for the event extraction task, we take the latent variable
$\tilde{\mathbf{z}}^{E}_{i}\in\mathcal{Z}^{E}$ that has the highest
likelihood, i.e.,
$\tilde{\mathbf{z}}^{E}_{i}=\text{argmax}_{\mathbf{z}_{j}^{E}\in\mathcal{Z}^{E}}\mathbb{P}(z_{j}^{E}|x^{E}_{i};\theta_{E})$.
The estimated latent variable $\tilde{\mathbf{z}}^{E}_{i}$ together with
$\mathbf{x}^{L}_{i}$ and $\mathbf{y}^{L}_{i}$ form a new instance and is added
into $\mathcal{U}_{L}$.

M-Step. For the event extraction task, we loop through the examples
$(\mathbf{x}^{E}_{i},\mathbf{y}^{E}_{i},\tilde{\mathbf{z}}^{L}_{i})$ in the
newly collected $\mathcal{U}_{E}$ event extraction dataset enhanced with
pseudo entity linking annotations. The Joint4E-EE model $f_{\theta}^{E}$
optimizes the log-likelihood of the true event extraction label
$\mathbf{y}^{E}_{i}$ based on the discrete latent variable
$\tilde{\mathbf{z}}^{L}_{i}$, i.e., the entity linking pseudo label. The loss
is computed as
$J_{E}(\theta_{E}|\mathbf{x}^{E}_{i},\tilde{\mathbf{z}}^{L}_{i})=-\log{\mathbb{P}(\mathbf{y}^{E}_{i}|\mathbf{x}^{E}_{i},\tilde{\mathbf{z}}^{L}_{i};\theta_{E})}$.
For the entity linking task, we loop through the examples
$(\mathbf{x}^{L}_{i},\mathbf{y}^{L}_{i},\tilde{\mathbf{z}}^{E}_{i})$ in the
newly collected $\mathcal{U}_{L}$ entity linking dataset enhanced with pseudo
event extraction annotations. The Joint4E-EL model $f_{\theta}^{L}$ optimizes
the log-likelihood of the true entity linking label $\mathbf{y}^{L}_{i}$ based
on the discrete latent variable $\tilde{\mathbf{z}}^{E}_{i}$, i.e., the event
pseudo label. The loss is computed as
$J_{L}(\theta_{L}|\mathbf{x}^{L}_{i},\tilde{\mathbf{z}}^{E}_{i})=-\log{\mathbb{P}(\mathbf{y}^{L}_{i}|\mathbf{x}^{L}_{i},\tilde{\mathbf{z}}^{E}_{i};\theta_{L})}$.

##  5 Experimental setup

###  5.1 Datasets

#### Event Extraction

We evaluate the performance of our approach for biomedical event extraction on
the Genia 2011 dataset (GE11) Kim et al. (2011), which defines 9 event types
with 6 argument roles. The text is based on the abstracts and full articles
from PubMed about biological processes related to proteins and genes. The
detailed statistics of GE11 are summarized in Table 1. Following previous
studies Ramponi et al. (2020); Trieu et al. (2020); Wang et al. (2020); Huang
et al. (2020); Zhao et al. (2021); Wang et al. (2022), we evaluate the
performance of biomedical event extraction using the precision (P), recall
(R), and F1 score (F1).

GE11 | Training | Development | Test
---|---|---|---
# Documents | 908 | 259 | 347
# Sentences | 8,664 | 2,888 | 3,363
# Entities | 11,625 | 4,690 | 5,301
# Events | 10,310 | 3,250 | 4,487
Table 1: Statistics of the Genia 2011 dataset for biomedical event extraction.

#### Entity Linking

For the entity linking task, we leverage the BioCreative IV GO (BC4GO) dataset
Van Auken et al. (2014) which contains annotations of Gene Ontology entities
for all the entity mentions in the dataset. Each entity mention in BC4GO is
mapped to a unique biomedical entity in the Gene Ontology knowledge base where
each entity is described with GO id, name, and definition. However, the
original BC4GO dataset was built in 2013. With the development of Vivo and
Vitro in biomedical science in the last decades, new definitions and
ontologies of biomedical concepts have been introduced into the Gene Ontology
knowledge base, which drastically changes the topology of the knowledge base
Park et al. (2011); Yon Rhee et al. (2008) and makes the mappings between the
entity mentions and their concepts in the original BC4GO outdated. In
addition, previous studies Balakrishnan et al. (2013) also suggest that the
mappings between entity mentions and entities in the Gene Ontology knowledge
base are not surjective, i.e., each entity mention can be mapped into multiple
entities. Thus, we propose to update the mappings between entity mentions in
BC4GO and entities in Gene Ontology by leveraging the official API
222https://www.ebi.ac.uk/QuickGO/ of Gene Ontology. We include more details on
how we build the mappings and process the BC4GO entity linking dataset in
Appendix A.1. We retrieve 30 candidates for each mention via querying the Gene
Ontology API. After the preprocessing, the expanded BC4GO dataset contains
29,037 mention-candidate pairs in the training set (9,027 positive and 20,010
negative pairs), 7,023 pairs in the dev set (2,352 positive and 4,671 negative
pairs), and 5,580 pairs in the test set (1,578 positive and 4,002 negative
pairs). During the evaluation, we set each mention with one candidate as one
pair, and use accuracy to calculate the correct prediction pair number over
the total number of mention-candidate pairs.

GO number | 10 | 15 | 20 | 25 | 30 | 35
---|---|---|---|---|---|---
Fraction | 0.56 | 0.68 | 0.79 | 0.82 | 0.83 | 0.83

Table 2: The fractions of mentions that can be found with at least one
positive candidate.

###  5.2 Baselines

#### Event Extraction

We compare Joint4E-EE with several recent state-of-the-art methods on
biomedical event extraction, including: TEES Björne and Salakoski (2011),
EventMine Pyysalo et al. (2012), Stacked generalization Majumder et al.
(2016), TEES-CNN Björne and Salakoski (2018), KB-driven Tree-LSTM Li et al.
(2019), QA with BERT Wang et al. (2020), GEANet Huang et al. (2020), BEESL
Ramponi et al. (2020), DeepEventMine Trieu et al. (2020), HANN Zhao et al.
(2021), and CPJE Wang et al. (2022).

#### Entity Linking

We compare our Joint4E-EL with the following baselines: LATTE Zhu et al.
(2020), Bootleg Orr et al. (2020), Fast Dual Encoder Bhowmik et al. (2021),
and PromptEL Zhu et al. (2021). Note that some of the existing entity linking
approaches Yuan et al. (2022) require to be explicitly grounded on other
knowledge bases. They are not comparable with our approach and thus we did not
include them in our experiments.

###  5.3 Implementation Details

We first train the base event extraction and entity linking models on Genia
2011 and BC4GO datasets, respectively. For the base event extraction and
entity linking models, we use AdamW Loshchilov and Hutter (2019) optimizer
with a learning rate of 5e-5, and a linear learning rate warm-up over the
first 10% of training steps is applied. The model is trained for 30 epochs
with a batch size of 16. We stop the training of these two base models if they
do not show a better performance for 5 consecutive epochs. For joint training
(both Joint4E-EE and Joint4E-EL), we use learning rates 2e-5, 1e-5, 1e-5,
5e-6, 5e-6, 5e-6 for 6 rounds respectively with a batch size of 16. We stop
the training of these two Joint4E-EE and Joint4E-EL models if they do not show
a better performance for 5 consecutive epochs.

##  6 Results and Discussions

###  6.1 Main Results

#### Event extraction

Table 3 shows the results of our approach Joint4E-EE and the baselines on
Genia 2011 dataset. Joint4E-EE achieves significant improvement over all the
strong baselines. Particularly, it outperforms the base model by 3.12% F1
score and the previous state-of-the-art by 1.6% F1 score, respectively. These
results demonstrate the effectiveness of our joint learning framework and our
approach effectively improves the base model that is not enhanced by the
entity linking model.

Method | Precision | Recall | F1 Score
---|---|---|---
TEES Björne and Salakoski (2011) | 57.65 | 49.56 | 53.30
EventMine Pyysalo et al. (2012) | 63.48 | 53.35 | 57.98
Stacked generalization Majumder et al. (2016) | 66.46 | 48.96 | 56.38
TEES-CNN Björne and Salakoski (2018) | 69.45 | 49.94 | 58.10
KB-driven Tree-LSTM Li et al. (2019) | 67.01 | 52.14 | 58.65
QA with BERT Wang et al. (2020) | 59.33 | 57.37 | 58.33
GEANet Huang et al. (2020) | 64.61 | 56.11 | 60.06
BEESL Ramponi et al. (2020) | 69.72 | 53.00 | 60.22
DeepEventMine Trieu et al. (2020) | 70.52 | 56.52 | 62.75
HANN Zhao et al. (2021) | 71.73 | 53.21 | 61.10
CPJE Wang et al. (2022) | 72.62 | 53.33 | 61.50
Base-EE (Ours) | 68.20 | 55.73 | 61.23
Joint4E-EE (Ours) | 69.66 | 59.75 | 64.35

Table 3: Performance comparison of various event extraction approaches on the
development set of BioNLP Genia 2011. (%). Bold highlights the highest
performance among all the approaches.

#### Entity linking

Table 4 shows the performance of various approaches for biomedical entity
linking based on the test set of BC4GO. We observe Joint4E-EL significantly
outperforms the four strong baselines by more than 2.73%. While our base
entity linking model shares a similar architecture as Sun et al. (2021), by
incorporating the additional event features from the local context, the
accuracy of Joint4E-EL is improved by a large margin (3.73% in accuracy),
demonstrating the benefit of event-based features to entity linking.

Method | Accuracy (%)
---|---
LATTE Zhu et al. (2020) | 82.71
Bootleg Orr et al. (2020) | 78.51
Fast Dual Encoder Bhowmik et al. (2021) | 82.03
PromptEL Zhu et al. (2021) | 81.32
Base-EL (Ours) | 81.35
Joint4E-EL (Ours) | 85.08

Table 4: Performance comparison of various entity linking approaches on BC4GO
in terms of accuracy (%). The best performance is highlighted in bold.

| Event Extraction | Entity Linking
---|---|---
Rounds | F1 Score | Accuracy
Base | 61.23 | 81.35
1st | 62.48 | 83.38
2nd | 63.85 | 84.06
3rd | 64.35 | 85.08
4th | 64.28 | 85.08
5th | 64.15 | 85.08
6th | 64.15 | 85.08

Table 5: F1 score (%) of event extraction on the Genia 2011 development set
and the accuracy (%) of entity linking on the test set of BC4GO at each round
of joint training. The best performance is highlighted in bold.

Event type | Base | Round 1 | Round 2 | Round 3 | Round 4 | Round 5 | Round 6
---|---|---|---|---|---|---|---
Gene expression | 78.15 | 79.08 | 80.23 | 81.03 | 81.12 | 81.00 | 81.00
Transcription | 69.46 | 70.60 | 71.14 | 72.08 | 72.08 | 72.08 | 72.08
Protein catabolism | 74.57 | 75.06 | 75.88 | 76.38 | 76.38 | 76.38 | 76.38
Phosphorylation | 83.67 | 84.12 | 84.95 | 85.67 | 85.67 | 85.67 | 85.67
Localization | 80.30 | 80.75 | 81.07 | 81.30 | 81.30 | 81.30 | 81.30
Simple events | 76.73 | 77.92 | 78.65 | 78.79 | 78.99 | 78.77 | 78.77
Binding | 52.19 | 55.26 | 56.73 | 58.36 | 58.36 | 58.36 | 58.36
Regulation | 45.52 | 46.03 | 47.94 | 49.03 | 48.53 | 48.53 | 48.53
Positive regulation | 49.52 | 50.82 | 52.02 | 52.95 | 52.52 | 52.41 | 52.41
Negative regulation | 57.48 | 57.78 | 58.09 | 58.52 | 58.33 | 58.33 | 58.33
Complex events | 50.84 | 51.54 | 52.68 | 53.83 | 53.42 | 53.31 | 53.31

Table 6: Results of event extraction on the Genia 2011 development set for
each fine-grained event type and three categories (simple events, binding
events, and complex events) at each round in terms of F1 score (%). The best
performance is highlighted in bold.

###  6.2 Analysis of Generalizability

We further conducted an experiment to evaluate the generalizability of our
framework on two additional datasets that are in another domain (i.e., drug-
disease association) and are supported by another knowledge base, i.e., the
Unified Medical Language System (UMLS). Specifically, for biomedical entity
linking, we validate our model on NCBI Disease corpus Doğan et al. (2014) the
disease mentions and their Concepts of Unique Identifiers (CUI) in UMLS from a
collection of 793 PubMed abstracts. For biomedical event extraction, we adopt
the pharmacovigilance (PHAEDRA) dataset Thompson et al. (2018) that contains 4
types of structured drug-disease event information with 3 argument roles.

We want to verify that the drug-disease event information on PHAEDRA can
provide extra information to help the entity linking task on NCBI (see results
in Table 7). On the other hand, we also want to verify that the meaning of the
disease mentions (disorder entities in PHAEDRA) from the UMLS knowledge base
will help the drug-disease event extraction on PHAEDRA (see results in Table
8). From both Table 7 and Table 8, our approach achieves promising performance
improvement compared with the base models and previous baselines on both
entity linking and event extraction tasks. Particularly, on NCBI, we
outperform the previous state-of-the-art biomedical entity linking model by
0.32% accuracy, while we obtain 0.78% improvement on biomedical event
extraction. The experiment effectively demonstrates that our approach can be
adopted to various domains.

Method | F1 Score (%)
---|---
NormCo Wright (2019) | 87.80
SparkNLP Kocaman and Talby (2021) | 89.13
BioLinkBERT Yasunaga et al. (2022) | 88.76
ConNER Jeong and Kang (2022) | 89.20
CompactBioBERT Rohanian et al. (2023) | 88.76
Base-EL (Ours) | 85.58
Joint4E-EL (Ours) | 89.52

Table 7: Performance comparison of various entity linking approaches on the
test set of NCBI Disease. Bold highlights the highest performance among all
the approaches.

Method | F1 Score(%)
---|---
EventMine Pyysalo et al. (2012) | 61.60
HYPHEN Thompson et al. (2018) | 65.00
Base-EE (Ours) | 61.29
Joint4E-EE (Ours) | 65.78

Table 8: Performance comparison of various event extraction approaches on the
test set of PHAEDRA. Bold highlights the highest performance among all the
approaches.

###  6.3 Impact of the Number of Training Rounds

Tables 5 show the performance of both event extraction and entity linking at
each round of joint training based on the EM-style iterative algorithm. We
observe that the performance of both models gradually increases with more
rounds of joint training and both models achieve the highest performance after
3 rounds. Compared with the base models, both Joint4E-EE and Joint4E-EL
achieve significant improvements with a large marge: 3.12% absolute F1 score
gain for event extraction and 3.73% absolute accuracy gain for entity linking,
demonstrating the effectiveness of our joint learning framework. Table 6 shows
the event extraction performance (i.e., F1 Score) on each fine-grained event
type and three event type categories (including simple events, binding events
and complex events) at each round during joint training. As we can see, with
3-4 rounds of joint training, Joint4E-EE achieves up to 2.26%, 6.17%, and
2.99% absolute F1 score gain on the simple, binding and complex events,
indicating that binding events benefit the most from the entity knowledge from
external knowledge bases. This is consistent with our observation as many
entity descriptions in the knowledge base indicate the binding functions of
the entities. We also observe that, with more rounds of joint training, the
performance of Joint4E-EE decreases more on complex events which contain
multiple arguments and nested events, such as regulation, positive regulation,
and negative regulation.

###  6.4 Qualitative Analysis

Table 9 shows three examples for which the event predictions are improved and
corrected within the first 3 rounds of joint training. Taking the first
sentence as an example, before the first round of joint training, Joint4E-EE
mistakenly predicts a Phosphorylation event triggered by “phospho” with
“STAT3” as the Theme argument due to the misinterpretation of the sentence.
However, the entity knowledge retrieved from the Gene Ontology (GO) by
Joint4E-EL indicates that “STAT3 is a regulation of tyrosine STAT protein and
BMP-6 is a regulation of BMP signaling pathway”, while the word "regulation"
from both GO definitions helps better disambiguate the context during the 1st
round of joint training and finally calibrates the previous wrong event
predictions to the Regulation event triggered by “changes” with two arguments:
“STAT3” as the Theme and “BMP-6 as the Cause argument.

Similarly, Table 10 also shows three examples for which the entity linking
results are improved and corrected within the first 3 rounds of joint
training. Taking the first sentence as an example, before the 1st round of
joint training, Joint4E-EL mistakenly links the entity mention “UNC-75” to the
entity defined by “Positive regulation of synaptic transmission”. However, by
incorporating the event knowledge with joint training, especially knowing that
“UNC-75” is the Theme of a Binding event, Joint4E-EL correctly links “UNC-75”
to the target entity defined by “single-stranded RNA binding” in the Gene
Ontology.

Rounds |
---|---
1st |  Text: We did not observe any significant changes in the level of phospho-STAT3 or phospho-p38 upon BMP-6 treatment of B cells.
|  Previous Results:  Event type: Phosphorylation; Trigger: phospho; Theme:
STAT3.
|  Entity Knowledge from Joint4E-EL: STAT3: regulation of tyrosine STAT
protein; BMP-6:regulation of BMP signaling pathway.
|  New Results:  Event type: Regulation; Trigger: changes; Theme: STAT3,
Cause: BMP-6.
2nd |  Text: Costimulation through CD28 and/or CD2 did not modulate, the CD3-dependent phosphorylation of HS1.
|  Previous Results:  Event type: Phosphorylation; Trigger: phosphorylation;
Theme: CD28.
|  Entity Knowledge from Joint4E-EL: CD28: immune response; CD2: regulation of
CD4, CD25 regulatory T cell differentiation.
|  New Results:  Event type: Regulation; Trigger: modulate; Theme:
phosphorylation (event), Cause: CD28.
3rd |  Text: When tested its ability to block calcineurin-dependent signaling in cells, the pivotal promoter element for interleukin-2 gene induction.
|  Previous Results:  Event type: Regulation; Trigger: induction; Theme:
interleukin-2.
|  Entity Knowledge from Joint4E-EL: interleukin-2: plastid gene expression.
|  New Results:  Event type: Gene expression; Trigger: induction; Theme:
interleukin-2.

Table 9: Example sentences and results for event extraction at each round
sampled from the Genia 2011 development set. For each sentence, before each
round of joint training, the event prediction is not correct while after
incorporating the entity knowledge from Joint4E-EL, the errors are corrected
with joint training. The bold words in each text highlight the candidate event
triggers while the italic words highlight the candidate arguments predicted by
Joint4E-EE.

Rounds |
---|---
1st |  Text: To determine the elements in the exon 7 region that UNC-75 directly and specifically recognizes in vitro.
|  Previous Results:  Entity: Positive regulation of synaptic transmission.
|  Event Knowledge from Joint4E-EE: Event type: Binding; Trigger: recognize;
Theme: UNC-75.
|  New Results:  Entity: single-stranded RNA binding.
2nd |  Text: These data suggest that ectopically expressed BP downregulates FIL activity genes in ovaries.
|  Previous Results:  Entity: DNA-binding transcription factor activity, RNA
polymerase II-specific.
|  Event Knowledge from Joint4E-EE: Event type: negative regulation; Trigger:
downregulates; Theme: FIL.
|  New Results:  Entity: negative regulation of gene expression.
3rd |  Text: Effects of the gei-8 mutation on gene expression were studied with whole genome microarrays.
|  Previous Results:  Entity: reciprocal meiotic recombination.
|  Event Knowledge from Joint4E-EE: Event type: gene expression; trigger:
expression; Theme: gei-8.
|  New Results:  Entity: transcription by RNA polymerase II.

Table 10: Example sentences and results for entity linking at each round
sampled from the development set of BC4GO dataset. For each sentence, before
each round of joint training, the entity linking result is not correct while
after incorporating the event knowledge from Joint4E-EE, the errors are
corrected with joint training. The bold words in each text highlight the
candidate entity mention for entity linking.

###  6.5 Error Analysis

We further sample 50 prediction errors for both entity linking and event
extraction based on their results on the development set of each dataset,
respectively. We summarize the main error categories for each task as follows:

#### Event Extraction

The main error (33/50) for event extraction lies in the missing or spurious
argument predictions. Most event types such as simple events (including gene
expression, transcription, localization, phosphorylation, and protein
catabolism) are defined with a fixed number of arguments while the complex
events and binding events are usually associated with up to four possible
arguments, thus the model tends to miss some arguments or predict spurious
arguments. Taking the following two sentences as examples:

  * •

S1: The FOXP3 (arg: Theme) inhibition (trigger: Negative regulation) by GATA
element in the FOXP3 promoter (redundant arg: Site).

  * •

S2: Disruption of the Jak1 (arg: Theme) binding (trigger: Binding), proline-
rich Box1 (arg: Site) region of IL-4R (missing arg: Theme) abolished signaling
by this chimeric receptor.

For S1, our model successfully predicts inhibition as a Negative regulation
event and FOXP3 as its Theme argument. However, it also mistakenly predicts
promoter as a Site argument, due to two possible reasons: (1) the entity
promoter is frequently labeled as a Site argument in the training set; and (2)
the protein FOXP3 is defined as “regulation of DNA-templated transcription” in
the Gene Ontology, which also tends to imply promoter as a Site argument. In
S2, our Joint4E-EE correctly predicts binding as a Binding event with two
arguments: Jak1 and Box1. However, it mistakenly misses another Theme argument
which is likely because the model treats IL-4R as Box1 which is already
labeled as a Site argument.

#### Entity Linking

76% (38/50) of the remaining error for entity linking lies in the candidate
retrieval where the candidate sets retrieved based on the Gene Ontology (GO)
API for some entity mentions do not contain their true target entities. For
example, for the entity mention “TAT-DeltaDBD-GATA3”, the candidate set
returned by GO API does not include the true target entity GO:0019799 with
name acetyl-CoA:alpha-tubulin-L-lysine 6-N-acetyltransferase activity.

##  7 Conclusion

In this work, we propose a joint biomedical entity linking and event
extraction framework, i.e., Joint4E-ELand Joint4E-EE, to leverage the benefit
of one task to the other. Our Joint4E-EEcan incorporate the domain knowledge
obtained by Joint4E-EL, while Joint4E-EL can be improved by the event
structural context provided by Joint4E-EE. To iteratively improve the two
tasks together, we propose a novel hard-EM-style learning strategy where we
first estimate missing variables for both two incomplete datasets based on the
current task-specific models, and then update the parameters of both models on
the datasets that are augmented by pseudo labels. We conduct extensive
experiments on the biomedical entity linking dataset, i.e., BC4GO, and
biomedical event extraction, i.e., Genia 11. We also provide several valuable
discussions such as error analysis that reveals the remaining challenges of
both two tasks. We hope this work can shed light on the following research on
biomedical information extraction and broader communities.

## Limitations

While our proposed joint framework and learning strategy showcase promising
results for entity linking and event extraction in the biomedical domain, one
limitation is the potential restriction of specific domains and knowledge
bases which both entity linking and event extraction tasks can share with.

## References

  * Angell et al. (2021) Rico Angell, Nicholas Monath, Sunil Mohan, Nishant Yadav, and Andrew McCallum. 2021\.  Clustering-based inference for biomedical entity linking.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2598–2608, Online. Association for Computational Linguistics.
  * Bairoch et al. (2005) Amos Bairoch, Rolf Apweiler, Cathy H Wu, Winona C Barker, Brigitte Boeckmann, Serenella Ferro, Elisabeth Gasteiger, Hongzhan Huang, Rodrigo Lopez, Michele Magrane, et al. 2005.  The universal protein resource (uniprot).  _Nucleic acids research_ , 33(suppl_1):D154–D159.
  * Balakrishnan et al. (2013) Rama Balakrishnan, Midori A Harris, Rachael Huntley, Kimberly Van Auken, and J Michael Cherry. 2013.  A guide to best practices for gene ontology (go) manual annotation.  _Database_ , 2013.
  * Beltagy et al. (2019) Iz Beltagy, Kyle Lo, and Arman Cohan. 2019.  SciBERT: A pretrained language model for scientific text.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3615–3620, Hong Kong, China. Association for Computational Linguistics.
  * Bhowmik et al. (2021) Rajarshi Bhowmik, Karl Stratos, and Gerard de Melo. 2021.  Fast and effective biomedical entity linking using a dual encoder.  In _Proceedings of the 12th International Workshop on Health Text Mining and Information Analysis_ , pages 28–37, online. Association for Computational Linguistics.
  * Björne and Salakoski (2011) Jari Björne and Tapio Salakoski. 2011.  Generalizing biomedical event extraction.  In _Proceedings of BioNLP Shared Task 2011 Workshop_ , pages 183–191, Portland, Oregon, USA. Association for Computational Linguistics.
  * Björne and Salakoski (2018) Jari Björne and Tapio Salakoski. 2018.  Biomedical event extraction using convolutional neural networks and dependency parsing.  In _Proceedings of the BioNLP 2018 workshop_ , pages 98–108.
  * Bodenreider (2004) Olivier Bodenreider. 2004.  The unified medical language system (umls): integrating biomedical terminology.  _Nucleic acids research_ , 32(suppl_1):D267–D270.
  * Consortium (2012a) Gene Ontology Consortium. 2012a.  Gene ontology annotations and resources.  _Nucleic acids research_ , 41(D1):D530–D535.
  * Consortium (2012b) Gene Ontology Consortium. 2012b.  Gene ontology annotations and resources.  _Nucleic acids research_ , 41(D1):D530–D535.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers)_ , pages 4171–4186. Association for Computational Linguistics.
  * Doğan et al. (2014) Rezarta Islamaj Doğan, Robert Leaman, and Zhiyong Lu. 2014.  Ncbi disease corpus: a resource for disease name recognition and concept normalization.  _Journal of biomedical informatics_ , 47:1–10.
  * Espinosa et al. (2019) Kurt Junshean Espinosa, Makoto Miwa, and Sophia Ananiadou. 2019.  A search-based neural model for biomedical nested and overlapping event detection.  pages 3677–3684.
  * Han et al. (2019) Rujun Han, Qiang Ning, and Nanyun Peng. 2019.  Joint event and temporal relation extraction with shared representations and structured prediction.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 434–444, Hong Kong, China. Association for Computational Linguistics.
  * He et al. (2020) Yun He, Ziwei Zhu, Yin Zhang, Qin Chen, and James Caverlee. 2020.  Infusing disease knowledge into bert for health question answering, medical inference and disease name recognition.  _arXiv preprint arXiv:2010.03746_.
  * Huang et al. (2020) Kung-Hsiang Huang, Mu Yang, and Nanyun Peng. 2020.  Biomedical event extraction with hierarchical knowledge graphs.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 1277–1285, Online. Association for Computational Linguistics.
  * Jeong and Kang (2022) Minbyul Jeong and Jaewoo Kang. 2022.  Enhancing label consistency on document-level named entity recognition.  _arXiv preprint arXiv:2210.12949_.
  * Kanz et al. (2005) Carola Kanz, Philippe Aldebert, Nicola Althorpe, Wendy Baker, Alastair Baldwin, Kirsty Bates, Paul Browne, Alexandra van den Broek, Matias Castro, Guy Cochrane, et al. 2005.  The embl nucleotide sequence database.  _Nucleic acids research_ , 33(suppl_1):D29–D33.
  * Kim et al. (2009) Jin-Dong Kim, Tomoko Ohta, Sampo Pyysalo, Yoshinobu Kano, and Jun’ichi Tsujii. 2009.  Overview of bionlp’09 shared task on event extraction.  In _Proceedings of the BioNLP 2009 workshop companion volume for shared task_ , pages 1–9.
  * Kim et al. (2011) Jin-Dong Kim, Yue Wang, Toshihisa Takagi, and Akinori Yonezawa. 2011.  Overview of Genia event task in BioNLP shared task 2011.  In _Proceedings of BioNLP Shared Task 2011 Workshop_ , pages 7–15, Portland, Oregon, USA. Association for Computational Linguistics.
  * Kocaman and Talby (2021) Veysel Kocaman and David Talby. 2021.  Biomedical named entity recognition at scale.  In _Pattern Recognition. ICPR International Workshops and Challenges: Virtual Event, January 10–15, 2021, Proceedings, Part I_ , pages 635–646. Springer.
  * Leitner et al. (2010) Florian Leitner, Scott A Mardis, Martin Krallinger, Gianni Cesareni, Lynette A Hirschman, and Alfonso Valencia. 2010.  An overview of biocreative ii. 5.  _IEEE/ACM Transactions on Computational Biology and Bioinformatics_ , 7(3):385–399.
  * Li et al. (2019) Diya Li, Lifu Huang, Heng Ji, and Jiawei Han. 2019.  Biomedical event extraction based on knowledge-driven tree-lstm.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 1421–1430.
  * Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. 2019.  Decoupled weight decay regularization.  In _International Conference on Learning Representations_.
  * Majumder et al. (2016) Amit Majumder, Asif Ekbal, and Sudip Kumar Naskar. 2016.  Biomolecular event extraction using a stacked generalization based classifier.  In _Proceedings of the 13th International Conference on Natural Language Processing_ , pages 55–64, Varanasi, India. NLP Association of India.
  * Orr et al. (2020) Laurel Orr, Megan Leszczynski, Simran Arora, Sen Wu, Neel Guha, Xiao Ling, and Christopher Re. 2020.  Bootleg: Chasing the tail with self-supervised named entity disambiguation.  _arXiv preprint arXiv:2010.10363_.
  * Park et al. (2011) Yu Rang Park, Jihun Kim, Hye Won Lee, Young Jo Yoon, and Ju Han Kim. 2011.  Gochase-ii: correcting semantic inconsistencies from gene ontology-based annotations for gene products.  _BMC bioinformatics_ , 12(1):1–7.
  * Pyysalo et al. (2012) Sampo Pyysalo, Tomoko Ohta, Makoto Miwa, Han-Cheol Cho, Jun’ichi Tsujii, and Sophia Ananiadou. 2012.  Event extraction across multiple levels of biological organization.  _Bioinformatics (Oxford, England)_ , 28(18):i575—i581.
  * Ramponi et al. (2020) Alan Ramponi, Rob van der Goot, Rosario Lombardo, and Barbara Plank. 2020.  Biomedical event extraction as sequence labeling.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 5357–5367.
  * Rohanian et al. (2023) Omid Rohanian, Mohammadmahdi Nouriborji, Samaneh Kouchaki, and David A Clifton. 2023\.  On the effectiveness of compact biomedical transformers.  _Bioinformatics_ , 39(3):btad103.
  * Segura-Bedmar et al. (2013) Isabel Segura-Bedmar, Paloma Martínez Fernández, and María Herrero Zazo. 2013.  Semeval-2013 task 9: Extraction of drug-drug interactions from biomedical texts (ddiextraction 2013).  Association for Computational Linguistics.
  * Sun et al. (2021) Cong Sun, Zhihao Yang, Lei Wang, Yin Zhang, Hongfei Lin, and Jian Wang. 2021.  Biomedical named entity recognition using bert in the machine reading comprehension framework.  _Journal of Biomedical Informatics_ , 118:103799.
  * Thompson et al. (2018) Paul Thompson, Sophia Daikou, Kenju Ueno, Riza Batista-Navarro, Jun’ichi Tsujii, and Sophia Ananiadou. 2018.  Annotation and detection of drug effects in text for pharmacovigilance.  _Journal of cheminformatics_ , 10(1):1–33.
  * Trieu et al. (2020) Hai-Long Trieu, Thy Thy Tran, Khoa NA Duong, Anh Nguyen, Makoto Miwa, and Sophia Ananiadou. 2020.  Deepeventmine: end-to-end neural nested event extraction from biomedical texts.  _Bioinformatics_ , 36(19):4910–4917.
  * Vader et al. (2006) Gerben Vader, René H Medema, and Susanne MA Lens. 2006.  The chromosomal passenger complex: guiding aurora-b through mitosis.  _The Journal of cell biology_ , 173(6):833–837.
  * Van Auken et al. (2014) Kimberly Van Auken, Mary L Schaeffer, Peter McQuilton, Stanley JF Laulederkind, Donghui Li, Shur-Jen Wang, G Thomas Hayman, Susan Tweedie, Cecilia N Arighi, James Done, et al. 2014.  Bc4go: a full-text corpus for the biocreative iv go task.  _Database_ , 2014.
  * Varma et al. (2021) Maya Varma, Laurel Orr, Sen Wu, Megan Leszczynski, Xiao Ling, and Christopher Ré. 2021.  Cross-domain data integration for named entity disambiguation in biomedical text.  _arXiv preprint arXiv:2110.08228_.
  * Wang et al. (2020) Xing David Wang, Leon Weber, and Ulf Leser. 2020.  Biomedical event extraction as multi-turn question answering.  In _Proceedings of the 11th International Workshop on Health Text Mining and Information Analysis_ , pages 88–96.
  * Wang et al. (2022) Yan Wang, Jian Wang, Huiyi Lu, Bing Xu, Yijia Zhang, Santosh Kumar Banbhrani, Hongfei Lin, et al. 2022.  Conditional probability joint extraction of nested biomedical events: Design of a unified extraction framework based on neural networks.  _JMIR Medical Informatics_ , 10(6):e37804.
  * Wright (2019) Dustin Wright. 2019.  _NormCo: Deep disease normalization for biomedical knowledge base construction_.  University of California, San Diego.
  * Wu et al. (2020) Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. 2020.  Scalable zero-shot entity linking with dense entity retrieval.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020_ , pages 6397–6407. Association for Computational Linguistics.
  * Xu et al. (2020) Dongfang Xu, Zeyu Zhang, and Steven Bethard. 2020.  A generate-and-rank framework with semantic type regularization for biomedical concept normalization.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8452–8464, Online. Association for Computational Linguistics.
  * Yasunaga et al. (2022) Michihiro Yasunaga, Jure Leskovec, and Percy Liang. 2022.  Linkbert: Pretraining language models with document links.  _arXiv preprint arXiv:2203.15827_.
  * Yon Rhee et al. (2008) Seung Yon Rhee, Valerie Wood, Kara Dolinski, and Sorin Draghici. 2008.  Use and misuse of the gene ontology annotations.  _Nature Reviews Genetics_ , 9(7):509–515.
  * Yuan et al. (2022) Hongyi Yuan, Zheng Yuan, and Sheng Yu. 2022.  Generative biomedical entity linking via knowledge base-guided pre-training and synonyms-aware fine-tuning.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022_ , pages 4038–4048. Association for Computational Linguistics.
  * Zhao et al. (2021) Weizhong Zhao, Jinyong Zhang, Jincai Yang, Tingting He, Huifang Ma, and Zhixin Li. 2021.  A novel joint biomedical event extraction framework via two-level modeling of documents.  _Information Sciences_ , 550:27–40.
  * Zhu et al. (2020) Ming Zhu, Busra Celikkaya, Parminder Bhatia, and Chandan K Reddy. 2020.  Latte: Latent type modeling for biomedical entity linking.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 9757–9764.
  * Zhu et al. (2021) Tiantian Zhu, Yang Qin, Qingcai Chen, Baotian Hu, and Yang Xiang. 2021.  Enhancing entity representations with prompt learning for biomedical entity linking.

##  Appendix A More Details on Experiment Setup

###  A.1 Dataset Processing for Entity Linking

In this section, we elaborate on how we construct the mappings between the
mentions in BC4GO and the entities in Gene Ontology. Specifically, Gene
Ontology consists of three directed acyclic graphs (DAG) and there is no
connected component between them. All the nodes in the three DAGs are
biomedical concepts and each DAG has a hierarchical structure called an
ancestor chart. The biomedical concepts at the upper level of an ancestor
chart have broader meanings compared with the biomedical concepts at the lower
level. The three nodes at the top of the three ancestor charts in Gene
Ontology are: Cellular Component, Molecular Function, and Biological Process,
which are the broadest concepts in biomedical science and any biomedical
concept is contained in one of them. We refer to the three topmost nodes in
the three ancestor charts as root nodes in the rest of the section. For each
node in an ancestor chart, the nodes above it are broader concepts while the
nodes below it are finer concepts, and there is no edge between the nodes from
the same level. For each entity mention $m_{i}$ from a sentence $\mathbf{w}$
of the BC4GO dataset, we denote the annotated target entity from Gene Ontology
for $m_{i}$ in the original BC4GO dataset as $c^{gold}_{i}$. We also take
$m_{i}$ as a query to the search engine via the API of Gene Ontology and
obtain a set of candidate entities
$\mathcal{C}_{i}=\\{c_{i1},c_{i2},...,c_{ij}\\}$. Then, for each
$c_{ij}\in\mathcal{C}_{i}$, we find a path between $c_{ij}$ and the root node
of a particular ancestor chart, denoted as $\mathbf{p}_{ij}$. Following a
similar process, we also find the path $\mathbf{p}_{gold}$ between the gold
target entity $c^{gold}_{i}$ and a root node. Based on these paths, we design
the following four strategies to determine a new set of gold entities from
$\mathcal{C}_{i}$ for each mention $m_{i}$ as the reference in Gene Ontology:

  * •

If a candidate entity $c_{ij}\in\mathcal{C}_{i}$ for the mention $m_{i}$ is
the same as the ground-truth concept $c^{gold}_{i}$ in the original BC4GO
dataset, we add $c_{ij}$ to the gold target entity set.

  * •

If a candidate entity $c_{ij}\in\mathcal{C}_{i}$ for the mention $m_{i}$ is on
the path $\mathbf{p}_{gold}$, we add $c_{ij}$ to the gold target entity set.

  * •

If the gold target entity $c^{gold}_{i}$ annotated in the original BC4GO
dataset is on the path $\mathbf{p}_{ij}$ for a candidate entity $c_{ij}$, we
add $c_{ij}$ to the gold target entity set.

  * •

If the path $\mathbf{p}_{ij}$ for a particular candidate entity $c_{ij}$ have
more than 4 overlapped nodes with the path $\mathbf{p}_{gold}$ for the
original gold entity, we add $c_{ij}$ to the gold target entity set.