#  EnriCo: Enriched Representation and Globally Constrained Inference for
Entity and Relation Extraction

Urchade Zaratiana1,2, Nadi Tomeh2, Yann Dauxais1, Pierre Holat1,2, Thierry
Charnois2
1 FI Group, 2 LIPN, CNRS UMR 7030, France
<EMAIL_ADDRESS>
Code: https://github.com/urchade/enrico

###### Abstract

Joint entity and relation extraction plays a pivotal role in various
applications, notably in the construction of knowledge graphs. Despite recent
progress, existing approaches often fall short in two key aspects: richness of
representation and coherence in output structure. These models often rely on
handcrafted heuristics for computing entity and relation representations,
potentially leading to loss of crucial information. Furthermore, they
disregard task and/or dataset-specific constraints, resulting in output
structures that lack coherence. In our work, we introduce EnriCo, which
mitigates these shortcomings. Firstly, to foster rich and expressive
representation, our model leverage attention mechanisms that allow both
entities and relations to dynamically determine the pertinent information
required for accurate extraction. Secondly, we introduce a series of decoding
algorithms designed to infer the highest scoring solutions while adhering to
task and dataset-specific constraints, thus promoting structured and coherent
outputs. Our model demonstrates competitive performance compared to baselines
when evaluated on Joint IE datasets.

EnriCo: Enriched Representation and Globally Constrained Inference for Entity
and Relation Extraction

Urchade Zaratiana1,2, Nadi Tomeh2, Yann Dauxais1, Pierre Holat1,2, Thierry
Charnois2 1 FI Group, 2 LIPN, CNRS UMR 7030, France<EMAIL_ADDRESS>Code:
https://github.com/urchade/enrico

##  1 Introduction

Joint entity and relation extraction is a pivotal task in Natural Language
Processing (NLP), aiming to identify entities (such as “Person” or
“Organization”) within raw text and to discern the relationships between them
(such as “Work_for”). This process forms the cornerstone for numerous
applications, including the construction of Knowledge Graphs (Nickel et al.,
2016). Traditionally, this task is tackled via pipeline models that
independently trained and implemented entity recognition and relation
extraction, often leading to error propagation (Brin, 1999; Nadeau and Sekine,
2007). The advent of deep learning has facilitated the development of end-to-
end and multitask models for this task, enabling the utilization of shared
representations and the simultaneous optimization of loss functions for both
tasks. (Wadden et al., 2019; Wang and Lu, 2020; Zhao et al., 2021; Zhong and
Chen, 2021; Yan et al., 2021). Despite this advancement, these models
essentially remain pipeline-based, with entity and relation predictions
executed by separate classification heads, thereby ignoring potential
interactions between these tasks.

While end-to-end models have been proposed (Lin et al., 2020), they often
resort to hand-coded operations like concatenation for computing entity and
relation representations, thereby limiting their flexibility. Moreover, these
representations ignore potential inter-span and inter-relation interactions,
as well as their interactions with the input text. Integrating these
interactions could enrich the representations by preserving valuable
contextual information overlooked during pooling operations. Moreover,
existing approaches tend to overlook the structured nature of the output. In
many real-world scenarios, the relationships between entities follow certain
patterns or constraints, which may vary depending on the domain or dataset.
However, current models typically treat entity and relation extraction as
separate classification tasks without considering these constraints
explicitly. Consequently, the extracted entities and relations may lack
coherence or violate domain-specific rules, limiting the utility of the
extracted knowledge.

Figure 1:  The model consists of three key components: (1) Word
Representation, responsible for computing word embeddings for each word in the
input sentence. (2) Entity Classification Module, which calculates, prunes,
enriches span representations, and classifies them. (3) Relation
Classification Module, which similarly calculates, prunes, enriches span
representations, and classifies them. The pruning and enrichment of entity and
relation representations are performed by a “Filter and Refine” layer, as
described in Section 2.5 and illustrated in Figure 2.

In this work, we address these limitations by proposing EnriCo, a novel
framework for joint entity and relation extraction. EnriCo aims to provide
richer representation and promote coherence in output structures by leveraging
attention mechanisms (Vaswani et al., 2017) and incorporating task and
dataset-specific constraints during decoding. To enhance representation
richness, EnriCo employs attention mechanisms that allow entities and
relations to dynamically attend to relevant parts of the input text (Fig. 4).
This allows for richer and more expressive representations by preserving
valuable contextual information that may be overlooked by traditional pooling
operations. In addition, it also incorporates span and relation-level
interactions, enabling each candidate entity or relation to update its
representation based on the presence and characteristics of other candidate
entities or relations in the text. This fosters a more holistic understanding
of the relationships between different spans and relations, helping to resolve
ambiguities and improve extraction accuracy (Table 8). To address the
computational complexity arising from a large number of spans and relations,
the model integrates a filtering layer to prune candidates, retaining only the
most relevant ones, enabling efficient processing without sacrificing
accuracy. Previous works have also leveraged rich high-level interactions. For
instance, Zaratiana et al. (2022a) employed span-level interaction, yet their
application is confined to entity recognition and lacks incorporation of
pruning, making it computationally inefficient due to large number of spans.
Similarly, Zhu et al. (2023) proposed span-to-token interaction for NER, but
our work extends this approach to the relation (span-pair) level.

Finally, to ensure structural coherence of the output, we introduce a series
of decoding algorithms to boost model performance by integrating task-specific
and dataset-specific constraints. To achieve this, we formulate entity and
relation prediction using an Answer Set Programming (ASP) solver, enabling the
derivation of exact solutions. Experiments across benchmark datasets
demonstrate the efficacy and performance of our proposed model.

##  2 Architecture

In this section, we present the architecture of our proposed model, EnriCo,
for joint entity and relation extraction. The overall architecture comprises
three main components: (1) Word Representation, (2) Entity Classification, and
(3) Relation Classification modules. Figure 1 illustrates the architecture,
offering a visual overview of how these modules interact.

###  2.1 Token Representation

The primary purpose of this module is to generate word embeddings from the
input sentences. For that, we use a transformer layer that takes an input text
sequence $\mathbf{x}$ and outputs token representations
$\mathbf{H}\in\mathbb{R}^{L\times D}$, where $D$ is the model dimension. In
practice, this component is a pretrained transformer encoder such as BERT
(Devlin et al., 2019).

###  2.2 Entity Module

In the Entity Module, the objective is to identify and classify spans in the
input text as entities. A span refers to a contiguous sequence of words in the
text that represents a candidate entity. Each entity is defined by its start
and end positions within the input sentence, as well as its associated entity
type. For example, the spans “Alain Farley” and “Montreal” could be classified
as entities of type “Person” and “Location”, respectively.

#### Span representation

To compute span representations, we first enumerate all possible spans from
the input sentence (up to a maximum span length in practice). Then, for each
span, we concatenate the embeddings of its start and end words to compute a
span representation. More formally, the span representation $\mathbf{S}$ for a
span starting at word $i$ and ending at word $j$ is given by:

$\mathbf{S}_{ij}={\bm{w}}_{ent}^{T}(\mathbf{h}_{i}^{s}\oplus\mathbf{h}_{j}^{e})$
(1)

where $\mathbf{h}_{i}^{s}$ and $\mathbf{h}_{j}^{e}$ are the embeddings of the
start and end words, ${\bm{w}}_{ent}\in\mathbb{R}^{2D\times D}$ is a learned
weight matrix, and $\oplus$ denotes concatenation. In total, we compute
$L\times M$ span vectors (we mask invalids), where $L$ represents the sentence
length and $M$ represents the maximum span length, thus
$\mathbf{S}\in\mathbb{R}^{LM\times D}$. The spans are then passed into a
Filter and Refine (Sec. 2.5) layer to prune the number of spans to $K$ and
update their representation, resulting in
$\mathbf{S}_{f}\in\mathbb{R}^{K\times D}$. The span representation
$\mathbf{S}_{f}$ will serve for both span classification in the next paragraph
and the relation representation in Sec. 2.3.

#### Span classification

For span classification, we feed the representation of the filtered span
$\mathbf{S}_{f}$ into a feed-forward network to obtain the span classification
score:

$\begin{split}\mathbf{Y}^{ent}=\texttt{FFN}(\mathbf{S}_{f})\in\mathbb{R}^{K\times|\mathcal{E}|}\end{split}$
(2)

where $|\mathcal{E}|$ corresponds to the number of entity types, including the
non-entity type.

###  2.3 Relation Module

In the Relation Module, the goal is to classify pairs of spans in the input
text as specific relations. For instance, when presented with two spans “Alain
Farley” and “McGill University”, this module has to predict the relation
between them, such as “Work_for” in this case.

#### Relation representation

To compute the representation of a relation between two spans $(i,j)$ and
$(k,l)$, we simply concatenate their respective span representations using

$\mathbf{R}_{ij|kl}={\bm{w}}_{rel}^{T}(\mathbf{S}_{f_{ij}}^{head}\oplus\mathbf{S}_{f_{kl}}^{tail})$
(3)

where $\mathbf{S}_{f_{ij}}^{head}$ and $\mathbf{S}_{f_{kl}}^{tail}$ are the
span representations for the spans $(i,j)$ and $(k,l)$ respectively and
${\bm{w}}_{rel}\in\mathbb{R}^{2D\times D}$ is a learned weight matrix. This
operation results in $K\times K$ candidate relations, corresponding to all
pairs of candidate entities. Similarly to the entities, we process the
relation representations through a Filter and Refine (Sec. 2.5) layer to
reduce their quantity to $K$, thereby updating their representation, which
results in $\mathbf{R}_{f}\in\mathbb{R}^{K\times D}$.

#### Relation classification

Finally, we compute the relation classification score for each relation
representation using a feed-forward network:

$\begin{split}\mathbf{Y}^{rel}=\texttt{FFN}(\mathbf{R}_{f})\in\mathbb{R}^{K\times|\mathcal{R}|}\end{split}$
(4)

where $|\mathcal{R}|$ represents the number of relation types, including the
no-relation type.

###  2.4 Entity-Relation Biases

To facilitate a more nuanced interaction between entity and relation
prediction, our model incorporates a bias score for each combination of (head
and tail) entity types and relation type (see Fig. 3 for an illustrative
example). In the training phase, these bias scores are learned and seamlessly
integrated into the relation score. Specifically, we augment the relation
logits (all $y_{r}^{rel}\in\mathbf{Y}^{rel}$) by incorporating information
about the predicted head entity type $h\in\mathcal{E}$ and the tail entity
type $t\in\mathcal{E}$ in the following manner:

$y_{rht}^{rel}=y_{r}^{rel}+\textbf{b}(h,t,r)$ (5)

where $\textbf{b}(h,t,r)\in\mathbb{R}$, a learned bias score for the triplet
$(h,t,r)$ defined as follow:

$\displaystyle\bm{b}(h,t,r)$
$\displaystyle=\bm{\phi}(h,t,r)+\bm{\phi}(h,r)+\bm{\phi}(t,r)+\bm{\phi}(h,t)$
(6)

In the above, we use the Gumbel-Softmax trick (Jang et al., 2017) to predict
discrete entity types $h$ and $t$, enabling gradient-based optimization of the
whole process. The term $\bm{\phi}(h,t,r)$ captures the joint affinity score
between a specific head, tail, and relation type. For instance, if the head
entity is Person and the tail entity is Organization, the relation score would
be higher for works_for than born_in. Meanwhile, $\bm{\phi}(h,r)$ and
$\bm{\phi}(t,r)$ capture the general tendencies for entities (head or tail) of
certain types to engage in specific relations. Lastly, $\bm{\phi}(h,t)$
capture any intrinsic compatibility between an head and tail types.
Furthermore, another utility of the bias term is that it allows to incorporate
domain constraints by manually assigning large negative values to invalid
triples (see Table 1 and 2).

Figure 2: Filter and Refine. This layer processes either span or relation
representations. It first computes a ranking score for each span or relation,
selecting those with the highest top-k values. The selected spans or relations
are then passed through a “Read & Process” layer. Figure 3: Biases value (Sec.
2.4) for ACE 05 dataset. This figure shows the values of learned biases for
different associations of entity and relation types. (left) $\bm{\phi}(h,r)$,
bias scores between head entity type and relation type. (middle)
$\bm{\phi}(t,r)$, bias scores between tail entity type and relation type.
(right) $\bm{\phi}(h,t)$, bias scores between head entity type and tail entity
type.

###  2.5 Filter and Refine

In this section, we detail the Filter and Refine layer (see Figure 2), a
crucial component utilized in both the entity and relation blocks. The purpose
of this layer is to prune the candidate elements (entities or relations) and
then enhance their representations. Let $\mathbf{Z}\in\mathbb{R}^{N\times D}$
denote the matrix containing the representations of either entities or
relations (i.e., $\mathbf{S}$ or $\mathbf{R}$), where $N$ represents the
number of entities or relations, and $D$ is the dimension of the model.

#### Filtering mechanism

The filtering first computes ranking scores for each element in $\mathbf{Z}$
using a FFN:

$\mathbf{F}=\texttt{FFN}(\mathbf{Z})\in\mathbb{R}^{N\times 1}$ (7)

Let $\texttt{argtopK}(\mathbf{F})$ denote the indices of the top $K$ elements
in the vector $\mathbf{F}$. Then, the filtered set
$\mathbf{Z}_{f}\in\mathbb{R}^{K\times D}$ is defined as:

$\begin{split}\mathbf{Z}_{f}&=\texttt{select\\_topk}(\mathbf{Z},\mathbf{F})\\\
&=\left[\mathbf{Z}[i]\,|\,i\in\texttt{argtopK}(\mathbf{F})\right]\end{split}$
(8)

This equation defines $\mathbf{Z}_{f}$ as the subset of $\mathbf{Z}$
containing only those elements that are ranked within the top $K$ according to
their scores.

#### Refine mechanism

The refine module updates the representations of ${\bm{Z}}_{f}$ using two
layers: READ and PROCESS. The READ layer updates each element in
${\bm{Z}}_{f}$ by incorporating information from the original token
representation ${\bm{H}}$, using multi-head attention:

${\bm{Z}}_{f}={\bm{Z}}_{f}+\texttt{MHA}({\bm{Z}}_{f},{\bm{H}})$ (9)

where $\texttt{MHA}(\mathbf{Z}_{f},\mathbf{H})$ represents a multi-head
attention mechanism with Queries $\mathbf{Z}_{f}$ and Keys and Values
$\mathbf{H}$. This operation proves beneficial as some information may be lost
during the hand-crafted representation computation via concatenation
(Equations 1 and 3). Allowing the entity and relation representations to
attend to the original input sequence enables them to dynamically gather
crucial information, thereby enhancing overall performance (see ablation study
in Table 8). Furthermore, this introduces an additional layer of
interpretability to our model, as illustrated in Figure 4. Additionally, the
PROCESS layer updates the representations by enabling each element ($\in$
${\bm{Z}}_{f}$) to aggregate information from others ($\in$ ${\bm{Z}}_{f}$):

$\displaystyle{\bm{Z}}_{f}$
$\displaystyle={\bm{Z}}_{f}+\texttt{MHA}({\bm{Z}}_{f},{\bm{Z}}_{f}),$ (10)
$\displaystyle{\bm{Z}}_{f}$
$\displaystyle={\bm{Z}}_{f}+\texttt{FFN}({\bm{Z}}_{f})$

where $\texttt{MHA}(\mathbf{Z}_{f},\mathbf{Z}_{f})$ is a multi-head self-
attention layer, and $\texttt{FFN}(\mathbf{Z}_{f})$ is a feed-forward network.
While inter-span interactions have been explored in previous works (Zaratiana
et al., 2022a; Floquet et al., 2023), we are the first to employ this
mechanism at the relation level.

###  2.6 Training

During training, our model employs multi-task learning by jointly minimizing
the filtering and classification losses. We utilize a pairwise ranking loss
with margin for the filtering: Usunier et al. (2009):

$\mathcal{L}_{f}=\sum_{p=1}^{N}\sum_{n=1}^{N}\max(0,{\bm{F}}_{n}-{\bm{F}}_{p}+\alpha)\cdot\delta(y_{p},y_{n})$
(11)

In this equation, $\mathbf{F}_{p}$ and $\mathbf{F}_{n}$ represent the
filtering scores for positive and negative samples (computed in Sec. 2.5),
respectively. The term $\alpha$ is the margin, and $\delta(y_{p},y_{n})$ is an
indicator function that is active when $y_{p}=1$ and $y_{n}=0$. This loss
function encourages the model to prioritize positive samples over negative
ones. This loss is applied at both the entity and relation levels. For the
classification, we minimize the negative log-likelihood of the gold label
spans and relations (on $\mathbf{Y}^{ent}$ and $\mathbf{Y}^{rel}$). Finally,
the total loss function is a sum of all losses:

$\mathcal{L}_{total}=\mathcal{L}_{f}^{ent}+\mathcal{L}_{f}^{rel}+\mathcal{L}_{cl}^{ent}+\mathcal{L}_{cl}^{rel}$
(12)

Here, $\mathcal{L}_{total}$ is the sum of filtering losses
($\mathcal{L}_{f}^{ent}+\mathcal{L}_{f}^{rel}$) and classification losses
($\mathcal{L}_{cl}^{ent}+\mathcal{L}_{cl}^{rel}$) for entities and relations.
To maintain simplicity, we do not add weighting terms for individual losses.

##  3 Decoding

In this section, we details the different decoding algorithm we employed in
this paper. The role of decoding is to produce the final output, which
comprises the prediction of entity types (span prediction) and relation types
(span pair prediction).

###  3.1 Unconstrained Decoding

Our baseline is unconstrained decoding, which corresponds to the raw
predictions of the model for both entities and relations. The predictions for
entities are obtained as follows:

$E_{\textit{p}}=\left\\{(i,j,c)\;\middle|\;\begin{array}[]{@{}l@{\;}l}&c=\operatorname*{arg\,max}_{c^{\prime}}{\bm{Y}}_{ijc^{\prime}}^{ent}\\\
&c\neq\texttt{non-entity}\end{array}\right\\}$ (13)

where $\mathbf{Y}_{ijc^{\prime}}^{ent}$ is the score of the spans $(i,j)$
having entity type $c^{\prime}\in\mathbb{R}^{|\mathcal{E}|}$ (see the
computation of span classification score in Equation 2). Furthermore, the
prediction of the relations are obtained as follows:

$R_{\textit{p}}=\left\\{(h,t,r)\;\middle|\;\begin{array}[]{@{}l@{\;}l}&r=\operatorname*{arg\,max}_{r^{\prime}}{\bm{Y}}_{htr^{\prime}}^{rel}\\\
&r\neq\texttt{no-relation}\end{array}\right\\}$ (14)

where ${\bm{Y}}_{htr^{\prime}}^{rel}$ is the score of the pairs of span $h$
and $t$ having relation type $r^{\prime}\in\mathbb{R}^{|\mathcal{R}|}$ (the
computation of relation classification score is in equation 4).

###  3.2 Constrained Decoding

#### Motivations

The unconstrained decoding we describe before, does not consider the task-
specific which are crucial for producing well-formed and coherent outputs. For
instance, the Joint IE task has the following constraints:

  * •

Unique Type Assignment: Each entity and relation must have a unique type
assigned to it. (Trivial)

  * •

Non-overlapping Entity Spans: Predicted entity spans must not overlap with
each other.

  * •

Consistency: A valid relation can only be formed by two valid entities, i.e.,
a relation cannot be formed by a non-entity span.

Moreover, each dataset may have its specific constraints. For instance, in the
CoNLL 04 dataset, if the head entity is people and the tail is Org, the
relation type should be work_for (or non-relation) (see Table 1 for an
exhaustive list).

Head | Tail | Relation
---|---|---
Peop | Org | Work_For
Peop | Loc | Live_in
Org | Loc | OrgBased_in
Loc | Loc | Located_in
Peop | Peop | Kill
Table 1: CoNLL 04 dataset constraints. The description of the entity and
relation types are detailed in the appendix.

#### Inference with ASP

In our work, we formulate the decoding problem using ASP (Answer Set
Programming) (Brewka et al., 2011; Gebser et al., 2014), a form of declarative
programming oriented towards combinatorial search problems. This framework is
particularly suitable for our task, as it allows for the integration of
various constraints in a straightforward manner. We implement three decoding
variants: Joint, which jointly optimizes the global score for entities and
relations; Entity First, which first finds the optimal solution for entities
and then for relations conditioned by predicted entities; and Relation First,
which initially finds the optimal solution for relations and then for entities
given the relations. For these decodings, we integrate both task-specific
(described above) and dataset constraints (Table 1 and 2). We provide pseudo-
code in the appendix (Figure 6).

#### Fast variant

While ASP provides strong performance, we find it is slow in practice. To
address this, we propose a more scalable solution, which is equivalent to the
Entity First variant of ASP, described before. Firstly, we predict candidate
entities $E_{\textit{p}}$ using Equation 13. Then, we search for the optimal
solution $\hat{E}_{\textit{p}}$, which is a subset of $E_{\textit{p}}$ with no
overlapping spans and the maximum score:

$\hat{E}_{\textit{p}}=\operatorname*{arg\,max}_{E\in\Psi(E_{\textit{p}})}\sum_{(i,j,c)\in
E}{\bm{Y}}_{ijc}^{ent}$ (15)

where $\Psi(E_{\textit{p}})$ contains all possible solution. The solution to
this problem is provided by Zaratiana et al. (2022b), who transform the
problem into a weighted graph search to derive exact solution. Then, once the
entities are determined, the goal is to predict the types of each candidate
relation based on these predicted entities. A key assumption is that the type
of one relation is independent of others, provided the entities are known (i.e
there is no inter-relation constraints). Therefore, we can predict each
relation types (for all $y^{rel}\in{\bm{Y}}^{rel}$) independently as follow:

$r=\operatorname*{arg\,max}_{r\in\mathcal{R}}y_{r}^{rel}+\textbf{b}(h,t,r)$
(16)

where $h$ and $t$ are respectively the type of head and tail entities. The
bias term $\textbf{b}(h,t,r)$ add entity prediction information in the
relation facilitate the integration of constraints into the prediction. It
does so by assigning a negative infinity value to any invalid entity-relation
type associations, as dictated by the specific dataset constraint (Table 1 and
2). As shown in the table 3, this algorithm is significantly faster than ASP-
based approached, while allowing the adherence to constraints.

Head | Tail | Relations
---|---|---
PER | FAC | ART, PHYS
PER | LOC | PHYS, GEN-AFF
PER | GPE | PHYS, ORG-AFF, GEN-AFF
PER | PER | PER-SOC, GEN-AFF
PER | ORG | ORG-AFF, GEN-AFF
PER | WEA | ART
PER | VEH | ART
FAC | FAC | PART-WHOLE, PHYS
FAC | GPE | PART-WHOLE, PHYS
FAC | LOC | PART-WHOLE, PHYS
GPE | FAC | PART-WHOLE, PHYS, ART
GPE | GPE | PART-WHOLE, PHYS, ORG-AFF
GPE | LOC | PART-WHOLE, PHYS
GPE | ORG | ORG-AFF
GPE | WEA | ART
GPE | VEH | ART
LOC | FAC | PART-WHOLE, PHYS
LOC | GPE | PART-WHOLE, PHYS
LOC | LOC | PART-WHOLE, PHYS
ORG | ORG | PART-WHOLE, ORG-AFF
ORG | GPE | PART-WHOLE, ORG-AFF, GEN-AFF
ORG | WEA | ART
ORG | VEH | ART
ORG | FAC | ART
ORG | LOC | GEN-AFF
VEH | VEH | PART-WHOLE
WEA | WEA | PART-WHOLE

Table 2: ACE 05 dataset constraints. The description of the entity and relation types are detailed in the appendix. | ASP-based | Fast variant
---|---|---
Joint | 6.7 | -
Relation first | 7.4 | -
Entity first | 5.5 | 21.7
Table 3: Decoding speed in sentence per second. All decoding can be
implemented using ASP solver. Entity first variant can be implemented without
ASP resulting in faster decoding.

##  4 Experimental Setup

###  4.1 Datasets

We evaluated our model on three datasets for joint entity-relation extraction,
namely SciERC (Luan et al., 2018), CoNLL04 (Carreras and Màrquez, 2004), and
ACE 05 (Walker et al., 2006). We provide details and statistics about the
datasets in the Table 4 and the description of entity and relation types in
Table 9.

#### ACE 05

is collected from a variety of domains, such as newswire, online forums and
broadcast news. It provides a diverse set of entity types such as Persons
(PER), Locations (LOC), Geopolitical Entities (GPE), and Organizations (ORG),
along with intricate relation types that include Artifact relationships (ART),
General affiliations (GEN-AFF), and Personal social relationships (PER-SOC).
This dataset is particularly notable for its complexity and wide coverage of
entity and relation types, making it a robust benchmark for evaluating the
performance of Joint IE models.

#### CoNLL04

is a popular benchmark dataset for entity-relation extraction in English. It
focuses on general entities such as People, Organizations, and Locations. The
dataset primarily includes simple and generic relations like Work_For and
Live_in.

#### SciERC

dataset is specifically designed for the AI domain. It includes entity and
relation annotations from a collection of documents from 500 AI paper
abstracts. It contains entity types such as Task, Method, Metric and relation
types such as Use-for, Part-of and Compare. SciERC is particularly suited for
constructing knowledge graphs in the AI domain.

###  4.2 Dataset Constraints

In this section, we discuss the dataset constraints used in our work. For the
CoNLL 04 dataset, the constraints are based on the seminal work of Roth and
Yih Roth and Yih (2004), which we report in Table 1. The constraints for this
dataset are relatively simple, allowing only five triplet combinations, for
instance, (Peop, Org, Work_For). For the ACE 05 dataset, no constraints were
publicly available. Thus, we decided to design the constraints manually by
examining the annotation guidelines provided by the Linguistic Data Consortium
dataset’s annotation guidelines
111https://www.ldc.upenn.edu/collaborations/past-projects/ace/annotation-
tasks-and-specifications, resulting in the set of constraints reported in
Table 2. As shown in the table, the task for the ACE 05 dataset is highly
complex, with more than 40 possible triples compared to CoNLL 04, which only
has 5. Finally, for SciERC, we do not include dataset-specific constraints as
the annotation guideline is not detailed enough to permit that, and the
presence of ill-defined entities such as Generic and Other-ScientificTerm
makes it difficult (see Table 9).

Dataset | $|\mathcal{E}|$ | $|\mathcal{R}|$ | # Train | # Dev | # Test
---|---|---|---|---|---
ACE05 | 7 | 6 | 10,051 | 2,424 | 2,050
CoNLL 04 | 4 | 5 | 922 | 231 | 288
SciERC | 6 | 7 | 1,861 | 275 | 551

Table 4: The statistics of the datasets. We use ACE04, ACE05, SciERC, and CoNLL 04 for evaluating end-to-end relation extraction. | ACE 05 | CoNLL 04 | SciERC
---|---|---|---
Backbone | ALB | ALB | SciB
Optimizer |  | AdamW |
lr backbone | 1e-5 | 3e-5 | 3e-5
lr others | 5e-5 | 5e-5 | 5e-5
Weight Decay | 1e-2 | 1e-4 | 1e-4
Dropout | 0.1 | 0.1 | 0.1
Hidden Size | 768 | 768 | 768
Train steps | 120k | 50k | 20k
Warmup | 5k | 5k | 2k
Batch size | 8 | 8 | 8
Span length | 12 | 12 | 12
Table 5: Hyperparameters. |  | ACE 05 | CoNLL 04 | SciERC
---|---|---|---|---
Model | Backbone | ENT | REL | REL+ | ENT | REL | REL+ | ENT | REL | REL+
DYGIE++ | BB & SciB | 88.6 | 63.4 | – | – | – | – | 67.5 | 48.4 | –
Tab-Seq | ALB | 89.5 | – | 64.3 | 90.1 | 73.8 | 73.6 | – | – | –
PURE | ALB & SciB | 89.7 | 69.0 | 65.6 | – | – | – | 66.6 | 48.2 | 35.6
PFN | ALB & SciB | 89.0 | – | 66.8 | – | – | – | 66.8 | – | 38.4
UniRE | ALB & SciB | 89.9 | – | 66.0 | – | – | – | 68.4 | – | 36.9
TablERT | ALB | 87.8 | 65.0 | 61.8 | 90.5 | 73.2 | 72.2 | – | – | –
UTC-IE | ALB & SciB | 89.9 | – | 67.8 | – | – | – | 69.0 | – | 38.8
UIE | T5 | – | – | 66.6 | – | 75.0 | – | – | – | 36.5
ChatGPT | (Few-shot) | – | 9.04 | – | – | 76.5 | – | – | 17.92 | –
EnriCo | ALB & SciB | 90.1 | 69.1 | 67.6 | 89.8 | 76.6 | 76.6 | 69.3 | 50.5 | 40.2
BL | 88.8 | 67.2 | 64.7 | 89.9 | 73.5 | 73.5 | – | – | –
Table 6: Main results. Entity refers to the F1 score for entity recognition,
REL for relaxed relation extraction, and REL+ for strict relation extraction.
The Backbone column indicates the underlying architecture for each model (ALB
for albert-xxlarge-v1 (Lan et al., 2019), BL for bert-large-cased (Devlin et
al., 2019), and SciB for scibert-base-uncased (Beltagy et al., 2019)).

###  4.3 Evaluation Metrics

For the named entity recognition (NER) task, we use span-level evaluation,
demanding precise entity boundary and type predictions. In evaluating
relations, we employ two metrics: (1) Boundary Evaluation (REL), which
requires correct prediction of entity boundaries and relation types, and (2)
Strict Evaluation (REL+), which also necessitates correct entity type
prediction. We report the micro-averaged F1 score following previous works.

###  4.4 Hyperparameters

In this study, we implemented our model using BERT (Devlin et al., 2019) or
ALBERT (Lan et al., 2019) for the CoNLL 04 and ACE 05 datasets. For the SciERC
dataset, we opted for SciBERT (Beltagy et al., 2019), aligning with previous
works. We detail the hyperparameters in Table 5. Our model was implemented
using PyTorch and trained on a server equipped with A100 GPUs.

###  4.5 Baselines

We primarily compare our model, EnriCo, with comparable approaches from the
literature in terms of model size. DyGIE++ (Wadden et al., 2019) is a model
that uses a pretrained transformer to compute contextualized representations
and employs graph propagation to update the representations of spans for
prediction. PURE (Zhong and Chen, 2021) is a pipeline model for the
information extraction task that learns distinct contextual representations
for entities and relations. PFN (Yan et al., 2021) introduces methods that
model two-way interactions between the task by partitioning and filtering
features. UniRE (Wang et al., 2021) proposes a joint entity and relation
extraction model that uses a unified label space for entity and relation
classification. Tab-Seq (Wang and Lu, 2020) tackles the task of joint
information extraction by treating it as a table-filling problem. Similarly,
in TablERT (Ma et al., 2022), entities and relations are treated as tables,
and the model utilizes two-dimensional CNNs to effectively capture and model
local dependencies within these table-like structures. Finally, UTC-IE (Yan et
al., 2023) treats the task as token-pair classification. It incorporates
Plusformer to facilitate axis-aware interactions through plus-shaped self-
attention and local interactions via Convolutional Neural Networks over token
pairs. We also included evaluations of generative approaches for information
extraction, comprising UIE (Lu et al., 2022), which fine-tunes a T5 model for
information extraction, and ChatGPT (Wadhwa et al., 2023) prompted using few-
shot demonstrations.

##  5 Results and Analysis

###  5.1 Main Results

The main results of our experiments are reported in Table 2. On ACE 05, our
model obtains the highest results in entity evaluation and is second in
relation prediction, slightly under-performing UTC-IE. On CoNLL 04, our model
surpasses the best non-generative baseline by a large margin. Specifically, it
obtains 76.6 on relation evaluation, achieving a +3 F1 improvement compared to
Tab-Seq. Similarly, on SciERC, it also obtains strong results for both
entities and relations, outperforming UTC-IE by 0.3 and 1.4 on entity and
relation F1, respectively. Furthermore, our model also show competitive
performance against generative models, UIE and ChatGPT. On CoNLL 04, ChatGPT
performs quite well due to the simplicity of relations in this dataset.
However, on more complex datasets (SciERC and ACE 05), its performance is far
behind, showing the benefits of fine-tuning task-specific models for the task.
Overall, our model showcases strong performance across all datasets,
demonstrating the utility of our proposed framework.

Dataset | Setting | ENT | REL | REL+
---|---|---|---|---
ACE 05 | Unconstr. | 60.2 | 68.6 | 67.1
| Joint | 90.1 | 68.9 | 67.5
| Ent First | 90.2 | 69.1 | 67.6
| Rel First | 90.0 | 68.7 | 66.9
CoNLL 04 | Unconstr. | 60.6 | 76.3 | 76.1
| Joint | 89.8 | 76.6 | 76.6
| Ent First | 89.8 | 76.7 | 76.7
| Rel First | 89.7 | 76.5 | 76.5
SciERC | Unconstr. | 69.4 | 51.4 | 39.7
| Joint | 69.3 | 50.8 | 40.5
| Ent First | 69.3 | 50.5 | 40.2
| Rel First | 69.4 | 51.1 | 40.7
Table 7: Performance Comparison of Decoding Algorithms. We compare
unconstrained and constrained approaches.

###  5.2 Decoding Algorithms

In Table 7, we report the performance of our model using different decoding
algorithms described in Section 3. We observe that, as expected, unconstrained
decoding is the least competitive, except on SciERC where we did not apply a
domain constraint. In particular, unconstrained decoding performance on entity
recognition can be very poor, especially for ACE 05 and CoNLL 04, where it
falls behind the constrained method by almost 30 points in terms of F1, mainly
due to span boundary and span overlap errors. For relation extraction,
constrained decoding can improve by up to 0.5, 0.6, and 1.0 points in terms of
the F1 score on ACE 05, CoNLL 04, and SciERC, respectively. These results
demonstrate that structural and domain constraints are important not only for
improving coherence but also for performance. Furthermore, we notice that the
performance difference between different constrained decoding methods (Joint,
Entity First, and Relation First) is minimal across datasets. However, Entity
First is the most beneficial one as it can be implemented efficiently without
the need for using an ASP solver, making it up to 3x to 4x faster than other
alternatives (Table 3).

Dataset | Setting | ENT | REL | REL+
---|---|---|---|---
ACE 05 | Full. | 90.1 | 69.1 | 67.6
| No ent. | 89.7 | 67.6 | 65.9
| No rel. | 90.2 | 68.1 | 66.5
| No both | 89.7 | 67.6 | 65.7
CoNLL 04 | Full. | 89.8 | 76.6 | 76.6
| No ent. | 89.7 | 76.1 | 76.1
| No rel. | 89.8 | 76.4 | 76.4
| No both | 89.5 | 75.7 | 75.7
SciERC | Full. | 69.3 | 50.5 | 40.2
| No ent. | 69.8 | 50.4 | 39.8
| No rel. | 69.0 | 50.3 | 39.6
| No both | 68.0 | 50.0 | 38.8
Table 8: Ablation experiment. With and without refine layer at the
entity/relation level.

###  5.3 Refine Layer Ablation

We perform an ablation analysis in Table 8 to assess the effectiveness of the
refine layer, specifically examining the contributions of the entity-level and
relation-level refine layers described in Section 2.5. To ensure a fair
comparison, we maintain a similar number of parameters for all compared
variants. In general, our model with the full configuration—incorporating both
entity and relation level interactions—achieves the most competitive scores
across the datasets. However, removing either the entity or relation level
interaction does not significantly impact performance, whereas removing both
leads to a more substantial drop in performance.

Figure 4: Attention visualization. This illustrates the attention scores of
candidate entities and candidate relations within the input sequence, averaged
across attention heads.

###  5.4 Attention Visualization

In Figure 4, we present the attention visualization of the READ module for
entities and relations, highlighting their interaction with the input
sequence. This visualization depicts the attention scores averaged across all
attention heads. The examples illustrated demonstrate that each span generally
attends most to its corresponding position in the input text. However,
intriguingly, we also observe attention to certain clue words such as “on” and
“using”, which may contribute to type prediction. For relations, attention is
directed to both head and tail spans constituting the relation. However,
contextual information beyond the spans is attended to; for example, the word
“evaluated” receives significant attention from the (“accuracy metric”,
“AlexNet”) relation, indicating the Evaluate-For relation between the two
spans. Similarly, in the same line, the word “trained” is highly attended to
by the (“ImageNet”, “AlexNet”) pair.

##  6 Related Works

#### Joint IE

The field of information extraction (IE) has evolved from traditional pipeline
models, which sequentially handle entity recognition (Chiu and Nichols, 2015;
Lample et al., 2016) and relation extraction (Zelenko et al., 2002; Bach and
Badaskar, 2007; Lin et al., 2016; Wu et al., 2017), to end-to-end models.
These approaches aim to mitigate error propagation (Brin, 1999; Nadeau and
Sekine, 2007) by jointly optimizing entity and relation extraction (Roth and
Yih, 2004; Fu et al., 2019; Sun et al., 2021; Ye et al., 2022), enhancing the
interaction between the two task and overall performance. Proposed approaches
include table-filling methods (Wang and Lu, 2020; Ma et al., 2022), span pair
classification (Eberts and Ulges, 2019; Wadden et al., 2019), set prediction
(Sui et al., 2020), augmented sequence tagging (Ji et al., 2020) and the use
of unified labels for the task (Wang et al., 2021; Yan et al., 2023). In
addition, recently, the usage of generative models (OpenAI, 2023) has become
popular for this task, where input texts are encoded and decoded into
augmented language (Paolini et al., 2021). Some of these approaches conduct
fine-tuning on labeled datasets (Lu et al., 2022; Fei et al., 2022; Zaratiana
et al., 2024), and others prompt large language models such as ChatGPT (Wadhwa
et al., 2023).

#### Higher-order attention

Recent works have proposed higher-order interactions for structured prediction
models. For instance, Floquet et al. (2023) employed span-level attention for
parsing, utilizing linear transformers to circumvent quadratic complexity of
dot-product attention. The work of Zaratiana et al. (2022a) employed span-
level Graph Attention Networks (Velickovic et al., 2017) to enhance span
representations for Named Entity Recognition (NER), using overlap information
as edges. However, their approach is slow and takes huge memory due to the
substantial size of the overlap graph, characterized by numerous nodes and
edges. In our work, we address this challenge by implementing a filtering
mechanism to alleviate computational inefficiencies. Similarly, Ji et al.
(2023) leveraged span-level attention by restricting the number of attended
spans for each span using predefined heuristic. In contrast, our proposed
method dynamically selects them. Additionally, Zhu et al. (2023) utilized
span-to-token attention for Named Entity Recognition (NER). Our model extends
their approach by incorporating both span- and relation-level interaction.

##  7 Conclusion

In summary, this paper introduces EnriCo, a novel model crafted for joint
entity and relation extraction tasks. By integrating span-level and relation-
level attention mechanisms, our model fosters richer representations of spans
and their interactions. The incorporation of a filtering mechanism efficiently
manages computational complexity, while the integration of learned biases and
constraint-based decoding further enhances the precision of model predictions.
Experimental evaluations across benchmark datasets demonstrate the efficacy
and performance of our proposed model.

## Acknowledgments

This work was granted access to the HPC resources of IDRIS under the
allocation 2023-AD011014472 and AD011013682R1 made by GENCI. This work is
partially supported by a public grantoverseen by the French National Research
Agency (ANR) as part of the program Investissements d’Avenir
(ANR-10-LABX-0083).

## References

  * Bach and Badaskar (2007) Nguyen Bach and Sameer Badaskar. 2007.  A review of relation extraction.
  * Beltagy et al. (2019) Iz Beltagy, Kyle Lo, and Arman Cohan. 2019.  Scibert: A pretrained language model for scientific text.  In _Conference on Empirical Methods in Natural Language Processing_.
  * Brewka et al. (2011) Gerhard Brewka, Thomas Eiter, and Miroslaw Truszczynski. 2011.  Answer set programming at a glance.  _Communications of the ACM_ , 54:92 – 103.
  * Brin (1999) Sergey Brin. 1999.  Extracting patterns and relations from the world wide web.  In _The World Wide Web and Databases_ , pages 172–183, Berlin, Heidelberg. Springer Berlin Heidelberg.
  * Carreras and Màrquez (2004) Xavier Carreras and Lluís Màrquez. 2004.  Introduction to the CoNLL-2004 shared task: Semantic role labeling.  In _Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004_ , pages 89–97, Boston, Massachusetts, USA. Association for Computational Linguistics.
  * Chiu and Nichols (2015) Jason P. C. Chiu and Eric Nichols. 2015.  Named entity recognition with bidirectional lstm-cnns.  _Transactions of the Association for Computational Linguistics_ , 4:357–370.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _ArXiv_ , abs/1810.04805.
  * Eberts and Ulges (2019) Markus Eberts and Adrian Ulges. 2019.  Span-based joint entity and relation extraction with transformer pre-training.  _ArXiv_ , abs/1909.07755.
  * Fei et al. (2022) Hao Fei, Shengqiong Wu, Jingye Li, Bobo Li, Fei Li, Libo Qin, Meishan Zhang, Min Zhang, and Tat-Seng Chua. 2022.  LasUIE: Unifying information extraction with latent adaptive structure-aware generative language model.  In _Advances in Neural Information Processing Systems_.
  * Floquet et al. (2023) Nicolas Floquet, Nadi Tomeh, Joseph Le Roux, and Thierry Charnois. 2023.  Attention sur les spans pour l’analyse syntaxique en constituants.  In _Actes de CORIA-TALN 2023. Actes de la 30e Conférence sur le Traitement Automatique des Langues Naturelles (TALN), volume 2 : travaux de recherche originaux – articles courts_ , pages 37–45, Paris, France. ATALA.
  * Fu et al. (2019) Tsu-Jui Fu, Peng-Hsuan Li, and Wei-Yun Ma. 2019.  Graphrel: Modeling text as relational graphs for joint entity and relation extraction.  In _Annual Meeting of the Association for Computational Linguistics_.
  * Gebser et al. (2014) Martin Gebser, Roland Kaminski, Benjamin Kaufmann, and Torsten Schaub. 2014.  Clingo= asp+ control: Preliminary report.  _arXiv preprint arXiv:1405.3694_.
  * Jang et al. (2017) Eric Jang, Shixiang Gu, and Ben Poole. 2017.  Categorical reparameterization with gumbel-softmax.  In _International Conference on Learning Representations_.
  * Ji et al. (2020) Bin Ji, Jie Yu, Shasha Li, Jun Ma, Qingbo Wu, Yusong Tan, and Huijun Liu. 2020.  Span-based joint entity and relation extraction with attention-based span-specific and contextual semantic representations.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 88–99, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  * Ji et al. (2023) Pengyu Ji, Songlin Yang, and Kewei Tu. 2023.  Improving span representation by efficient span-level attention.  In _Findings of the Association for Computational Linguistics: EMNLP 2023_ , pages 11184–11192, Singapore. Association for Computational Linguistics.
  * Lample et al. (2016) Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016.  Neural architectures for named entity recognition.  In _North American Chapter of the Association for Computational Linguistics_.
  * Lan et al. (2019) Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019.  Albert: A lite bert for self-supervised learning of language representations.  _ArXiv_ , abs/1909.11942.
  * Lin et al. (2016) Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2016.  Neural relation extraction with selective attention over instances.  In _Annual Meeting of the Association for Computational Linguistics_.
  * Lin et al. (2020) Ying Lin, Heng Ji, Fei Huang, and Lingfei Wu. 2020.  A joint neural model for information extraction with global features.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7999–8009, Online. Association for Computational Linguistics.
  * Lu et al. (2022) Yaojie Lu, Qing Liu, Dai Dai, Xinyan Xiao, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu. 2022.  Unified structure generation for universal information extraction.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 5755–5772, Dublin, Ireland. Association for Computational Linguistics.
  * Luan et al. (2018) Yi Luan, Luheng He, Mari Ostendorf, and Hannaneh Hajishirzi. 2018.  Multi-task identification of entities, relations, and coreferencefor scientific knowledge graph construction.  In _Proc. Conf. Empirical Methods Natural Language Process. (EMNLP)_.
  * Ma et al. (2022) Youmi Ma, Tatsuya Hiraoka, and Naoaki Okazaki. 2022.  Joint entity and relation extraction based on table labeling using convolutional neural networks.  In _Proceedings of the Sixth Workshop on Structured Prediction for NLP_ , pages 11–21, Dublin, Ireland. Association for Computational Linguistics.
  * Nadeau and Sekine (2007) David Nadeau and Satoshi Sekine. 2007.  A survey of named entity recognition and classification.  _Lingvisticae Investigationes_ , 30:3–26.
  * Nickel et al. (2016) Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. 2016.  A review of relational machine learning for knowledge graphs.  _Proceedings of the IEEE_ , 104(1):11–33.
  * OpenAI (2023) OpenAI. 2023.  Gpt-4 technical report.
  * Paolini et al. (2021) Giovanni Paolini, Ben Athiwaratkun, Jason Krone, Jie Ma, Alessandro Achille, RISHITA ANUBHAI, Cicero Nogueira dos Santos, Bing Xiang, and Stefano Soatto. 2021.  Structured prediction as translation between augmented natural languages.  In _International Conference on Learning Representations_.
  * Roth and Yih (2004) Dan Roth and Wen-tau Yih. 2004.  A linear programming formulation for global inference in natural language tasks.  In _Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004_ , pages 1–8, Boston, Massachusetts, USA. Association for Computational Linguistics.
  * Sui et al. (2020) Dianbo Sui, Yubo Chen, Kang Liu, Jun Zhao, Xiangrong Zeng, and Shengping Liu. 2020.  Joint entity and relation extraction with set prediction networks.  _IEEE transactions on neural networks and learning systems_ , PP.
  * Sun et al. (2021) Kai Sun, Richong Zhang, Samuel Mensah, Yongyi Mao, and Xudong Liu. 2021.  Progressive multi-task learning with controlled information flow for joint entity and relation extraction.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 35(15):13851–13859.
  * Usunier et al. (2009) Nicolas Usunier, David Buffoni, and Patrick Gallinari. 2009.  Ranking with ordered weighted pairwise classification.  In _Proceedings of the 26th Annual International Conference on Machine Learning_ , ICML ’09, page 1057–1064, New York, NY, USA. Association for Computing Machinery.
  * Vaswani et al. (2017) Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _NIPS_.
  * Velickovic et al. (2017) Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio’, and Yoshua Bengio. 2017.  Graph attention networks.  _ArXiv_ , abs/1710.10903.
  * Wadden et al. (2019) David Wadden, Ulme Wennberg, Yi Luan, and Hannaneh Hajishirzi. 2019.  Entity, relation, and event extraction with contextualized span representations.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 5784–5789, Hong Kong, China. Association for Computational Linguistics.
  * Wadhwa et al. (2023) Somin Wadhwa, Silvio Amir, and Byron Wallace. 2023.  Revisiting relation extraction in the era of large language models.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 15566–15589, Toronto, Canada. Association for Computational Linguistics.
  * Walker et al. (2006) Christopher Walker, Stephanie Strassel, Julie Medero, and Kazuaki Maeda. 2006.  Ace 2005 multilingual training corpus.
  * Wang and Lu (2020) Jue Wang and Wei Lu. 2020.  Two are better than one: Joint entity and relation extraction with table-sequence encoders.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1706–1721, Online. Association for Computational Linguistics.
  * Wang et al. (2021) Yijun Wang, Changzhi Sun, Yuanbin Wu, Hao Zhou, Lei Li, and Junchi Yan. 2021.  UniRE: A unified label space for entity relation extraction.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 220–231, Online. Association for Computational Linguistics.
  * Wu et al. (2017) Yi Wu, David Bamman, and Stuart J. Russell. 2017.  Adversarial training for relation extraction.  In _Conference on Empirical Methods in Natural Language Processing_.
  * Yan et al. (2023) Hang Yan, Yu Sun, Xiaonan Li, Yunhua Zhou, Xuanjing Huang, and Xipeng Qiu. 2023.  UTC-IE: A unified token-pair classification architecture for information extraction.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 4096–4122, Toronto, Canada. Association for Computational Linguistics.
  * Yan et al. (2021) Zhiheng Yan, Chong Zhang, Jinlan Fu, Qi Zhang, and Zhongyu Wei. 2021.  A partition filter network for joint entity and relation extraction.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 185–197, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Ye et al. (2022) Deming Ye, Yankai Lin, Peng Li, and Maosong Sun. 2022.  Packed levitated marker for entity and relation extraction.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 4904–4917, Dublin, Ireland. Association for Computational Linguistics.
  * Zaratiana et al. (2022a) Urchade Zaratiana, Nadi Tomeh, Pierre Holat, and Thierry Charnois. 2022a.  GNNer: Reducing overlapping in span-based NER using graph neural networks.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop_ , pages 97–103, Dublin, Ireland. Association for Computational Linguistics.
  * Zaratiana et al. (2022b) Urchade Zaratiana, Nadi Tomeh, Pierre Holat, and Thierry Charnois. 2022b.  Named entity recognition as structured span prediction.  In _Proceedings of the Workshop on Unimodal and Multimodal Induction of Linguistic Structures (UM-IoS)_ , pages 1–10, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  * Zaratiana et al. (2024) Urchade Zaratiana, Nadi Tomeh, Pierre Holat, and Thierry Charnois. 2024.  An autoregressive text-to-graph framework for joint entity and relation extraction.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 38(17):19477–19487.
  * Zelenko et al. (2002) Dmitry Zelenko, Chinatsu Aone, and Anthony Richardella. 2002.  Kernel methods for relation extraction.  In _Journal of machine learning research_.
  * Zhao et al. (2021) Tianyang Zhao, Zhao Yan, Yunbo Cao, and Zhoujun Li. 2021.  A unified multi-task learning framework for joint extraction of entities and relations.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 35(16):14524–14531.
  * Zhong and Chen (2021) Zexuan Zhong and Danqi Chen. 2021.  A frustratingly easy approach for entity and relation extraction.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 50–61, Online. Association for Computational Linguistics.
  * Zhu et al. (2023) Enwei Zhu, Yiyang Liu, and Jinpeng Li. 2023.  Deep span representations for named entity recognition.  In _Findings of the Association for Computational Linguistics: ACL 2023_ , pages 10565–10582, Toronto, Canada. Association for Computational Linguistics.

Dataset | Type | Description
---|---|---
ACE 05 | Entity Types
| PER | Individual people or groups of people
| FAC | Buildings, airports, highways, bridges, etc.
| LOC | Geographical areas and landmasses
| GPE | Countries, cities, states, etc.
| ORG | Companies, governments, institutions, etc.
| WEA | Tools or instruments designed to inflict harm
| VEH | Means of transportation
| Relation Types
| ART | Artifact relationship, indicating ownership or use
| PHYS | Physical proximity or location relationships
| GEN-AFF | General affiliations or relationships
| ORG-AFF | Affiliation with an organization
| PER-SOC | Personal relationships between people
| PART-WHOLE | One entity is a part of another entity
CoNLL 2004 | Entity Types
| Peop | Individuals or groups of people
| Org | Companies, institutions, governments, etc.
| Loc | Geographical entities like cities, countries, etc.
| Relation Types
| Work_For | A person works for an organization
| Live_in | A person lives in a location
| OrgBased_in | An organization is based in a location
| Located_in | One location is situated within another
| Kill | One person is responsible for the death of another
SciERC | Entity Types
| Task | Specific task or problem in a scientific context
| Method | Approach or technique used in research
| Metric | Standard of measurement in experiments
| Material | Tools, datasets, or materials used in research
| Other-ScientificTerm | Other relevant scientific terms
| Generic | General entities relevant in context
| Relation Types
| Used-for | A method/material used for a task
| Feature-of | An entity is a feature of another
| Part-of | One entity is a component of another
| Evaluate-for | Evaluating a method/material/task
| Compare | Comparison between two or more entities
| Conjunction | Entities related in a conjunctive manner
Table 9: Combined Descriptions of Entity and Relation Types in ACE 05, CoNLL
2004, and SciERC datasets. Figure 5: This figure illustrate the highest
scoring relation type for each pairs of entity types. Figure 6: ASP pseudo
code for globally constrained decoding for joint entity and relation
extraction.