# FinMatcher at FinSim-2: Hypernym Detection in the Financial Services Domain
using Knowledge Graphs

Jan Portisch  University of Mannheim, Data and Web Science GroupB 6,
26MannheimGermany43017-6221<EMAIL_ADDRESS>0000-0001-5420-0663
,  Michael Hladik  SAP SE Product Engineering Financial ServicesDietmar-Hopp-
Allee 16WalldorfGermany<EMAIL_ADDRESS>0000-0002-2204-3138 and  Heiko
Paulheim  0000-0001-5420-0663 University of Mannheim, Data and Web Science
Group B 6, 26MannheimGermany43017-6221<EMAIL_ADDRESS>0000-0003-4386-8195

(2021)

###### Abstract.

This paper presents the _FinMatcher_ system and its results for the FinSim
2021 shared task which is co-located with the Workshop on Financial Technology
on the Web (FinWeb) in conjunction with The Web Conference. The FinSim-2
shared task consists of a set of concept labels from the financial services
domain. The goal is to find the most relevant top-level concept from a given
set of concepts. The FinMatcher system exploits three publicly available
knowledge graphs, namely WordNet, Wikidata, and WebIsALOD. The graphs are used
to generate explicit features as well as latent features which are fed into a
neural classifier to predict the closest hypernym.

financial services, knowledge graphs, wikidata, knowledge graph embeddings,
RDF2vec, hypernymy detection

††journalyear: 2021††conference: Companion Proceedings of the Web Conference
2021; April 19–23, 2021; Ljubljana, Slovenia††booktitle: Companion Proceedings
of the Web Conference 2021 (WWW ’21 Companion), April 19–23, 2021, Ljubljana,
Slovenia††doi: 10.1145/3442442.3451382††isbn:
978-1-4503-8313-4/21/04††submissionid: FinW02imS††ccs: Theory of computation
Semantics and reasoning††ccs: Information systems Web mining††ccs: Information
systems Language models

##  1\. Introduction

A _hypernym_ or _hyperonym_ is a concept which is superordinate to another
one. In computer science, it is often represented as an _IS-A_ relationship.
For example, _animal_ is a hypernym of _cat_ and _equity index_ is a hypernym
of _S &P 500 Index_. A _hyponym_ , on the other hand, is a concept which is
subordinate to another one. For example, _cat_ is a hyponym of _animal_ and _S
&P 500 Index_ is a hyponym of _equity index_. (Murphy, 2003) Hypernymy
detection can be broadly applied in real-world applications. The detection of
hypernyms in the financial services domain is particularly interesting due to
a domain specific vocabulary and a lack of publicly available domain-specific
resources and concept representations.

The FinSim task models the hypernym detection task as a multi class
classification problem: Given a concept label (i.e., the hyponym), the correct
hypernym is to be found from a set of 10 mutually exclusive classes (i.e.,
hypernyms). A system participating in this task can return a sorted list of
classes. The task is evaluated with two performance metrics: mean rank and
accuracy.

The FinMatcher system uses two very broad publicly available knowledge graphs
(Wikidata and WebIsALOD) as well as a small linguistic graph resource
(WordNet). A knowledge graph contains real world entities from various domains
and the relationships that hold between them in a graph format (Paulheim,
2017). The system presented in this paper calculates multiple explicit
features and uses RDF2vec embeddings obtained from WebIsALOD. The features are
concatenated into a feature vector which is presented to a neural classifier
which was trained with the provided FinSim training data.

In the following section, related work is introduced. Afterwards, the provided
dataset is quickly described. In Section 4, the FinMatcher system is
presented. The results of FinSim task are given in Section 5 together with an
ablation study. The paper is concluded in Section 6 where future research
directions are also presented.

##  2\. Related Work

###  2.1. Shared Tasks for Hypernym Detection

Hypernym discovery has been addressed before as challenge, for example at
SemEval-2018 (Camacho-Collados et al., 2018). Unique to the FinSim task is the
focus on the financial services industry. The evaluation campaign premiered in
2020 (El Maarouf et al., 2021) and has been extended for the 2021 campaign,
also referred to as FinSim-2 (Mansar et al., 2021): Two additional tags have
been introduced and the training and evaluation datasets have been extended.

###  2.2. Knowledge Graphs

FinMatcher uses three external knowledge graphs as background knowledge for
the task of hypernym detection.

_WordNet_ (Fellbaum, 1998) is a well known lexical resource. It is a database
of English words grouped in sets which represent a particular meaning, called
_synsets_ ; further semantic relations such as hypernymy also exist in the
database. The resource is publicly available.111see
https://wordnet.princeton.edu/download

_Wikidata_ is a knowledge graph hosted by the Wikimedia Foundation which is
publicly available222see https://www.wikidata.org/wiki/Wikidata:Main_Page and
maintained by an open community. The graph contains class-like entities, such
as “stock market index”, and also instance-like entities, such as “MSCI
World”. An example for a Wikidata statement would be _“MSCI World” instance of
“stock market index”_ 333see https://www.wikidata.org/wiki/Q1881843. The graph
can be queried using SPARQL444see https://query.wikidata.org/.

A frequent problem that occurs when working with external background knowledge
in the financial services domain is the fact that less common entities – so
called _long tail entities_ – are not contained within a knowledge base. The
_WebIsA_ (Seitner et al., 2016) database is an attempt to tackle this problem
by providing a dataset which is not based on a single source of knowledge –
like _DBpedia_ (Lehmann et al., 2015) – but instead on the whole Web: The
dataset consists of hypernymy relations extracted from the _Common Crawl_
555see http://commoncrawl.org/, a freely downloadable crawl of a significant
portion of the Web. For the automated extraction, lexico-syntactic patterns
similar to those presented by Hearst (Hearst, 1992) were used. Like Wikidata,
the graph contains class-like and instance-like concepts. A sample triple from
the dataset is “zero-coupon bond” skos:broader “bond”666see
http://webisa.webdatacommons.org/concept/zero-coupon_bond_. The dataset is
also available via a Linked Open Data (LOD) endpoint777see
http://webisa.webdatacommons.org/ under the name _WebIsALOD_ (Hertling and
Paulheim, 2017) – hence, it can be queried like Wikidata using SPARQL.

###  2.3. Knowledge Graph Embeddings

In recent years, latent representations have gained traction not only in
natural language processing but also in other data science communities.
_RDF2vec_ (Ristoski et al., 2019) is a knowledge graph embedding approach
which allows to obtain a latent representation for the elements of a knowledge
graph, i.e. a vector, for each node and each edge in a graph. It applies the
_word2vec_ (Mikolov et al., 2013a; Mikolov et al., 2013b) model to RDF data:
Random walks are performed for each node and are interpreted as sentences.
After the walk generation, the sentences are used as input for the word2vec
algorithm. As a result, one obtains a vector for each word, i.e., a concept in
the RDF graph. Multiple flavors of RDF2vec have been developed in the past
such as biased walks (Cochez et al., 2017) or _RDF2vec Light_ (Portisch et
al., 2020c).888For a good overview of the RDF2vec approach and its
applications, refer to
http://www.rdf2vec.org/ The calculation of knowledge graph embeddings on large
graphs can require a significant amount of resources. Therefore, _KGvec2go_
999see http://www.kgvec2go.org (Portisch et al., 2020b) provides pre-trained
RDF2vec knowledge graph embeddings through a Web API as well as via download.
For the system presented in this paper, a pre-trained embedding of WebIsALOD
has been downloaded from KGvec2go.

Both, RDF2vec and WebIsALOD, have been used for integration tasks in the
financial services domain before (Monych et al., 2020; Portisch, 2018).

##  3\. FinSim Dataset Description

The FinSim dataset consists of 614 hyponym-hypernym pairs. There are 10 class
labels, i.e. hypernyms:

  1. (1)

Equity Index

  2. (2)

Credit Index

  3. (3)

Bonds

  4. (4)

Swap

  5. (5)

Option

  6. (6)

Funds

  7. (7)

Future

  8. (8)

MMIs

  9. (9)

Stocks

  10. (10)

Forward

The class labels presented above classify concepts not according to their
features but instead according to their prototypical kind. The distribution of
class labels is not balanced. As shown in Figure 1, the distribution of labels
follows a power-law with 286 entries for “equity index” and only 9 entries for
“forward”. This is a challenging setting for multiple reasons: (i) the
training dataset is comparatively small, (ii) the hypernyms are semantically
very related, (iii) industry abbreviations are used, and (iv) there are
textual overlaps. The FinSim-2 test dataset consists of 212 entries, the
distribution of class labels is not known.

Figure 1. Distribution of class labels in the FinSim training dataset

Compared to other evaluation campaigns where participants have to submit their
implementations, such as the Ontology Alignment Evaluation Initiative (OAEI),
participants of the FinSim task run their system on their own premises and
submit the predictions made by their system.

##  4\. System Description

The FinMatcher system combines explicit and latent features. In total, there
are five groups of features which will be presented in the following. The
overall architecture is shown in Figure 2.

###  4.1. Features

#### Word Overlap

The overlap between hyponym and class label is a strong signal for a match. An
example would be “Supranational Bond” which is a “Bond”. As such
constellations are relatively frequent in the provided dataset, the first
feature vector encodes whether the label contains the class label. For this
feature minimal text pre-processing is applied including lower-casing and
removal of the plural suffix “s”. As this step is performed for each class
label, a vector of length 10 is obtained. The overlap feature vector is
displayed in green in Figure 2.

#### Wikidata Hypernym Lookup

Wikidata is a large general-purpose knowledge graph which is not tailored to
the financial domain. Nonetheless, the data source contains many financial
concepts and relations between them. For example, the concept “UCITS” can be
linked to“Undertakings for Collective Investment in Transferable Securities”
via the _also known as_ label; due to the annotated relation _subclass of_ ,
it is easily recognizable that “UCITS” is an “investment fund”.101010see
https://www.wikidata.org/wiki/Q25323628 This notion is exploited in this set
of features: A comprehensive linking mechanism from the MELT
framework111111The Matching EvaLuation Toolkit (MELT) is a framework for
ontology and instance matching (development, evaluation, visualization
(Portisch et al., 2020a)). However, components can also be exploited for other
tasks. For a better overview, see https://github.com/dwslab/melt/ (Hertling et
al., 2019, 2020) is used to link classes (the hypernyms) as well as labels
(the hyponyms) to Wikidata concepts and then relations $P31$ (instance of) and
$P279$ (sublcass of) are followed up to two hops to evaluate whether the class
label appears. Distant matches receive a lower signal strength which is
calculated through the inverse hop-distance: A direct hypernym annotation (as
in the UCITS example stated earlier) receives the value $\frac{1}{1}=1$
whereas a two-hop match would receive a value of $\frac{1}{2}=0.5$. As this
step is performed for each class label, a vector of length 10 is obtained. The
Wikidata lookup feature vector is displayed in blue in Figure 2.

#### WordNet Hypernym Lookup

The same exploitation approach chosen for Wikidata is applied on the WordNet
graph: Hypernyms and hyponyms are linked into WordNet and then the inverse
hop-distance is used as feature value. This is done for each class label that
could be linked. The WordNet lookup feature vector is displayed in yellow in
Figure 2.

#### WebIsALOD Hypernym Lookup

In a similar fashion to the Wikidata hypernym lookup, class labels as well as
hyponym labels are linked to the WebIsALOD graph using a linker from the MELT
framework. In this graph, there exists only one significant relation:
_skos:broader_. For each hyponym, the broader concepts are obtained and it is
checked whether the hypernym appears. Due to a high level of noise, the number
of upwards hops is limited to 1. As this step is performed for each class
label, a vector of length 10 is obtained. The WebIsALOD lookup feature vector
is displayed in purple in Figure 2.

#### WebIsALOD RDF2vec Similarity

For the embedding feature, each class label as well as each hyponym label is
linked again into the WebIsALOD knowledge graph. Each concept in WebIsALOD has
an associated embedding vector $v\in I\\!R^{200}$. For comparisons, the cosine
similarity between the hyponym and the class label is calculated.

If the whole concept cannot be linked, multiple sub-concepts are detected and
linked. Within this linking process, longer sub-concepts are favored. For
example, the string “CDX Emerging Markets” cannot be directly linked –
however, the longest substring that can be linked here is “Emerging Markets”;
in addition, “CDX” can also be linked. Comparisons in such cases are performed
as follows:

(1) $\frac{\sum^{I}_{i=0}\max^{J}_{j=0}(sim(v_{i},v_{j})}{|I|}$

where $I$ represents the set of links of the hyponym, $J$ represents the set
of links of the hypernym, $v_{i}$ and $v_{j}$ correspond to the vectors of the
links and $sim$ refers to a similarity function. In this case, the cosine is
used as similarity function. As this step is performed for each class label, a
vector of length 10 is obtained. The WebIsALOD lookup feature vector is
displayed in salmon in Figure 2.

#### Feature Composition

Each of the features $i$ returns a signal vector $s_{i}\in I\\!R^{10}$. All
vectors are concatenated to form the final signal vector
$S=\mathbin{\|}^{5}_{i=1}s_{i}$, which is used as input for the classifier.

###  4.2. Classifier

Due to the small total number of training examples, a very simple artificial
neural network architecture has been chosen. It is configured with one fully
connected layer of size 10 and mean squared error as loss. The network was
trained with 100 epochs and a batch size of 25 on a consumer PC. The vector
that is to be predicted is of size 10 and represents the one-hot-encoded class
label. The neural network classifier performed best among the classifiers
evaluated: Naïve Bayes, J48 decision trees, random forests, and a regression.

As the distribution of class labels is skewed (see Figure 1), we applied the
synthetic minority oversampling technique (SMOTE) (Chawla et al., 2002) to
upsample underrepresented class labels. We experimentally chose 33% of the
majority class total as the upsampling barrier; this means that if the
majority class in the training split totals to 229 records, upsampling for
class labels with less than $\frac{1}{3}*229=76$ records will be performed so
that there are 76 records for the underrepresented class label.

Figure 2. Architecture of FinMatcher

##  5\. Results

###  5.1. Results on the Training Data and Ablation Study

We evaluated our matching system by performing a stratified five-fold cross
validation on the training data. We trained each ANN configuration 10 times
and report the average results for accuracy and mean rank. We further
performed an ablation study by training and evaluating the performance when
leaving out each of the five feature groups. The results can be found in Table
1.

It is visible that the most important feature group in terms of accuracy is
word overlap. This is not surprising given the high number of labels that
contain the hypernym within their name (for example “green bonds”
$\rightarrow$ “bonds”) and shows that it is sensible for the task at hand to
combine explicit and latent features. The observation that the inclusion of
the target label in the term is a significant signal has also been made in the
last FinSim campaign (El Maarouf et al., 2021). The negligible role of WordNet
in terms of accuracy is also comprehensible since this particular external
background knowledge dataset contains merely general-purpose class knowledge
(such as “call option”) but no knowledge about instances (such as “MSCI EMU
Index”). For the FinSim dataset, very large knowledge graphs that contain
class as well as instance knowledge are more beneficial due to their higher
concept coverage. However, the information in the knowledge graphs used also
contain some redundancy, as can be observed in Table 1: leaving out a single
knowledge graph does not significantly change the results.

Table 1. Ablation Study Left-out Vector Group | Mean Accuracy | Mean Rank (HITS@10)
---|---|---
Submission | 86.69 | 1.432
No SMOTE | 85.55 | 1.371
Word Overlap | 60.51 | 2.007
Wikidata Hypernyms | 85.50 | 1.490
WordNet Hypernyms | 86.66 | 1.440
ALOD Hypernyms | 85.88 | 1.361
ALOD RDF2vec | 85.56 | 1.481

To further analyze the contribution of the different signals, we plotted the
weights of the input features. As the weight of each input neuron $s_{i}$
relates to label $i$, we can directly observe which features the trained model
considers relevant to identify which label.

Table 2 shows the summed absolute weight per feature group. This allows to
analyze the overall contribution of the individual feature group. Here, it is
visible that the latent RDF2vec feature group receives the highest weight –
higher than the word overlap group. While the word overlap feature is
important for the majority labels (equity index, credit index), it is not
equally important for all labels and does not have the overall highest weight:
Figure 3 shows the summed absolute weight per feature group and class label.
The class labels are sorted in descending order by frequency. Here, it is
visible that the word overlap has the highest contribution for the equity
index as well as a high contribution for the credit index but low weights for
the remaining minority classes.

Table 2. Absolute Weights per Feature Group Vector Group | Weight
---|---
Word Overlap | 13.64
Wikidata Hypernyms | 13.01
WordNet Hypernyms | 13.35
ALOD Hypernyms | 9.12
ALOD RDF2vec | 14.01
Figure 3. Heatmap of the Absolute Weights per Feature Group and Class Label

###  5.2. Results on the Reference Data

FinMatcher participated only with one configuration and achieved an accuracy
of 81.1% and a mean rank of 1.415 on the reference data below the expected
scores from the training data shown in Table 1.

##  6\. Conclusion

In this paper, we presented _FinMatcher_ , a hypernym detection system for the
financial services domain which exploits multiple knowledge graphs by
combining explicit and latent features. We could show that the task can be
addressed by including external knowledge in the form of knowledge graphs and
that the combination of multiple graphs is overall beneficial.

In the future, we strive to improve the results through the inclusion of more
advanced embedding techniques as well as the exploration of additional
external datasets.

###### Acknowledgements.

We would like to thank the FinSim-2 organizers (Youness Mansar, Ismaïl El
Maarouf, and Juyeon Kang) for compiling the data, conducting the evaluation
campaign, and for promptly answering all questions.

## References

  * (1)
  * Camacho-Collados et al. (2018) José Camacho-Collados, Claudio Delli Bovi, Luis Espinosa Anke, Sergio Oramas, Tommaso Pasini, Enrico Santus, Vered Shwartz, Roberto Navigli, and Horacio Saggion. 2018\.  SemEval-2018 Task 9: Hypernym Discovery. In _Proceedings of The 12th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2018, New Orleans, Louisiana, USA, June 5-6, 2018_ , Marianna Apidianaki, Saif M. Mohammad, Jonathan May, Ekaterina Shutova, Steven Bethard, and Marine Carpuat (Eds.). Association for Computational Linguistics, 712–724.  https://doi.org/10.18653/v1/s18-1115
  * Chawla et al. (2002) Nitesh V. Chawla, Kevin W. Bowyer, Lawrence O. Hall, and W. Philip Kegelmeyer. 2002.  SMOTE: Synthetic Minority Over-sampling Technique.  _J. Artif. Intell. Res._ 16 (2002), 321–357.  https://doi.org/10.1613/jair.953
  * Cochez et al. (2017) Michael Cochez, Petar Ristoski, Simone Paolo Ponzetto, and Heiko Paulheim. 2017.  Biased graph walks for RDF graph embeddings. In _Proceedings of the 7th International Conference on Web Intelligence, Mining and Semantics, WIMS 2017, Amantea, Italy, June 19-22, 2017_ , Rajendra Akerkar, Alfredo Cuzzocrea, Jannong Cao, and Mohand-Said Hacid (Eds.). ACM, 21:1–21:12.  https://doi.org/10.1145/3102254.3102279
  * El Maarouf et al. (2021) Ismail El Maarouf, Youness Mansar, Virginie Mouilleron, and Dialekti Valsamou-Stanislawski. 2021\.  The finsim 2020 shared task: Learning semantic representations for the financial domain. In _Proceedings of the Second Workshop on Financial Technology and Natural Language Processing_. 81–86.
  * Fellbaum (1998) Christiane Fellbaum (Ed.). 1998\.  _WordNet: An Electronic Lexical Database_.  MIT Press, Cambridge, Massachusetts.
  * Hearst (1992) Marti A. Hearst. 1992\.  Automatic Acquisition of Hyponyms from Large Text Corpora. In _14th International Conference on Computational Linguistics, COLING 1992, Nantes, France, August 23-28, 1992_. 539–545.  https://www.aclweb.org/anthology/C92-2082/
  * Hertling and Paulheim (2017) Sven Hertling and Heiko Paulheim. 2017.  WebIsALOD: Providing Hypernymy Relations Extracted from the Web as Linked Open Data. In _The Semantic Web - ISWC 2017 - 16th International Semantic Web Conference, Vienna, Austria, October 21-25, 2017, Proceedings, Part II_ _(Lecture Notes in Computer Science, Vol. 10588)_ , Claudia d’Amato, Miriam Fernández, Valentina A. M. Tamma, Freddy Lécué, Philippe Cudré-Mauroux, Juan F. Sequeda, Christoph Lange, and Jeff Heflin (Eds.). Springer, 111–119.  https://doi.org/10.1007/978-3-319-68204-4_11
  * Hertling et al. (2019) Sven Hertling, Jan Portisch, and Heiko Paulheim. 2019\.  MELT - Matching EvaLuation Toolkit. In _Semantic Systems. The Power of AI and Knowledge Graphs - 15th International Conference, SEMANTiCS 2019, Karlsruhe, Germany, September 9-12, 2019, Proceedings_ _(Lecture Notes in Computer Science, Vol. 11702)_ , Maribel Acosta, Philippe Cudré-Mauroux, Maria Maleshkova, Tassilo Pellegrini, Harald Sack, and York Sure-Vetter (Eds.). Springer, 231–245.  https://doi.org/10.1007/978-3-030-33220-4_17
  * Hertling et al. (2020) Sven Hertling, Jan Portisch, and Heiko Paulheim. 2020\.  Supervised ontology and instance matching with MELT. In _Proceedings of the 15th International Workshop on Ontology Matching co-located with the 19th International Semantic Web Conference (ISWC 2020), Virtual conference (originally planned to be in Athens, Greece), November 2, 2020_ _(CEUR Workshop Proceedings, Vol. 2788)_ , Pavel Shvaiko, Jérôme Euzenat, Ernesto Jiménez-Ruiz, Oktie Hassanzadeh, and Cássia Trojahn (Eds.). CEUR-WS.org, 60–71.  http://ceur-ws.org/Vol-2788/om2020_LTpaper6.pdf
  * Lehmann et al. (2015) Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N. Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick van Kleef, Sören Auer, and Christian Bizer. 2015.  DBpedia - A large-scale, multilingual knowledge base extracted from Wikipedia.  _Semantic Web_ 6, 2 (2015), 167–195.  https://doi.org/10.3233/SW-140134
  * Mansar et al. (2021) Youness Mansar, Juyeon Kang, and Ismaïl El Maarouf. 2021\.  FinSim-2 2021 Shared Task: Learning Semantic Similarities for the Financial Domain. In _Proceedings of The Web Conference 2021, Virtual Edition_.
  * Mikolov et al. (2013a) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a.  Efficient Estimation of Word Representations in Vector Space. In _1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, May 2-4, 2013, Workshop Track Proceedings_ , Yoshua Bengio and Yann LeCun (Eds.).  http://arxiv.org/abs/1301.3781
  * Mikolov et al. (2013b) Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013b.  Distributed Representations of Words and Phrases and their Compositionality. In _Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States_ , Christopher J. C. Burges, Léon Bottou, Zoubin Ghahramani, and Kilian Q. Weinberger (Eds.). 3111–3119.  http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality
  * Monych et al. (2020) Michael Monych, Jan Portisch, Michael Hladik, and Heiko Paulheim. 2020.  DESKMatcher. In _Proceedings of the 15th International Workshop on Ontology Matching co-located with the 19th International Semantic Web Conference (ISWC 2020), Virtual conference (originally planned to be in Athens, Greece), November 2, 2020_ _(CEUR Workshop Proceedings, Vol. 2788)_ , Pavel Shvaiko, Jérôme Euzenat, Ernesto Jiménez-Ruiz, Oktie Hassanzadeh, and Cássia Trojahn (Eds.). CEUR-WS.org, 181–186.  http://ceur-ws.org/Vol-2788/oaei20_paper7.pdf
  * Murphy (2003) Lynne Murphy. 2003\.  _Semantic relations and the lexicon : antonymy, synonymy, and other paradigms_.  Cambridge University Press, Cambridge, UK New York. 2016–236 pages.
  * Paulheim (2017) Heiko Paulheim. 2017\.  Knowledge graph refinement: A survey of approaches and evaluation methods.  _Semantic Web_ 8, 3 (2017), 489–508.  https://doi.org/10.3233/SW-160218
  * Portisch (2018) Jan Portisch. 2018\.  Automatic schema matching utilizing hypernymy relations extracted from the web.
  * Portisch et al. (2020a) Jan Portisch, Sven Hertling, and Heiko Paulheim. 2020a.  Visual Analysis of Ontology Matching Results with the MELT Dashboard. In _The Semantic Web: ESWC 2020 Satellite Events - ESWC 2020 Satellite Events, Heraklion, Crete, Greece, May 31 - June 4, 2020, Revised Selected Papers_ _(Lecture Notes in Computer Science, Vol. 12124)_ , Andreas Harth, Valentina Presutti, Raphaël Troncy, Maribel Acosta, Axel Polleres, Javier D. Fernández, Josiane Xavier Parreira, Olaf Hartig, Katja Hose, and Michael Cochez (Eds.). Springer, 186–190.  https://doi.org/10.1007/978-3-030-62327-2_32
  * Portisch et al. (2020b) Jan Portisch, Michael Hladik, and Heiko Paulheim. 2020b.  KGvec2go - Knowledge Graph Embeddings as a Service. In _Proceedings of The 12th Language Resources and Evaluation Conference, LREC 2020, Marseille, France, May 11-16, 2020_ , Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asunción Moreno, Jan Odijk, and Stelios Piperidis (Eds.). European Language Resources Association, 5641–5647.  https://www.aclweb.org/anthology/2020.lrec-1.692/
  * Portisch et al. (2020c) Jan Portisch, Michael Hladik, and Heiko Paulheim. 2020c.  RDF2Vec Light - A Lightweight Approachfor Knowledge Graph Embeddings. In _Proceedings of the ISWC 2020 Demos and Industry Tracks: From Novel Ideas to Industrial Practice co-located with 19th International Semantic Web Conference (ISWC 2020), Globally online, November 1-6, 2020 (UTC)_ _(CEUR Workshop Proceedings, Vol. 2721)_ , Kerry L. Taylor, Rafael S. Gonçalves, Freddy Lécué, and Jun Yan (Eds.). CEUR-WS.org, 79–84.  http://ceur-ws.org/Vol-2721/paper520.pdf
  * Ristoski et al. (2019) Petar Ristoski, Jessica Rosati, Tommaso Di Noia, Renato De Leone, and Heiko Paulheim. 2019\.  RDF2Vec: RDF graph embeddings and their applications.  _Semantic Web_ 10, 4 (2019), 721–752.  https://doi.org/10.3233/SW-180317
  * Seitner et al. (2016) Julian Seitner, Christian Bizer, Kai Eckert, Stefano Faralli, Robert Meusel, Heiko Paulheim, and Simone Paolo Ponzetto. 2016.  A Large DataBase of Hypernymy Relations Extracted from the Web. In _Proceedings of the Tenth International Conference on Language Resources and Evaluation LREC 2016, Portorož, Slovenia, May 23-28, 2016_ , Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asunción Moreno, Jan Odijk, and Stelios Piperidis (Eds.). European Language Resources Association (ELRA).  http://www.lrec-conf.org/proceedings/lrec2016/summaries/204.html