# Simulating the aftermath of
Northern European Enclosure Dam (NEED) break
and flooding of European coast

Paweł Maczuga1, Marcin Łoś1, Eirik Valseth2,3,4, Albert Oliver Serra5,
Leszek Siwik1, Elisabede Alberdi Celaya6, Anna Paszyńska7, Maciej Paszyński1

(1 AGH University of Krakow, Poland
2 Norwegian University of Life Sciences, Norway
3 The Oden Institute for Computational Engineering and Sciences
at The University of Texas at Austin, Texas, USA
4 Simula Research Laboratory, Norway
5 University of Las Palmas de Gran Canaria, Spain
6 The University of the Basque Country, Bilbao, Spain
7 Jagiellonian University, Kraków, Poland )

###### Abstract

The Northern European Enclosure Dam (NEED) is a hypothetical project to
prevent flooding in European countries following the rising ocean level due to
melting arctic glaciers. This project involves the construction of two large
dams between Scotland and Norway, as well as England and France. The
anticipated cost of this project is 250 to 500 billion euros. In this paper,
we present the simulation of the aftermath of flooding on the European
coastline caused by a catastrophic break of this hypothetical dam. From our
simulation results, we can observe that there is a traveling wave after the
accident, with a velocity of around 10 kilometers per hour, raising the sea
level permanently inside the dammed region. This observation implies a need to
construct additional dams or barriers protecting the northern coastline of the
Netherlands and the interior of the Baltic Sea. Our simulations have been
obtained using the following building blocks. First, a graph transformation
model was applied to generate an adaptive mesh approximating the topography of
the Earth. We employ the composition graph grammar model for breaking
triangular elements in the mesh without the generation of hanging nodes.
Second, the wave equation is formulated in a spherical latitude-longitude
system of coordinates and solved by a high-order time integration scheme using
the generalized $\alpha$ method.

##  1 Introduction

Rising ocean levels caused by climate change and melting Arctic glaciers have
worried European environmentalists to the point that a plan has been made to
build a large dam to separate the oceans from the North Sea and Baltic Sea
basins. The Northern European Enclosure Dam (NEED) project [1, 2] contemplates
building two huge dams, the first between Scotland and Norway and the second
between England and France. This would require the construction of four
sections of dams in the north and one section in southern England. The first
dam, 331 kilometers long respectively, would connect the west coast of Norway
with the Scottish island of Shetland. Another section between the Isle of
Shetland and Scotland with a total length of 145 kilometers, and another
covering the Orkney islands. In addition, a dam would need to be built in the
south of England between England’s Cornwall and Bretagne in France, with a
length of 161 kilometers.

The water depth at the dam sites reaches 110 meters at its deepest points.
Technical details of the construction of the NEED dam are under consideration,
but nevertheless, preliminary estimates have put the cost of building such a
dam at an expense of 250-500 billion euros. Despite such an extreme cost, the
project is being considered very seriously since the Arctic glaciers are
melting [1]. Since 1880, the average sea level has risen by 21 centimeters
[4], and the predicted average increase in ocean levels is 2.3 meters for
every one degree Celsius increase in average temperature [5]. The result, as
noted by [3], is an inevitable rise in sea levels in the range of 5-11 meters
over the next centuries. The bleakest scenarios assume a more than 10-meter
rise in global average ocean levels by the year 2500 [6, 7]. More cautious
scenarios described in [8, 9, 10] predict a 1-2 meter rise in average ocean
levels by 2100. A recent paper [28] shows that the West Antarctic Ice Sheet
encountered rapid melting 8000 years ago, which could cause an increase of the
average ocean level up to 4 meters in 200 years.

We focus on a hypothetical scenario where we assume that the Northern Sea and
Atlantic Ocean levels rise by 6 meters and the Northern and Baltic Seas are
protected by the NEED. This is the starting point of our consideration. At
this moment, we assume a massive dam break. There may be several potential
causes of such a scenario, including natural causes like an earthquake or
asteroid strike, as well as material fatigue or design flaws. Another
theoretically possible scenario is a terrorist attack against the dam.

To perform our simulation, we employ the following tools:

  * •

A computational mesh is generated using a novel composition graph grammar
model described in Section 2. The topography of the Earth and the seabed is
based on the Global Multi-Resolution Topography Data Synthesis database [25].

  * •

A simulation that employs the wave equations summarized in Section 3
discretized in space using finite element method over the entire Earth mesh.

  * •

A time integration scheme which employs a high-order generalized $\alpha$
method described in Section 4.

The simulation of the NEED dam break using all the above tools is summarized
in Section 5. In Section 6 we also discuss the parallel scalability of our
implementation. The paper is concluded in Section 7.

##  2 The first building block, a simple composition graph grammar model for
breaking triangular elements in the mesh without generation of hanging nodes

The first building block for our simulator is the composition graph grammar
model. It models Rivara’s algorithm [20, 21] for breaking triangular elements
in a way that does not generate hanging nodes. All the triangles of the mesh
are represented by the composition graph grammar [13]. The way we break the
triangular elements is described by some rules that transform the composition
graph. These rules are defined following the idea of the composition graph
grammar proposed by Ewa Grabska [14, 15]. The formal definition of the
composition graph grammar can be found in [13].

The Rivara longest-edge refinement algorithm is summarized in Figure 2. The
main assumption of the Rivara algorithm is that we can only break the longest
edges of elements. This prevents the creation of elongated elements that may
cause numerical problems. We start with four triangular finite element meshes.
We want to break the first triangle; see panel (a) in Figure 2. We mark its
longest edge by the red dot; see panel (a) in Figure 2. The general condition
says that _we can break the marked edge only if two neighboring triangles
share it, and for both triangles, this edge is the longest. Another option is
when this edge is located on the boundary._ This is not the case here since
the second element has another longest edge. Thus, we mark the other longest
edge of the second element, which we denote again by the red dot, see panel
(a) in Figure 2. This procedure is repeated; see panel (b) in Figure 2 until
we end up with the edge located on the side of the domain. This time it can be
broken. We break the last element; see panel (b) in Figure 2. After this
operation, the previous red dot is located on the edge that fulfills the
breaking condition; see panel (c) in Figure 2. We can break this edge and the
two adjacent elements; see panel (c) in Figure 2. The same happens to the
second edge denoted by the red dot in panel (c) in Figure 2. We can break this
edge. Finally, the first edge denoted by the red dot also fulfills the
breaking condition; see panel (c) in Figure 2. We can break the two adjacent
elements, including the first element that was our original target, see panel
(c) in Figure 2.

The composition graph grammars have been successfully used to model mesh
refinements with triangular or rectangular elements [16, 17, 18, 19]. The
definition of the Rivara algorithm by another kind of transformative rules
called the hypergraph grammar, has already been described in [30]. There are
the following differences between the previous model described in [30] and the
new model presented in this paper:

  * •

The previous model employs the hypergraph grammar model, while the new model
uses the composition graph grammar model.

  * •

The previous model needs six hypergraph grammar productions presented in
Figures 5-10 in [30], while the new model needs only four composition graph
grammar productions presented in Figures 3-7.

  * •

The six productions from the hypergraph grammar model described in [30],
presented in Figures 5-10 there, have complex left-hand sides, which are very
costly to implement. The subgraphs of the left-hand sides of these productions
need to be identified as subgraphs of the big hypergraph representing the
entire computational mesh there. They consist of triangular elements with
between 3 to 5 vertices and several hyperedges. The four productions of our
new composition graph model presented in Figures 3-7 have only one or two
vertices, so their identification is straightforward.

Summing up, our new model presented in Figure 3-7 is simpler and cheaper to
implement than our previous model described in [30].

The Rivara algorithm has also been expressed by graph-grammar in [23]. There
are the following differences between [23] and this paper:

  * •

In [23], the graph-grammar-based mesh generation has been performed in the
spherical system of coordinates, but the PDE has been formulated in the
Cartesian system of coordinates, and the simulation was performed only on
small part of the domain. In our new paper, the PDEs are formulated in the
spherical system of coordinates, and the simulation is performed on the entire
mesh.

  * •

The graph grammar model presented in [23] expressed the Rivara algorithm by
two productions, with four connected vertices and six connected vertices on
the left-hand sides. These left-hand sides of the productions are still
expensive to identify. In this new paper, we introduce graph-grammar
productions that have only one or two vertices on the left-hand sides, and
they are easy to identify to execute graph grammar productions.

  * •

The computations performed in [23] were sequential, while the computations
presented in this paper are parallel.

Let us now introduce the new composition graph grammar model for breaking two-
dimensional triangular elements without the creation of the hanging nodes.
Figure 1 presents an exemplary two-element mesh and the corresponding CP-
graph. A node represents a triangular element. Each node has label $T$ and
attributes $C1,C2,C3$ the coordinates of the triangle vertices. Each node has
three bonds representing the edges of a triangle. Each edge has three
attributes: attribute $L$ \- the length of the edge; attribute $B$, which
equals $1$ if the edge is boundary edge and ”0” in the other case; and
attribute $BR$, which equals $1$ if the edge should be broken and ”0” in the
other case.

Figure 1: An exemplary two-element mesh and the corresponding CP-graph.

The graph grammar modeling the longest edge Rivara algorithm consists of four
productions:

  * •

Production (graph transformation) $GT1$ (Figure 3) which marks for refinement
the longest edge of an element that should be refined (element for which the
refinement criteria ($RC$) is fulfilled). If an element has more than one
longest edge, the boundary edge will be chosen. The value of the attribute
$BR$ for the longest edge is set to $1$.

  * •

Production (graph transformation) $GT2$ (Figure 4) which propagates the
refinement - marks for refinement of the longest edge of the neighboring
element. If the neighboring element has more than one longest edge, the
boundary edge will be chosen. The value of the attribute $BR$ for the longest
edge is set to $1$.

  * •

Production (graph transformation) $GT3$ (Figure 5), which propagates the
refinement in the case when the neighboring element’s longest edge is the
common edge and performs refinement of both elements to the longest common
edge. This production replaces the CP-graph representing two big neighboring
elements with a CP-graph representing four smaller elements. Figure 6 presents
changes in the two-element mesh when we perform refinement (graph
transformation $GT3$). The function $Calculate1(C1,C2,C3)$ and
$Calculate2(C1,C2,C3)$ calculate the coordinates of vertices of the newly
created first and second small triangle, respectively, on the base of
coordinates of vertices of the big triangle. The function
$CalculateDistance(C1,C2,C3)$ calculates the length of the newly created
common edge.

  * •

Production (graph transformation) $GT4$, presented in Figure 7, performs
refinement of the longest edge, which is the boundary edge. This production
replaces the CP-graph representing one big boundary element with a boundary
edge marked for refinement by a CP-graph representing two smaller elements.
The function $Calculate1(C1,C2,C3)$ and $Calculate2(C1,C2,C3)$ calculate the
coordinates of vertices of the newly created first and second small triangle,
respectively, on the base of coordinates of vertices of the big triangle.

Figure 2: Rivara algorithm of refinement, step by step. Figure 3: Production
$GT1$, marking element for breaking. Figure 4: Production $GT2$, propagation
of refinement marker. Figure 5: Production $GT3$, breaking of common longest
edge. Figure 6: Breaking of two neighboring elements. Figure 7: Production
$GT4$, breaking of boundary edge.

Figure 8 presents derivation modeling the process of Rivara refinement
algorithm as presented in Figure 9.

Figure 8: An exemplary derivation modeling the process of Rivara algorithm as
presented in Figure 9. Figure 9: The process of Rivara refinement algorithm
for the simple two element mesh in the case the left element should be
refined.

##  3 The second building block modeling the dam break with the wave equations

The second building block for our simulations are the wave equations. A
comprehensive review of modeling tsunami waves is described in [26]. Seawater
flow is often modeled using shallow water theory and leads to a set of
transient and nonlinear PDEs, i.e., the shallow water equations (SWE) [24].
The solution of the SWE for physically relevant flows requires significant
resources due to its nonlinear nature and multiple unknowns. Multiple
surrogates exist that are valid in certain limited cases and admit accurate
solutions within their region of validity. In this case of tsunami modeling,
we will focus on the case in which the wave propagation is governed by long
wave theory.

The approach we use has been studied extensively and appears in, e.g., [11,
22, 23]. Starting with nonlinear shallow water theory and performing modeling
assumptions (see [26]), the SWE can be reduced to the following wave equation:

$\displaystyle\frac{\partial^{2}u}{\partial
t^{2}}-\nabla\left(g(u-h_{b})\nabla u\right)=0\text{ in }\,\Omega.$ (1)

In this formula, the unknown scalar field $u$ represents the water level,
thus, $u-h_{b}$ is the water depth relative to the seabed denoted by the
$h_{b}$. The acceleration due to gravity is denoted by
$g=9.81\frac{m}{s^{2}}$. The computations are performed on the model of the
whole Earth, where the North Pole and South Pole are modeled as the zero
Neumann baoundary conditions. Here, we have assumed that the wave speed is the
shallow water wave celerity, i.e., $c^{2}=g(u-h_{b})$. Note that (1) is the
standard representation of the nonlinear wave equation in $x,y,z$ coordinates.
As the computational domain of consideration, we include the entire Earth. In
our graph grammar-based mesh generation procedure, described in Section 2, we
produce finite element meshes that are defined using spherical coordinates
with a fixed radius $R$. Hence, classical longitude-latitude coordinates
$(\lambda,\psi)$. The reference water level corresponds to the sphere. The
wave equation seeks a water level $(\lambda,\psi)\rightarrow
u(\lambda,\psi)\in{\cal R}$. The sea floor is defined by
$(\lambda,\psi)\rightarrow h_{b}(\lambda,\psi)\in{\cal R}$. The three-
dimensional mesh obtained by the graph-grammar model approximates the
bathymetry, topography, and coastline based on the data provided by [25]. The
generated computational mesh for the whole Earth is presented in Figures
10-12.

Figure 10: Generation of the topography of the Earth (1/3) with composition
graph grammar expressing the Rivara method.

Figure 11: Generation of the topography of the Earth (2/3) with composition
graph grammar expressing the Rivara method.

Figure 12: Generation of the topography of the Earth (3/3) with composition
graph grammar expressing the Rivara method.

To accomplish this, we use standard transformations to this reference frame
using the following transformations to a longitude-latitude coordinate system,
i.e.,

$\displaystyle x$
$\displaystyle=(\lambda-\lambda_{0})\,R\,\text{cos}\,\phi_{0},$ (2)
$\displaystyle y$ $\displaystyle=\,R\,\phi.$

Here $(\lambda_{0},\phi_{0})$ are coordinates of a reference point. Using the
chain rule, we can perform a substitution and coordinate shift such that:

$\nabla\left(g(u-h_{b})\nabla
u\right)=\nabla\\{g(u-h_{b})(R\text{cos}\phi_{0})^{-1}\frac{\partial
u}{\partial\lambda},g(u-h_{b})R^{-1}\frac{\partial
u}{\partial\phi}\\}^{\text{T}},$ (3)

Thus:

$\displaystyle\frac{\partial^{2}u}{\partial
t^{2}}-R^{2}\left(\frac{1}{\text{cos}^{2}\phi_{0}}\frac{\partial}{\partial\lambda}\\{g(u-h_{b})\frac{\partial
u}{\partial\lambda}\\}\right)-R^{2}\frac{\partial}{\partial\phi}\left(g(u-h_{b})\frac{\partial
u}{\partial\phi}\right)=0\text{ in }\,\Omega.$ (4)

##  4 Numerical scheme

To develop numerical simulations using (4), we discretize in space using the
Finite Element method and use the generalized alpha scheme [27] to perform
time stepping. After integration by parts and application of boundary
conditions, we can write the equivalent weak formulation of (4) as:

$\left({\frac{\partial^{2}u}{\partial t^{2}}},{v}\right)+b(u,v)=0\quad\forall
v\in V,$ (5)

for a bilinear form $b(\cdot,\cdot)$ and trial space $V$:

$\displaystyle b(u,v)=R^{2}g\left((u-h_{b})\frac{\partial
u}{\partial\phi},\frac{\partial
v}{\partial\phi}\right)-R^{2}g\frac{1}{\text{cos}^{2}\phi_{0}}\left((u-h_{b})\frac{\partial
u}{\partial\lambda},\frac{\partial v}{\partial\lambda}\right)\quad\forall v\in
V,$ (6)

Since we solve the whole Earth, there are no boundary conditions. To alleviate
encountered stability issues, we also introduce a non-zero dampening term of
the form $c\frac{\partial u}{\partial t}$, where

$c=c_{0}\frac{h_{0}}{\min\\{h_{0},h_{b}\\}},$ (7)

In deep regions of the ocean ($h_{b}\ll h_{0}$) we have $c\approx 0$, while in
the shallow waters ($h_{b}\approx h_{0}$) we have $c\approx c_{0}$, meaning
that the damping effect is only meaningful near the shores, where we
encountered numerical instabilities. In the numerical experiments presented in
the paper, we used $c_{0}=5\times 10^{-4}$ and $h_{0}=-5$.

Applying the Bubnov-Galerkin method results in a semi-discrete formulation,

$M\ddot{\mathbf{u}}+C\dot{\mathbf{u}}+B\mathbf{u}=0.$ (8)

The first term involves the mass matrix, the second term involves the damping
matrix, and the last term involves the matrix of our bilinear form. The
solution vector $\mathbf{u}$ contains the coefficients of the discrete
solution $u_{h}$. We use the generalized-$\alpha$ scheme with time step size
$\tau$. We introduce the acceleration $\mathbf{a}=\ddot{\mathbf{u}}$ as well
as we introduce the velocity $\mathbf{v}=\dot{\mathbf{u}}$. Using this
notation, we can rewrite our problem (8) into

$M\mathbf{a}+C\mathbf{v}+B\mathbf{u}=0,$ (9)

The generalized-$\alpha$ method parameterized with $\alpha_{1}$, $\alpha_{2}$,
$\beta$, $\gamma$ involves the following sequence of steps

$M\mathbf{a}_{n+1-\alpha_{1}}+C\mathbf{v}_{n+1-\alpha_{2}}+B\mathbf{u}_{n+1-\alpha_{f}}=0,\\\
$ (10)

where the symbols with fractional indices are defined as

$\displaystyle\mathbf{a}_{n+1-\alpha_{1}}$
$\displaystyle=(1-\alpha_{1})\mathbf{a}_{n+1}+\alpha_{m}\mathbf{a}_{n},$ (11)
$\displaystyle\mathbf{v}_{n+1-\alpha_{2}}$
$\displaystyle=(1-\alpha_{2})\mathbf{v}_{n+1}+\alpha_{f}\mathbf{v}_{n},$
$\displaystyle\mathbf{u}_{n+1-\alpha_{2}}$
$\displaystyle=(1-\alpha_{2})\mathbf{u}_{n+1}+\alpha_{f}\mathbf{u}_{n},$

and new values of $\mathbf{v}$, $\mathbf{u}$ are given by

$\displaystyle\mathbf{v}_{n+1}$
$\displaystyle=\mathbf{v}_{n}+\tau\left[(1-\gamma)\mathbf{a}_{n}+\gamma\mathbf{a}_{n+1}\right],$
(12) $\displaystyle\mathbf{u}_{n+1}$
$\displaystyle=\mathbf{u}_{n}+\tau\mathbf{v}_{n}+\frac{\tau^{2}}{2}\left[(1-2\beta)\mathbf{a}_{n}+\beta\mathbf{a}_{n+1}\right].$

In the numerical tests, their values are

$\displaystyle\alpha_{1}$ $\displaystyle=\frac{2\rho-1}{\rho+1},$ (13)
$\displaystyle\alpha_{2}$ $\displaystyle=\frac{\rho}{\rho+1},$
$\displaystyle\beta$ $\displaystyle=\frac{1}{4}(1-\alpha_{1}-\alpha_{2})^{2},$
$\displaystyle\gamma$ $\displaystyle=\frac{1}{2}-\alpha_{1}-\alpha_{2},$

where $\rho=0.2$. Such the setup of the method parameters makes sure that this
time integration scheme is of the second order [27].

##  5 Simulation of a tsunami caused by dam break

We present a numerical simulation of the aftermath of the hypothetical NEED
failure, caused by either natural causes, structural failure, or terrorist
attack. While all these scenarios could cause slightly different initial
conditions, we assume that in our initial configuration, the NEED has
disappeared, including the Northern and Southern parts, and we perform a
simulation of the aftermath.

The simulation is performed on a model of the entire Earth, presented in
Figure 13, generated using composition graph grammar adapting the coastline
and the seabed to the Global Multi-Resolution Topography Data Synthesis
database [25]. We employ a spherical coordinate system since our numerical
model describes the ocean water level with respect to a reference level which
is shown in blue on the presented sphere. In our further pictures, we focus on
the part of the adaptive mesh describing the Northern Sea, Baltic Sea, and the
Northern European Enclosure Dam (NEED).

Figure 13: The computational domain of Earth including the coastline, seabed,
and the reference sea level.

The simulation performed with our homemade code takes around 2 hours on a
laptop having an 11th Generation Intel(R) Core(TM) i5-11500H processor with a
2.92 GHz clock with 32 GB of RAM. In Section 6, we also summarize the
scalability of the parallel code executed on a multi-core node of Ares
parallel machine [29]. The simulated time period is 12500 minutes, or 8 days
16 hours and 20 minutes.

Subsequent snapshots from the computer simulation, summarized in Table 1 are
shown in Figures 14-28.

Figure | time step | simulated time
---|---|---
18 | 1 | 41 minutes
19 | 2 | 1 hours 23 minutes
20 | 3 | 2 hours 5 minutes
21 | 4 | 2 hours 46 minutes
22 | 5 | 3 hours 28 minutes
23 | 10 | 6 hours 56 minutes
24 | 15 | 10 hours 25 minutes
25 | 20 | 13 hours 3 minutes
26 | 40 | 1 day 3 hours 46 minutes
27 | 60 | 1 day 17 hours 40 minutes
28 | 100 | 2 days 21 hours 26 minutes
29 | 150 | 4 days 8 hours 10 minutes
30 | 200 | 5 days 18 hours 53 minutes
31 | 250 | 7 days 5 hours 36 minutes
32 | 300 | 8 days 16 hours 20 minutes
Table 1: Snapshots of the simulation

These figures show the systematic flooding of the North Sea basin and then the
Baltic Sea by a diffusing wave with a speed of about 10 kilometers per hour.
In this stage, the level of the North Sea and the Baltic Sea gradually rises
to the initially assumed level of 6 meters above the original average sea
level.

Figure 14: 41 minutes after breaking of the NEED. Figure 15: 1 hour 23 minutes
after breaking of the NEED. Figure 16: 2 hours 5 minutes after breaking of the
NEED. Figure 17: 2 hours 46 minutes after breaking of the NEED. Figure 18: 3
hours 28 minutes after breaking of the NEED. Figure 19: 6 hours 56 minutes
after breaking of the NEED. Figure 20: 10 hours 25 minutes after breaking of
the NEED. Figure 21: 13 hours 3 minutes after breaking of the NEED. Figure 22:
1 day 3 hours 46 minutes after breaking of the NEED. Figure 23: 1 day 17 hours
40 minutes after breaking of the NEED. Figure 24: 2 days 21 hours 26 minutes
after breaking of the NEED. Figure 25: 4 days 8 hours 10 minutes after
breaking of the NEED. Figure 26: 5 days 18 hours 53 minutes after breaking of
the NEED. Figure 27: 7 days 5 hours 36 minutes after breaking of the NEED.
Figure 28: 8 days 16 hours 20 minutes after breaking of the NEED.

##  6 Parallel computations

Figure 29: Execution time of parallel code on a single node of ARES with 48
cores and 192 GB of RAM. Figure 30: Fractions of the execution times of
particular parts of the parallel code executed on a single node of ARES with
48 cores and 192 GB of RAM.

We have run 20 time iterations of our simulator on a single node of ARES
supercomputer [29] equipped with 48 cores and 192 GB of RAM, measuring:

  * •

”mesh loading” - mesh file loading time,

  * •

”setup” - time to allocate memory, pre-process the mesh, initial state, etc.

  * •

”assembly” - time spent generating matrices and right sides for the system of
equations,

  * •

”solver” - time of linear solver,

  * •

”step update” - the updates of the velocity and wave shape fields, estimated
from the accelerations solved,

  * •

”file output” - saving the results to a file.

The execution times of different parts of the algorithm, measured as we
increase the number of cores, from 1 to 48 cores, are presented in Figure 29.
The fractional decompositions of the total simulation time into different
parts for increasing number of cores are outlined in Figure 30.

Figure 31: Speedup of parallel execution on a single node of ARES with 48
cores and 192 GB of RAM. Figure 32: Efficiency of parallel execution on a
single node of ARES with 48 cores and 192 GB of RAM.

The summary of the speedup of the particular parts, measured for 1-48 cores,
is presented in Figure 31. The efficiency for 1-48 cores is presented in
Figure 32. To summarize the execution times, we can perform 20 iterations of
the simulator using 48 cores in less than 4 seconds; see Figure 29.

##  7 Conclusions

The massive damage to the Northern European Enclosure Dam is predicted to have
huge consequences for the European population living in coastal areas.

We combined a unique graph transformation model to generate the entire Earth
to perform a simulation of the NEED break using wave equation, finite element
method solver with the higher-order generalized-$\alpha$ method.

From our simulations, we can draw the following conclusions: a potential NEED
should be built with great care and certainly with additional coastal
protection systems in the event of a catastrophic break.

##  8 Acknowledgements

The authors are grateful for support from the funds the Polish Ministry of
Science and Higher Education assigned to AGH University of Krakow. The work
supported by “Excellence initiative - research university” for the AGH
University of Krakow. The work of Albert Oliver-Serra was supported by ”Ayudas
para la recualificación del sistema universitario español” grant funded by the
ULPGC, the Ministry of Universities by Order UNI/501/2021 of 26 May, and the
European Union-Next Generation EU Funds.

##  9 Computer Code Availability

NEED dam break simulation was carried out using tsunami-europe script,
developed by Marcin Łoś, using FEniCS framework

https://fenicsproject.org/download/archive/

Running it requires Python (version at least 3.10), FEniCS framework and
meshio Python package. No special hardware is required. The script consists of
170 lines of Python code. The source code and the mesh file are freely
available at

https://github.com/marcinlos/tsunami-europe

under MIT License, with a brief instruction.

The meshes used in the NEED dam break simulation have been generated using the
tsunami-europe-mesh script developed by Paweł Maczuga and Albert Oliver-Serra.
You can access the script in the following publicly available repository:

https://github.com/albert-oliver/Tsunami-Europe-mesh

The code runs in the Julia programming language, and it depends on the
following two libraries, also developed by Paweł Maczuga and Albert Oliver-
Serra:

https://github.com/albert-oliver/MeshGraphs.jl

This library implements the Graph Grammar for the Rivara refinement and
provides an API for generating and refining a mesh.

https://github.com/albert-oliver/TerrainGraphs.jl

This library implements the mesh generation of terrains (either flat or
spherical). It reads a GeoTIFF file, generates an initial coarse mesh, checks
the refinement criteria specific to terrains, and refines the mesh until it
obtains the final mesh.

Please note that the software code is available under the MIT license

## References

  * [1] S. Groeskamp, J. Kjellsson, Northern Europe Enclosed: Engineering a Solution to Sea Level Rise, _Bulletin of the American Meteorological Society_ 101(11) (2020) 971-974.
  * [2] S. Groeskamp, J. Kjellsson, NEED Northern European Enclosure Dam, _Europhysics News_ 52(2) (2021)
  * [3] S. Groeskamp, J. Kjellsson, The Northern European Enclosure Dam for if Climate Change Mitigation Fails, _American Metheorological Society_ , (2020) E1174-E1189
  * [4] J. A. Church, N. J. White, Sea-level rise from the late 19th to the early 21st century. _Surveys in Geophysics_ , 32 (2011) 585–602
  * [5] A. Levermann, P. U. Clark, B. Marzeion, G. A. Milne, D. Pollard, V. Radic, A. Robinson, The multimillennial sea-level commitment of global warming, _Proceedings of the National Academy of Sciences USA_ , 110 (2013) 13745–13750,
  * [6] R. M. DeConto, D. Pollard, Contribution of Antarctica to past and future sea-level rise, _Nature_ , 531 (2016) 591–597.
  * [7] T. L. Edwards, M. A. Brandon, G. Durand, N. R. Edwards, N. R. Golledge, P. B. Holden, I. J. Nias, A. J. Payne, C. Ritz, A. Wernecke, Revisiting antarctic ice loss due to marine ice-cliff instability, _Nature_ , 566 (2019) 58–64.
  * [8] S. Jevrejeva, A. Grinsted, J. C. Moore, Upper limit for sea level projections by 2100, _Environmental Research Letters_ , 9 (2016) 104008.
  * [9] R. E. Kopp, R. M. Horton, C. M. Little, J. X. Mitrovica, M. Oppenheimer, D. J. Rasmussen, B. H. Strauss, and C. Tebaldi, Probabilistic 21st and 22nd century sea-level projections at a global network of tide-gauge sites, _Earth’s Future_ , 2 (2014) 383–406.
  * [10] D. L. Bars, S. Drijfhout, H. de Vries, A high-end sea level rise probabilistic projection including rapid antarctic ice sheet mass loss. _Environmental Research Letters_ , 12 (2017) 044013.
  * [11] G. F. Carrier, Y. Harry, Tsunami Propagation from a Finite Source, Computer Modeling in Engineering & Sciences, 10(2) (2005) 113-121.
  * [12] Y. S. Cho, S. B. Yoon, S. B. A modified leap-frog scheme for linear shallow-water equations, Coastal Engineering Journal, 40(02) (1998) 191-205.
  * [13] I. Ryszka, A. Paszyńska, E. Grabska, M. Sieniek, M. Paszyński, Graph transformation systems for modeling three dimensional finite element method. Part II, _Fundamenta Informaticae_ ,(2015), 140, 173–203
  * [14] E. Grabska, Theoretical Concepts of Graphical Modeling. Part One: Realization of CP-Graphs, _Machine Graphics and Vision_ , (1993), 2(1), 3–38.
  * [15] E. Grabska: Theoretical Concepts of Graphical Modeling. Part Two: CP-Graph Grammars and Languages, _Machine Graphics and Vision_ , (1993), 2(2), 149–178
  * [16] A. Paszyńska, M. Paszyński, E. Grabska, Graph Transformations for Modeling hp-Adaptive Finite Element Method with Triangular Elements, Computational Science–ICCS 2008: 8th International Conference, Krakow, Poland, June 23-25, 2008, Proceedings, Part III (2008) 604-613
  * [17] M. Paszyński, A. Paszyńska, Graph Transformations for Modeling Parallel hp-Adaptive Finite Element Method, Parallel Processing and Applied Mathematics: 7th International Conference, PPAM 2007, Gdansk, Poland, September 9-12 (2007) 1313-1322
  * [18] M. Paszyński, On the parallelization of self-adaptive hp-finite element methods part I. composite programmable graph grammar model, Fundamenta Informaticae, 93(4) (2009) 411-434
  * [19] M. Paszyński, On the parallelization of self-adaptive hp-finite element methods part II. partitioning communication agglomeration mapping (PCAM) analysis, Fundamenta Informaticae, 93(4) (2009) 435-457
  * [20] M.C. Rivara, Algorithms for refining triangular grids suitable for adaptive and multigrid techniques, International Journal Numerical Methods in Engineering, 20(4) (1984) 745–756.
  * [21] M.C. Rivara, Mesh refinement processes based on the generalized bisection of simplices, SIAM Journal Numerical Analysis, 21(3) (1984) 604–613.
  * [22] U. Kanoglu, C. E. Synolakis Long wave runup on piecewise linear topographies. Journal of Fluid Mechanics, 374 (1998) 1-28.
  * [23] P. Maczuga, A. Oliver-Serra, A. Paszyńska, E. Valseth, M. Paszyński, Graph-grammar based algorithm for asteroid tsunami simulations, Journal of Computational Science, Volume 64, 2022, 101856,
  * [24] W.Y. Tan, Shallow water hydrodynamics: Mathematical theory and numerical solution for a two-dimensional system of shallow-water equations, Elsevier, (1992).
  * [25] Global Multi-Resolution Topography Data Synthesis, _https://www.gmrt.org/_
  * [26] E. Pelinovsky, Hydrodynamics of tsunami waves In: Grue, J., Trulsen, K. (eds) Waves in Geophysical Fluids. CISM International Centre for Mechanical Sciences vol.489 (2006) 1-48. Springer.
  * [27] J. Chung, G. M. Hulbert, A Time Integration Algorithm for Structural Dynamics With Improved Numerical Dissipation: The Generalized-$\alpha$ Method, Journal of Applied Mechanics, 60 (1993) 371–375.
  * [28] M.M. Grieman, C. Nehrbass-Ahles, H. M. Hoffmann, T. K. Bauska, A.C.F. King, R. Mulvaney, R. H. Rhodes, I. F. Rowell, E. R. Thomas, E.W. Wolff, Abrupt Holocene ice loss due to thinning and ungrounding in the Weddell Sea Embayment, Nature Geosciences (2024). https://doi.org/10.1038/s41561-024-01375-8
  * [29] _https://www.cyfronet.pl/en/computers/4581,main.html_
  * [30] K. Podsiadło, A. O. Serra, A. Paszyńska, R. Montenegro, I. Henriksen, M. Paszyński, K. Pingali, Parallel graph-grammar-based algorithm for the longest-edge refinement of triangular meshes and the pollution simulations in Lesser Poland area. Engineering with Computers 37 (2021) 3857–3880.


# FACOS: Finding API Relevant Contents on Stack Overflow with Semantic and
Syntactic Analysis

Kien Luong1, Mohammad Hadi2, Ferdian Thung1, Fatemeh Fard2, and David Lo1
1School of Computing and Information Systems, Singapore Management University
2Irving K. Barber Faculty of Science, University of British Columbia
{kiengialuong, ferdianthung<EMAIL_ADDRESS>{mohammad.hadi,
<EMAIL_ADDRESS>
###### Abstract

Collecting API examples, usages, and mentions relevant to a specific API
method over discussions on venues such as Stack Overflow is not a trivial
problem. It requires efforts to correctly recognize whether the discussion
refers to the API method that developers/tools are searching for. The content
of the thread, which consists of both text paragraphs describing the
involvement of the API method in the discussion and the code snippets
containing the API invocation, may refer to the given API method. Leveraging
this observation, we develop FACOS, a context-specific algorithm to capture
the semantic and syntactic information of the paragraphs and code snippets in
a discussion. FACOS combines a syntactic word-based score with a score from a
predictive model fine-tuned from CodeBERT. FACOS beats the state-of-the-art
approach by 13.9% in terms of $F_{\textit{1}}\textit{-}score$.

##  I Introduction

Developers typically use existing libraries or frameworks to implement certain
common functionalities. Understanding which APIs to use, the methods they
offer, their distinctive names, and how to use them is vital in this regard.
There may be hundreds or even thousands of APIs in a large-scale software
library such as the .NET framework and JDK. Microsoft conducted a survey in
2009 in which 67.6% of respondents said that inadequate or absent resources
hindered learning APIs [1].

In order to gain a deeper understanding of APIs and their usage information,
developers need to inspect many web pages manually and they use automated code
search tools. Most of the Code search tools do not consider the semantics of
natural language queries because they are based on keyword matching. Stack
Overflow is the second most common place for the developers to discover APIs,
their simple method names, and their usage through crowd-sourced questions and
answers. As many API names share simple names but provide different
functionality, it is difficult to find code snippets and APIs that correspond
to the specific problem searched by the developers on these platforms.
Moreover, API mentions in the informal text content of Stack Overflow are
often ambiguous, which makes it difficult to track down APIs and learn their
uses.

Developers frequently discuss and mention APIs in natural language in online
discussion and question answering forums like Stack Overflow [2, 3, 4, 5].
When developers or automated tools are looking for a specific API, names of
API methods sharing the same name can be ambiguous. Therefore, we require API
disambiguation to support several downstream tasks such as API recommendation
and API mining. To properly index and link APIs to their related information
in various sources (e.g., Stack Overflow, Javadoc, etc.), it is important to
link ambiguous API mentions to their actual APIs correctly.

Luong et al. recently proposed DATYS [6], which uses type-scoping to
disambiguate API mentions in informal text content on Stack Overflow. In type
scoping, they considered API methods whose types appear in more parts (i.e.,
scopes) of a Stack Overflow thread as more likely to refer to the searched API
method. However, the statistical word alignment model it uses is based on the
appearance of words in a sentence rather than considering in which context the
sequence of words are being used and what connotations do these words relaying
to the readers.

APIs are often discussed and mentioned in natural language in online forums
such as Stack Overflow to better understand them. Developers or automated
tools looking for a specific API would be confused by API methods with the
same name. In Stack Overflow, API disambiguation is crucial to finding APIs.
Several downstream tasks, such as API recommendation [7, 8], are supported by
this collection, including API mining [7, 8], which relies on interpreting API
mentions correctly to index and link APIs to relevant information in various
data sources, including Stack Overflow, Javadoc, etc.

To incorporate a deeper understanding of the underlying semantics in a natural
language text content of Stack Overflow, we introduce FACOS, a context-
specific algorithm to capture the semantic and syntactic information of the
paragraphs and code snippets in a crowd-sourced discussion. We call this API
resource retrieval task because FACOS focuses on finding Stack Overflow
threads mentioning a given API method. Our work also modified DATYS to perform
better search over the code snippets in the Stack Overflow discussion threads.
The modified DATYS, denoted as DATYS+ provides an additional metric to better
capture the occurrence of API method type in the Stack Overflow thread. By
greedily matching the type name with the tokens in the code snippet, DATYS+
performs the syntactic search in FACOS. Yet, both DATYS and DATYS+ are only
searching based on the syntactic information provided by the fully qualified
name of a target API method. It cannot capture the semantic meaning in
paragraphs and code snippets of threads on Stack Overflow and how similar they
are to the target API method. Thus, to capture the semantics, in addition to
the weighted syntactic information provided by DATYS+, FACOS has a semantic
search component that leverages a deep attention based Transformer model,
CodeBERT [9]. This semantic search component measures the similarity between
the paragraphs and code snippets of a Stack Overflow thread with the target
API method comment and implementation code. The more similar they are, the
more likely the thread to be relevant to the target API method that we search
for. To efficiently leverage both semantic and syntactic knowledge of Stack
Overflow thread and the API method, FACOS joined the semantic and the
syntactic search element to get the joint relevance score that determines
whether a thread relates to a given API method. The contributions of each
element in the joint relevance score is defined by a weighting factor.

In this paper, we are going to answer with the following research questions:

  1. RQ1

Can FACOS perform better than the baseline (DATYS)?

  2. RQ2

How well does each component of FACOS perform?

  3. RQ3

How does the weighting factor affect the F1-score of FACOS?

These research questions will help us understand the effectiveness of our
approach FACOS and the internal mechanism through which it yields better
results than the current baseline. Our work has offered the following main
contributions:

  1. 1.

To our knowledge, we are the first to adopt a transformer-based deep learning
technique to incorporate semantic knowledge understanding for API resource
retrieval task.

  2. 2.

As compared to state-of-the-art techniques, our approach performs better while
searching for the contents related to the queried API. On a dataset of 380
Stack Overflow threads, FACOS beats the state-of-the-art by 13.9%.

  3. 3.

We designed an ablation study to understand how the integrated components of
our approach are performing. We found that each component contributes to the
effectiveness of FACOS.

  4. 4.

We have also open sourced our code and additional artifacts required for
recreating the results and re-purposing our approach for other tasks. The
source code for FACOS is available at
https://anonymous.4open.science/r/facos-E5C6/

The rest of the paper is structured as follows: Section II deals with the
preliminary knowledge about the components on top of which we have built our
method. Section III provides an overview of our proposed approach, while
Section IV elaborates the various components of our proposed approach. We
describe our experiment details and results in Section V and VI, respectively.
The related works and the threats to validity are presented in Sections VIII
and VII-D. Finally, we concluded our work and present future work in Section
IX.

##  II Preliminaries

###  II-A DATYS

Two steps are involved in finding the API mentioned in informal text content:
(1) API mentions extraction, and (2) API mentions disambiguation. API mention
extraction aims to identify common words that refer to the APIs. API mentions
disambiguation, on the other hand, links API mentions with the APIs they
reference. DATYS [6] specifically deals with the API mention disambiguation
via type scoping in the informal text of Stack Overflow to resolve ambiguous
mentions of Java API methods, where the mentions have been identified.

After extracting API method candidates from input Java libraries, DATYS scores
API method candidates based on how often their types (i.e., classes or
interfaces) appear in different parts (i.e., scopes) of the Stack Overflow
thread with identified API mentions. Having a type that appears in more scopes
will increase the API candidate score. Here, DATYS considers three scopes:
Mention scope, which covers the mention itself. Text scope, which covers the
textual content of the thread, including the mentions. Code scope, which
covers the code snippets in the thread. API candidates are ranked according to
their scores for each API mention in the thread. DATYS takes the top API
candidate with a non-zero score as the mentioned API. If the leading API
candidate has a zero score, DATYS considers the mention as an unknown API.
Luong et al. built a ground truth dataset containing 807 Java API mentions
from 380 threads in Stack Overflow.

###  II-B CodeBERT

CodeBERT [9] was developed using a multilayered attention-based Transformer
model, BERT [10]. As a result of its effectiveness in learning contextual
representation from massive unlabeled text with self-supervised objectives,
the BERT model has been adopted widely to develop large pre-trained models.
Thanks to the multilayer Transformer [11], CodeBERT developers adopted two
different approaches than BERT to learn semantic connections between Natural
Language (NL) - Programming Language (PL) more effectively.

Firstly, The CodeBERT developers make use of both bimodal instances of NL-PL
pairs (i.e., code snippets and function-level comments or documentations) and
a large amount of available unimodal codes. In addition, the developers have
pre-trained CodeBERT using a hybrid objective function, which includes masked
language modeling [10] and replaced token detection [12]. The incorporation of
unimodal codes helps the replaced token detection task, which in turn produces
better natural language by detecting plausible alternatives sampled from
generators.

Developers trained CodeBERT from Github code repositories in 6 programming
languages, where only one pre-trained model is learned for all six programming
languages with no explicit indicators used to mark an instance to the one out
of six input programming languages. CodeBERT was evaluated on two downstream
tasks: natural language code search and code documentation generation. The
study found that fine-tuning the parameters of CodeBERT obtained state-of-the-
art results on both tasks.

##  III Approach Overview

###  III-A Task Definition

Our goal is to find Stack Overflow threads that mention a given API
method111In this paper, we use the terms API, method, and API method
interchangeably.. Specifically, given an API method, we strive to find Stack
Overflow threads containing words matching the simple name of the given API
method. In Java, the simple name of an API method is the name of the method
without the class and the package names. For example, m is the simple name of
API method com.example.Class.m. We want to classify whether the threads having
the simple name m is actually relevant to API method com.example.Class.m. In
summary, the task is defined as: “For each API method in a set of given API
methods, identify Stack Overflow threads that refer to it.”

###  III-B Architecture

Figure 1: The architecture of FACOS (Finding API Relevant Contents on Stack
Overflow with Semantic and Syntactic Analysis)

The pipeline of FACOS is presented in Figure 1. It is divided into 2 main
steps:

(1) Collecting various API-related resources from a given API method name; and
(2) Recommending relevant threads using the collected API-related resources.

In the step (1), FACOS finds Potential Threads from Stack Overflow using the
simple name of the given API method as the query. Potential Threads are the
threads that have at least one word matching with the simple name of given
API. The API method comment and implementation code are directly obtained from
the source code repository of the given API. Last but not least, the API
Candidates are obtained from a database of API methods. The API Candidates are
API methods that have the same simple name as the given API method.

The objective of step (2) is to identify whether each Stack Overflow thread in
the Potential Threads actually refers to the given API. FACOS has two
components: API relevance classifier and DATYS+. API relevance classifier is
designed to draw the relevance between a thread and an API method by capturing
the semantic similarity between (1) paragraphs and code snippets in the
thread; and (2) API method comment and implementation code. API relevance
classifier outputs a semantic relevance score representing the relevance it
measures. In contrast, DATYS+ outputs a syntactic relevance score based on the
existence of the terms from the fully qualified name of the given API in
different scopes of a Stack Overflow thread. For example, API ”A.B.C” has
terms such as ”A”, ”B”, and ”C”. The last term, ”C”, is the simple name of the
API method. The second last term, ”B”, is the type of the API method. Both
DATYS and DATYS+ use type scoping [6] to give score based on the existence of
the type of the API (i.e., ”B” in the example) in different scopes of the
Stack Overflow thread (code scope, text scope, etc.). However, type scoping of
DATYS+ is modified to be suitable for the search task and we are going to
describe it in Section IV-A. It outputs a score that indicates the syntactic
relevance between the given API and the thread, we call it DATYS+ score.

After step (2), each thread will have a score indicating if the thread refers
to the given API method. This score is combined from semantic relevance score
and DATYS+ score and is called joint relevance score. Threads predicted as
referred to the given API method are then returned to the user. We describe
FACOS components (i.e., DATYS+ and API relevance classifier) in detail in
Section IV.

##  IV FACOS

FACOS consists of two main components: DATYS+ and API relevance classifier.
DATYS+ takes as inputs Potential Threads and API Candidates and outputs scores
indicating its confidence that the given API is referred to in the threads
(Section IV-A). Given Potential Threads and API method comment and
implementation code, FACOS first converts them to API relevance embedding
(Section IV-B). The API relevance embedding is input to API relevance
classifier, which outputs confidence scores indicating the likelihood that the
given threads refer to the API (Section IV-C). Finally, the scores from DATYS+
and API relevance classifier are combined to a joint relevance score, and
threads with scores larger than a threshold are returned as the relevant
threads (Section IV-D).

###  IV-A DATYS+

DATYS+ is an extension of DATYS. DATYS used regular expressions to capture the
types of API method invocations available in code snippets of the thread.
However, these regular expressions are limited and thus DATYS may miss some
mentions in code snippets. To capture more types, DATYS+ modifies the type
scoping algorithm by adding a new score.

Algorithm 1 indicates how modified type scoping works. Compared to DATYS’s,
DATYS+’s type scoping algorithm receives $\mathit{CodeSnippets}$ as another
input. $\mathit{CodeSnippets}$ represents the content available in code
snippets of the Stack Overflow thread. In addition, inputs of the original
type scoping algorithm are also considered. $\mathit{APIMention}$,
$\mathit{PTypeList}$, $\mathit{APIMethodCandidate}$, and
$\mathit{ThreadContent}$ stand for the simple name of the given API, the list
of possible types extracted from code snippets following the algorithm used by
DATYS, the API Candidates, and the thread’s textual content (i.e., title,
text, tags), respectively. The three scopes used by DATYS are also used in
DATYS+. In Mention Scope (Lines 3-8), DATYS+ increases an API score if its
type appear within the API mention. In Text Scope (Lines 10-13), DATYS+
increases an API score if its type appear within the textual content of the
thread. In Code Scope (Lines 17-21), DATYS+ increases an API score if its type
matches with the type of method invocation or imported types in the code
snippet. Additionally, in Code Scope, DATYS+ also looks at the content of the
code snippets and increases the API score of the corresponding API candidate
if there are tokens in the code snippets that match with the API type (Lines
14-16). This score helps to capture the occurrence of types that would be
missed by a more accurate matching used in DATYS. Thus, we call the scope of
this score Extended Code Scope

After executing type scoping, DATYS+ returns scores for the API Candidates.
The scores are then normalized to a range of [0, 1] following the minimum and
the maximum score from the API Candidates. DATYS+ then takes the normalized
score of the given API method and passes it to the next step.

Algorithm 1 Scoring an API Candidate with Type Scoping in DATYS+

0: $ApiMention,PTypesList,APIMethodCandidate,$$ThreadContent,CodeSnippets$

0: $CandScore$

1: $CandScore$ = 0

2: $CandType=getType(APIMethodCandidate)$

3: if $hasPrefix(ApiMention)$ then

4: $Prefix=getPrefix(ApiMention)$

5: if $endsWith(Prefix,CandType)$ then

6: $CandScore=CandScore+1$

7: end if

8: end if

9: $TextualTokens=tokenize(ThreadContent)$

10: $CodeTokens=tokenize(CodeSnippets)$

11: if $CandType$ $in$ $TextualTokens$ then

12: $CandScore=CandScore+1$

13: end if

14: if $CandType$ $in$ $CodeTokens$ then

15: $CandScore=CandScore+1$

16: end if

17: for $PType$ in $PTypesList$ do

18: if $isSameType(PType,CandType)$ then

19: $CandScore=CandScore+1$

20: end if

21: end for

22: return $CandScore$

###  IV-B API relevance embedding

We follow the process described in Figure 2 to build API relevance embedding.
Firstly, each thread in Potential Threads needs to be converted into an
embedding. A thread may contain m paragraphs and n code snippets. A paragraph
is a piece of textual content on a Stack Overflow thread that is separated
from other contents in the thread via a newline character. Code snippet is a
piece of code content on a Stack Overflow thread. It is typically enclosed
with a starting tag $\langle pre\rangle\langle code\rangle$ and an ending tag
$\langle/code\rangle\langle/pre\rangle$. Each paragraph is paired with each
code snippet to create a pair of thread content. Therefore, a Stack Overflow
thread would have $m\times n$ thread content pairs. A natural-programming
language model, CodeBERT222https://github.com/microsoft/CodeBERT, is used to
extract the semantic meaning of each thread content pair. It encodes the
$m\times n$ thread content pairs into $m\times n$ thread embeddings. thread
embedding is the representation vector of thread content that created by
CodeBERT’s encoder. By converting the pairs from a textual form to a numerical
vector form with a pre-trained CodeBERT model, the semantic relationship
between the paragraphs and code snippets is extracted. Before feeding the
thread content pairs into the encoder of CodeBERT, each pair is pre-processed
following the format:

$\langle CLS\rangle\ paragraph\ \langle SEP\rangle\ code\ snippet\ \langle
EOS\rangle$

$\langle CLS\rangle$ is the token that informs the start of the pair according
to the design of RoBERTa model [13] which CodeBERT is based on. $\langle
SEP\rangle$ is the token that separates a Paragraph from a Code Snippet and
$\langle EOS\rangle$ indicates the end of the pair. In detail, the maximum
number of tokens in a pair before being fed into CodeBERT encoder is 512. We
set the number of tokens for a paragraph and a code snippet to 254 and 255
tokens, respectively. The two numbers add up to 512 when the three tokens such
as $\langle CLS\rangle$, $\langle SEP\rangle$, and $\langle EOS\rangle$ are
counted. If the number of tokens in the paragraph is less than 254, then
padding tokens would be added to reach 254 tokens. On the other hand, if the
number of tokens in the paragraph is more than 254, we truncate the paragraph
and take the first 254 tokens. The same process is applied to the code snippet
with 255 tokens. The CodeBERT encoder receives these thread content pairs
under this format as inputs and outputs embedding vectors. For a thread with
$m\times n$ thread content pairs, there would be $m\times n$ thread embedding
vectors created and each thread embedding vector has a length of 768.

Figure 2: How API relevance embeddings are created

Secondly, to build API relevance embedding, API comment and implementation
code also need to be converted into an embedding. The API method comment is a
piece of textual content that describes the functionality of the API method
and how to use it. The API implementation code is the code inside the API
method body that implements the described functionality. The API comment and
implementation code are extracted from the Javadoc and the JAR files,
respectively, they are pre-processed to the following format:

$\langle CLS\rangle\ comment\ \langle SEP\rangle\ implementation\ code\
\langle EOS\rangle$

They are then transformed into a numerical representation vector via the
CodeBERT encoder.

Finally, each thread embedding vector and the method embedding vector are then
concatenated to a vector. We call this concatenated vector API relevance
embedding. In total, $m\times n$ API relevance embedding vectors would be
created.

###  IV-C API relevance classifier

The API relevance classifier is a binary classifier that utilizes a neural
network with two fully connected layers to predict whether the API relevance
embedding comes from a Stack Overflow thread that refers to the given API
method.

The API relevance classifier has two modes of operation: training and
deployment modes. In the training mode, the API relevance embeddings are used
to train the API relevance classifier. When there is an imbalance between
positive and negative labels, API relevance classifier upsamples the minority
label. Whenever the thread refers to the given API method, all API relevance
embedding created from the thread would be considered as positive by the
classifier. Otherwise, in case the given API method is not referred to by the
thread, every API relevance embedding of the thread would have negative
labels. In the deployment mode, API relevance classifier produces probability
scores for the $m\times n$ API relevance embedding. These scores are averaged
and passed to the next step. The averaged score indicates the likelihood that
the thread refers to the given API.

###  IV-D Computing joint relevance score

Figure 3: Computing joint relevance score

We follow the process in Figure 3 to compute the joint relevance score. DATYS+
and API relevance classifier output scores $A{}$ and $B{}$, respectively. Both
represent their confidence that the given API method is mentioned in the
thread. The two scores are then combined to a joint relevance score $C$
following this formula:

$C{}=x\times A{}+(1-x)\times B{}$ (1)

The weighting factor $x{}$ decides the contributions of DATYS+ score and API
relevance classifier in joint relevance score $C$. The higher the value of
$x{}$ is, the more DATYS+ score contributes to the final joint relevance
score. The range of $A{}$, $B{}$, and $x{}$ is from $0$ to $1$. A thread is
considered to refer to the given API if the joint relevance score $C$ is
larger than a threshold $t$. Otherwise, the thread is considered not to refer
to the given API. By default, $t$ is set to $0.5$.

The value of $x{}$ will be estimated based on the training data. In detail, we
let $x{}$ increase gradually from $0$ to $1$ with a step of 0.1. There are ten
possible values of $x{}$: $\\{0,0.1,0.2,...,0.9,1.0\\}$. The value of $x{}$
giving the highest performance in the training data is then chosen.

##  V Experiment

###  V-A Dataset and Experimental Settings

We utilize the dataset provided in DATYS work [6] to evaluate both FACOS and
DATYS. We split 380 Stack Overflow threads to 253 training threads and 127
testing threads with the ratio of 2:1. The training threads are utilized to
train the API relevance classifier while the testing threads are used to
evaluate FACOS and DATYS. Next, as mentioned in Section IV-B, for each Stack
Overflow thread in the training threads, we extract its thread embeddings and
these thread embeddings are grouped into a training set. Similarly, for each
Stack Overflow thread in the testing threads, we extract its thread embeddings
and these thread embeddings are grouped into a testing set. The numbers of API
relevance embeddings of the dataset are shown in Table I. The numbers of the
embeddings in training set and testing set are $57\,690$ and $26\,212$,
respectively.

TABLE I: Number of API relevance embeddings in each set | # API relevance embeddings
---|---
Training set | 57,690
Testing set | 26,212

To generate API relevance embeddings for the API relevance classifier for
training, for each thread, if the given API appears in the thread, we generate
API relevance embeddings for thread contents and method contents as described
in Section IV-B. These embeddings would have positive label because they are
created from the API that is referred to by the thread. To generate embeddings
with a negative label for a thread, we find APIs that have the same simple
name as the given API and are not mentioned in the thread. We then create API
relevance embeddings from these APIs and label these API relevance embeddings
as negative.

To train the API relevance classifier, there are $344$ APIs. These APIs are
used to generate the training method embeddings. In the testing set, there are
$181$ APIs. These APIs are used to generate the testing method embeddings.
Table II shows the numbers of positive and negative API relevance embeddings
created in training and testing sets. The number of negative API relevance
embeddings is approximately 4 times more compared to the positive ones in the
same set of thread. Due to this imbalance, positive API relevance embeddings
are randomly up-sampled to balance the two classes within the API relevance
classifier training process.

TABLE II: Number of positive and negative API relevance embeddings in each set Positive Embeddings in Training set | 9,934
---|---
Negative Embeddings in Training set | 47,756
Positive Embeddings in Testing set | 5,607
Negative Embeddings in Testing set | 20,605

The API relevance classifier is trained using 6 epochs on the training data.
After the first 6 epochs, the value of the loss function has relatively
converged. The learning rate of the training is set to $10^{-3}$.

###  V-B Metrics

To evaluate the proposed approach on identifying threads that are relevant to
an API, we use three metrics: Precision, Recall, and
$F_{\textit{1}}\textit{-}score$. In order to calculate the three
aforementioned metrics, True Positive, False Positive, and False Negative
should be defined first. Our task focuses on finding threads that actually
refer to a given API. True Positive is the case where a thread is deemed to be
relevant by the approach is indeed relevant. False Positive is the case where
the thread that is deemed to be relevant by the approach is actually
irrelevant. False Negative is the case where a threads is deemed to be
irrelevant by the approach is actually relevant. The metrics are calculated
using the following formulas:

$Precision=\frac{True\ Positive}{True\ Positive+False\ Positive}$ (2)
$Recall=\frac{True\ Positive}{True\ Positive+False\ Negative}$ (3)
$F_{\textit{1}}\textit{-}score=\frac{2\times Precision\times
Recall}{Precision+Recall}$ (4)

We measure the above scores of all given APIs in the testing set and report
the averages of the scores.

###  V-C Research Questions

Research Question 1: Can FACOS perform better than the baseline (DATYS)?

The baseline, DATYS, was designed for a task of API mention disambiguation. We
adopt it to our task of finding threads that are relevant to an API. If DATYS
finds an API is mentioned in the thread, the thread is considered to be
relevant to the API. To evaluate the improvement that FACOS over DATYS, we
evaluate them in the testing data set and compare them in terms of
$F_{\textit{1}}\textit{-}score$. We also analyse some cases that FACOS can
resolve and DATYS can not in Section VII-A.

Research Question 2: How well does each component of FACOS perform?

There are three possible variants of of FACOS depending on which component
that comes along with it. The variants are (1) FACOS with API relevance
classifier; (2) FACOS with DATYS+; and (3) FACOS with DATYS+ and API relevance
classifier. API relevance classifier is a semantic-based algorithm while
DATYS+ is a syntactic-based algorithm. In this study, we aim to analyze the
contribution of each component in FACOS. From the analysis, we would like to
answer the question whether combining a semantic-based algorithm and a
syntactic-based algorithm leads to a better result than running them
individually.

Research Question 3: How does the weighting factor affect the F1-score of the
relevant thread classification? Does our strategy work well?

The weighting factor is an importance factor that would affect how well FACOS
perform. We select the importance factor based on the best performance in the
training data. We analyze whether our strategy leads to the best performance
in the testing data. We vary the values of weighting factor in both the
training data and the testing data. The values that we use are
$\\{0,0.1,0.2,...,0.9,1.0\\}$. We analyze whether picking values in the
training data that leads to the best performance in the training data also
leads to the best performance in the testing data.

##  VI Result

###  VI-A RQ1: FACOS Effectiveness

TABLE III: FACOS vs DATYS in terms of $F_{\textit{1}}$-score in the testing set Approach | Avg. | Avg. | Avg.
---|---|---|---
| Precision | Recall |  $F_{\textit{1}}$-score
DATYS | 0.7441 | 0.7703 | 0.7340
FACOS | 0.8697 | 0.9016 | 0.8730
TABLE IV: Contribution of FACOS Components Components | Avg. | Avg. | Avg.
---|---|---|---
| Precision | Recall |  $F_{\textit{1}}$-score
FACOS | 0.8697 | 0.9016 | 0.8730
FACOS with only API | 0.3408 | 0.3658 | 0.3408
relevance classifier |  |  |
FACOS with only DATYS+ | 0.8620 | 0.8723 | 0.8530

Table III shows the performance of the DATYS and FACOS in finding threads that
are relevant to the given API. FACOS, in general, outperforms DATYS. On
average, FACOS achieves an F1-score of 0.873, which is an improvement of 13.9%
compared to DATYS. FACOS also beats DATYS in terms of precision and recall.

###  VI-B RQ2: Ablation Study

TABLE V: Average Precision, average Recall, and average $F_{\textit{1}}$-score of testing sets when weighting factor varies x | Avg. Precision | Avg. Recall | Avg. $F_{\textit{1}}$-score
---|---|---|---
$0$ | 0.3420 | 0.3658 | 0.3408
$0.1$ | 0.4485 | 0.4653 | 0.4441
$0.2$ | 0.8650 | 0.8925 | 0.8641
$0.3$ | 0.8697 | 0.9016 | 0.8730
$0.4$ | 0.8684 | 0.8934 | 0.8689
$0.5$ | 0.8588 | 0.8723 | 0.8097
$0.6$ | 0.8588 | 0.8723 | 0.8510
$0.7$ | 0.8606 | 0.8723 | 0.8521
$0.8$ | 0.8606 | 0.8723 | 0.8521
$0.9$ | 0.8606 | 0.8723 | 0.8521
$1.0$ | 0.8620 | 0.8723 | 0.8530

Table IV shows how well each component in FACOS is. Since the “API relevance
classifier-only” version of FACOS gives worst result, API relevance classifier
may not be able to resolve the task well independently. Partly, this might be
be due to the limited amount of the training data (i.e., only 253 training
threads). In addition, the “DATYS+ only” version of FACOS performs much better
compared to the “API relevance classifier-only” version. However, FACOS is
still better than both of them. It demonstrates that both components are
useful and essential.

###  VI-C RQ3: Effect of the weighting factor

Table V shows the performance of FACOS in the training set when we vary the
values of weighting factor. The bold numbers in each row of the table are the
average $F_{\textit{1}}\textit{-}scores$ of the chosen values of x in the
training sets. Similarly, Table VI the performance of FACOS in the test set
when we vary the values of weighting factor. The highest
$F_{\textit{1}}\textit{-}score$ for both the training and testing set is
achieved when the value of the weighting factor is equal to 0.3. It
demonstrates that our strategy to pick the value of the weighting factor that
leads to the best performance in the training data works really well.

TABLE VI: Average Precision, average Recall, and average $F_{\textit{1}}$-score of training sets when weighting factor varies x | Avg. Precision | Avg. Recall | Avg. $F_{\textit{1}}$-score
---|---|---|---
$0$ | 0.6565 | 0.6685 | 0.6473
$0.1$ | 0.7111 | 0.7272 | 0.7080
$0.2$ | 0.8261 | 0.8506 | 0.8269
$0.3$ | 0.8328 | 0.8498 | 0.8329
$0.4$ | 0.8265 | 0.8410 | 0.8254
$0.5$ | 0.8159 | 0.8234 | 0.8097
$0.6$ | 0.8132 | 0.8234 | 0.8079
$0.7$ | 0.8132 | 0.8234 | 0.8079
$0.8$ | 0.8132 | 0.8234 | 0.8079
$0.9$ | 0.8132 | 0.8234 | 0.8079
$1.0$ | 0.8180 | 0.8191 | 0.8073

##  VII Discussion

###  VII-A Cases where FACOS outperforms DATYS

(1) The relevant thread does not contain the type name of the given API method

Figure 4 shows the example of a case where the content of the thread does not
relate to the given API method. The thread contains paragraph and code snippet
of a Stack Overflow thread with ID
56135373333https://stackoverflow.com/questions/56135373/.
org.mockito.stubbing.OngoingStubbing.thenReturn444https://javadoc.io/doc/org.mockito/mockito-
all/2.0.2-beta/org/mockito/stubbing/OngoingStubbing.html is the API method the
thread refers to.

From the content of the thread, it would be difficult to find the relevance
between the text written in the paragraphs and the given API method (i.e.,
org.mockito.stubbing.OngoingStubbing.thenReturn) since the type (i.e.,
OngoingStubbing) does not appear in the thread. The text only shows the user
view towards the code snippet without having a description mentioning the
application or usage of the observed API method invocation (e.g., thenReturn
in the code snippet of Figure 4). Sentences such as ”This works like charm!”
do not provide much information to identify whether the observed API method
refers to the given API.

Therefore, we leverage the content of the thread which might be relevant to
the content of the API method. For example, in the thread above, its title
which is shown in Figure 6(a), ”Optional cannot be returned by stream() in
Mockito Test classes”, relates to the comment of the given API which is Sets a
return value to be returned when the method is called in Figure 6(b). Due to
this feature, FACOS can successfully consider this thread as relevant while
DATYS missed it.

Figure 4: Thread 56135373 on Stack Overflow where API is referred by a code
snippet of the thread

(2) The irrelevant thread contains the type name of the given API method

An example of this case is shown in Figure 5. In the
thread555https://stackoverflow.com/questions/16919751/, the given API method
is com.google.common.base.CharMatcher.is and there is a word that matches the
simple name of the API method is which we highlighted. Since the type of the
given API method (e.g., CharMatcher) appears in both the textual content and
the code snippet, DATYS mistakenly accepts the thread as referring to the
given API. By leveraging the semantic knowledge learnt by the API relevance
classifier, FACOS is able to detect the irrelevance between the textual
content, code snippet around the word is and the API comment and
implementation code. FACOS can conclude that the thread is irrelevant to the
given API com.google.common.base.CharMatcher.is.

Figure 5: Thread 16919751 on Stack Overflow where API
com.google.common.base.CharMatcher.is is not referred by content of the
thread.

(a)

(b)

Figure 6: The similarity in semantic meaning between the API comment of method
org.mockito.stubbing.OngoingStubbing.thenReturn in Figure 6(a) and the textual
content (i.e., the title) of thread 56135373 in Figure 6(b)

###  VII-B Case where FACOS fail to exclude irrelevant threads

Figure 7 shows a case where FACOS fail to exclude the
thread666https://stackoverflow.com/questions/30127057/ out of the relevant
results for the given API method org.mockito.Mockito.mock. The issue occurs
when there is an API method that has a similar functionality as the given API
method. These two methods usually have the same simple name and highly similar
functionality description.

In Figure 7, PowerMock and Mockito, perform similar functions such as mocking
(i.e., creating a version of a service in order to quickly and reliably run
tests on that service777https://circleci.com/blog/how-to-test-software-part-i-
mocking-stubbing-and-contract-testing/). Since both of them have the API
method whose simple name is mock, and both their mock methods have the same
API signature (i.e., parameters, return type), it would be easy to mistakenly
recognize one as the other, even for a human. Figure 8 shows the comment of
API method
org.mockito.Mockito.mock888https://javadoc.io/static/org.mockito/mockito-
all/2.0.2-beta/org/mockito/Mockito.html, which is Creates mock object of given
class or interface. Because of the similarity between the API method from
Mockito library and the title of thread 30127057 in Figure 7, FACOS wrongly
recognizes that the simple API name mock in the thread refers to the given API
method org.mockito.Mockito.mock. In fact, the simple API name mock refers to
the one from PowerMock library.

Figure 7: Thread 30127057 on Stack Overflow that FACOS falsely recognize as
referring to the API method org.mockito.Mockito.mock.

Figure 8: The API comment of method org.mockito.Mockito.mock

###  VII-C Adding Semantic Information for API Content Search

The API method and relevant information search does not always utilize
syntactic information. For some libraries, some methods are chained (as in
Figure 4), so the type is not displayed. It requires a type system to
determine its type. In addition, the Type system is not 100 percent reliable
due to a lack of import information, variables, etc. It is ineffective to use
the similarity of word representations provided by language models for API
contents search. API contents search is unique to other tasks such as code
search. In code search, due to the term ”transfer” and ”convert” being related
to currency, the method named ”transferSGDToUSD” might be the answer to the
query ”convert SGD to USD.” However, searching for an API method requires
explicit mention of the API name in the query, which could be difficult as
different APIs may share the same API method names. To solve the problem,
solely using a model that only learns the syntactic meaning of textual and
code content is insufficient. Therefore, we incorporated semantic information
for API content search, which generated better results, as we have
demonstrated from the results of our experiment.

###  VII-D Threats to validity

A threat to internal validity is related to experiment bias. We obtain our
dataset from another work. We also run the baseline using the code provided by
the author. We then check our code multiple times to ensure that we do not
make mistakes. We believe there should be little threats to internal validity.
We also release the dataset and the code for our experiments for all to use.

A threat to external validity is on whether the approach is applicable to
other platform other than Stack Overflow. This experiment mainly focuses on
Stack Overflow and Java programming language, therefore, it is uncertain
whether FACOS can be applied on other discussion venues that also talk about
API issues. The potential platform might be Reddit, which has sub-reddit
(i.e., a place gathering Reddit’s threads discussing a particular problem)
discussing programming languages and frameworks. It has title, textual and
code content as same as Stack Overflow. Thus, the similarity suggests that we
can potentially apply FACOS to Reddit too. We leave this possibility for a
future work. Regarding the threat that changing targeting programming language
would affect the accuracy of FACOS, although we focus only on Java, the
features (e.g., API comment/documentation, API implementation code, fully
qualified name, class/type name, etc.) required for the approach can be found
from other programming languages. Therefore, we leave this as a future work to
study whether it may work well when applied to other programming languages.

There is a also threat of construct validity on whether precision, recall, and
$F_{\textit{1}}\textit{-}score$ is a suitable evaluation metric for our task.
Our task is a classification task. Many work in software engineering has used
precision, recall, and F1-score as the evaluation metric for classification
task [6, 14, 15, 16]. Thus, we believe the threats are minimal.

##  VIII Related Work

###  VIII-A API Disambiguation

Many past works [14, 17, 18, 19, 20, 21, 22, 23] deals with API
disambiguation. There are two main groups: informal text disambiguation [18,
19, 20, 21, 14] and code snippet disambiguation [22, 23, 17]. As suggested by
its name, the first aims to disambiguate API mentions in textual content while
the second deals with disambiguating API mention in code snippets.

For informal text disambiguation, several work utilizes classical information
retrieval approaches such as Vector Space Model and Latent Semantic Indexing
to disambiguate the API mentions [18, 19, 20] while some others use heuristics
[21]. Bacchelli et al [20] combined string matching and information retrieval
algorithms to link emails to source code entities. Dagenais and Robillard [21]
identified Java APIs mentioned in support channels (e.g., mailing list,
forums), documents, and code snippets. Ye et al. [14] worked on API
disambiguation in the textual content of Stack Overflow thread by utilizing
mention-mention similarity, mention-entry similarity, and scope filter. Luong
et al.[6] used type scoping to disambiguate the API mentions in Stack Overflow
thread.

The work on API disambiguation on Stack Overflow thread can be viewed as
another side of the coin of the task in finding threads that are relevant to
the API. When we disambiguate an API in a thread, the disambiguated API is
relevant to the thread as the thread is talking about the API.

###  VIII-B API Resource Retrieval

Several studies have explored how to search for the code for API and related
information retrieval. Lv et al. [24] proposed Codehow to deal with the lack
of query understanding ability of the existing tool. By expanding a user query
with APIs, Codehow can identify potential APIs and perform a code search based
on the Extended Boolean model, which considers the impact of APIs on code
search. Gu et al. [25] proposed DeepAPI to search for API usage sequences. As
opposed to assuming a bag of words, it learns the sequence of words within a
query and the sequence of APIs associated with it. DeepAPI encodes a single
user query into a fixed-length context vector to generate an API sequence.

Other studies have also exploited different aspects of APIs and natural
language to better retrieve the APIs and their related information. The
techniques include: using global and local contexts of the queries [26],
leveraging usage similarity for effective retrieval of API examples [27],
employing word embeddings to document similarities for improved API retrieval
[28], exploiting user knowledge [29], and task-API knowledge gap [30] during
retrieval of semantically annotated API operations.

Wang et al. [31] developed a transformer-based framework for unifying code
summarization and code search. Shahbazi et al. [32] proposed API2Com to
improve automatically generated code comments by fetching API documentations.
Alhamzeh et al. [33] built DistilBERT-based argumentation retrieval for
answering comparative questions. Dibia et al. [34] and Vale et al.[35]
developed a usable library for question answering with contextual query
expansion and a question-answering assistant for software development using a
transformer-based language model, respectively. Ciniselli et al.[36] performed
an empirical study on the usage of Transformer Models for code search and
completion.

Our study also work on API resource retrieval. Specifically, we retrieve Stack
Overflow threads that are relevant to a target API that we are searching for.

###  VIII-C Contribution of StackOverflow for API documentation

Treude et al. [37] studied the augmenting API documentation with insights from
stack overflow. [4] explored the Crowd documentation by examining the dynamics
of API discussions on Stack Overflow, whereas [38] dubbed the Stack Overflow
as the Social Media for Developer Support in terms of provided utilities. [39]
and [40] worked on classifying stack overflow posts on API issues and
contextual documentation referencing on stack overflow. The dichotomy of these
studies is notable where some research like [41] and [42] studies how API
documentation fails via the API misuse on stack overflow, other studies [43,
44] heavily lean on the Crowdsourced knowledge on stack overflow for automated
API documentation with tutorials. Similarly, crowdsourced knowledge was hailed
by [45] and [46], who explored the innovation diffusion and web resource
recommendation for hyperlinks through link sharing on stack overflow.

Our work support the effort in this line of study. FACOS can automatically
find threads about a particular API in Stack Overflow that can be augmented to
the corresponding API documentation.

###  VIII-D Word Sense and Entity Disambiguation Study

There are several works focused on disambiguation task [21, 22, 47, 23]. We
also have found a variety of word sense and entity disambiguation methods
employed for different objectives [48, 49, 50, 51, 52, 53, 54, 55, 56]. These
studies have solved myriads of problems via solving lexical disambiguations in
literature. The task of word sense disambiguation is to identify a target
word’s intended meaning by examining its context. Researchers have used Word
Sense Disambiguation to predict election results by enhanced sentiment
analysis on Twitter data. Researchers have associated place-name mentions in
unstructured text with their actual references in geographic space using word
disambiguation. Other research has also proposed unsupervised, knowledge-Free,
and interpretable Word Sense Disambiguation for various applications.
Researchers used this approach to add meaning to social network posts when it
comes to named entity recognition and disambiguation. Different Entity-fishing
tools were also developed for facilitating the recognition and disambiguation
service. In recent years, tools that allow researchers to recognize and
extract named entities have become increasingly popular.

##  IX Conclusion and Future Work

We present FACOS, an approach to search Stack Overflow threads that refer to
API of which users or tools may want to find the usage. We utilize the
semantic and syntactic features of the paragraphs and code snippets in a
thread to determine whether the thread is related to a given API. Our
evaluation shows that FACOS has an improvement compared to DATYS when adapting
both approaches to the search task. We have added a weight parameter to
balance the usage of syntactic and semantic information for retrieving API
mentions and related threads. We have proved the utility of the weight factor
by incorporating an ablation study. In future, we plan to improve our approach
with larger dataset which has more threads and APIs. Also, we plan to make our
approach become robust with more programming languages so that it can be more
useful to developers.

Replication Package. The source code for FACOS is available at
https://anonymous.4open.science/r/facos-E5C6/.

## References

  * [1] M. P. Robillard, “What makes apis hard to learn? answers from developers,” _IEEE Software_ , vol. 26, no. 6, pp. 27–34, 2009.
  * [2] P. K. Venkatesh, S. Wang, F. Zhang, Y. Zou, and A. E. Hassan, “What do client developers concern when using web apis? an empirical study on developer forums and stack overflow,” in _2016 IEEE International Conference on Web Services (ICWS)_. IEEE, 2016, pp. 131–138.
  * [3] M. Linares-Vásquez, G. Bavota, M. Di Penta, R. Oliveto, and D. Poshyvanyk, “How do api changes trigger stack overflow discussions? a study on the android sdk,” in _proceedings of the 22nd International Conference on Program Comprehension_ , 2014, pp. 83–94.
  * [4] C. Parnin, C. Treude, L. Grammel, and M.-A. Storey, “Crowd documentation: Exploring the coverage and the dynamics of api discussions on stack overflow,” _Georgia Institute of Technology, Tech. Rep_ , vol. 11, 2012.
  * [5] G. Uddin and F. Khomh, “Automatic mining of opinions expressed about apis in stack overflow,” _IEEE Transactions on Software Engineering_ , 2019.
  * [6] K. Luong, F. Thung, and D. Lo, “Disambiguating mentions of api methods in stack overflow via type scoping,” in _ICSME_. IEEE, 2021.
  * [7] Q. Huang, X. Xia, Z. Xing, D. Lo, and X. Wang, “Api method recommendation without worrying about the task-api knowledge gap,” in _ASE_. IEEE, 2018.
  * [8] M. M. Rahman, C. K. Roy, and D. Lo, “Rack: Automatic api recommendation using crowdsourced knowledge,” in _SANER 2016_ , vol. 1. IEEE, 2016.
  * [9] Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang _et al._ , “Codebert: A pre-trained model for programming and natural languages,” _arXiv preprint arXiv:2002.08155_ , 2020.
  * [10] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” _arXiv preprint arXiv:1810.04805_ , 2018.
  * [11] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in _Advances in neural information processing systems_ , 2017, pp. 5998–6008.
  * [12] K. Clark, M.-T. Luong, Q. V. Le, and C. D. Manning, “Electra: Pre-training text encoders as discriminators rather than generators,” _arXiv preprint arXiv:2003.10555_ , 2020.
  * [13] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” _arXiv preprint arXiv:1907.11692_ , 2019.
  * [14] D. Ye, L. Bao, Z. Xing, and S.-W. Lin, “Apireal: an api recognition and linking approach for online developer forums,” _Empirical Software Engineering_ , vol. 23, no. 6, pp. 3129–3160, 2018.
  * [15] Q. Huang, E. Shihab, X. Xia, D. Lo, and S. Li, “Identifying self-admitted technical debt in open source projects using text mining,” _Empirical Software Engineering_ , vol. 23, no. 1, pp. 418–451, 2018.
  * [16] G. A. A. Prana, C. Treude, F. Thung, T. Atapattu, and D. Lo, “Categorizing the content of github readme files,” _Empirical Software Engineering_ , vol. 24, no. 3, pp. 1296–1327, 2019.
  * [17] C. K. Saifullah, M. Asaduzzaman, and C. K. Roy, “Learning from examples to find fully qualified names of api elements in code snippets,” in _ASE_. IEEE, 2019.
  * [18] G. Antoniol, G. Canfora, G. Casazza, A. De Lucia, and E. Merlo, “Recovering traceability links between code and documentation,” _TSE_ , vol. 28, no. 10, 2002.
  * [19] A. Marcus and J. I. Maletic, “Recovering documentation-to-source-code traceability links using latent semantic indexing,” in _ICSE_. IEEE, 2003.
  * [20] A. Bacchelli, M. Lanza, and R. Robbes, “Linking e-mails and source code artifacts,” in _ICSE_ , 2010.
  * [21] B. Dagenais and M. P. Robillard, “Recovering traceability links between an api and its learning resources,” in _2012 34th international conference on software engineering (icse)_. IEEE, 2012, pp. 47–57.
  * [22] S. Subramanian, L. Inozemtseva, and R. Holmes, “Live api documentation,” in _Proceedings of the 36th International Conference on Software Engineering_ , 2014, pp. 643–652.
  * [23] H. Phan, H. A. Nguyen, N. M. Tran, L. H. Truong, A. T. Nguyen, and T. N. Nguyen, “Statistical learning of api fully qualified names in code snippets of online forums,” in _2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE)_. IEEE, 2018, pp. 632–642.
  * [24] F. Lv, H. Zhang, J.-g. Lou, S. Wang, D. Zhang, and J. Zhao, “Codehow: Effective code search based on api understanding and extended boolean model (e),” in _2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE)_ , 2015, pp. 260–270.
  * [25] X. Gu, H. Zhang, D. Zhang, and S. Kim, “Deep api learning,” in _Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering_ , ser. FSE 2016. New York, NY, USA: Association for Computing Machinery, 2016, p. 631–642. [Online]. Available: https://doi.org/10.1145/2950290.2950334
  * [26] T. Nguyen, N. Tran, H. Phan, T. Nguyen, L. Truong, A. T. Nguyen, H. A. Nguyen, and T. N. Nguyen, “Complementing global and local contexts in representing api descriptions to improve api retrieval tasks,” in _Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ , ser. ESEC/FSE 2018\. New York, NY, USA: Association for Computing Machinery, 2018, p. 551–562. [Online]. Available: https://doi.org/10.1145/3236024.3236036
  * [27] S. K. Bajracharya, J. Ossher, and C. V. Lopes, “Leveraging usage similarity for effective retrieval of examples in code repositories,” in _Proceedings of the Eighteenth ACM SIGSOFT International Symposium on Foundations of Software Engineering_ , ser. FSE ’10. New York, NY, USA: Association for Computing Machinery, 2010, p. 157–166. [Online]. Available: https://doi.org/10.1145/1882291.1882316
  * [28] X. Ye, H. Shen, X. Ma, R. Bunescu, and C. Liu, “From word embeddings to document similarities for improved information retrieval in software engineering,” in _Proceedings of the 38th International Conference on Software Engineering_ , ser. ICSE ’16. New York, NY, USA: Association for Computing Machinery, 2016, p. 404–415. [Online]. Available: https://doi.org/10.1145/2884781.2884862
  * [29] M. Rój, “Exploiting user knowledge during retrieval of semantically annotated api operations,” in _Proceedings of the Fourth Workshop on Exploiting Semantic Annotations in Information Retrieval_ , ser. ESAIR ’11. New York, NY, USA: Association for Computing Machinery, 2011, p. 21–22. [Online]. Available: https://doi.org/10.1145/2064713.2064726
  * [30] Q. Huang, X. Xia, Z. Xing, D. Lo, and X. Wang, “Api method recommendation without worrying about the task-api knowledge gap,” in _2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE)_ , 2018, pp. 293–304.
  * [31] W. Wang, Y. Zhang, Z. Zeng, and G. Xu, “Trans^ 3: A transformer-based framework for unifying code summarization and code search,” _arXiv preprint arXiv:2003.03238_ , 2020.
  * [32] R. Shahbazi, R. Sharma, and F. H. Fard, “Api2com: On the improvement of automatically generated code comments using api documentations,” _arXiv preprint arXiv:2103.10668_ , 2021.
  * [33] A. Alhamzeh, M. Bouhaouel, E. Egyed-Zsigmond, and J. Mitrović, “Distilbert-based argumentation retrieval for answering comparative questions,” _Working Notes of CLEF_ , 2021.
  * [34] V. Dibia, “Neuralqa: A usable library for question answering (contextual query expansion+ bert) on large datasets,” _arXiv preprint arXiv:2007.15211_ , 2020\.
  * [35] L. d. N. Vale and M. d. A. Maia, “Towards a question answering assistant for software development using a transformer-based language model,” _arXiv preprint arXiv:2103.09423_ , 2021.
  * [36] M. Ciniselli, N. Cooper, L. Pascarella, A. Mastropaolo, E. Aghajani, D. Poshyvanyk, M. Di Penta, and G. Bavota, “An empirical study on the usage of transformer models for code completion,” _arXiv preprint arXiv:2108.01585_ , 2021.
  * [37] C. Treude and M. P. Robillard, “Augmenting api documentation with insights from stack overflow,” in _2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE)_. IEEE, 2016, pp. 392–403.
  * [38] M. Squire, “” should we move to stack overflow?” measuring the utility of social media for developer support,” in _2015 IEEE/ACM 37th IEEE International Conference on Software Engineering_ , vol. 2. IEEE, 2015, pp. 219–228.
  * [39] M. Ahasanuzzaman, M. Asaduzzaman, C. K. Roy, and K. A. Schneider, “Classifying stack overflow posts on api issues,” in _2018 IEEE 25th international conference on software analysis, evolution and reengineering (SANER)_. IEEE, 2018, pp. 244–254.
  * [40] S. Baltes, C. Treude, and M. P. Robillard, “Contextual documentation referencing on stack overflow,” _IEEE Transactions on Software Engineering_ , 2020.
  * [41] G. Uddin and M. P. Robillard, “How api documentation fails,” _Ieee software_ , vol. 32, no. 4, pp. 68–75, 2015.
  * [42] T. Zhang, G. Upadhyaya, A. Reinhardt, H. Rajan, and M. Kim, “Are code examples on an online q&a forum reliable?: a study of api misuse on stack overflow,” in _2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE)_. IEEE, 2018, pp. 886–896.
  * [43] S. Meldrum, S. A. Licorish, and B. T. R. Savarimuthu, “Crowdsourced knowledge on stack overflow: A systematic mapping study,” in _Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering_ , 2017, pp. 180–185.
  * [44] A. M. Rocha and M. A. Maia, “Automated api documentation with tutorials generated from stack overflow,” in _Proceedings of the 30th Brazilian Symposium on Software Engineering_ , 2016, pp. 33–42.
  * [45] C. Gómez, B. Cleary, and L. Singer, “A study of innovation diffusion through link sharing on stack overflow,” in _2013 10th Working Conference on Mining Software Repositories (MSR)_. IEEE, 2013, pp. 81–84.
  * [46] J. Li, Z. Xing, D. Ye, and X. Zhao, “From discussion to wisdom: web resource recommendation for hyperlinks in stack overflow,” in _Proceedings of the 31st Annual ACM Symposium on Applied Computing_ , 2016, pp. 1127–1133.
  * [47] A. T. Nguyen, P. C. Rigby, T. Nguyen, D. Palani, M. Karanfil, and T. N. Nguyen, “Statistical translation of english texts to api code templates,” in _2018 IEEE International Conference on Software Maintenance and Evolution (ICSME)_. IEEE, 2018, pp. 194–205.
  * [48] T. Steiner, R. Verborgh, J. Gabarró Vallés, and R. Van de Walle, “Adding meaning to social network microposts via multiple named entity disambiguation apis and tracking their data provenance,” _International Journal of Computer Information Systems and Industrial Management_ , vol. 5, pp. 69–78, 2013.
  * [49] M. Karimzadeh, W. Huang, S. Banerjee, J. O. Wallgrün, F. Hardisty, S. Pezanowski, P. Mitra, and A. M. MacEachren, “Geotxt: A web api to leverage place references in text,” in _Proceedings of the 7th Workshop on Geographic Information Retrieval_ , ser. GIR ’13. New York, NY, USA: Association for Computing Machinery, 2013, p. 72–73. [Online]. Available: https://doi.org/10.1145/2533888.2533942
  * [50] S. Patwardhan, S. Banerjee, and T. Pedersen, “Senserelate:: Targetword-a generalized framework for word sense disambiguation,” in _ACL_ , vol. 2005, 2005, pp. 73–76.
  * [51] R. Jose and V. S. Chooralil, “Prediction of election result by enhanced sentiment analysis on twitter data using word sense disambiguation,” in _2015 International Conference on Control Communication Computing India (ICCC)_ , 2015, pp. 638–641.
  * [52] L. Foppiano and L. Romary, “entity-fishing: a dariah entity recognition and disambiguation service,” _Journal of the Japanese Association for Digital Humanities_ , vol. 5, no. 1, pp. 22–60, 2020.
  * [53] S. Zwicklbauer, C. Seifert, and M. Granitzer, “Do we need entity-centric knowledge bases for entity disambiguation?” in _Proceedings of the 13th International Conference on Knowledge Management and Knowledge Technologies_ , 2013, pp. 1–8.
  * [54] A. Mandalios, K. Tzamaloukas, A. Chortaras, and G. Stamou, “Geek: Incremental graph-based entity disambiguation,” in _LDOW@ WWW_ , 2018.
  * [55] D. Klein, K. Toutanova, H. T. Ilhan, S. D. Kamvar, and C. D. Manning, “Combining heterogeneous classifiers for word sense disambiguation,” in _Proceedings of the ACL-02 workshop on Word sense disambiguation: recent successes and future directions_ , 2002, pp. 74–80.
  * [56] P. Chen, W. Ding, C. Bowes, and D. Brown, “A fully unsupervised word sense disambiguation method using dependency knowledge,” in _Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics_ , 2009, pp. 28–36.