# Lessons learned from hyper-parameter tuning for microservice candidate
identification

Rahul Yedida1, Rahul Krishna2, Anup Kalia2, Tim Menzies1, Jin Xiao2, Maja
Vukovic2
1Department of Computer Science, NC State, USA; 2IBM Research, USA
<EMAIL_ADDRESS>{rkrsn<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

When optimizing software for the cloud, monolithic applications need to be
partitioned into many smaller microservices. While many tools have been
proposed for this task, we warn that the evaluation of those approaches has
been incomplete; e.g. minimal prior exploration of hyperparameter
optimization. Using a set of open source Java EE applications, we show here
that (a) such optimization can significantly improve microservice
partitioning; and that (b) an open issue for future work is how to find which
optimizer works best for different problems. To facilitate that future work,
see https://github.com/yrahul3910/ase-tuned-mono2micro for a reproduction
package for this research.

###### Index Terms:

microservices, hyper-parameter optimization

##  I Introduction

Traditional software is “monolithic”; i.e. one large entity that handles all
concerns. Such software needs to be divided into “microservices” to take full
advantage of the flexibility and scalability offered by cloud computing
environments [15]. That microservice architecture allows for independent
scalability, the use of different programming languages for each microservice,
and does not have a single point of failure.

In our experience, the process of manually refactoring a monolithic
application to a microservice architecture can be a time-consuming, expensive,
and an error-prone endeavor. Different architects may propose different
candidate sets when manually constructing a set of microservices. Also,
different software systems may have different inherent architectures. Lastly,
different engineers may have differing opinions on the number of microservices
to split the software into. Hence, like many other researchers [11, 8, 12, 16]
we explore publicly available machine learning methods and algorithms to
identify a candidate set of microservices from a monolithic application. But
even here, there are still problems. For example, as shown in column one of
Table I, there are multiple methods to find these partitions. Also, as seen in
column two of that table, these methods contain one or more parameters that
must be set via engineering judgment. However, different software architects
(and different types of applications) may have different views on the best
values for these configurations.

TABLE I: Hyper-parameter choices studied in this paper. All the methods take runtime traces (or uses cases) as their input Algorithm |  Hyper-parameter | Range
---|---|---
mono2micro [12] |  Number of clusters | $\\{2,3,\ldots\\}$
FoSCI [11] |  Number of clusters | $\\{2,3,\ldots\\}$
Number of iterations to run NSGA-II | $\\{1,2,\ldots\\}$
Population size for NSGA-II | $\\{5,6,\ldots\\}$
Parent size to use in NSGA-II | $\\{5,6,\ldots\\}$
Threshold for clustering to stop | $\mathbb{R}^{+}$
MEM [16] |  Maximum partition size | $\\{2,3,\ldots\\}$
Number of partitions | $\\{2,3,\ldots\\}$
Bunch [17] |  Number of partitions | $\\{2,3,\ldots\\}$
Initial population size for hill climbing | $\\{2,3,\ldots\\}$
Number of neighbors to consider in hill-climbing iterations | $[0,1]$
CoGCN [8] |  Number of clusters | $\\{2,3,\ldots\\}$
Loss function coefficients $\alpha_{1},\alpha_{2},\alpha_{3}$ | $\alpha_{i}\in\mathbb{R}$
Number of hidden units in each layer, $h_{1},h_{2}$ | $h_{i}\in\mathbb{Z}^{+}$
Dropout rate | $d\in[0,1)$

We characterize all these issues as hyperparameter optimization (HPO); i.e.
the automatic selection of (a) which features/algorithms to use; and (b) what
settings to apply to those algorithms. In our case, the algorithms are the
methods used to find partitions or microservices. Using the partitioning
methods in Table I the paper compares off-the-shelf partitioning methods to
two tuning (i.e., HPO) methods. The rest of the paper does the following:

  * •

Documents the improvement seen to microservice creation when the methods of
Table I are tuned via HPO. Specifically, tuning can significantly improve
results, with the highest being 266% (FoSCI on the plants dataset, on the SM
metric).

  * •

Reveals a new research challenge microservice design strategies. While our
results shows that tuning is better than non-tuning, we also show that no
single approach is a clear winner across multiple datasets or multiple
performance goals.

In summary, the lessons learned from this work are:

  1. 1.

Do not use these partitioning methods off-the-shelf. since these these tend to
have subpar results.

  2. 2.

The choice of tuning algorithm is not as important as the decision to tune.
The random and hyperopt optimizers performed differently on different metrics
and datasets, but in a broader view, shared the same number of highest Scott-
Knott rankings. Therefore, the decision to tune is important, but the choice
of algorithm may not be as important.

  3. 3.

More research is required on tuning. This work suggest that a new research
challenge is appropriate: microservice design strategies. While our results
shows that tuning is better than non-tuning, we also show that no single
approach is a clear winner across multiple datasets or multiple performance
goals.

  4. 4.

There is no best partitioning algorithm. We will show later in this paper that
the priorities of the stakeholders matters when choosing the right algorithm,
and that there is no one best algorithm across all the metrics.

More generally, going forward, we say that industrial cloud applications need
to understand and apply automatic tuning methods. That said, we will conclude
that it is still an open issue which tuner to use. More research is needed in
that direction.

##  II Business Case

A recent report shows that only 20% of the enterprise workloads are in the
cloud, and they were predominately written for native cloud architectures [6].
This leaves 80% of legacy applications on-premises, waiting to be refactored
and modernized for the cloud. Once ported to microservices, business services
can be independently enhanced and scaled, providing agility and improved speed
of delivery.

Application refactoring is the process of restructuring existing code into a
set of smaller independent groups of code, ideally without changing the
external behavior and semantics. Currently, refactoring is usually done
manually and is expensive, time-consuming, and error-prone. There are several
methods to implement application refactoring. For example, Table I lists
different ways to partition a monolith. In the table, the core idea is to
understand what parts of the code call each other other (e.g., by reflecting
on test cases or uses cases), then cluster those parts of the code into
separate microservices.

Software remodularization aims to repair the architectural design of a
software system to reduce the technical debt. Remodularization tools such as
Bunch [17] perform such refactoring with the underlying assumption that
developers want to maximize cohesion and minimize coupling [5].

As opposed to service-oriented architectures, the construction of
microservices (as explored here) is motivated by enterprises’ need to refactor
their code into independent services, guided by some automatically-inferred
partitions. The resulting application architecture may or may not conform
perfectly to SOA, as the partitions are reversed engineered from test cases or
use cases describing taken from some pre-exisiting monolithic application. The
benefit is these microservices can be created relatively cheaply and built
incrementally by porting existing monolithic code to the cloud. Well observed
practices in the industry, such as the strangler pattern, follows this
approach.

##  III Background

Several approaches have been proposed to solve this problem. These may broadly
be categorized into (a) Optimization (or search) based approaches, and (b)
unsupervised clustering based approaches. We provide a brief overview of each
of these approaches below.

###  III-A Unsupervised Clustering Based Partitioning

####  III-A1 Mono2Micro

This approach, introduced by Kalia et al. [12], uses hierarchical
decomposition that dynamically collects runtime traces, under the execution of
specific business use cases of the application. Then, using Jaccard similarity
and hierarchical clustering over the runtime traces, the applications classes
are segregated into disjoint partitions.

####  III-A2 Microservice Extractor (aka. MEM)

Mazlami et al. [16] propose MEM, which first constructs a graph based on
logical, semantic, and change history based coupling. Next, they pose
microservice partitioning as a graph cut problem that uses minimum spanning
trees to find min-cut partitions from the constructed graph. Classes in each
min-cut represents a candidate microservice partition.

###  III-B Optimization/Search Based Partitioning

Some authors prefer to cast software partitioning as an optimization problem,
whose objective is to optimize a linear combination of the quality metrics.
Our reading of the literature is that the following list of methods samples
much of the current state-of-the-art in the research arena.

####  III-B1 CoGCN

Desai et al. [8] proposed a method called CO-GCN, that uses graph
convolutional networks (GCN) [13] with a loss function that incorporates
several metrics from Tab. II directly the end-to-end training phase. The
partitions are then discovered using the k-means clustering approach.

####  III-B2 FoSCI

Jin et al. [11] propose FoSCI, which uses runtime traces as its data source.
They first remove redundant traces using a trace reduction algorithm, and then
perform hierarchical clustering to create “functional atoms” (logical units
that provide some function). These functional atoms are then merged using a
multi-objective optimization algorithm, namely NSGA-II [7].

####  III-B3 Bunch

Mitchell and Mancoridis [17] proposed Bunch, where the partition problem is
characterized as a search problem. Computational search procedures are then
used to find a partition that maximizes the modular quality (MQ) and the
structural modular quality (SMQ) metrics. In particular, they use the hill
climbing search algorithm [17, §4.1].

###  III-C Hyper-parameter optimization

We note that across all of these approaches, a common theme is the existence
of hyper-parameters, listed in Table I. Because these papers are focused on
proposing a novel approach, little attention is paid to tuning these hyper-
parameters, or discussing reasonable defaults for them Therefore, we ask
whether tuning can improve the results.

Hyper-parameter optimizers find best suited parameters (i.e., meta-options)
that guide the specifics of an algorithm’s execution. These include parameters
such as the number of clusters for a partitioning or a clustering algorithm,
or the number of layers in a deep learner.

Bergstra and Bengio [2] advocate for random hyper-parameter tuning, in part,
due to the ease of implementation. A more sophisticated optimizer is the Tree
of Parzen Estimators (TPE) [3] that uses an “acquisition function” to reflect
on the model built so far in order to learn the next most informative sample
to be evaluated. Prior results are divided into best and rest and each
division is modelled as a Gaussian distribution. Candidate evaluations are
then ranked by how well they select for best and avoid rest. The better
candidates are then evaluated which, in turn, updates best and rest and their
associated distributions. We use the hyperopt [14] implementation of the TPE
algorithm, and in the rest of the paper, we use “hyperopt” to refer to “TPE”.

TABLE II: The metrics used to assess the quality of microservice partitions. In the first column, a $\Downarrow$ means we want to minimize the metric; a $\Uparrow$ means that higher values are better. Metric | Name |  Description |  Goal
---|---|---|---
BCP $\Downarrow$ [12] | Business context purity |  Mean entropy of business use cases per partition |  If minimized then more business cases handled locally
ICP $\Downarrow$ [12] | Inter-partition call percentage |  Percentage of runtime calls that are between partitions |  If minimized then less traffic between clusters
SM $\Uparrow$ [11] | Structural modularity |  A combination of cohesion and coupling defined by Jin et al. [11] |  If maximized then more self-contained clusters with fewer connections between them
MQ $\Uparrow$ [17] | Modular quality |  A ratio involving cohesion and coupling, defined by Mitchell and Mancoridis [17] |  If maximized then more processing is local to a cluster
IFN $\Downarrow$ [17] | Interface number |  Number of interfaces needed in the microservice architecture |  If minimized then fewer calls between clusters

Several papers [9, 18, 1, 19] demonstrate that the hyper-parameters learned
via such optimizers can lead to models that significantly out-perform the
untuned (or off-the-shelf) defaults. Yet, none of the approaches we study do
such tuning. We investigate this research gap by using two of the optimizers
recommended by Bergstra et al. [3]: random and TPE. Both of these optimizers
have been endorsed in major venues (NeurIPS) and have since been widely used
by the community.

TABLE III: Summary of Results. Each column is a separate clustering method.
Each row is a different performance metric. The final row highlights the wins
across all metrics.

(a) mono2micro Metric | Treatment | Wins
---|---|---
BCP $\Downarrow$ | untuned | 0
random |  4
hyperopt |  4
ICP $\Downarrow$ | untuned |  4
random | 0
hyperopt | 0
SM $\Uparrow$ | untuned |  2
random |  3
hyperopt |  3
MQ $\Uparrow$ | untuned |  1
random |  3
hyperopt |  3
IFN $\Downarrow$ | untuned |  1
random |  4
hyperopt |  4
Total | untuned |  1
tuned | 4
tie | 0

(b) FoSCI Metric | Treatment | Wins
---|---|---
BCP $\Downarrow$ | untuned | 0
random | 0
hyperopt |  4
ICP $\Downarrow$ | untuned |  1
random |  3
hyperopt | 0
SM $\Uparrow$ | untuned | 0
random |  4
hyperopt | 0
MQ $\Uparrow$ | untuned | 0
random | 0
hyperopt |  4
IFN $\Downarrow$ | untuned |  1
random |  4
hyperopt |  1
Total | untuned | 0
tuned | 5
tie | 0

(c) MEM Metric | Treatment | Wins
---|---|---
BCP $\Downarrow$ | untuned |  3
random | 0
hyperopt |  1
ICP $\Downarrow$ | untuned |  2
random |  2
hyperopt | 0
SM $\Uparrow$ | untuned |  3
random | 0
hyperopt |  1
MQ $\Uparrow$ | untuned |  1
random |  2
hyperopt |  4
IFN $\Downarrow$ | untuned |  2
random |  2
hyperopt |  1
Total | untuned | 2
tuned |  1
tie |  2

(d) Bunch Metric | Treatment | Wins
---|---|---
BCP $\Downarrow$ | untuned |  2
random |  3
hyperopt |  3
ICP $\Downarrow$ | untuned |  2
random |  4
hyperopt |  4
SM $\Uparrow$ | untuned |  4
random |  2
hyperopt |  2
MQ $\Uparrow$ | untuned |  3
random |  3
hyperopt |  3
IFN $\Downarrow$ | untuned |  2
random |  4
hyperopt |  4
Total | untuned |  1
tuned | 3
tie |  1

(e) CoGCN Metric | Treatment | Wins
---|---|---
BCP $\Downarrow$ | untuned |  2
random | 0
hyperopt |  2
ICP $\Downarrow$ | untuned | 0
random | 0
hyperopt |  4
SM $\Uparrow$ | untuned | 0
random |  2
hyperopt |  2
MQ $\Uparrow$ | untuned |  4
random | 0
hyperopt | 0
IFN $\Downarrow$ | untuned |  1
random | 0
hyperopt |  3
Total | untuned |  1
tuned | 3
tie |  1

TABLE IV: Another summary of the results. Here, we show the best algorithm for an dataset for a metric (i.e., the best of untuned, random, and hyperopt). Where untuned had the best result, we mark it with an asterisk. A blue background indicates the best result. The last row denotes the number of wins for each algorithm across all projects. Dataset | Algorithm | BCP $\Downarrow$ | ICP $\Downarrow$ | SM $\Uparrow$ | MQ $\Uparrow$ | IFN $\Downarrow$
---|---|---|---|---|---|---
jpetstore | mono2micro | 1.67 | 0.51* | 0.06* | 3.29 | 3.77
Bunch | 2.43 | 0.48 | 0.21 | 7.95 | 2.64
CO-GCN | 1.97* | 0.15 | 0.19 | 6.11* | 1.8
MEM | 2.69* | 0.47* | 0.23* | 4.56 | 4
FoSCI | 2.04 | 0.04 | 0.12 | 3.63 | 1
plants | mono2micro | 1.73 | 0.51* | 0.1 | 2.82* | 5.1
Bunch | 1.75 | 0.51 | 0.22* | 4.71 | 3.88
CO-GCN | 1.87 | 0.02 | 0.27 | 3.51* | 1
MEM | 2.04 | 0.32 | 0.25 | 2.83 | 4.17
FoSCI | 2.04 | 0.08 | 0.11 | 3.63 | 3
daytrader | mono2micro | 1.4 | 0.4* | 0.07 | 5.52 | 6.13
Bunch | 1.75 | 0.51 | 0.27* | 11.33* | 3.88
CO-GCN | 1.02* | 0.01 | 0.15 | 7.99* | 0.5
MEM | 1.94* | 0.38 | 0.11 | 4.73 | 4.08
FoSCI | 2.04 | 0.04 | 0.12 | 0.12 | 1
acmeair | mono2micro | 1.04 | 0.4* | 0.1 | 3.44 | 3.3
Bunch | 1.67* | 0.51 | 0.17 | 4.71 | 3.88
CO-GCN | 1.31 | 0.29 | 0.15 | 3.71* | 3
MEM | 1.92* | 0.66 | 0.09 | 2.77* | 4
Wins | mono2micro | 3 | 0 | 0 | 0 | 0
Bunch | 0 | 0 | 2 | 4 | 0
CO-GCN | 1 | 2 | 1 | 0 | 2
MEM | 0 | 1 | 1 | 0 | 1
FoSCI | 0 | 2 | 0 | 0 | 2

##  IV Experimental Setup

We run each algorithm using the default settings (untuned) and using the
settings obtained from hyper-parameter optimizations (tuned). We run each
algorithm for 30 times both for tuned and untuned settings.

Data. We use four open-source projects to test the algorithms under study.
These are: (1) daytrader111https://github.com/WASdev/sample.daytrader7, a
sample online stock trading system; (2)
plants222https://github.com/WASdev/sample.plantsbywebsphere, a sample online
store to purchase plants and pets; (3)
acmeair333https://github.com/acmeair/acmeair, a sample airline booking
application; and (4) jpetstore444https://github.com/mybatis/jpetstore-6, a
sample pet store. These applications are representative web apps built using
various Java EE frameworks such as Spring, Apache Struts, etc.

Metrics. For a fair comparison with prior work, we must compare using the same
set of metrics. We choose a total of five metrics to evaluate our core
hypothesis that hyper-parameter tuning improves microservice extraction
algorithms. These are detailed in Table II. These metrics have been used in
prior studies, although different papers used different metrics in their
evaluations. For fairness, we use metrics from all prior papers (cf. Table
II). These metrics evaluate different aspects of the utility of an algorithm
that might be more useful to different sets of users (detailed in the RQ1),
e.g., BCP evaluates how well different business use cases are separated across
the microservices.

Hyper-parameter optimization. We use two hyper-parameter optimization
approaches, random sampling [2], and Tree of Parzen Estimators (TPE) [3],
using the hyperopt framework [4]. These are two commonly used approaches from
the AI literature, and have been compared against in recent SE literature [1].
We run both for 100 iterations. We use two hyper-parameter optimization
approaches for the following reasons. Firstly, it avoids discussing the choice
of hyper-parameter optimizers by demonstrating the use of two different
approaches. Our argument is for the use of hyper-parameter optimization in
general, as opposed to a specific approach.

Because we use several metrics, we need to guide the hyper-parameter
optimization algorithms to achieve the results that we want. To achieve this,
we define a loss function $\ell(P)=\sum\limits_{i=1}^{5}\alpha_{i}m_{i}(P)$,
where $P$ is the partition, $m_{i}$ are the five metrics. For simplicity, we
set each weight to 1 if the goal is to minimize the metric, and -1 if the goal
is to maximize the metric. Having framed our loss function, we use the hyper-
parameter optimization algorithms to minimize this goal. We note that this is
not necessarily the best loss function; but it is the simplest.

Statistics. Our comparison method of choice for the results is the Scott-Knott
test, which was endorsed at ICSE’15 [10]. The Scott-Knott test is a recursive
bi-clustering algorithm that terminates when the difference between the two
split groups is insignificant. Scott-Knott searches for split points that
maximize the expected value of the difference between the means of the two
resulting groups. The result of the Scott-Knott test is ranks assigned to each
result set (two results have the same rank if their difference is
insignificant).

##  V Results

To evaluate the outcome of hyperparameter tuning for microservice candidate
identification, we ask and answer the following thse research questions:

  * •

RQ1.  How does hyperparameter tuning affect microservice candidate
identification?

  * •

RQ2.  Is there an exemplary approach that outperforms the others across
multiple datasets?

For brevity we show only the summary tables in this paper. We provide the
details online555https://github.com/yrahul3910/tuned-microservices-results.

RQ1. How does hyperparameter tuning affect microservice candidate
identification?

Table III shows our results. In that figure, each row shows results from one
assessment metric (BCP, ICP, SM, MQ, IDM) and each column shows results from a
different technology from Table 1. Each result is a three bars chart showing
how often the untuned, random tuned, and TPE (hyperopt) tuner achieved best
results first (where “best” was determined by our statistical tests) within
our four data sets (daytrader, plants, acmeair, jpetstore). For example, top-
left, we see that random and hyperopt both achieved best BCP scores for
mono2micro.

The general trend across all the results is (a) untuned usually lost to
tuning; but (b) there is no stable pattern about which tuner (random or
hyperopt) worked best for a particular performance measure. The exception to
this general pattern is that (c) untuned MEM which performed best in 3/5
metrics (BCP, ICP, SM). Our investigation of MEM revealed that it ignores its
hyperparameter options and uses its own internal methods for deciding actual
parameters.

Summary.  Tuning is better than not tuning, but it is still an open issue is a
particular tuner works better.

RQ2. Is there an exemplary approach that outperforms the others across
multiple datasets?

Our review of different approaches in §II showed a variety of techniques
applied to the problem. These tools were all available as open-source, and
could be easily run and evaluated. Table IV shows a summary view of the
results. Here, we pick the best among the untuned and tuned results (from RQ1)
and compare the algorithms across all the datasets and metrics. We see that
there is no one best algorithm, and that the “best” algorithm depends on the
specific use case and metric used to reflect that use case. For example, a
business user might be more interested in the BCP metric (i.e., the business
cases are more cleanly separated), and would be recommended the mono2micro
algorithm. Similarly, a systems engineer who wants low coupling (i.e., low ICP
and low IFN) would choose the CoGCN or the FoSCI approach. Therefore, the
conclusion is that the right algorithm depends on the user goals and
priorities.

Summary. The best partitioning algorithm depends on the business use case of
interest to the practitioners.

## References

  * Agrawal et al. [2021] A. Agrawal, X. Yang, R. Agrawal, R. Yedida, X. Shen, and T. Menzies, “Simpler hyperparameter optimization for software analytics: Why, how, when,” _IEEE Transactions on Software Engineering_ , 2021.
  * Bergstra and Bengio [2012] J. Bergstra and Y. Bengio, “Random search for hyper-parameter optimization.” _Journal of machine learning research_ , vol. 13, no. 2, 2012.
  * Bergstra et al. [2011] J. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl, “Algorithms for hyper-parameter optimization,” in _25th annual conference on neural information processing systems (NIPS 2011)_ , vol. 24, 2011.
  * Bergstra et al. [2013] J. Bergstra, D. Yamins, D. D. Cox _et al._ , “Hyperopt: A python library for optimizing the hyperparameters of machine learning algorithms,” in _Proceedings of the 12th Python in science conference_ , vol. 13. Citeseer, 2013.
  * Candela et al. [2016] I. Candela, G. Bavota, B. Russo, and R. Oliveto, “Using cohesion and coupling for software remodularization: Is it enough?” _ACM Transactions on Software Engineering and Methodology (TOSEM)_ , vol. 25, no. 3, pp. 1–28, 2016\.
  * Daya et al. [2016] S. Daya, N. Van Duy, K. Eati, C. Ferreira, D. Glozic, V. Gucer, M. Gupta, S. Joshi, V. Lampkin, M. Martins _et al._ , _Microservices from Theory to Practice: Creating Applications in IBM Bluemix Using the Microservices Approach_. IBM Redbooks, 2016\. [Online]. Available: https://books.google.com/books?id=eOZyCgAAQBAJ
  * Deb et al. [2002] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist multiobjective genetic algorithm: Nsga-ii,” _IEEE transactions on evolutionary computation_ , vol. 6, no. 2, pp. 182–197, 2002.
  * Desai et al. [2021] U. Desai, S. Bandyopadhyay, and S. Tamilselvam, “Graph neural network to dilute outliers for refactoring monolith application,” _arXiv preprint arXiv:2102.03827_ , 2021.
  * Fu et al. [2016] W. Fu, T. Menzies, and X. Shen, “Tuning for software analytics: Is it really necessary?” _Information and Software Technology_ , vol. 76, pp. 135–146, 2016.
  * Ghotra et al. [2015] B. Ghotra, S. McIntosh, and A. E. Hassan, “Revisiting the impact of classification techniques on the performance of defect prediction models,” in _2015 IEEE/ACM 37th IEEE International Conference on Software Engineering_ , vol. 1. IEEE, 2015, pp. 789–800.
  * Jin et al. [2019] W. Jin, T. Liu, Y. Cai, R. Kazman, R. Mo, and Q. Zheng, “Service candidate identification from monolithic systems based on execution traces,” _IEEE Transactions on Software Engineering_ , 2019.
  * Kalia et al. [2020] A. K. Kalia, J. Xiao, C. Lin, S. Sinha, J. Rofrano, M. Vukovic, and D. Banerjee, “Mono2micro: an ai-based toolchain for evolving monolithic enterprise applications to a microservice architecture,” in _Proceedings of the 28th ACM Joint Meeting on ESEC/FSE_ , 2020, pp. 1606–1610.
  * Kipf and Welling [2016] T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” _arXiv preprint arXiv:1609.02907_ , 2016.
  * Komer et al. [2014] B. Komer, J. Bergstra, and C. Eliasmith, “Hyperopt-sklearn: automatic hyperparameter configuration for scikit-learn,” in _ICML workshop on AutoML_ , vol. 9. Citeseer, 2014, p. 50.
  * Larrucea et al. [2018] X. Larrucea, I. Santamaria, R. Colomo-Palacios, and C. Ebert, “Microservices,” _IEEE Software_ , vol. 35, no. 3, pp. 96–100, 2018.
  * Mazlami et al. [2017] G. Mazlami, J. Cito, and P. Leitner, “Extraction of microservices from monolithic software architectures,” in _2017 IEEE International Conference on Web Services (ICWS)_. IEEE, 2017, pp. 524–531.
  * Mitchell and Mancoridis [2006] B. S. Mitchell and S. Mancoridis, “On the automatic modularization of software systems using the bunch tool,” _IEEE Transactions on Software Engineering_ , vol. 32, no. 3, pp. 193–208, 2006.
  * Nair et al. [2018] V. Nair, Z. Yu, T. Menzies, N. Siegmund, and S. Apel, “Finding faster configurations using flash,” _IEEE Transactions on Software Engineering_ , vol. 46, no. 7, pp. 794–811, 2018.
  * Yedida and Menzies [2021] R. Yedida and T. Menzies, “On the value of oversampling for deep learning in software defect prediction,” _IEEE Transactions on Software Engineering_ , 2021.