# OTSU BASED DIFFERENTIAL EVOLUTION METHOD FOR IMAGE SEGMENTATION

Afreen Shaikh
National Institute of Technology, Warangal
<EMAIL_ADDRESS>
Botcha Sharmila
National Institute of Technology, Warangal
<EMAIL_ADDRESS>
&Murali Krishna
National Institute of Technology, Warangal
<EMAIL_ADDRESS>
&Dr. Sushil Kumar
National Institute of Technology, Warangal
<EMAIL_ADDRESS>

###### Abstract

This paper proposes an OTSU-based differential evolution method for satellite
image segmentation and compares it with four other methods such as Modified
Artificial Bee Colony Optimizer (MABC)[1], Artificial Bee Colony (ABC),
Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) using the
objective function proposed by Otsu for optimal multilevel thresholding. The
experiments conducted and their results illustrate that our proposed DE+OTSU
algorithm segmentation can effectively and precisely segment the input image,
close to results obtained by the other methods. In the proposed DE+OTSU
algorithm, instead of passing the fitness function variables, the entire image
is passed as an input to the DE algorithm after obtaining the threshold values
for the input number of levels in the OTSU’s algorithm. The image segmentation
results are obtained after learning about the image instead of learning about
the fitness variables. In comparison to other segmentation methods examined,
the proposed DE+OTSU algorithm yields promising results with minimized
computational time comparison to some algorithms.

##  1 INTRODUCTION

Image segmentation has a vital part as a pre-processing step in image
processing. In particular, it aims at grouping and partitioning pixels within
meaningful regions to analyze the image in a detailed manner. Image
segmentation applies to varied Computer Vision tasks like Content-based image
retrieval, Medical Imaging, Object detection and identification, and Object
recognition tasks (like face and fingerprint), among others.

Segmentation methods can be subdivided broadly as (a) edge and line-oriented
segmentation methods, (b) region growing methods, (c) clustering, and (d)
region splitting methods. Ant Colony Optimisation (ACO), Particle Swarm
Optimization (PSO), Bacterial Foraging (BF), and Differential Evolution (DE)
are regarded as popular choices for the segmentation of complex images due to
their exhaustive searching capability. Because of its clearness and stability,
thresholding is coined as one of the most popular image segmentation methods.
Typically, satellite images have insignificant illumination features because
of multiple kinds of environmental distributions. They contain diverse objects
(regions), like vegetation, water bodies, and territory. However, these areas
lack clear demarcation because of low spatial resolution. Hence, segmenting
various land cover parts in these images is a complicated task.

One of the key aspects of image segmentation is thresholding (Agrawal et al.,
2013)[12]. Many algorithms to perform global level thresholding can be found
in the existing literature which aim at segmenting images and extract
compelling and meaningful patterns (Rosin, 2001; Portesde et al., 2004; Zahara
et al., 2005).

Otsu in the year 1979 [13] presented a method for selecting threshold from
grey-level histograms. But, inadequate articulation of between-class variance
raises the cost of computation of the algorithm, specifically in the selection
of multi-level threshold. Otsu’s thresholding aims at automatic threshold
selection and region-based segmentation and is one of the most accurate
techniques for image thresholding because of its simple calculation.
Differential Evolution is a reasonably recent population-based evolutionary
model which depends on the mutation operation as its prominent step. It
efficiently explores large search spaces and exhibits superior results in the
following criteria: (1) shorter convergence speed than other evolutionary
algorithms, (2) fewer number of parameter adjustments, making it particularly
easy to implement.

Tsai (1985)[14] uses the moment-preserving principle to determine thresholds
of grey-level input images - the Tsallis entropy technique, a prominent
function used for image thresholding. Kittler et al.(1986) work with the
assumption that the grey levels of every entity within an image are usually
distributed. Kapur et al. (1985) further introduced a robust method for grey-
level image thresholding by utilizing the histogram entropy (Kapur et al.,
1985)[15].

Entropy-based approaches have earned popularity among all the distinctive
image thresholding methods. PCO, influenced by the social behaviours like
flocking of birds or schooling of fishes (Akay et al., 2013[16]; Maitra et al.
2008[17]; Yin, et al. 2007[18]), ant colony optimization (ACO) influenced by
the pasturing conduct of ant colonies (Ye et al., 2005[20]; Tao et al.,
2007[19]) Artificial Bee Colony (ABC) inspired by the pasturing conduct of
honey bee swarm (Cuevaset al., 2012 [21], Akay et al., 2013;).

Another recent area of research inspired by the coordinated intelligence in a
swarm of insects or animals has also emerged called Swarm intelligence. One of
the widespread and recently developed Swarm Intelligence-based techniques is
the ABC algorithm, presented by Karaboga D. (2005)[22], that simulates the
foraging behavior of honey bee colonies. Numerous studies have demonstrated
that the performance of Artificial Bee Colony (ABC) method is competitive and
comparable with other population-based techniques.

The MABC method given by Bhandari et al.(2014) introduces an advanced solution
search formula (Gao et al., 2012) in charge for its more promising search
solution. Within this search equation, the bee explores only closer to the
best solution of the iteration before to improve exploitation. The results
demonstrate that MABC is best suited for multilevel thresholding of images to
attain optimal thresholds for satellite images in comparison to Artificial Bee
Colony based algorithms and Particle Swarm Optimization based techniques.
Evaluating the advantages, such algorithms are the preferred choice for
finding the optimum thresholds in simple images. For example, the Genetic
Algorithm (GA) and the improved Genetic Algorithm (Zhang et al., 2014 [23])
are frequently in use to multilevel thresholding scenarios.

The Swarm Intelligence based computing methods can find efficient and optimal
solutions for any objective function and have been widely used with an ability
to generate highly accurate results in case of complex problems as well. It
has also been found by statistical analysis that the Swarm Intelligence based
algorithms perform well in multi-level thresholding scenarios (Kurban et al.,
2014 [24]).

##  2 BRIEF EXPLANATION OF THE ALGORITHMS USED IN THIS STUDY

###  2.1 Otsu’s thresholding

####  2.1.1 Bi-level thresholding

In Bi-level thresholding, the aim is to find a threshold to minimize the
variance between classes in the segmented image. Otsu’s algorithm tends to
achieve better results when two different peaks are present in the histogram
of the original image, one corresponding to the background and the other to
the foreground. The entire range of pixels are iterated and the Otsu’s
threshold is determined when the between-class variances are minimum. Hence,
the Otsu’s threshold tends to be decided by the class with greater
variance.Hence, Otsu’s method tends to produce sub-optimal results when there
is an occurrence of two or more peaks within the histogram of the image or if
one of the classes has a significant variance.

The total mean and variance are calculated based on the following formulas:
The entire set of pixels are distributed into 2 classes,

C1 $\leftarrow$ pixels having grey levels [1, t]
C2 $\leftarrow$ pixels having grey levels $[t+1,...,L]$.
where $t$ corresponds to Otsu’s threshold

The probability distribution of the two classes is denoted by:

$C_{1}:{\frac{p_{1}}{w_{1}(t)},...,\frac{p_{r}}{w_{1}(t)}}$ (1)

and

$C_{2}:{\frac{p_{r+1}}{w_{2}(t)},...,\frac{p_{L}}{w_{2}(t)}}$ (2)

Where, $w_{1}(t)=\sum_{i=1}^{r}p_{i}$ and $w_{2}(t)=\sum_{i=r+1}^{L}p_{i}$

The class mean for the two classes $\mu_{1}$ and $\mu_{2}$ are defined by:

$\mu_{1}=\sum_{i=1}^{r}\frac{i*p_{i}}{w_{1}(t)}$ (3)

$\mu_{2}=\sum_{i=r+1}^{L}\frac{i*p_{i}}{w_{2}(t)}$ (4)

Otsu’s between-class variance based on discriminate analysis of the threshold
image is defined as:

$\sigma^{2}_{B}=w_{1}(\mu_{1}-\mu_{T})^{2}+w_{2}(\mu_{2}-\mu_{T})^{2}$ (5)

For bi-level thresholding, the optimal threshold t* is chosen to maximize
between-class variance n, i.e.

$t^{*}=arg_{t<l}max{\sigma^{2}_{B}(t)}$ (6)

####  2.1.2 Multilevel Thresholding

The first step in the algorithm is to obtain the Otsu’s threshold and the
class means of the two classes divided by the Otsu’s threshold. Further, in
multi-level thresholding, the pixels of the image are divided into three
categories instead of two categories determined by Otsu’s threshold. The three
categories correspond to the following: (a) the ’foreground’ region - group of
pixel with values >= the larger mean, (b) the ’background’ region - group of
pixels with values <= to the smaller mean and (c) the ’to-be-determined (TBD)’
region - group of pixels with values between the two class means.

In iteration $i+1$, the algorithm retains the ’foreground’ and ’background’
regions from iteration $i$ and re-applies the Otsu’s method only on the ’to-
be-determined’ region to further divide it into three classes again. When this
iteration stops after satisfying a pre-defined criterion, the final ’to-be-
determined (TBD)’ region is then divided into two classes - foreground and
background instead of three. Lastly, the foreground regions from all the
iterations are combined to get the final foreground class and the final
background region is also determined likewise.

For an image represented by $L$ no.of grey levels $0,1,...,L-1$, we could
develop the image histogram $H=\\{f_{0},f_{1},...,f_{L-1}\\}$, where $f_{i}$
is the frequency of grey level $i$ in the image. Let

$N=\sum_{i=0}^{L-1}f_{i}$

determine the total no. of pixels in the image. The occurrence probability of
$i^{th}$ grey level is defined by:

$p_{i}=\frac{f_{i}}{N}$ (7)

It can be easily be illustrated that $p_{i}\geq 0$ and
$\sum_{i=0}^{L-1}p_{i}=1$.

Otsu’s algorithm divides the image into $K+1$ clusters $\\{C0,C1,...,C_{K}\\}$
using $K$ no. of thresholds chosen from the set
$T=\\{(t_{1},t_{2},...,t_{K})|0<t1<...<t_{K}<L\\}$ where $C_{K}$ is the set of
pixels with grey levels $\\{t_{K},t_{K}+1,...,t_{K+1}-1\\}$. Where $t_{0}=0$
and $t_{K+1}=L$. For every cluster $C_{K}$, the cumulative probability $w_{K}$
and mean grey level $\mu_{K}$ are defined by:

$w_{k}=\sum_{i\epsilon C_{k}},k\epsilon\\{0,1,2,...,K\\}$ (8)

$\mu_{k}=\sum_{i\epsilon
C_{k}}\frac{i*p_{i}}{w_{k}},k\epsilon\\{0,1,2,...,K\\}$ (9)

The mean intensity of the whole image $\mu_{T}$ and the between-class variance
$\sigma_{B}^{2}$ are defined by:

$\mu_{T}=\sum_{k=0}^{K}w_{k}*\mu_{k}=\sum_{i=0}^{L-1}i*p_{i}$ (10)

and

$\sigma_{B}^{2}=\sum_{k=0}^{K}w_{k}*(\mu_{k}-\mu_{T})^{2}=\sum_{k=0}^{K}w_{k}*\mu_{k}^{2}-\mu_{T}^{2}$
(11)

In Otsu’s method, the threshold levels for each cluster are selected on the
basis of maximizing the variance among means of the cluster (Huang et al.,
2011). The optimal thresholds at maximum between-class variance are determined
by:

$(t_{1}^{*},t_{2}^{*},...,t_{K}^{*})=\\{\sigma_{B}^{2}(t_{1},t_{2},...,t_{K})\\}$
(12)

##  3 DIFFERENTIAL EVOLUTION (DE)

In comparison to other evolutionary algorithms, the Differential Evolution
(DE) algorithm aims to reserve the global search strategy based on population
and employs a simple mutation function of the differential and one-on-one
competition, reducing the operation’s genetic complexity. At the same time,
the specific memory ability of DE allows it to dynamically track the current
search to adjust its search strategy with robust global convergence. Hence, it
is suited for the complex environments of the optimization problem.
Fundamental operations like selection, crossover, and mutation are the
foundation of the Differential Evolution algorithm.

###  3.1 How DE is used in the proposed algorithm?

Initially, the image to be segmented is made to run through OTSU’s multilevel
thresholding, on the basis of the number of levels($L$) the image has to be
segmented. The output of this step yields us with ‘$L$’ threshold values which
act as partitions for the pixel intensities. These partitions divide the pixel
intensity range (0-255) into ‘$L+1$’ clusters. Based on the pixels belonging
to the image that fall into a particular cluster, the cluster centre is
defined using the mean of the pixels (falling in that cluster).

Now, the image to be segmented is given as input to the DE algorithm. Each of
the ‘$G$’ generations has Np number of populations, where population size is
equal to the count of the image pixels. After each generation, the best
population is evaluated based on the least value of fitness function, which is
a measure to the MSE (mean square error) distance of each pixel in the
population to its corresponding cluster centre. The next generations are
populated using this best population as described later in the algorithm. The
best population of the last generation is again reshaped into image format,
which form the segmented image.

##  4 PROPOSED ALGORITHM

###  4.1 Algorithm pre-processing

In our implementation an input image and the number of levels ($L$) are given
as input to the algorithm which would return the cluster partition values
$(cp_{1},cp_{2},...,cp_{L})$.

Cluster ranges, $(0,cp_{1}),(cp_{1}+1,cp_{2}),...,(cp_{L}+1,255)$ are computed
from the cluster partition values obtained after the thresholding of the
image.

Cluster centres, $(cc_{1},cc_{2},...,cc_{L+1})$ are calculated as follows :

$cc_{i}=\frac{\sum pix^{i}}{|pix^{i}|}$ (13)

Where,

$pix^{i}$ = pixels in the cluster range $(0,cp_{i})$

$|pix^{i}|$ = pixels in the cluster range $(0,cp_{i})$

###  4.2 Differential Evolution

Consider the image has ‘$m$’ pixels $(pix_{1},pix_{2},..,pix_{m})$, and
algorithm runs for ‘$G$’ generations with ‘$Np$’ no. of populations per
generation.

$X_{n,i}^{g}$ : $i^{th}$ pixel value of $n^{th}$ population in $g^{th}$
generation

$X_{n}^{g}:[X_{n,1}^{g},X_{n,2}^{g},...,X_{n,m}^{g}]$ (14)

$n:1\rightarrow Np(populationindex)$

$g:1\rightarrow G(generationindex)$

$i:1\rightarrow m(pixelindex)$

Algorithm 1 An algorithm with caption

$n=0$

$i=0$

while $n\neq 0$ do

while $i\neq P$ do

$[Xlow,Xhigh]\leftarrow clusterrangeforpixel,i$

$X^{1}_{n,i}\leftarrow X_{low}+rand(0,1)*[X_{high}-X_{low}]$

end while

end while

$n=0$

while $n\neq 0$ do

$MSE_{X^{g=1}_{n}}=\frac{1}{n}\sum_{i=1}^{m}(X_{n,i}^{1}-C_{i})^{2}$

$X_{best}^{1}\leftarrow populationwithminimumfitnessvalue$

end while
Mutant vector :
for each population n in generation g+1

$V_{n}^{g+1}\leftarrow X_{best}^{g}+F*(X_{r1}^{g}-X_{r2}^{g})$

if $rand(0,1)<=Cr$ then

$U_{n}^{g+1}\leftarrow V_{n}^{g+1}$

else if $rand(0,1)>Cr$ then

$U_{n}^{g+1}\leftarrow X_{n}^{g}$

end if

if $fitness(U_{n}^{g+1}<fitness(X_{n}^{g})$ then

$X_{n}^{g+1}\leftarrow U_{n}^{g+1}$

else if $fitness(U_{n}^{g+1}>fitness(X_{n}^{g})$ then

$X_{n}^{g+1}\leftarrow X_{n}^{g}$

end if

Fitness function and mutant vector calculation will be repeated for G
generations. After G generations the $X_{best}^{G}$ population is considered
the solution to image segmentation and the ‘m’ pixels in $X_{best}^{G}$ is
converted into output image

##  5 RESULTS

###  5.1 Performance Metrics

This section aims at presenting results gathered from various satellite images
using the proposed OTSU based differential evolution algorithm are discussed
and those results are set side by side the results obtained from the Modified
Artificial Bee Colony algorithm (MABC) [11], Artificial Bee Colony algorithm
(ABC), Particle Swarm Optimization algorithm (PSO) and Genetic Algorithm (GA).
Along with quality estimation factor of Peak Signal-to-Noise Ratio (PSNR)
,algorithm efficiency (based on CPU Timing) and feature assessment is also
measured using Structural Similarity Index measure (SSIM).

Table 6-14 compares the PSNR, SSIM and CPU time values gathered by applying
the proposed DE+OTSU method and set side by side for comparison with the
result obtained using Modified Artificial Bee Colony algorithm (MABC),
Artificial Bee Colony algorithm (ABC), Particle Swarm Optimization algorithm
(PSO) and Genetic Algorithm (GA) methods respectively.

###  5.2 Input Images and References

(a)

(b)

(c)

Figure 1: Satellite images used to perform and compare the experiments
(a) an image from the NASA Earth Observatory, Yellowstone National Park, June
2009 (b)Pléiades Satellite Image: Shanghai, China (c)Pléiades Satellite Image:
Kabul Airport, Afghanistan

###  5.3 Parameters configuration in the Proposed Algorithm

Table 1: Parameters used in DE+OTSU Parameters | Value
---|---
No. of generations and population size | 10 and 100
Crossover and mutation probability | 0.2, 0.3
Image size | 250*250 pixels
Table 2: Parameters configuration in ABC Parameters | Value
---|---
Swam Size and iterations | 10 and 100
Lower and Upper bounds $(lb(X_{min}),ub(X_{max}))$ | 1 and 256
Trial limit at max | 10
$F_{i}(\phi)$ | [0,1]
Table 3: Parameters configuration in MABC Parameters | Value
---|---
Swam Size | 10
Iterations (Max FE) | 100
No. of Iterations with chaos | 300
Lower and Upper bounds $(lb(X_{min}),ub(X_{max}))$ | 1 and 256
Table 4: Parameters configuration in PSO Parameters | Value
---|---
Swam Size and iterations | 200 and 100
Cognitive, social and neighbourhood acceleration (C1, C2, C3) | 2,2,1
Lower and Upper bounds $(lb(X_{min}),ub(X_{max}))$ | 1 and 256
Error goal and Trial limit at max |  $1e^{-}7$ and 500
Starting velocity weight | 0.95
Ending velocity weight | 0.4
The proportion of iterations, for linear varied W | 0.7
Constriction Factor, Maximum Velocity Step and Neighborhood Size | 1
Global Minima | 0
Table 5: Parameters configuration in GA Parameters | Value
---|---
Size of the population and iterations | 10 and 100
Crossover and mutation probability | 0.5 and 0.1
Bits used in variable representation | [8]
$eta(\eta)$ | 1
Lower and Upper bounds $(lb(X_{min}),ub(X_{max}))$ | 1 and 256

###  5.4 Comparison of Output Images

Table 6 : Comparison between Modified Artificial Bee Colony algorithm (MABC),
Artificial Bee Colony algorithm (ABC), Particle Swarm Optimization algorithm
(PSO) and Genetic Algorithm (GA) algorithms using Kapur’s entropy with
proposed DE+OTSU based on CPU timing (seconds).

$\begin{array}[]{|c|c|l|c|c|c|c|}\hline\cr\hbox{\multirowsetup\text{ Image
}}&\hbox{\multirowsetup\text{ Level }}&\hbox{\multirowsetup\text{ DE+OTSU
}}&\vrule\lx@intercol\hfil\text{ KAPUR'S ENTROPY
}\hfil\lx@intercol\vrule\lx@intercol\\\ \cline{4-7}\cr&&&\text{ MABC }&\text{
ABC }&\text{ PSO }&\text{ GA }\\\ \hline\cr\hbox{\multirowsetup
1}&2&10.62&2.843121&6.890611&21.00694&185.94096\\\
\cline{2-7}\cr&3&10.96&2.48574&6.641995&22.24080&181.80301\\\
\cline{2-7}\cr&4&8.26&2.528806&6.636246&21.84823&184.04964\\\
\cline{2-7}\cr&5&9.92&3.181302&6.661542&23.07188&121.46275\\\
\hline\cr\hbox{\multirowsetup
2}&2&12.57&2.947659&6.510813&24.00778&268.01849\\\
\cline{2-7}\cr&3&10.56&2.487444&6.295622&23.27194&266.77671\\\
\cline{2-7}\cr&4&8.69&2.621718&7.009322&23.85787&268.79580\\\
\cline{2-7}\cr&5&10.65&2.733291&6.570533&24.95336&183.32426\\\
\hline\cr\hbox{\multirowsetup 3}&2&9.93&2.81911&8.235545&25.22988&436.11369\\\
\cline{2-7}\cr&3&11.36&3.286868&8.075897&24.80887&433.87366\\\
\cline{2-7}\cr&4&8.11&2.612452&6.917417&24.86667&438.91758\\\
\cline{2-7}\cr&5&10.34&2.821316&8.247721&25.73357&296.15088\\\
\hline\cr\end{array}$

Table 7 : Comparison between Modified Artificial Bee Colony algorithm (MABC),
Artificial Bee Colony algorithm (ABC), Particle Swarm Optimization algorithm
(PSO) and Genetic Algorithm (GA) algorithms using Kapur’s entropy with
proposed DE+OTSU based on PSNR (dB) [25] values.

$\begin{array}[]{|c|l|l|l|l|l|l|}\hline\cr\hbox{\multirowsetup\text{ Image
}}&\hbox{\multirowsetup\text{ Level }}&\hbox{\multirowsetup\text{ DE+OTSU
}}&\vrule\lx@intercol\hfil\text{ KAPUR'S ENTROPY
}\hfil\lx@intercol\vrule\lx@intercol\\\ \cline{4-7}\cr&&&\text{ MABC }&\text{
ABC }&\text{ PSO }&\text{ GA }\\\ \hline\cr\hbox{\multirowsetup
1}&2&16.43&24.54&24.53&24.55&24.58\\\
\cline{2-7}\cr&3&15.53&24.76&24.79&24.74&24.756\\\
\cline{2-7}\cr&4&20.16&24.97&25.01&24.969&25.89\\\
\cline{2-7}\cr&5&18.94&25.82&25.76&25.188&25.676\\\
\hline\cr\hbox{\multirowsetup 2}&2&16.77&24.6265&24.606&24.575&24.591\\\
\cline{2-7}\cr&3&18.85&24.80&24.801&24.84&24.811\\\
\cline{2-7}\cr&4&20.12&25.024&25.008&25.008&25.979\\\
\cline{2-7}\cr&5&21.71&25.542&25.238&26.106&25.54\\\
\hline\cr\hbox{\multirowsetup 3}&2&14.36&24.64&24.659&24.649&24.691\\\
\cline{2-7}\cr&3&19.53&24.83&24.86&24.84&24.90\\\
\cline{2-7}\cr&4&20.99&25.065&25.039&25.063&25.978\\\
\cline{2-7}\cr&5&20.54&24.5451&24.542&24.551&24.582\\\ \hline\cr\end{array}$

Table 8 : Comparison between Modified Artificial Bee Colony algorithm (MABC),
Artificial Bee Colony algorithm (ABC), Particle Swarm Optimization algorithm
(PSO) and Genetic Algorithm (GA) algorithms using Kapur’s entropy with
proposed DE+OTSU based on SSIM [25] values.

$\begin{array}[]{|c|c|l|c|c|c|c|}\hline\cr\hbox{\multirowsetup\text{ Image
}}&\hbox{\multirowsetup\text{ Level }}&\hbox{\multirowsetup\text{ DE+OTSU
}}&\vrule\lx@intercol\hfil\text{ KAPUR\textquoteright S ENTROPY
}\hfil\lx@intercol\vrule\lx@intercol\\\ \cline{4-7}\cr&&&\text{ MABC }&\text{
ABC }&\text{ PSO }&\text{ GA }\\\ \hline\cr\hbox{\multirowsetup
1}&2&0.37&0.900747&0.865230&0.936475&0.941127\\\
\cline{2-7}\cr&3&0.312&0.966901&0.953207&0.965857&0.962966\\\
\cline{2-7}\cr&4&0.59&0.97696&0.970332&0.977355&0.969124\\\
\cline{2-7}\cr&5&0.492&0.989512&0.981282&0.984123&0.979080\\\
\hline\cr\hbox{\multirowsetup 2}&2&0.53&0.886746&0.877771&0.945456&0.945627\\\
\cline{2-7}\cr&3&0.59&0.958891&0.949243&0.967704&0.965602\\\
\cline{2-7}\cr&4&0.761&0.976986&0.974446&0.979841&0.975580\\\
\cline{2-7}\cr&5&0.672&0.985457&0.978247&0.985389&0.982706\\\
\hline\cr\hbox{\multirowsetup
3}&2&0.421&0.918842&0.902112&0.943819&0.947394\\\
\cline{2-7}\cr&3&0.672&0.961164&0.959592&0.968163&0.963107\\\
\cline{2-7}\cr&4&0.73&0.978343&0.961763&0.979184&0.976280\\\
\cline{2-7}\cr&5&0.704&0.989280&0.977779&0.985351&0.981165\\\
\hline\cr\end{array}$

Table 9 : Comparison between Modified Artificial Bee Colony algorithm (MABC),
Artificial Bee Colony algorithm (ABC), Particle Swarm Optimization algorithm
(PSO) and Genetic Algorithm (GA) algorithms using Between-class variance with
proposed DE+OTSU based on CPU timing (seconds).

$\begin{array}[]{|c|c|l|c|c|c|c|}\hline\cr\hbox{\multirowsetup\text{ Image
}}&\hbox{\multirowsetup\text{ Level }}&\hbox{\multirowsetup\text{ DE+OTSU
}}&\vrule\lx@intercol\hfil\text{ BETWEEN CLASS VARIANCE
}\hfil\lx@intercol\vrule\lx@intercol\\\ \cline{4-7}\cr&&&\text{ MABC }&\text{
ABC }&\text{ PSO }&\text{ GA }\\\ \hline\cr\hbox{\multirowsetup
1}&2&10.62&2.251826&2.511814&13.23507&121.92735\\\
\cline{2-7}\cr&3&10.96&2.383504&2.495365&4.30062&118.78825\\\
\cline{2-7}\cr&4&8.26&2.243267&2.596011&4.000213&119.73264\\\
\cline{2-7}\cr&5&9.92&2.228959&2.434094&4.336584&119.81306\\\
\hline\cr\hbox{\multirowsetup
2}&2&12.57&3.244759&3.504219&5.748055&174.33693\\\
\cline{2-7}\cr&3&10.56&3.437252&3.588553&5.722739&174.18969\\\
\cline{2-7}\cr&4&8.69&3.440058&3.744837&5.670794&174.57630\\\
\cline{2-7}\cr&5&10.65&3.406708&3.732828&5.793264&173.69577\\\
\hline\cr\hbox{\multirowsetup 3}&2&9.93&4.667568&5.007778&7.29691&272.89039\\\
\cline{2-7}\cr&3&11.36&5.363554&5.276051&6.587852&275.03171\\\
\cline{2-7}\cr&4&8.11&5.076405&5.519536&7.154403&272.78504\\\
\cline{2-7}\cr&5&10.34&4.784902&5.427858&6.753919&275.44405\\\
\hline\cr\end{array}$

Table 10 : Comparison between Modified Artificial Bee Colony algorithm (MABC),
Artificial Bee Colony algorithm (ABC), Particle Swarm Optimization algorithm
(PSO) and Genetic Algorithm (GA) algorithms using Between-class variance with
proposed DE+OTSU based on PSNR (dB) [25] values.

$\begin{array}[]{|c|c|c|c|c|c|c|}\hline\cr\hbox{\multirowsetup\text{ Image
}}&\hbox{\multirowsetup\text{ Level }}&\hbox{\multirowsetup\text{ DE+OTSU
}}&\vrule\lx@intercol\hfil\text{ BETWEEN CLASS VARIANCE
}\hfil\lx@intercol\vrule\lx@intercol\\\ \cline{4-7}\cr&&&\text{ MABC }&\text{
ABC }&\text{ PSO }&\text{ GA }\\\ \hline\cr\hbox{\multirowsetup
1}&2&16.43&24.56352&24.579535&24.546290&24.552551\\\
\cline{2-7}\cr&3&15.53&24.780481&24.759637&24.732067&24.764200\\\
\cline{2-7}\cr&4&20.16&24.981269&25.006653&25.012021&24.874078\\\
\cline{2-7}\cr&5&18.94&25.627989&25.4729200&25.197580&25.004700\\\
\hline\cr\hbox{\multirowsetup
2}&2&16.77&24.621500&24.583300&24.585789&24.570418\\\
\cline{2-7}\cr&3&18.85&24.800300&24.794900&24.836735&24.786600\\\
\cline{2-7}\cr&4&20.12&24.968400&25.004600&24.973612&24.974895\\\
\cline{2-7}\cr&5&21.71&25.771400&25.300500&25.234673&25.158200\\\
\hline\cr\hbox{\multirowsetup
3}&2&14.36&24.653841&24.648178&24.644845&24.645697\\\
\cline{2-7}\cr&3&19.53&24.856153&24.870026&24.859707&24.870600\\\
\cline{2-7}\cr&4&20.99&25.067745&25.054861&25.066102&26.279928\\\
\cline{2-7}\cr&5&20.54&25.769200&25.082472&25.293798&25.124000\\\
\hline\cr\end{array}$

Table 11 : Comparison between Modified Artificial Bee Colony algorithm (MABC),
Artificial Bee Colony algorithm (ABC), Particle Swarm Optimization algorithm
(PSO) and Genetic Algorithm (GA) algorithms using Between-class variance with
proposed DE+OTSU based on SSIM [25] values.

$\begin{array}[]{|c|c|c|c|c|c|c|}\hline\cr\hbox{\multirowsetup\text{ Image
}}&\hbox{\multirowsetup\text{ Level }}&\hbox{\multirowsetup\text{ DE+OTSU
}}&\vrule\lx@intercol\hfil\text{ BETWEEN CLASS VARIANCE
}\hfil\lx@intercol\vrule\lx@intercol\\\ \cline{4-7}\cr&&&\text{ MABC }&\text{
ABC }&\text{ PSO }&\text{ GA }\\\ \hline\cr\hbox{\multirowsetup
1}&2&0.37&0.907199&0.898708&0.9341685&0.936426\\\
\cline{2-7}\cr&3&0.312&0.958494&0.950182&0.9613643&0.964400\\\
\cline{2-7}\cr&4&0.59&0.97805&0.976883&0.9771952&0.924582\\\
\cline{2-7}\cr&5&0.492&0.984679&0.978082&0.9837941&0.972000\\\
\hline\cr\hbox{\multirowsetup
2}&2&0.53&0.885600&0.869500&0.9450091&0.995003\\\
\cline{2-7}\cr&3&0.59&0.958300&0.957800&0.9666987&0.962600\\\
\cline{2-7}\cr&4&0.761&0.975500&0.978500&0.9784969&0.960120\\\
\cline{2-7}\cr&5&0.672&0.981600&0.979800&0.9806148&0.947300\\\
\hline\cr\hbox{\multirowsetup
3}&2&0.421&0.920762&0.904131&0.9423958&0.942725\\\
\cline{2-7}\cr&3&0.672&0.963812&0.968801&0.9687778&0.962500\\\
\cline{2-7}\cr&4&0.73&0.971162&0.972597&0.9778467&0.955299\\\
\cline{2-7}\cr&5&0.704&0.982051&0.967690&0.9819275&0.95570000\\\
\hline\cr\end{array}$

Table 12 : Comparison between Modified Artificial Bee Colony algorithm (MABC),
Artificial Bee Colony algorithm (ABC), Particle Swarm Optimization algorithm
(PSO) and Genetic Algorithm (GA) algorithms using Tsallis entropy with
proposed DE+OTSU based on CPU timing (seconds).

$\begin{array}[]{|l|l|l|l|l|c|c|}\hline\cr\hbox{\multirowsetup\text{ Image
}}&\hbox{\multirowsetup\text{ Level }}&\hbox{\multirowsetup\text{ DE+OTSU
}}&\vrule\lx@intercol\hfil\text{ TSALLIS ENTROPY
}\hfil\lx@intercol\vrule\lx@intercol\\\ \cline{4-7}\cr&&&\text{ MABC
}&\vrule\lx@intercol\hfil\text{ ABC
}\hfil\lx@intercol\vrule\lx@intercol&\text{ PSO }&\text{ GA }\\\
\hline\cr\hbox{\multirowsetup 1}&2&10.62&4.191848&9.2953&5.285757&1287.455\\\
\cline{2-7}\cr&3&10.96&4.28949&10.4439&5.165659&1243.575\\\
\cline{2-7}\cr&4&8.26&5.003336&9.4859&5.261193&0129.445\\\
\cline{2-7}\cr&5&9.92&5.064827&8.9783&5.146160&0122.208\\\
\hline\cr\hbox{\multirowsetup 2}&2&12.57&4.687289&9.9036&7.547844&1804.159\\\
\cline{2-7}\cr&3&10.56&5.839503&8.8693&6.745230&1813.836\\\
\cline{2-7}\cr&4&8.69&5.344366&9.3367&6.553589&0178.122\\\
\cline{2-7}\cr&5&10.65&6.290270&9.0729&7.058079&0180.25\\\
\hline\cr\hbox{\multirowsetup 3}&2&9.93&6.721415&8.9362&7.839406&2672.361\\\
\cline{2-7}\cr&3&11.36&6.710296&10.7472&7.785332&2644.285\\\
\cline{2-7}\cr&4&8.11&6.903615&10.5427&7.210223&297.826\\\
\cline{2-7}\cr&5&10.34&7.755440&10.1722&7.334682&263.15\\\
\hline\cr\end{array}$

Table 13 : Comparison between Modified Artificial Bee Colony algorithm (MABC),
Artificial Bee Colony algorithm (ABC), Particle Swarm Optimization algorithm
(PSO) and Genetic Algorithm (GA) algorithms using Tsallis entropy with
proposed DE+OTSU based on PSNR (dB) [25] values.

$\begin{array}[]{|l|l|l|c|c|c|c|}\hline\cr\hbox{\multirowsetup\text{ Image
}}&\hbox{\multirowsetup\text{ Level }}&\hbox{\multirowsetup\text{ DE+OTSU
}}&\vrule\lx@intercol\hfil\text{ TSALLIS ENTROPY
}\hfil\lx@intercol\vrule\lx@intercol\\\ \cline{4-7}\cr&&&\text{ MABC }&\text{
ABC }&\text{ PSO }&\text{ GA }\\\ \hline\cr\hbox{\multirowsetup
1}&2&16.43&24.377843&24.351485&24.519494&24.104899\\\
\cline{2-7}\cr&3&15.53&24.636986&24.577900&24.657204&25.929835\\\
\cline{2-7}\cr&4&20.16&24.960974&24.751543&24.982344&25.065115\\\
\cline{2-7}\cr&5&18.94&25.448985&25.553804&24.860717&26.549726\\\
\hline\cr\hbox{\multirowsetup
2}&2&16.77&24.48804&24.584723&24.650583&24.747813\\\
\cline{2-7}\cr&3&18.85&24.866677&24.582997&24.559801&25.454637\\\
\cline{2-7}\cr&4&20.12&24.964328&24.782696&25.054277&25.260387\\\
\cline{2-7}\cr&5&21.71&26.76419&25.037394&25.151130&26.946907\\\
\hline\cr\hbox{\multirowsetup
3}&2&14.36&24.649292&24.477749&24.655701&25.288265\\\
\cline{2-7}\cr&3&19.53&24.829821&24.644722&24.769577&25.408871\\\
\cline{2-7}\cr&4&20.99&24.865398&24.870877&25.056663&25.196562\\\
\cline{2-7}\cr&5&20.54&25.093037&24.858501&25.064660&26.927766\\\
\hline\cr\end{array}$

Table 14 : Comparison between Modified Artificial Bee Colony algorithm (MABC),
Artificial Bee Colony algorithm (ABC), Particle Swarm Optimization algorithm
(PSO) and Genetic Algorithm (GA) algorithms using Tsallis entropy with
proposed DE+OTSU based on SSIM [25] values.

$\begin{array}[]{|l|l|l|c|c|c|c|}\hline\cr\hbox{\multirowsetup\text{ Image
}}&\hbox{\multirowsetup\text{ Level }}&\hbox{\multirowsetup\text{ DE+OTSU
}}&\vrule\lx@intercol\hfil\text{ TSALLIS ENTROPY
}\hfil\lx@intercol\vrule\lx@intercol\\\ \cline{4-7}\cr&&&\text{ MABC }&\text{
ABC }&\text{ PSO }&\text{ GA }\\\ \hline\cr\hbox{\multirowsetup
1}&2&0.37&0.873413&0.848868&0.904213&0.877822\\\
\cline{2-7}\cr&3&0.312&0.938499&0.927630&0.854607&0.922841\\\
\cline{2-7}\cr&4&0.59&0.964424&0.946915&0.962464&0.961415\\\
\cline{2-7}\cr&5&0.492&0.979107&0.978985&0.944744&0.964881\\\
\hline\cr\hbox{\multirowsetup 2}&2&0.53&0.8708&0.839311&0.888775&0.902111\\\
\cline{2-7}\cr&3&0.59&0.93286&0.847479&0.848088&0.945562\\\
\cline{2-7}\cr&4&0.761&0.96772&0.948136&0.931402&0.963708\\\
\cline{2-7}\cr&5&0.672&0.978641&0.969851&0.977266&0.924099\\\
\hline\cr\hbox{\multirowsetup
3}&2&0.421&0.923064&0.860053&0.906366&0.877859\\\
\cline{2-7}\cr&3&0.672&0.939928&0.915346&0.891213&0.816022\\\
\cline{2-7}\cr&4&0.73&0.960756&0.943489&0.942365&0.935665\\\
\cline{2-7}\cr&5&0.704&0.968972&0.953955&0.964825&0.967876\\\
\hline\cr\end{array}$

##  6 CONCLUSIONS

In this study, an algorithm for image segmentation task based on a combination
of Differential evolution and Otsu’s multi-level thresholding was developed
and the results were experimentally examined against three different satellite
images. The study also explored the comparison between the proposed algorithm
and the developed MABC[11], ABC, PSO, GA using Kapur’s entropy, Between-class
variance and Tsallis entropy (one at a time) based on the three satellite
images based on PSNR, SSIM and CPU time parameters. The proposed Differential
evolution based Otsu method(DE+OTSU) yields promising results which are close
to the results obtained from the other algorithms (MABC, ABC and PSO) and we
observed to be better than Genetic algorithm(GA). On proper parameter setting,
there is a positive probability of increased performance results when using
the proposed algorithm.

DE+OTSU

(q)

(r)

(s)

(t)

Figure 2: Segmented images obtained for 1st satellite image using MABC, ABC,
PSO, GA with Kapur’s entropy and proposed DE+OSU methods. (a-d): levels 2-5
from MABC using Kapur’s entropy. (e-h): levels 2-5 from ABC using Kapur’s
entropy. (i-l): levels 2-5 from PSO using Kapur’s entropy. (m-p): levels 2-5
from GA using Kapur’s entropy. (q-t): levels 2-5 from proposed DE+OTSU
algorithm.

DE+OTSU

(q)

(r)

(s)

(t)

Figure 3: Segmented images obtained for 1st satellite image using
MABC,ABC,PSO,GA with Between-class variance and proposed DE+OSU methods. (a-d)
: levels 2-5 from MABC using Between-class variance. (e-h) : levels 2-5 from
ABC using Between-class variance. (i-l) : levels 2-5 from PSO using Between-
class variance.(m-p) : levels 2-5 from GA using Between-class variance. (q-t)
: levels 2-5 from proposed DE+OTSU algorithm.

DE+OTSU

(q)

(r)

(s)

(t)

Figure 4: Segmented images obtained for 1st satellite image using
MABC,ABC,PSO,GA with Tsallis entropy and proposed DE+OSU methods. (a-d) :
levels 2-5 from MABC using Tsallis entropy. (e-h) : levels 2-5 from ABC using
Tsallis entropy. (i-l) : levels 2-5 from PSO using Tsallis entropy.(m-p) :
levels 2-5 from GA using Tsallis entropy. (q-t) : levels 2-5 from proposed
DE+OTSU algorithm.

levels 2-5 from MABC using Kapur’s entropy.

levels 2-5 from MABC using Between-Class variance.

levels 2-5 from MABC using Tsallis entropy.

(a)

(b)

(c)

(d)

levels 2-5 from proposed DE+OTSU algorithm.

Figure 5: Segmented images obtained for 2nd satellite image using MABC and
proposed DE+OSU methods.

(r)

(s)

(t)

Figure 6: Segmented images obtained for 3rd satellite image using
MABC,ABC,PSO,GA with Kapur’s entropy and proposed DE+OSU methods.(a-d) :
levels 2-5 from MABC using Kapur’s entropy.(e-h) : levels 2-5 from ABC using
Kapur’s entropy.(i-l) : levels 2-5 from PSO using Kapur’s entropy.(m-p) :
levels 2-5 from GA using Kapur’s entropy.(q-t) : levels 2-5 from proposed
DE+OTSU algorithm.

DE+OTSU

(q)

(r)

(s)

(t)

Figure 7: Segmented images obtained for 3rd satellite image using
MABC,ABC,PSO,GA with Between-class variance and proposed DE+OSU methods. (a-d)
: levels 2-5 from MABC using Between-class variance. (e-h) : levels 2-5 from
ABC using Between-class variance. (i-l) : levels 2-5 from PSO using Between-
class variance.(m-p) : levels 2-5 from GA using Between-class variance. (q-t)
: levels 2-5 from proposed DE+OTSU algorithm.

DE+OTSU

(q)

(r)

(s)

(t)

Figure 8: Segmented images obtained for 3rd satellite image using
MABC,ABC,PSO,GA with Tsallis entopy and proposed DE+OSU methods. (a-d) :
levels 2-5 from MABC using Tsallis entopy. (e-h) : levels 2-5 from ABC using
Tsallis entopy. (i-l) : levels 2-5 from PSO using Tsallis entopy.(m-p) :
levels 2-5 from GA using Tsallis entopy. (q-t) : levels 2-5 from proposed
DE+OTSU algorithm.

DE+OTSU

(q)

## References

  * [1] P.N.Varalakshmi.K, D. Uma Devi, 2012, Image segmentation of Dermatitis Images Using Differential Evolution, INTERNATIONAL JOURNAL OF ENGINEERING RESEARCH & TECHNOLOGY (IJERT) Volume 01, Issue 08 (October 2012)
  * [2] Cuevas, Erik & Zaldivar, Daniel & Cisneros, Marco. (2016). Image Segmentation Based on Differential Evolution Optimization. 10.1007/978-3-319-26462-2-2.
  * [3] Storn, R., Price, K. Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces. Journal of Global Optimization 11, 341–359 (1997).
  * [4] Hamed Soleimani, Govindan Kannan, A hybrid particle swarm optimization and genetic algorithm for closed-loop supply chain network design in large-scale networks, Applied Mathematical Modelling, Volume 39, Issue 14, 2015, Pages 3990-4012, ISSN 0307-904X, https://doi.org/10.1016/j.apm.2014.12.016. (https://www.sciencedirect.com/science/article/pii/S0307904X14006842)
  * [5] S. Choubey, "Genetic algorithm, particle swam optimization and differential evolution algorithm for optimal hydro generation scheduling," 2017 International Conference on Intelligent Computing and Control Systems (ICICCS), 2017, pp. 486-491, doi: 10.1109/ICCONS.2017.8250770.
  * [6] Pham Xuan, Thuy & Siarry, Patrick & Oulhadj, Hamouche. (2018). Integrating fuzzy entropy clustering with an improved PSO for MRI brain image segmentation. Applied Soft Computing. 65. 10.1016/j.asoc.2018.01.003.
  * [7] I. Khanfir Kallel, S. Almouahed, B. Solaiman, É. Bossé, An iterative possibilistic knowledge diffusion approach for blind medical image segmentation, Pattern Recognition, Volume 78, 2018, Pages 182-197, ISSN 0031-3203
  * [8] Priya M.S and Dr. G.M. Kadhar Nawaz, "Multilevel Image Thresholding using OTSU’s algorithm in image segmentation", International Journal of Scientific & Engineering Research Volume 8, Issue 5, May-2017
  * [9] Ta Yang Goh, Shafriza Nisha Basah, Haniza Yazid, Muhammad Juhairi Aziz Safar, Fathinul Syahir Ahmad Saad, Performance analysis of image thresholding: Otsu technique, Measurement, Volume 114, 2018, Pages 298-307, ISSN 0263-2241, https://doi.org/10.1016/j.measurement.2017.09.052.
  * [10] Kachitvichyanukul, Voratas. (2012). Comparison of three evolutionary algorithms: GA, PSO, and DE. Industrial Engineering and Management Systems. 12. 215-223. 10.7232/iems.2012.11.3.215.
  * [11] A.K. Bhandari, A. Kumar, G.K. Singh, Modified artificial bee colony based computationally efficient multilevel thresholding for satellite image segmentation using Kapur’s, Otsu and Tsallis functions, Expert Systems with Applications, Volume 42, Issue 3, 2015, Pages 1573-1601, ISSN 0957-4174, https://doi.org/10.1016/j.eswa.2014.09.049.
  * [12] Sanjay Agrawal, Rutuparna Panda, Sudipta Bhuyan, B.K. Panigrahi, Tsallis entropy based optimal multilevel thresholding using cuckoo search algorithm, Swarm and Evolutionary Computation, Volume 11, 2013, Pages 16-30, ISSN 2210-6502, https://doi.org/10.1016/j.swevo.2013.02.001.
  * [13] N. Otsu, "A Threshold Selection Method from Gray-Level Histograms," in IEEE Transactions on Systems, Man, and Cybernetics, vol. 9, no. 1, pp. 62-66, Jan. 1979, doi: 10.1109/TSMC.1979.4310076.
  * [14] Wen-Hsiang Tsai, Moment-preserving thresolding: A new approach, Computer Vision, Graphics, and Image Processing, Volume 29, Issue 3, 1985, Pages 377-393, ISSN 0734-189X, https://doi.org/10.1016/0734-189X(85)90133-1.
  * [15] J.N. Kapur, P.K. Sahoo, A.K.C. Wong, A new method for gray-level picture thresholding using the entropy of the histogram, Computer Vision, Graphics, and Image Processing, Volume 29, Issue 3, 1985, Pages 273-285, ISSN 0734-189X, https://doi.org/10.1016/0734-189X(85)90125-2.
  * [16] Bahriye Akay, A study on particle swarm optimization and artificial bee colony algorithms for multilevel thresholding, Applied Soft Computing, Volume 13, Issue 6, 2013, Pages 3066-3091, ISSN 1568-4946, https://doi.org/10.1016/j.asoc.2012.03.072.
  * [17] Madhubanti Maitra, Amitava Chatterjee, A hybrid cooperative–comprehensive learning based PSO algorithm for image segmentation using multilevel thresholding, Expert Systems with Applications, Volume 34, Issue 2, 2008, Pages 1341-1350, ISSN 0957-4174, https://doi.org/10.1016/j.eswa.2007.01.002.
  * [18] Peng-Yeng Yin, Multilevel minimum cross entropy threshold selection based on particle swarm optimization, Applied Mathematics and Computation, Volume 184, Issue 2, 2007, Pages 503-513, ISSN 0096-3003, https://doi.org/10.1016/j.amc.2006.06.057.
  * [19] Wenbing Tao, Hai Jin, Liman Liu, Object segmentation using ant colony optimization algorithm and fuzzy entropy, Pattern Recognition Letters, Volume 28, Issue 7, 2007, Pages 788-796, ISSN 0167-8655, https://doi.org/10.1016/j.patrec.2006.11.007.
  * [20] Ye, Z., Zheng, Z., Yu, X., & Ning, X., ”Automatic threshold selection based on ant colony optimization algorithm”, IEEE international conference on neural networks and brain (2005) (vol. 2, pp. 728–732).
  * [21] E. Cuevas, F. Sencion, D. Zaldivar, M. Perez-Cisneros, H. Sossa, “A multi-threshold segmentation approach based on artificial bee colony optimization”, Applied Intelligence, 37 (3) (2012), pp. 321-336
  * [22] Karaboga D., “An idea based on honey bee swarm for numerical optimization”,Technical Report-TR06, Kayseri, Turkey: Erciyes University, (2005), p. 200.
  * [23] Jing Zhang, Jinfeng Liu, Two triggered information transmission algorithms for distributed moving horizon state estimation, Systems & Control Letters, Volume 65, 2014, Pages 1-12, ISSN 0167-6911, https://doi.org/10.1016/j.sysconle.2013.12.003.
  * [24] Tuba Kurban, Pinar Civicioglu, Rifat Kurban, Erkan Besdok, Comparison of evolutionary and swarm based computational techniques for multilevel color image thresholding, Applied Soft Computing, Volume 23, 2014, Pages 128-143, ISSN 1568-4946, https://doi.org/10.1016/j.asoc.2014.05.037.
  * [25] Ashish Kumar Bhandari, Vineet Kumar Singh, Anil Kumar, Girish Kumar Singh, Cuckoo search algorithm and wind driven optimization based study of satellite image segmentation for multilevel thresholding using Kapur’s entropy, Expert Systems with Applications, Volume 41, Issue 7, 2014, Pages 3538-3560, ISSN 0957-4174, https://doi.org/10.1016/j.eswa.2013.10.059.