# Suppressing evolution through environmental switching

Bryce Morsky<EMAIL_ADDRESS>Department of Physics, University of Notre
Dame, Nieuwland Science Hall, Notre Dame, IN, USA  Department of Biology,
University of Pennsylvania, Carolyn Lynch Laboratories, Philadelphia, PA, USA
Dervis Can Vural<EMAIL_ADDRESS>Department of Physics, University of Notre
Dame, Nieuwland Science Hall, Notre Dame, IN, USA

###### Abstract

Ecology and evolution under changing environments are important in many
subfields of biology with implications for medicine. Here, we explore an
example: the consequences of fluctuating environments on the emergence of
antibiotic resistance, which is an immense and growing problem. Typically,
high doses of antibiotics are employed to eliminate the infection quickly and
minimize the time under which resistance may emerge. However, this strategy
may not be optimal. Since competition can reduce fitness and resistance
typically has a reproductive cost, resistant mutants’ fitness can depend on
their environment. Here we show conditions under which environmental varying
fitness can be exploited to prevent the emergence of resistance. We develop a
stochastic Lotka-Volterra model of a microbial system with competing
phenotypes: a wild strain susceptible to the antibiotic, and a mutant strain
that is resistant. We investigate the impact of various pulsed applications of
antibiotics on population suppression. Leveraging competition, we show how a
strategy of environmental switching can suppress the infection while avoiding
resistant mutants. We discuss limitations of the procedure depending on the
microbe and pharmacodynamics and methods to ameliorate them.

##  1 Introduction

Populations will face a variety of environmental fluctuations of both biotic
and abiotic nature. Since phenotypes typically have different reproductive
success in differing environments, the dynamics of these fluctuations can be
crucial in determining phenotypic composition. Here, we consider the effects
of varying environments on the emergence and maintenance of antibiotic
resistance.

The rise of microbial resistance is a looming catastrophe, and prudential use
of antimicrobials is a fundamental means to prevent it [33]. Such strategies
to limit the chance of resistance can be made at all levels of disease
dynamics, from population level protocols to individual patient therapies.
Studies of antibiotic resistance in vivo, in hospitals, and in the community
at large using mathematical models can help address the pharmacodynamics,
pharmacokinetics, and epidemiology of resistance [34, 9, 4, 14, 20, 22, 40].
Such models have found use in effectively modelling real-world experimental
data [55, 48, 8, 39]. In particular, modelling has been used in identifying
dosing regimens that suppress the emergence of resistance [54, 53].

There are several mechanisms by which bacteria can be resistant to antibiotics
[46], an example of which is overexpression of the efflux pump [10, 59, 52],
which bacteria use to expel antibiotics. Typically, such resistant mechanisms
have a fitness cost, which can result in trade-offs between resistance and
growth [37, 16, 58, 7]. Example costs of resistance include less energy for
other cellular processes, and impaired motility. When the antibiotic is
present, the resistant mechanism pays for its cost by providing a fitness
advantage relative to the susceptible strain. However, when the antibiotic is
not present, the resistant mechanism incurs a fitness disadvantage.
Resistance, therefore, can be reversed under evolutionary forces by altering
the environment [3]. As such, a pulsed protocol, where the antibiotic is
periodically applied so that the environment switches from antibiotic to
antibiotic-free regimes, may be able to eliminate the bacteria. However, there
is a risk that resistant mutants evolve to reduce the fitness cost of
resistance rather than lose the resistance mechanism, whereby they could be
competitive whether the antibiotic is present or not [43]. Further, mutations
in regulatory genes can produce phenotypes of irreversible resistance [56].
These risks can cripple pulsed protocols aimed at controlling the infection
while preventing resistance. Preventing a sustained presence of resistance is
therefore a high priority.

Here, we develop a mathematical model of pulsed protocols of antibiotic and
antibiotic-free regimes, switching rapidly from one environment to the other,
to control a bacterial population comparable to Escherichia coli. We consider
concentration-independent (i.e. time-dependent) bactericides such as
$\beta$-lactams (e.g. penicillins and cephalosporins), which require high
maintained concentrations to be effective [2]. We assume that there is a
maximum benefit to the concentration amount (due either to the
pharmacodynamics of the bactericidal mechanism or tolerance of the patient to
the antimicrobial). Therefore, we fix the dose concentration when applied, and
find the proper periods for each regime that prevent the emergence of
resistance while eliminating the infection.

Previous theoretical studies have shown that pulsed protocols of antibiotics
can eliminate bacteria [32, 12, 13, 1]. However, these studies feature only a
“persistent phenotype”, that neither grows nor dies under application of the
antimicrobial agent [6, 61]. Bacteria may transition between the persister
type and wild type, depending on the environmental conditions. The number of
persisters remain at low levels and act as a staging ground for the bacteria
to repopulate after the antimicrobial is removed. Pulsed protocols of
antibiotics, however, can disrupt this process and lead to the elimination of
the bacteria. Experimental studies have shown that pulsed protocols can be
effective in controlling such a system [50].

Although pulsed protocols can eliminate non-persister and persister colonies,
they have more difficulty in eliminating colonies with resistant phenotypes
that can grow when antibiotics are present. However, these protocols have been
shown to be effective in containing an infection both theoretically and
experimentally [5, 25]. In such cases, antimicrobials can act as ecological
disturbances and can be approximately as effective as a constant application
of the antimicrobial in controlling the bacterial load while also diminishing
the probability of the emergence of resistance [5]. With short durations of
high concentrations of drugs, the period under which resistance is selected
for can be minimized.

The above studies have explored pulsed protocols in different ways:
controlling persisters, and controlling emergence of resistance. The resistant
strain we consider here does grow in the presence of antibiotics, and thus are
not persisters. Our scenario is thus more similar to, and an extension of,
[5]. Our main contribution is to show how leveraging competition can not only
suppress the emergence of resistance as in [5], but also reduce the overall
bacterial load. Additionally, we explore the impact of other important
mechanisms on pulsed protocols including the evolvability of the bacteria and
the lethality of the antibiotic. We compare these results to a protocol of
constant application. Though pulsed protocols can, on average, outperform a
constant application, constant applications are more likely to completely
eliminate the bacteria. However, they are also more likely to result in an
uncontrolled population of resistant mutants. Thus, with pulsed protocols we
aim to mitigate the emergence of resistance and reduce the risk of the
evolutionary escape from the antibiotic.

The key mechanism of our models in suppressing the population is competition
between the two phenotypes. Two common models of microbial competition are
resource-competition models [5] and the competitive Lotka-Volterra equation
[51, 21]. The latter of which we employ here. Competition can be low when the
total size of the population is small (e.g. the population is well below the
carrying capacity or there is a high amount of resource relative to the number
of bacteria). In such a case, both phenotypes can grow. Yet, we can still
suppress the number of resistant bacteria and the average bacterial load over
time. We explore the impact of various parameters and pulsed protocol
durations on the average bacterial load over time. Our models also features
stochasticity, which we develop in a stochastic kinetic framework [60]. We
show that only when selection against the resistant type is high when the
antibiotic is off can pulsed protocols effectively control the population.

##  2 Methods

Stochastic birth-death processes are widely used in biological modelling [41],
and, in particular, stochastic modelling of the Lotka-Volterra system [27].
Our stochastic model features microscopic processes of birth, death,
competition, and mutation, as detailed in Box 2. These processes operate on
two phenotypes $X$ and $Y$, which represent a wild-type strain, which is
susceptible to the antibiotic, and a mutant strain, which is resistant,
respectively.

Consider first the dynamics of the birth and death processes without the
presence of antibiotics, i.e. in the antibiotic-free regime. Reaction set 1
represents these processes, where $b$ and $d$ are the birth and death rates,
respectively, for the wild-type strain with $b>d>0$. We assume that the death
rate for the resistant strain is also $d$. However, assuming a cost $c>0$ to
the birth rate for resistance, the birth rate of the resistant strain is $b-c$
(costs applied to birth rather than death rates have also been applied
similarly in ecological games [26]).

In the presence of the antibiotic, the above processes still occur, but with
an additional set of reactions involving the antibiotic. Since we are
considering a concentration-independent or time-dependent antibiotic, we will
assume that the amount of antibiotic, $\bar{A}$, remains unchanged while we
are in the antibiotic regime. At the maximum dose $\bar{A}=1$, normalized. The
antibiotic is bactericidal, and kills both types of bacteria. Reaction set 3
represents death from the antibiotic with rates $\alpha$ and $\alpha^{\prime}$
for the susceptible and resistant strains, respectively. Note that for $X$ to
be susceptible and $Y$ to be resistant, we must have
$\alpha>b-d>c+\alpha^{\prime}$.

The species also die from competition as represented in Reaction set 2. Death
occurs due to the bacteria competing for resources; the loser receives less
and thus has some chance of dying. Let $\gamma$ be the rate at which two
bacteria compete over a critical resource in a well-mixed population. We use
the birth rate minus the death rate as a proxy for their competitiveness. We
then assume that the rate of death of the focal type from competition is
related to the ratio of their opponent’s competitiveness to their own. i.e.
since the wild-type is more competitive than the mutant, a mutant is more
likely to die when competing for a resource with a wild-type bacterium. If the
focal type and its competitor are the same type, then the ratio is equal to
one. We define $\kappa$ as the rate of death of a mutant in competition with a
wild-type. And, since competition may scale nonlinearly with respect to the
ratio of competitiveness, we consider several values of $\kappa\geq 1$. We
then assume that the rate of death of a wild-type in competition with a mutant
is the inverse, i.e. $1/\kappa$. Frequently in Lotka-Volterra systems, intra-
specific competition is assumed to be greater than inter-specific. As such,
the rate of death would be greater for competition between bacteria of the
same type than those that are different. However, we think it reasonable that
between kin interactions can be greater than within kin interaction in our
case, because the bacteria are well mixed.

The bacteria can turn into the other type via mutation at rate $\mu$, which
occurs in both regimes (Reaction set 4). However, under the stress of the
antibiotic, the susceptible type $X$ will mutate to $Y$ at a higher rate
$\mu^{\prime}>\mu$.

Box 1: Stochastic Lotka-Volterra processes Birth/death: $\displaystyle
X\xrightarrow{b}2X$ $\displaystyle Y\xrightarrow{b-c}2Y$ $\displaystyle
X\xrightarrow{d}\emptyset$ $\displaystyle Y\xrightarrow{d}\emptyset$ (1)
Competition: $\displaystyle 2X\xrightarrow{\gamma}X$ $\displaystyle
2Y\xrightarrow{\gamma}Y$ $\displaystyle XY\xrightarrow{\gamma/\kappa}Y$
$\displaystyle XY\xrightarrow{\gamma\kappa}X$ (2) Death via antibiotic:
$\displaystyle\bar{A}X\xrightarrow{\alpha}\emptyset$
$\displaystyle\bar{A}Y\xrightarrow{\alpha^{\prime}}\emptyset$ (3) Mutation:
$\displaystyle X\xrightarrow{\mu(1-\bar{A})+\mu^{\prime}\bar{A}}Y$
$\displaystyle Y\xrightarrow{\mu}X$ (4)

The environmental switching is controlled by a choice of the on and off
durations of the drug. We assume $100\%$ bioavailability of the drug at
application, e.g. intravenous application. Thus, when the antibiotic is
“turned on,” its effects are immediate. Further, when it is “turned off,” the
dissipation of the antibiotic — i.e. the rate at which it breaks into
ineffective material, is metabolized, etc. — is rapid, which is common for
concentration-independent and time-dependent antibiotics. In the antibiotic
regime, we apply the maximum effective dose. Thus, the set of pulsed protocols
we consider are sequences of durations of the regimes, where $\bar{A}=1$ when
the antibiotic is on and $\bar{A}=0$ when it is off.

Parameter and default value | Definition
---|---
$b=0.35$ | Birth rate.
$c=0.05$ | Cost for resistance.
$d=0.1$ | Death rate.
$\alpha=0.4$ | Death rate of the wild-type via antibiotic.
$\alpha^{\prime}=0.1\alpha=0.04$ | Death rate of the mutant via antibiotic.
$\gamma=10^{-5}$ | Competition rate.
$\kappa>1$ | Death rate of mutants from competition with the wild-type.
$\mu=10^{-5}$ | Mutation rate.
$\mu^{\prime}=10\mu=10^{-4}$ | Stress induced mutation rate of susceptible to resistant bacteria.
Table 1: Summary definitions of parameters and variables.

We conducted numerical simulations of the model to test the effects of various
protocols. We average realizations for each parameter combination. We use the
Gillespie algorithm [19] from Julia’s DifferentialEquations and Catalyst
packages [47]. Table 1 lists the default parameter values used with rates per
$15$ minutes. We vary these values to explore nearby parameter space. We
assume that a new generation occurs after $1/(b-d)=1$ hour. We estimate that
the relative fitness of the resistant strain in the antibiotic-free
environment is $(b-c-d)/(b-d)\approx 0.8\implies c=0.05$, which is within
experimentally evaluated values [38]. E. Coli has an average rate of mutation
per genome per generation on the order of $10^{-5}$ [44]. Further, under
stress from the antibiotic, the mutation rate can be even larger, up to ten
times the non-stressed rate [31]. Thus, we consider $\mu^{\prime}=10\mu$. We
assume that resistant bacteria die from the antibiotic at $1/\nth{10}$ the
rate susceptibles do [11], i.e. $\alpha^{\prime}=0.1\alpha$. The initial
condition is a population of $100\%$ susceptible bacteria, $X_{0}=10^{5}$. We
explore a variety of competition parameters $\kappa$. We consider fixed on/off
durations, where we repeat switching until the population is extinct or $t=14$
days.

##  3 Results

We first consider the mean field (i.e. non-stochastic) behaviour of the system
with average values $\langle X\rangle$ and $\langle Y\rangle$ (for details of
the derivation see Appendix A.)

$\displaystyle\dot{\langle X\rangle}$
$\displaystyle=(b-d-\mu(1-\bar{A})-(\alpha+\mu^{\prime})\bar{A})\langle
X\rangle+\mu\langle Y\rangle-\gamma\langle
X\rangle^{2}-\frac{\gamma}{\kappa}\langle X\rangle\langle Y\rangle,$ (5)
$\displaystyle\dot{\langle Y\rangle}$
$\displaystyle=(b-c-d-\mu-\alpha^{\prime}\bar{A})\langle
Y\rangle+(\mu(1-\bar{A})+\mu^{\prime}\bar{A})\langle
X\rangle-\gamma\kappa\langle X\rangle\langle Y\rangle-\gamma\langle
Y\rangle^{2}.$ (6)

In either environment, both types will coexist at equilibrium due to
mutations, though the less adapted type will remain at low frequency. When the
population size is sufficiently low (i.e. below the zero isoclines),
competition will also be low, which will allow both types to increase in
abundance. However, the higher the competition term $\kappa$, the smaller this
region is. A large $\kappa$ will cause the mutant strain to be suppressed in
the antibiotic-free environment, and in the stochastic scenario, this effect
increases the chance that the mutant strain be eliminated. In the remainder of
the results, we detail the effects of switching the drug on and off,
competition, and stochasticity in the stochastic scenario.

(a)

(b)

(c)

(d)

Figure 1: Representative times series for resistance emerging under constant
application, panel (a), and results for a pulsed protocol (on and off for
$2$hrs each), panels (b)-(d). Blue are wild strain and red resistant. Pulsed
protocols can suppress the bacterial load (b with $\alpha=0.4$ and
$\kappa=4$). However, if we increase the antibiotic kill rate to
$\alpha=0.45$, the pulsed protocol fails (c). We may also observe oscillations
(d with $\alpha=0.5$, $\gamma=0.5\times 10^{-5}$, $\kappa=20$, and
$\mu=10^{-4}$). $X_{0}=10^{5}$ wild-type strain bacteria, and the remaining
parameters, if not mentioned here, are from Table 1. Note that we only depict
results $\leq 10^{4}$, since the population is quickly contained to this
region.

Figure 1 depicts a representative time series for a switching protocol vs. a
constant application of the antibiotic. With a sufficient competitive
disadvantage for resistance, i.e. high $\kappa$, we can effectively suppress
the average bacterial load over time and resistant bacteria relative to a
constant application as depicted in Figures 1(b) (pulsed protocol) and 1(a)
(constant application). However, due to stochastic effects, resistance can
briefly rise as seen in Figure 1(d) before it is brought under control again.
This phenomenon emerges from the interactions between the two types at
different time scales. Oscillations between the types emerge at a time scale
longer than the period of the protocol. When the mutants are at their peak,
the wild-type strain is not, and vice verse. However, peaks of the wild-type
strain are relatively low as can be seen in panel d. Between day $10$ until
sometime during day $12$, the mutants are suppressed and the wild-type are at
a relative high. This high, however, is not much higher than when they are
suppressed. As such, the total bacterial load is relatively low. Regardless of
these waves of resistant bacteria, the average bacterial load over time can
still be less than when the mutant becomes established under constant
application, as we shall show.

Though Figure 1(a) depicts a specific instance where the mutant becomes
established under the constant application protocol, resistance can be
prevented by rapid elimination of the population. Since the mutations and
fluctuations in abundances are stochastic, it is possible that we are
fortunate and constant application drives the population extinct before
resistance emerges. Thus, to better understand the effectiveness of therapies
we must evaluate the statistics of the bacterial load as a function of system
parameters. We will show that constant application of an antibiotic tends to
lead to either extreme: elimination of the entire population, or the
establishment of a majority resistant population.

Averaged over $50$ realizations, we calculate average bacterial load over time
for pulsed protocols with various on and off durations and compare these to
the bacterial load for constant application of the antibiotics. We plot these
results in heat maps, where the colour indicates the long-term bacterial load
relative to the outcome from constant antibiotic application. Red indicates
that the therapy is on average worse than constant application, yellow is on
average equal, and blue indicates that it is on average better.

We observe that pulsed protocols along a diagonal do best. One reason for this
is that the switching times explored here are much less than the time to reach
carrying capacity in either regime. For example, even a day-period protocol
will not reach carrying capacity (the expected time to reach carrying capacity
is between one and two days). In such a case, the population can swing from
predominantly one type to the other (see Appendix B for an example time
series). However, this behaviour can still be beneficial, since each
application of the antibiotic is another chance of eliminating the population,
since switching environments drives the dominant type down potentially to
extinction before the other type can become established. In addition to
plotting heat maps, we plot the average bacterial load over time for constant
and pulsed ($2$hrs on and off each) for various values of each parameter
averaged over $50$ realizations.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

Figure 2: Heatmaps of the average bacterial load over time from pulsed
protocols relative to that of constant application of the antibiotic for
$\kappa=1,2,4,8,10,20$. Protocols that matched the average outcome of the
constant application therapy are coloured in yellow. Those protocols that did
worse are in red, and those that did better are in blue. Panel g depicts the
average bacterial load over time for constant and pulsed ($2$hrs on and off
each) for various $\kappa$. The points are the results for individual
realizations and the curves their average.

Figure 2 shows that the higher the competition, the lower the diagonal (i.e.
the best results come from protocols where the duration on is greater than the
duration off). The increased competition suppresses the emergence of
resistance even in the antibiotic environment, and thus the duration of
application can be longer. However, we note that the effectiveness of the
optimal pulsed protocols drastically falls if the duration on relative to off
is too high. We do not see this sensitivity when reducing the time on. We also
observe an intermediate level of competition is best for pulsed protocols
relative to constant application. We can see this effect in Figure 2(g).
Increasing $\kappa$ decreases the average of the bacterial load of the
individual realizations for the constant application as we would expect.
Since, high competition between the types will suppress the emergence of
resistant mutants (which is true in both environments). However, increasing
competition has an initially steeper effect upon pulsed protocols before it
levels off. A sufficient amount of competition is required for pulsed
protocols to work. As $\kappa$ is increased, the difference between the
outcomes of the two approaches decreases.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

Figure 3: Heatmaps of the average bacterial load over time from pulsed
protocols relative to that of constant application of the antibiotic for
$\alpha=0.35,0.4,0.45,0.5,0.55,0.6$ and $\kappa=4$ (panels a-f). Protocols
that matched the average outcome of the constant application therapy are
coloured in yellow. Those protocols that did worse are in red, and those that
did better are in blue. Panel g depicts the average bacterial load over time
for constant and pulsed ($2$hrs on and off each) for various $\alpha$. The
points are the results for individual realizations and the curves their
average.

Another reason for angle of the optimal diagonal of successful protocols in
Figure 2, is due to the relationships between the mean growth rates and the
antibiotic kill rates. Figure 3 depicts the results for various values of
$\alpha$. The higher the antibiotic kill rate, the shorter the duration on for
the most successful protocols. Like in the case of $\kappa$, $\alpha$ impacts
the effectiveness of pulsed protocols nonlinearly. Figure 3(g) depicts the
mean results for various $\alpha$. The higher the $\alpha$, the better
constant application does. However, this is not true for pulsed protocols. An
intermediate value is best. This result is due to the impact of $\alpha$ on
competitiveness. If $\alpha$ is too high, then the wild-type is suppressed too
much, and thus cannot be used to suppress the mutant strain through
competition. Figure 1(c) depicts a time series of the case where $\alpha$ is
too high, resulting in failure of the pulsed protocol. Increasing competition
$\kappa$, however, can mitigate this effect, shifting the minimum to the right
(see Appendix B for an example).

(a)

(b)

(c)

(d)

(e)

(f)

(g)

Figure 4: Heatmaps of the average bacterial load over time from pulsed
protocols relative to that of constant application of the antibiotic for
$\mu=10^{-6},10^{-5},10^{-4}$, $\mu^{\prime}=\mu,10\mu$, and $\kappa=4$.
Protocols that matched the average outcome of the constant application therapy
are coloured in yellow. Those protocols that did worse are in red, and those
that did better are in blue. Panel g depicts the average bacterial load over
time for constant and pulsed ($2$hrs on and off each) for various $\mu$. The
points are the results for individual realizations and the curves their
average.

To explore how robust our results are to mutation rates, we considered various
values of $\mu$ and $\mu^{\prime}$. We can see the effects of various $\mu$ in
the rows of Figure 4, which show that the pulsed protocols are more effective
under a higher mutation rate. The first row depicts the case where there is a
stress induced mutation rate from wild-type to mutant ($\mu^{\prime}=10\mu$).
The second row depicts the results where stress does not increase the mutation
rate ($\mu^{\prime}=\mu$). The stress induced mutation makes the antibiotic
environment more conducive to generating resistance, and thus makes it harder
to control the emergence of resistance. The impact is a small relative effect
on the pulsed protocols vs. constant application protocols. Figure 4(g) shows
that the mutation rate impacts the constant application more so than the
pulsed protocol. This result matches intuition; the higher the mutation rate,
the less likely a constant application can eliminate the colony before a
mutant arises and becomes established. In summary, the more evolvable the
system, the better switching environments works.

(a)

(b)

(c)

(d)

Figure 5: Heatmaps of the average bacterial load over time from pulsed
protocols relative to that of constant application of the antibiotic for
$\gamma=0.5\times 10^{-5},10^{-5},2\times 10^{-5}$ and $\kappa=4$. Protocols
that matched the average outcome of the constant application therapy are
coloured in yellow. Those protocols that did worse are in red, and those that
did better are in blue. Panel d depicts the average bacterial load over time
for constant and pulsed ($2$hrs on and off each) for various $\gamma$. The
points are the results for individual realizations and the curves their
average.

In Figure 5 we explored the effect of varying the contact rate $\gamma$, and
observed that switching was more effective for a low $\gamma$. For a high
$\gamma$, the region where both types can grow is small, which magnifies the
impact of stochastic effects leading to elimination of emerging mutants.
Further, same-type competitive interactions are also more intense, and thus
the population is driven to extinction more quickly. Figure 5(d) shows that
the effectiveness of both constant application and pulsed protocols increases
as $\gamma$ increases (and thus the carrying capacity is lower). However, the
gap between the two shrinks. Thus, for a high $\gamma$ system, the average of
the bacterial load of the individual realizations for the constant application
is similarly to the low level of the pulsed protocol.

##  4 Discussion

Mathematical modelling has been important in the fight against resistance
through increasing our understanding of the dynamics and emergence of
resistance [42]. This paper contributes to this endeavour by showing that the
average bacterial load over time can be reduced and the emergence of resistant
mutants can be mitigated via pulsed protocols. Previous research has shown
that pulsed protocols with sufficiently long periods between switching can
eliminate the population while rapidly changing environments are ineffective
[36]. Our model would also exhibits this behaviour. Long durations between
pulses can (re)establish the wild-type. Upon application of the antibiotic,
the population would then either be eliminated or rescued by resistant
mutants. However, in a clinical setting, this may not be feasible. Though
quickly varying the environment can often not eliminate the population, it can
suppress the bacterial load and resistance. This strategy can be thought of as
“playing to not lose”, which contrasts with a constant application (or long
durations) where the bacteria are either eliminated or resistance flourishes
(i.e. “playing to win or lose”) [18].

We found that pulsed protocols only work when competition is high. Else, at
low population levels, resistance can be maintained. Previous empirical and
theoretical research has also found the importance of high competition in
containing an infection [25]. Further, pulsed protocols and competition can be
effective in containing an infection even in a well-mixed population [25].
Spatial effects, such as those found in biofilms, could heighten the degree of
competition and thus the effectiveness of the pulsed protocols. Since, spatial
heterogeneity due to clumping could keep the competition level between
different types high even when the population is small relative to the
carrying capacity. The competition effects we consider can be interpreted
crudely as arising from such effects.

Bactericides with significant post-antibiotic effects (PAE), such as
fluoroquinolones, may hamper our control strategy. Such antimicrobials can
impact the bacteria at sub-MIC levels long after they have been removed from
the system [2], and as such can select for resistant bacteria after the
antibiotic has been turned off. Additionally, sub MIC levels can lead to
multidrug resistance through radical-induced mutagensis [29]. Therefore, we
must have a rapid dissipation of the antibiotic once below the MIC to prevent
selection for the mutant (the range between the MIC and the point at which the
susceptible strain is selected for) [23]. PAE is frequently caused by
antibiotics that impair DNA functioning. Hence, $\beta$-lactams, which inhibit
cell wall production, are a good choice for our therapies. Antibiotics with a
short half life would also be effective with our protocols.

Intermediate drug concentration is a dimension we did not consider here. The
effects of concentration on the spread of the disease due to within host and
between host dynamics have been found to be an important factor [49]. It is
not always beneficial to have a high concentration due to a U-shaped
probability of resistance emerging vs. drug concentration [15]. Here we only
considered a specific concentration for all applications.

We restrict ourselves to the case where the system cannot be well monitored.
Clearly, the best strategy is to alter the durations dependent on the state of
the system, which can both maximize the duration of the antibiotic regime and
prevent the emergence of resistance. The more repeated applications of the
treatment, the less likely it will work. However, such observations may not be
feasible, especially when the bacterial load is small and heterogeneous.

Future models could incorporate other biological factors. For example, the
setting and source of resistance (source-sink dynamics) are important factors
in controlling antibiotic resistance [45]. The method by which resistance is
spread is another important factor such as where plasmids confer resistance.
In such a scenario, resistance can reemerge rapidly. Since, plasmids can
remain in the population due to horizontal transfer even when the plasmid
confers a cost [35]. Although, the transfer rate has been show to dramatically
fall once the population is low [24]. Compensatory mutations could also be
added by which the cost to resistance could be reduced ($c$ or $\kappa$ could
be reduced). However, we did explore the case where $c=0$, and found that
though pulsed protocols were less effective than when there was a cost, they
could still suppress the population and resistance (see Appendix B for these
results). Future models could also incorporate more of the complexity of
interactions between the bacteria and the patients’ natural flora [57, 17].

More generally, our work fits within the theory of controlling evolving
populations. Which, outside of bacteria, has been used to study cancer [30,
28, 18] and which families of chemotherapies will work best. Our model can be
viewed as control of “species” in conflict under directed actions of an
external force (in our case, the application of antibiotics) or under
environmental fluctuation (which could be undirected). Though our aim here has
been to lower the overall bacterial load, in some simulations (see Figure
1(d)), alternating environments prevents one species dominating thereby
sustaining coexistence. This temporal heterogeneity in competitiveness can
thus act as a stabilizing mechanism that promotes diversity (as measured by
the relative proportions of each type over time). This observation has broader
theoretical implications to abundance and diversity of phenotypes or species
competing with one another. In a switching environment, intermediate levels of
interaction between different phenotypes or species can result in higher
diversity. e.g. in our case, both phenotypes may coexist when the environment
is varying while a constant environment leads to extinction or the resistant
strain dominating (i.e less relative diversity than if both types coexist at
low levels). Though our model is within a framework of bacterial competition,
this phenomenon would apply to competitive Lotka-Volterra systems under
environmental switching more generally. As such, we envisage further research
that explores such phenomena under scenarios other than control of antibiotic
resistance and through models related to our own.

### Acknowledgements

We would like to thank Plamen Kamenov and Giuseppe Forte for assistance with
earlier versions of this project. This material is based upon work supported
by the Defense Advanced Research Projects Agency under Contract No.
HR0011-16-C-0062, and the University of Pennsylvania.

### Author contributions

Both authors contributed to the conception of the study and the final
manuscript. B.M. developed the code for and analyzed the numerical
simulations, and wrote the first draft.

#### Data availability

The code to run the numerical simulations and make figures and the data for
the figures are available at https://github.com/bmorsky/antibioticresistance.

## References

  * [1] Acar, N., and Cogan, N. G. Enhanced disinfection of bacterial populations by nutrient and antibiotic challenge timing.  Mathematical biosciences 313 (2019), 12–32.
  * [2] AliAbadi, F. S., and Lees, P. Antibiotic treatment for animals: effect on bacterial population and dosage regimen optimisation.  International Journal of Antimicrobial Agents 14, 4 (2000), 307–313.
  * [3] Andersson, D. I., and Hughes, D. Antibiotic resistance and its cost: is it possible to reverse resistance?  Nature Reviews Microbiology 8, 4 (2010), 260.
  * [4] Austin, D., and Anderson, R. Studies of antibiotic resistance within the patient, hospitals and the community using simple mathematical models.  Philosophical Transactions of the Royal Society of London B: Biological Sciences 354, 1384 (1999), 721–738.
  * [5] Baker, C. M., Ferrari, M. J., and Shea, K. Beyond dose: Pulsed antibiotic treatment schedules can maintain individual benefit while reducing resistance.  Scientific Reports 8, 1 (2018), 5866.
  * [6] Balaban, N. Q., Merrin, J., Chait, R., Kowalik, L., and Leibler, S. Bacterial persistence as a phenotypic switch.  Science 305, 5690 (2004), 1622–1625.
  * [7] Basra, P., Alsaadi, A., Bernal-Astrain, G., O’Sullivan, M. L., Hazlett, B., Clarke, L. M., Schoenrock, A., Pitre, S., and Wong, A. Fitness tradeoffs of antibiotic resistance in extraintestinal pathogenic escherichia coli.  Genome Biology and Evolution 10, 2 (2018), 667–679.
  * [8] Bhagunde, P., Singh, R., Ledesma, K. R., Chang, K.-T., Nikolaou, M., and Tam, V. H. Modelling biphasic killing of fluoroquinolones: guiding optimal dosing regimen design.  Journal of Antimicrobial Chemotherapy 66, 5 (2011), 1079–1086.
  * [9] Bonhoeffer, S., Lipsitch, M., and Levin, B. R. Evaluating treatment protocols to prevent antibiotic resistance.  Proceedings of the National Academy of Sciences 94, 22 (1997), 12106–12111.
  * [10] Borges-Walmsley, M. I., McKeegan, K. S., and Walmsley, A. R. Structure and function of efflux pumps that confer resistance to drugs.  Biochemical Journal 376, 2 (2003), 313–338.
  * [11] Coates, J., Park, B. R., Le, D., Şimşek, E., Chaudhry, W., and Kim, M. Antibiotic-induced population fluctuations and stochastic clearance of bacteria.  eLife 7 (2018), e32976.
  * [12] Cogan, N. Effects of persister formation on bacterial response to dosing.  Journal of Theoretical Biology 238, 3 (2006), 694–703.
  * [13] Cogan, N., Brown, J., Darres, K., and Petty, K. Optimal control strategies for disinfection of bacterial populations with persister and susceptible dynamics.  Antimicrobial Agents and Chemotherapy 56, 9 (2012), 4816–4826.
  * [14] Czock, D., and Keller, F. Mechanism-based pharmacokinetic–pharmacodynamic modeling of antimicrobial drug effects.  Journal of Pharmacokinetics and Pharmacodynamics 34, 6 (2007), 727–751.
  * [15] Day, T., and Read, A. F. Does high-dose antimicrobial chemotherapy prevent the evolution of resistance?  PLoS Computational Biology 12, 1 (2016), e1004689.
  * [16] Ender, M., McCallum, N., Adhikari, R., and Berger-Bächi, B. Fitness cost of sccmec and methicillin resistance levels in staphylococcus aureus.  Antimicrobial Agents and Chemotherapy 48, 6 (2004), 2295–2297.
  * [17] Estrela, S., and Brown, S. P. Community interactions and spatial structure shape selection on antibiotic resistant lineages.  PLoS Computational Biology 14, 6 (2018), e1006179.
  * [18] Fischer, A., Vázquez-García, I., and Mustonen, V. The value of monitoring to control evolving populations.  Proceedings of the National Academy of Sciences 112, 4 (2015), 1007–1012.
  * [19] Gillespie, D. T. A general method for numerically simulating the stochastic time evolution of coupled chemical reactions.  Journal of Computational Physics 22, 4 (1976), 403–434.
  * [20] Gloede, J., Scheerans, C., Derendorf, H., and Kloft, C. In vitro pharmacodynamic models to determine the effect of antibacterial drugs.  Journal of Antimicrobial Chemotherapy 65, 2 (2009), 186–201.
  * [21] Gonze, D., Coyte, K. Z., Lahti, L., and Faust, K. Microbial communities as dynamical systems.  Current opinion in microbiology 44 (2018), 41–49.
  * [22] Greulich, P., Doležal, J., Scott, M., Evans, M. R., and Allen, R. J. Predicting the dynamics of bacterial growth inhibition by ribosome-targeting antibiotics.  Physical biology 14, 6 (2017), 065005.
  * [23] Gullberg, E., Cao, S., Berg, O. G., Ilbäck, C., Sandegren, L., Hughes, D., and Andersson, D. I. Selection of resistant bacteria at very low antibiotic concentrations.  PLoS Pathogens 7, 7 (2011), e1002158.
  * [24] Händel, N., Otte, S., Jonker, M., Brul, S., and ter Kuile, B. H. Factors that affect transfer of the inci1 $\beta$-lactam resistance plasmid pesbl-283 between e. coli strains.  PloS One 10, 4 (2015), e0123039.
  * [25] Hansen, E., Karslake, J., Woods, R. J., Read, A. F., and Wood, K. B. Antibiotics can be used to contain drug-resistant bacteria by maintaining sufficiently large sensitive populations.  PLoS Biology 18, 5 (2020), e3000713.
  * [26] Hauert, C., Wakano, J. Y., and Doebeli, M. Ecological public goods games: cooperation and bifurcation.  Theoretical Population Biology 73, 2 (2008), 257–263.
  * [27] Huang, W., Hauert, C., and Traulsen, A. Stochastic game dynamics under demographic fluctuations.  Proceedings of the National Academy of Sciences 112, 29 (2015), 9064–9069.
  * [28] Katouli, A. A., and Komarova, N. L. The worst drug rule revisited: mathematical modeling of cyclic cancer treatments.  Bulletin of Mathematical Biology 73, 3 (2011), 549–584.
  * [29] Kohanski, M. A., DePristo, M. A., and Collins, J. J. Sublethal antibiotic treatment leads to multidrug resistance via radical-induced mutagenesis.  Molecular Cell 37, 3 (2010), 311–320.
  * [30] Komarova, N. Stochastic modeling of drug resistance in cancer.  Journal of Theoretical Biology 239, 3 (2006), 351–366.
  * [31] Kuban, W., Jonczyk, P., Gawel, D., Malanowska, K., Schaaper, R. M., and Fijalkowska, I. J. Role of escherichia coli dna polymerase iv in in vivo replication fidelity.  Journal of Bacteriology 186, 14 (2004), 4802–4807.
  * [32] Kussell, E., Kishony, R., Balaban, N. Q., and Leibler, S. Bacterial persistence: a model of survival in changing environments.  Genetics 169, 4 (2005), 1807–1814.
  * [33] Laxminarayan, R., Duse, A., Wattal, C., Zaidi, A. K., Wertheim, H. F., Sumpradit, N., Vlieghe, E., Hara, G. L., Gould, I. M., Goossens, H., et al. Antibiotic resistance—the need for global solutions.  The Lancet Infectious Diseases 13, 12 (2013), 1057–1098.
  * [34] Lipsitch, M., and Levin, B. R. The population dynamics of antimicrobial chemotherapy.  Antimicrobial Agents and Chemotherapy 41, 2 (1997), 363–373.
  * [35] Lopatkin, A. J., Meredith, H. R., Srimani, J. K., Pfeiffer, C., Durrett, R., and You, L. Persistence and reversal of plasmid-mediated antibiotic resistance.  Nature Communications 8, 1 (2017), 1689.
  * [36] Marrec, L., and Bitbol, A.-F. Resist or perish: Fate of a microbial population subjected to a periodic presence of antimicrobial.  PLoS computational biology 16, 4 (2020), e1007798.
  * [37] Martínez, J. L., and Baquero, F. Interactions among strategies associated with bacterial infection: pathogenicity, epidemicity, and antibiotic resistance.  Clinical Microbiology Reviews 15, 4 (2002), 647–679.
  * [38] Melnyk, A. H., Wong, A., and Kassen, R. The fitness costs of antibiotic resistance mutations.  Evolutionary Applications 8, 3 (2015), 273–283.
  * [39] Nielsen, E. I., Cars, O., and Friberg, L. E. Pk/pd indices of antibiotics predicted by a semi-mechanistic pkpd model–a step towards model-based dose optimization.  Antimicrobial Agents and Chemotherapy (2011), AAC–00182.
  * [40] Nielsen, E. I., and Friberg, L. E. Pharmacokinetic-pharmacodynamic modeling of antibacterial drugs.  Pharmacological Reviews 65, 3 (2013), 1053–1090.
  * [41] Novozhilov, A. S., Karev, G. P., and Koonin, E. V. Biological applications of the theory of birth-and-death processes.  Briefings in Bioinformatics 7, 1 (2006), 70–85.
  * [42] Opatowski, L., Guillemot, D., Boelle, P.-Y., and Temime, L. Contribution of mathematical modeling to the fight against bacterial antibiotic resistance.  Current Opinion in Infectious Diseases 24, 3 (2011), 279–287.
  * [43] Pacheco, J. O., Alvarez-Ortega, C., Rico, M. A., and Martínez, J. L. Metabolic compensation of fitness costs is a general outcome for antibiotic-resistant pseudomonas aeruginosa mutants overexpressing efflux pumps.  mBio 8, 4 (2017), e00500–17.
  * [44] Perfeito, L., Fernandes, L., Mota, C., and Gordo, I. Adaptive mutations in bacteria: high rate and small effects.  Science 317, 5839 (2007), 813–815.
  * [45] Perron, G. G., Gonzalez, A., and Buckling, A. Source–sink dynamics shape the evolution of antibiotic resistance and its pleiotropic fitness cost.  Proceedings of the Royal Society of London B: Biological Sciences 274, 1623 (2007), 2351–2356.
  * [46] Poole, K. Mechanisms of bacterial biocide and antibiotic resistance.  Journal of Applied Microbiology 92 (2002), 55S–64S.
  * [47] Rackauckas, C., and Nie, Q. Differentialequations. jl – a performant and feature-rich ecosystem for solving differential equations in julia.  Journal of Open Research Software 5, 1 (2017).  DOI:http://doi.org/10.5334/jors.151.
  * [48] Schmidt, S., Sabarinath, S. N., Barbour, A., Abbanat, D., Manitpisitkul, P., Sha, S., and Derendorf, H. Pharmacokinetic-pharmacodynamic modeling of the in vitro activities of oxazolidinone antimicrobial agents against methicillin-resistant staphylococcus aureus.  Antimicrobial Agents and Chemotherapy 53, 12 (2009), 5039–5045.
  * [49] Scire, J., Hozé, N., and Uecker, H. Aggressive or moderate drug therapy for infectious diseases? trade-offs between different treatment goals at the individual and population levels.  PLoS Computational Biology 15, 8 (2019), e1007223.
  * [50] Sharma, B., Brown, A. V., Matluck, N. E., Hu, L. T., and Lewis, K. Borrelia burgdorferi, the causative agent of lyme disease, forms drug-tolerant persister cells.  Antimicrobial Agents and Chemotherapy 59, 8 (2015), 4616–4624.
  * [51] Stein, R. R., Bucci, V., Toussaint, N. C., Buffie, C. G., Rätsch, G., Pamer, E. G., Sander, C., and Xavier, J. B. Ecological modeling from time-series inference: insight into dynamics and stability of intestinal microbiota.  PLoS Comput Biol 9, 12 (2013), e1003388.
  * [52] Sun, J., Deng, Z., and Yan, A. Bacterial multidrug efflux pumps: mechanisms, physiology and pharmacological exploitations.  Biochemical and Biophysical Research Communications 453, 2 (2014), 254–267.
  * [53] Tam, V. H., Ledesma, K. R., Vo, G., Kabbara, S., Lim, T.-P., and Nikolaou, M. Pharmacodynamic modeling of aminoglycosides against pseudomonas aeruginosa and acinetobacter baumannii: identifying dosing regimens to suppress resistance development.  Antimicrobial Agents and Chemotherapy 52, 11 (2008), 3987–3993.
  * [54] Tam, V. H., Louie, A., Deziel, M. R., Liu, W., Leary, R., and Drusano, G. L. Bacterial-population responses to drug-selective pressure: examination of garenoxacin’s effect on pseudomonas aeruginosa.  The Journal of Infectious Diseases 192, 3 (2005), 420–428.
  * [55] Tam, V. H., Schilling, A. N., Poole, K., and Nikolaou, M. Mathematical modelling response of pseudomonas aeruginosa to meropenem.  Journal of Antimicrobial Chemotherapy 60, 6 (2007), 1302–1309.
  * [56] Van Bambeke, F., Balzi, E., and Tulkens, P. M. Antibiotic efflux pumps.  Biochemical Pharmacology 60, 4 (2000), 457–470.
  * [57] Wade, M. J., Harmand, J., Benyahia, B., Bouchez, T., Chaillou, S., Cloez, B., Godon, J.-J., Boudjemaa, B. M., Rapaport, A., Sari, T., et al. Perspectives in mathematical modelling for microbial ecology.  Ecological Modelling 321 (2016), 64–74.
  * [58] Wang-Kan, X., Blair, J. M., Chirullo, B., Betts, J., La Ragione, R. M., Ivens, A., Ricci, V., Opperman, T. J., and Piddock, L. J. Lack of acrb efflux function confers loss of virulence on salmonella enterica serovar typhimurium.  mBio 8, 4 (2017), e00968–17.
  * [59] Webber, M., and Piddock, L. The importance of efflux pumps in bacterial antibiotic resistance.  Journal of Antimicrobial Chemotherapy 51, 1 (2003), 9–11.
  * [60] Wilkinson, D. J. Stochastic modelling for systems biology.  CRC press, 2011.
  * [61] Zhang, Y., Yew, W. W., and Barer, M. R. Targeting persisters for tuberculosis control.  Antimicrobial Agents and Chemotherapy 56, 5 (2012), 2223–2230.

##  Appendix A Derivation of the Fokker-Planck and mean field equations

Here we derive the mean field equations. This model includes a stress induced
mutation rate $\mu^{\prime}$ (see Appendix for details). Let $P(X,Y,t)$ be the
probability of $X$ and $Y$ numbers of each type at time $t$. Then, given that
we have a two dimensional Markov process in continuous time, the master
equation is

$\displaystyle\frac{\partial P(X,Y,t)}{\partial t}$
$\displaystyle=b(X-1)P(X-1,Y,t)+(b-c)(Y-1)P(X,Y-1,t)$
$\displaystyle+\Big{[}d+\alpha\bar{A}+\gamma(X+1)+\frac{\gamma}{\kappa}Y\Big{]}(X+1)P(X+1,Y,t)$
$\displaystyle+\Big{[}d+\alpha^{\prime}\bar{A}+\gamma\kappa
X+\gamma(Y+1)\Big{]}(Y+1)P(X,Y+1,t)$
$\displaystyle+\mu(Y+1)P(X-1,Y+1,t)+(\mu(1-\bar{A})+\mu^{\prime}\bar{A})(X+1)P(X+1,Y-1,t)$
$\displaystyle-\Big{[}(b+d+(1-\bar{A})\mu+(\alpha+\mu^{\prime})\bar{A})X+(b-c+d+\mu+\alpha^{\prime}\bar{A})Y$
$\displaystyle+\gamma X^{2}+\Big{(}\kappa+\frac{1}{\kappa}\Big{)}\gamma
XY+\gamma Y^{2}\Big{]}P(X,Y,t).$ (7)

Taking the time derivative of the average values $\langle
X(t)\rangle=\sum_{X,Y}P(X,Y,t)$ and $\langle Y(t)\rangle=\sum_{X,Y}P(X,Y,t)$,
we can find the mean field model:

$\displaystyle\dot{\langle X\rangle}$
$\displaystyle=(b-d-\mu(1-\bar{A})-(\alpha+\mu^{\prime})\bar{A})\langle
X\rangle+\mu\langle Y\rangle-\gamma\langle
X\rangle^{2}-\frac{\gamma}{\kappa}\langle X\rangle\langle Y\rangle,$ (8)
$\displaystyle\dot{\langle Y\rangle}$
$\displaystyle=(b-c-d-\mu-\alpha^{\prime}\bar{A})\langle
Y\rangle+(\mu(1-\bar{A})+\mu^{\prime}\bar{A})\langle
X\rangle-\gamma\kappa\langle X\rangle\langle Y\rangle-\gamma\langle
Y\rangle^{2}.$ (9)

Here we are assuming that $\langle XX\rangle=\langle X\rangle^{2}$, $\langle
XY\rangle=\langle X\rangle\langle Y\rangle$, and $\langle Y^{2}\rangle=\langle
Y\rangle^{2}$.

##  Appendix B Further simulations results

Here we depicts other simulations results. In Figure 6 we depict an example
time series for a one day on and off protocol where the bacteria is not
eliminated. Longer durations on or off can lead to the population reaching the
carrying capacity, after which they will be markedly less successful at
suppressing the bacteria than the constant application therapy.

Figure 6: Representative times series for a switching protocol (on and off for
$12$hrs each). $\kappa=4$ and the remaining parameters are from Table 1.

In order to explore the effects of greater stochasticity in the birth and
death rates on the outcomes of these therapies, we ran simulations with birth
and death rates $b+\xi$ and $d+\xi$, which keeps the mean growth rates
unchanged. Figure 7 depicts the results. We see that if the higher
stochasticity can reduce the relative effectiveness of pulsed protocols.
However, this effect is not large. In Figure 7(g), we can see that the gulf
between the outcomes of constant applications and pulsed protocols decreases
as $\xi$ increases.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

Figure 7: Heatmaps of the average bacterial load over time from pulsed
protocols relative to that of constant application of the antibiotic for
$\xi=-0.05,0.05,0.1,0.15,0.2,0.25$ and $\kappa=4$. Protocols that matched the
average outcome of the constant application are coloured in yellow, those
worse in red, and those better in blue. Panel g depicts the average bacterial
load over time for constant and pulsed ($2$hrs on and off each) for various
$\xi$ The points are the results for individual realizations and the curves
the average.

Figure 8 shows how increasing the competitiveness $\kappa$ can shift the
minimum of the average bacterial load to the right. Thus, we can counteract
the effect of a high antibiotic kill rate. Thereby, competition from the wild-
type can be used to effectively suppress the resistant mutants.

(a)

(b)

Figure 8: The average bacterial load over time for constant and pulsed ($2$hrs
on and off each) for various $\alpha$. The points are the results for
individual realizations and the curves their average. Increasing $\kappa$ from
$\kappa=4$ (panel a) to $\kappa=20$ (panel b) shifts the minimum of the
average bacterial load to the right.

Though compensatory mutations could remove the cost to resistance and thus
reduce the effectiveness of pulsed protocols, if there is a competitive
disadvantage, resistance can still be mitigated. This results can be seen in
Figure 9, which depicts the results for various $\kappa$ and zero cost of
resistance.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 9: Heatmaps of the average bacterial load over time from pulsed
protocols relative to that of constant application of the antibiotic for
$\kappa=1,2,4,8,10,20$. Protocols that matched the average outcome of the
constant application therapy are coloured in yellow. Those protocols that did
worse are in red, and those that did better are in blue. Here the cost to
resistance is zero ($c=0$).


11institutetext: University of Pisa, Pisa, Italy 11email:
<EMAIL_ADDRESS>
22institutetext: ISTI-CNR, via G. Moruzzi 1, 56124, Pisa, Italy 22email:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
# Generative Adversarial Networks for Astronomical Images Generation

Davide Coccomini 11 Nicola Messina 22 Claudio Gennaro 22 Fabrizio Falchi 22

###### Abstract

Space exploration has always been a source of inspiration for humankind, and
thanks to modern telescopes, it is now possible to observe celestial bodies
far away from us. With a growing number of real and imaginary images of space
available on the web and exploiting modern Deep Learning architectures such as
Generative Adversarial Networks, it is now possible to generate new
representations of space. In this research, using a Lightweight GAN, a dataset
of images obtained from the web, and the Galaxy Zoo Dataset, we have generated
thousands of new images of celestial bodies, galaxies, and finally, by
combining them, a wide view of the universe. The code for reproducing our
results is publicly available at https://github.com/davide-coccomini/GAN-
Universe, and the generated images can be explored at https://davide-
coccomini.github.io/GAN-Universe/.

###### Keywords:

Astronomy Generative Adversarial Networks Deep Learning

##  1 Introduction

Generative Adversarial Networks (GANs) [5] have recently achieved astonishing
results on many image generation tasks [8], producing figures that are
difficult to distinguish from real ones in many different contexts. At the
same time, space exploration has enabled us to observe the universe with
increasing clarity and detail, as in the case of the Hubble telescope, which
was able to take an image of a portion of the universe from the combination of
multiple individual photographs or the first-ever picture of a black hole
obtained in 2019111https://www.nationalgeographic.com/science/article/first-
picture-black-hole-revealed-m87-event-horizon-telescope-astrophysics.
Galloping technology is also giving rise to so-called space tourism with
billionaires and ordinary people making space flights. All this is helping to
fascinate new generations with the wonders of the universe and inspiring the
companies to progress faster and faster in this area [1]. So we are now faced
with an extremely high level of interest in space exploration, and curiosity
about the wonders out there is higher than ever.

In this research, we aim to exploit Generative Adversarial Networks to
generate new images of celestial bodies (planets, stars, galaxies, nebulae,
etc.) based on actually captured photographs of the universe or artistic
productions available on the web. We will then also exploit the Galaxy Zoo
dataset [14] containing hundreds of thousands of photographs of real galaxies
to generate new ones and combine them in a wide view of the universe, similar
to the Hubble image processing pipeline.

##  2 Related Works

###  2.1 Generative Adversarial Networks

Generative Adversarial Networks were first introduced by [5], and they
immediately obtained a great success, being considered one of the biggest
breakthroughs in the history of AI. GANs employ two distinct networks. The
discriminator, the one that must be able to identify when an image is fake or
not, and the generator, the network that actually generates new images in a
sufficiently credible way to deceive its counterpart. With this technique, it
was possible to generate new works of art [2], improve the resolution of
existing photographs [15] and even create highly credible deepfake images and
videos [12]. StyleGAN2 currently represents the state-of-the-art of these
networks, with its adaptive discriminator augmentation (ADA) [8]. The authors
proposed a particular mechanism of data augmentation that significantly
stabilizes training in limited data regimes, reducing the possibility of
discriminator overfitting with the consequent divergence of the training
process. This is a big step forward because this new structure makes it
possible to achieve excellent results even with limited data. A further
alternative for working in limited data situations was taken with the
Lightweight GAN [10], a further simplified version of this model capable of
achieving good results but with a lighter and shorter training process. To do
that, the architecture uses a skip-layer channel-wise excitation module and a
self-supervised discriminator trained as a feature-encoder.

###  2.2 Previous applications of GANs in Astronomy

GANs have been used in astronomy before, for example, to retrieve features in
astrophysical images of galaxies with the so-called GalaxyGAN [9]. Very often,
astrophysical images are disturbed by noise; deconvolution techniques are
traditionally used to improve the quality of these observations, but these are
rather limited. Using GANs, it is possible to recover lost features obtaining
more faithful and reliable results. In [13], the authors exploited GANs,
particularly Spatial-GANs [7], to generate views of space and thus of several
celestial bodies in the same area, inspired by the data collected by the
Hubble Space Telescope, obtaining quite realistic results. Using chained
Generative Adversarial Networks [3], they were able to generate new galaxies
with the same physical properties, highlighting how these architectures can be
useful for data augmentation in this area as well. GANs in this sector have
also been useful for atmospheric retrievals on exoplanets to obtain a model,
ExoGAN [16], capable of carrying out this task with a lower computational cost
than traditional ones and able to recognize molecular features, atmospheric
trace-gas abundances, and planetary parameters.

##  3 Methodology

To train the network to generate celestial bodies, it is necessary to create a
sufficiently large and heterogeneous dataset. To do this, we collected both
real space images and artistic representations of the universe using the
Flickr Scraper library222https://github.com/ultralytics/flickr_scraper,
together with some handwork for manually downloading images from the web. The
collected dataset was then manually revised to discard inconsistent images
(e.g., the trivial ones or those having too low resolution). As the images
collected online were all of different sizes, they were then cropped in a
centered squared way, ensuring that the image was not deformed during the
resize phase. In the end of this process, we obtained a dataset of 283
coherent, good quality, and squared images.

(a) Examples of collected dataset images.

(b) Examples of Galaxy Zoo Dataset images.

Figure 1: Examples of images in the dataset.

We also exploited the Galaxy Zoo Dataset [14], a large collection with
hundreds of thousands of space images collected by telescopes, to carry out
further tests and obtain real galaxy images to be merged into a single wide
view.

The Generative Adversarial Network chosen to carry out the experiments is a
Lightweight GAN [10], a version very similar to the state-of-the-art StyleGAN2
but lighter and easier to train. In fact, it has been demonstrated that this
network is able to converge in a few hours, on a single GPU, with a few
hundred training samples and achieving remarkable quality results. For these
reasons, it is the more suitable architecture in our context.

##  4 Experiments

We trained two different instances of the Lightweight GAN using first our
collected data and then the one from the Galaxy Zoo Dataset. In both cases,
the only data augmentation technique applied was to vary the color of the
input images with a probability of 25%.

###  4.1 Training

Firstly, a Lightweight GAN was trained on the 283 images collected from the
web. Subsequently, another instance of the network was trained on a portion of
the Galaxy Zoo Dataset consisting of 61636 images. In both cases, we used a
batch size of 3 with an Adam optimizer and a learning rate of $2e^{-4}$ . The
generated images are 128x128 pixels for the first dataset, to simplify the
network’s training process with such a low number of input images, and 256x256
for the second one, in which we have a larger number of available images.

###  4.2 Results

The lightweight GAN trained on the celestial bodies dataset obtained from the
web produced on a single Tesla T4 GPU, after three days of training, some
excellent and very credible images of planets, nebulae, and galaxies, with
style very similar to that provided as input as shown in Figure 2(a).

(a) Examples of generated images from Lightweight GAN trained on scraped
images from web.

(b) Examples of generated images from Lightweight GAN trained on Galaxy Zoo
Dataset.

Figure 2: Examples of images generated by the GAN.

After two days of training in the same environment, we obtained very credible
images on the Galaxy Zoo Dataset. In this case, thanks to the simplicity of
the input images and their abundance, the trained network could produce
results hardly distinguishable from the original ones, as shown in Figure
2(b).

Finally, inspired by the work done by astronomers who in 2019 used 7500
individual exposures obtained by the Hubble telescope to create a mosaic of a
wide view of the universe333https://www.nasa.gov/feature/goddard/2019/hubble-
astronomers-assemble-wide-view-of-the-evolving-universe, we exploited the
Lightweight GAN trained on the Galaxy Zoo dataset to generate a wide view of
the universe. We first obtained 3000 images of galaxies from the network, and
a further 10, extremely simple and almost totally black, were selected as
blank space. These images were rotated, resized, and randomly combined to
obtain a mosaic of 25000 images, thus generating the desired wide view, which
can be seen in Figure 3.

Figure 3: Wide view of universe generated by Lightweight GAN.

Although the obtained galaxies may look very similar to the original ones to
the naked eye, we decided to conduct an additional experiment using a pre-
trained detector trained on Galaxy Zoo to get a more quantitative evaluation.
The network chosen is the one trained by [4] and based on YOLOv2. By looking
at the detection results from this network, we can get a more objective idea
of the results obtained.

As shown in Figure 4, using as input the wide-view generated by the
combination of galaxies obtained from our GAN, the detector is able to
identify many of the larger galaxies in the scene. Repeating the test on
zoomed portions of the image, the detector continues to detect a good number
of galaxies perfectly in line with the tests conducted in the paper.

To obtain a quantitative measure of the quality of the images generated by the
GAN, we used the Fréchet Inception Distance (FID) [6]. This score is a widely
used metric to assess the quality of a GAN’s output. It gives an assessment
based on both the variety of images generated and their credibility taking
into account the classes distribution from the training set. The network
chosen to calculate this metric is an XceptionNet developed for Kaggle Galaxy
Zoo competition444https://www.kaggle.com/c/galaxy-zoo-the-galaxy-challenge and
publicly available555https://www.kaggle.com/hironobukawaguchi/galaxy-zoo-
xception/notebook trained on 61577 images from GalaxyZoo training set to
classify into three distinct classes, spiral, eliptical and irregular. The
training images were used together with 3000 images generated by GAN to obtain
the Fréchet Inception Distance value. The FID obtained is 0.007, very close to
0, indicating a good fidelity of the generated images with respect to the
original ones. The distribution is also very similar, as can be seen in the
Table 1:

Figure 4: Galaxies detection on the entire image. Table 1: Classes
distribution on training and generated sets

Set | Spiral | Eliptical | Irregular
---|---|---|---
Training | 25775 (41.8%) | 35738 (58.0%) | 65 (0.1%)
Generated | 920 (30.6%) | 2064 (68.8%) | 16 (0.5%)

However, an FID value so close to zero and therefore so good could also result
from a GAN memorizing the examples shown to it during training, a danger
highlighted in [11]. In order to rule out this possibility, we carried out a
similarity search between 40 images randomly extracted from those generated by
GAN and all the images in the training set.

Figure 5: The figure shows 10 images generated by the GAN training (first
row), and for each of them, on the same column, the five most similar images
identified within the training set.

The comparison of each image pair (fake/real) was conducted by calculating the
Structural Similarity Index Measure (SSIM), a commonly used measure of
similarity between two images. As shown in Figure 5, in none of the cases
considered was an image from the training set found to be perfectly identical
to the one generated, but it is often very clear that the image is completely
new. In the end, the conducted experiments show good results in both
qualitative and quantitative terms.

##  5 Conclusions

In this research we demonstrated the ability of a Lightweight GAN to obtain
extremely credible results even with a low amount of data and in an almost
unexplored context. We also generated a multitude of credible images of a wide
variety of individual celestial bodies, galaxies, and finally, a spectacular
wide-view of a portion of the universe obtained from the combination of the
created images. The quality of the results has been validated in multiple
ways: (a) exploiting purely aesthetic evaluations and detection techniques
using pre-trained networks and (b) measuring objective metrics and parameters
widely used within the Generative Adversarial Networks framework. The
resulting network can be used to generate images useful in the world of art
and graphics and as a data augmentation tool for the classification of planets
or galaxies, once again highlighting the usefulness and effectiveness of
Generative Adversarial Networks.

## References

  * [1] Crawford, I.A.: The long-term scientific benefits of a space economy (2016)
  * [2] Elgammal, A., Liu, B., Elhoseiny, M., Mazzone, M.: Can: Creative adversarial networks, generating "art" by learning about styles and deviating from style norms (2017)
  * [3] Fussell, L., Moews, B.: Forging new worlds: high-resolution synthetic galaxies with chained generative adversarial networks. Monthly Notices of the Royal Astronomical Society 485(3), 3203–3214 (Mar 2019). doi:10.1093/mnras/stz602, http://dx.doi.org/10.1093/mnras/stz602
  * [4] González, R.E., Muñoz, R.P., Hernández, C.A.: Galaxy detection and identification using deep learning and data augmentation (2018)
  * [5] Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks (2014)
  * [6] Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium (2018)
  * [7] Jetchev, N., Bergmann, U., Vollgraf, R.: Texture synthesis with spatial generative adversarial networks (11 2016)
  * [8] Karras, T., Aittala, M., Hellsten, J., Laine, S., Lehtinen, J., Aila, T.: Training generative adversarial networks with limited data (2020)
  * [9] Kevin, S., Ce, Z., Hantian, Z., Lucas, F., Krishnan, S.G.: Generative adversarial networks recover features in astrophysical images of galaxies beyond the deconvolution limit. Monthly Notices of the Royal Astronomical Society: Letters p. slx008 (Jan 2017). doi:10.1093/mnrasl/slx008, http://dx.doi.org/10.1093/mnrasl/slx008
  * [10] Liu, B., Zhu, Y., Song, K., Elgammal, A.: Towards faster and stabilized gan training for high-fidelity few-shot image synthesis (2021)
  * [11] Lucic, M., Kurach, K., Michalski, M., Gelly, S., Bousquet, O.: Are gans created equal? a large-scale study (2018)
  * [12] Mirsky, Y., Lee, W.: The creation and detection of deepfakes. ACM Computing Surveys 54(1), 1–41 (Apr 2021). doi:10.1145/3425780, http://dx.doi.org/10.1145/3425780
  * [13] Smith, M.J., Geach, J.E.: Generative deep fields: arbitrarily sized, random synthetic astronomical images through deep learning. Monthly Notices of the Royal Astronomical Society 490(4), 4985–4990 (Oct 2019). doi:10.1093/mnras/stz2886, http://dx.doi.org/10.1093/mnras/stz2886
  * [14] Willett, K.W., et al., L.: Galaxy zoo 2: detailed morphological classifications for 304 122 galaxies from the sloan digital sky survey. Monthly Notices of the Royal Astronomical Society 435(4), 2835–2860 (Sep 2013). doi:10.1093/mnras/stt1458, http://dx.doi.org/10.1093/mnras/stt1458
  * [15] Zhang, W., Liu, Y., Dong, C., Qiao, Y.: Ranksrgan: Super resolution generative adversarial networks with learning to rank (2021)
  * [16] Zingales, T., Waldmann, I.P.: ExoGAN: Retrieving exoplanetary atmospheres using deep convolutional generative adversarial networks. The Astronomical Journal 156(6), 268 (nov 2018). doi:10.3847/1538-3881/aae77c, https://doi.org/10.3847/1538-3881/aae77c