# Revisiting GANs by Best-Response Constraint: Perspective, Methodology, and
Application

Risheng Liu${}^{\ast 1,2}\quad$ Jiaxin Gao${}^{1}\quad$ Xuan Liu${}^{1}\quad$
Xin Fan1,2
1International School of Information Science & Engineering, DUT

###### Abstract

In past years, the minimax type single-level optimization formulation and its
variations have been widely utilized to address Generative Adversarial
Networks (GANs). Unfortunately, it has been proved that these alternating
learning strategies cannot exactly reveal the intrinsic relationship between
the generator and discriminator, thus easily result in a series of issues,
including mode collapse, vanishing gradients and oscillations in the training
phase, etc. In this work, by investigating the fundamental mechanism of GANs
from the perspective of hierarchical optimization, we propose Best-Response
Constraint (BRC), a general learning framework, that can explicitly formulate
the potential dependency of the generator on the discriminator. Rather than
adopting these existing time-consuming bilevel iterations, we design an
implicit gradient scheme with outer-product Hessian approximation as our fast
solution strategy. _Noteworthy, we demonstrate that even with different
motivations and formulations, a variety of existing GANs ALL can be uniformly
improved by our flexible BRC methodology._ Extensive quantitative and
qualitative experimental results verify the effectiveness, flexibility and
stability of our proposed framework.

##  1 Introduction

Generative Adversarial Networks (GANs), one of the most popular frameworks of
generative models, aim to construct the problem as a adversarial learning
procedure with the generator and the discriminator. Under the adversarial
learning mechanism, Vanilla GAN (VGAN) training procedure seeks to find the
solution to a minimax formulation by means of an alternating gradient strategy
greedily [1]. In recent years, GAN and its variants have been exploited with
significant success for a broad spectrum of complex vision and learning
problems, such as image generation [2, 3], low-level image and video
enhancement [4, 5], high-level recognition and detection [6, 7], and
reinforcement learning [8, 9], just to name a few. Formally, the goal of GANs
is to produce a generator network $G$ (with parameters $\bm{\theta}_{G}$) that
is able to accurately reveal some unknown data distribution $P_{data}$. It can
be typically analyzed from the divergence minimization perspective

$\min_{\bm{\theta}_{G}}``\mathtt{distance}"(P_{G},P_{data})\rightarrow\min_{\bm{\theta}_{G}}\max_{\bm{\theta}_{D}}F(\bm{\theta}_{G},\bm{\theta}_{D}).$
(1)

Here, $``\mathtt{distance}"$ generally denotes some given statistical distance
(e.g., Jensen-Shannon divergence, f-divergence, and Wasserstein distance)
between the distribution of the generated data $P_{G}$ and the latent
distribution $P_{data}$. By introducing an adversarial discriminator $D$
(parameter $\bm{\theta}_{D}$), VGAN dynamics defines a single-level minimax
multivariate function that includes the competition of the two networks. A
further explanation is to utilize the above-mentioned maximum objective on the
discriminator to assist in computing the divergence minimization process.
However, this modeling formulation of GANs puts the discriminator $D$ and the
generator $G$ in the identical status, and could not characterize the coupling
relationship between the generator $G$ and the discriminator $D$, thus is
always notoriously fragile and unstable. Following the widespread adoption of
joint cross-entropy loss in VGAN and its variants, i.e.,
$F(\bm{\theta}_{G},\bm{\theta}_{D})=E_{\mathbf{x}\sim
P_{data}}\left[\log(D(\mathbf{x}))\right]+E_{\mathbf{z}\sim
P_{\mathbf{z}}}\left[\log(1-D(G(\mathbf{z})))\right],$ various variations of
objective function $F$ suitable for specific applications have emerged.

In recent years, researchers are keen on a variety of GAN improvements mainly
including four categories, selecting suitable loss functions [10, 11],
utilizing better normalization [12, 13], designing delicate architectures [14,
3] and implementing various training strategies [15, 16]. Some approaches
extended in the context of VGAN, combine spectral normalization with gradient
penalty to prevent the discriminator from learning too fast and triggering
gradient disappearance [12, 10]. Other methods [17, 18] introduce
regularization of the gradient norm to improve the diversity of the generated
samples, thus improving the robustness of the framework under different
structures. Although numerous GAN variants coming from different disciplines
and perspectives exist in the previous literature, it still remains unclear
how these techniques combine with each other and how the combination affects
the overall performance. Conversely, significant manual parameter tuning and
redundant network structures pose a significant obstacle to the wider adoption
of current GANs as a off-the-shelf framework. Intuitively speaking, current
common failures regarding training GANs include generating poor generation
quality, vanishing or exploding gradients, training instability, and mode
collapse [19]. To date none of these methods are completely effective in
alleviating all the above difficulties.

---
Figure 1: Illustration of the learning principle of GAN. The traditional VGAN
(a) and our proposed BRC modeling framework (b), as well as the solution
strategy (c) are displayed from left to right. The dashed boxes exhibit why
our proposed BRC framework can find the optimal solution from three generator
variables compared to VGAN.

###  1.1 Our Motivations and Contributions

Despite the well-intended nature of Eq. (1), existing mathematical theory has
proven that the convergence of minimax methods can be particularly fragile,
even in simple, bilinear problems [20]. Algorithmically, traditional GANs
trained by taking simultaneous or alternating gradient updating rules (w.r.t.
$\bm{\theta}_{D}$ and $\bm{\theta}_{G}$), lead to two separate optimization
dynamics for generator and discriminator [21]. Both forms share the same
drawbacks, stemming from the fact that they generally do not consider ordering
aspects, thus causing various stability problems. This also means that the
gradient updates of VGAN exhibit cyclic behavior, which may slow down
convergence or cause phenomena such as mode collapse, mode hopping, etc [22].

In Figure 1, we provide an illustration in terms of the comparisons of
modeling and solving strategy under different optimization frameworks (i.e.,
VGAN and GAN with BRC). It exhibits how to find the optimal solution from
three sampled variables of generators (i.e., $\bm{\theta}_{G}^{1}$,
$\bm{\theta}_{G}^{2}$, and $\bm{\theta}_{G}^{3}$) by performing VGAN (a) and
VGAN with BRC (b), respectively. The dotted box indicates the solution process
by executing two methods separately. For VGAN, even for a locally optimal
discriminator parameter $\theta_{D}^{*}$, only a suboptimal result
$\theta_{G}^{2}$ can be found, i.e.,
$F(\bm{\theta}_{G}^{3},\bm{\theta}_{D}^{*})\geq
F(\bm{\theta}_{G}^{1},\bm{\theta}_{D}^{*})\geq
F(\bm{\theta}_{G}^{2},\bm{\theta}_{D}^{*})$. This corresponds exactly to the
fact that the alternate training strategy of VGAN will force the optimization
of two networks into a non-correspondence state, thus always resulting in
oscillating, unstable situations. On the contrary, our proposed framework
calculates the corresponding BRC for three sampled variables of generator,
that is, the optimal discriminator $\bm{\theta}_{D}(\bm{\theta}_{G})$ given
$\bm{\theta}_{G}$, and then transports it to the generator based on the value-
function $\varphi(\bm{\theta}_{G})$, and finally obtains the optimal solution
$\theta_{G}^{1}$, explained by
$F(\bm{\theta}_{G}^{3},\bm{\theta}_{D}(\bm{\theta}_{G}^{3}))\geq
F(\bm{\theta}_{G}^{2},\bm{\theta}_{D}(\bm{\theta}_{G}^{2}))\geq
F(\bm{\theta}_{G}^{1},\bm{\theta}_{D}(\bm{\theta}_{G}^{1}))$. Based on this
analysis, it can be concluded that traditional single-level minimax based
modeling and training techniques are two main limiting factors to relieve the
unstable state of GANs, while our proposed BRC framework can essentially
depict the hierarchical training dynamics and explicitly formulate the
potential dependency of the generator on the discriminator.

In this paper, we provide a novel reformulation and algorithm perspective to
re-comprehend and interpret the intrinsic mechanism of GAN training procedure.
Specifically, we define a best-response constraint with respect to the
discriminator maximization to depict the coupling nested relationship with
respect to generator and discriminator, which is then incorporated into the
divergence minimization procedure in terms of generator. Algorithmically,
instead of using the traditional hierarchical updating rule, we construct a
fast solution strategy from the perspective of implicit gradient with an
outer-product-based Hessian approximation technique. Through this,
unprecedented improvements in training stability and alleviation of mode
collapse challenges can be achieved by a significant margin. Our contributions
can be summarized as follows:

  * •

Abandoning the inherent single-level minimax formulation, we introduce Best-
Response Constraint (BRC), a novel hierarchical optimization framework to
revisit learning principles of GANs, which can explicitly formulate the
potential dependency of the generator on the discriminator.

  * •

To alleviate high computational complexity issues associated with traditional
hierarchical optimization, we design an implicit gradient scheme with outer-
product Hessian approximation as fast solution strategy, which is more
computation-friendly and suitable for diverse high-dimensional large-scale
real-world applications.

  * •

As a general learning platform, our BRC methodology accompanied with the fast
solution strategy can uniformly improve existing GAN variants by a significant
margin in a flexible manner. Abundant experimental results demonstrate the
effectiveness and efficiency especially in training stability and mode
collapse.

##  2 The Proposed Framework

In this section, we introduce Best-Response Constraint (BRC) to accurately
portray the coupling relationship, and then propose a fast solution strategy
with implicit gradient approximation. We also explore how to uniformly improve
existing GAN variants under BRC platform.

###  2.1 Adversarial Learning with Best-Response Constraints

As the single-level type modeling formulation of GAN have proven to be still
notoriously unstable, we seek to construct a novel hierarchical learning
perspective to portray the $``\mathtt{distance}"$ in Eq. (1). Starting from
maximization optimization of discriminator, we define a BRC, i.e.,
$\bm{\theta}_{D}(\bm{\theta}_{G})\in\mathcal{S}(\bm{\theta}_{G})$ to govern
the dynamic update process of the divergence minimization of generator. Given
a generator with parameter $\bm{\theta}_{G}$, the BRC can be formulated as the
optimization of discriminator network followed by a process of maximizing
optimization,

$\mathcal{S}(\bm{\theta}_{G}):=\arg\max\limits_{\bm{\theta}_{D}}f(\bm{\theta}_{D},\bm{\theta}_{G}).$
(2)

$\mathcal{S}(\bm{\theta}_{G})$ denotes the solution set with respect to any
given $\bm{\theta}_{G}$. It can be intuitively interpreted as the training of
the discriminator parameters until optimal when current $\bm{\theta}_{G}$ is
fixed. The concept of an optimal discriminator is particularly important
because its optimality would ensure that the discriminator produces meaningful
feedback to the generator. Then by enforcing the optimal discriminator
$\bm{\theta}_{D}(\bm{\theta}_{G})$ as a best-response to the divergence
objective, we define the value-function $\varphi(\bm{\theta}_{G})$ to more
accurately depict the divergence minimization problem, i.e.,

$\min\limits_{\bm{\theta}_{G}}\varphi(\bm{\theta}_{G}):=F(\bm{\theta}_{G},\bm{\theta}_{D}(\bm{\theta}_{G})).$
(3)

Intuitively, the above process contains two levels of optimization tasks where
the optimization of discriminator as a constraint is nested within the
generative learning task. From this perspective, generative adversarial
learning corresponds to a hierarchical leader-follower optimization problem

$\min\limits_{\bm{\theta}_{G}}F(\bm{\theta}_{G},\bm{\theta}_{D}(\bm{\theta}_{G})),\
s.t.\ \bm{\theta}_{D}(\bm{\theta}_{G})\in\mathcal{S}(\bm{\theta}_{G}),$ (4)

where the leader objective $F$ tries to minimize the expected divergence
between characteristic functions of real and generated data distributions, the
maximizition of the follower objective $f$ targets at obtaining the optimal
discriminator transmitted to the generator divergence. As illustrated in
Figure 1, for any generator $G$, there is a corresponding (parametric)
discriminator $D$ in the follower problem to be learned that provides the
rational (optimal) response to the leader generator. It is not symmetric in
terms of two levels and the discriminator $D$ is served as a constraint on the
generator $G$ to optimize $\bm{\theta}_{G}$, which is in accordance with the
intrinsic mechanism of GANs as a generative framework.

Indeed, the gradient of best-response constraint (i.e.,
$\frac{\partial\bm{\theta}_{D}(\bm{\theta}_{G})}{\partial\bm{\theta}_{G}}$)
captures how the optimal discriminator $\bm{\theta}_{D}(\bm{\theta}_{G})$
responds to the change in generator. Thus, the optimization (w.r.t.
$\bm{\theta}_{G}$) for the divergence minimization problem can be further
derived as

$\frac{\partial\varphi(\bm{\theta}_{G})}{\partial\bm{\theta}_{G}}=\overbrace{\frac{\partial
F(\bm{\theta}_{G},\bm{\theta}_{D})}{\partial\bm{\theta}_{G}}}^{\mbox{Direct-
Response Gradient}}+\overbrace{\frac{\partial
F(\bm{\theta}_{G},\bm{\theta}_{D})}{\partial\bm{\theta}_{D}}\underbrace{\frac{\partial\bm{\theta}_{D}(\bm{\theta}_{G})}{\partial\bm{\theta}_{G}}}_{\mbox{Best-
Response (BR) Gradient}}}^{\mbox{Hierarchical Coupled-Response Gradient}}.$
(5)

Here, the BR gradient term denotes the gradient of best-response constraint,
and the direct-response gradient term reveals straightforward dependence on
generator. The intuition is that capturing how a discriminator would react
given a change in discriminator and feeding it into a generator would allow
the generator to react, and prevent mode collapse. More specifically, the
response gradient term has the ability to embed rich adversarial information
learned from the lower-level discriminator $D$ to promote the propagation of
the generator $G$. Indeed, the principle mechanism inherent of GANs can be
further understood, i.e., the optimal feedback to the generator $G$ is
acquired by multiple updates to the discriminator $D$. Thus the BR Gradient
term of the follower actually plays the key role for the adversarial learning
process.

However, as for most of existing GANs, they just establish gradient flows
simply based on the direct-response gradient term of Eq. (5) to update
$\bm{\theta}_{D}$ and $\bm{\theta}_{G}$, i.e.,
$\bm{\theta}_{D}^{t+1}=\bm{\theta}_{D}^{t}+s_{t}\frac{\partial
F\left(\bm{\theta}_{G}^{t},\bm{\theta}_{D}^{t}\right)}{\partial\bm{\theta}_{D}^{t}},$
and $\bm{\theta}_{G}^{t+1}=\bm{\theta}_{G}^{t}-l_{t}\frac{\partial
F\left(\bm{\theta}_{G}^{t},\bm{\theta}_{D}^{t}\right)}{\partial\bm{\theta}_{G}^{t}}$
($s_{t}$ and $l_{t}$ represent the learning rates for $G$ and $D$ updates),
which either ignores BR gradient or inaccurately approximates best-response
without unrolled gradient. Thus all fail to exactly evaluate the complicated
dependency of the generator induced by $\bm{\theta}_{D}(\bm{\theta}_{G})$. It
can be directly reflected by why the optimal solution cannot be found by VGAN
even for a locally optimal discriminator parameter $\theta_{D}^{*}$, as
illustrated in (a) of Figure 1. Explained from this perspective, our proposed
BRC framework has the ability to better capture the potential dependency of
the generator on the discriminator, thus maintaining better generalization
capabilities, refer to (b) in Figure 1.

###  2.2 Fast Solution Strategy with Implicit Gradient Approximation

In this section, we propose a fast and efficient training strategy, Implicit
Gradient Approximation (IGA), to alleviate the complexity and instability of
adversarial learning process. The key step in the preceding discussion is
providing the accurately calculated BR gradient (i.e.,
$\frac{\partial\bm{\theta}_{D}(\bm{\theta}_{G})}{\partial\bm{\theta}_{G}}$ of
Eq. (5). However, it is computationally challenging for most existing
strategies to evaluate the exact BR gradient. This difficulty is further
aggregated when $\bm{\theta}_{D}$ and $\bm{\theta}_{G}$ are high-dimensional.
Considering that implicit methods can directly and precisely estimate the
optimal gradient, we derive the following equation in the case of optimal
discriminator, i.e., $\partial f/\partial\bm{\theta}_{D}=0$, inspired by
implicit function theorem. Thus, the BR gradient that is inextricably linked
with discriminator maximization is then replaced with an implicit equation,
i.e.,

$\frac{\partial\bm{\theta}_{D}(\bm{\theta}_{G})}{\partial\bm{\theta}_{G}}=-\left(\frac{\partial^{2}f(\bm{\theta}_{G},\bm{\theta}_{D}(\bm{\theta}_{G}))}{\partial\bm{\theta}_{D}\partial\bm{\theta}_{D}}\right)^{-1}\frac{\partial^{2}f(\bm{\theta}_{G},\bm{\theta}_{D}(\bm{\theta}_{G}))}{\partial\bm{\theta}_{D}\partial\bm{\theta}_{G}}.$
(6)

On account of the extreme difficulty of counting Hessian and its inverse, we
seek to establish an efficient solving strategy from the perspective of
simplification of second derivative to first derivative to compute the best-
response Jacobian, accompanied by two key computational steps, implicit
gradient estimation and outer-product approximation.

Implicit Gradient Estimation. For convenience, we first denote the
hierarchical coupled-response gradient term in Eq. (5) as $\mathbf{G}_{R}$,
i.e., $\mathbf{G}_{R}=\frac{\partial
F(\bm{\theta}_{G},\bm{\theta}_{D})}{\partial\bm{\theta}_{D}}\frac{\partial\bm{\theta}_{D}(\bm{\theta}_{G})}{\partial\bm{\theta}_{G}}.$
To avoid directly calculating the products of various Hessians and their
inversions, we further introduce a linear solver system $\mathbf{B}$ based on
Eq. (6), for evading the complexity of calculating $\mathbf{G}_{R}$. Hence,
the indirect response gradient can be reformulated as

$\mathbf{G}_{R}=\left(\frac{\partial^{2}f}{\partial\bm{\theta}_{D}\partial\bm{\theta}_{G}}\right)^{\top}\mathbf{B},\
\mbox{where}\
\frac{\partial^{2}f}{\partial\bm{\theta}_{D}\partial\bm{\theta}_{D}}\mathbf{B}=-\frac{\partial
F}{\partial\bm{\theta}_{D}},$ (7)

where $(\cdot)^{\top}$ denotes the transposition operation. The indirect
response gradient $\mathbf{G}_{R}$ only depends on the first-order condition,
and decouples the computational burden from the solution trajectory of the
lower-level discriminator, which can greatly reduce the pressure to propagate
the backward gradient in the discriminator maximization dynamics. However, it
can be recognized that the calculation of second-order derivatives in
$\mathbf{G}_{R}$ is still intractable. The urgent requirements from
approximating the repeated computation of two Hessian matrix
$\frac{\partial^{2}f}{\partial\bm{\theta}_{D}\partial\bm{\theta}_{D}}$ and
$\frac{\partial^{2}f}{\partial\bm{\theta}_{D}\partial\bm{\theta}_{G}}$ spawned
the following concept of outer-product approximation.

Outer-Product Approximation. To further suppress the complexity of
discriminator optimization, we consider replacing the original Hessian
operation based on Gauss-Newton method and introduce two approximations using
corresponding outer products, as follows:

$\frac{\partial^{2}f}{\partial\bm{\theta}_{D}\partial\bm{\theta}_{D}}\approx\frac{\partial
f}{\partial\bm{\theta}_{D}}\frac{\partial f^{\top}}{\partial\bm{\theta}_{D}},\
\frac{\partial^{2}f}{\partial\bm{\theta}_{D}\partial\bm{\theta}_{G}}\approx\frac{\partial
f}{\partial\bm{\theta}_{D}}\frac{\partial f^{\top}}{\partial\bm{\theta}_{G}}.$
(8)

This way of gradient separation converts the second-order derivative of high
complexity into a simple product operation of the first-order derivative,
which can greatly reduce the complexity of the algorithm, especially in the
memory consumption. By combining Eqs. (7)-(8), we establish the nonlinear
least squares problem by approximating the Gauss-Newton formula. Plugging into
the Eq. (7), thus we can obtain $\mathbf{B}\approx-\left(\left(\frac{\partial
f}{\partial\bm{\theta}_{D}}\frac{\partial
f^{\top}}{\partial\bm{\theta}_{D}}\right)^{\top}\frac{\partial
f}{\partial\bm{\theta}_{D}}\frac{\partial
f^{\top}}{\partial\bm{\theta}_{D}}\right)^{-1}\left(\frac{\partial
f}{\partial\bm{\theta}_{D}}\frac{\partial
f^{\top}}{\partial\bm{\theta}_{D}}\right)^{\top}\frac{\partial
F}{\partial\bm{\theta}_{D}}.$ To over simplify, we can have
$\mathbf{B}=\left(\frac{\partial f}{\partial\bm{\theta}_{D}}\frac{\partial
f^{\top}}{\partial\bm{\theta}_{D}}\frac{\partial
f}{\partial\bm{\theta}_{D}}\frac{\partial
f^{\top}}{\partial\bm{\theta}_{D}}\right)^{-1}\left(\frac{\partial
f^{\top}}{\partial\bm{\theta}_{D}}\frac{\partial
f}{\partial\bm{\theta}_{D}}\right)\frac{\partial F}{\partial\bm{\theta}_{D}}.$
Finally, the representation of the response gradient $\mathbf{G}_{R}$ is
approximately obtained, i.e.,

$\mathbf{G}_{R}\approx-\frac{\partial
f}{\partial\bm{\theta}_{G}}\left({\frac{\partial
f^{\top}}{\partial\bm{\theta}_{D}}\frac{\partial
F}{\partial\bm{\theta}_{D}}}\bigg{/}{\frac{\partial
f^{\top}}{\partial\bm{\theta}_{D}}\frac{\partial
f}{\partial\bm{\theta}_{D}}}\right).$ (9)

Algorithm 1 Implicit Gradient Approximation (IGA).

0: Initialization $\bm{\theta}_{G}^{0},\bm{\theta}_{D}^{0}$ and necessary
parameters.

0: The optimal generator $G$.

1: while not converge do

2: Obtain approximation $\hat{\bm{\theta}}_{D}$ to $\bm{\theta}_{D}$ by
solving Eq. (2).

3: Calculate $\mathbf{G}_{R}$ with $\hat{\bm{\theta}}_{D}$ and current
$\bm{\theta}_{G}$ by Eq. (9).

4: Calculate
$\frac{\partial\varphi(\bm{\theta}_{G})}{\partial\bm{\theta}_{G}}$ by Eq. (5)
.

5: Update
$\bm{\theta}_{G}\leftarrow\bm{\theta}_{G}-\alpha\frac{\partial\varphi(\bm{\theta}_{G})}{\partial\bm{\theta}_{G}}$
($\alpha$: learning rate).

6: end while

7: return The optimal parameters $\bm{\theta}_{G}$.

Based on the above derivations, we can summarize the complete solution
strategy in Alg. 1. In the training process, with the current generator
parameters $\bm{\theta}_{G}$, we first optimize the discriminator according to
the discriminator maximization objective $f$ in several steps to approximate
the optimal discriminator, i.e.,
$\hat{\bm{\theta}}_{D}(\bm{\theta}_{G})\approx\bm{\theta}_{D}(\bm{\theta}_{G})$.
The optimal discriminator is then back-propagated to the generator,
$\mathbf{G}_{R}$ is calculated based on our proposed implicit gradient
strategy (i.e., Eq. (9)), which in turn updates the parameters of the
generator until convergence.

In terms of computational complexity, Alg. 1 does not involve the unfolded
dynamical system by recurrent iteration or Hessian inverse, without any
computation of Hessian- or Jacobian-vector products. All the complexity of our
strategy comes from computing the first order gradient. The calculation of the
first derivative of the function and the Hessian vector product have similar
time and space complexity, our approximate method simplifies the process of
solving the response gradient $\mathbf{G}_{R}$ to only calculate the first-
order derivative of a few fixed times. We also provide a fair assessment of
the various traditional hierarchical methods in Table 3.1, verifying the
superiority of our solution strategy in terms of convergence speed and memory
consumption.

###  2.3 BRC vs. Existing GAN variants

Issues of Existing GAN variants. In recent years, a plethora of GAN variants
have been established to improve performance, from utilizing better loss
functions (e.g., WGAN [10], LSGAN [11], SNGAN [12]) to designing suitable
training strategies (e.g., UnrolledGAN [15], ProxGAN [16]). Even with
different formulations and complex strategies, these methods are always
notoriously fragile and unstable in most cases. For these GAN methods, the
core problem is that the generator and discriminator are modeled and optimized
through the same objective $F$ based on single-level minimax type formulation,
and then perform the alternate propagation strategy under two parallel
patterns. With this setting, the optimization of generator always depends on
parameters of discriminator in the previous step rather than the current step,
as illustrated in (a) of Figure 1. This directly results in deviating from the
essential goal of divergence minimization process in Eq. (1), and triggers
unstable situations like mode collapse and gradient disappearance.

Uniformly Improving GAN Variants by BRC. Despite different motivations and
formulations, all existing GANs can be uniformly improved under our flexible
BRC platform. With BRC methodology, we can first construct the discriminator
dynamics corresponding to the maximization optimization of Eq. (2). As
illustrated in Figure 1 from (a) to (b), the optimization of discriminator can
be depicted by an exact estimated BRC. Then the BRC can be dynamically back-
propagated to the optimization process of Eq. (4) for the generator dynamics.
Thus, it can accurately support the solving procedure of divergence
minimization dynamics between generator and discriminator for all GANs. With
the proposed fast solution strategy, we compute the hierarchical response
gradient for generator with the outer-product approximation via Eq. (7) and
Eq. (8). Through this, the potential dependency of the generator on the
current discriminator can be accurately formulated and well-portrayed. We
demonstrate the effectiveness and efficiency of BRC in terms of more stable
training performance and better generalization capabilities through abundant
experimental results.

##  3 Experimental Results

In this section, we conduct comprehensive experiments on the synthetic and
real-world applications to evaluate the effectiveness and flexibility of our
proposed BRC framework and solution strategy.

###  3.1 Evaluations on Synthesized Data

Table 1:  Comparison results on three representative GAN methods on 2D Ring MOG experiment. Best and second best results are highlighted and underlined. Method | 2D Ring (Max Mode=5)
---|---
FID$\downarrow$ | JS$\downarrow$ | Mode$\uparrow$
VGAN | 295.5$\pm$16.20 | 0.798 $\pm$0.08 | 2.54$\pm$0.51
WGAN | 28.93$\pm$4.10 | 0.728$\pm$0.07 | 3.80$\pm$1.50
UnrolledGAN | 246.9$\pm$7.60 | 0.796$\pm$0.02 | 1.90$\pm$1.26
VGAN(BRC) | 15.43$\pm$0.77 | 0.60$\pm$0.27 | 5.00$\pm$0
VGAN(BRC-Rf) | 0.87$\pm$0.07 | 0.39$\pm$0.31 | 5.00$\pm$0

Implementation Details. First, on the synthetic dataset, we discuss the
feasibility of the proposed algorithm in terms of operational efficiency and
convergence performance compared with the current mainstream GAN networks. To
fully demonstrate the effectiveness of our method, we perform visual analysis
and comprehensive performance evaluation on 2D and 3D Mixed of Gaussian (MOG)
distribution data. We compare performance with several state-of-the-art GAN
architectures that mitigate mode collapse and maintain stable training,
including VGAN [1], WGAN [10], ProxGAN [16], LCGAN [23], etc.

In the 2D case, we create 2D Ring MOG distribution with 5 or 8 2D Gaussian
functions (denoted by “Max Mode=5/8”). In the 3D scene, we conduct this
experiment with 27 3D Gaussian functions in a cube (denoted by “Max Mode=27”).
The variance of each Gaussian distribution is set to 0.02. During the training
phase, we create 512 generated data and real data samples from each mixture of
Gaussian models for each training batch, and sample 512 generated images for
testing. The Adam optimizer is uniformly set to optimize two networks, and the
learning rates of the discriminator $D$ and generator $G$ are $10^{-4}$ and
$10^{-3}$, respectively. For the generator and discriminator, we use a 3-layer
linear network with a width of 256 and a leaky relu with a threshold of 0.2 as
the activation function. We employ three well-known metrics, i.e., Frechet
Inception Distance (FID) [14], Jensen-Shannon divergence (JS) [1], number of
Modes (Mode), to provide a comprehensive comparison. It should be noted that
in the synthetic experiments, we directly compute the data itself as a feature
to measure FID, which requires the use of a pretrained network for feature
extraction.

Performance Evaluation. On 2D Ring MOG experiment, we first evaluate the
effectiveness and efficiency of our method for the improvement of VGAN, and
demonstrate the flexibility in choosing lower-level loss function under our
framework. As shown in Table 3.1, compared to WGAN and UnrolledGAN, original
VGAN combining with BRC is distinguished by immensely increased significant
performance improvement with better score (FID, JS and Mode) and achieves
comparable training efficiency in speed. On this basis, we introduce an extra
regular term for the discriminator based on VGAN, denoted by BRC-Rf, which can
further stabilize training and improve performance. In this way, the
discriminator loss function can be expressed as
$f(\bm{\theta}_{G},\bm{\theta}_{D})=f_{1}(\bm{\theta}_{G},\bm{\theta}_{D})+f_{2}(\bm{\theta}_{G},\bm{\theta}_{D}),$
where $f_{1}(\bm{\theta}_{G},\bm{\theta}_{D})=E_{\mathbf{x}\sim
P_{data}}\left[\log(D_{2}(D_{1}(\mathbf{x})),\mathtt{mean}\left\\{D_{1}(\mathbf{x}),\mathtt{var}(D_{1}(\mathbf{x}))\right\\}\right],$
$f_{2}(\bm{\theta}_{G},\bm{\theta}_{D})=E_{\mathbf{z}\sim
P_{\mathbf{z}}}\left[\log(1-D_{2}(D_{1}(G(\mathbf{z})),h(D_{1})))\right],$ and
$h(D_{1})=\mathtt{mean}\left\\{D_{1}(G(\mathbf{z})),\mathtt{var}(D_{1}(G(\mathbf{z})))\right\\}$.
Here, $D=D_{2}\circ D_{1}$, $D_{1}$ and $D_{2}$ are the feature extraction
layer and the linear classification layer of $D$, respectively. That is, we
add the mean and variance of data features as input to the discriminator
classification layer. Thanks to the flexibility of our BRC framework to revise
the discriminator objective function, VGAN can achieve optimal performance.

|
---|---
Figure 2: Comparison results of FID and JS at each training iteration in 2D
Ring MOG experiment. To improve the convergence of the algorithm, an
additional rule term (denoted as BRC-Rf) is introduced in the discriminator
loss of VGAN.

Figure 2 illustrates the FID and JS curves in terms of various methods (i.e.,
VGAN, WGAN, UnrolledGAN, VGAN(BRC) and VGAN(BRC-Rf)). It can be seen that
VGAN(BRC-Rf) and VGAN(BRC) can finally achieve the best and second best
performance metrics.

To further verify the flexibility and effectiveness of our method, both visual
and quantitative comparisons among various existing GAN methods (VGAN, LCGAN,
WGAN and ProxGAN) on two synthetic datasets are conducted. Noteworthy, the
same number of iterations among various GAN methods is strictly enforced to
facilitate a fair comparison. Compared with original GAN methods, it can be
observed in Table 2 and Figure 3 that GAN variants combined with our BRC
framework exhibit better performance in the validation dataset.

Table 2 provides a comparison of results among existing GAN methods (i.e.,
VGAN, LCGAN, WGAN and ProxGAN) with or without our BRC framework across three
metrics listed above (i.e., FID, JS and Mode) on two synthetic datasets (i.e.,
2D Ring and 3D Cube). As can be seen, all of the results illustrate that GAN
methods with our BRC framework can outperform original GAN methods for all
metrics. In particular, the mode collapse problem has been well improved. It
can be observed that WGAN combined with BRC can obtain the smallest FID score
in 2D ring distribution, and gets the lowest JS score in 3D cube dataset.
Experiments on all synthesized data validate the superiority and flexibility
for all existing GAN methods retrained in our proposed BRC framework.

Target | VGAN | WGAN | ProxGAN | LCGAN | w/o
---|---|---|---|---|---
|  |  |  |
|  |  |  | w/
2D Ring
Target | VGAN | WGAN | ProxGAN | LCGAN | w/o
|  |  |  |
|  |  |  | w/
3D Cube
Figure 3: Comparison results among four mainstream GAN methods on two synthetic MOG distribution datasets. Heatmaps for all distributions are generated through the same iterative steps. The diversity of generated samples (number of generated classes/number of target classes) is listed at the top. All experiments were terminated when their performance stabilized. The shading of the dots represents the density of the final distribution, with darker dots representing greater density. Table 2: Comparison of evaluation results of four mainstream GAN methods (i.e., VGAN, LCGAN, WGAN, and ProxGAN) with or without BRC. We report the average FID$\downarrow$, JSD $\downarrow$ and Mode $\uparrow$ scores on two MOG synthesized datasets. Best and second best results are highlighted and underlined. Method | BRC | 2D Ring (Max Mode=8) | 3D Cube (Max Mode=27)
---|---|---|---
FID$\downarrow$ |  JS$\downarrow$ |  Mode$\uparrow$ |  FID$\downarrow$ |  JS$\downarrow$ |  Mode$\uparrow$
VGAN | ✗ |  193.20$\pm$65.30 |  0.63$\pm$0.12 |  3.50$\pm$1.00 |  12.28$\pm$26.10 | 0.48$\pm$0.08 |  8.50$\pm$1.00
✓ |  34.16$\pm$10.23 | 0.40$\pm$0.16 | 7.50$\pm$1.00 | 0.52$\pm$0.16 |  0.61$\pm$0.30 | 23.00$\pm$1.40
LCGAN | ✗ |  15.01$\pm$15.30 |  0.63$\pm$0.12 |  3.50$\pm$1.00 |  68.80$\pm$60.85 |  0.87$\pm$0.17 |  6.00$\pm$3.16
✓ |  1.19$\pm$1.23 | 0.40$\pm$0.16 | 7.50$\pm$1.00 |  30.5$\pm$45.81 |  0.70$\pm$0.30 |  17.33$\pm$14.15
WGAN | ✗ | 0.42$\pm$0.21 |  0.65$\pm$0.13 |  6.25$\pm$1.53 |  12.00$\pm$0.90 |  0.62$\pm$0.07 |  15.00$\pm$0.41
✓ | 0.16$\pm$0.09 | 0.27$\pm$0.25 | 7.00$\pm$0.82 | 0.80$\pm$0.20 | 0.21$\pm$0.03 | 24.00$\pm$0.51
ProxGAN | ✗ |  15.01$\pm$15.30 |  0.63$\pm$0.12 |  3.50$\pm$1.00 |  111.64$\pm$63.96 |  0.85$\pm$0.03 |  3.67$\pm$2.89
✓ |  8.82$\pm$17.25 |  0.58$\pm$0.26 |  6.50$\pm$3.00 |  52.10$\pm$86.98 |  0.69$\pm$0.30 |  17.67$\pm$14.47

Figure 3 shows a comparison of results regarding the number of samples
generated among current advanced GAN methods with or without our BRC
framework. VGAN and LCGAN in our framework already generate all modes and are
significantly better than corresponding original GAN methods in terms of
similarity to the real target under 2D ring distribution. It can be seen
original GANs only capture a small number of distributions, getting into the
most severe mode collapse dilemma and failing to achieve satisfactory
performance. While under our framework, these method can capture a relatively
large number of distributions with the assistance of our method, and generate
abundant real distributions. On 3D Cube distribution, ProxGAN and LCGAN
combined with our method can fit almost all Gaussian distributions well with
plenty of details. Experiments on all synthesized data validate the
superiority of original GAN methods trained in our framework.

Table 3: Comparison of the time complexity (S) and space complexity (MB) on 2D and 3D MOG datasets for each step of calculating the gradient of $\bm{\theta}_{G}$ against existing gradient-based algorithms. Note that “T” and “M” represent “Time” and “Memory”, respectively. Best and second best results are highlighted and underlined.  Dataset | Metric | RHG | BDA | CG | Neumann | IGA
---|---|---|---|---|---|---
2D MOG | T | 1083.97 | 1512.74 | 317.96 | 311.16 | 107.77
M | 94.85 | 106.31 | 45.56 | 43.92 | 16.34
3D MOG | T | 1031.49 | 1537.86 | 373.62 | 309.21 | 94.98
M | 74.90 | 96.85 | 23.16 | 21.05 | 12.56

Computational efficiency. We also compare time and space complexity among our
strategy and current mainstream hierarchical optimization solution strategies
on 2D and 3D MOG synthetic datasets. As can be seen from the Table 3.1, the
implicit gradient methods (i.e. CG [24] and Neumann [25]) keep the
computational complexity lower compared to the explicit gradient methods (i.e.
RHG [26] and BDA [27]) because they are efficient to avoid the computationally
expensive Hessian inverse. Our method outperforms these methods and can
guarantee less time consumption and less memory consumption, especially in
high-dimensional tasks.

###  3.2 Validation on Real-world Image Generation Task

Implementation Details. In this part, we evaluate the flexibility and
effectiveness of our framework on the real-world image generation task. In
terms of image generation, we first use the popular CIFAR10 [28], CIFAR100
[29] benchmark for evaluation and in-depth analysis of our method training
strategy. Both CIFAR10 and CIFAR100 are composed of 60,000 colour images with
$32\times 32$ size, where 50,000 images are used as training set and the rest
10,000 images are used as test set. We use Inception Score (IS) [17] for
evaluating generation quality and diversity, and FID for capturing model
issues, e.g., mode collapse. All the results show our algorithm can further
facilitate the training of original GAN methods and obtain better sample
generation quality and performance score. We train DCGAN, LSGAN and SNGAN by
executing 200k, 200k and 20k iterations, respectively. For the network
architecture and other training details, we follow the settings in [30].

Table 4:  Comparison results of FID and IS score on CIFAR10 and CIFAR100 dataset training with or without BRC. Best and second best results are highlighted and underlined.  Method | BRC | CIFAR10 | CIFAR100
---|---|---|---
IS$\uparrow$ | FID$\downarrow$ | IS$\uparrow$ | FID$\downarrow$
DCGAN | ✗ | 6.63 | 49.03 | 6.56 | 57.37
✓ | 7.06 | 42.23 | 6.87 | 44.18
LSGAN | ✗ | 5.57 | 66.68 | 3.81 | 145.54
✓ | 7.54 | 32.50 | 7.30 | 35.72
SNGAN | ✗ | 7.48 | 26.51 | 7.99 | 25.33
✓ | 7.58 | 22.81 | 8.27 | 21.37
|
---|---
CIFAR10 | CIFAR100
Figure 4: Comparison of training efficiency results measured by FID and IS for
LSGAN networks on CIFAR10 dataset (listed left) and CIFAR100 datasets (listed
right). More stable training convergence curves and better FID and IS
performance scores can be obtained by LSGAN combined with BRC.

Performance Evaluation. First, we qualitatively evaluate three representative
GAN networks (i.e., DCGAN [31], LSGAN [11], SNGAN [12]) combined with our BRC
framework on CIFAR10 and CIFAR100 datasets. When not training in our BRC
framework, the generated images through original GANs tend to appear as
collapsed and distorted fake shapes. In contrast, retraining by our algorithm
will enable the generated samples to contain more variety and abundant
details. As shown in Figure 4, the trend of two curves implies that LSGAN
retrained in our framework is able to converge stably and obtain lower FID and
higher IS scores. Table 3.2 also demonstrates that our method achieves higher
performance than the original training strategy. Especially on CIFAR10, LSGAN
within our framework outperforms original LSGAN by a large margin,
significantly reducing the performance gap between DCGAN and SNGAN in terms of
IS and FID scores.

##  4 Conclusions and Future Works

To alleviate a series of issues like mode collapse, vanishing gradients and
oscillations caused by the single-level minimax type GANs, this paper
introduces a novel BRC framework based on a hierarchical optimization
perspective to explicitly formulate the potential dependency of the generator
on the discriminator. As a general learning framework, most of current
mainstream GANs can reformulate the optimization of discriminator as a
constraint and incorporate it into the generator optimization. On this basis,
we also design a fast solution strategy that combines implicit function theory
with an outer-product-based Hessian approximation technique, to mitigate the
high complexity of original hierarchical strategies. Extensive quantitative
and qualitative experimental results verify the effectiveness, flexibility and
stability in both synthetic and real-world scenarios. Future research should
be extended to apply BRC framework to more challenging and complex high-level
visual fields, or to explore faster and more stable learning strategies while
maintaining performance.

## References

  * Goodfellow et al. [2014] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  _NeurIPS_ , 27, 2014.
  * Li et al. [2019] Bowen Li, Xiaojuan Qi, Thomas Lukasiewicz, and Philip HS Torr.  Controllable text-to-image generation.  _arXiv:1909.07083_ , 2019.
  * Zhu et al. [2017] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros.  Unpaired image-to-image translation using cycle-consistent adversarial networks.  In _ICCV_ , pages 2223–2232, 2017.
  * Ledig et al. [2017] Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al.  Photo-realistic single image super-resolution using a generative adversarial network.  In _CVPR_ , 2017.
  * Wang et al. [2018] Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy.  Esrgan: Enhanced super-resolution generative adversarial networks.  In _Proceedings of the European conference on computer vision (ECCV) workshops_ , 2018.
  * Marriott et al. [2021] Richard T Marriott, Sami Romdhani, and Liming Chen.  A 3d gan for improved large-pose facial recognition.  In _CVPR_ , 2021.
  * Tran et al. [2017] Luan Tran, Xi Yin, and Xiaoming Liu.  Disentangled representation learning gan for pose-invariant face recognition.  In _CVPR_ , 2017.
  * Pfau and Vinyals [2016] David Pfau and Oriol Vinyals.  Connecting generative adversarial networks and actor-critic methods.  _arXiv:1610.01945_ , 2016.
  * Chen et al. [2020] Minshuo Chen, Yizhou Wang, Tianyi Liu, Zhuoran Yang, Xingguo Li, Zhaoran Wang, and Tuo Zhao.  On computation and generalization of generative adversarial imitation learning.  _arXiv:2001.02792_ , 2020.
  * Arjovsky et al. [2017] Martin Arjovsky, Soumith Chintala, and Léon Bottou.  Wasserstein generative adversarial networks.  In _ICML_ , pages 214–223, 2017.
  * Mao et al. [2017] Xudong Mao, Qing Li, Haoran Xie, Raymond YK Lau, Zhen Wang, and Stephen Paul Smolley.  Least squares generative adversarial networks.  In _ICCV_ , 2017.
  * Miyato et al. [2018] Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida.  Spectral normalization for generative adversarial networks.  _arXiv preprint arXiv:1802.05957_ , 2018.
  * Che et al. [2016] Tong Che, Yanran Li, Athul Paul Jacob, Yoshua Bengio, and Wenjie Li.  Mode regularized generative adversarial networks.  _arXiv:1612.02136_ , 2016.
  * Heusel et al. [2017] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.  Gans trained by a two time-scale update rule converge to a local nash equilibrium.  _Advances in neural information processing systems_ , 30, 2017.
  * Metz et al. [2016] Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein.  Unrolled generative adversarial networks.  _arXiv:1611.02163_ , 2016.
  * Farnia and Ozdaglar [2020] Farzan Farnia and Asuman Ozdaglar.  Do gans always have nash equilibria?  In _ICML_ , 2020.
  * Salimans et al. [2016] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen.  Improved techniques for training gans.  _NeurIPS_ , 2016.
  * Tolstikhin et al. [2017] Ilya Tolstikhin, Sylvain Gelly, Olivier Bousquet, Carl-Johann Simon-Gabriel, and Bernhard Schölkopf.  Adagan: Boosting generative models.  _arXiv:1701.02386_ , 2017.
  * Pavan Kumar and Jayagopal [2021] MR Pavan Kumar and Prabhu Jayagopal.  Generative adversarial networks: a survey on applications and challenges.  _International Journal of Multimedia Information Retrieval_ , 10(1):1–24, 2021.
  * Daskalakis et al. [2021] Constantinos Daskalakis, Stratis Skoulakis, and Manolis Zampetakis.  The complexity of constrained min-max optimization.  In _Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing_ , pages 1466–1478, 2021.
  * Mescheder et al. [2018] Lars Mescheder, Andreas Geiger, and Sebastian Nowozin.  Which training methods for gans do actually converge?  In _ICML_ , 2018.
  * Gemp and Mahadevan [2018] Ian Gemp and Sridhar Mahadevan.  Global convergence to the equilibrium of gans using variational inequalities.  _arXiv preprint arXiv:1808.01531_ , 2018.
  * Engel et al. [2017] Jesse Engel, Matthew Hoffman, and Adam Roberts.  Latent constraints: Learning to generate conditionally from unconditional generative models.  _arXiv preprint arXiv:1711.05772_ , 2017.
  * Pedregosa [2016] Fabian Pedregosa.  Hyperparameter optimization with approximate gradient.  In _International conference on machine learning_ , pages 737–746. PMLR, 2016.
  * Lorraine et al. [2020] Jonathan Lorraine, Paul Vicol, and David Duvenaud.  Optimizing millions of hyperparameters by implicit differentiation.  In _International Conference on Artificial Intelligence and Statistics_ , pages 1540–1552. PMLR, 2020.
  * Franceschi et al. [2017] Luca Franceschi, Michele Donini, Paolo Frasconi, and Massimiliano Pontil.  Forward and reverse gradient-based hyperparameter optimization.  In _International Conference on Machine Learning_ , pages 1165–1173. PMLR, 2017.
  * Liu et al. [2022] Risheng Liu, Pan Mu, Xiaoming Yuan, Shangzhi Zeng, and Jin Zhang.  A general descent aggregation framework for gradient-based bi-level optimization.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2022.
  * Krizhevsky and Hinton [2010] Alex Krizhevsky and Geoff Hinton.  Convolutional deep belief networks on cifar-10.  _Unpublished manuscript_ , 40(7):1–9, 2010.
  * Krizhevsky et al. [2009] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * Minguk Kang and Park [2021] Minsu Cho Minguk Kang, Woohyeon Shim and Jaesik Park.  Rebooting ACGAN: Auxiliary Classifier GANs with Stable Training.  2021\.
  * Gao et al. [2018] Fei Gao, Yue Yang, Jun Wang, Jinping Sun, Erfu Yang, and Huiyu Zhou.  A deep convolutional generative adversarial networks (dcgans)-based semi-supervised method for object recognition in synthetic aperture radar (sar) images.  _Remote Sensing_ , 10(6):846, 2018.