DVD-GAN
DVD-GAN
**DVD-GAN** is a generative adversarial network for video generation built upon the [BigGAN](https://paperswithcode.com/method/biggan) architecture.

DVD-GAN uses two discriminators: a Spatial Discriminator $\mathcal{D}\_{S}$ and a
Temporal Discriminator $\mathcal{D}\_{T}$. $\mathcal{D}\_{S}$ critiques single frame content and structure by randomly sampling $k$ full-resolution frames and judging them individually.  The temporal discriminator $\mathcal{D}\_{T}$ must provide $G$ with the learning signal to generate movement (not evaluated by $\mathcal{D}\_{S}$).

The input to $G$ consists of a Gaussian latent noise $z \sim N\left(0, I\right)$ and a learned linear embedding $e\left(y\right)$ of the desired class $y$. Both inputs are 120-dimensional vectors. $G$ starts by computing an affine transformation of $\left[z; e\left(y\right)\right]$ to a $\left[4, 4, ch\_{0}\right]$-shaped tensor. $\left[z; e\left(y\right)\right]$ is used as the input to all class-[conditional Batch Normalization](https://paperswithcode.com/method/conditional-batch-normalization) layers
throughout $G$. This is then treated as the input (at each frame we would like to generate) to a Convolutional [GRU](https://paperswithcode.com/method/gru).

This RNN is unrolled once per frame. The output of this RNN is processed by two residual blocks. The time dimension is combined with the batch dimension here, so each frame proceeds through the blocks independently. The output of these blocks has width and height dimensions which
are doubled (we skip upsampling in the first block). This is repeated a number of times, with the
output of one RNN + residual group fed as the input to the next group, until the output tensors have
the desired spatial dimensions. 

The spatial discriminator $\mathcal{D}\_{S}$ functions almost identically to BigGAN’s discriminator. A score is calculated for each of the uniformly sampled $k$ frames (default $k = 8$) and the $\mathcal{D}\_{S}$ output is the sum over per-frame scores. The temporal discriminator $\mathcal{D}\_{T}$ has a similar architecture, but pre-processes the real or generated video with a $2 \times 2$ average-pooling downsampling function $\phi$. Furthermore, the first two residual blocks of $\mathcal{D}\_{T}$ are 3-D, where every [convolution](https://paperswithcode.com/method/convolution) is replaced with a 3-D convolution with a kernel size of $3 \times 3 \times 3$. The rest of the architecture follows BigGAN.
Adversarial Video Generation on Complex Datasets
https://paperswithcode.com/paper/efficient-video-generation-on-complex
Adversarial Video Generation on Complex Datasets