Exploring Deep Convolution Generative Adversarial Nets
Deep Convolution GAN
The Generative Adversarial Network (GAN) is among the most innovative discovery in deep learning in recent times. In the paper (Goodfellow et al.) that introduced the GAN, two competing networks, the generator and the discriminator play the minimax game — one tries to minimize the minimax function whereas the other tries to maximize it.
Initially a GAN was plagued with several key problems. Firstly, it was unstable to train a GAN and secondly, there was limited research in trying to understand what a GAN actually learnt during the training process.
To address both these limitations, Radford et al. wrote the paper Unsupervised Representation Learning with Deep Convolution Generative Adversarial Nets and introduced the Deep Convolution GAN (DCGAN). To make the GAN more stable, the authors introduced the following model architecture changes and specifications:
- Deterministic Spatial Pooling functions (such as max pooling) were replaced with strided convolutions
- Fully Connected Layers were eliminated
- Batch Normalization was introduced for both the generator and the discriminator nets
- ReLU activation was used for the generator while Leaky ReLU was used for the discriminator
The authors further experimented by training DCGANs on various datasets and provided illustrative visuals to showcase the GAN learning process.
DCGAN on PyTorch
Once I read the DCGAN paper, my next objective was to code the network and see how it behaved. With help from some tutorials, I managed to code both the networks and their training process. For training, I used the MNIST digits dataset.
Here’s my Discriminator Network: We can note the use of Leaky ReLU activation and Batch Norm.
And here’s the Generator Network: Note the use of ReLU activation and Batch Norm
I trained the network for just 1 epoch. Here’s the random noise at the first step of the train process on Tensorboard:
And here’s step 10 of the first epoch of the train process:
One can see how the network is slowly creating the numerical digits similar to the MNIST dataset. Of course, training for more epochs would have given us much better results.
