We examine the omissions of a GAN in two ways:
- What does a GAN miss in its overall distribution?
- What does a GAN miss in each individual image?
Seeing Omissions in a GAN Distribution
To understand what the GAN's output distribution is missing, we
gather segmentation statistics over the outputs, and compare the number
of generated pixels in each output object class with the expected
number in the training distribution.
A Progressive GAN trained to generate LSUN outdoor church images
is analyzed below.
Of course the model is not perfect, but this graph reveals that
the omissions of the model are specific. This model does not generate
enough pixels of people, cars, palm trees, or signboards compared
to the training distribution.
Instead of drawing such complex objects, it draws too many pixels of
simple things like earth and rivers and rock.
Seeing Omissions in Individual GAN Images
Omissions in the distribution lead us to ask: how do these mistakes
appear in individual images?
Seeing what a GAN does not generate requires us to compare
the GAN's output with real photos. So instead of examining random
images on their own, we use the GAN model to reconstruct real images
from the training set. The differences reveal specific cases of what
the GAN should ideally be able to draw, but cannot.
