Targeted style transfer using cycle consistent generative adversarial networks with quantitative analysis of different loss functions 1

Abstract

Targeted style transfer is the visual computing and deep learning problem where the input and target image sets are used to train the network by learning the mapping between those for conversion of the input image to the style of the target image. One of the popular methods for this task is Cycle-GANs (Cycle Consistent Generative Adversarial Networks), with Mean Squared Error, Binary Cross Entropy Error, and L1 loss functions. In this paper, our network is trained for image-to-image translation where the style or content of the Target image is changed by the network by modifying loss functions of Cycle GANs. Most accurate translation could be trained to the network through the use of paired images i.e. Supervised Learning where the input image and output images are known and thus, the network learns to minimize the gap between the expected output and observed output. However, this kind of paired data is not readily available and is strenuous to mass produce. Cycle GANs uses unpaired data, and our work is dedicated to finding the best possible loss function combination for making it even more efficient.

In Cycle GANs, there is a combination of 2 networks: Discriminators and Generators for each data set, which compete against each other to out-perform the other. Discriminator network uses Classification loss functions for distinguishing the images for the 2 datasets, while the Generator network uses Regression loss functions for determining Cycle loss and Identity loss. These loss functions play a vital role in the style transfer as they determine how much the images have been modified. We have worked on various loss functions like Mean Square Error loss, Binary Cross Entropy Error loss, Hinge loss, Huber loss, Log loss, Square loss and L1 loss for experimentation for the best losses combination to be used. We discuss the strengths and limitations of the loss functions already used and propose different combinations of loss functions for better accuracy. A separate classifier was trained extensively for performance evaluation purpose, which gives the most optimal combination of loss functions which is Binary Cross Entropy loss for Classification loss function and Huber loss for Regression loss function.