I recently finished a Monet-style CycleGAN (Cycle Generative Adversarial Network) model inspired by the "GAN Getting Started" Kaggle competition. The model trained from hundreds of Monet paintings and thousands of real photographs. The goal of this project was to develop a model using CycleGAN architecture to learn the mapping between photo and Monet painting domains without requiring paired training data.
This project started out as a Kaggle submission for my Reinforcement Learning graduate course, but with too large of a Jupyter notebook and way too long of a runtime (~7 hours past the 5 hour max) this quickly turned into a larger effort to train a model to transfer photos to actually resemble Monet's art style.
This project used the TensorFlow library for my implementation of the CycleGAN architecture, as well as loss functions and optimizers to help train the model. Each of the 50 epochs took about 15-25 minutes as the notebook ran overnight. The first 10 epochs of training revealed clear progress with a lot of work to go, as seen in the blurry images with dark artifacts splotched around the edges:
By epoch 30, we can observe outputs that exhibit a canvas-style look with more uniform colors that certainly look more like a painting, but many photographs were still seeing large artifacts after transfer:
Fortunately, the training corrected itself by epoch 50, where we see the vast majority of output images are consistently more Monet-style with no artifacting:
This is a work in progress, as one of many changes I'd like to make include developing a user interface to actually interact with this model by uploading photographs to be transferred.